PURPOSE OPERATION OPTIONS COMMAND LINES RELATED PROGRAMS
(64 Bit version is available in beta form. Please call if intetested.)
As of 2003, the 16 bit versions are no
longer being supported, updated or maintained.
This version also can copy files from a SHADOW COPY location if used under the proper server environments. (see --SHADOW_COPY, in options section). If you wish to get a copy of this "unicode" version, request it specifically upon ordering.
The upcopy program is designed to read a source and destination directory and copy only newer files from the source to destination. UPCOPY operates similar to xcopy or robocopy except with better e-discovery/forensic options. Upcopy supports the Unicode Microsoft path/name max length of 32767 characters.
A second use is to provide the program a list of files to copy to a new destination. This option (-S) is especially useful when doing some sort of update and you have a known list of files that need to be copied, and a destination root path for all of them to go to. Those using the Hashkeeper program will find this especially useful.
Generally it should be used to copy (update) files from one directory to another. An excellent use is to update certain files from a hard disk to a floppy or visa versa.
It can be used to copy files only newer than certain number of days old, or all newer files regardless of age. It can be set to only copy those files that already exist, thus not adding superfluous files to the destination.
For forensic purposes, it can be used to forensically copy files or folders from a live system to a destination drive. This is especially useful when the users can't or won't give permission to shut down their running systems. Most often used with servers.
Also, when dealing with servers, and especially EDB exchange data bases. The user should consider creating a "SHADOW FILE COPY". This is a special process by which the OS properly captures the state of an open file (.EDB or other), and places it in a secure location on the drive. It can then be copied with the unicode version of the software and the user is assured of an uncorrupted data file. (see the --SHADOW option below).
Additional capabilities are added as necessary:
One cabaility is to provide a list of top level folders/paths (--dirs=filename) from which to copy source files. This allows the user to specify specific folders (ie: \documents and setting, \other_folders) and not traverse the entire file system.
Another capability/other option added Nov. 2007, is to provide a list (--bypass=filename) of top level folders which are to be BYPASSED. So you would start at the top of the root, but bypass \docs and settings and bypass \windows, and others.
UNC paths such as \\cpu-00\x\progs (where x is the drive letter) are acceptable to use as the source (-p option). However, they will not work if using them within a -s source_file text file.
The logging options have been changed significantly (--logfile, --error, .ini). The -2, -3 logging options are no longer available after Feb 2008. All logging options have been replaced. However, the -1 (one) logging option still creates a basic log file, but without lists of filenames checked, copied or passed.
During the copy process the file dates and times of the original file are maintained in the destination file attributes. However, without the -R (Reset) option, the original access time is adjusted accordingly.
On Sept. 8, 2010 another capability was added. Occassionally the investigator has a tree structure of many files and many sub-directories. The need here is to take all the files (or just specific named files, ie: -f *.doc) from the entire tree structure, and copy them to a single level directory. You could do it one at a time, but the added --split option will perform this task. It causes upcopy to copy all the specified files in the source path, to a SINGLE top level directory, and does NOT create sub-directories as is experienced in the normal usage. The --split command can also specify how many files are to be in a top level directory. So if you have a tremendous amount of files, you can, if necessary only copy X number per destination folder
Source file type(s) (i.e. *.c) if necessary are provided by the user. The default is to copy all files (*.*)
The program locates all files in the source directory meeting the file specifications. If no source file types are provided, *.* is assumed. Source file types can contain wildcards, multiple file specs, or be blank for all (*.*) A source directory must be specified, but source file types defaults to (*.*).
If the source file list (-S) is used, then the Source files are taken to be all the files listed in the provided list. This list is a text file containing one filename (including full path) per line.
If the filetypes file (-F textfile) is used, then the filetypes found in the textfile will be used as if they were input individually using the -f filetype option. (4/2009)
When the program starts, the destination drive or directory is examined. If using the default of all files, the destination is searched for a file of similar name. If one is found, the parameters (options selected) are compared from the source file against the destination file. If the destination needs to be updated, it is. I.e. when the same file is found on the destination as in the source directory, and the destination file is older than the source file, the newer source file is copied to the destination.
If there is no destination file of the correct name, then the source file is copied to the destination. (-e overrides this operation to copy ONLY existing files).
The destination MUST be a directory or drive. If it is only a drive (-d A:) All subdirectories under the source are copied to appropriately created subdirectories on the destination.
Note:
If you are using only a drive as destination, make certain the default directory on that drive is the top level directory to start writing to. I.e. if the default on A: was A: \tmp, when the operation started, then all the copying would begin at A:\tmp and continue to create subdirectories below. If you wanted to start at A:\, then the default directory on A: should be root.
There is no check to see if the destination file is read-only. Destination Read-only files are NOT written over. (To force an overwrite of protected destination files, use the -O option. The -O option is not documented on the programs help screen.)
Various options exist to allow the user to “program” the file selection process, by file name, size, age, whether the destination exists or not and other options.
The 32-bit version will copy long file name files, and any file that meets the traditional long filename parameters.
The copy list option (-S) is ONLY available in the 32-bit version.
During the copy process the file dates and times of the original file are maintained in the destination file attributes. However, without the -R (Reset) option, the original access time is adjusted accordingly.
Copy Errors - ERROR FILE, updated/changed 2/2008
In some instances, especially when processing files using the -S (source list) option, you may find that there are some files that are NOT COPIED. In most instances this is because the file name has been generated by a program that writes UNICODE file names into a traditional text file. Thus loosing a significant amount of filename characters. These files are most often found in the internet cache area, and are usually .url type files. In most cases, they are of no consequence. However, they cannot be copied, because the text file which the program is reading as sources, does not contain the true unicode filename, and thus cannot find the file.
In any case, there are files which cannot be copied. Sometimes, it is purely a windows permission problem, which the user must overcome.
The purpose of the -E error file (replaced Feb 2008, by the --error=16 option), is to provide the user with a list of files which were not copied, and provide the user with information which may be useful in performing some other manual review or copy process. In cases where it is merely a system permission problem, once the permissions have been properly set, the files can be copied. For this reason, if the -E option is chosen, another file with the same root name as the error file, and has an extension of .lst. This file, has as its format, one that can be passed to the -S option, once the problem has been solved, and another recovery run is needed. In effect, this new file, has a clean listing of all those not copied.
C:>upcopy [source_directory] [destination_directory] [-[options]]
VERY IMPORTANT NOTE: The program can be run with only the source and destination directory on the command line (upcopy a: d:) without using any options. However, IF the destination is used without an option, the source item on the command line must also be present. This is because the source and destination (without options) are positionally specific on the command line. This means the source MUST occur before the destination. However the reverse is not true. You can list the source without an option, and pick up with the -d and other options. This capability of running without options is for quick operation (and for lazy people like myself).
C:>upcopy C:\tmp D:\tmp\old
/* copy the tree structure from C:\tmp to a new directory D:\tmp\old */
C:>upcopy -p C:\tmp -d d:\tmp\old
/* same as the first one, except this one makes use of the options */
C:>upcopy -p c:\tmp -d d:\tmp\old -f *.doc
/* copy only the *.doc files*/
C:>upcopy -p c:\tmp -d d:\tmp\old -f *.doc *.ppt
/* copy *.doc and *.ppt files */
C:>upcopy -p c:\tmp -d d:\tmp\old
--logfile=c:\path\logfilename!63
/* 2008: create ALL available logfiles named logfilename, in path
C:>upcopy -d d:\work_dir -S listfile.nam
/* copy all the files identified in the text file listfile.nam to the
d:\work_dir tree */
C:>upcopy -p . -d d:\work_dir -f *.doc --split
/* copy all the .doc files found in the current tree, and copy them all the the d:
\work_dir. NOT creating any subdirectories.. */
C:>upcopy -p . -d d:\work_dir -f *.doc --split=1000
/* copy all the .doc files found in the current tree, and copy them all the the d:
\work_dir. Placing a max of 1000 files per output directory, and creating a single
sub-directory for each additional 1000 files as necessary. Sub-directories are named
0001, 0002, etc as needed. */
The latest versions, (after 6/2009) have enhanced option usage. The options can also now be literal words preceded by the double -- (minus) signs. This syntax is similar to the *IX formats. The enhanced options are not included here. You should check the help screen of the program to see any particular option which has this format implemented. an example would be: instead of the -d path\folder, you could use: --destination=path\folder, or instead of the -S fileoption, you could use: --list=file_containing_list_to_copy.
Format NOTE When an option is listed of the following format: -option + filename: the plus sign (+) is indicated to mean that you must include an item following the option. DO NOT INCLUDE the plus (+) in the command line.Note: the -p and -d are required so they are placed at the top of the list
-p + src_dir: Use this directory as the source (starting point). Generally the format is: (-p C:\TOP_LEVEL_FOLDER\ANY_SUBFOLDERS). The source directory can be a network designation (i.e. \\COMPUTER_NAME\C\FOLDER_NAME). Not used if the -S option is used. (-p and -S are mutually exclusive)
-d + dest_dir: Use this as the destination directory. This is the top level destination path. Generally the format is: (-d D:\TOP_LEVEL_DEST_FOLDER\ANY_SUBFOLDERS). All files will be created under this destination, and original path will be maintained below this destination. The destination directory can be a network designation (i.e. \\COMPUTER_NAME\C\FOLDER_NAME) However, tests should be conducted to see if the destination paths are properly created.
--split[=xx]: Use this option when you want all the source files to be placed in a SINGLE -d destination directory. The program finds all the specified source files, and copies them to the single top level -d directory. The caveat here, is that files found with duplicate names, will not be copied, as the program will not create duplicate files. So use caution that duplicate file names are not attempted to be copied. If the =XX is replaced by a value ie: split=100, then there will be a maximum of 100 files placed in the top level -d directory, and additional subdirectories will be created as needed, each with a max of 100 files.
-f + filetype(s): Copy only those files meeting this file type. Additional file type can be added by separating each one by a space. (i.e. -f *.c *.doc *.tmp *.ppt ). This option is overridden by -S.
-F + filetype(s)file.txt: The file: filetypesfile.txt contains (one per line) the file type to process. This is used in place of the -f option if you have many filetypes (over the 10 supported by the -f option) to process. This file can contain up to 200 filetypes to process. This option may conflict with the -S option. (4/2009)
-x + exclude_filetype(s): Exclude files meeting this file type. Additional file type(s) (up to 32) can be added by separating each one by a space. (i.e. -x *.c *.doc *.tmp *.ppt ). This option is overridden by -S.
-S + filename: Text file (filename) containing a list, one per line, of source files to copy to the destination directory. (The names in the file MUST be filenames and not merely directories. If the source file can't be found, the program continues with the next.) The destination directory ( -d d:\option) is the top level tree where all the files will be copied. The copy maintains tree structure below the destination directory. Hashkeeper users find this very useful. . (-p and -S are mutually exclusive).
The line of text containing the filename MUST be pipe ( | ) delimited if the line of text contains more information than just the filename, see samples below. (i.e. if it is the output of diskcat which also contains file size etc.). The filename MUST be left justified, without leading spaces and be the first item on the line.
If the text files contain only the filename, (which is the preferred format), then there is no need for the pipe delimeters.
This file format:
C:\anydir\anyfile
C:\another_dir\another_file
C:\as_many_files\as_are_necessary
Or:
F:\anydir\another\dir\filename.ext | 123456 | 09-12-2002 12:26AM
C:\anydir\anyfile | 123 | 08-12-2002 09:25AM
C:\another_dir\another_file | 12387 | 04-12-2002 09:25AM
C:\as_many_files\as_are_necessary | 122344 | 02-12-2002 09:25AM
-A Copy (overwrite) ALL files. This causes all the files in the source tree to be copied. If this option is not used, only newer files, and ones that don’t exist in the destination are copied. This option causes a clean sweep of the tree. Consider adding the -O option to ensure overwrite of protected files.
-e Only copy over existing files. If the destination file doesn’t it won’t be created. Without this option, all newer and files where the destination doesn’t exist are copied. Possible use is, suppose the destination only has *.doc files in the tree, but the source has *.docs, some of which DON”T already exist on the destination. If the -f *.doc option is used, ALL the doc files would be used, use this if only the existing files are to be overwritten.
-E + errorfilename "Replaced 2/2008": Create an error file which holds information regarding files not copied. This file contains a listing of all files not copied to the destination. Some reasons the file could not be copied are that the destination disk is full or the destination file is locked by the OS (ie NT locks certain files, such as system files, and they can't be accessed while locked).
-h: Perform hash of both source and destination file. This confirms a good copy.
-hs: Perform hash on ONLY the source file. (this option can only be used with the -H outputfile option).
-hd: Perform hash on ONLY the destination file. (this option can only be used with the -H outputfile option).
Note: the -hs and -hd options are mutually exclusive, and are only useful when outputting hashes to an output file. If they are used without the -H option, results are unpredictable.
-H + hashfile: Perform hash of both source and destination file, and record results in the file named by hashfile. This can almost serve as a replacement for the -2 logfile option. This confirms a good copy.
--SHA: In addition to the hashing, calculate the SHA1 of the files. This option can only be used if the -h and -H logfile is used, as the SHA1 values are put in the logfile along with the MD5 hashes. This is a very very time consuming operation. Included in versions only after (12/9/2009)
-i Proceed Immediately with the copy. Without this option the source tree is first scanned and files are counted so the user knows how many files are involved.
-m: Automatically make the first destination if it does not exist. Without this option, if the first (top level) destination directory doesn’t exist, the user is prompted for an OK to create it. All subsequent subdirectories are automatically created without user intervention. Use this in batch file operation so no user input is needed.
-M: Same as -m, except the final destination directory will maintain the date and time of the source directory.
-nN: The lower case -n prints only the source filename to the
screen. DO NOT perform the copy operation. Use this to first confirm the items which will be
copied.
The (-N) upper case version also prints the file
size and the destination filename to the screen. And if the --delimiter=xx option is used, it
will format with delimeters the "source | filesize | destination name" to the screen in a
single line. This output can then be redirected >> to an output file for later examination or
input into a spreadsheet, or passed to rm or rmd
to remove the files once the copy process has taken place.
-r: DO NOT recurs through the source directory for file. The default is that the source directory is recursed and ALL subsequent files and directories are copied. The default operation emulates the XCOPY command.
-O Force and overwrite of any file that is protected, such as read only files. This has no effect on OS locked files. It has only been shown to be effective with files with the read only or hidden attribute set. This option is not documented on the program help screen.
-g + #:
-l + #: Copy only those files (g)reater than
or (l)ess than # days old. Replace the # with a valid number of
days.
-g + mm-dd-yyyy[acw]
-l + mm-dd-yyyy[acw]: (that's and ell, not a one).
Process only those files (g)reater (older) than or (l)ess than (newer)
than this mm-dd-yyyy date. The date MUST be in the form mm-dd-yyyy. It
MUST have two digit month and days (leading 0 IS required), and it MUST
have a 4 digit year. The date given mm-dd-yyyy is NOT included in the
calculation. Ie. if today was 01-10-2003 and you entered -l 01-09-2003
you would only process todays files. If you wanted to include those on
01-09, you should have entered -l 01-08-2003.
if any of the acw items are included, restrict the date to that type.
(access, create, write)
-t[acw]: In the 32 bit version, this is used to modify the -g or -l option to specify which time type to use in the calculations. The a==access, c==create, w==last write time. Don’t forget, in WIN9X, there is no access time.
-G + #:
-L + #: Copy only those files (G)reater than
or (L)ess than # bytes in size. Replace the # with a valid file
size.
-R: Because the files are opened and read, on WINNT and WIN9X the access date is modified. This option attempts to reset the source file date back to its original.
During the copy process the file dates and times of the original file are maintained in the destination file attributes. However, without the -R (Reset) option, the original access time is adjusted accordingly..
-1 logfile Create an output logfile with statistical information relating to the programs operation.
-2 and -3 options discontinued as of Feb 2008 version
-2 logfile Create an output log file with statistical information. This file is more verbose than that used in the -1 logfile option, and contains a list, in 2 line format, of all the source files their destination. This file can get quite large depending on the number of files copied.
-3 logfile Create an output log file with statistical information. This file is more verbose than that used in the -1 logfile option, and contains a list, in single line format, of all the source files their destination. This file can get quite large depending on the number of files copied.
As of Feb 2008 there are 3 new ways to get logfile creations.
-1 + logfilename[[!;|]xx]: filename to create logging file(s).
this '!' '|' or ';' seperator can be any one of the three but only 1
the [xx] syntax == any value below added together to get matrix of files
logging files that may be created are:
1: default logfile: contains command line, and statistics
2: list logfile: contains list of ALL files scanned for copy
4: copied logfile: contains list of ALL files successfully copied
8: not_copied logfile: contains list of files scanned but not copied
this file contains files not meeting criteria PLUS copy errors
16: error logfile: contains files meeting criteria, but had COPY error
32: renamed dupes log: destination file renamed due to duplicate
For ALL logfiles use: \"logfilename!63\" or greater. Normal is !31
Alternate -- option to the -1 logfile
--logfile=logfilename[!|;]xx Replace !xx with similar value as -1 option OR
--logfile=logfilename[!|;][list,copy,not,dupe,error] comma seperated file types
list==listfile==2, copy==copied file==4 not==notcopied==8
error==errorfile==16, dupes==duperenamefile==32
--error=xx replace xx with a value above. no filename allowed
\nTHE '! ; or |' seperator is required between filename and matrix value
any items or options values are always OR'd together
example: --logfile=logname.ext!2 --err=12
results in list, copied and notcopied logs + default logfile
upcopy.ini file syntax: [LOGFILE=name[ [[!|;] [bitmask] keyword,keyword..]]]
[LOGS=keyword,keyword...]
--dirs=filename Filename is a text file containing
the full top level path of the folder(s) you wish to copy. No error
checking is performed to see if the folders exist or not. If it exists,
the files below are copied in the normal fashion. The file is a text
file, with one path/folder per line, (or pipe delimeted, see help
screen). This option is mutually exclusive
with the -S option. So don't use them together. Sample
f:\documents and settings\johns folder
f:\documents and settings\other folder | other stuff on the line
f:\program files\virus generators
f:\documents and settings\johns folder
f:\documents and settings\other folder | other stuff on the line
f:\program files\virus generators
--SHADOW_COPY: If the user is attempting to copy all or part of a shadow copy file (see: Microsoft Volume Shadow Copy) the upcopy program may not be able to properly locate the volume shadow file. If the program cannot locate the shadow file, it is suggested that the user add this --SHADOW_COPY option. It may assist in finding and opening the file.
The pathname provided on the command line to locate the VSS (Volume Shadow Copy) is that
which was provided to the user when they ran the
VSSADMIN
program with the command:
vssdamin List Shadows.
The program will include all the shadows is maintains, and
the proper syntax for the -p option would be:
-p \\?\actual name of the shadow copy file which was provide by the command.
--bypass=filename Filename is a text file
containing the full top level path of the folder(s) of folders you wish
to bypass. No error checking is performed to see if the folders exist
or not. If it exists, it will be bypassed and no files below will be
copied.
The file is a text file, with one path/folder per line.
If the "filename" is not a file containing paths, but the actual path,
it is treated as such. This --bypass= option can be included multiple
times on the command line. Once for each folder, and/or for the file.
This option is mutually exclusive with the -S option. So don't use them
together. Sample
f:\documents and settings\johns folder bypass
f:\documents and settings\other folder bypass
f:\program files\virus checkers bypass
If you have an upcopy.ini file, the ini file contents are
bypass=f:\foldername
bypass=f:\anotherfoldername
bypass=etc.
Here are samples of the two types of output formats.
-2 Logfile_Option
512 |D:\DRa00632\DRa00632\AAA00632|
==>D:\NEWDIR\DRa00632\DRa00632\\AAA00632|
512 |D:\DRa00632\DRa00632\AAA00635|
==>D:\NEWDIR\DRa00632\DRa00632\AAA00635|
-3 Logfile_Option
512 |D:\DRa00632\AAA00632| ==>D:\NEWDIR\DRa00632\AAA00632|
512 |D:\DRa00632\AAB00632| ==>D:\NEWDIR\DRa00632\AAB00632|