Robocopy is a robust file copying program built into Windows similar to UNIX rsync. It is a much better method of copying large datasets or lots of files across volumes and is a great tool for backing up data. It has the ability to resume copies if interrupted, various options and logging during copying.
Robust File and Folder Copy.
By default Robocopy will only copy a file if the source and destination have different time stamps or different file sizes.
Syntax
ROBOCOPY Source_folder Destination_folder [files_to_copy] [options]
Key
file(s)_to_copy : A list of files or a wildcard.
(defaults to copying *.*)
Source options
/S : Copy Subfolders.
/E : Copy Subfolders, including Empty Subfolders.
/A : Copy only files with the Archive attribute set.
/M : like /A, but remove Archive attribute from source files.
/LEV:n : Only copy the top n LEVels of the source tree.
/MAXAGE:n : MAXimum file AGE - exclude files older than n days/date.
/MINAGE:n : MINimum file AGE - exclude files newer than n days/date.
(If n < 1900 then n = no of days, else n = YYYYMMDD date).
/FFT : Assume FAT File Times (2-second date/time granularity).
Use this when copying files to a linux NAS or other non-windows file system.
/256 : Turn off very long path (> 256 characters) support.
Copy options
/COPY:copyflag[s] : What to COPY (default is /COPY:DAT)
(copyflags : D=Data, A=Attributes, T=Timestamps,
S=Security=NTFS ACLs, O=Owner info, U=aUditing info)
File Data (D) always includes file Timestamps (T).
/DCOPY:DCflag[s] : What to COPY for Directories (default is /DCOPY:DA)
(DCflags : D=Data, A=Attributes, T=Timestamps,
E=EAs, X=Skip alt data streams).
/COPYALL : Copy ALL file info (equivalent to /COPY:DATSOU).
/NOCOPY : Copy NO file info (useful with /PURGE).
/NODCOPY : COPY No directory info (by default /DCOPY:DA is done).
/SEC : Copy files with SECurity (equivalent to /COPY:DATS).
/B : Copy files in Backup mode.
/EFSRAW : Copy any encrypted files using EFS RAW mode.
/L : List only - don’t copy, timestamp or delete any files.
/MOV : MOVe files (delete from source after copying).
/MOVE : Move files and dirs (delete from source after copying).
/SJ : Copy Junctions as Junctions instead of the junction target.
/SL : Copy file Symbolic Links instead of the SL target [see notes below].
Performance tweaks
/COMPRESS : Request SMB network compression during file transfer, if applicable.
/J : Copy using unbuffered I/O (recommended for large files).
/NOOFFLOAD : Copy files without using the Windows Copy Offload mechanism.
/IPG:n : Inter-Packet Gap (ms), to free bandwidth on slow lines.
/MT[:n] : Multithreaded copying, n = no. of threads to use (1-128)
default = 8 threads, not compatible with /IPG and /EFSRAW
Redirecting output using /LOG is recommended for even better performance.
/R:n : Number of Retries on failed copies - default is 1 million.
/W:n : Wait time between retries - default is 30 seconds.
/REG : Save /R:n and /W:n in the Registry as default settings.
/TBD : Wait for sharenames To Be Defined (retry error 67).
Destination options
/A+:[RASHCNET] : Set file Attribute(s) on destination files + add.
/A-:[RASHCNET] : UnSet file Attribute(s) on destination files - remove.
/FAT : Create destination files using 8.3 FAT file names only.
/CREATE : CREATE directory tree structure + zero-length files only.
/DST : Compensate for one-hour DST time differences.
/PURGE : Delete dest files/folders that no longer exist in source.
/MIR : MIRror a directory tree - equivalent to /PURGE plus all subfolders (/E)
/SECFIX : Update ACL file SECurity on all files, even skipped files.
When using /SECFIX, specify the type of security information you want to copy by
also using one of these options: /COPYALL /COPY:O /COPY:S /COPY:U /SEC
Logging options
/L : List only - don’t copy, timestamp or delete any files.
/NP : No Progress - don’t display % copied.
/unicode : Display the status output as Unicode text.
/LOG:file : Output status to LOG file (overwrite existing log).
/UNILOG:file : Output status to Unicode Log file (overwrite)
/LOG+:file : Output status to LOG file (append to existing log).
/UNILOG+:file : Output status to Unicode Log file (append)
/TS : Include Source file Time Stamps in the output.
/FP : Include Full Pathname of files in the output.
/NS : No Size - don’t log file sizes.
/NC : No Class - don’t log file classes.
/NFL : No File List - don’t log file names.
/NDL : No Directory List - don’t log directory names.
/TEE : Output to console window, as well as the log file.
/NJH : No Job Header, ROBOCOPY © header.
/NJS : No Job Summary: Stats for Copied,Skipped,Mismatch,Failed,Extras.
/X : Report all eXtra files, not just those selected & copied.
/V : Produce Verbose output log, showing skipped files.
Job Options
/JOB:jobname : Read parameters from the named JOB file, e.g. multiple /XF files /XD dirs to exclude.
/SAVE:jobname : SAVE parameters to the named job file, this must be the last option, the command will not be run.
The .RCJ job file is an editable text file.
/NOSD : NO Source Directory is specified.
/NODD : NO Destination Directory is specified.
The command-line and job file cannot contain conflicting Source/Destination paths.
In the .RCJ job file, Source/Destination paths can be given as /SD:path and /DD:path
/NOSD and /NODD can be used either on the command line or in the job file.
/QUIT : QUIT after processing command line (to view parameters).
/IF : Include the following Files.
Repeated Copy Options
/MON:n : MONitor source; run again when more than n changes seen.
/MOT:m : MOnitor source; run again in m minutes Time, if changed.
This will continue until you terminate the Robocopy process with Ctrl-C.
The Robocopy /MON process can also be started/stopped by setting the run hours, see below.
/RH:hhmm-hhmm : Run Hours - e.g. /RH:1800-0200
If the robocopy command is launched outside the given hours it will pause until the next start time.
/PF : Check run hours on a Per File (not per pass) basis.
Exclude/Include
/XO : Exclude Older - if destination file exists and is the same date
or newer than the source - don’t bother to overwrite it.
/XC : Exclude Changed files
/XN : Exclude Newer files
/XL : Exclude "Lonely" files and dirs (present in source but not destination)
This will prevent any new files being added to the destination.
/XX : Exclude "eXtra" files and dirs (present in destination but not source)
This will prevent any deletions from the destination. (this is the default)
Will also suppress listing extra destination files in the log.
/XF file [file]... : Exclude Files matching given names/paths/wildcards.
/XD dirs [dirs]... : Exclude Directories matching given names/paths.
XF and XD can be used in combination e.g.
ROBOCOPY c:\source d:\dest /XF *.dmp "pagefile.sys" /XD "c:\System Volume Information" /S
/IA:[RASHCNETO] : Include files with any of the given Attributes.
/XA:[RASHCNETO] : Exclude files with any of the given Attributes.
/IM : Include modified files, copy if the Archive attribute is changed on both the source and destination.
By default robocopy ignores the Archive attribute and looks only at the file size and modified time.
/IS : Include Same, overwrite files even if they are already the same.
/IT : Include Tweaked files.
/XJ : Exclude Junction points from source. (included by default).
/XJD : Exclude junction points from source directories.
/XJF : Exclude junction points from source files.
/MAX:n : MAXimum file size - exclude files bigger than n bytes.
/MIN:n : MINimum file size - exclude files smaller than n bytes.
/MAXLAD:n : MAXimum Last Access Date - exclude files unused since n.
/MINLAD:n : MINimum Last Access Date - exclude files used since n.
(If n < 1900 then n = n days, else n = YYYYMMDD date).
Advanced options
/BYTES : Print sizes as bytes.
/DEBUG : Show debug volume information (undocumented)
/ETA : Show Estimated Time of Arrival of copied files.
/LFSM : Operate in "low free space" mode, enabling copy, pause and resume, see notes below.
/LFSM:n[KMG] : /LFSM, specifying the floor size in [K:Kilo,M:Mega,G:Giga] bytes.
/TIMFIX : FIX file TIMes on all files, even skipped files.
/Z : Copy files in restartable mode (survive network glitch) use with caution as
this significantly reduces copy performance due to the extra logging.
/ZB : Use restartable mode; if access denied use Backup mode. This option significantly
reduces copy performance because of checkpointing.Robocopy EXIT CODES
By copying only the files that have changed, robocopy can be used to backup very large volumes.
If either the source or desination are a "quoted long foldername" do not include a trailing backslash as this will be treated as an escape character, i.e. "C:\some path\" will fail but "C:\some path\\" or "C:\some path\." or "C:\some path" will work.
If creating a progress logfile with /LOG , specify a destination directory that already exists, robocopy will create the file but will not create a log directory automatically.
/REG Writes to the registry at HKCU\Software\Microsoft\ResKit\Robocopy
/XX (exclude extra) If used in conjunction with /Purge or /Mir, the exclude extra switch will take precedence and prevent any files being deleted from the destination.
The Robocopy /MO 'Monitor source' option can be used to regularly copy new/changed files e.g. database transaction logs.
Running Robocopy commands under PowerShell will add quotation marks to the options creating an "invalid parameter" error, [details], this issue is fixed in the latest version of PowerShell 7.2. In most cases you will find it easier to write a simple batch file and call that from PowerShell.
Long FileNames
When copying a large tree of multiple files and subfolders, it is likely some files will hit or even exceed the NTFS 256 character limit.
ROBOCOPY will accept UNC pathnames including long pathnames over 256 characters long.
Even though Robocopy successfully copies such files, choosing a destination folder with a shorter name than the source folder, can avoid issues such as difficulty accessing the files with Windows Explorer.ROBOCOPY \work \backup_files\monday — destination needs 15 extra chars in path ✗
ROBOCOPY \workgroups \backup — destination is shorter than source ✓
Junctions and Symbolic Links
For a Directory Symbolic Link, a Junction, or a Hard Link, RoboCopy /S (Copy Subfolders) will follow the source and copy the contents to a standard folder in the destination. For a backup program this is usually the desired behaviour.
For compatibility with old software the User Profile folder contains a number of Junction Points, 'Application Data', 'My Documents', Recent, 'Send To' etc. These can be listed with DIR /ALL %userprofile%
To exclude Junction points from being copied you can use the options /XJ, /XJD, /XJF this will prevent the source files from being copied. The Junction Point itself will not be copied, with or without these flags.
/SL Will copy symbolic links, this will copy a File Symbolic Link only, creating a new symbolic link at the destination.
An NTFS junction point in the Destination will not be deleted by RoboCopy, if you select any options which delete from the destination (/purge, /mir) and the destination contains a junction, RoboCopy will blindly follow that junction and delete the contents.
The /XJ options do not affect this as they only exclude junctions from the source.
File Attributes [RASHCNETO]
These options accept any combination of the following letters, when several are specified, will match if any or all items match:
R – Read only A – Archive S – System H – Hidden C – Compressed N – Not content indexed E – Encrypted T – Temporary O - Offline
Server 2016 Deduplicated files.
If you run Robocopy /MIR to mirror a system drive, it will attempt to delete 'dedup' files from \System Volume Information\ if also running in backup mode /B or /ZB it will have the permissions to actually delete them.
To avoid this either exclude \System Volume Information with /XD, or just avoid that combination of switches.
Performance
Multithreaded copying with /MT:n will improve performance until the network and file system latency become the limiting factor, /MT:16 is a good place to start testing.
To limit the network bandwidth used by robocopy, specify the Inter-Packet Gap parameter /IPG:n
This will send packets of 64 KB each followed by a delay of n Milliseconds.Robocopy will fail to copy files that are locked by other users or applications, so limiting the number of retries with /R:0 will speed up copying by skipping any in-use files. The Windows Volume Shadow Copy service is the only Windows subsystem that can copy open files. Robocopy does not use the Volume Shadow Copy service, but it can backup a volume shadow that has already been created with VSHADOW or DISKSHADOW.
NTFS Permissions
/B (backup mode) will allow Robocopy to override file and folder permission settings (ACLs).
ERROR 5 (0x00000005) Changing File Attributes ... Access is denied
This error usually means that File/Folder permissions or Share permissions on either the source or the destination are preventing the copy, either change the permissions or run the command in backup mode with /B.To run ROBOCOPY under a non-administrator account will require SeBackupPrivilege (backup files privilege), to copy security information auditing privilege is also required, plus of course you need at least read access to the files and folders.
All versions of Robocopy will copy security information (ACLs) for files and directories, but will not copy ACL security changes unless the file itself has also changed. The /SEC /SECFIX options can be used to refresh any ACL changes even if the file/folder itself is unchanged.
File Classes
These file classes can be used to include/exclude files from the copy operation.
File Class In source folder In destination folder Source/destination
File last modified timesSource/destination
File SizesSource/destination
File attributesLonely Yes No n/a n/a n/a Tweaked Yes Yes Equal Equal Different Same Yes Yes Equal Equal Equal Changed Yes Yes Equal Different n/a Newer Yes Yes Source > Destination n/a n/a Older Yes Yes Source < Destination n/a n/a Extra No Yes n/a n/a n/a Mismatched Yes (file) Yes (directory) n/a n/a n/a 'Tweaked' files will only be identified and copied when the /IT option is used
Low Free Space Mode (LFSM)
LFSM was added specifically for use with a target enabled for Azure File Sync cloud tiering. This switch is useful only for targets with tiered storage that might run out of local capacity before Robocopy finishes.
In LFSM mode, robocopy will pause whenever a file copy would cause the destination volume’s free space to go below a ‘floor’ value, which can be explicitly specified by the LFSM:n form of the flag. If LFSM is specified with no explicit floor value, the floor is set to 10% of the destination volume’s size.
Low Free Space Mode is incompatible with the /MT, /EFSRAW, /B and /ZB options.
Bugs
Robocopy /MOVE or /PURGE can be used to delete empty folders by setting source and destination to the same folder, but this does not always deal with nested empty folders in a single pass. It will work if Windows Explorer is closed.
When copying files larger than 2 GB from some iSCSI/SAN volumes the copy operation may fill all available RAM and then stall out. This typically happens when the RAID virtual disk on the controller is set to write-through caching rather than write-back caching.
Copy utilities such as TeraCopy ($) and BITS will workaround this issue because they copy the file in larger blocks, however the cost of this is that copying smaller files may then be slower than using Robocopy. The /IPG option may help with this issue.If the /FAT option is used to copy files into a directory containing existing files with long filenames, if any short filenames match between source and destination, those files will be replaced, as you would expect, but the files will retain their old long filenames even if they don’t match the long filenames of the source file.
Simple copy of all files from one folder to another:
ROBOCOPY \\Server1\reports \\Server2\backup
Copy Everything: all files, subfolders (even empty ones /E), NTFS ACL permissions, ownership, and file attributes. If this command is run repeatedly it will skip any files already in the destination, however this is not a true mirror as any files deleted from the source will remain in the destination. The *.* is optional as this is the default.
ROBOCOPY \\Server1\reports \\Server2\backup *.* /E /COPYALL
Copy all .jpg and .bmp files from one folder to another:
ROBOCOPY \\Server1\reports \\Server2\backup *.jpg *.bmp
List files over 32 MBytes in size:
ROBOCOPY C:\work /MAX:33554432 /L
Move files over 14 days old: (note the MOVE option will fail if any files are open and locked.)
ROBOCOPY C:\work C:\destination /move /minage:14
Backup a Server:
The script below copies data from FileServ1 to FileServ2, the destination holds a full mirror along with file security info. When run regularly to synchronize the source and destination, robocopy will only copy those files that have changed (change in time stamp or size).
@ECHO OFF
SETLOCALSET _source=\\FileServ1\e$\users
SET _dest=\\FileServ2\e$\BackupUsers
SET _what=/COPYALL /B /MIR
:: /COPYALL :: COPY ALL file info
:: /B :: copy files in Backup mode.
:: /MIR :: MIRror a directory tree
SET _options=/R:0 /W:0 /LOG:C:\batch\RoboLog.txt /NFL /NDL
:: /R:n :: number of Retries
:: /W:n :: Wait time between retries
:: /LOG :: Output log file
:: /NFL :: No file logging
:: /NDL :: No dir logging
ROBOCOPY %_source% %_dest% %_what% %_options%
Run two robocopy jobs at the same time with START /Min
Start /Min "Job one" Robocopy \\FileServA\C$\Database1 \\FileServeBackupA\c$\Backups Start /Min "Job two" Robocopy \\FileServB\C$\Database2 \\FileServeBackupB\c$\Backups
Copy only permission changes (additions and removals) assuming we already have a copy of the data:
ROBOCOPY \\FileServer\C$ \\SVR-Backups\c$\Backups /E /Copy:S /IS /IT
How to use Robocopy to copy files over the network fast
Using Robocopy over the network is a two-step process. You must first configure file sharing on the source device and then use Robocopy in the destination device to perform the transfer.
1. Confiture file sharing on Windows 10
To share files in the local network on Windows 10, use these steps:
Open File Explorer on Windows 10.
Navigate to the folder with the files to migrate.
Right-click the folder and select the Properties option.
On the “Properties” page, click the Sharing tab.
Click the Share button.
Use the drop-down menu to select the user or group to share files or folders – For example, the Everyone group.
Click the Add button.
Under “Permission Level,” select the sharing permissions you want the folder to have. For example, you can choose Read (default) if you only want users to view and open files. If you select Read/Write, users can view, open, modify, and delete the content in your shared folder.
Click the Share button.
Note the network path for the folder that other users will need to access the content over the network, and click the Done button.
Click the Close button.
After you complete the steps, make a note of the folder path and the IP address of your source computer.
2. Use Robocopy to copy files on Windows 10 fast over the network
Once file sharing is configured on the source device, you can copy the files using Robocopy from the destination device.
To use Robocopy to copy files fast on Windows 10, use these steps:
Open Start.
Search for Command Prompt, right-click the top result, and select the Run as administrator option.
Type the following command to copy the files over the network and press Enter:
robocopy \\source-device-ip\path\to\share\folder C:\destination-device\path\to\store\files /E /Z /ZB /R:5 /W:5 /TBD /NP /V /MT:16Command example:
robocopy \\10.1.2.111\Users\admin\Documents C:\Users\admin\Documents /E /Z /ZB /R:5 /W:5 /TBD /NP /V /MT:16
In the above command, change the source and destination paths with your configuration.
Robocopy command options breakdown
Robocopy has many features that you can use, and in the command shown in this guide, we’re using the following options to make the copy reliable and fast.
- /S — Copy subdirectories, but not empty ones.
- /E — Copy Subdirectories, including empty ones.
- /Z — Copy files in restartable mode.
- /ZB — Uses restartable mode. If access is denied, use backup mode.
- /R:5 — Retry 5 times (you can specify a different number, the default is 1 million).
- /W:5 — Wait 5 seconds before retrying (you can specify a different number, the default is 30 seconds).
- /TBD — Wait for share names To Be Defined (retry error 67).
- /NP — No Progress – don’t display percentage copied.
- /V — Produce verbose output, showing skipped files.
- /MT:16 — Do multithreaded copies with n threads (default is 8).
Perhaps the most important switch to pay attention to is /MT, an option that enables Robocopy to copy files in multithreaded mode. Typically, when you copy files from File Explorer, you only copy one file at a time. However, with multithreaded enabled, you can copy multiple files simultaneously, better utilizing the bandwidth and significantly speeding up the process.
If you do not set a number when using the “/MT” switch, then the default number will be “8,” which means that Robocopy will try to copy eight files at the same time, but the tool supports “1” to “128” threads.
In the command shown in this guide, we use “16,” but you can set it to a higher number. The only caveat is that more processing power and bandwidth will be used when using higher numbers. If you have an older processor and an unreliable network connection, it could cause issues. As such, test the command before executing the command with a high number of threads.
You can always view all the available switches, run the robocopy /? command.
0 comments:
Post a Comment