Unsolved
This post is more than 5 years old
26 Posts
0
6606
Alternative tool to EMCOPY?
I have approximately 5-6 Tb of data I need to copy from a Celerra NS502G to a NS480. It has to keep all the file and folder permissons and properties intact. I found a tool called Beyond Compare which worked well, but come to find out, it didnt retain the "file created date" property of the file, so that messed up deduplication. Emcopy while it does work most of the time seems to give me problems any time I try to use it.The way we have our data is it is spread across 5 different file systems. So I am trying to run emcopy from 5 different command prompt windows, each copy to/from a seperate file system. Running all 5 at the same time seems to give me a lot of problems, so I usually only one one or two at a time. What usually happens is for whatever reason emcopy just stops. I see a lot of "unable to browse" errors start occuring and then it just stops. I have to restart it, then it runs for a while and then stops again. Each time it seems to copy a little more than it did last time, but then it stops again. So I have to babysit the process all day long and keep restarting it. Here is an example of the command syntax I use. [emcopy m:\ n:\ *.* /o /a /de /i /s /sdd /r:1 /w:1 /log:c:\emcopy\logfile.txt /tee] It would be great to have a more reliable tool that didnt continously fail all day long. Any suggestions?
dynamox
1 Rookie
1 Rookie
•
20.4K Posts
1
January 16th, 2012 10:00
emcopy has been good to use, as a matter of fact i am in the middle of migration of 23TB of data (tiny little files too) from Celerra to Isilon and i have multiple emcopy running in different command windows. I have not seen the same issues, You could try robocopy from a Windows 2008 box (where robocopy supports multi-thread /MT:XX parameter).
bergec
275 Posts
1
January 16th, 2012 23:00
By default EMCopy uses 64 threads, running 5 copies at the same time might put too much load on the system. I suggest you decrease the # of threads (/th)
Also, make sure you understand how /de behaves when comparing source and destination, see emc224742
Claude
btrotter
26 Posts
0
January 17th, 2012 05:00
bergec, I am reading that Primus, but I dont think I understand what it is saying. Here is what the primus says:
An additional note regarding EMCOPY documentation:
EMCOPY readme.txt file has documentation error regarding /de option.
The /de option of EMCOPY was introduced in order to copy the file from source to destination in case the previous copy was interrupted in the middle of transferring data (in that case, the file date is more recent on the destination and /d will not re-transfer the file).
With EMCOPY 4.00, the readme.txt file that comes with EMCOPY is incorrect when it says that the file will be transferred when “LAST MODIFICATION time greater than the existing target copy or the files that have the same LAST MODIFICATION time and a difference in the size .“ The message displayed when EMCOPY is run with no arguments is correct: “Copy the source files when the last modification is not the same as the destination file's modification time or when files size are different.”
In the next EMCOPY release, the readme.txt file which comes with EMCOPY will be updated to reflect this behavior.btrotter
26 Posts
0
January 17th, 2012 06:00
Got it, thank you for your help. I am going to change the /th variable to see if I can find the optimal setting.
bergec
275 Posts
0
January 17th, 2012 06:00
The /d option will copy files if they are more recent on source than on destination.
The /de option will copy files if more recent on source than destination OR if filesize differ between source and destination.
I just wanted to make it clear that if using /de and files were modified on destination (e.g. change to a script), then the file will be over written on destination next time you run EMCopy with /de.
During a migration, no one usually gets access to the destination, but some admins might have acess (e.g. to their own files) and could start editing their own files on the destination ...
Claude
Rainer_EMC
8.6K Posts
1
January 17th, 2012 08:00
If you have such problem with emcopy you will have it with any other tool.
Emcopy is the best tested and most widely used copy tool with EMC
Make you have the latest version of emcopy from Powerlink
As Claude said the newer multi-threaded emcopy can actually overwhelm slower servers – or the Windows client you are using.
Use less threads in that case or a more powerfull copy client.
Not having on-access virus scanning turned on helps as well.
Rainer
christopher_ime
2K Posts
0
January 18th, 2012 01:00
Just curious, was Replicator (v2) ever considered as an option? Possibly ruled out because of a lack of additional SavVol space required by the supporting checkpoints for the sessions, or maybe using this opportunity to resize the filesystems (migrate from larger to smaller)?
I'll assume the NS480 is at DART 6.x by now, which would then require at least DART v5.6.47 on the NS500G (v5.6.52 is the latest available).
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
0
January 19th, 2012 11:00
Use the /c option as it will continue if it encounters an error. Additionally, you could run the following on the target control station:
# server_param server_2 -facility cifs -modify acl.mappingErrorAction -value 3
Note: This parameter defines rules for unknown mappings between SID, UID, GID on ACL setting, either because DC does not know SID or the username is not yet mapped to UID/GID.
Bit 0 = store unknown SID
Bit 1 = store SID without UNIX GID/UID mapping
Bit 2 = enable debug traces
Bit 3 = turning on bits 0 and 1
# server_param server_2 -facility cifs -modify acl.retryAuthSid -value 600
Note: This parameter sets the timeout value of 600 seconds between authentication retries for unknown SIDs.
# server_param server_2 -facility cifs -modify acl.FailOnSDRestoreError -value 0
This is the symtax i use and it works every time:
emcopy “Source” “Dest” /o /a /s /secfix /d /c /purge /preserveSIDh /log:c:\emclog\log_e_fs1.txt
Good Luck!