Advice needed EMCOPY or ROBOCOPY???

Thank you very much @Amolc  and @Gav for your fast and rich responses !!

emcopy is a fancy version of robocopy ( if you are familiar with it). It allows you to copy data while preserving all NTFS ACLs, it's also multi-threaded so it's very fast. Sharedup allows you to migrate your shares/share permissions from your windows host to Celerra/Isilon so you don't have to recreate those manually. Great tools.

Here is a sample syntax, i am copying data from x: drive to z: drive

emcopy x:\ z:\ /o /s /d /q /secfix /purge /stream /c /r:1 /w:1 /log:c:\emcopylog.txt

if you open command prompt and type emcopy ..you will see all the parameters.

Thanks @chughh for your fast response, but I would like to hear opinions and responses to my first post.

Best Regards

EMCOPY is supported by EMC and Robocopy is not. EMCOPY preserves NTFS ACLs and also it is multithreaded so faster. Honestly I cannot tell which one is better, I guess you will have to decide for yourself. I prefer EMCOPY just because it is faster.

EMCOPY is Windows based tool, meaning it will run only on system which has windows on it. Isilon is based on FreeBSD Linux, so you cannot run EMCOPY on Isilon. Also, it is not advisable to run any kind of scripts or tools on Isilon that are not Isilon related or otherwise suggested by EMC, it will void EMC support policy.

Hope this helps!

PEG, I have recently migrated about 30TB of file data to an Isilon cluster. I can tell you that EMCopy is significantly faster and works really well. If you're familiar with Robocopy, you won't have any issue learning EMCopy syntax. The format is the same, just different syntax. I would also recommend a product called Beyond Compare. It will do file comparison of the source and destination and if there are any differences, it gives you the ability to move the delta's over. It is also extremely affordable and offers a 30+ day trial.

Thank you for your opinions!!. They are all very useful.

In the past, I used robocopy a lot.  But now we have to copy hundreds of W2k servers and some filers (EVA, Netapp) to a new Isilon.

Last doubts,

1) What would be the best way to migrate another NAS filer such as (Netapp with CIFS) to a One File Isilon (CIFS too) ?? I think the only way is using a windows host (intermediary copy hosts) this is correct? I have some alternative?

2) If I need to use the intermediary Windows there are difference between using w2008r64 or w2003r32  in a portable computer?? Or better I use Windows 7-64 ? (Thinking in speed performance, etc.) or better use some old server with Windows 2003 instead?¿

3) What is the best practice?? Start to copy with emcopy in old filer LAN to the ISILON by WAN or start the copy process in the DESTINATION LAN and copy files from the old filers?   (the ISILON is a cloud service connected to old filers by WAN)

Best Regards.

1) i have never used it myself but there is isi_vol_copy, take a look at this paper for more details/ideas

https://support.emc.com/docu49401_White-Paper:-SMB-File-Migration-to-EMC-Isilon.pdf?language=en_US

2) does not matter, as long as it has good network connection and plenty of memory.

3) do you have this data on tape (not in NDMP format), that could be restored onto Isilon and only then run emcopy/robocopy to sync up new/changed files ...so not to copy everything over WAN.

I agree with Dyanmox above.

1) ISI_VOL_COPY is magnitudes faster than any host based tool when it comes to migrating data from OnTAp.

It totally eliminates the need for the intermediary host. It uses NDMP to copy data as well as permissions from the source to the target.

2) For your other windows file servers, you can use EMCopy64 on a 64 Bit Windows 2008 host. It just performs better when multithreading is used on a 64-Bit host from our tests and then you can use a higher memory VM to run the copies. I would suggest atleast 16GB VRAM if its Virtual or RAM if its physical server.

3) Dynamox is  right on with his solution to restore from a tape device locally if your only option is WAN and then incremental/Syncup using EMCopy.

How to use sidmapfile function and in what version of emcopy is it available

emcopy 4.17 supports sidmapfile.

the format for sidmapfile is

this is a comment

#

#this file is read by emcopy when option /sidmapfile is specified

#each line is a single mapping record which leads emcopy to replace the sid source by the specified sid

#There are four different types of record supported

#SID_TO_SID, record format is : :

#HSID_TO_HSID, record format is : :

#ENUMTYPE, record format is : " "," "," "," "

#                              and are not used by emcopy.

#ISI_RECORD, record format is : ,{SID|HSID|GROUP|USER|NAME}: , ,{SID|HSID|GROUP|USER|NAME}:

#There is no limitation of record type directive

#A record type directive can be folowed by several records,

#next line will be discarded, no record type previously specified

JOE,"SID":S-1-5-21-1643508158-2780356154-507200429-1010,ISILON1,SID:"S-1-5-21-3787548895-340410874-2009791390-1002"

SID_TO_SID

S-1-5-21-1643508158-2780356154-507200429-1000:S-1-5-21-3787548895-340410874-2009791390-1006

HSID_TO_HSID

S-1-5-15-61F5F1BE-A5B8DE3A-1E3B43AD-3E8:S-1-5-15-6112CBD5-8F6E3909-A3DF4B75-3E8

# next line indicates a revoked SID

S-1-5-15-61F5F1BE-A5B8DE3A-1E3B43AD-1F5:REVOKED

ENUM_TYPE

"ISILON1","User","localuser1","S-1-5-21-3787548895-340410874-2009791390-1005"

"ISILON1","group","localgroup3","S-1-5-21-3787548895-340410874-2009791390-1004"

# next line indicates a revoked SID

"ISILON1","group","guests",REVOKED

ISI_RECORD

WSJDO,NAME:USER1,ISILON1,NAME:USER2

WSJDO,HSID:S-1-5-15-47AF4AC5-451AE0F4-A4519B61-3E8,ISILON1,HSID:S-1-5-15-47AF4AC5-451AE0F4-A4519B61-3EF

JOE,"SID":S-1-5-21-1643508158-2780356154-507200429-1010,ISILON1,SID:"S-1-5-21-3787548895-340410874-2009791390-1002"

# next line indicates a revoked SID

JOE,"SID":S-1-5-21-1643508158-2780356154-507200429-1010,REVOKED

#next line will be discarded

This is a Bad record

Beware of the SID map feature.  I had some trouble with it on EMCOPY 4.16 - it flat out didn't work reliably.

I reported the bug, so perhaps 4.17 works.  But I would verify its operation if I were you.

Gentlemen,

If you check the release notes for 4.17, you'll see that these items were addressed, in addition to adding the revoked functionality described above. 

---begin paste--

04.16.01 :  - Fixed : emcopy could sometime crash when a log file that can't be open is specified.

            - Fixed : owner or group SID are sometime not well migrated when a sidmapfile is used.

04.16.02 :  - Added : SID revocation.Revoked SID are specified in the SID migration file by the REVOKED

              keyword in place of a regular target SID in mapping record, ACE relative to revoked SID are

              discarded.

04.17:  - Added : support of file and directory compressed attribute when target volume supports compression.

        - Added : option /compress which drives emcopy compress policy.

                  Compress policy are applied only when files or directories are created.

                  ignore      : compress attribute is ignored

                  forcefile   : force compress attribute on every files

                  forcedir    : force compress atribute on every directories

                  forceall    : force compress attribute on every files and directories

        - Added : option /trfwindow which specified the maximum size of the sum of all files

                  in tranfer.

----end paste---

Hope this helps,

Chris Klosterman

chris.klosterman@emc.com

Senior Solution Architect

EMC Isilon Offer & Enablement Team

I'm glad to hear it has been fixed.  That said, I would trust, but verify.