We have a huge file system about 6 TB in size. There is a lot of stale data on the server and also quite very scrambled across random folders. We want to restructure the NAS (folder structure). As in, I just want to move the existing selective data (just the data) and move it to the new folder structure created within the same file system. Post that we'll apply the new NTFS permissions separately as even AD SecGroups are being modified.
I tried to move the data internally using default Linux command to copy/move (cp/mv) the file from the old location to the new location I created within the file system. But the transfer rate is very slow. We have a few areas of 1.5 TB which will need movement too. At that rate it would take ages to complete the movement.
I would appreciate suggestions on how to perform this. NTFS permissions are not important here. Just the data.
thanks in advance. Best regards,
if i were you i would use emcopy, running on a host that has pretty good connection to the CIFS server. server_archive is good for one time copies but it sounds like you may have to take a few passes before final cutover and emcopy is great for that.
Thanks for the advise. We have a 1 Gb/s bandwidth in our environment. I'm using a LAN connected host to perform the copy using EMCopy. However, just to transfer 12 GB worth data, it's taking over 2 hours. I'm using the following command:
emcopy64.exe \\<sourece_location> \\<dest_location> /nosec /o /i /s /c /z /sd /de /r:1 /w:1 /log+:EMCopy64_Test_Log.txt
The previous copy I did was from the server to my laptop of the same file (12 GB) which took 1 hour 5 minutes.
Copy engine Statistics
File(s) copied : 1
File(s) recovered : 0
Directory(ies) created : 0
Amount of copied Byte(s) : 11 GB (12 819 335 680 Byte(s))
Estimated copy bitrate : 3212.378 KB/s
Global copy duration :3897.076
Elapsed time: hours: 01, mins: 04, secs: 57
Would you be able to tell me what could possibly be the reason here ?
these are extremely slow rates, even with a 100 megabit connection it should not take more than 20 minutes. What kind of file is it ? Instead of using your workstation, can you run this from a server in the datacenter ..maybe something that is on the same subnet as your CIFS server.
You should know that various factor could cause that.
First EMCOPY take advantages of paralelism or threads. By default it uses 64 threads, but if you are coping only one file you don´t benefit from this.
Second you have to check your network to see if there is any bottleneck anywhere.
Also if the file is terrible fragmented in the source or the source is heavily working with another random access you could get very poor copy perfomance with one file.
dynamox: I'm going to give that a shot right now from a server in the same vlan. Hopefully it works.
@alvaro_gajardo: I don't suppose we have bottlenecks in the network as per the network trace captured from the host workstation to the server and vice-versa.
@rainer_emc: We do have McAfee running but I've made sure I disable it before running the utility. I'll try what Dynamox has to suggest.