Announcement Banner
UNSOLVED

kim_stofa

updated

12 years ago

K

kim_stofa

1 Rookie

•

21 Posts

1

8622

September 12th, 2014 12:00

Move data locally in filesystem

I am currently copying a large amount of data (50TB) from an old SAN to Isilon. The transfer was done using

FTP, and I have seen transfer rates around 5-10Gbps. The transfer is now done.

I now need to copy the same files to another location on the local filesystem. The issue is that the local copy is

much slower than the external FTP?

The internal copy process is done by CLI using SSH to the first node in the cluster, and trying the below commands

one by one.

I have tried using the "cp" and the "rsync" command, but nothing seems to change the copy speed.

I have used the following commands:

rsync -avW /ifs/data/source/ /ifs/data/newcopy/

cp -v -n *.ts /ifs/data/source/ /ifs/data/newcopy/


No matter what parameters I try, the transfer does not go above 1.5Gbps through the infiniband links.

From the CLI I do see the trafic moved over the Infiniband link, but the copy is much slower internally, then when

using an external source over FTP?  I tested this using the command: netstat -ib


One option is to use the "cp -c" command, for at clone file copy, but this cannot list the files in the source folder (more than 10.000 files).


Have anyone seen the same issue, and/or know how to go beyond this limit?


We are using the Isilon 7.1.1.0 build.


/ Kim