Incremental Sync really slow on millions of files

The network is 1Gbps and I believe network is not an issue. Whenever the incremental syncs run, nothing gets sent over to the target site because the specs are fully synched up during real time. Average file size is around 50k -100k.

spec 1) 13 millions files, 790 GB, 44 hours to complete incremental sync, attribute compare only

spec 2) 7.5 million files, 1.38 TB, 20 hours to complete incremental sync, attribute compare only

Hi,

Is a network band between sites 1gbps?
Is it 100mbps?

How many is one file size when you average it?
How many is size of the total file?

I will advise you.

Please change the setting.
This setting had best performance.

Option ---> Performance

TCP/IP Buffer Size 64KB
Full Sync Buffer Size 128KB
Maximum Batch Size 256KB

Target Connections 1
Compression Level 0


Please full synchronization first.

In other words please exclude a check of Incremental.

If there is much number of the files, the incremental synchronization takes time adversely.

When the size of the file is 100KB, reading and writing is very slow in the specification of NTFS.
*This is not a issue of RepliStor.

Therefore, the full synchronization is earlier than the incremental synchronization.

Thank you for the suggestion, I will give it a try this weekend. Both sites are already fully synchronized. Can I ask why you advise to set the Target Connections to 1? Our current settings in the Performance tab is:

TCP/IP Buffer Size: 256
Full Sync Buffer Size: 256
Maximum Batch Size: 512
Target COnnections: 18

Also, the Disable Communications Compression checkbox is NOT checked.

I will teach special information.

As a result, a performance was the fastest.
"Comp Level 0 Target Connections 1"

Comp Level 0
Target Connections Transfer Rate Average(MB/Sec)
1---------------------------40.9
2---------------------------32.8
4---------------------------29.7
8---------------------------28.4
16--------------------------23.0
32--------------------------20.5


Comp Level 1
Target Connections Transfer Rate Average(MB/Sec)
1---------------------------32.0
2---------------------------33.4
4---------------------------29.5
8---------------------------28.3
16--------------------------27.0
32--------------------------22.3


Comp Level 2
Target Connections Transfer Rate Average(MB/Sec)
1---------------------------7.8
2---------------------------14.3
4---------------------------17.1
8---------------------------17.0
16--------------------------17.0
32--------------------------17.0


Comp Level 3
Target Connections Transfer Rate Average(MB/Sec)
1---------------------------7.8
2---------------------------14.1
4---------------------------17.1
8---------------------------17.2
16--------------------------17.0
32--------------------------16.9


Are you dissatisfied at even this?

Thank you for the suggestions! I will try them out this weekend and let you know. By the way, are you using version 6.3 of Replistor?? We are using 6.2. Do you think the newer version works faster?

Hi,

The version that I tested is 6.4.

However, I do not think that the speed changes.

It doesn't change in Full SYnc though the performance of Incremental Sync has improved in 6.4.

Hi,

Do you know how I can download a trial verison of 6.4? I wasn't able to find this on EMC's website. I tested the incremental sync this weekend and it's slower than before. So, maybe our version of 6.2 is not good. Thank you!

6.4 is now available for download via PowerLink. You can upgrade or install a eval copy good for 1 month.

Here is a 4 week evaluation license: QOGUJXOL-YMAD3KF3CA9BK77N

Hi, I am able to download 6.4 so now I want to test the incremental sync by using both 6.2 and 6.4 However, when I request for an Evaluation License, I get this error, "An invalid license key was entered. Please re-enter the License Key". I uninstall the previous Replistor installation and tried re-install, then I got this error. Anyway to workaround this without reimaging???

OK, I have been testing version 6.4 and 6.2 recently. In my test of 1.1 million tiff images (43 GB), 6.4's incremental synch did not improve as indicated in the release notes. In fact, it was slower than version 6.2 using the same settings.

I also tested the incremental synch by reducing the compression level and number of targets, it's actually slower. The best rate was achieved using 8 target connections with compression level of 1. That means what we have in production seems to be the best and there is no way to improve this...