Start a Conversation

Unsolved

This post is more than 5 years old

DJ

1235

June 1st, 2009 06:00

Replication throughtput: doesn't fill the pipe...

Replicating NS40G to NS80G. No bandwidth throttling set. NYC to Philadelphia over Yipes 75 Mbit ethernet, 7ms average latency. Riverbed Steelheads on both ends of link, so latency should not be a huge factor. Steelhead data reduction on the replication sessions is about 40%. I have yet to see a replication utilize more than 3.5 Mbit of the WAN link. Utilization over the yipes link always stays below 50Mbit. I'd like to push the replication a bit harder. Are there tweaks? Is there a debug log for the replication?

26 Posts

June 1st, 2009 07:00

There are two parameters you can tune that may help in this case. You can adjust your TCP window for Replication using the following parameters. These only apply to celerra replicator and fs_copy, and you will need to reboot the datamover for these to take effect.

Use the following syntax to set the new parameters:

server_param server_# -facility -modify -value

[nasadmin@ns600_sales_cs nasadmin]$ server_param server_2 -f RCP -l
server_2 :
param_name facility default current
configured
tcpwindowlowat RCP 0 0
tcpwindow RCP 0 0


Here is a write up I got from our EMC² folks which helped tremendously when replicating over wide area links.

detailed_description
Allows you to change the window size used by replication and fs_copy.
This value indicates the amount of data that can be sent before it is
acknowledged by the receiving site. If you change the parameter after
you start these processes, you must reset the connection for the change
to take effect. Values are entered in bytes. param RCP tcpwindow=0 sets
the window size to 128 KB. Increasing the window size is most effective
when working with with latent, low-loss networks. The basic equation for
calculating window size is: WindowSize = Round Trip Delay * Desired Rate
For example, if there is a round trip delay of 100 ms and we plan to
send 10 MB/s across the IP network, we would need a window size of 1 MB
(.1sec * 10 MB/s = 1MB). Enter the value of the example parameter
as:param RCP tcpwindow= 1048576 This parameter applies to all
replications and fs_copy operations on the Data Mover.

------Other comments that I dug up out of an email---------------
Then, one must configure the 'RCP.tcpwindow' parameter on both DM's of
the communications to maximize the TCP throughput based on the bandwidth
and round-trip time of the communications. (The minimum value of the
'RCP.tcpwindow' parameter on the sending DM should be 64K, even if a
lower value of the TCP window size would still maximize the TCP data
stream throughput. This is the only case in which the 'RCP.tcpwindow'
parameters may normally be different on the DM's at the ends of the
communications.) Finally, I recommend that one should configure the
'RCP.tcpwindowlowat' parameter on the sending DM to be at least
three-quarters the value of the 'RCP.tcpwindow' parameter. (To
absolutely maximize the bandwidth utilization and thus the throughput
one may need to analyze the network traffic behavior when varying each
of the parameters.
----------------------------------------------------------

1 Rookie

 • 

92 Posts

June 1st, 2009 13:00

Thanks for the response. I checked these parameters and the GUI says they apply to V1 replication and fs_copy...

1 Rookie

 • 

92 Posts

June 4th, 2009 09:00

We may have exceeded the outbound bandwidth rating on our aging Steelhead 5010, so I'm going to try bypassing it.
No Events found!

Top