Data Domain: Measuring Replication Throughput

APPLIES TO:

• All Data Domain models
• All Software Releases
• Replication

PURPOSE:

This article describes how to measure the network throughput of the replication service and if there is any lag in the data reaching the destination restorer.

This helps the administrator determine if replication is working at all, and if so if there is a lag condition which requires further troubleshooting.

This article assumes that replication has been configured and is running.

SOLUTION:

What is lag?

Lag measures how far out of sync two copies of data are. In order to measure the lag between two contexts, at the Data Domain system prompt type:

replication status 

CTX   Destination                                              Enabled   Connection     Sync'ed-as-of-time   
---   ------------------------------------------------------   -------   ------------   ------------------   
0     col://test.datadomain.com                                yes       Connected      Mon Jul 28 13:32
1     dir://test.datadomain.com/backup/repl1                   yes       connected      Mon Jul 28 13:32   
---   ------------------------------------------------------   -------   ------------   ------------------  

• • • • Context (CTX) 0 is for collection replication and CTX 1 is for directory replication.
      • The sync ed-as-of-time column tells the administrator when the data on the destination was current.
      • If the sync d-as-of-time is different from the current time, and data has been written to the source DDR since the sync d-as-of-time, the administrator may be experiencing lag.

How to tell if replication is working:

Review the "Replication History" section in a recent auto support (ASUP).

For Directory Replication:

-------------------
Directory Replication:

Date Time           CTX  Pre-Comp (KB) Pre-Comp (KB) Replicated (KB) Sync-as-of Written Remaining Pre-Comp Network Time
-------------------------------------------------------------------------------------------------------------------------------------------------- 
2010/07/27 08:40:31  1   11,495,148 0  11,495,800    1,782,104       Tue Jul 27 08:03
2010/07/27 09:40:31  1   95,114,487 0  95,116,039    11,517,645      Tue Jul 27 09:04
2010/07/27 10:40:31  1   0          0  0             223             Tue Jul 27 10:04
-------------------------------------------------------------------------------------------------------------------------------------------------- 

• • • Data in the "Network" column under the "Replicated KB" section indicates how much data was written across the network during that hour, in total kilobytes.
    • Convert it to an average bits-per-second value, the administrator must multiply the total value by 8192 (to convert KB to bits), and then divide it by 3600 (the number of seconds in an hour).
    • Once the administrator has the bits per second value, it can be converted to kilo-, mega-, or giga-bits per second as appropriate:
      • During the hour leading up to 09:40:31, the DDR sent 11,517,645 * 8192 / 3600 = 26159332 bits/second, or roughly 25 megabits per second on average.
      • This was equivalent to 95,116,039 kilobytes of uncompressed data, for a virtual throughput of 206 megabits per second.
    • Values less than 1,000 typically indicate conditions in which no real data was sent across to the replica.
    • The "Pre-Comp (KB) Remaining" column tells the administrator how much data is left to copy from the source to the destination at the end of that hour.
    • In the example above, the "Pre-Comp (KB) Remaining" column is always 0, indicating there is no more data waiting to be sent across the network
    • If the "Pre-Comp (KB) Remaining" field is growing each hour, and the "Replicated (KB) Network" column shows a significant amount of data being sent (that is over 1000 KB), the Data Domain may have a lag condition which requires troubleshooting.

For Collection Replication:

Collection Replication:

Date Time           Data(KB)        Data(KB)   Sync-as-of  Remaining Replicated Time
---------------------------------------------------------------------------------------------- 
2010/07/27 08:40:31 462,898,258     4,058,875  Mon Jul 3   08:03
2010/07/27 09:40:31 461,419,298     3,573,790  Mon Jul 3   08:03
2010/07/27 10:40:31 458,489,689     3,606,086  Mon Jul 3   08:03
---------------------------------------------------------------------------------------------- 

• • • Data Remaining indicates how much data is left to replicate and Data Replicated indicates how much data was written to the disk.