Data Domain: How to ensure that MTU is consistent along the path

This document describes two mechanisms that can be used to ensure that the Maximum Transfer Unit (MTU) is consistent along the path and to calculate the path MTU which is the minimal MTU that is consistent along the path between two hosts.

The MTU is the largest possible frame on an OSI Model Layer 2 data network.
The MTU size depends on the physical properties of the communications media.

For most Ethernet networks, this is set to 1500 bytes by default.
Ethernet Version 2 networks have a standard frame size of 1518 bytes (including the 14-byte Ethernet II header and 4-byte Frame Check Sequence (FCS)
More recently Ethernet frames with more than 1500 bytes of payload are becoming popular (they are called Jumbo Frames) and normally they carry 9000 bytes of payload.

The problem with increasing the MTU over the default value of 1500 is that all network devices along the path must support the increased MTU value.

This is not always the case. We must ensure that the MTU is consistent all over the path, otherwise weird problems can appear.

Some problems that can be caused by having an inconsistent MTU along the path:

• DD Boost devices residing on Data Domain are mounted in Dell NetWorker, when the backup starts it gets stuck without showing any progress.
• Mtree replication between two devices it is shown as connected, but replication never starts.
• The destination Data Domain only receives some traffic from the source Data Domain in a replication pair.

All network devices in the path between two hosts must support and be configured to use the same MTU.

In order to avoid problems with inconsistent MTU, we must calculate the path MTU, which is the minimum MTU on any network segment that is currently in the path between two hosts.

Tools for calculating the Path MTU:

1. Tracepath
   Tracepath traces the path from source to destination showing the MTU of each hop.
   It helps to detect any inconsistent MTU along the path.
   An example of an inconsistent MTU problem below:

   tracepath -n 172.26.33.11
   1: 172.26.61.97 (172.26.61.97) 0.126ms pmtu 1500
   1: 172.26.61.250 (172.26.61.250) 0.559ms
   2: 172.26.61.250 (172.26.61.250) asymm 1 0.459ms pmt u 1476
   3: no reply
   4: no reply

   The -n parameter is for not resolving hostnames and works only with IP addresses.

   Note: In the example, the change of the MTU value from 1500 to 1476 (hop 2) and the inability to detect the newer MTU value after that hop.

   An example of an MTU consistent along the path:

   tracepath -n 146.176.1.130
   1: 146.176.3.6 0.070ms pmtu 1500
   1: 146.176.3.254 (146.176.3.254) 0.235ms
   2: 146.176.209.2 (146.176.209.2) 0.325ms
   3: crl-nbmed-1.napier.ac.uk (146.176.1.130) 0.213ms reached
   Resume: pmtu 1500 hops 3 back 3

   Note: The consistent MTU of 1500 in all the path in the example above

2. Using ping to calculate the path MTU.
   Ping can be used to calculate the path MTU.
   The key is to use it, not in the standard form, but with two modifiers.
   One modifier is used for specifying the packet size and the other to prohibit fragmentation.

   The packet size must be the real packet size we want to test minus twenty-eight.
   This is because the header of ICMP is eight bytes and the IP header is twenty bytes.

   How to use the ping command in LINUX to detect MTU problems
   In Linux, the command is ping -M do -s <size_of_the_package>-28 <IP address> -c 4
   Let us say that we want to test if an MTU of 1500 is possible along the path between one machine and another. In source, we issue a command like the following.

   # ping -M do -s 1472 10.64.102.30 -c 4
   PING 10.64.102.30 (10.64.102.30) 1472(1500) bytes of data.
   1480 bytes from 10.64.102.30: icmp_seq=0 ttl=64 time=0.279 ms
   1480 bytes from 10.64.102.30: icmp_seq=1 ttl=64 time=0.152 ms
   1480 bytes from 10.64.102.30: icmp_seq=2 ttl=64 time=0.181 ms
   1480 bytes from 10.64.102.30: icmp_seq=3 ttl=64 time=0.205 ms

   We are confirming above that it is possible to reach 10.64.102.30 with an MTU of 1500.

   Let us see if we can increase the MTU size to 1518. The package size to use in that case with the ping command would be c1518-28=1490

   ping -M do -s 1490 10.64.102.30 -c 4
   PING 10.64.102.30 (10.64.102.30) 1490(1518) bytes of data.
   From 10.64.102.25 icmp_seq=0 Frag needed and DF set (mtu = 1500)
   From 10.64.102.25 icmp_seq=0 Frag needed and DF set (mtu = 1500)
   From 10.64.102.25 icmp_seq=0 Frag needed and DF set (mtu = 1500)
   From 10.64.102.25 icmp_seq=0 Frag needed and DF set (mtu = 1500)

   We can see that the destination is not reachable as fragmentation would be needed.

   Then we can say that this MTU of 1518 bytes cannot be used in this environment.

   How to use the ping command from DDSH (command-line interface) (Starting in DD OS 5.4) to detect MTU problems
   From the DDSH prompt:

   net ping [destinationIP] path-mtu do packet-size [packet size -28] count 3
     EXAMPLE: sysadmin@dd2500# net ping 1.2.3.4 path-mtu do packet-size 1472 count 3

   How to use the ping command in WINDOWS to detect MTU problems
   In Windows, the command is similar, but the modifiers are different.

   ping -f -l <size_of_the_package>-28 [destinationIP]

   Note: -f is the modifier for setting the 'do not fragment bit'

   So, for testing if an MTU of 1500 is possible between one source machine and a destination machine in Windows, the command to issue would be:

   ping -f -l 1472 [destinationIP]

   If the calculated MTU between two points is less than the configured MTU of the source, then this inconsistency must be investigated by your network team. MTU should be large enough along the whole link from the source of the data stream to minimize re-transmission or loss of communication.

If the MTU is different in DD, then to modify MTU settings in DD, following command can be used to change the MTU settings.

net config ifname {[ipaddr [netmask mask]] | [ipv6addr/prefix] | [type {fixed | floating}] | [dhcp {yes [ipversion {ipv4 | ipv6}] | no}]} {[autoneg] | [duplex {full | half} speed {10| 100|1000|10000}] [up | down] [mtu {size | default}] [txqueuelen size]