Data Domain: Setting up Link Aggregation

Setting Up Link Aggregation

SUMMARY

Using the link aggregation feature in DDOS can result in an increase in throughput, however depending on several factors it could result in the opposite. The purpose of this article is to help determine and implement the appropriate configuration of Link Aggregation. In DD OS, since dynamic port participation methods like PAGp and MLT are not supported, it is crucial to statically configure LAG ports on the switch.

APPLIES TO

All DD OS versions that support the link aggregation feature

SOLUTION

Determine Appropriate Link Aggregation:

The following is what should be considered when trying to decide on aggregation. If aggregation is not required, consider failover. Therefore, the last choice given is failover as an alternative to aggregation.

Implementation of Data Domain Link Aggregation and Failover

• Round Robin and
• Balanced-xor (setup manually on both sides)
• LACP (starting in 5.0 for 1 Gb and 5.1 for 10 Gb)

• Layer 2 or
• Layer 3+4
• Layer 2+3 (starting in 5.0)

• veth0
• veth1
• veth2
• veth3

Before aggregation can be used, a virtual interface must be created. This can be done by running the following command:

net create virtual <veth#>

Example:

net create virtual veth0

After a virtual interface is created, any of the physical links available (eth0, eth1, eth2, and so on) can be included in the aggregation group. The onboard links (eht0 and eth1) have only recently been allowed to be added. Older installations of the Data Domain software may not allow those two links to be aggregated. 

To specify the aggregation of the eth2 and eth3 interfaces in the virtual interface veth0, one of the following commands would be used:

net aggregate add veth0 mode round-robin interfaces eth2 eth3

The first network packet sent to veth0 will be forwarded to one of the interfaces, and the next packet would be forwarded to the other.

Sending of packets continues to alternate between the interfaces until there are no more packets, or a link fails.

If eth3 loses physical connection, all packets are sent through eth2 until the eth3 link is brought back up.

To make this effective, the other side of the network must also be configured as round robin. For direct connect (the only topology that is recommended for round robin) the media server must be able to be set up (and support) round robin.

net aggregate add veth0 mode xor-L2 interfaces eth2 eth3

The aggregation used would be balanced-xor.

The packets are distributed across eth2 and eth3 based on XOR of the source and destination MAC addresses.

As there are only two links to be aggregated the lowest bit is used to determine the interface to use for the packet.

If the result is 0, one interface is chosen. If the result is 1, the other interface is used.

To get the packets spread across the two links, the data is must be sent to more than one destination. The MAC addresses of the destination must be different in such a way that XOR results provide a different number.

This means that one address must be odd and the other must be even.

If there are three links that are aggregate, the XOR result is split three ways. There have to be at least two media servers with odd and even MAC addresses to get any aggregation at all. In general, this aggregation should not be used with less than four media servers.

net aggregate add veth0 mode xor-L3L4 interfaces eth2 eth3

The aggregation used with this command will also be balanced-xor.

Eth2 and eth3 distribute packets using XOR of source and destination IP addresses and port numbers.

The result gives a number in which the lowest bit is used to determine which link to use to send the packet.

An even result goes over one, and an odd result goes over the other. With three links, the result is divided by 3 with the remainder determining which interface to use.

This aggregation would be used when there are a lot of connections (there is one connection per stream) or a lot of media servers or both.

This is the mode of choice for Data Domain, but some switches do not support this type of hashing.

net aggregate add veth0 mode LACP hash xor-L3L4 interfaces eth4a eth4b

The aggregation used with this command will also be lacp-xor.

Eth2 and eth3 distribute packets using XOR of source and destination IP addresses and port numbers.

The data flow control follows the same mechanism used by balanced mode. The exception is that it adds a control protocol to monitor the interfaces with a minimal amount of automated interface administration, including better interface failure sensing.

The sensing goes beyond the sensing of carrier loss to the sensing of the ability to send and receive data.

The heartbeat can be sent out every second or every 30 seconds (default). The interval determines how fast the bonding senses the link is no longer communicating and stops using the interface. Once every 30s is less invasive, but it takes longer to mark the link as down, and connection timeouts may occur while it is waiting.

net failover add veth0 interfaces eth2 eth3

This is not aggregation, but the command will group together interfaces eth2 and eth3 for failover.

There is only one failover type supported. If the active physical link goes away, the data is sent to the second physical link.

The active interface is determined by which link comes up first when it is set up.

This is nondeterministic. It depends on several factors such as switch activity, network activity, and which interface is brought up first when they are enabled.

The active interface can be determined by specifying one of the links as primary.

The primary interface is always set as active if it is UP and RUNNING.

Configure and enable the virtual interface.

After the virtual interface has been created (and the aggregation method assigned), the interface must be configured with an IP address (and subnet mask), and enabled. The following command can be used to do accomplish this:

net config <veth#> <IP address> netmask <subnet address> up

Example:

net config veth1 192.168.192.168 netmask 255.255.255.0 up