Connectrix - B Series Brocade: How to configure LS LD long-distance Link modes
Summary: Brocade Extended Fabrics License is an optional licensed feature for the Brocade SAN deployment over distances beyond 10 km. To implement long-distance dynamic (LD) and long-distance static (LS) distance levels, a Brocade Extended Fabrics license is required. The LD and LS settings are necessary to achieve maximum performance results over interswitch links (ISLs and IFLs) that are greater than 10 km. ...
Instructions
Fabric interconnectivity over Fibre Channel at longer distances: ISLs and IFLs can use long-distance dark fiber connections to transfer data. Wavelength-division multiplexing, such as dense wavelength-division multiplexing (DWDM), coarse wavelength-division multiplexing (CWDM), and time-division multiplexing (TDM), can increase the capacity of the links. As the Fibre Channel speeds increase, the maximum distance decreases for each switch. The Extended Fabrics feature extends the distance the ISLs/IFLs can reach over an extended fiber. This extension is accomplished by providing enough buffer credits on each side of the link to compensate for latency introduced by the extended distance.
Optimized switch buffering: When Extended Fabrics is installed on gateway switches (with E_Port connectivity from one switch to another), the ISLs/IFLs (E_Ports) are configured with a large pool of buffer credits. The enhanced switch buffers help ensure that that data transfer can occur at the near-full bandwidth to use the connection over the extended links efficiently. This efficiency ensures the highest possible performance on ISLs/IFLs.
Long-Distance Link Modes:
Use the portCfgLongDistance command to support long-distance links and to allocate sufficient numbers of full-size frame buffers on a specific port. Changes made by this command are persistent across switch reboots and power cycles. The portCfgLongDistance command supports the following long-distance link modes:
Normal Mode (L0): L0 is the normal (default) mode for an E_Port. It configures the E_Port as a standard (not long-distance) ISL. A total of 20 full-size frame buffers are reserved for data traffic, regardless of the E_Port’s operating speed. The maximum supported link distance is up to 5 km at 2G, up to 2 km at 4G, and up to 1 km at 8G, 10G, and 16G.
Extended Mode (LE): LE configures the distance for an E_Port when that distance is greater than 5 km and up to 10 km. LE does not require an Extended Fabrics license. The baseline for the buffer credit calculation is one buffer credit per km at 2G. This allocation yields the following values for 10 km:
Ten buffer credits per port at 2G
Twenty buffer credits per port at 4G
Forty buffer credits per port at 8G
Fifty buffer credits per port at 10G
Eighty buffer credits per port at 16G
One Hundred Eighty buffer credits per port at 32G
Dynamic Mode (LD): LD calculates buffer credits based on the distance measured during port initialization. Brocade switches use a proprietary algorithm to estimate distance across an ISL. The estimated distance determines the buffer credits required in LD (dynamic) extended link mode based on a maximum Fibre Channel payload size of 2,112 bytes. You can place an upper limit on the calculation by providing a desired_distance value. Fabric OS confines user entries to no larger than what it has estimated the distance to be. When the measured distance is more than the specified wanted distance, the wanted distance (the smaller value) is used in the calculation.
Static Mode (LS): LS calculates a static number of buffer credits based only on a user-defined desired_distance value. LS mode also assumes that all FC payloads are 2,112 bytes. Specify LS mode to configure a static long-distance link.
Example of configuring the LS type on an ISL:
Below Port is an ISL and configured as "E-Ports" initially having buffer credit assigned as 34.
- Check the Port speed and type set. Verify or confirm the actual distance between the ISL ports from one end to the other end of the switch. If the distance is more than 10 km, then LS or LD must be configured on ISL port.
/fabos/bin/switchshow : switchName: Switch A switchType: 162.6 switchState: Online switchMode: Native switchRole: Principal switchDomain: 10 switchId: fffc0a switchWwn: xx:xx:xx:xx:xx:xx:xx:xx zoning: ON (FABRIC_A) Index Port Address Media Speed State Proto ================================================== 43 43 0a2b00 id 16G Online FC E-Port (Trunk master) /fabos/cliexec/portbuffershow : User Port Lx Max/Resv Avg Buffer Usage & FrameSize Buffer Needed Link Remaining Port Type Mode Buffers Tx Rx Usage Buffers Distance Buffers ---- ---- ---- ---- ------- ---------------------------- ------ ------- --------- ---------- 43 E - 20 3(1364) - ( 308) 34 34 <2km
- Run the "portBufferCalc" command to calculate the number of buffers required per port.
Ex : switch:admin> portbuffercalc 9/4 \ -distance 100 -speed 8 -framesize 512 1606 buffers required for 100km at 8G and framesize of 512 bytes
- Verifying Extended Fabric on switch:
/fabos/link_sbin/license --show: License Id : xx:xx:xx:xx:xx:xx:xx:xx License 1 : ------------------------------------------------------------- License features : Extended Fabric
- Disable the ISL port and run below commands to configure "LS" mode on ISL port:
switch:admin:FID128:admin> portdisable 43 switch:admin:FID128:admin> portcfglongdistance 43 LS 1 -distance 100 Warning: port (43) may be reserving more credits depending on port speed. switch:admin:FID128:admin> portenable 43
- Verify the buffer credits:
CURRENT CONTEXT -- 0, 128 /fabos/cliexec/portbuffershow : User Port Lx Max/Resv Avg Buffer Usage & FrameSize Buffer Needed Link Remaining Port Type Mode Buffers Tx Rx Usage Buffers Distance Buffers ---- ---- ---- ---- ------- ---------------------------- ------ ------- --------- ---------- 43 E LS 806 187(1904) 185(1884) 806 806 100km
- Verify the switch ports status from the "Switchshow" command:
Index Port Address Media Speed State Proto ================================================== 43 43 0a2b00 id N16 Online FC LS
Additional Information
Command reference guide: https://docs.broadcom.com/doc/FOS-82x-Command-RM
