Force 10 MXL 10/40 GB configuration with PC 8132F Swicthes.

What is your switch inter connectivity?

An Equallogic SAN should use a single network for iSCSI. This network should then also have enough bandwidth between the switches.

Each MXL has 2 40Gbit/s QSFP+ ports by default and has 2 slots for optional expansion ports (e.g. 4 x 10Gbit SFP+ ports, 4 x 10Gbit RJ45 ports or 2 more QSFP+ ports).

If your MXL and the 8132F switches will all be dedicated to iSCSI traffic only, I'd suggest redundant connections between everything, but not stack any of the switches.

You could do:

Option 1:

- B1 to B2 via a QSFP+ to QSFP+ cable (giving 40Gbit/s in bandwidth between the MXL switches)

- B1 to 8132F #1 via 2 SFP+ connections (make them a port-channel)

- B2 to 8132F #2 via 2 SFP+ connections (make them a port-channel)

- 8132F #1 to 8132F #2 via 2 or 4 SFP+ connections (make them a port-channel)

Spanning-tree will block one of the connections to prevent a loop, but if a link between switches goes down anywhere, spanning-tree will renegotiate and bring the blocked link up to allow things to work via the alternate path.

Option 2:

- B1 to B2 via a QSFP+ to QSFP+ cable (giving 40Gbit/s in bandwidth between the MXL switches)

- B1 to 8132F #1 via the second QSFP+ port (requires the optional QSFP+ module on the 8132F)

- B2 to 8132F #2 via the second QSFP+ port (requires the optional QSFP+ module on the 8132F)

- 8132F #1 to 8132F #2 via the second QSFP+ port on the QSFP+ module

This looks like a setup with no redundant connections between the switches, but due to the loop design, you get redundancy that way and spanning-tree will take care of this to prevent issues.

Note: On the 8132F spanning-tree is enabled by default, but on the MXL switches it is disabled by default. I'd recommend to set spanning-tree on the inter-switch connections to portfast/edgeport just like you would on an iSCSI port that goes to the 6510 or servers from any of the switches.

For bandwidth reasons I'd highly recommend at least dual connections between the switches (and make them a port-channel to actually get the bandwidth out of the links).

What connection options do you have on the MXL switches? Do you have just the native 2 QSFP+ ports and do you then maybe have 2 QSFP+-to-4xSFP+ breakout cables and 1 QSFP+-to-QSFP+ cables? Or do you have SFP+ or RJ45 modules for the MXL switches?

I have MXL with 4*10SFP+ module. On each MXL. Two QSFP + on each MXL has been already stacked with each other.

Are the 2 MXL switches stacked? It's a little less overhead, but the major downside is that if you ever want to update their firmware, you will need 100% downtime on any services that require storage access. To me that's a major drawback with stacking ethernet switches. It does however give you the option to run a port-channel to each of the 8132F switches where 1 port for each port-channel is coming from one of the MXL switches.

The 4 SFP+ ports on each MXL are probably ports 41 to 44.

So, if you opt to keep the MXL switches, assuming the 2 8132F switches aren't stacked, and you do want to have the bandwidth between the switches, here are the commands to set up the MXL switch stack and the 8132F switches (note, this assumes the SFP+ ports are 41 and 42 and that both 41's go to one 8132F and the 42-ports go to the other 8132F:

config

interface range te0/41 , te1/41

port-channel-protocol lacp

port-channel 1 mode active

exit

interface range te0/42 , te1/42

port-channel-protocol lacp

port-channel 2 mode active

exit

interface port-channel 1

no shutdown

description "Port-channel to 8132F-1"

exit

interface port-channel 2

no shutdown

switchport

description "Port-channel to 8132F-2"

exit

interface range te0/1-32 , te1/1-32

no shutdown

switchport

mtu 9252

spanning-tree rstp edge-port

flowcontrol rx on tx off

exit

protocol spanning-tree rstp

no disable

exit

For your iSCSI VLAN (as you don't want to leave iSCSI on the default VLAN (best practices), you could either untag the port-channels if you don't think you'll ever run any other traffic across these switches, or if you want to be more ready in case you ever do need to run other traffic (e.g. heartbeat or vMotion), then you tag the port-channels for the iSCSI VLAN.

To do the VLAN part, assuming your iSCSI VLAN is 20 and you do want to tag the iSCSI traffic, the commands would be:

interface vlan 20

tagged port-channel 1

tagged port-channel 2

untagged te0/1-32,te1/1-32

Then on the 8132F-1 and 8132F-2 switches you use these commands (note that in this example I'm assuming you're using ports 1 and 24 for the connection to the 2 MXL switches, if this is not the case, change this in the commands below to reflect the actual ports being used):

interface range te1/0/1,te1/0/24

channel-group 1 mode auto

exit

interface port-channel 1

mtu 9216

switchport mode trunk

switchport trunk allowed vlan add 20

And on the 8132F-2 you do the same, but optionally change the port-channel number to 2 (though this isn't a requirement).

If you then also want to have a link between the 2 8132F's, you would use a similar setup between them. Assuming these switches would be linked together via ports 2 and 23, the commands for that are:

interface range te1/0/2,te1/0/23

channel-group 3 mode auto

exit

interface port-channel 3

mtu 9216

On the 8132 ports that go to the Equallogic SAN, or to any servers that aren't in the M1000e blade chassis, you would use these commands:

interface range te1/0/3-1/0/22

mtu 9216

spanning-tree portfast

switchport access vlan 20

exit

interface port-channel 1-3

switchport mode trunk

switchport trunk allowed vlan add 20

exit

Do remember to save the settings ('write memory' on Force10 and 'copy running-config startup-config' on PowerConnect), and I'd also suggest to pull a backup of the configs, just in case you ever have to replace a whole switch.

Let me know if you need any clarification or a different config.