Dell Force 10 MXL 10/40GbE switch

Hi cswright,

What model blade is it and which NIC? What fabric is the MXL in the chassis? If the autonegotiate is setting it to 1Gb there is probably something that is causing it to not connect at 10G and why hard setting the speed drops the connection.

Poweredge M610, Broadcom BCM57711 NetXtreme II 10 GigE. The fabric is B1.

Thanks. Are you using the latest driver? http://www.dell.com/support/home/us/en/04/Drivers/DriversDetails?driverId=GPJ8K&fileId=3389738638&osCode=WS8R2&productCode=poweredge-m610&languageCode=EN&categoryId=NI

Is this a new blade chassis or an existing one with an upgraded switch? Is the CMC firmware up to date? http://www.dell.com/support/home/us/en/04/Drivers/DriversDetails?driverId=D2KPX&fileId=3325047092&osCode=WNET&productCode=poweredge-m1000e&languageCode=EN&categoryId=CE

What does it show for the midplane version?

I am using the latest driver. this is a new blade chassis for us. The blade Chassis is running  version 4.50 and the midplane version is 1.0

Thanks for your help. That is the issue. I contacted the company got the Chassis from to get that midplane replaced with a 1.1. Thank you for your help!

You shouldn’t need to do anything else. The auto negotiate is still at 1Gb and if you hard set 10Gb no network still? We can see if there are any errors in the CMC logs. If you use Putty to connect to the CMC and enable logging of the output we can create a file for the logs. From the command prompt run racdump. The output should a section that looks like the following

Switch-5  Dell PowerConnect M8024        10 GbE XAUI        Present      OK           ON          Master  
Switch-6  Dell PowerConnect M8024        10 GbE XAUI        Present      OK           ON          Master 

                                
server-1   Present       Gigabit Ethernet    OK            10 GbE XAUI         OK            
server-2   Present       Gigabit Ethernet    OK            10 GbE XAUI         OK            
server-3   Present       Gigabit Ethernet    OK            10 GbE XAUI         OK            
server-4   Present       Gigabit Ethernet    OK            10 GbE XAUI         OK            
server-5   Present       Gigabit Ethernet    OK            10 GbE XAUI         OK            
server-6   Present       Gigabit Ethernet    OK            10 GbE KR           OK 

I have replaced the midplane with a 1.1 though my interfaces for my blades are still showing 1gb connection. Anything else I need to do to make this connect at 10gb? Thanks.

This is the only switch I have in there right now. Here is the information you asked for

Switch-3  MXL 10/40GbE                   10 GbE KR          Present      OK                                                                ON          Master

That looks correct, what do the servers show?

Server shows Broadcom BCM57711 Netxtreme 2 10GB card is connected at 1.0GBPS. Current driver on that nic card is 2/7/2014 driver version 7.10.6.0

I am having the same problem. I have a load of R610 servers with BCM57711 NICs in them. They were originally installed in a M1000e which had a v1.1 midplane and a SFP+ passthrough module going to a standard 10Gbe switch. We moved one of the R610s to a newer M1000e blade centre with a v1.1 midplane and a pair of Force10 MXL10/40Gbe switches. They are in fabric B and configured as a stack. The M610s will only connect to this switch at 1Gbps. Try to force it to 10Gbps and it says Connection Disconnected.

I have updated the firmware and drivers to the latest QLogic ones found on the Dell website. I have tried this with two R610s in two slots and it's the same in each configuration.

The same blade centre hosts newer M820 and M520 servers with BCM58xxx NICs and those connect to the Force10 stack at 10Gbps as expected. The R610s in our midplane v1.0 enclosure connect to our core switches using the SFP+ passthrough modules with no issue. Is there a way around this?

I am having the same problem. I have a load of R610 servers with BCM57711 NICs in them. They were originally installed in a M1000e which had a v1.0 midplane and a SFP+ passthrough module going to a standard 10Gbe switch. We moved one of the R610s to a newer M1000e blade centre with a v1.1 midplane and a pair of Force10 MXL10/40Gbe switches. They are in fabric B and configured as a stack. The M610s will only connect to this switch at 1Gbps. Try to force it to 10Gbps and it says Connection Disconnected.

I have updated the firmware and drivers to the latest QLogic ones found on the Dell website. I have tried this with two R610s in two slots and it's the same in each configuration.

The same blade centre hosts newer M820 and M520 servers with BCM58xxx NICs and those connect to the Force10 stack at 10Gbps as expected. The R610s in our midplane v1.0 enclosure connect to our core switches using the SFP+ passthrough modules with no issue. Is there a way around this?

I figured out what the issue was. The 1.0 backplane we had will only do 1GB. So we replaced it with a 1.1. We still had the issue with the 1gb connection and disconnecting when forcing 10gb. The reason for that is because BCM57711 are legacy cards and will not do 10gb. We purchased Intel x520 mezzanine cards the part number is 8F6NV. We now have 3 m1000e chassis and all our blades are running these intels at 10gb. Hopefully you are able to get your cards swapped out. Took ALOT of work to get this information.

Thanks, after a bit of messing about I've come to the same conclusion. Unfortunately this servers are two years out of warranty so any new mezzanine cards would have to be purchased. I'm going to contact my account manager and see how much they cost.

Cheers,

Ian