Start a Conversation

Unsolved

C

43 Posts

3656

June 16th, 2020 14:00

Differences in Mezzanine Bridge Cards for PowerEdge C

I'm trying to figure out which of the following bridge cards is the right one to use for installing Mellanox part MCX384A-BCAA // Dell part 3CYRK into C6320....  (and if 3CYRK is compatible with C6320 at all). 

Here are part numbers for different PCIe bridge card PCB's that may fit but I cant really tell the differences between them.  These descriptions are from a random parts list on the Internet so I don't know if they are correct.

JKM5M  Printed Wiring Assy.Daughterboard.Serial Attached Scsi.Bridge.X3.Data Center Solutions
HH4P1 Card.Interface.Mezzanine.Bridge.C6220.CLE
2NYRW  Card.Expansion. Expand.Bridge.C8000
4NWN5   Printed Wiring Assy.Daughterboard.Mezzanine.Bridge.V2.V1.6300
MW5RY   Printed Wiring Assy.Daughterboard.Mezzanine.Bridge.V2.V2.6300
 
The 4NWN5 card seems most often listed as compatible with C6320 on random websites and ebay listings.  Is MW5RY interchangable with 4NWN5 or are they different?  The shape is slightly diffferent but MW5RY also frequenly is said to be for the C6320.
 
But do different devices use different bridge cards?  I found a photo in an eBay listing (see attached) Screenshot 2020-06-16 at 2.08.51 PM.png

 

... that has a HH4P1 card plugged into what they say is a Mellanox  XDTDX // MCX344A 10Gb NIC, and that listing said it was for the C6320.   When I tried to use an HH4P1 with MCX3 84A-BCAA // Dell part 3CYRK my system would not power on.   My current theory is that HH4P1 may be for C6220 and that I need 4NWN5 for C6320 for either of these mezzanine boards.
 
Can any one offer any insight?  Frustrating that there is so little documenation on Dell parts.

43 Posts

July 2nd, 2020 14:00

That page is for switches, not mezzanine PCI cards for servers.

I am looking for firmware like at the following link...
https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=0dtdc

... and I assume the RedHat option will work even though I am trying to update from CentOS.   If you can tell me what server models you show as compatible systems for 3CYRK then it *MAY* be listed under firmware downloads for that system (but I will need to go through process described below to confirm).

There are dozens of Dell firmware pages that come up like that when searching for "dell connectx3 firmware" on google.  But the pages do not say what dell part number they are for.  So I started blindly trying the different updates, and then I learned that in the logs and by unpacking the BIN I can see the Dell part number in the payload filenames (shown in the list in previous post for what I've dug through so far).

The problem (I'm guessing) is that inside the BIN files, the firmware only appears to apply to very specific NIC cards. That's what I listed above.  So for fimware M8H5D ... the payload files inside the BIN include "fw-ConnectX3Pro-rel-2_42_5000-0C8Y42_0R3F0N-UEFI-15.11.45-FlexBoot-3.4.752.bin"

and then I google the part numbers in the filename and find out that Dell part R3F0N is a regular PCIe card and  equivalent to Mellanox part CX324A etc.   It is still a Connectx3 card but wrong dell and mellanox part numbers for what I need.   My C6320 system is listed as a compatible system but only if that specific card were in my system, I guess.

If I try to find Connectx3 pro firmware under C6320 downloads I seem to be having a hard time seeing anything at all, even though there are DOZENS of the above pages listing my system.   

This can't be the only way to find the firmware for a device, I hope?!  I'm really baffled as to why things are so difficult with Dell hardware.  There is so little information available.   And I am guessing that you guys don't have a system to find everything there is to know about a part by entering the dell number?

43 Posts

July 2nd, 2020 15:00

A little update/clue -- I did manage to bring up some firmware under C6320.   Versions are very old, however I did find some clues inside one of them.

https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=fhfx6&oscode=naa&productcode=poweredge-c6320

https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=fhfx6&oscode=naa&productcode=poweredge-c6320

Under both of these is an option for "base drivers for imaging" which I assume are the "payload" files found embedded in the other type of BIN files (the Dell installer BIN).  These "base drives" are intended to be updated using MFT (Mellanox Firmware Tools).

Inside the FHFX6 firmware there is a PDF with release notes saying that the RELEASE applies to many different Dell part numbers, including 3CYRK (and it shows that PN crossing to PSID DEL1110001023).   So at least we're in the right family of firmware.   But the ONLY DELL PN in the firmware filename is XDTDX ..... and that part number isn't even listed with others in the release notes!!

Many if not all of the other PN's in the document are mezzanine cards, so perhaps this same binary applies to all of them. 

So, on a hunch, I checked under C6220-II to see what Mellanox firmware I could find for that system, since the 3CYRK card may have been sold for that model, and I found this:

https://www.dell.com/support/home/en-gu/drivers/driversdetails?driverid=h6vxh&oscode=naa&productcode=poweredge-c6220-2

Downloaded Firmware_H6VXH_LN_2.32.5100.BIN file and extracted as below.   Things to note:

  • we can see the 0XDTDX part number again, and even though there are no release notes in this package,  under the C6320 download package the release notes did suggest we are in the right family of card (even though XDTDX was not in release notes itself... ugh)
  • DEVICE ID MATCHES MY CARD!
  • SUBDEVICE ID DOES NOT match my card!

 

penguin:~$ chmod +x Firmware_H6VXH_LN_2.32.5100.BIN
penguin:~$ ./Firmware_H6VXH_LN_2.32.5100.BIN --extract H6VXH
penguin:~$ ls H6VXH/payload/
configfile.xml  fw-ConnectX3Pro-rel-2_32_5100-0XDTDX-FlexBoot-3.4.306_ETH-UEFI-10.5.16.bin  UpdRollBack.lst
penguin:~$ cat H6VXH/payload/configfile.xml 


  
    
      
        
      
    
    
    
      00000000-0000-0000-0000000000000000
      0
      
      
    
    
      3792094f-4eb9-11e2-9d5af9377442e396
      2339170
      02.33.5100
      fw-ConnectX3Pro-rel-2_32_5100-0XDTDX-FlexBoot-3.4.306_ETH-UEFI-10.5.16.bin
    
  

 

 

Moderator

 • 

3.4K Posts

July 2nd, 2020 18:00

Hi @CA_Tallguy,

 

The URL https://dell.to/3iszJYs i gave did go to the switch firmware but it's the 1st tab it loaded in, you can click on the 2nd tab which brings you to adapters? Under Dell EMC ConnectX-3 Firmware Download Center, there's firmware 2.42.5 with different PN#. 

 

Hope this helps? Let me know.

43 Posts

July 2nd, 2020 19:00

Ahh yes I did miss that tab.  Thanks.

This link will go straight to it (posting for anyone else coming along who might miss that too): https://www.mellanox.com/support/firmware/dell?mtag=oem_firmware_download&linkId=92599977&linkId=92710159#tabs-2

I feel like we're getting warmer and zeroing in on the right firmware.  I sent an email to Mellanox support to see if they can point to the specific firmware to use.

Thanks for your help.

43 Posts

July 5th, 2020 12:00

Mellanox support isn't providing help as I am not their customer and they consider the product to be a Dell product.

So I am back here hoping someone at Dell can tell me where to find the latest firmware for this card.  It has to be available somewhere.

The card has an automated update utility built in but it only works for their own directly sold cards.

Device Type: ConnectX3Pro
Part Number: 03CYRK
Description: MCX384A-BCCA ConnectX-3 Pro Eth; dual port QSFP; 40Gb/s Mezz card for Dell
PSID: DEL1110001023
PCI Device Name: 13:00.0
Port1 MAC: f45214057490
Port2 MAC: f45214057491
Versions: Current Available
FW 2.30.4212 N/A
PXE 3.4.0134 N/A

Status: No matching image found

The firmware for this device is not distributed inside Mellanox driver: 0000:13:00.0 (PSID: DEL1110001023)
To obtain firmware for this device, please contact your HW vendor.

Moderator

 • 

3.4K Posts

July 5th, 2020 19:00

Hello,

 

I traced back which the card is compatible with on systems, I only found DSS1500 and DSS7500. And based on the systems, I found:

 

https://dell.to/3gvSgS8 DSS1500

 

https://dell.to/2ZEuQ6e DSS7500

 

I hope this helps. I'm not very verse with DSS or DSC systems, so I'm unsure if the firmware may work on C6320.

 

Though, the card PN# Y5WKX has the firmware in the download page, you're not planning to use Y5WKX instead?

43 Posts

July 6th, 2020 01:00

Thanks Joey.  I have updated the Y5WKX card from the link posted to Mellanox site but I'm still looking for 3CYRK firmware for 3 other cards.

Unfortunately, however, I don't think the linked downloads in your last post are quite right.  The 3CYRK cards are DeviceID 1007 and SubDeviceID 10 and those are not present in the configuration package and  also there is no firmware file named for that card.  The Dell product ID should be in the filename, I believe, or at least I believe that the SubDeviceID needs to be present in the config.  I think it is likely both should match.

Here's how I am examining them.... under most of these Dell driver pages there are BIN and EXE dowload options, or sometimes a ZIP file labeled as "base drivers for imaging" as in one of your links.   I showed the series of commands I am using to extract from the BIN files (the main command is in format below (see the full sequence a few posts back).....

./Firmware_H6VXH_LN_2.32.5100.BIN --extract H6VXH

That unpacks the BIN into a directory (H6VXH) in above example.

The zip files can of course be examined and the EXE files I have turned to as a last resort have been self extracting archives so those files can also be examined after execution.

So on your most recent links they have nothing for 3CYRK cards are DeviceID 1007 and SubDeviceID 10..

T5Y3H download config file....
































T5Y3H download firmware files.... (3CYRK not present)....

 fw-ConnectX3Pro-rel-2_34_5060-019RNV_0YHTD6-FlexBoot-3.4.521-UEFI-14.7.28.bin
 fw-ConnectX3Pro-rel-2_34_5060-02T0WW-FlexBoot-3.4.521-CLP-8006.0.0-UEFI-14.7.28.bin
 fw-ConnectX3Pro-rel-2_34_5060-0C8Y42_0R3F0N-FlexBoot-3.4.521-UEFI-14.7.28.bin
 fw-ConnectX3-rel-2_34_5060-06256K-FlexBoot-3.4.521-CLP-8005-UEFI-14.7.28.bin
 fw-ConnectX3-rel-2_34_5060-08KP6W_0M9NW6-FlexBoot-3.4.521-UEFI-14.7.28.bin
 fw-ConnectX3-rel-2_34_5060-0W0RM9_0Y3KKR-FlexBoot-3.4.521-UEFI-14.7.28.bin

X1T4W zip download contains only one firmware file (for Dell product ID R17HV

fw-ConnectX3Pro-rel-2_35_5100-0R17HV_Ax-FlexBoot-3.4.648-UEFI-14.8.43.bin

Moderator

 • 

3.4K Posts

July 6th, 2020 02:00

Hi,

 

I'm out of idea on how I can search for the firmware. I can see that you have posted on Mellanox's forum on it. Probably one would be able to assist you. 

43 Posts

July 6th, 2020 12:00

I sincerely appreciate your help, Joey, and that of your associates.  I don't expect the Mellanox people to help.  The support email already told me to post on their boards, and I have seen similar firmware inquiries on their message boards and they pretty much tell the users to contact the manufacturer for any custom firmware.

But there is already another Dell user who replied to the thread over there looking for the exact same firmware.

I have looked at the manuals for the DSS systems you referenced and they don't appear to be compatible with the 3CYRK card.   They look to only have PCIe slots (standard) not mezzanine boards like the 3CYRK.    So I am still unsure of what system this card was sold for, and the way Dell has the firmware organized, it makes it difficult to find any firmware without that information.

Mellanox support has already punted the question over to their community forum but similar forum questions often elicit a response from a Mellanox employee directing users back to the "manufacturer" such as Dell or HPE and telling the user they do not have a support contract.

It's shocking that it is this difficult to find a simple firmware file. I can understand selling premium support if a customer is out of warranty and has a technical question but asking for a firmware isn't really "support" for this type of product. It should only rise to "support" if a customer is asking for assistance to APPLY the update.

We are talking about a product of a vintage (CX3) where we can see there are many available updates still posted... it's just impossible to FIND what is needed (or there has been an oversight and this update is missing).

Hopefully someone at Mellanox will realize how  the firmware situation is over at Dell and maybe reach out to their contacts to get it fixed. Finding any Mellanox firmware over here is more luck than anything. And that isn't good for the Mellanox brand.

The specific problem is that Dell download pages have no specific identifiers about Dell or Mellanox part numbers for which they apply. The most specific they get is CX3, CX4, pro etc. and maybe speed and number of ports. Further, there is no index of the pages. So it's actually easier to find Mellanox firmware pages at Dell via Google than via Dell Support pages. But without Dell/Mellanox product ID's on the pages, it's random chance that you will find the update you actually need.

No Events found!

Top