Highlighted
2 Iron

Differences in Mezzanine Bridge Cards for PowerEdge C

I'm trying to figure out which of the following bridge cards is the right one to use for installing Mellanox part MCX384A-BCAA // Dell part 3CYRK into C6320....  (and if 3CYRK is compatible with C6320 at all). 

Here are part numbers for different PCIe bridge card PCB's that may fit but I cant really tell the differences between them.  These descriptions are from a random parts list on the Internet so I don't know if they are correct.

JKM5M  Printed Wiring Assy.Daughterboard.Serial Attached Scsi.Bridge.X3.Data Center Solutions
HH4P1 Card.Interface.Mezzanine.Bridge.C6220.CLE
2NYRW  Card.Expansion. Expand.Bridge.C8000
4NWN5   Printed Wiring Assy.Daughterboard.Mezzanine.Bridge.V2.V1.6300
MW5RY   Printed Wiring Assy.Daughterboard.Mezzanine.Bridge.V2.V2.6300
 
The 4NWN5 card seems most often listed as compatible with C6320 on random websites and ebay listings.  Is MW5RY interchangable with 4NWN5 or are they different?  The shape is slightly diffferent but MW5RY also frequenly is said to be for the C6320.
 
But do different devices use different bridge cards?  I found a photo in an eBay listing (see attached) Screenshot 2020-06-16 at 2.08.51 PM.png

 

... that has a HH4P1 card plugged into what they say is a Mellanox XDTDX // MCX344A 10Gb NIC, and that listing said it was for the C6320.   When I tried to use an HH4P1 with MCX384A-BCAA // Dell part 3CYRK my system would not power on.   My current theory is that HH4P1 may be for C6220 and that I need 4NWN5 for C6320 for either of these mezzanine boards.
 
Can any one offer any insight?  Frustrating that there is so little documenation on Dell parts.
0 Kudos
24 Replies
Highlighted
Moderator
Moderator

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

Hello,

 

Part# 4NWN5 and MW5RY seems to be on the right track. Both of them are alternate parts of each other. Though, there is only 1 difference between them is in the description, V2.V2 and V2.V1. You may need to know which version of main board that you are using. 

 

Though 3CYRK; I'm not seeing it listed in C6320 as compatible part. 

 

 


Thanks,
Joey Chong
Dell EMC Enterprise Support Services
Get support on Twitter @DellCaresPro

Download the Dell Quick Resource Locator app today to access PowerEdge support content on your mobile device! (iOS, Android, Windows)

0 Kudos
Highlighted
2 Iron

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

Thanks Joey.  Do you happen to have a list of compatible Mellanox mezzanine boards for that unit?   Mostly looking for dual port options, probably QSFP..

The manual has a list but no part numbers.... based on this list maybe only the VPI models or QSFP may work.  That would be a bummer.   I was hoping to get away with this model because it has the features I need and the base model number is the same so that made me think there was a chance. 

https://www.dell.com/support/manuals/us/en/19/poweredge-c6320/c6320_pub/optional-mezzanine-cards?gui...

 

0 Kudos
Highlighted
Moderator
Moderator

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

Hi,

 

PN# Y5WKX would fit your request. 

 

There is something I stumble upon searching the part for you. I found HH4P1 and 4NWN5 is listed in C6320 component list, and so probably you might want to keep HH4P1.


Thanks,
Joey Chong
Dell EMC Enterprise Support Services
Get support on Twitter @DellCaresPro

Download the Dell Quick Resource Locator app today to access PowerEdge support content on your mobile device! (iOS, Android, Windows)

Highlighted
2 Iron

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

Thank you for the info.

As a test, I decided to plug HH4P1 into the motherboard (without a mezzanine card attached) and the sled would NOT power up.   I would *think* that the system should still power up like this.   So that makes me think that one of these applies: (1) not the right card for my system, (2) sysem requires mezzanine card to complete the circuitry or something and it is expected to fail without it, or (3) the HH4P1 card could be defective but I think this is unlikely.

I may place an order for board Y5WKX to go along wih the 4NWN5 bridge cards on the way just in case these other MCX384A units still don't work.  Then I wil be fairly confident I'll have a concrete solution next week.... with two mezzanine adapter types and three brdge card types on hand to test.

0 Kudos
Highlighted
2 Iron

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

Here are a couple of card variations and as can be seen there appear to be variations in the PCB and mapping of the pins.   So these boards may be phsically interchangable but probably not compatible with each other.   I'm still trying to figure out if the socket on motherboard is universal across systems or if the pinout on that even changes.

2NYRW below seems to be for C8220x.  That big hole in the middle is curious... wonder if it is used somehow.

I'm under the assumption that we're dealing with standard PCIe over a NON-standard interconnect.  Fairly sure these devices show up as PCIe in the system.   My best guess is that different devices need different bridge cards , probably in part because some devices need more power.  When the designers are adapting PCIe cards for mezzanine use it looks like they may use wider lanes on the PCB for carrying it.

It sure is frustrating that Dell doesn't seem to have minimal public specs available for their products, nor do they list part numbers in the manual. Seems like a concious decision, perhaps to try to drive professional/tech support services and to hasten the retirement of older equipment.   Not great for the earth to push things to obsolescence.  Is it too much to expect SOME type of information when typing a part number like HH4P1 on Dell.com?

 

IMG_3804.jpeg

 

IMG_3805.jpeg

 

IMG_3806.jpeg

 

IMG_3807.jpeg

0 Kudos
Highlighted
2 Iron

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

@DELL-Joey C  I'm very confused... I have both Y5WKX and 3CYRK now.   Which card is the better one?

You said that Y5WKX is showing as compatible with C6320 and it comes up as NOT PRO model, whereas the 3CYRK is I think the older card sold for the C6220 and it is coming up as PRO model??

3CYRK: MT27500 Family [ConnectX-3] <<< NOT PRO

Y5WKX: MT27520 Family [ConnectX-3 Pro]

(also I can confirm that these work with 4NWN5 bridge card.... the sleds will not even power on wwith HH card installed)

Thanks for your help!

0 Kudos
Highlighted
2 Iron

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

CORRECTION... I reversed the part numbers in my last post!!  These are coming up for me as follows....

Y5WKX: MT27500 Family [ConnectX-3] <<< NOT PRO

3CYRK: MT27520 Family [ConnectX-3 Pro]

0 Kudos
Highlighted
Moderator
Moderator

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

Hello,

 

Well 3CYRK would not work on PowerEdge C. It's for DCS systems. I would wonder if the system is booting up and detecting it? If yes, do let me know.

 

Between the Pro and non Pro model, it's the feature in the cards. https://dell.to/2B6ZsF0


Thanks,
Joey Chong
Dell EMC Enterprise Support Services
Get support on Twitter @DellCaresPro

Download the Dell Quick Resource Locator app today to access PowerEdge support content on your mobile device! (iOS, Android, Windows)

0 Kudos
Highlighted
2 Iron

Re: Differences in Mezzanine Bridge Cards for PowerEdge C

Yes 3CYRK is working great.  It shows as PRO and the other one only shows as ConnectX-3 (not pro) inside the system devices.

The only difference in the manual link you provide (scanning "applies to" and "available in" sections) is support for NVGRE in the PRO version.

On a Mellanox chart, the differences I can see are (1) asterisk showing "partial support" for RoCE on the non-pro card, (2) the aforementioned NVGRE difference as under "Overlay Network/Stateless Offload" it says SUPPORTED for the PRO card, and (3) RoCE Congestion Control shows SUPPORTED for PRO card.

https://community.mellanox.com/s/article/mellanox-adapters---comparison-table

So as far as I can tell, there are only two options for QSFP mezzanine:

MCX384A-BCAA: ConnectX-3 PRO with at least extra features listed above but NO VPI

MCX384A-FCAA: ConnectX-3 (not pro) WITH VPI but lacking above features.

So if planning to run the card in Ethernet mode, it seems the BCAA is the better card.

 

0 Kudos