I'm trying to figure out which of the following bridge cards is the right one to use for installing Mellanox part MCX384A-BCAA // Dell part 3CYRK into C6320.... (and if 3CYRK is compatible with C6320 at all).
Here are part numbers for different PCIe bridge card PCB's that may fit but I cant really tell the differences between them. These descriptions are from a random parts list on the Internet so I don't know if they are correct.
... that has a HH4P1 card plugged into what they say is a Mellanox XDTDX // MCX344A 10Gb NIC, and that listing said it was for the C6320. When I tried to use an HH4P1 with MCX384A-BCAA // Dell part 3CYRK my system would not power on. My current theory is that HH4P1 may be for C6220 and that I need 4NWN5 for C6320 for either of these mezzanine boards.
Part# 4NWN5 and MW5RY seems to be on the right track. Both of them are alternate parts of each other. Though, there is only 1 difference between them is in the description, V2.V2 and V2.V1. You may need to know which version of main board that you are using.
Though 3CYRK; I'm not seeing it listed in C6320 as compatible part.
Thanks Joey. Do you happen to have a list of compatible Mellanox mezzanine boards for that unit? Mostly looking for dual port options, probably QSFP..
The manual has a list but no part numbers.... based on this list maybe only the VPI models or QSFP may work. That would be a bummer. I was hoping to get away with this model because it has the features I need and the base model number is the same so that made me think there was a chance.
PN# Y5WKX would fit your request.
There is something I stumble upon searching the part for you. I found HH4P1 and 4NWN5 is listed in C6320 component list, and so probably you might want to keep HH4P1.
Thank you for the info.
As a test, I decided to plug HH4P1 into the motherboard (without a mezzanine card attached) and the sled would NOT power up. I would *think* that the system should still power up like this. So that makes me think that one of these applies: (1) not the right card for my system, (2) sysem requires mezzanine card to complete the circuitry or something and it is expected to fail without it, or (3) the HH4P1 card could be defective but I think this is unlikely.
I may place an order for board Y5WKX to go along wih the 4NWN5 bridge cards on the way just in case these other MCX384A units still don't work. Then I wil be fairly confident I'll have a concrete solution next week.... with two mezzanine adapter types and three brdge card types on hand to test.
Here are a couple of card variations and as can be seen there appear to be variations in the PCB and mapping of the pins. So these boards may be phsically interchangable but probably not compatible with each other. I'm still trying to figure out if the socket on motherboard is universal across systems or if the pinout on that even changes.
2NYRW below seems to be for C8220x. That big hole in the middle is curious... wonder if it is used somehow.
I'm under the assumption that we're dealing with standard PCIe over a NON-standard interconnect. Fairly sure these devices show up as PCIe in the system. My best guess is that different devices need different bridge cards , probably in part because some devices need more power. When the designers are adapting PCIe cards for mezzanine use it looks like they may use wider lanes on the PCB for carrying it.
It sure is frustrating that Dell doesn't seem to have minimal public specs available for their products, nor do they list part numbers in the manual. Seems like a concious decision, perhaps to try to drive professional/tech support services and to hasten the retirement of older equipment. Not great for the earth to push things to obsolescence. Is it too much to expect SOME type of information when typing a part number like HH4P1 on Dell.com?
@DELL-Joey C I'm very confused... I have both Y5WKX and 3CYRK now. Which card is the better one?
You said that Y5WKX is showing as compatible with C6320 and it comes up as NOT PRO model, whereas the 3CYRK is I think the older card sold for the C6220 and it is coming up as PRO model??
3CYRK: MT27500 Family [ConnectX-3] <<< NOT PRO
Y5WKX: MT27520 Family [ConnectX-3 Pro]
(also I can confirm that these work with 4NWN5 bridge card.... the sleds will not even power on wwith HH card installed)
Thanks for your help!
CORRECTION... I reversed the part numbers in my last post!! These are coming up for me as follows....
Y5WKX: MT27500 Family [ConnectX-3] <<< NOT PRO
3CYRK: MT27520 Family [ConnectX-3 Pro]
Well 3CYRK would not work on PowerEdge C. It's for DCS systems. I would wonder if the system is booting up and detecting it? If yes, do let me know.
Between the Pro and non Pro model, it's the feature in the cards. https://dell.to/2B6ZsF0
Yes 3CYRK is working great. It shows as PRO and the other one only shows as ConnectX-3 (not pro) inside the system devices.
The only difference in the manual link you provide (scanning "applies to" and "available in" sections) is support for NVGRE in the PRO version.
On a Mellanox chart, the differences I can see are (1) asterisk showing "partial support" for RoCE on the non-pro card, (2) the aforementioned NVGRE difference as under "Overlay Network/Stateless Offload" it says SUPPORTED for the PRO card, and (3) RoCE Congestion Control shows SUPPORTED for PRO card.
So as far as I can tell, there are only two options for QSFP mezzanine:
MCX384A-BCAA: ConnectX-3 PRO with at least extra features listed above but NO VPI
MCX384A-FCAA: ConnectX-3 (not pro) WITH VPI but lacking above features.
So if planning to run the card in Ethernet mode, it seems the BCAA is the better card.