Power Class Mismatch Between Active Copper Breakout Cable to PowerScale Node Causes No Link

A 400GbE capable switch using a 400GbE QSFP56 to 4xQSFP28 100GbE active copper breakout cable connected to a PowerScale node 100GbE port does not come up.

The output of ifconfig -v mce<#> from the node returns a status: no carrier.

cluster-1# ifconfig -v mce3
...
status: no carrier
...

The output of mst status shows a ConnectX-5 NIC. Here pci0:59 is the Front-End (FE) NIC, and pci0:216 is the Backend-End (BE) NIC.

cluster-1# mst status
MST devices:
------------
  pci0:59:0:0   -       MT28800 Family [ConnectX-5 Ex]
  pci0:216:0:0  -       MT28800 Family [ConnectX-5 Ex]

The output of mlxlink -d <pci value> -m from the node (where the pci value can be obtained from the output of mst status and the last digit of the pci value can be either 0 or 1 depending on port # cable is plugged into) shows the power class greater than 4 for the cable in the Power Class field.

cluster-1# mlxlink -d pci0:59:0:0 -m

Module Info
-----------
...
Power Class                     : 5.0 W max
...

The following log line can be seen in /var/log/messages file:

2024-08-13T17:13:37.185278+01:00 <0.7> - /boot/kernel.amd64/kernel: mlx5_core3: INFO: Module 1, status: error, Power Budget Exceeded

The QSFP28 100GbE cable end of the 400GbE QSFP56 to 4xQSFP28 100GbE active copper breakout cable is rated at Power Class 5, and the ConnectX-5 NIC is only capable of Power Class 4.

One way forward is to use the following 400GbE QSFP56 to 4xQSFP28 100GbE active optical (AOC) breakout cables. Each QSFP28 100GbE AOC cable end is rated at Power Class 3.

AOC-Q56DD-4Q28-100G-3M JHG00 470-BCKY
AOC-Q56DD-4Q28-100G-7M J6T7 470-AFNJ
AOC-Q56DD-4Q28-100G-15M 9T7JR 470-AFNF
AOC-Q56DD-4Q28-100G-30M W0WXF 470-AFNK