UNSOLVED

Htoumi

updated

4 years ago

H

Htoumi

1 Rookie

•

6 Posts

0

2558

July 13th, 2022 17:00

vrtx (cmc) redundancy is lost

Hello,

We had a current problem that caused the dell VRTX to shut down, the problem is that after the startup we had a critical alert "Chassis management controller (CMC) redundancy is lost".

Actually it is the CMC 2 that works, one of the network cables drives the server is in orange, the reboot of the server has not solved the problem, also the reset of the CMC.

for info after this current incident I have one of the 4 M620 hosts install on the VRTX that no longer wants to light the on/off button flashes green, I do not know if this has anything to do with the problem of the CMC 1.

CMC-< Removed by Moderator >

Version : 2.01

need your help if someone has an idea please.

  • DELL-Joey C

    Moderator

    •

    4170 Posts

    •

    20965 Points

    864

    0

    Posted July 14th, 2022 00:00

    Hi @Htoumi,

     

    It's unlikely that the CMC is causing the M620 not able to be powered on. Previously what happen before the M620 not able to be turned on? I'm not able to understand what you mean "one of the network cables drives the server is in orange", could you rephrase. 

     

    You might need to look into CMC 2's logs to check if there are any logs that informed any hardware issue. 

    DELL-Joey C

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’ if I did. 

  • DiegoLopez

    6 Operator

    •

    2668 Posts

    856

    0

    Posted July 14th, 2022 04:00

    Hello @Htoumi,


    Do you have access to the CMC? Can you check the hardware logs looking for any hardware error? If there is no CMC access you can export the log using a serial connection: https://dell.to/3IAwA6o and generating the logs:  PowerEdge Server: How to generate enclosure logs for CMC/VRTX/FX2 https://dell.to/3aEnaui


    This logs are what Dell support will need to check for hardware issues. However, Dell support will only check the logs if the VRTX is in warranty. If your VRTX is no longer on warranty you will need to open an out of warranty case and ask for a quote for out of warranty reparation.


    Regards.

  • Htoumi

    1 Rookie

    •

    6 Posts

    854

    0

    Posted July 14th, 2022 04:00

    Hello DiegoLopez,

    I have access to the cmc since the CMC2 works, and this is what is in the hardware logs when the VRTX is launched "see capture"

    2022-07-14_12h50_15.png

  • Htoumi

    1 Rookie

    •

    6 Posts

    858

    0

    Posted July 14th, 2022 04:00

    Hi Joey C,

    Thank you very much for your answer.

    Before the server is shut down everything works well, but when I turned on the vrtx I got the lost redundancy message, and I noticed that the M620 doesn’t want to start anymore.

    When I swiped the location of the M620 with one that is below it could start but the one I put on top did not start.

    Regarding the CMC LEDs, there are usually 2 blue LEDs with 2 network cables, now there is an orange LED on the mgmt console, see the capture.

    2022-07-14_11h30_19.png

     

  • DiegoLopez

    6 Operator

    •

    2668 Posts

    847

    0

    Posted July 14th, 2022 05:00

    Hello @Htoumi,


    Why the CMC firmware is so outdated? I think as per your first message is 2.01. Last version is 3.20.
    Is there any reasons to not update the firmware?


    Please, check this article: How to update the CMC of a Dell PowerEdge VRTX - https://dell.to/3P0I8Ct


    Regards.

  • Htoumi

    1 Rookie

    •

    6 Posts

    844

    0

    Posted July 14th, 2022 06:00

    Hello DiegoLopez,

    Thanks,

    Unfortunately the old IT has neglected the firmware update, currently I am trying to recover the CMC 1 to be able to do the update.

    Best regards

  • Htoumi

    1 Rookie

    •

    6 Posts

    810

    0

    Posted July 15th, 2022 05:00

    Hello,

    I wanted to ask you if the update will be able to solve the redundancy problem.

    thanks.

    Best regard

  • DELL-Charles R

    Moderator

    •

    4730 Posts

    •

    25476 Points

    807

    0

    Posted July 15th, 2022 05:00

    Hello Htoumi,

     

    For the error Chassis management controller (CMC) redundancy is lost will be resolved when you can get both CMC online.

     

    You may try shut down the chassis and swap CMC slots to see if the issue stays in the slot or follows the CMC.

    When you do the swap I recommend a power drain to the chassis.

    Shut down any running systems, power down the chassis.

    Remove all power and NIC cables for five minutes, then swap CMC slots.

    Reconnect everything and check results.

     

    DELL-Charles R

    Social Media and Communities Professional
    Dell Technologies | Enterprise Support Services
    #IWork4Dell

    Did I answer your query? Please click on ‘Accept as Solution’. ‘Thumbs up’ the posts you like!

  • Htoumi

    1 Rookie

    •

    6 Posts

    764

    0

    Posted July 16th, 2022 10:00

    Hello Mr, @Dell -Charles R

    I followed the steps you gave me :

    I did a power drain to the chassis

    I removed all power and network adapter cables for five minutes, then swapped the CMC slots.

    After launching the VRTX the problem follows the CMC, currently I have the CMC 2 DOWN and always the concern of redundancy, see the captures.

    Best regards

    2022-07-16_18h46_32.png2022-07-16_18h58_24.png

  • DELL-Joey C

    Moderator

    •

    4170 Posts

    •

    20965 Points

    170

    0

    Posted July 18th, 2022 02:00

    Hi @Htoumi,

     

    It looks like the CMC card in the server is the root cause since you have swapped it. To confirm hardware faulty, you probably need to look into the CMC logs. Else, if you have a space CMC card, you can try replacing it to check on the issue. 

    DELL-Joey C

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’ if I did.