Dell FC640 Lifecycle Controller not working

Question

Trying to update the firmware on a number of FC640 servers and am really struggling.

The upgrades are going on very inconsistently. We're using OME.

I suspect the cause of my problems is Lifecycle Controller which doesn't seem to be working properly on some of the blades. On a large number of the blades, they cannot be booted into Lifecycle Controller - the server just hangs at the boot screen with the message 'Entering Lifecycle Controller...'. The same blades also hang for hours on end when trying to run CSIOR.

Some of the updates appear to be executing in extreme slow motion. I have one particular blade where I started the Intel NIC driver update yesterday morning and it's still running! I have a feeling this is LC related too.

iDRAC/LC is version 3.21.23.22 BIOS is 1.4.8 all other firmware is latest available

Has anyone else come across this issue? I'd be grateful for any suggestions on how best to proceed. I have quite a few blades left to do that I dare not touch yet

Daniel My · Answer

Hello

If the iDRAC/LCC is not functioning properly then that is where you should start troubleshooting. I suggest that you connect directly to the iDRAC embedded web server on the system. Verify that the iDRAC is functional. Check the logs for any issues. If you are not able to locate and fix the issue then I would re-flash the iDRAC/LCC firmware. You will not be able to push updates through the iDRAC/LCC until they are functional.

http://www.dell.com/idracmanuals/

http://www.dell.com/support/

Thanks

Shepguy · Answer

It's very odd. iDRACS seems to be working fine, but LC has a problem on the affected blades. It is working, but has gone into extreme slow motion, taking days to complete operations on the affected blades.

A possible clue in the LC logs seems to be numerous mentions of 'the previous operation has been repeated hundreds of (sometimes 1000) times' where the previous operation has been 'successfully logged in using username, from ip and wsman.' I did wonder if the OME appliance was causing a problem, but I've shut that down without any change.

Daniel My · Answer

Something may be overwhelming the iDRAC/LCC with requests, or there could be an issue with the iDRAC/LCC causing it to take a long time to respond to requests. I haven't seen anything to indicate whether the requests are the problem or a symptom of the problem. You may try taking the iDRAC/LCC off of the network to see if that has any affect.

Shepguy · Answer

Thank you for that. We have already tried removing all network connections on an affected blade. It had no effect on the LC issue, so I guess it must have been either coincidence or a consequence of the LC issues. We did discover that if we remove the BOSS card then it works, but swapping a BOSS card from a working blade didn't make the dodgy blade work, Bit stumped now - we're thinking the update has broken something, but it doesn't explain why not all the blades have this issue.

Daniel My · Answer

You should check all of the installed hardware. If you have any unsupported hardware(USB, PCIe, or any other devices) then I would remove it for testing. Unsupported or faulty hardware can cause odd intermittent issues. Something as simple as a USB pen drive could cause issues throughout the system.

Shepguy · Answer

Thanks, but the servers are all as they were delivered. Not even a monitor plugged into the chassis.

PowerEdge Hardware General

Dell FC640 Lifecycle Controller not working

Was this post helpful?