system update hangs

Hi Naseem,

Thanks for the post. You are right, OME copies files from dell website & iDRAC/Server downloads from OME.

Few questions to understand scenario better

1. Are you using in-band or Out-of-band (iDRAC) for system update?

2. If it is Out of Band, can you check for following possibilities ?

         - Whether iDRAC and OME in same network? or From iDRAC network are you able to reach OME system ?

         - Does OME machine has firewall ports to allow in-bound connections?

3. What are the generation of servers you are managing via OME ?

4. If all connectivity is fine, can you try system update after clearing iDRAC Job queue / iDRAC reset?

Thanks,

Arun

OOps, answer to first question is: out of band/idrac

Thanks!

Naseem

Hi Arun, thanks you so much for the reply. Let me try to answer the questions:

1. Are you using in-band or Out-of-band (iDRAC) for system update?

Yes.

2. If it is Out of Band, can you check for following possibilities ?

         - Whether iDRAC and OME in same network? or From iDRAC network are you able to reach OME system ?

         - Does OME machine has firewall ports to allow in-bound connections?

iDracs are not on the same network as OME. the various iDracs pertain to different networks in fact. What i do know is that from the OME network i can reach all the various iDrac networks. I am not sure if from iDRAC i can reach OME, is there a way to test that? in any case i think if something needs to be opened, if possible, it should beopened on OME side since there are so many different networks with idracs on them that i want to manage.

As for OME machine having fw ports to allow in bound connections, could you elaborate on that please so i can forward info to our ip team?

3. What are the generation of servers you are managing via OME ?

mostly 11th and 12th gen, recently we have been acquiring 12th gen as well.

4. If all connectivity is fine, can you try system update after clearing iDRAC Job queue / iDRAC reset?

Sure, while on this topic, would you be so kind as to list the simplest way to launch idrac reset (**not** server power reset) using OME on various machines? would it be a remote racadm task?

Thanks,

You can use "racreset" command to reset the iDRAC using racadm command line task in OME. But before that we would want to know if there are any pending jobs in Lifecycle which may be causing the tasks to fail. You can run the following the command to list the jobs in iDRAC:

winrm e cimv2/root/dcim/DCIM_LifecycleJob -u: -p: -r:https:// /wsman:443 -SkipCNCheck -SkipCACheck -SkipRevocationCheck -auth:basic -encoding:utf-8

If there are pending jobs, then we need clear the job queue before performing any system update through OME. This command will allow you to clear the job queue:

winrm invoke DeleteJobQueue cimv2/root/dcim/DCIM_JobService?CreationClassName=DCIM_JobService+Name=JobService+SystemName=Idrac+SystemCreationClassName=DCIM_ComputerSystem @{JobID="JID_CLEARALL"} -u: -p: -r:https:// /wsman:443 -SkipCNCheck -SkipCACheck  -auth:basic -encoding:utf-8

Try this on one system. If it works, then we have a catch.

Hi thanks for the reply Dell Pupul! i have 17 idracs where system update is hanging (in fact system update has never successfully worked yet) can i run the above command on many idracs at once via OME? if so what kind of remote task would it be (RACADM, General, etc.) and would it be written out differently?

Thanks,

Naseem

Hi pupul, update: I have used "jobqueue delete -i JID_CLEARALL_FORCE" on all nodes via OME, then i did racreset on all nodes as well. system update still hangs... :(

Hi Pupul, they are a mix of iDrac 6,7 and 8 so mostly 1.98,1.65.65 and i believe 2.10, respectively.

Small information needed: Are all these iDRACs where system update task is failing, are at the same firmware version? What is the iDRAC firmware version on these iDRACs?

Hi Pupul, after discussing with some co workers, I believe the issue is that the network on which OME is installed is not reachable by the idracs on various networks.

Thanks for your help!

Naseem