Dell iSM on ESXi 6.5 Ramdisk Issues

Question

VMware ESXi 6.5 build 13004031Dell PowerEdge R640Dell iSM VIB: 3.4.0 Hi All, Has anyone seen issues with the Dell iDRAC Service Module (iSM) 3.4.0 and ESXi 6.5 where iSM writes lots of zero length files to /etc/cim/dell/srvadmin/iSM/ini and exhausts the available inodes on the /etc ramdisk? We've had a number of ESXi hosts begin logging errors writing to /etc and in the worst cases the hosts have become disconnected from vCenter due to services like hostd crashing. We've removed iSM 3.4 from all our ESXi 6.5 hosts as a precaution until we fully understand the issue, but not all hosts showed the problem.

DavidPWhite · Accepted Answer

Hi I've seen it in one of my customers with the 3.4.0 iSM release.

There is an emergency patch for that issue, released as 3.4.0.1 which seems to have fixed the runaway iNodes.

BTW, is there a way to remove the iSM without having to put a node in Maintenance Mode?

TIA

#iwork4dell

Dell-DylanJ · Answer

Hello, Your post is the first I've heard of the issue, but I'll try to replicate it on my end and see if I get the same behavior. If I do, I'll get the information passed along to get the iSM fixed.

rjs_uk · Answer

Thanks Dylan.

We had earlier versions of iSM run without problems, and 3.4 seemed to run ok for some time but this issue cropped up out of the blue and on all hosts in a single cluster.

It is amongst our busier clusters, but also our only cluster with PowerEdge R640s.

Running the command, esxcli system visorfs ramdisk list, showed that all inodes on the /etc ramdisk were allocated and used (4096 from memory). I un-installed the iSM service in all cases, in some cases I had to kill the process and manually remove some of the files in the /etc/cim/dell/srvadmin/iSM/ini directory before it would un-install.

Dell-DylanJ · Answer

I have the test system set up. What BIOS and iDRAC revisions are on your server? I want to try to replicate your system in as many areas that may be a factor as possible. Since the iSM interacts with the iDRAC, and the iDRAc with the BIOS, I figure it would be a good idea to be thorough.

rjs_uk · Answer

BIOS: 1.6.13. iDRAC: 3.30.30.30. All hosts are identical hardware/firmware.

etienne falcon · Answer

We're seeing the same issue.iSM 3.4 ESXi host 6.7u2 build: 13644319  I suspect the issue is the dell iSM only appears to be designed for 6.7u1 not update2.I'll uninstall iSM for now, and try again when update2 is officially supported.   -Etienne

Jason_N · Answer

Same issue here on 6.7.0 (13006603) with iSM 3.4. We are seeing the below error in the logs.

esx.problem.visorfs.ramdisk.inodetable.full

Are we still waiting on a version of iSM with support for 6.7u2?

Jason_N · Answer

Just to add we have a vSAN cluster of 5 x R7415 with iDRAC 3.34.34.34 and BIOS 1.8.7. I've just removed iSM entirely for now until Dell confirm a version that has 6.7u2 support.

Jason_N · Answer

Thanks David. 3.4.0.1 doesn't appear in the VUM repository yet but I've just found it on the Dell download site. I'll give it a go on a new cluster as it's still in pre-production.

Re. maintenance mode I just removed the VIB in the usual way (esxicli software vib remove..). I don't recall if it requests maintenance mode. If it does I've never seen away to avoid this. That being said with sufficient resources and DRS enabled never had an issue with entering maintenance mode during production, usually during an off peak period.

rjs_uk · Answer

Good news. Thanks for the info on the new iSM version, DavidPWhite. I'll go ahead and mark your post as the accepted solution.

Jason_N · Answer

I've just tried to install 3.4.0.1, it failed via update manager due to an expired cert, see error below. I was able to install via cli using the --force switch. [Installati Dell_bootbank_dcism_3.4.0.1.ESXi6-1000 Please refer to the log file for more details.

Jason_N · Answer

Following the install I still have issues on the first host I have tried. The iSM does not communitcate, in iDRAC I have the error 'Not running (TLS error)'. I see this is a known issue which advises to renstall the iSM but even following this and host reboot I still get the TLS error.

Jason_N · Answer

We resolved this one in the end by removing iSM then a hard/factory reset of the iDRAC followed by a reinstall of iSM.

DellForumUser · Answer

I have experienced this issue on one of our PowerEdge 6415 hosts. installed the vib through the host gui and going to keep an eye on it.

tamere22 · Answer

Hello, Where did you get the download URL for this package ?  ok found it : https://downloads.dell.com/FOLDER05748534M/1/ISM-Dell-Web-3.4.0.1-1000.VIB-ESX6i-Live_A00.zip I also have the cert error (srsly, DELL ?) via VUM I wish I could avoid to hard reset all our idracs ...   Also did the update resolve the communication problem between idrac and ism ? Since latest idrac firmwares, ism is not communicating anymore with idrac. Yes it does ! Now ism show as 'running' on idrac connectivity overview status page ! yay !   Also support assist can't talk with idracs too, due to expired certs. I did recreate the certs on idrac but not working.     Anyone looking for complete guide : Remove old ism :  esxcli software vib remove -n dcism Download new ism (https://downloads.dell.com/FOLDER05748534M/1/ISM-Dell-Web-3.4.0.1-1000.VIB-ESX6i-Live_A00.zip) and upload via scp to /tmp Install new ism : esxcli software vib install -d '/tmp/ISM-Dell-Web-3.4.0.1-1000.VIB-ESX6i-Live_A00.zip' --force   Thanks to contributors of this thread !

Systems Management General

Dell iSM on ESXi 6.5 Ramdisk Issues

Was this post helpful?