Start a Conversation

Unsolved

This post is more than 5 years old

T

45748

February 1st, 2010 15:00

Inventory Collector Agent and IPMI not accessable from ITA

I manage about 10 servers from ITA and last week one of my new servers (R710/ Windows 2008 R2 X64) stopped updating its inventory. I noticed this behaviour after updating all firmware/drivers on this machine. in software updates ITA kept showing all the old versions, while OMSA on the server confirmed new versions where installed.

First i tried to remove/re-discover the server in ITA, but stil the old version numbers where shown.  in C:\Program Files (x86)\Dell\SysMgt\oma\log i dicovered a file called Inventory.xml.1/2 stil containing the old inventory data, so i decided to remove OMSA (6.1) and installe the new version (6.2).

After this reinstall an re-discovering the device in ITA i do not see any inventory data anymore, also no new inventory files are apearing in C:\Program Files (x86)\Dell\SysMgt\oma\log.

on the 'server information' screen (device details) i only see

Agent Global Status Agent Name Agent Description Agent Manufacturer Agent Version
Performance and Power Monitoring Agent NA Dell Inc. NA
Server Administrator (Storage Management) Configuration and monitoring of disk storage devices. Dell Inc. 3.2.0
Server Administrator Management software for Dell systems. Dell Inc. 6.2.0

on my other server i see

Agent Global Status Agent Name Agent Description Agent Manufacturer Agent Version
Performance and Power Monitoring Agent NA Dell Inc. NA
Server Administrator (Storage Management) Configuration and monitoring of disk storage devices. Dell Inc. 3.2.0
IPMI IPMI Management Software NA 2.0
Server Administrator Management software for Dell systems. Dell Inc. 6.2.0
Inventory Collector Agent Provides information about devices running on the local system Dell Inc. 6.2.0

 

so it looks like the inventory Collector agent is not working on this server.

Anyone has any clue where to look for this problem ?

 

Kind Regards,

Tom

347 Posts

February 2nd, 2010 07:00

the first thing to note is that if OMSA is showing the right versions, you're on the right path. Second, ITA gets its inventory from OMSA's Inventory Collector based on the pollimg intervals specified in ITA. There can be a delay of minutes, hours or even a whole day depending on the settings in ITA for polling interval.

Try this: go to your discovery range for that IP, right click the range and choose perform Discovery and Inventory now, and it should update.

if that still doesnt work, it would be a good idea to use the Trouble shooting tool in ITA to verify ping, and SNMP connectivity. You can get to the tool in ITA by going to Tools>Trouble shooting tool> then enter the IP address of the R710 and click Get Test Configuration. On the list of things to test, first try ping, if that fails, fix that first. then try port connectivity on port 1311 (assuming you havnt changed the omsa standard port). Then, try the get community name (set name will proably fail).

2 Intern

 • 

793 Posts

February 2nd, 2010 15:00

You should also make sure ITA is up to date as well.

16 Posts

February 3rd, 2010 07:00

I have already tried all the steps you mentioned (i'm pretty familiar with ITA)

I have rediscovered an reinventoried the  machine and the range it is in. I even deleted te server from ITA and rediscovered it. all without any progress to the problem.

I also have tried this from 3 different ITA installations (ITA version 8.5.0), and all give the same result.

I also have tried the troubleshouting tool and all test are succesfull (Ping Test, SNMP test, OMSA port test, CIM test)

I'p pretty sure the problem is OMSA related, as ITA works fine with all other servers.

Anyone else has another suggestion ?

347 Posts

February 5th, 2010 15:00

ok, try this. First, lets be sure inventory collector runs on the local system.
go to the R710, c:\program files (x86)\dell\sysmgt\cm\invcol folder
then run invcol.exe   (you can run invcol.exe -outc=results.txt if you want to save the output in a txt file, open it with wordpad, not notepad)
you should see a big dump of information. If invcol.exe fails to run, thats the first problem.
assuming you get a nice dump with all the system infomation, invcol is working.

If that works, make sure that these services are running in Services.msc
DSM SA Connection Service
DSM SA Data Manager
DSM SA Event Manager
DSM SA Shared Services
if all these are running properly, lets go to the next step

locate the DSM SA Shared service, in Services.msc
Open up WIndows task manager and click the Processes tab.
back in Services, restart the DSM SA Shared service, then flip back to the Processes tab in Task Manager
what you are looking for is invcol.exe *32 to appear (2 of them), and it should stay visible for about 15-30 seconds and then go away. 
This is normal behavior. If any of the above services are not running or invcol doesnt run, thats the first issue to tackle and we can go from there.

Let us know how these steps go.

 

 

 

 

 

 

 

 

16 Posts

February 5th, 2010 16:00

reybeast,

thanks for your reply,

Things already go wrong at step 1. When i run invcol.exe (even in an commandprompt with admin rights) is stops right away without any output. Also i can't find any errormessage anywhere (eventlog, logfiles).

Kind regards,

Tom

347 Posts

February 8th, 2010 07:00

does it eventually drop you to a command prompt again?
here is how to see help on this: incvol.exe /?
Thats the first test, do you get the help feedback, if not, I would say the invcol is corrupted. or you may have a restriction on running 32bit apps.

Try this invcol.exe -logc=error.xml (post the contents of the resulting file here.)
Little more info here. invcol.exe is a self-extracting zip file. What normally happens is that the contents of the exe are extracted to a temporary folder, then the individual modules are run and the info is compiled and then either posted to the screen or written to an .xml file.
Also, the files are extracted wherever your user's Temp folder is set to. Type "Set" at the command prompt to see where that is.
When invcol.exe is run, in the Temp location, a folder called invXXXX_tmp is created and there will be lots of subfolders in there. If this temp folder cant be created, invcol will not run properly. There can be a policy setting that restricts the creation of a tmp folder.  
If you look at the processes tab in Task Manager, does the invcol.exe run forever?

16 Posts

February 8th, 2010 08:00

1: yes i'm using terminal server, but on other servers i works normaly with the same kind of connection

2 : files are writen to C:\Users\[my user name]\AppData\Local\Temp\inv989C_tmp\

3: yes all 4 omsa services are running normaly.

i was hoping thats by replacing de incol.exe package with the unpacked invcol.exe (and all other files) this version would be called bij the service

if you want i can e-mail you the output from procmon

Kind regards, Tom

347 Posts

February 8th, 2010 08:00

well, you did all the right things, you know your invcol is good since you copied it from another system. So what this sounds like is that someone has enabled File Redirection and/or User Access Control is giving you problems.  
Questions:
1. Are you using Terminal Server?
2. where are the files written to according to Procmon? (You might need to enable file logging.)
3. were the services listed above all running properly?

Its important to note that even though you can run invcol manually after the files are extracted, ITA wont get updated because it didnt call the above services to "deliver" the inventory when it asked.

 

16 Posts

February 8th, 2010 08:00

Reybeast,

You have a nice timing, i just tried exactly the things jou suggest and i was just wanted to update the thread with bij conclusions

- incvol.exe indded just returns me to the command prompt, without any message, after a few miliseconds.

- when i user invcol -? for help is has the same behavior, no helpteksts is displayed and i'm back at the coomand prompt

- i tried to copy incol.exe from a server with a working inventory, but the results stay the same

- i tried invcol -logc=error.xml, but no errorlog file is created

- Then i run invcol i see a directory appearin in my temp folder, but it desapears so fast i cannot even read the name. but with procmon (sysinternals) i can see a lot of files are created succesful in this directory.

- When i extract al files form invcol.exe to a subdirectory in mu temp folder, i can run invcol extracted from this directory and it seems to work !, i get normal XML output !

- i even tried to replace the invcol.exe archine with al of its contents, when i run invcol manualy from there it works, but i stil get no inventory in ITA

Kind regards, Tom

 

 

347 Posts

February 8th, 2010 12:00

sounds good, so to clarify, you see an invxxxx_tmp folder created, and then it gets deleted after about 30-40 seconds?
and just to be sure, you should not be seeing alot of these invxxxx_tmp folders, one should appear for less than a minute, then it should remove itself, is that what you are seeing?
I forgot to ask an obvious question, you have full admin rights?
Does the invcol show up in the Processes tab as i described above?

 

 

16 Posts

February 10th, 2010 05:00

almost true, the invxxx_tmp directory and also the invcol process disappear in less then a second (so not 30-40 seconds) about the same time it costs to be returned to the command prompt. is looks like something is going wrong in the process, and it cleans-up the directory and stops the question now is what is going wrong ? and yes, i have full admin richts and i'm running from an elevated cmd prompt

347 Posts

February 15th, 2010 09:00

I'm not sure where else to go on this one. If invcol.exe can't run properly, your inventory isnt ever going to show up. The issue you seem to be having is the execution of the sub-invcol.exe. Something is restricting the calling/running of the sub invcol.exe that happens after the files are extracted to the temp folder. if you have an antivirus on the system, try disabling that temporarily. Trend Micro officescan can cause problems for updates. TCP port 135 must be opened in the firewall and the Temp  folder for the user applying the updates must be excluded from the real time scans.

16 Posts

February 16th, 2010 04:00

Last night i made another try to solve this problem brute force

Ideinstalled OMSA and manula deletede al directoryies left.

After installing OMSA again the inventory worked !

The bad news is i did not figure out what the problem was, but i'm glad it is wordking again

Thanks for your help, Tom

No Events found!

Top