Invalid package signature after upgrade

Hi Chris,

Thanks for the post.

It would be helpful to get more information on the issue. Are you saying the same packages worked for you earlier or is this the first time you are trying to deploy these packages?

Is the error seen soon after the task is scheduled? OME starts downloading the package in the background and verifies the signature when the download completes.

Do you see the file getting downloaded to the following folder "C:\Program Files (x86)\Dell\SysMgt\Essentials\SystemUpdate"

You can also try deleting the package manually and try to schedule the task again.

Regards

Abhijit

We used the old version for several weeks. We applied a catalog update at the same time as the upgrade to 1.0.1 so not sure where the breakdown is happening. When watching the packages directory, packages are momentarily downloaded and then deleted. Not everything has the invalid sig, but more than one thing have it. It caused the entire update to pause at 66% without ever running anything on the target machine.

Also, I've noticed the target machine result output includes an inventory run that contains a lot of stuff that has already been updated. For example:

NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth0)

The version of this Update Package is newer than the currently installed version.

Software application name: NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth0)

Package version: 6.4.4

Installed version: 6.2.16

All the NIC firmware on this machine is already 6.4.5, so I have no idea why/where it is pulling out old inventory status messages for DRAC, PERC, NIC, etc. during a fresh run.

Seems pretty broken.

Thanks for the additional information.

The update application logic did not change between the last version and this one. OME does have the logic to delete the package if the calculated signature does not match the signature specified in the catalog. In the past we had some packages with bad signatures listed in the catalog. It would be helpful if you can specify the package names which are failing the signature check and we can verify those on our end.

Regarding the information you have posted about NetXtreme package, are you saying that the NIC firmware is updated to 6.4.5 on the target server but the logs specify that the installed version is 6.2.16 instead of 6.4.5?

OME gets the version information about the installed components from Inventory Collector which is installed as part of OpenManage Server Administrator. If the inventory information reports 6.2.16 as installed version, OME will send the latest applicable update to that server. Can you verify if you have latest version of OMSA installed on the target server?

If you already have latest OMSA and you are seeing old inventory versions being pulled, most likely the inventory collector is not pulling the right versions. Is it possible for you to restart the target server or OMSA services? That would force inventory collector to run and refresh the information.

Interestingly the firmwares that are failing have lower case filenames, whereas the successful ones are uppercase. Coincidence?

Incidentally, Inventory.xml.1 also shows the same current firmwares. Invcol is running normally each time OMSA services restart, unless the bad versions are being pulled from a different inventory or temp file somewhere. After the target result above is acheived OME's system update task hangs at 66% and never completes.

I think some of the RHEL package signatures may be borked following the catalog update, or 1.01 is not handling them correctly.

I did a completely clean install of 1.0.1 and the problem persists.

In decrypting the system output result:

Package uploaded

ftp.dell.com/.../LC_APP_LX_R300967.BIN

ftp.dell.com/.../RAID_FRMW_LX_R313336.BIN

Invalid package signature

ftp.dell.com/.../DRVPK_APP_LX_R316354.BIN

ftp.dell.com/.../FRMW_LX_R310809.BIN

ftp.dell.com/.../FRMW_LX_R310809.BIN

That's three on a single machine that OME deemed as have (actually one of them is 2x). These are downloaded to the OME machine, but OME immediately deletes them when the *.sign is "bad".

Lots of spam below, but this is all from one machine. You can see below the output result from the target system, and it's OMSA /opt/dell/srvadmin/var/log/openmanage/Inventory.xml.2 file at the time that the OME result message was generated.

OME Result Message

PackageStatus:   The Update Package was applied successfully.
PackageFileName:   /var/tmp/zipydnxYD/IDRAC6_FRMW_LX_R313606.BIN
PackageLog:   Collecting inventory...
Running validation...

iDRAC6

The version of this Update Package is newer than the currently installed version.
Software application name: iDRAC6
Package version: 1.80
Installed version: 1.70

Executing update...
WARNING: DO NOT STOP THIS PROCESS OR INSTALL OTHER DELL PRODUCTS WHILE UPDATE IS IN PROGRESS.
THESE ACTIONS MAY CAUSE YOUR SYSTEM TO BECOME UNSTABLE!
Update Successful.
PackageReleaseId:   Not Available
PackageState:   complete
PackageCode:   2
PackageFileName:   /var/tmp/zipydnxYD/NETW_FRMW_LX_R309327.BIN
PackageLog:   Collecting inventory...
Running validation...

NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth0)

The version of this Update Package is newer than the currently installed version.
Software application name: NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth0)
Package version: 6.4.4
Installed version: 6.2.16

NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth1)

The version of this Update Package is newer than the currently installed version.
Software application name: NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth1)
Package version: 6.4.4
Installed version: 6.2.16

NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth2)

The version of this Update Package is newer than the currently installed version.
Software application name: NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth2)
Package version: 6.4.4
Installed version: 6.2.16

NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth3)

The version of this Update Package is newer than the currently installed version.
Software application name: NetXtreme II BCM5709 Gigabit Ethernet  rev 20 (eth3)
Package version: 6.4.4
Installed version: 6.2.16

Executing update...
WARNING: DO NOT STOP THIS PROCESS OR INSTALL OTHER DELL PRODUCTS WHILE UPDATE IS IN PROGRESS.
THESE ACTIONS MAY CAUSE YOUR SYSTEM TO BECOME UNSTABLE!
The system should be restarted for the update to take effect.
PackageReleaseId:   Not Available
PackageState:   complete
PackageStatus:   3
PackageCode:   2
PackageFileName:   /var/tmp/zipydnxYD/PER710_BIOS_LX_6.0.7.BIN
PackageLog:   Collecting inventory...
Running validation...

BIOS

The version of this Update Package is newer than the currently installed version.
Software application name: BIOS
Package version: 6.0.7
Installed version: 3.0.0

Executing update...
WARNING: DO NOT STOP THIS PROCESS OR INSTALL OTHER DELL PRODUCTS WHILE UPDATE IS IN PROGRESS.
THESE ACTIONS MAY CAUSE YOUR SYSTEM TO BECOME UNSTABLE!
The system should be restarted for the update to take effect.
PackageReleaseId:   Not Available
PackageState:   complete
PackageStatus:   3
TotalPercentageComplete:   66
TotalStatus:   All update packages executed successfully.  System reboot required for updates to be applied.  Rebooting the system automatically.
TotalStatusMessage:   Software update in-progress.

OMSA Inventory.xml.2

       
       


         display="BIOS"/>
       


       


         History="0" SequenceNum="3">


       


       


       


       
       
       
        mponentType="FRMW" version="1.07" display="SAS/SATA Backplane 0:0 Backplane Firmware"/>
       
               
       
       
               
       
       
               
       
       
               
       
       

The other firmwares were not successful. BIOS and NIC were good long ago. The five that were to be applied were

LCC 1.50.672 (good)

PERC 6/i 6.3.1-0003 (good)

OS Drivers 6.5.3 (bad)

drive fw ST3300657SS ES65 (x2 bad)

The update task hung at 66% without updating anything, and spat out that false result saying that it updated a bunch of stuff that was already up-to-date.

Can you tell me where that bad update result is coming from if not the Inventory.xml files that invcol generates? The bad result showing old updates is appearing after many, many reboots and manual reinventories from OME.

By "good" I mean that the download was good, but the updates were not applied by OME.

Hi Chris,

Thanks for your patience.

From the description above, the problem may be related to inventory data not being refreshed in OME. As a result, OME is not aware of the updated versions and sends those updates again. Since those components are already up to date, you get the result that the components are updated.

The two bad components you listed above are the same ones which are reporting invalid signature errors. As a result, those components will not be sent to the target nodes. As I mentioned, we are working on verifying those components to check if there is a signature issue in catalog for those.

Meanwhile there are couple of things you can try. Check with troubleshooting tool if you are able to communicate the target server over SNMP successfully. You can also delete the server from the tree and try rediscovering. Once you get the inventory data, the compiance reports will automatically refresh and the updated components will not show up in the report.

Regards

Abhijit

Thanks Again.

As you informed we did find signature issues in the catalog and we have informed the catalog team to correct those and post updated catalog. I agree with you that manually updating mutliple servers is painful and that's where OME's update feature is supposed to help.

I believe once the package signatures are corrected, package application should be successful from OME. We'll definitely look into it if you get any issues again.

Signatures will have an impact on repository manager as well. There is a newer version of repository manager available here. It may fix some of the issues you are seeing.

ftp://ftp.dell.com/FOLDER00313115M/1/Dell_Repository_Manager_1.4.113.msi

Regards

Abhijit

Well, I'm done with that server now, since it was in downtime for maintenance and I needed an immediate solution.

I manually applied the drive firmware and the driver pack (bad sig packages), restarted srvadmin-services.sh (wait for invcol to stop running), and then reinventoried through OME. The machine came back as compliant, but I suspect that the problem with the bad report still exists and probably will exist until OME successfully applies firmware to the machine again.

I'm not going to mess with dropping and readding, since I have no way to test it right now anyhow, but next time it needs an update, I can try it.

How long does it normally take to get these package sigs fixed? We have about 100 machines that need the Seagate S65 firmware and the driver package update (in addition to others that do work through OME). Manually updating is no fun, now that we are spoiled.

Also, can you tell if this signature issue will also cause problems with Repo Manager?  I'm trying to create a new OME repo through Repo Manager. It says it needs to process 105 updates. It makes it through 13/105 and then errors "Getting Inventory", "Error retrieving inventory from Dell OpenManage Essentials". Are there some logs for Repo Manager somewhere that will tell me what is going on here, or is it a simple case of RM trying to pull these same bad sig updates?

Thanks for all your help. I've moved my RM issue to another thread discussing this:

en.community.dell.com/.../19429076.aspx

It might be good to get some RM and OME devs looking at some of this stuff and fast tracking to the catalog team, as these are now released, supported products, and sales is using these products in their materials as selling points for the 12G servers.