PowerPath cannot run powermt command in AIX host

configuration change (HBA / disk / etc..)

Environment:

OS: AIX
DELL EMC SW: PowerPath for AIX 6.0
DELL EMC SW: PowerPath for AIX 6.0 SP1
DELL EMC Product: Symmetrix / VMAX
Customer replaced an HBA and rebooted the host. After reboot of the AIX host, the "powermt display" command showed no devices, and reported below error:

ERROR  File or directory not found.

"powermt" commands are failing with "ERROR: File or directory not found." is also observed after a configuration change.

as another symptom for the issue, we can observe that the /etc/emc/mpaa.lams file is empty. 

PowerPath bug. 

PowerPath for AIX 6.0/ PowerPath for AIX 6.0 SP1 are missing a check for VCM DB device type which causes it to incorrectly handle the Symmetrix ACLX device.

This is documented in PowerPath 6.1 Release notes as follows : 

 

Problem number	Problem summary	Found in version	Fixed in version
PPAI-186	The powermt display command displays Error:File or directory not found in PowerPath 6.0 SP1	6.0 SP1	6.1

"found in Version" should read PowerPath 6.0

Fix:
long term : Upgrade to PowerPath for AIX 6.1 or above
short term : run a "powermt save" to populate /etc/emc/mpaa.lams

The most common cause for the reported error message is the missing or corrupted mpaa.lams file. Running the command  *powermt save* will generate the file. Note that in such a case, the file is also regenerated when an emcgrab is run; the "*powermt check_registration" is doing the trick ! In reality, any powermt outside of the "display" flavours will regenerate the mpaa.lams file in this PowerPath release.

Workaround (which will prevent the problem from re-occurring in PowerPath 6.0 / PowerPath 6.0 SP1):

1. unmap the ACLX device from affected host;
2. rescan devices (if that does not work, reboot host)

Only the combination of Symmetrix and PowerPath for AIX 6.0/PowerPath for AIX 6.0 SP1 is impacted by this issue.

Boot from SAN environments.
Customers impacted by this bug with BFS environments can run into another problem.  The "pprootdev" shell script has embedded powermt commands which will cause pprootdev to behave incorrectly. 

The following workaround can be used to get PowerPath commands work on this customer's host and upgrade to PowerPath for AIX 6.1:
1. Rename "/etc/powermt_custom.xml" to "/etc/powermt_custom.xml.bkup"
# mv /etc/powermt_custom.xml /etc/powermt_custom.xml.bkup
2. Reboot
3. Upgrade to PowerPath for AIX 6.1