unfortunately i can't provide a solution but to share your frustration. We always have these issue, even on the latest 5.x RHEL boxes. The other day we had to upgrade PowerPath from 5.1 to 5.5 and PowerPath kept complaining that devices were in use, how can they be in use if all file system were un-mounted and volume groups were exported. Stopping PowerPath service would not help either. I literally had to disable ports on the switch, bounce the box and only then we were able to upgrade PowerPath. Linux after so many years is still so weak when it comes to LVM and disk subsystem. You very rarely see these issues on AIX/HPUX/Solaris ..real *nix boxes

Hello.  Have you contacted anyone from EMC's Customer Support about this issue?

Thanks, Brion

Please keep me informed on your progress. I’d like to see this issue researched.

Thank you for the responses.  I have not opened a case with EMC, mostly because I suspect, based on some past experience, the answer will be reboot.

Today I made the same change on three more servers (another RAC cluster). Befor unpresenting the LUNs from the Clariion I issued the 'powermt remove' command, but the answer was -

#powermt remove force dev=emcpoweri
Cannot remove device that is in use: emcpoweri

Even though there was nothing mounted on the device.

Interestingly, the issue on these three nodes is consistent with what happened on the first three. One path remains on the EMC pseudo device in a display -

#  powermt display dev=emcpoweri
Pseudo name=emcpoweri
CLARiiON ID=APM00083101547 [bkvdwracp RAC cluster]
Logical device ID=6006016072C020003E7F9D2EC585DE11 [bkvdwracp - /exports on bkvdw03p - LUN407]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   0 lpfc                      sdh       SP A4     active  dead       0      1

I cannot reboot at this time, but it does not appear to be causing any issues in this state.

I have two more clusters to do this to. They are each only two nodes. Maybe I will find out more before then!

Please review the steps in the "PowerPath for Linux Installation and Administration Guide" available from PowerLink in the following breadcrumb trail:

Home > Support > Technical Documentation and Advisories > Software ~ P-R ~ Documentation > PowerPath Family > PowerPath > Installation/Configuration

Specifically skip to the section: "Dynamically removing a LUN".  There are a few steps listed that you didn't mention whether or not you had performed and you may or may not be performing them in the recommended order.  From the few times I've needed to do this, I've followed this specific process and don't recall seeing the behavior you are observing, but I also would have to review my projects to see what version of PowerPath (and version of Red HatI was working on when I last performed it.  Based on the process, as we all agree a reboot will most likely clean it up as Linux rescans/rebuilds its pseudo devices and PowerPath rebuilds its configuration.

Just a thought.

Thank you for all the helpful posts. i have performed the same task on eight servers in three RAC clusters, all with the same result. I have tried unpresenting the LUN's before cleaning up PowerPath and tried the 'powermt remove' before unpresenting..with no difference. PowerPath thinks one of the paths is still busy even though the filesystem is unmounted. It will removed all the dead paths for the device except for one in every case -

Pseudo name=emcpowerd
CLARiiON ID=APM00083101547 [bkvoltpda RAC cluster]
Logical device ID=6006016072C020007EBABDD310E0DE11 [bkvoltpda - /exports on bkvoltp01da - LUN504]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 lpfc                      sdaf      SP B5     active  dead       0      1

Atleast it is consistent! I expect these will finally clean up on a reboot. I have one more cluster (two servers) to do.

Thanks,

Michele

On the final time doing this the result was consistent and one dead path remains.

I am planning to remove LUNs from a standalone server next week and am curious to see if the result is still the same.

I will update this forum with my results.

Thanks.

since you mentioned RAC cluster, are those LUNs part of ASM disk group? if yes, the devices cant be released till you bunce the ASM on the node.

Thanks for your response. No these are not ASM disks, strickly LVM.

Although I am curious to see if perhaps the clusterware may be the culprit. Tomorrow I will be removing a LUN from a standalone server...although same OS and version of PowerPath.

I will update with result.

when you said you tried to remove when the device is mounted, but was the device part of the VG that time? were u able to do a vgexport ?