Policy manager hung in stopping state

We are running the ESRS gateway and policy manager on the same system (version 2.0.2.1-145 (2011-07-14_10-52-24)).  The system is a Windows 2008 R2 with SP1 system.  Approximately every two weeks the policy manager will hang in a stopping state while trying to run the nightly ESRS backup.  The only way to fix this is to kill the tomcat5.exe process.  Does anyone have a way to prevent this from happening?  Anyone else having the same problem?

Tags (2)
0 Kudos
12 Replies
BChused
2 Iron

Re: Policy manager hung in stopping state

Apologies for the delay in response. When you say you are "running the ESRS Gateway and Policy Manager on the same system," do you mean that you are utilizing a single server to host both the ESRS application and the Policy Manager application?

0 Kudos

Re: Policy manager hung in stopping state

You are correct, we are using a single server to host both the ESRS application and the policy manager.  I know this is not recommended for production applications but since we only have one VMAX system behind the gateway we couldn't justify two VMs.  This system is dedicated to ESRS use; no other applications are installed on this system.

0 Kudos
BChused
2 Iron

Re: Policy manager hung in stopping state

Yes, the ESRS Gateway server should be dedicated and not used for any additional applications (including the Policy Manager), so that could be your issue. Another thought -- if you only have a single VMAX array, there is an embedded ESRS application within the service processer that can be configured by EMC or authorized partners, and it does not require a separate server. If you utilize this ESRS configuration ("ESRS Device Client for Symmetrix") then perhaps you could dedicate the VM to the Policy Manager?

0 Kudos

Re: Policy manager hung in stopping state

I was not aware of the embedded ESRS application on the VMAX array.  That would be an option for us today but we may have other products serviced by this ESRS gateway in the future so I don't want to get locked-in just yet.

We did review our decision to use single system for both the ESRS gateway and policy manager with our EMC technical architecture and they agreed with our reasoning.  This is a supported configuration.  We were told it is not recommended because it doesn't scale well not because it doesn't work well.  Since we no plans to scale beyond 5-10 devices the scale issue is not a concern for us.

Please explain how running the gateway and policy manager on the same host would cause the the script that runs as part of the policy manager backup to fail once every two or three weeks.  To me, it is unacceptable to need to run these on separate systems.  We are already disappointed with the administrative overhead needed to manage one storage array (solutions enabler host, SMI-S host, Unisphere host, ESRS host), please don't increase it.

There is a known issue with around the policy manager backup; I started this discussion hoping to find someone that has experienced this and found a work around.

0 Kudos
Doxx2
2 Bronze

Re: Policy manager hung in stopping state

This has been an on going problem w/ eth PM and W2kR2 - we are aware of teh issue and it definitely willo be fixed in the next release of PM code. Tihis issue is cuased by teh 32 bit java randomly generating a split thread which cause the PM not to stop cleanly when called to do so bu the scheduled job ( which is triggered day @ 0300 local time ) and aas the PM did not stop cleanly the backup also does not occur and the PM becomes non responsive we are working with teh vendor to address trhis issue in this version of code and  trying to develop a work around for the problem

We have a Primus solution which has several possible work arounds that may address you issue, it is available n support.emc.com .

It is emc272481: Policy Manager service hangs in "stopping" state. If non of these methods are sucessful please open a Service Request and will will investigate furter

Re: Policy manager hung in stopping state

Unfortunately I do not have access to that Primus article.  We have however already opened a service request.  Based on the SR we have reinstalled the policy manager and deleted and recreated the scheduled task for the backup.  Neither helped.  Any ETA on the next release of the PM code?

0 Kudos
BChused
2 Iron

Re: Policy manager hung in stopping state

FYI - new Policy Manager code was released last week (v6.6). See the "announcements" section at the top of the ESRS Support Forum for additional details.

0 Kudos
dynamox
6 Thallium

Re: Policy manager hung in stopping state

Ben,

i have briefly looked at 6.6 operation guide, could not find a section on "What's New". What is really new in 6.6  (i am currently on 2.0.2.1). I also saw that this is a migration, not really an upgrade in place ?

0 Kudos
Highlighted
Doxx2
2 Bronze

Re: Policy manager hung in stopping state

No it is NOT an in place upgrade where you install the new code over the old code. The operations Guide dfor Policy Manager 6.6 provides extensive instruction to perform this action.

0 Kudos