VNX / PowerPath issue

Question

Hey friends, I am seeing an issue that I am hoping to get some advice on. At my DR site, I have a 3 node MS SQL cluster and it has been fine for quite some time. For some reason I was on over the weekend to do some server maintenance and tried failing over my SQL instances and they would never move. I then noticed tons of degraded paths in PowerPath. I logged into the VNX and nothing appears out of the ordinary. I rebooted the servers, updated firmware and drivers and still i am seeing tons of errors.

We do have RecoverPoint in play and it is showing a ton of timeouts on the DR side writing from he RPAs to the VNX. The fiber switches appear to be OK as well. I am at a loss of what the issue could be. Just thought I would get some good ideas here. Thanks!

Madhug1 · Answer

Hi,

Let me know the version of PP and OS installed, As mentioned, there was no issues from switch and VNX end.

Are the firmware and drivers compatible as per EMC E-Lab.

this thread is for your reference.

https://community.emc.com/thread/112465?start=0&tstart=0

Thanks

Madhusudan

anilsedha · Answer

Received your tweet. I have seen a similar issue in the past - it was on Cisco UCS and had to do with the firmware. Since you are not using UCS and have upgraded firmware and drivers that should eliminate the driver/firmware problem.

Another issue I observed recently had to do with PowerPath. The problem was that PowerPath showed error messages like -

PowerPath: EmcpEsxLogEvent:1271:Error:Mpx:SCSI-2 release failed with SCSI status

Are you receiving any "flogi" errors as well ? It would be helpful if you can post some error messages from your logs.

Note that this was a virtual env running Microsoft clusters - file & SQL. We had to uninstall Powerpath and attempt a reinstall. It did resolve the problem.

There is also a Primus article about the other errors we were seeing and we found out that the problem was known issue. PowerPath problem was resolved in v5.2sp1 and we were at v5.3

We were thinking that maybe the problem was still lingering and not fully resolved. I'll try to find the primus for you.

HyperIsa · Answer

Hi anilsedha, we have a very similar problem, and we use Cisco UCS. Could you explain better what you've got to experience in the past? You say that in that case it was firmware, but what exactly? and what were the symptoms? In the last 2 weeks some of our systems (mainly SQL cluster but also Oracle) experience a strong latency in r/w on disks during peak of I/O up to crash. We use UCS B-Series Blade connect to MDS C9513 and EMC Symmetrix Storage, can you help please? thanks Isabella

anilsedha · Answer

Hi Isabella,

There was a known issue with the UCS firmware that we were running on - it had to do with flogi errors and path error messages. If you actually search the UCS knowledge base with the error message you get you would see a KB. Sorry, I will have to go way back and look at my emails but I'll try to find them for you.

Your issue as you are relating is about latency r/w on disks during peak I/O. I am not sure if that is related to path errors unless you clearly see that. Since this seems like an unrelated issue can you start a new post and I can try to help you there. It is good for the community as well to look at threads related to specific issues. Give me the new link via email and I will try my best to help (anil.sedha@gmail.com).

Do list some details - ESXi version, Oracle version, Unix/windows platform, LUN/Pool configuration and so on that is necessary to understand your environment.

Thanks

VNX

VNX / PowerPath issue

Was this post helpful?