Solved!
Go to SolutionPowerPath Connection to CX4
Hello All,
Mods, please direct this Discussion to the correct venue if I have placed it in the wrong location.
I have a host that has started behaving very oddly. Here is my setup:
- Windows Server 2008 R2 Host
- PowerPath 5.5 SP1 (Unlicensed)
- Server acts as a Media agent for CommVault
- Server has approximately 15 LUN's mapped to it
- Two LUNS are NTFS
- 13 LUNS are VMFS
- Using CommVault VSA to backup the VMWare VM's via SAN Mode
- QLogic Two port FC
- HBA, each port is individually zoned to each SP
- The server also has 6 iSCSI disks that are mapped to NON-EMC storage
The host has been exhibiting the following behavior:
- When my CommVault Backups kick off, I have been getting a tremendously high number of EMC Alerts indicating that "The storage system received too many events in this poll cycle to display here. This does not necessarily indicate a problem."
- When I restart the host, if both SPA and SPB are connected to the host, the server goes into a perpetual reboot cycle, and never will boot into windows. If I unplug one of the FC cables, the server boots successfully into the OS.
- In PowerPath, if I expand Path Management > Storage Arrays > Storage ID; SP A1 has all of the Disks (NTFS and VMFS) - Mode: Active, however SP B1 ONLY has the two NTFS Volumes - Mode: Unlicensed.
My questions are:
- Other than Windows updates, nothing has changed on this host from a configuration perspective . I cannot figure out why it all of a sudden started this behavior.
- Why would SP B1 only two disks whereas SP A1 shows all disks? Is this part of the problem?
- I understand that PowerPath is unlicensed - does this mean that the server would not be able to have disks that are on both SPA and SPB on the CX4? Since PowerPath is unlicensed, must all disks reside on a single SP for the host to be able to properly access the data on those disks?
- When viewing the list of disks in PowerPath, it says that the current owner of ALL disks is SPA. However, when in Navisphere, it indicates that the majority of the VMFS LUN's are on SPB whereas only two NTFS and two VMFS LUN's are on SPA, can this be a contributing factor?
- Has anyone experienced the perpetual reboot scenario when you have both paths connected to a Windows Host?
Thanks in advance for your advice - I will be glad to amend/append as needed.
Message was edited by: seanka - Formatting issues
RobertDudley
2 Intern
•
448 Posts
977
0
Posted October 15th, 2012 09:00
Unlicensed powerpath only provides failover at the array level not the host hba level. All I/O will use one path to the storage system. If luns are owned by both SP's when this host tried to access them for backup it may be causing the lun's to trespass to the other SP.
As dynamox suggested license powerpath or remove it and use the built-in DMP. I would suggest getting a PP license.
here is an example for you (this is Linux but same concept)
Pseudo name=emcpowera
CLARiiON ID=APM00000000000 [server07]
Logical device ID=600601609E402D00008BDBBA4612E211 [LUN 366]
state=alive; policy=BasicFailover; priority=0; queued-IOs=0;
Owner: default=SP A, current=SP A Array failover mode: 4
==============================================================================
--------------- Host --------------- - Stor - -- I/O Path -- -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
1 fnic sdb SP B3 active alive 0 0
1 fnic sdc SP A0 active alive 0 0
2 fnic sdd SP A3 unlic alive 0 0
2 fnic sde SP B0 unlic alive 0 0
PowerPath is not licensed but each HBA is zoned to SPA and SPB so you can access LUN on either SP using HBA 1 but from failover and load-balancing perspective the second (unlic) HBA is not in use. So if you were to experience port failure on the switch, HBA 1 failure, cable failure ..this host would lose total storage access.
Can you post output from "powermt display dev=all" ? (scramble your serial numbers/host names)
Thanks Guys,
So, to clarify. In theory I need to have a licensed copy of PowerPath on this host since I need to access the LUNS on both SP's. Since this single host is accessing LUNS that are shared amongst other hosts, it will need access to both SP's concurrently.
If I have PowerPath licensed, and the host tries to access a disk on SPB OR SPA at any given time, it will be able to communicate with both of them?
HI Dynamox,
That is correct. The VMFS luns are presented to the windows box. Then, the backup software (CommVault) takes a snapshot of the virtual machine and backs up the VM. The backups are done via FC directly connected to the VMFS Datastore.
The fact that the VMFS LUNS are not presented to the second HBA is what I am wondering is causing the issues. Plus, according to Navisphere there are about 10 luns that are sitting on SPB. So, is my guess correct that when this server tries to backup the data on those luns located on SPB, it is tresspassing the luns to SPA so that it can complete?
Is there a reason that you can think of that PowerPath would see the the VMFS luns on the second HBA?
interesting, VMFS LUNs are not presented to the second HBA (even not licensed). Is this VCB type of backup, where you present VMFS LUNs to a windows box and then spin vmdks to tape.
You have a couple of options:
1) Best case - get PowerPath license, get better load-balancing, throughput, failover. Zone each HBA to SPA and SPB.
2) If not licensed - might as well disconnect the second HBA, it's not going to buy you anything. Zone first HBA to SPA and SPB and see if you problem re-appears. Right now , based on the output your provided, if LUN trespasses to SPB, you are hosed because HBA 2 (unlicesed) is the only path to that SP.
The VMFS Luns show up in Disk Management on the Windows Server as "disks" but don't show up My Computer as expected.
The Tug of war that you are describing sounds exactly like what is happening. I think I am going to get a trial license of PowerPath and test it. If it resolves the problem, I think I'll have a good justification for the License.
Dynamox - many thanks for your advice on this. You provided insite and an explanation that unfortunately EMC Support couldn't.
Sean
dynamox
11 Legend
•
20419 Posts
•
87439 Points
977
0
Posted October 15th, 2012 10:00
if you are presenting VMFS LUNs to the Windows server storage group, i would expect to see them listed just like your two NTFS LUNs (unless CommVault is doing some kind of magic and hides one path).
if this in fact is happening ( it could), you can get into trouble (trespass storm), where your backup server tresspasses the LUN to SPA and then VMware trespasses it back to SPB ..so you get into this tug of war. I would make sure each HBA is zoned to SPA and SPB.