Zoning for this server was not done by me, But there were some changes made to the zoning because we were in the process on migrating an MDS switch to a Brocade one. I will check the zoning and get back to you.
by the way, What should the best practise for zoning, Should I zone all 4 HBA's to all 4 ports to get 16 paths?
this is the same HBA that is showing as dead and alive at the same time. Draw a little diagram that explains how you zoned this server because if i assume this then its' only 8 paths:
I'm assuming your topology includes two fabrics, two FA ports in each fabric. The AIX host has 4 HBAs; 2 HBAs per fabric. The only time you would zone all 4 HBAs is if you are worried about overloading an HBA on the host. Keep in mind that HBA1 and HBA2 would be zones to the same FA ports (since they are connected to the same fabric) so you are not gaining anything from the VMAX perspective. The only thing you are gaining is load-balance across all 4 HBAs. Now is that really necessary? I don't know your environment but most likely it is overkill.
This AIX server is the most critical server in our environment as the most critical banking applications are running on this server. We need top notch performance on this server because it keeps the bank in business.
@Ernes you are correct about the topology, I believe the zoning is screwed up somewhere, I am checking it right now.
I just wanted to drop in a quick reply to what Ernes has posted as this fixed an issue we were having after migrating from a DMX-4 to a new VNX-5600 with a number of AIX servers. I initially was not going to scroll down and read the post as the original poster indicated that they were missing paths. Our issue was that after migrating we still were seeing the DMX paths in PowerPath - even though the zoning on our MDSes had clearly removed all visibility to the DMX. Here is what we were seeing and this was AFTER removing all visibility to the DMX:
# powermt display dev=all
Pseudo name=hdiskpower0
Symmetrix ID=000190103951
Logical device ID=0259
Device WWN=60060480000190103951533030
323539 state=dead; policy=SymmOpt; queued-IOs=0 ============================================================================== --------------- Host --------------- - Stor - -- I/O Path -- -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 3 UNKNOWN unknown FA 8c:01 active dead 0 0 3 UNKNOWN unknown FA 9c:01 active dead 0 0 3 UNKNOWN unknown FA 9b:01 active dead 0 0 3 UNKNOWN unknown FA 8b:01 active dead 0 0 3 UNKNOWN unknown FA 8a:01 active dead 0 0 3 UNKNOWN unknown FA 9a:01 active dead 0 0 2 UNKNOWN unknown FA 9d:00 active dead 0 0 2 UNKNOWN unknown FA 8d:00 active dead 0 0 2 UNKNOWN unknown FA 9c:00 active dead 0 0
# lsdev -Cc disk hdisk144 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk145 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk146 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk147 Defined C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk148 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk149 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk150 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk151 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk152 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk153 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk154 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk155 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk156 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk157 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk158 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk159 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk160 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk161 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk162 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdisk163 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdisk164 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdisk165 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdisk166 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdisk167 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdisk168 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk169 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk170 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk hdisk171 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk172 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk hdisk173 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk174 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk hdisk175 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdisk176 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk hdiskpower0 Available PowerPath Device hdiskpower1 Available PowerPath Device hdiskpower2 Available PowerPath Device hdiskpower3 Available PowerPath Device hdiskpower4 Available C6-T1-01 PowerPath Device hdiskpower5 Available C3-T1-01 PowerPath Device hdiskpower6 Available C6-T1-01 PowerPath Device hdiskpower7 Available C6-T1-01 PowerPath Device
The AIX guru was removing the paths, but as soon as we would reboot, they were back. Again, it was odd because the zoning should not have allowed PowerPath to see anything towards the DMX. It was almost like PowerPath was caching/storing that path information somewhere.
After finding the post by Ernes in this thread we ran through the steps he listed out (and we had NOT been running the emc_cfgmrg - just the AIX cfgmgr - command) it was like Christmas in June! :-) All of the UNKNOWN entries were gone and even after we rebooted it had cleaned up everything permanently for us. We even did a quick Live Partition Mobility test and moved the LPAR to a few different VIO servers with no issues. Thank you so much, Ernes, for taking the time to post up your response which really helped us out! With 13 more AIX LPARs to go in this migration effort, this has saved us a ton of concern!
A-J1
1 Rookie
•
49 Posts
0
August 27th, 2013 05:00
Hi dynamox,
The HBA WWN's are
WWN : 10000000c9955f66
WWN : 10000000c995602e
WWN : 10000000c9955f67
WWN : 10000000c995602f
Output of symaccess -sid xx list logins -dirport xx:xx command is below,can you suggest if they are all ok.
C:\>symaccess -sid 60 list logins -dirport 1f:0
Symmetrix ID :
Director Identification : FA-1F
Director Port : 0
User-generated Logged On
Identifier Type Node Name Port Name FCID In Fabric
---------------- ----- --------------------------------- ------ ------ ------
10000000c950b40d Fibre NULL NULL 01df00 Yes Yes
10000000c96c003e Fibre 10000000c96c003e 10000000c96c003e 01de00 Yes Yes
10000000c99432b5 Fibre NULL NULL 011d00 Yes Yes
10000000c994399e Fibre NULL NULL 012e00 Yes Yes
10000000c9955f66 Fibre NULL NULL 010100 Yes Yes
10000000c9955f67 Fibre NULL NULL 010200 Yes Yes
10000000c9955fb2 Fibre 10000000c9955fb2 10000000c9955fb2 011900 Yes Yes
10000000c9955ff6 Fibre NULL NULL 010300 Yes Yes
10000000c9956768 Fibre NULL NULL 6a0000 No No
c050760240460008 Fibre c050760240460008 c050760240460008 012703 Yes Yes
c0507602404d0002 Fibre NULL NULL 6a005f No No
c0507602404d0006 Fibre NULL NULL 012a01 Yes Yes
2511000dec2e3c80 Fibre NULL NULL fffc0a No No
2511000dec581fc0 Fibre NULL NULL fffc6a No No
C:\>symaccess -sid 60 list logins -dirport 2f:0
Symmetrix ID :
Director Identification : FA-2F
Director Port : 0
User-generated Logged On
Identifier Type Node Name Port Name FCID In Fabric
---------------- ----- --------------------------------- ------ ------ ------
10000000c950b55c Fibre NULL NULL 01df00 Yes Yes
10000000c96c3196 Fibre 10000000c96c3196 10000000c96c3196 01de00 Yes Yes
10000000c98c90ed Fibre NULL NULL 011d00 Yes Yes
10000000c9955fc0 Fibre NULL NULL 011900 Yes Yes
10000000c995602e Fibre NULL NULL 010100 Yes Yes
10000000c995602f Fibre 10000000c995602f 10000000c995602f 010200 Yes Yes
10000000c9956046 Fibre NULL NULL 010300 Yes Yes
10000000c9956f66 Fibre NULL NULL 010044 No No
10000000c99729ba Fibre 10000000c99729ba 10000000c99729ba 012e00 Yes Yes
c05076024046000a Fibre NULL NULL 012607 Yes Yes
c0507602404d0004 Fibre NULL NULL 010078 No No
c0507602404d000a Fibre c0507602404d000a c0507602404d000a 012d01 Yes Yes
2511000dec346ac0 Fibre NULL NULL fffc01 No No
C:\>symaccess -sid 60 list logins -dirport 1g:1
Symmetrix ID :
Director Identification : FA-1G
Director Port : 1
User-generated Logged On
Identifier Type Node Name Port Name FCID In Fabric
---------------- ----- --------------------------------- ------ ------ ------
10000000c98df621 Fibre 10000000c98df621 10000000c98df621 011c00 Yes Yes
10000000c992b625 Fibre 10000000c992b625 10000000c992b625 012800 Yes Yes
10000000c992c306 Fibre 10000000c992c306 10000000c992c306 012700 Yes Yes
10000000c993a2ac Fibre NULL NULL 012900 Yes Yes
10000000c993ac13 Fibre 10000000c993ac13 10000000c993ac13 012600 Yes Yes
10000000c99431fc Fibre NULL NULL 012c00 Yes Yes
10000000c9955f66 Fibre NULL NULL 010100 Yes Yes
10000000c9955f67 Fibre NULL NULL 010200 Yes Yes
c050760240460004 Fibre NULL NULL 012702 Yes Yes
2511000dec346ac0 Fibre NULL NULL fffc01 No No
2511000dec581fc0 Fibre NULL NULL fffc6a No No
C:\>symaccess -sid 60 list logins -dirport 2g:1
Symmetrix ID :
Director Identification : FA-2G
Director Port : 1
User-generated Logged On
Identifier Type Node Name Port Name FCID In Fabric
---------------- ----- --------------------------------- ------ ------ ------
10000000c98df620 Fibre 10000000c98df620 10000000c98df620 011e00 Yes Yes
10000000c992b624 Fibre NULL NULL 012800 Yes Yes
10000000c992b625 Fibre 10000000c992b625 10000000c992b625 012800 No No
10000000c992c307 Fibre 10000000c992c307 10000000c992c307 012700 Yes Yes
10000000c993a2ad Fibre 10000000c993a2ad 10000000c993a2ad 012a00 Yes Yes
10000000c993ac12 Fibre 10000000c993ac12 10000000c993ac12 012600 Yes Yes
10000000c99431fd Fibre 10000000c99431fd 10000000c99431fd 012c00 Yes Yes
10000000c995602e Fibre NULL NULL 010100 Yes Yes
10000000c995602f Fibre 10000000c995602f 10000000c995602f 010200 Yes Yes
10000000c99727fe Fibre NULL NULL 010054 No No
c050760240460006 Fibre NULL NULL 012606 Yes Yes
c05076024046000e Fibre c05076024046000e c05076024046000e 012604 Yes Yes
2511000dec2e3c80 Fibre NULL NULL fffc0a No No
2511000dec346ac0 Fibre NULL NULL fffc01 No No
2511000dec581fc0 Fibre NULL NULL fffc6a No No
etaljic81
1K Posts
0
August 27th, 2013 05:00
If cfgmgr doesn't work as dynamox suggested try running emc_cfgmgr. If you don't have it you can download it from here: ftp://ftp.emc.com/pub/elab/aix/ODM_DEFINITIONS
dynamox
9 Legend
•
20.4K Posts
0
August 27th, 2013 05:00
run "cfgmgr" or the server and them make sure it shows up as logged-in.
etaljic81
1K Posts
1
August 27th, 2013 06:00
Also, run a powerpath cleanup and perform a disk scan again:
A-J1
1 Rookie
•
49 Posts
0
August 27th, 2013 06:00
Hi,
Zoning for this server was not done by me, But there were some changes made to the zoning because we were in the process on migrating an MDS switch to a Brocade one. I will check the zoning and get back to you.
by the way, What should the best practise for zoning, Should I zone all 4 HBA's to all 4 ports to get 16 paths?
dynamox
9 Legend
•
20.4K Posts
1
August 27th, 2013 06:00
1 fscsi0 hdisk218 FA 2fA active dead 0 0
1 fscsi0 hdisk21 FA 2fA active alive 0 0
this is the same HBA that is showing as dead and alive at the same time. Draw a little diagram that explains how you zoned this server because if i assume this then its' only 8 paths:
Fabric A
HBA1 - 1F:0
HBA2 - 1F:0
HBA1 - 2G:1
HBA2 - 2G:1
Fabric B
HBA3 - 2F:0
HBA4 - 2F:0
HBA3 - 1G:1
HBA4 - 1G:1
etaljic81
1K Posts
0
August 27th, 2013 07:00
I'm assuming your topology includes two fabrics, two FA ports in each fabric. The AIX host has 4 HBAs; 2 HBAs per fabric. The only time you would zone all 4 HBAs is if you are worried about overloading an HBA on the host. Keep in mind that HBA1 and HBA2 would be zones to the same FA ports (since they are connected to the same fabric) so you are not gaining anything from the VMAX perspective. The only thing you are gaining is load-balance across all 4 HBAs. Now is that really necessary? I don't know your environment but most likely it is overkill.
A-J1
1 Rookie
•
49 Posts
0
August 27th, 2013 07:00
This AIX server is the most critical server in our environment as the most critical banking applications are running on this server. We need top notch performance on this server because it keeps the bank in business.
@Ernes you are correct about the topology, I believe the zoning is screwed up somewhere, I am checking it right now.
Thanks a lot for your greate help everyone.
dynamox
9 Legend
•
20.4K Posts
0
August 27th, 2013 07:00
Does your host need that many paths because of performance requirements ? Its your call.
travispbonfigli
4 Posts
1
June 2nd, 2016 07:00
Hello and Good Morning!
I just wanted to drop in a quick reply to what Ernes has posted as this fixed an issue we were having after migrating from a DMX-4 to a new VNX-5600 with a number of AIX servers. I initially was not going to scroll down and read the post as the original poster indicated that they were missing paths. Our issue was that after migrating we still were seeing the DMX paths in PowerPath - even though the zoning on our MDSes had clearly removed all visibility to the DMX. Here is what we were seeing and this was AFTER removing all visibility to the DMX:
# powermt display dev=all
Pseudo name=hdiskpower0
Symmetrix ID=000190103951
Logical device ID=0259
Device WWN=60060480000190103951533030
323539
state=dead; policy=SymmOpt; queued-IOs=0
==============================================================================
--------------- Host --------------- - Stor - -- I/O Path -- -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3 UNKNOWN unknown FA 8c:01 active dead 0 0
3 UNKNOWN unknown FA 9c:01 active dead 0 0
3 UNKNOWN unknown FA 9b:01 active dead 0 0
3 UNKNOWN unknown FA 8b:01 active dead 0 0
3 UNKNOWN unknown FA 8a:01 active dead 0 0
3 UNKNOWN unknown FA 9a:01 active dead 0 0
2 UNKNOWN unknown FA 9d:00 active dead 0 0
2 UNKNOWN unknown FA 8d:00 active dead 0 0
2 UNKNOWN unknown FA 9c:00 active dead 0 0
# lsdev -Cc disk
hdisk144 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk145 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk146 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk147 Defined C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk148 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk149 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk150 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk151 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk152 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk153 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk154 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk155 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk156 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk157 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk158 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk159 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk160 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk161 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk162 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk163 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk164 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk165 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk166 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk167 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk168 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk169 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk170 Available C3-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk171 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk172 Available C4-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk173 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk174 Available C5-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk175 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdisk176 Available C6-T1-01 EMC CLARiiON FCP VRAID Disk
hdiskpower0 Available PowerPath Device
hdiskpower1 Available PowerPath Device
hdiskpower2 Available PowerPath Device
hdiskpower3 Available PowerPath Device
hdiskpower4 Available C6-T1-01 PowerPath Device
hdiskpower5 Available C3-T1-01 PowerPath Device
hdiskpower6 Available C6-T1-01 PowerPath Device
hdiskpower7 Available C6-T1-01 PowerPath Device
The AIX guru was removing the paths, but as soon as we would reboot, they were back. Again, it was odd because the zoning should not have allowed PowerPath to see anything towards the DMX. It was almost like PowerPath was caching/storing that path information somewhere.
After finding the post by Ernes in this thread we ran through the steps he listed out (and we had NOT been running the emc_cfgmrg - just the AIX cfgmgr - command) it was like Christmas in June! :-) All of the UNKNOWN entries were gone and even after we rebooted it had cleaned up everything permanently for us. We even did a quick Live Partition Mobility test and moved the LPAR to a few different VIO servers with no issues. Thank you so much, Ernes, for taking the time to post up your response which really helped us out! With 13 more AIX LPARs to go in this migration effort, this has saved us a ton of concern!
Cheers,
Travis