Unsolved
This post is more than 5 years old
152 Posts
0
5489
high queued -i/o's in powermt output
We had a issue with a node of a windows cluster. There are 6 devices mapped and masked to that node from DMX-4. When i see the powermt display dev=all out put i see that out of 6 devices 1 of them has High queued i/o's .The rest 5 dont have any queued i/o's . I did some analysis on that device and i see the below readings.
DEVICE IO/sec KB/sec % Hits %Seq Num WP
19:48:11 READ WRITE READ WRITE RD WRT RD WRT Tracks
19:48:11 DEV001 (0100) 124 2154 16776 5241 100 96 13 53 753
The FA port utilization also seems to be high
19:47:04 DIRECTOR PORT IO/sec Kbytes/sec
19:47:04 FA-8C 1 2589 27628
19:47:04 FA-9C 1 5085 84102
can anyone tell me how to reduce the queued I/o's for that particular device (0100)
Pseudo
Symmetrix
Logical device
state=alive; policy=SymmOpt; priority=0; queued-IOs=28
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
2 port2\path0\tgt0\lun20 c2t0d20 FA 9cB active alive 14 0
4 port4\path0\tgt0\lun20 c4t0d20 FA 8cB active alive 14 0
Quincy561
1.3K Posts
0
September 2nd, 2010 20:00
Sounds like you answered you own question. The FA utilization is high, so adding more FA CPUs (not just ports) will most likely reduce the queues.
Nollaig1
1 Rookie
1 Rookie
•
137 Posts
0
September 3rd, 2010 02:00
Hello,
Just to also add thats queued I/O's are historic as well. If they are stable at 14 and not incrementing they may indicate that there was an issue at one time, but not necessarily now.
The queued I/Os counter is only cleared at reboot when it is returned to zero.
Hope this helps,
Nollaig
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
September 3rd, 2010 02:00
Quincy56,
wouldn't high FA utilization affect all devices in that case and not just one ?
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
September 3rd, 2010 02:00
Nollaig,
are you talking about PowerPath "Stats Q-IOs" counter ?
Quincy561
1.3K Posts
0
September 3rd, 2010 04:00
I am assuming all the devices do not have the same IO rate. The devices with low IO rates won't show high queues.
vimad
152 Posts
0
September 3rd, 2010 04:00
That disk(0100) is a transaction log disk which is experiencing slow writes. The remaining disks have very less I/O or no I/O at all. The Queued I/O's on this Particular transaction log disk is varying from time to time and they are not able to process transactions fastly. So, can anyone tell me how to minimize this effect. Will adding FA's help me to improve the perfromance or there any other ways for better perfromance for that particular drive (0100).
Quincy561
1.3K Posts
1
September 3rd, 2010 04:00
Of all the IO types the FA can process, writes to slots not yet write pending are the hardest for it to process. They also tend to come in bursts. I would bet good money that adding two more paths will significantly reduce the response times.
ZepHead
88 Posts
0
September 3rd, 2010 04:00
So it appears that a particular device is singled out and can focus there. What is different about this device than the others..? Is it a quorum device..? Did this just start happening and the only occurance..? Also, you can set "throttling-related" values per device, see: "PowerPath CLI and System Messages Reference Guide". In addtition, can also use powermt display with arguments (ex: powermt display dev=<> every=<>), to watch in intervals.. I know that powermt restore clears "historical errors" but unable to conclude this will help with queued I/O, due to being 'real time'. As Nolliag stated, need to see if these values change while being monitored...
Nollaig1
1 Rookie
1 Rookie
•
137 Posts
0
September 3rd, 2010 07:00
Hello,
Sorry Dynamox, just saw query. Looks like possible issue has been nailed down now.
Thanks again!
Nollaig
vimad
152 Posts
0
September 3rd, 2010 09:00
I decided to add two more FA's for the existing windows cluster. But I need some confirmation on the FA port settings. This is what we have at present on
on 8c:1 which is conencted to the cluster
SCSI Flags
{
Negotiate_Reset(N) : Disabled
Soft_Reset(S) : Disabled
Environ_Set(E) : Disabled
HP3000_Mode(B) : Disabled
Common_Serial_Number(C) : Enabled
Disable_Q_Reset_on_UA(D) : Disabled
Sunapee(SCL) : Disabled
Siemens(S) : Disabled
Sequent(SEQ) : Disabled
Avoid_Reset_Broadcast(ARB) : Disabled
Server_On_AS400(A4S) : Disabled
SCSI_3(SC3) : Enabled
SPC2_Protocol_Version(SPC2) : Enabled
SCSI_Support1(OS2007) : Disabled
}
The other FA (8a:1)which we thought of adding to the cluster has the below settings
SCSI Flags
{
Negotiate_Reset(N) : Disabled
Soft_Reset(S) : Disabled
Environ_Set(E) : Disabled
HP3000_Mode(B) : Disabled
Common_Serial_Number(C) : Enabled
Disable_Q_Reset_on_UA(D) : Disabled
Sunapee(SCL) : Disabled
Siemens(S) : Disabled
Sequent(SEQ) : Disabled
Avoid_Reset_Broadcast(ARB) : Disabled
Server_On_AS400(A4S) : Disabled
SCSI_3(SC3) : Enabled
SPC2_Protocol_Version(SPC2) : Disabled
SCSI_Support1(OS2007) : Disabled
}
Can any one confirm if i can go ahead and use 8a:1 and 9a:1 as additional FA's for the windows cluster. SPC2 bit is different on both .
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
September 3rd, 2010 09:00
what version of windows ?
Boom1
131 Posts
0
September 3rd, 2010 10:00
Compliance
2007
vimad
152 Posts
0
September 3rd, 2010 10:00
If we set via hba_flags do we need to reboot the cluster in order to take effect the settings.
dynamox
2 Intern
2 Intern
•
20.4K Posts
1
September 3rd, 2010 10:00
or you can drop the zone from that host to the new FAs
Message was edited by: dynamox
vimad
152 Posts
0
September 3rd, 2010 10:00
These are the Windows cluster details..