I couldn¿t help but to notice your interesting conversation and I would like to add my two pence worth to the mix¿
According to my notes from my EXCELLENT EMC CLARiiON performance course by Steve Stead: ¿The Write Intent Log (WIL), will, if used, always be updated when the primary image changes. This imposes the cost of an extra write, and adds additional latency to the host I/O.¿
Hy, I also can't imagine that the bitmapping process count as an IO... but I don't know. Another thing to consider for calculation: add the percents of the disk crossings to iops calc.
That's a good one: counting misalignment as extra IOps ! The problem with this is that you can't tell how much extra IOps there will be, because it depends on the block size. For 4kB blocks the extra IOps will be 12,5% (4kB / 32kB = 1/8). And if you can't have full stripe writes because of this, worst case it doubles the number of I/Os, because misalignment then generates 2 full stripe writes instead of one..... Worst case of course.
I'm not sure what you mean.... If everything is aligned and you know the IOps behaviour of your application, IMHO this should be enough to start a calculation. Disk crossings are a natural fenomenon due to the fact that data is spread accross multiple disks and full stripe writes are the writes that determine the number of IOps that will take place. In a misaligned setup however there will be more I/Os taking place than in an aligned setup. And besides that: a disk crossing by itself is an electronical switch / delay measured in micro seconds and cannot be taken into account in the whole IOps calculation.
So what percentage should I add to my calculation due to disk crossings ? I think zero.
Even if your LUN is aligned you have disk crossings as you stated normal phenomenon. You are right it normally suffice if you know the iops and the behavior of your app. if you doing 4k writes and somebody is spitting you a 2k IO between (can happen everytime, can't?), then you'll have disk crossings -> this is what I mean. (maybe it is anyway wrong..)
I thought being accuratly when doing a calculation: Here is what i'm thinking. Application: 1000 IOps 4k random (most of them) 30% writes 70% reads R 1/0 Disk Crossings maximum.(4k/64k=6,25%)
300 writes/sek --> 600 writes/sek to disk + %diskcrossing(6,25) = 637,5
R5 : --> 1200 writes (parity write cannot cause disk crossing and so->) 600 writes +diskcrossing% = again 637,5 + parity 600 = 1237,5
Your I/O capacity is based on the number of spindels of a certain type (10k of 15k) and what your storage can handle is the sum of IOps per spindel.
My problem with your "story" is that you state that a disk crossing is an extra I/O, while in fact IMHO it doesn't matter where this I/O is placed: on the next block, track, or even another disk. I suppose you could try to measure this and drill down in the number of IOs and do the math. Based on that I would like to believe you, other wise I must say I'd rather stay with the general rule of thumbs about read IOs and write penalties.
First thank you for the interessing discussion. The Clariion track called "Element Size" measured in Blocks is default 128 =64k. An earlier Flare version (i think prior Flare 19 ) you had the chance to change the element size to 32k or higher (i think up to 256k...).
I had telephone calls and even on site visits from an EMC Performance guru and we discussed the disk crossing issue.... but I'm still not sure about this. I'm simply lost there....
After an hour or two on the phone with him I decided it's probably best to stik with the best practice - known - values and perhaps reconcider those when it's going to be a tight setup with a customer who really sits on his money and really really reallly wants to get the most out of every penny he's paying.... But then again: for every few hours of me, he could also buy an extra spindle.... hehehehe
Blessing ? Ok... for sure... but still you can see developments going on where technology from Clariion migrates to DMX For example snapshots, virtual lun technology in the software section and the DAE's in the hardware section..... Clariion isn't all that bad.
I've had the same course with Steve Stead, but in my memory and in all my notes I can't find any hint saying that you need to take an extra IO into account when you're using the WIL. If you could translate a WIL biitmap I/O to a true I/O, I'm sure Steve would have mentioned this. I'm asking this question because I'm just not sure what to do with this. Perhaps it's even an option to simply send an email to him. I think I will do just that. Thanks for reminding me !
calle2
133 Posts
0
December 7th, 2007 10:00
I couldn¿t help but to notice your interesting conversation and I would like to add my two pence worth to the mix¿
According to my notes from my EXCELLENT EMC CLARiiON performance course by Steve Stead: ¿The Write Intent Log (WIL), will, if used, always be updated when the primary image changes. This imposes the cost of an extra write, and adds additional latency to the host I/O.¿
I hope this helps
Carl
mpi2
2 Intern
•
215 Posts
0
November 27th, 2007 04:00
I also can't imagine that the bitmapping process count as an IO... but I don't know.
Another thing to consider for calculation: add the percents of the disk crossings to iops calc.
best regards
Manfred
RRR
4 Operator
•
5.7K Posts
0
November 28th, 2007 02:00
And if you can't have full stripe writes because of this, worst case it doubles the number of I/Os, because misalignment then generates 2 full stripe writes instead of one..... Worst case of course.
mpi2
2 Intern
•
215 Posts
0
November 28th, 2007 03:00
If the access pattern of the application is clear why not?
RRR
4 Operator
•
5.7K Posts
0
November 28th, 2007 03:00
And besides that: a disk crossing by itself is an electronical switch / delay measured in micro seconds and cannot be taken into account in the whole IOps calculation.
So what percentage should I add to my calculation due to disk crossings ? I think zero.
mpi2
2 Intern
•
215 Posts
0
November 28th, 2007 04:00
You are right it normally suffice if you know the iops and the behavior of your app.
if you doing 4k writes and somebody is spitting you a 2k IO between (can happen everytime, can't?), then you'll have disk crossings -> this is what I mean. (maybe it is anyway wrong..)
I thought being accuratly when doing a calculation:
Here is what i'm thinking.
Application: 1000 IOps 4k random (most of them) 30% writes 70% reads R 1/0
Disk Crossings maximum.(4k/64k=6,25%)
300 writes/sek --> 600 writes/sek to disk + %diskcrossing(6,25) = 637,5
R5 :
--> 1200 writes (parity write cannot cause disk crossing and so->) 600 writes +diskcrossing% = again 637,5 + parity 600 = 1237,5
Maybe it is a big error in reasoning...
RRR
4 Operator
•
5.7K Posts
0
November 29th, 2007 03:00
My problem with your "story" is that you state that a disk crossing is an extra I/O, while in fact IMHO it doesn't matter where this I/O is placed: on the next block, track, or even another disk.
I suppose you could try to measure this and drill down in the number of IOs and do the math. Based on that I would like to believe you, other wise I must say I'd rather stay with the general rule of thumbs about read IOs and write penalties.
mpi2
2 Intern
•
215 Posts
0
November 29th, 2007 03:00
First thank you for the interessing discussion.
The Clariion track called "Element Size" measured in Blocks is default 128 =64k.
An earlier Flare version (i think prior Flare 19 ) you had the chance to change the element size to 32k or higher (i think up to 256k...).
best regards
Manfred
RRR
4 Operator
•
5.7K Posts
0
November 29th, 2007 03:00
dynamox
9 Legend
•
20.4K Posts
0
November 29th, 2007 07:00
haven't touched a Clariion in ages.... being stuck
here with a few DMX's...)
that's a blessing
RRR
4 Operator
•
5.7K Posts
0
November 29th, 2007 07:00
After an hour or two on the phone with him I decided it's probably best to stik with the best practice - known - values and perhaps reconcider those when it's going to be a tight setup with a customer who really sits on his money and really really reallly wants to get the most out of every penny he's paying.... But then again: for every few hours of me, he could also buy an extra spindle.... hehehehe
RRR
4 Operator
•
5.7K Posts
0
November 29th, 2007 07:00
RRR
4 Operator
•
5.7K Posts
0
November 30th, 2007 02:00
RRR
4 Operator
•
5.7K Posts
0
December 10th, 2007 06:00
I'm asking this question because I'm just not sure what to do with this. Perhaps it's even an option to simply send an email to him. I think I will do just that. Thanks for reminding me !