I was hoping someone could clarify a query I have had from one of our storage guys. We have a CX3-40 with a number DAE3P shelves. If we had 1 Shelf with FC disk and another shelf with SATA II disk on the same Bus and loop would the SATA disks effect the performance of the FC disks due to the slower rotational speeds and transfer times of the SATA disk on the same FC-AL.
Whoa there Kelleg...as the original author of both the Fundamentals and Best practices guides, and who shepherded them along for 5 years, killing many myths along the way...I must find myself again killing myths. In this case:
"Writing to a 7200RPM disk will take longer than writing to a 15K disk. So the slower disk will hold the bus longer."
The latter statement is not true. I can categorically state that we have never made that statement in either guide. But I understand that the misconception is widely held.
Here's the lowdown:
There have been times when we suggested using a different bus for certain drives - when they were at a lower interface rating, 2 Gb/s interface vs. 4 Gb/s. If you had the original 10K rpm drives for the CX3 series, for example, they were 2 Gb/s. You would not want to put them on a bus with 4 Gb drives as then the bus arbitrates down to 2 Gb/s. But our SATA drives are 4 Gb/s -- they will work on the bus just as well as any FC 4 Gb/s drive. So, why do folks think they 'slow down' the bus.
The confusion stems I think from 2 things:
1) statements that guided the user to separate high-bandwidth disk traffic (which is often but not always on SATA) from random. You don't want your backup to disk traffic to soak up all the bus bandwidth, affecting your online apps like Exchange. This has NOTHING to do with the disk transfer rate - it is about the workload you are asking the disks and the bus to absorb. If your BU2D is close to or more than 300 MB/s, you are using up most of one bus's bandwidth. You should put your exchange drives on another bus.
2) the misunderstanding that SATA drives somehow 'slow down' the bus. Nothing could be further from the truth. There are 3 factors at work here:
2-a) The disk transfer rate is immaterial - the transfer onto the bus is made from buffers in the HADA electronics, long (in computer terms) after the transfer from the platter is made, and it is controlled by the interface electronics.
2-b) The arbitration for the bus is handled by the interface electronics which, in the SATA II product we have shipped for over a year now, is 4 Gb/s capable. So, no slowdown from that.
2-c) The transfer PATTERN is really controlled by the SP. Those manufacturer specs for MB/s transfer rates are really just pie-in-the-sky (e.g., marketing) and only applicable to a single disk on a busmastering PCIe controller on a server (doing nothing else on that bus). In the context of a storage system, where you have multiple drives arbitrating & exchanging information in a controlled manner on a FC bus, those transfer times are immaterial. The transfer rates of the bus will only be matched by multiple drives, which will hit maybe 30 MB/s each with a good IO profile. Whether it is 10 drives or 20, it's all the same to the bus. There is a cost to having MORE drives on the bus, but that is a small effect.
Lastly, the proof is always in the pudding. If SATA drives 'slowed down' the bus then you'd wonder why we hit some of our higherst bandwidth numbers WITH SATA DRIVES. And, we do.
So, please do the reading yourself. Place drives on buses based on general aggregation of workload - if you have very different workload profiles then you might separate groups of disk by bus. Certainly do so if you have disks rated at different interface speeds (e.g., 2 Gb/s vs. 4 Gb/s). But in the majority of cases, mixing SATA and FC in an of itself has no effect in the real world.
The FC and SATA drives have to arbitrate for the shared bus to communicate. I have heard that the FC drives detach from the bus during any head movement on the drive itself (thus leaving the bus free for other attached drives to receive/send communication requests) and that SATA drives do not detach from the bus.
If this is the case then the slower SATA drives will hold up the bus while it transfers data.
Hmmm...ok - So the NCQ reduces latency on the SATA drives so releasing the bus for communication earlier.....?
What i am trying to understand is the architecture of the shelves (FC-AL) and how the use of SATA and FC disks effect the performance on the same bus. The SATA II disks have a 4Gbps paddle card on them so there is no difference in bus speed between the disks.
I read in the Ultrapoint Technology guide that the new shelves have a loop switch which may get round this?
I have read all the documentation that i can find and i can't seem to get a definitive answer.
All backend IO to and from the Storage Processors is done via Fibre Channel, even if the IO interface on the disk drive is something else. The "paddle card" you mention is part of that implementation for SATA drives. When transferring data, devices obtain the backend bus, transfer information at bus speed, and then release the bus. And within current disk shelves, each LCC (Link Control Card, which communicates via Fibre Channel to a Storage Processor) has direct connectivity to each disk slot (that's consistent with the "loop switch" comment). The effect of all this is that disk shelves of the same bus speed (e.g. 4Gbit/sec) can be freely intermixed, as the interface type and/or rotational speed of the drive itself are not directly coupled to the backend bus. Just be aware that placing a 2Gbit/sec member (for example, a legacy DAE2 shelf) on a 4Gbit/sec capable bus will reduce the speed of the bus it is placed on down to 2Gbit/sec.
Thanks for the reply that has helped clear a few things up!
You have mentioned that when data is transfered to the disk it is transferred at bus speed and then the disk releases the bus. Would the time it takes to physically transfer the data to the platters effect the lenght of time the disk holds the bus for. i.e the time it takes to transfer data to a SATAII (7200)disk that holds the bus would be alot slower than a FC (15k) disk even if the the bus is connected to both disks at 4Gbit/sec over Fibre Channel.
Maybe onboard cache on the disk and the NCQ releases the bus quicker?
Not sure about all the parts of your question, but if you look at the Best Practices guide, you'll see that differernt rotational speed disks will transfer data at different rates.
Writing to a 7200RPM disk will take longer than writing to a 15K disk. So the slower disk will hold the bus longer.
On-disk write cache is disabled for data protection - the Write cache on the storage processors is protected by the SPS battery backup, the disks are not. If you have write data in an on-board disk cache, you could lose this data if you have a power outage.
EMC CLARiiON Best Practices for Fibre Channel Storage: FLARE Release 26 Firmware Update - Best Practices Planning
ok so even though EMC support the mixing of SATA and FC on the same Bus, it would be wise not to mix them because the SATA ones rotate and transfer data at a slower rate and can therefore slow down the overall throughput due to holding the bus for longer.