3 reasons all-flash storage might be here to stay

By Jim O’Reilly, Contributor

Storage is in a state of revolution. New ideas are appearing regularly, primarily driven by flash storage. The performance mark of 150 IOPS per drive, that stood for 20 years, is now utterly broken. SSD drives are creeping towards 750K IOPS, which offers more random performance in a single drive than whole large storage farms could achieve just five years ago.

There are drawbacks in that speed, though. A storage farm can work with all the servers attached to the SAN, and allows a method for transferring information, and preventing single points of failure. This need has driven the availability of all-flash arrays, which have the benefit of plugging in to existing SANs. These shareable appliances have as much as 2 million IOPS.

At this point you might ask, “When does the spinning hard-drive go away?”That may be the most complex question in the industry today!

First, we have to note that the storage industry, especially in the enterprise, is a bit conservative. Mainly, this is due to 30 years of not having serious alternatives to RAID arrays or NAS. There is also the concern to protect data and a building a resistance to change. SANS and NAS will be around for a while.

Next, flash wears out with use. This was partly resolved in the past by using Single-Level Cell (SLC) technology, which had over 100 times the durability of the much less expensive Multi-Level Cell (MLC) devices. Advances in processes have brought the durability of MLC up to near the levels of SLC. Wear-out problems can be ignored for almost all applications.

That leaves three other major issues on the table: cost per terabyte, availability, and the infrastructure/applications to best use flash performance. Herein lie the three reasons why all-flash storage might be here to stay.

1. Cost has to be carefully compared. An enterprise-class SSD needs to be compared with its nearest enterprise hard disk rival, which costs around $500. On that basis, SSD is about two to three times the price per terabyte, though perhaps 1,000 times cheaper on dollars/IOPS.

Here’s where complexity creeps in. The extra cost of SSD is often offset by needing many fewer servers or drives, usually with a net saving. For example, running a virtual machine environment needs 10’s of HDDs per server – or 1 SSD. Capacity isn’t the issue, it’s IOPS/virtual machines.

There are also prosumer-class 1TB SSDs which retail for $450. These don’t have all the durability extension features of the enterprise-class drives, but we’ll come back to them as we look at new storage tiers.

2. The availability question is based on assessments that we won’t have enough flash die capacity to replace HDD anytime soon. One result is that flash prices are stagnant. Technology is also running into a wall, since we’ve reached sub-20nm levels in wafer production, and any process advantages are difficult to come by. New approaches like 3D stacking and even 3D devices hold promise of future evolution, but it will be a while before they impact the market.

3. Finally, the apps and OS are beginning to change. Big data solutionsrecognize that and have re-optimized around flash storage, with substantial improvements. More generally, PCIe-based SSD such as Dell’s Express Flash are entering the fray, and these will use a combination of PCIe and NVMe to really speed up access.

Apps, unfortunately, are not immune from redesign. With DRAM expanding rapidly, and fast storage, the file IO sections of the code need a rethink to get the best out of the new configurations.

It’s worth it, since gains of 10 times or so are possible. Quoting from Dell’s website:

“IOPS-hungry applications like Oracle, SQL, and SAP can move business faster, better inform decisions, and improve customer service, but providing the performance they need can be costly. Our flash-optimized solution can reduce latency up to 90%, in 84% less rack space, for 80% less cost/GB than conventional spinning disk solutions. With this price/performance balance, plus the ability to scale on demand, why wouldn’t you want to move critical workloads to SSDs? Finally, flash is within reach—take it.”

That brings us to the storage tiering question. Hard-disk RAID arrays are no longer the top dog for external storage. In the short haul, we’ll see a contest on the SAN side between all-flash arrays fronting existing HDD arrays versus hybrid arrays with SSD and HDD. Servers will be similarly conflicted, with fast SSD and bulk SATA drives competing against all-SSD configurations.

Over time, HDD will be relegated to just bulk storage and archiving, and all the active data will be on flash or SSD. At this point we can expect a couple of solid-state tiers, with superfast top-tier storage and slower, cheaper prosumer SSD as a capacity tier.

It will take a decade, but at some point the supply side for SSD will catch up to demand, new technologies will increase capacity per die and SSD will ease out the remaining bulk storage HDD. The economics of low failure rates, low power consumption, ability for zero-chill operation and performance will add to the equation, and the transition to an all-flash environment is inevitable.

About the Author: Power More