In my last post I kept it simple (chili-simple) in describing what OneFS® has done for enterprise data storage management. But for those of you who want to step behind the wizard’s curtain and into the technical details, these next three posts are for you.
In the year 2000, work began in earnest on OneFS. The most important design choice – and fundamental difference of OneFS – is that the storage system does not rely on hardware as a critical part of the storage architecture. Instead, the hardware is near-commodity; enterprise-quality, Isilon-certified components produced by commodity hardware manufacturers, such as Intel, Hitachi, and SuperMicro. Nearly all aspects of the storage system are provided in software, by OneFS. This includes data protection capabilities, automated data balancing and migration, and the ability to seamlessly add storage and performance capabilities on the fly. The two components which are not-quite-commodity are a high-speed battery-backed NVRAM journal and a 20Gbps point-to-point microsecond-latency interconnect, Infiniband.
In order to provide a high-quality, single vendor experience, OneFS works exclusively with the Isilon scale-out storage system, referred to as a “cluster.” A single Isilon cluster consists of multiple storage “nodes”, which are constructed as rack-mountable enterprise appliances – containing memory, CPU, networking, NVRAM, Infiniband, and storage media. An Isilon cluster must start with 3 nodes at a minimum, but scales as high as 144 nodes; at the time this paper was written, total capacity ranges from a minimum of 6 TB to a maximum of 10 PB. This is all in a single storage system, a single file system, and a single volume – so the complexity to the storage administrator does not change as the system scales.
OneFS stripes data across these storage nodes; as data is sent from client machines to the cluster (using industry-standard protocols, such as NFS and CIFS), OneFS automatically divides the content and allocates it to different storages nodes in parallel. This occurs on the private Infiniband network, which eliminates any unnecessary network traffic. Clients see an Isilon cluster as a single file system; this data distribution is completely transparent. When a client wishes to read a file, OneFS will retrieve the appropriate blocks from multiple storage nodes in parallel, automatically recombining the file such that the initiating client sees exactly what was originally written. This ability for OneFS to automatically distribute data across multiple nodes in a transparent manner is fundamental in how it provides transparent growth, next-generation data protection and extreme performance.
Adding additional capabilities to an Isilon cluster is significantly easier than other storage systems; the storage administrator simply adds another node into the rack, attaches the private networking, and instructs the cluster to add the additional node. This new node will provide not only additional capacity but also additional performance (since each node includes CPU, memory, and network). OneFS will automatically AutoBalance – move data across the Infiniband network in an automatic, coherent manner – existing data that resides on the cluster onto this new storage node. This balancing ensures that the new node doesn’t become a hot spot for new data and that existing data is able to gain the benefits of a more powerful storage system. The AutoBalance feature of OneFS is completely transparent to the end-user and can be throttled in order to minimize impact on high-performance workloads. This capability alone allows OneFS to scale transparently, on-the-fly.
In my next post, I’ll detail how OneFS solves the traditional “bin-packing” dilemma and unique data protection capabilities… Stay tuned!
