Re: Re: Ask the Expert: Introducing DSSD & Rack-Scale Flash
Q:How is the storage presented to the host: Object/LUN/NAS/SMB/NFS?
A: Hi "@AI..." thank you for your excellent question. The D5 offers several ways for storage to be presented to the host. Keep in mind that up to 48 hosts are redundantly connected to the D5 via PCIe I/O cables for maximum performance and for sharing the storage. With that "PCIe Mesh Fabric" architecture in mind, for host connectivity, we offer three ways to configure and present DSSD D5 storage to your applications on your hosts:
DSSD Block Driver: DSSD created a high performance DSSD Block Driver interface that allows customers to use legacy block device applications without the need to modify their existing application I/O source code in any way. There is also the “DSSD Block Service” running in the user space that handles the management of the data path. To control which hosts can see which Block Devices, the concept of a DSSD volumes exists. On the D5 appliance an administrator first creates volumes. Those volumes are assigned to hosts. Once a host has a volume assigned, the host administrator can carve up D5 storage into objects of whatever size and of varying block lengths (512Byte and 4K to be usable by the Linux OS).
Once the host administrator has created these block objects, they next configure and start the DSSD Block Device Service. Once the DSSD Block Device service is running on the host(s), a block device entry is made in /dev. At this point your block device looks like any other block device and the device can be used as a raw device or you can create and mount a file system on it. Here is an example from my D5 where I have created three separate block devices, each device is a 1TB device. They are named dssd0000, dssd0001 and dssd0002 and are lncluded in the list of other block devices also installed in my host, such as the root directory and an SSD device installed in the server as well.
There are two other access methods that are available to applications for using DSSD D5 storage. The Flood Direct Memory API and via DSSD Plug-Ins.
Flood Direct Memory API: Any application can be modified, or new Applications can be developed to use the Flood Direct Memory API “verbs”. All data is stored on the D5 as some type of an Object. The “libflood” API C-library includes commands to create, modify, destroy, read and write to objects. A DSSD Block object can be accessed directly from an application using the Flood Direct Memory API, and achieve maximum D5 performance as opposed to using the DSSD Block Driver.
DSSD PLUG-INS: Essentially DSSD may create an API interface for certain categories of applications – like HDFS to allow the upper level application code to remain unchanged yet still provide direct objct access to the D5. The key difference here is that DSSD can create API Plug-Ins or other common application interfaces for applications that support modification of their file system or I/O subsystem, as opposed to a customer “rolling their own API code”.
The first example of this is the DSSD Hadoop Plug-In that will be available at GA. That Plug-In will enable HDFS distributions to install the DSSD Hadoop Plug-In and then make no changes to the upper level application interface. On the back end the Plug-In will perform native D5 I/O and bypass the Kernel. The first HDFS distribution that will be certified to use the DSSD Hadoop Plug-In is “Cloudera”. Other certifications will be made available as they are completed. For example Pivotal, Hortonworks or any other HDFS distribution supporting standard Hadoop Plug-Ins.
For More information I suggest going to our technology briefs linked on our DSSD web page.
Specifically checkout our "Modular Storage Architecture" tech brief: the https://www.emc.com/en-us/collateral/data-sheet/h14867-ds-dssd-d5-object-oriented-next-gen-analytics...
On our support.emc.com portal you can access all of our DSSD D5 Documentation. Here is a link to our DSSD Client Guide:
Review chapter 3 - Managing Clients. There is a section in there that discusses the creation of block devices.
I hope this answers your question. Look forward to more discussion!