Ask the Expert: Introducing ScaleIO 2.0

This Ask the Expert session is now open for questions. For the next couple of weeks our Subject Matter Experts will be around to reply to your questions, comments or inquiries about our topic.

Let’s make this conversation useful, respectful and entertaining for all. Enjoy!

Afaik (pls correct me if I'm wrong), ScaleIO keeps 2 copies of each data block distributed between fault sets (i.e. in 2 separate fault sets), and this is always the case and not configurable.

Q1: Is there any documentation available how the algorithm distributes data between more than 2 fault sets and/or between SDSs that are members of a fault set and those that aren't?

Q2: 2 copies of data is not very redundant, so can you pls share your experiences how customers overcome this limitation?

Q3: Do you know about any plans/time frames to extend this to 3 copies in at least 3 separate fault sets?

Thanks a lot!

Thanks for the questions Groer.

You are correct, we are doing 2x mesh mirroring. Answers to your questions below:

Q1: Faults sets are a delineation of a set of nodes that could fail together. Example: by have the nodes in a rack within a fault set, the algorithm makes sure that all mirrors go to a nodes outside of it’s fault set. (Ie a different rack). That way, if multiple nodes within a fault set fail, then the data is still available. By default, each node is a fault set. The minimum number of fault sets is 3, but be aware that fault sets can affect usable space. Especially if you are have a small number of them.

Q2: When a devices or node fails, the rebuilt is done in a massively parallel fashion as every node storing data within that storage pool is working to redistribute that data. And since there are so many nodes involved and the data is mirrored, these rebuilds happen in a matter of minutes. So that window of risk is very low – key is to understand that since the rebuilds are so fast (as long as you follow some basic best practice) you actually have lower probability of DU/DL compared to typical RAID-6 and even 3-mirror (which suffers from really, really long rebuild times) . We have an internal tool that will give the availability of a specific configuration based on all the components and the network connectivity to the nodes. What we find is that we have very high availability with 2x mesh mirroring and have very large service providers using this for critical applications. Risk can additionally be mitigated by creating production domains and storage pools as cluster grow to larger and larger sizes as well as using fault sets.

Q3: Today we do 2x mesh mirror but are exploring other data layout options for the future. However, availability has not been something that has been an issue or driving these discussions.

Thanks,

Jason

Thanks for the question BaDMaN,

Great question. We are exploring both of these options. If you want more details, reach out and we can discuss the roadmap.

Thanks,

Jason

Hello osaddict,

What would be the state of the cluster if the fault set fails and there is not space for rebuild?

Ex, 4 Rack cluster , Rack is a fault set and lets assume the fault set fails . It might be difficult to persuade a customer to have the capacity of the rack as spare capacity .

Hi Experts

I have question about scaleIO performance

1, how many factors would be have impact in performance?

(disk type,network,server spec,number of SDS, and any other?)

2, will the number of devices in each SDS have impact in performance

(eg, 3 SDS with 3 disks in each one VS 3 SDS with 6 disks in each one)

3, the devices be added to SDS can be disk or an unmounted partition,

is performance different if we use a raw disk from several unmounted partition

(eg, add 1T raw disk VS add 10*100G partition from the disk )

Best regards

Wangzz

do you intend to develop an internal replication (no recoverpoint) or even a stretch cluster

Hi all,

Do you have resp. are you willing to share data about the performance impact of integrity features like checksum, zero padding, background scanner? Even orders of magnitude would be interesting, i.e. are we talking rather tenths of a percent or 1 or 10 percent?

Thanks a lot!

Are there any plans to certify ScaleIO to work in stretched multi-site mode?

Technically it should not be a big problem even now: one site = one fault set. But supportability and interaction with host-side solutions (say vMSC cluster) is a question.

Q1) I will be interested about if there were plans for dedupe and compression to match the feature sets of other hyperconverged solutions. 

Q2) Any plans for getting ScaleIO on VMWare's HCL list?  I don't see it listed.

Q3) Is there going to be a recommended Hardware list for ScaleIO such as cards, drives, etc... ?

Q4) Any thoughts on tiering for ScaleIO?  Would be nice to have a SSD tier and 10K tier on each server presenting storage.

Hi Aiyappa,

The system will not allow you to use the space so that this situation would not occur. I agree that with 4 racks, the overhead of reserved space becomes very high. This is why it doesn't really make a lot of sense until you get to around 10 racks, if you are using racks at the the fault set.

Fault set are about grouping common components that might fail together. Most datacenters don't have whole racks that fail, so the rack may not be the best unit to use. Another way I have seen fault sets use is in chassis that have multiple nodes. Example: a 2u chassis with 4 nodes inside of it. If there is concern that the chassis could fail and take down all 4 nodes, making them a fault set makes sense.

I hope that help,

Jason

Hi Aglidic,

We are looking at adding replication. Please contact us to discuss roadmap timelines.

Thanks,

Jason

Hi Groer,

Yes, happy to share this information.

The impact of checksum depends on the OS:

Linux: no real impact (the implementation is using specific Intel CPU commands that make this almost a “HW solution”).

Windows: ~30% max impact (pure SW implementation), please note that if you are not close to the limits of the max IOPs, you will not have an impact.

ESX: on the SDS, no impact (“HW” implementation) on the SDC, ~30% max impact (pure SW implementation), please note that if you are not close to the limits of the max IOPs, you will not have an impact.

Zero padding: in real life has almost no impact, similar to Thin etc because this feature only affects “new” writes so unless this is a fresh new volume or a test, the impact is really small.

Background scanner: no impact, the scanning is done at a low rate that is almost noise to the system.

Best,

Dan

Hi usakwilliamson,

1. Yes, there are plans to add compression. dedup I am not sure. we find that for most generic applications the benefit is not high enough to justify the huge amount of resources dedup consumes from a converged config.
2. Sorry, I do not control VMware and will let someone else address this question
3. ScaleIO is HW agnostic. Just like when you buy Oracle, they don’t tell you what HW to buy, ScaleIO doesn’t either. We only specify what Operating Systems we support and any vendor that wants to sell its HW needs to qual its HW on that OS. What you can do if you like is look at the VxRack Flex or VxRack Node HW options (which is a ScaleIO HW+SW solution) and use the same exact components.
4. We do have Tiering.  In the VxRack solutions we use CacheCade which allows you to configure an SSD as Read and Write cache for the HDDs. You can also use RFCache which comes free with ScaleIO (from 2.0) and use that to configure an SSD or any Flash device as a large Read Cache for the HDDs.

Best,

Dan