Highlighted
3 Argentum

Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

hello,

we are doing netapp to isilon migration using isi_vol_copy, we noticed very strange thing related to size. initially we bought isilon with sizes that matches / little more than netapp so once we started migration isilon is consuming lot more space than we anticipated.

here i picked one volume on netapp ( its not a flex clone , original volume ) which is 55gb ( attached the screen shot ). once we do isi_vol_copy we noticed the size on isilon its taking 263gb ( attached the screen shot ) also we verified the data on both its same even application team verified after migration they are good and they matched the data as well. which includes the data-protection overhead still its way way  high.

curios , is netapp has dedup feature ? which isilon didn't.  bec this is my qa volume which has prod / staging / test ..   if this the case then our storage sizing is not right.

0 Kudos
1 Solution

Accepted Solutions
Highlighted
3 Argentum

Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

For rough calculations this is more like RAID 5.  Even though you get 2 disks worth of data protection, it is done by using a layout trick so that no additional overhead is required in most situations.

So 1 node's worth of storage is being pulled out of your 6 node cluster.  You can look at this as 1/6 which is 16% of the total cluster is overhead or as a 1 node of overhead / 5 nodes of storage which gives you 20% overhead.  Both calculations are correct and say slightly different things.

Let's assume for simplicity sake you have 6 x 100 TB nodes for a total raw capacity of 600 TB.  Your usable space would be roughly 500 TB.

600TB * .16666 = 100 TB of overhead

At small node counts both numbers are easy to calculate using approximations.  Things get more complicated when you exceed 16 nodes.  The overhead stays fixed after a certain point due to the maximum number of nodes we will stripe data across (16) and the number of parity stripes required (anywhere from 1 to 4).

View solution in original post

8 Replies
Highlighted
4 Beryllium

Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

Your Netapp has dedupe on ("Status -> sis").

If your three stage areas are 99% or so identical, dedupe might reduce space to 1/3.

Un-deduped those 55G will become 3 * 55G = 165G.

Protection overhead is not included on the Netapp listing.

Assuming a 3 node Isilon cluster, OneFS protection overhead

will be 50% of the user data (or more when mirroring):

165G user data -> 1.5 * 165G = 248G space consumption (incl. ovh.)

Pretty close to the 263G you are seeing...

You can double check, du -A (or du --apparent-size)

will report 3 x 55G on both Netapp and Isilon, I bet.

(Isilon will have dedupe later this year.)

Cheers

Peter

Highlighted
3 Argentum

Re: Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

Hi Peter,

thank you for the useful info. this particular volume is on 6 node isilon cluster and the source dedupe has 3/4 volumes on netapp [ prod.dev,qa,xxx ] so that make sence.

question so mentioned about onefs protection overhead right so let assume at this point on isilon it showing 100Tb available storage so if i want to keep folder of 50gb [ independent one not part of any de-dupe pairs stuff involved ] then it will occupay 50 * 1.5 = 75GB ??

i am under impression that what over its showing here in this case 100TB includes the overhead .am i wrong ?

0 Kudos
Highlighted
3 Argentum

Re: Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

Your actual space usage is going to vary depending on the file mix.  For discussion sake, let's assume your 50 TB of source data has no files < 128 Kib.  Then the actual physical space taken on the Isilon, at a protection level of N+2:1 should be approximately 58.3 TiB.  On a 6 node cluster, the overhead is approximately 16%.

Highlighted
3 Argentum

Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

may be dumb question so i understand default prot level > 18TB cluster ( in my case ) is N+2:1 so can i calculate this as RAID6 equivalent?  also it would be great if u explain how u come up with overhead of 6 node cluster is 16% ?  each node of my cluster is 34 drives.

0 Kudos
Highlighted
3 Argentum

Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

For rough calculations this is more like RAID 5.  Even though you get 2 disks worth of data protection, it is done by using a layout trick so that no additional overhead is required in most situations.

So 1 node's worth of storage is being pulled out of your 6 node cluster.  You can look at this as 1/6 which is 16% of the total cluster is overhead or as a 1 node of overhead / 5 nodes of storage which gives you 20% overhead.  Both calculations are correct and say slightly different things.

Let's assume for simplicity sake you have 6 x 100 TB nodes for a total raw capacity of 600 TB.  Your usable space would be roughly 500 TB.

600TB * .16666 = 100 TB of overhead

At small node counts both numbers are easy to calculate using approximations.  Things get more complicated when you exceed 16 nodes.  The overhead stays fixed after a certain point due to the maximum number of nodes we will stripe data across (16) and the number of parity stripes required (anywhere from 1 to 4).

View solution in original post

Highlighted
3 Argentum

Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

Hi Andrew,.

thankx for ur detailed explanation. but one last my prev question   my isilon dashboard showing me HDD available is 100TB so this is not including the overhead correct.

0 Kudos
Highlighted
4 Ruthenium

Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

You are correct, the available space in the dashboard is not accounting for any anticipated overhead.  One way to look at why that would be difficult to forecast is:

1) Depending on the file size, the overhead will differ

Not sure if you had seen the post, but there is an awesome discussion (still ongoing) that talks about the overhead.  I am going to recommend to everyone that asks about the topic, to read the post.

https://community.emc.com/thread/178281

2) Even though each pool has a default protection level, as you know, you can set a different protection setting at a directory level (for instance, via File System Explorer)

Also, there is a very good 7 page paper (4 if you take away the coverpage and TOC ) that talks about managing your free space:

https://support.emc.com/docu48119_Best-Practices-Guide-for-Maintaining-Enough-Free-Space-on-Isilon-C...

Highlighted
3 Silver

Re: Netapp - Isilon volume migration : volume sizes mismatch

Jump to solution

Was your question answered correctly? If so, please remember to mark your question Answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue

0 Kudos