Isilon load balancing

sultana ,

That's not exactly how it works, If you have 3 nodes and write all the data to one node, the data will be protected evenly across the back-end so that all 3 nodes in the node pool fill evenly.  In the event that you add a 4th node, the addition will kick off 2 jobs:

Autobalance: To re-balance the data across the 4 nodes

Autobalance(LIN): To re-balance the LINs(Logical inodes) across the 4 nodes.

In that case you'd have 3.75TB of user data + parity overhead from erasure coding on each node.

But the term load-balancing means something else entirely in most cases.  That's talking about utilizing all the front-end interfaces to accept writes from the clients.  This is achieved using SmartConnect which is a method for using DNS Delegations to a custom DNS server on the Isilon cluster, and then balancing all the incoming connections across as many interfaces and nodes as are available.

I co-wrote a paper on SmartConnect a couple of years ago that would probably be a very good read for you as a primer:

EMC Isilon External Network Connectivity Guide: Routing, Network Topologies, and Best Practices for SmartConnect

Hope this helps,

~Chris Klosterman

Principal Pre-Sales Consultant, Datadobi

chris.klosterman@datadobi.com

thank you for your answer.

So how does Isilon cluster balance the data between nodes ? How does OneFS stripe data across all nodes and protect the files , directories and associated metadata?is there any software for that ?

for example: SmartConnect attribute a node1 to user1 who has lot of data to write and user2 to node2 with few data to write, for a user3 it will be attributed to node2 since node1 is still busy...this way, will we have the same capacity written in the both nodes?

or the scenario is that OneFS uses the InfiniBand back-end network to allocate and stripe data across all nodes in the cluster automatically?

My concern is about : is it necessary to have the same data capacity in all nodes of a cluster?or we can find a node with 6 TB and other 3TB for example?

thank you.

Sultana,

When a user connects to the cluster say node 1 the data they write to the cluster will not all be written to node 1 just because they are connected there. all nodes participate in data striping for protection of said data based on smartpools rules and data protection levels. this is what will allow you to lose an entire node and still have access to all data even with an entire node down in  default 2:1 protection levels.

That being said there are a variety of different reasons that you may see an imbalance of data between nodes. I would be looking into why any node would have 3TB of difference if they are in the same pool. Ideally what will happen is when nodes reach a 5% skew in capacity difference a job like auto-balance,auto-balancelin, or multiscan will run to start the balancing of that data out between nodes. there could be reasons why this job is not or has not run like for instance job priority levels can prevent autobalance from running.

An example of this is if you had autobalance running but then a drive failed. autobalance would be put on pause so that flexprotect could run (as it has higher priority for that job) to remove the failed drive and reprotect that data. after flex is done we would expect the autobalance to continue where it left off balancing the data between nodes in any given pool.

Cheers,

D_Trace

Sultana,

you can certainly purchase 2 different sized nodes for you cluster as long as you have the recommended number of node needed to create a new pool which in many cases is 3 nodes.  keep in mind that you open up another door of cluster management complications though when you have a cluster with 2 different type of nodes types/size nodes in it thought.  My opinion to from a management standpoint would be to try and stick with the same node type, and size throughout the life of the cluster.  There is only one time when I would welcome mixing different node types in my opinion... That is when your going to upgrade the entire cluster from older nodes to a newer node cluster.  I hope this helps.

Hi sultana,

I would like to expand on what chjatwork said and mention that not only do the nodes need to be same type, but the same hardware configuration. There is some compatibility between older/newer node types and there is documentation to assist. https://support.emc.com/docu44518

There is White Paper on the Isilon Job Engine that has more information about AutoBalance here: https://support.emc.com/docu51125