How to remove a node in a IQ6000 storage

Patrick,

There are 2 options:

1. SmartFail the node

2. Hard-removal of the node

You should ALWAYS do option #1, because it's a graceful failure, and re-protecting the data to other remaining nodes.  If I were going to fail out say node 4 in a cluster, personally I would do this:

1. Suspend the node from all existing smartconnect zones.

isi networks modify subnet0:pool0 --sc-suspend-node=4

isi networks modify subnet0:pool1 --sc-suspend-node=4

2. Watch the number of connections to that node, as they trickle down.

isi statistics client --nodes=4

3. When the number of connections has trickled down do an acceptable number, permanently remove the interfaces from the smartconnect pools.

isi networks modify subnet0:pool0 --remove-ifaces=1:ext-1

isi networks modify subnet0:pool0 --remove-ifaces=1:ext-2

isi networks modify subnet0:pool1 --remove-ifaces=1:ext-1

isi networks modify subnet0:pool1 --remove-ifaces=1:ext-2

4. Smartfail out the node:

isi devices -a smartfail -d 4

5. Watch until the node is removed from the cluster:

isi status -q

isi job status (watch the smartfail/flexprotect/flexprotectlin)

6. un-cable and remove the node from the rack for disposal

~Chris Klosterman

Senior Solution Architect

EMC Isilon Offer & Enablement Team

chris.klosterman@emc.com

Alternatively if you are replacing an entire node-pool/diskpool of IQ nodes as part of a cluster refresh, you can use SmartPools to move the data to the new nodepool across the back-end, and then smartfail out the nodes 1-by-1, which when they are empty is really fast.

~Chris

Hi Chris,

Thank you for the answer, this is clear.

This is exactly what we are going to do:

-          The existing storage is composed of 7x IQ6000 nodes. We must replace it with 4x new X-200 nodes.

-          The existing Infiniband switch has 17 ports free (it’s a 24 port switch). We’ll then connect the new X-200 nodes to the free ports.

-          Then we will smartfail the old IQ6000 nodes one by one as you described, in order to remove them smoothly.

Is this procedure valid?

Thank you,

Patrick

Patrick,

Given that you are going to be replacing all the nodes, the easiest approach is:

1. Swap out the back-end switches (if they are being upgrade)

2. Add the new nodes to the cluster and the connect their network interfaces

3. Change the default file pool policy to store data on your new X200 nodes.

4. Create a file pool policy to move all existing data under path /ifs to the X200 node pool

5. suspend all the existing nodes from smartconnect, and watch them as the number of connections trickle down.

6. When the old nodes are empty, smartfail them out 1 at a time.  Do the node with the lowest nodeID last.  Run isi config status advanced, to see the mapping between LNN and NodeID.  The logic behind this recommendation is that the node with the lowest NodeID will be running the smartconnect service, if you were to fail out node1, then node2 then node3, the smartconnect service has to keep hopping to the node with the next lowest NodeID over and over.  By doing this node last, the smartconnect service only has to move once.

7. After you're all done, ensure that you have the cluster and all firmware on the latest target versions.

EMC PS can do this for you as what we term a tech refresh, but either way it's not overly complicated, which is of course one of the key benefits of the platform.

Enjoy your new nodes,

~Chris Klosterman

Senior Solution Architect

EMC Isilon Offer & Enablement Team

chris.klosterman@emc.com

twitter: @croaking

Hi Chris,

thanks a lot.

We are currently in contact with local EMC support to see if they can support us. Anyway good to know the procedure, and that it's not complicated.

Patrick