Isilon smartpools migration with multiple node types

Hey Shane.  The easiest way to get where you want to be is to:

1.  Change your existing filepool policies to point to tiers instead of a specific node pool, so the tier might be called perf, an d comprised of just the X200s, and one called archive may be comprised of just the 108NLs..

2. Add the new nodes, and then put the new nodepools into the tiers.

3. Change the default filepool policy to write all new data to the X210s, so you stop making it worse.

4. Run a manual smartpools job, or wait until it runs on its own at 10PM, wait for that to finish.

5. Change your tiers by removing the old nodepools from them.

6. Run a manual smartpools job, or wait until it runs on its own at 10PM, wait for that to finish.

7. Watch and wait.  See how it goes, then start smart-failing if those nodes are getting empty.

These steps don't account for the networking changes on the front-end (one-hint there always remove the node with the lowest Node ID last, to minimize the number of group merges). You have to, of-course make sure you have enough back-end switch ports and cables to get them all connected at the same time, and that the cables are of the right type  X210 uses QDR/QSFP and X200 uses CX4/DDR IIRC, etc.

Anyway, hope it helps,

Chris Klosterman

Principal SE Datadobi

chris.klosterman@datadobi.com

Spillover=BAD

In general, you never want to run into a condition where a nodepool completely fills and spillover becomes necessary because it means that you've then lost control over where the data resides.  That said, of course leave in on, for those just-in-case scenarios. Will autobalance balance data between node pools in a performance tier?  I don't honestly know.  Perhaps someone else can answer that question.  However making sure that you have file pool policies that cover all data is really what you need to have here.  Something like move all data whose mtime is older than 180 days to the archive tier, and then another policy that says, move all data whose mtime is less than 180 days to the performance tier should get the job done, once you've pulled the old node pools out of the performance tier.  Using tiers is just an easier way to leave the policies you already have in place, and not have to mess with them, and accomplish the same goal.

Hope that helps,

~Chris

> Will autobalance balance data between node pools in a performance tier?

Tricky question... Just FWIW -- I wouldn't be surprised to see a two-stage approach here, in analogy to updating the target for newly created files.

SmartPools might not do it right away, because it's objective is not balancing. If it finds a file matching file pool rule for tier X, and that file is in disk pool A within tier X (while disk pool B in same tier X is empty), SmartPools could consider that file as fully satisfying its file pool rule, and therefore it will do nothing.

MultiScan/AutoBalance will find that file in pool A, but might not be aware that the file is supposed to be just somewhere in tier X (disk pools A + B), so it might have no clue about migrating the file from A to B.

Yet it must be possible somehow. My guess is that one should run SmartPools first after creating new tiers, which might update certain bits of information about pools and tiers in the filesystem, but not necessarily balance the pools. With that new information in the filesystem, MultiScan/AutoBalance could make proper decisions about balancing files over pools within the same tier.

The goal of this speculation, of course, is to provoke clarification from the Isilon team.

-- Peter

Did you  have any update about that question? (Will autobalance balance data between node pools in a performance tier?)

I think we all want to have an answer to this question. This would probably also apply to another situation where you want to seamlessly migrate your whole cluster to the cloud using CloudPools.

Hi!

     About this situation, where you need to migrate all cluster to a specific pool, you can create a FilePoolPolicy directing everything do that Pool.

     But about AutoBalance job balancing data between nodepools, i've had a real situation recently in a customer.

     We added new nodes with a different model of the ones that was already in the cluster and a new nodepool was created.

     After the execution of AutoBalance job, only some data was migrated to the new nodepool, but the space between nodepools was not balanced. Only some data was migrated to the new nodepool.

    So, we had to create some FilePoolPolicies to "force" the movement of some data to the new nodepool.

Regards,

Bruno

Hi,

I was wondering if there are 2 tiers on a cluster. One is hybrid and one is for a specific node pool. Can we run autobalance job on the hybrid tier only so that the other tier is not affected by this data balancing?

Thank you in advance.
Best Regards,

Nourhan