I have a customer that has an existing Isilon cluster that is due for a refresh. The existing cluster consists of x200 and IQ108 nodes. The newly purchased nodes that will be added to this cluster are X210 and NL410 nodes. I've checked node compatibility and we are good there. My concern is how to move the data from the X200 tier to the x210 tier and then move data on the IQ108 tier to the NL410 Tier. I've done smart pool migrations before with a single node type (NL400 to NL410) by changing the default file pool policy then smart failing out the old nodes.
How can I move data from one tier to another equivalent tier? I don't want to put data that lives on X200 nodes on NL410 node and cause a performance impact.
Will any existing file pool polices convert from the X200 SP to X210 SP and IQ108 SP to NL410 SP? If not, do I have to recreate the file pool policies?
Thanks in advance for any responses.
Hey Shane. The easiest way to get where you want to be is to:
1. Change your existing filepool policies to point to tiers instead of a specific node pool, so the tier might be called perf, an d comprised of just the X200s, and one called archive may be comprised of just the 108NLs..
2. Add the new nodes, and then put the new nodepools into the tiers.
3. Change the default filepool policy to write all new data to the X210s, so you stop making it worse.
4. Run a manual smartpools job, or wait until it runs on its own at 10PM, wait for that to finish.
5. Change your tiers by removing the old nodepools from them.
6. Run a manual smartpools job, or wait until it runs on its own at 10PM, wait for that to finish.
7. Watch and wait. See how it goes, then start smart-failing if those nodes are getting empty.
These steps don't account for the networking changes on the front-end (one-hint there always remove the node with the lowest Node ID last, to minimize the number of group merges). You have to, of-course make sure you have enough back-end switch ports and cables to get them all connected at the same time, and that the cables are of the right type X210 uses QDR/QSFP and X200 uses CX4/DDR IIRC, etc.
Anyway, hope it helps,
Principal SE Datadobi
Thanks Chris! Once I add the new nodepools (X210 and NL410) to the existing tiers I create (Perf & Archive) and run the smart pools job, will that balance the data across the old and new nodes within their respected tiers? Also, I'm assuming that when I remove the old node pools from the tiers and run the smart pools job, it will drain the old node pools (NL108 & X200). One other question, what would you set the spillover policy to?
In general, you never want to run into a condition where a nodepool completely fills and spillover becomes necessary because it means that you've then lost control over where the data resides. That said, of course leave in on, for those just-in-case scenarios. Will autobalance balance data between node pools in a performance tier? I don't honestly know. Perhaps someone else can answer that question. However making sure that you have file pool policies that cover all data is really what you need to have here. Something like move all data whose mtime is older than 180 days to the archive tier, and then another policy that says, move all data whose mtime is less than 180 days to the performance tier should get the job done, once you've pulled the old node pools out of the performance tier. Using tiers is just an easier way to leave the policies you already have in place, and not have to mess with them, and accomplish the same goal.
Hope that helps,
> Will autobalance balance data between node pools in a performance tier?
Tricky question... Just FWIW -- I wouldn't be surprised to see a two-stage approach here, in analogy to updating the target for newly created files.
SmartPools might not do it right away, because it's objective is not balancing. If it finds a file matching file pool rule for tier X, and that file is in disk pool A within tier X (while disk pool B in same tier X is empty), SmartPools could consider that file as fully satisfying its file pool rule, and therefore it will do nothing.
MultiScan/AutoBalance will find that file in pool A, but might not be aware that the file is supposed to be just somewhere in tier X (disk pools A + B), so it might have no clue about migrating the file from A to B.
Yet it must be possible somehow. My guess is that one should run SmartPools first after creating new tiers, which might update certain bits of information about pools and tiers in the filesystem, but not necessarily balance the pools. With that new information in the filesystem, MultiScan/AutoBalance could make proper decisions about balancing files over pools within the same tier.
The goal of this speculation, of course, is to provoke clarification from the Isilon team.
I think we all want to have an answer to this question. This would probably also apply to another situation where you want to seamlessly migrate your whole cluster to the cloud using CloudPools.
About this situation, where you need to migrate all cluster to a specific pool, you can create a FilePoolPolicy directing everything do that Pool.
But about AutoBalance job balancing data between nodepools, i've had a real situation recently in a customer.
We added new nodes with a different model of the ones that was already in the cluster and a new nodepool was created.
After the execution of AutoBalance job, only some data was migrated to the new nodepool, but the space between nodepools was not balanced. Only some data was migrated to the new nodepool.
So, we had to create some FilePoolPolicies to "force" the movement of some data to the new nodepool.
I was wondering if there are 2 tiers on a cluster. One is hybrid and one is for a specific node pool. Can we run autobalance job on the hybrid tier only so that the other tier is not affected by this data balancing?
Thank you in advance.