Isilon script: Which pool contains my data.

Couple of thoughts / questions:

Scanning the file system and inquiring the pool for each file (per isi get)

would be practical only if there are not too many files (say up to few millions

but not tens of millions).

Then, would you like to see each file in the spreadsheet?

Or aggregated by directories at a certain depth of the tree?

Still you would need to re-create the actual filepool

rules from the cluster for evaluating the data and to make a prediction.

Each successful SmartPools job gives you a report

with an exact file count per file pool policy. I'd first

look at these numbers -- they come for free -- to see

wether some useful trending over time can be derived.

The more often SmartPools jobs are run, the better of course.

Unfortunately the file capacities are not reported.

If one can "afford" to create more policies,

one can consider adding ranges of file size to the rules.

This will make "trending"  somewhat more quantitative.

just fwiw -- but makes sense?

-- Peter

Don't disagree, but let's remove the "file" level reporting question.  Can we do it based on "directory" so I could at least see every directory/folder/subfolder and which pool it is located?  From that I can run IIQ reports on each directory as to the capacity and from there I could set up SmartPool policies to put the data where I want to it to reside.  That's the end goal, to put the data where it needs to be because SmartPools doesn't provide the full functionality I need.

As for SmartPools itself, it needs to be smarter than "I've hit 97% capacity so I'm going to spill over to other pools" instead of saying "Ok, lets stop writing to this pool since it's at "X" capacity and handle the data between the other two pools until pool 3 is below X capacity".  The results of letting SmartPool railroad a specific pool, as you know, has a severe performance impact that in production is not tolerable.  Too much hand-holding with SmartPools as it is, I need it to actually be "smart"

1. Absolutely. Your concerns around a lack of SmartPools directed spillover threshold feature are definitely valid.

Pretty sure we have a product requirement for this due in the next couple of releases timeframe.

Scott G – care to add any color or context to this?

> Can we do it based on "directory" so I could at least see every directory/folder/subfolder and which pool it is located?

Keep in mind that with timestamp-based policies, each subfolder can contain

files in different pools. That makes it somewhat complex and I assume rather

than a having a generic script one will end up with a custom tool that

specifically reflects the rationale of the actual filepool policies in a given environment.

My suggestion to look at the job reports first comes from the fact

that these reports are already there with no additional efforts or overhead...

also @trimbn and @gentrs1:

"Smart" would be not just to migrate files until some threshold is reached,

but to migrate files in order of (whatever chosen) timestamp, from oldest

to newest, or even "coldest" to "hottest", up to a threshold or balance criterion.

definitely staying tuned...

-- Peter