What I do is have 2 "chargeback" pools - one for primary storage and one for secondary. Data that is SmartPooled down to secondary storage still gets charged at primary rates. I have SmartPool policies that will migrate data that becomes stale from S to NL, but if the data becomes active again, it will migrate back to S. That means that I have to have some percentage of free space in my S pools for this.
Yes, I can walk the file system and find out where every file lives but there's no way to do this quickly, and especially not real time. Even the FSA data can get stale pretty quickly on busy clusters and not complete a run for a month or two.
Our customers are internal, so "good enough" reporting is acceptable to us.
We don't charge our users for the overhead or the snapshots but obviously factor those into our cost per TB. Some applications use fewer snapshots or larger files and pay more than they should, but others have a high change rate and lots of little files and get a break. It all works out in the end but we've found that it's hard enough to get our customers to forecast disk space without them trying to figure out what the change rate is so we can calculate snapshot overhead. And, of course, this varies over the life of each project...
Peter_Sero
4 Operator
•
1.2K Posts
2
May 20th, 2014 04:00
Couple of thoughts here:
1. Unfortunately, InsightIQ can report the usage by directory and by pool/tier, but not by both at the same time...
2. The SmartPools job report shows usage by "filepool" (= SmartPool rule), but only globally....
3. Try out:
isi filepool apply -r -s
which is probably the closest match to your needs (in 6.5: isi smartpools)
If run shortly after a full SmartPools job, it shouldn't perform too many
actual file migrations, and thus might be reasonable fast.
4. Even faster is a dry run (no migrations):
isi filepool apply -r -s -n
but it only counts the files per filepool/rule, and doesn't report the consumed space.
5.
isi get -D
reveals the pool where a single files resides, and never
changes its placement or layout, nevertheless it is usually
too slow to be applied on very large file sets.
hth
-- Peter
Anonymous User
170 Posts
1
May 21st, 2014 06:00
I haven't found a good way to do this.
What I do is have 2 "chargeback" pools - one for primary storage and one for secondary. Data that is SmartPooled down to secondary storage still gets charged at primary rates. I have SmartPool policies that will migrate data that becomes stale from S to NL, but if the data becomes active again, it will migrate back to S. That means that I have to have some percentage of free space in my S pools for this.
Yes, I can walk the file system and find out where every file lives but there's no way to do this quickly, and especially not real time. Even the FSA data can get stale pretty quickly on busy clusters and not complete a run for a month or two.
Our customers are internal, so "good enough" reporting is acceptable to us.
Anonymous User
170 Posts
1
May 21st, 2014 09:00
We don't charge our users for the overhead or the snapshots but obviously factor those into our cost per TB. Some applications use fewer snapshots or larger files and pay more than they should, but others have a high change rate and lots of little files and get a break. It all works out in the end but we've found that it's hard enough to get our customers to forecast disk space without them trying to figure out what the change rate is so we can calculate snapshot overhead. And, of course, this varies over the life of each project...
Peter_Sero
4 Operator
•
1.2K Posts
0
May 21st, 2014 09:00
Do you include protection overhead, and snapshots?
We decided to stick to regular ("apparent") file sizes,
but it is not really fair...
All relevant directories here are either on X or on NL pool,
that makes it an easy call for us.