This post is more than 5 years old
2 Intern
•
356 Posts
0
2219
Isilon - (Script) - Run "du -hxcd1" against output
Community,
I need to know what script will allow me to run a "du - hxcd1" against the output of "Target Path" column for the command "isi sync target list".
If possible the output should be "Name","Source","Target Path","RESULT OF THE DU TOTAL"
Where you at? Peter Serocka
Thank you in advance!!
Peter_Sero
1.2K Posts
0
August 16th, 2017 08:00
Thanks for the feedback! I think you got the quote marks right by copy-pasting.
Had to add another line to match a newer output format:
isi sync target list -v |
awk -F": " '
{ print $0; }
/^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }
/^ *Target Path:/ { printf " du: "; system("du -hxd1 \""$2"\""); }
'
The script simple adds a line tagged with "du:" after the Target Path line.
The path is repeated within the output from the du command.
sample output:
Name: TEST
Source: isi810
Target Path: /ifs/data/target 1
du: 2.5K /ifs/data/target 1
Last Job State: finished
FOFB State: writes_disabled
Source Cluster GUID: 000c2982a34602941c59a81c747b2bfa5644
Last Source Coordinator IP: 127.0.0.1
Legacy Policy: No
Last Update: 2017-08-16T11:22:16
(Note: the example target path contains a space in "target 1" to show that spaces in paths are handled correctly.
That's what those multiple quote marks are for...)
If even different output formats are found, please post a sample.
Cheers
-- Peter
Peter_Sero
1.2K Posts
0
August 15th, 2017 11:00
Printing pretty tables would required some more lines of code,
but for the list-style of the isi sync output I can offer what is essentially a decorated one-liner:
isi sync target list -v |
awk -F": " '
{ print $0; }
/^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }
'
hth
-- Peter
chjatwork
2 Intern
2 Intern
•
356 Posts
0
August 16th, 2017 03:00
Peter Serocka
Thank you for taking time out to help me with this. Hey, maybe I am doing something wrong cause I am just getting the printout for the:
# isi sync target list -v
Its not providing any additional information in reference to disk usage on each filesystem. I am not sure if its because the script is broken up and I did not assemble it properly in a one liner? I ran it like this:
# isi sync target list -v | awk -F": " ' { print $0; } /^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }'
and I also just cut and paste it in how you had it above:
# isi sync target list -v |
pipe>
pipe> awk -F": " '
pipe quote>
pipe quote> { print $0; }
pipe quote>
pipe quote> /^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }
pipe quote>
pipe quote> '
Both instances I just got the output for "isi sync target list -v"
I will try and brake the script up into smaller pieces to see what I maybe doing wrong, but if you can see let me know.
chjatwork
2 Intern
2 Intern
•
356 Posts
0
August 17th, 2017 03:00
Now if I could find a way to make these scripts run faster
It takes forever when you got TBs of data.
sluetze
300 Posts
0
August 17th, 2017 04:00
The way to make it faster is against your requirement of using du –hxcd1.
Because that’s the process doing the “real” work. The rest of peters code is only formatting.
To be faster you could leverage FSA Jobs. Or generate the “du” metadata on a schedule and only use the data when needed.
Rgds
-- sluetze
chjatwork
2 Intern
2 Intern
•
356 Posts
0
August 17th, 2017 04:00
@sluetze how do I leverage FSA? I just added the cluster to InsightIQ so
it's going to take a serveral weeks before we can get enough data to start
seeing any kind of trend.
Peter_Sero
1.2K Posts
0
August 17th, 2017 04:00
With a SmartQuotas license you can set directory quotas
just for accounting and reporting, i.e. no hard limits if not needed.
Would volunteer to update the script accordingly
FSA reports can be accessed through the platform API,
scripts tend to get lengthy with boilerplate.
-- Peter
sluetze
300 Posts
0
August 17th, 2017 05:00
as peter said, you can access it through P-API. There are some ways to directly access the database but I would not recommend that, since upgrades brick the accessmethod regularly :P
You also won’t see any trends with du -hxcd1.
I suggest you to try to define what you want to achieve – maybe someone already faced this and can tell you a stable way to reach your aim.
Rgds
--sluetze
chjatwork
2 Intern
2 Intern
•
356 Posts
0
August 17th, 2017 09:00
I agree your not going to get trending from running du once, but if a daily du is achievable on a cluster then you can see those report over a period of time and establish a trend.
Peter Serocka
Id totally welcome the script that would create a directory quotas just for accounting and reporting.