8 Krypton

Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

Community,

I need to know what script will allow me to run a "du - hxcd1" against the output of "Target Path" column for the command "isi sync target list".

If possible the output should be "Name","Source","Target Path","RESULT OF THE DU TOTAL"

Where you at?  Peter Serocka

Thank you in advance!!

Labels (2)
0 Kudos
1 Solution

Accepted Solutions
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

Thanks for the feedback! I think you got the quote marks right by copy-pasting.

Had to add another line to match a newer output format:

isi sync target list -v |

awk -F": " '

{ print $0; }

/^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }

/^ *Target Path:/ { printf "                        du: "; system("du -hxd1 \""$2"\""); }

'

The script simple adds a line tagged with "du:" after the Target Path line.

The path is repeated within the output from the du command.

sample output:

                      Name: TEST

                    Source: isi810

               Target Path: /ifs/data/target 1

                        du: 2.5K /ifs/data/target 1

            Last Job State: finished

                FOFB State: writes_disabled

       Source Cluster GUID: 000c2982a34602941c59a81c747b2bfa5644

Last Source Coordinator IP: 127.0.0.1

             Legacy Policy: No

               Last Update: 2017-08-16T11:22:16

(Note: the example target path contains a space in "target 1" to show that spaces in paths are handled correctly.

That's what those multiple quote marks are for...)

If even different output formats are found, please post a sample.

Cheers

-- Peter

0 Kudos
9 Replies
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

Printing pretty tables would required some more lines of code,

but for the list-style of the isi sync output I can offer what is essentially a decorated one-liner:

isi sync target list -v |

awk -F": " '

{ print $0; }

/^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }

'


hth

-- Peter

0 Kudos
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

Peter Serocka

Thank you for taking time out to help me with this.  Hey, maybe I am doing something wrong cause I am just getting the printout for the:

# isi sync target list -v

Its not providing any additional information in reference to disk usage on each filesystem.  I am not sure if its because the script is broken up and I did not assemble it properly in a one liner?  I ran it like this:

# isi sync target list -v | awk -F": " ' { print $0; } /^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }'

and I also just cut and paste it in how you had it above:

# isi sync target list -v |

pipe>

pipe> awk -F": " '

pipe quote>

pipe quote> { print $0; }

pipe quote>

pipe quote> /^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }

pipe quote>

pipe quote> '

Both instances I just got the output for "isi sync target list -v"

I will try and brake the script up into smaller pieces to see what I maybe doing wrong, but if you can see let me know.

0 Kudos
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

Thanks for the feedback! I think you got the quote marks right by copy-pasting.

Had to add another line to match a newer output format:

isi sync target list -v |

awk -F": " '

{ print $0; }

/^Target Dir:/ { printf "du: "; system("du -hxd1 \""$2"\""); }

/^ *Target Path:/ { printf "                        du: "; system("du -hxd1 \""$2"\""); }

'

The script simple adds a line tagged with "du:" after the Target Path line.

The path is repeated within the output from the du command.

sample output:

                      Name: TEST

                    Source: isi810

               Target Path: /ifs/data/target 1

                        du: 2.5K /ifs/data/target 1

            Last Job State: finished

                FOFB State: writes_disabled

       Source Cluster GUID: 000c2982a34602941c59a81c747b2bfa5644

Last Source Coordinator IP: 127.0.0.1

             Legacy Policy: No

               Last Update: 2017-08-16T11:22:16

(Note: the example target path contains a space in "target 1" to show that spaces in paths are handled correctly.

That's what those multiple quote marks are for...)

If even different output formats are found, please post a sample.

Cheers

-- Peter

0 Kudos
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

Now if I could find a way to make these scripts run faster   It takes forever when you got TBs of data.

0 Kudos
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

The way to make it faster is against your requirement of using du –hxcd1.

Because that’s the process doing the “real” work. The rest of peters code is only formatting.

To be faster you could leverage FSA Jobs. Or generate the “du” metadata on a schedule and only use the data when needed.

Rgds

-- sluetze

0 Kudos
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

@sluetze how do I leverage FSA? I just added the cluster to InsightIQ so

it's going to take a serveral weeks before we can get enough data to start

seeing any kind of trend.

0 Kudos
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

With a SmartQuotas license you can set directory quotas

just for accounting and reporting, i.e. no hard limits if not needed.

Would volunteer to update the script accordingly

FSA reports can be accessed through the platform API,

scripts tend to get lengthy with boilerplate.

-- Peter

0 Kudos
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

as peter said, you can access it through P-API. There are some ways to directly access the database but I would not recommend that, since upgrades brick the accessmethod regularly 😛

You also won’t see any trends with du -hxcd1.

I suggest you to try to define what you want to achieve – maybe someone already faced this and can tell you a stable way to reach your aim.

Rgds

--sluetze

0 Kudos
Highlighted
8 Krypton

Re: Isilon - (Script) - Run "du -hxcd1" against output

Jump to solution

I agree your not going to get trending from running du once, but if a daily du is achievable on a cluster then you can see those report over a period of time and establish a trend.

Peter Serocka

Id totally welcome the script that would create a directory quotas just for accounting and reporting.

0 Kudos