castleknock ,

You can do quite a bit with the filtering in 'isi statistics'.

Here is an example:

That syntax if you want to copy/paste play with it instead of trying to retype my screenshot is:

'isi statistics client -nall --remote-addresses 192.168.11.66 --output NumOps,Ops,InMax,InMin,In,InAvg,OutMax,OutMin,Out,OutAvg,TimeMax,TimeMin,TimeAvg,Node,Proto,Class,UserID,UserName,LocalAddr,RemoteAddr -r 10 -i 10 -z'

You can also change the format to get it into a table(default), a csv, json, etc.

Hope it helps, that's just a start there are thousands of ways to manipulate those command combinations to see different things.  You can also use InsightIQ if you like GUIs, but of course it's not as flexible.

~Chris Klosterman

chris.klosterman@datadobi.com

And there's also

isi statistics drive

for disk IO activity, throughput, latency, queue length -- from the

perspective of the individual drives, but unrelated to specific NFS or file system operations.

You know you have lost when disk latencies go high (>10 ms beware,  >100 ms stuff hits the fan),

but with mixed workloads it will be hard to catch the thief.

Let's face it, a drill down for an individual NFS operation is not provided for customers.

castleknock, if you are lucky and can reproduce the issue with an isolated workload

under "lab conditions" (when the cluster is idle otherwise), the available tools

might provide some good clues -- recommended docs:

Troubleshoot Performance Issues on your Isilon Cluster

Measuring Cluster Performance

White Paper: Isilon OneFS Cluster Performance Metrics Tips and Tricks

hth

-- Peter

And of course feel free to change any of the variables that I used to fit your own needs. Different intervals, more of them, different format, etc.

~Chris

Tks,

Have run the capture, now comes the fun part ! making sense of the figures

Thanks for answer, looked at performance troubleshooting guide.