Unsolved
This post is more than 5 years old
4 Posts
1
12824
July 12th, 2017 00:00
isi statistics command document
Hello,
For get performance data on OneFS v7.2,
I have run "isi statistics system --timestamp --nodes -i 5" and "isi statistics client --long -i 5" command.
To understand data, I have referenced "man isi-statistics" manual on OneFS v7.2.
But "man isi-statistics" manual describes each column simply.
I'd like to know the meaning of the information on each output column.
For example, when I run the following command,
# isi statistics system --timestamp --nodes -i 5
the following output displayed as result.
Timestamp Node CPU SMB FTP HTTP ISCSI NFS HDFS Total NetIn NetOut DiskIn DiskOut
s LNN %Used B/s B/s B/s B/s B/s B/s B/s B/s B/s B/s B/s
1495568686.9 All 24.3 0.0 0.0 0.0 0.0 129703383.5 0.0 129703383.5 4573231.4 35919198.2 0.0 13632512.0
1495568686.9 1 18.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1899.0 28.6 0.0 1431859.2
1495568686.9 2 21.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 699.8 28.6 0.0 669593.6
1495568686.9 3 70.5 0.0 0.0 0.0 0.0 129703383.5 0.0 129703383.5 4332457.4 36613933.2 0.0 1072742.4
1495568686.9 4 21.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 878.2 28.6 0.0 1144490.7
1495568686.9 5 9.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1672.6 28.6 0.0 1252352.0
1495568686.9 6 21.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1234.8 1797.6 0.0 1229619.2
1495568686.9 7 17.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1873.2 22.0 0.0 1307750.4
1495568686.9 8 19.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 872.4 22.0 0.0 1513062.4
1495568686.9 9 20.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1380.7 0.0 0.0 1666150.4
1495568686.9 10 20.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 782.2 28.6 0.0 1663692.8
I'm searching any documents on Dell EMC support page.
But I can find only "man isi-statistics" command help.
I'm looking for the document which explains the output of each column more detail than "man isi-statistics" command help.
Is there such document ?
sjones51
252 Posts
2
July 12th, 2017 11:00
Hi k-ryu,
There is not much in the way of complete documentation. There are a couple of articles out there to assist but none that speak specifically to the command that you are using:
382740 : OneFS:Using the isi statistics command https://support.emc.com/kb/382740
459545 : How to Understand Isi Statistics Drive Output https://support.emc.com/kb/459545
459506 : How to Read and Understand the Output of Isi Statistics Protocol Command https://support.emc.com/kb/459506
Peter_Sero
2 Intern
2 Intern
•
1.2K Posts
1
July 12th, 2017 14:00
The throughputs are gathered from three distinct areas of the system,
and the example helps illustrating how Isilon OneFS works overall:
1. Protocols:
SMB, FTP, HTTP (also RAN=Restful Access to Namespace), ISCSI (deprectaed), NFS, HDFS (Hadoop)
Each colum is the combined read + write throughput.
"Total" is the computed sum of all protocols.
2. Network:
Only the external = client-facing network throughputs are shown.
NetIn = write to Isilon cluster, NetOut = read from cluster.
The sum NetIn+NetOut "could" match the protocols Total, but doesn't always (as in your example):
- network throughput is sampled differently -- things get clearer with more stable client workloads
- network throughput will also include other activities like SyncIQ or NDMP backups and other "noise"
3. Disk media:
DiskIn = write, DiskOut = read
On cluster level (the "All" row), the sum DiskIn + DiskOut "could" match protocols Total,
and in fact it roughly does in your example.
Looking at the individual nodes in the example, one can nicely see how all nodes' disk contribute
almost evenly to the only NFS activity on single node 3. That's an essential aspect of OneFS.
In other situations, looking at the entire cluster:
- disk throughput can be HIGHER than protocols throughput: e.g. with internal restripe jobs
- disk throughput can be LOWER than protocols throughput: e.g. with good cache hit rates
Also note that node 3 has a much higher CPU load than the rest,
due to the active NFS workload.
Makes sense? Too deep already? or left out other questions?
-- Peter
k-ryu
4 Posts
0
July 13th, 2017 01:00
Hello sjones5,
Thanks for reply.
I check the KB information you told.
k-ryu
4 Posts
0
July 13th, 2017 04:00
Hello Peter,
Thank you for telling me deep information the man isi-statistics command didn't show.
I could deepen my understanding about the "isi statistics system" command data.
So, Could you tell me about "isi statistics client" command data ?
When I run the following command,
# isi statistics client --long -i 5
the following output displayed as result.
In this data, is there deep information which "man isi-statistics" does not show ?
Peter_Sero
2 Intern
2 Intern
•
1.2K Posts
0
July 19th, 2017 09:00
Quick note, in the second row, all units starting from N/s should be shifted one column to the right,
NumOps has no unit, it's just the count of operations in the 5-second sampling interval,
while Ops is the per-seconds operations rate -- both columns have equivalent information.
Naming and order of the In/Out columns are confusing at best...
InMax, InMin and InAvg refer to request block sizes in Bytes,
whereas "In" shows the input data rates in Bytes/s.
Same for OutMax, OutMin, OutAvg and Out.
TimeMax, TimeMin and TimeAvg show the processing times or "latencies".
Starting from OneFS 7.2, the reported NFS protocol latencies can be impacted
by slow clients which makes performance analysis a bit more difficult than before.
The operation "Class" should be interpreted in the context of each Protocol.
Read and write classes are clear, but e.g. NFS namespace_read includes
NFS lookups as well as getattrs. The "isi statistics protocol" command
can break things further down to operations level (leaving out "Users" in exchange.)
UserName is the result of the cluster doing a user lookup for the
given numerical UserID -- in your example this was not possible for UID 2301.
LocalAdress and RemoteAddress refer to the used IP addresses on the Isilon Node
and on the client, respectively. The cluster tries to make DNS lookups for you
to produce LocalName and RemoteName.
General tips:
- select columns to show with the --output option
- get overviews by aggregating rows with the --totalby option,
e.g. --totalby UserID or --totalby Node
cool tip: --totalby accepts multipe fields like: --totalby UserID,Node
Happy monitoring; let me know if there are more questions!
-- Peter
k-ryu
4 Posts
0
July 21st, 2017 01:00
Hello Peter,
Thank you for reply.
If I want to get NFS network data transfer performance,
should I run "isi statistics protocol --protocols nfs3 --long" command
rather than "isi statistics system/client" command ?
And, does "isi statistics protocol --protocols nfs3 --long" command output data not include
other data (e.g. SyncIQ, NDMP backup and so on) ?
Peter_Sero
2 Intern
2 Intern
•
1.2K Posts
0
July 24th, 2017 03:00
The various statistics sub-commands offer different break-outs,
so you can choose any of them, according to your needs.
For just combined In+Out throughput by node, "isi statistics system" is fine,
for troughput and activity by users and/or clients, "isi statistics client ..." is the only way to go,
while insights down to protocol operations level are only provided by "isi statistics protcol ...".
In either case, reported NFS throughput never contains other traffic, such as SyncIQ or NDMP etc.
Cheers
-- Peter