PowerScale: Using 'dd' on the node CLI to troubleshoot the source of latency.
Summary: You can use the 'dd' command on a PowerScale node CLI to help determine the source of the bottleneck for a particular performance issue. For example, if you can replicate the latency directly on a node command line using 'dd' then this indicates an issue at the file system layer vs. the protocol (NFS or SMB) or networking layer. ...
Instructions
Here are tips on using the 'dd' command on a PowerScale node:
Being that our on-disk block size is 128k in OneFS, Dell Support recommends using the 'bs=128k' option (bs = block size) when using the 'dd' utility. This should be sending data to disk when 'dd' has enough to fill a full block.
Write speed test: 1GB file
# dd if=/dev/zero of=/ifs/data/Isilon_Support/1GB.out bs=128k count=7812
Write speed test: 10GB file
dd if=/dev/zero of=/ifs/data/Isilon_Support/10GB.out bs=128k count=78120
Read Speed Test:
For a read speed test, the 'cp' (copy) command is sufficient for testing:
# time cp /ifs/data/Isilon_Support/kevin1GB.file /ifs/data/Isilon_Support/cptestfile
Optionally, you can use the 'dd' utility as follows:
# dd if=/1GB.file of=/1GBoutput.file bs=128k # dd if=<actual filename you are reading> of=/ifs/data/Isilon_Support/file.out bs=128k
Here are some typical baselines, as tested in our lab:
Hardware: F800-4U-Single-256GB-1x1GE-2x40GE SFP+-24TB SSD
WRITE
1G Isilon-5# dd if=/dev/zero of=/ifs/data/Isilon_Support/1GB.out bs=128k count=7812 7812+0 records in 7812+0 records out 1023934464 bytes transferred in 1.037575 secs (986853056 bytes/sec) ---> 1GB/sec 10G Isilon-5# dd if=/dev/zero of=/ifs/data/Isilon_Support/10GB.out bs=128k count=78120 78120+0 records in 78120+0 records out 10239344640 bytes transferred in 9.491726 secs (1078765328 bytes/sec) ----> 1GB/sec
READ
1G Isilon-5# time cp /ifs/1GB.out /ifs/data/Isilon_Support/kevintest22 cp -i /ifs/1GB.out /ifs/data/Isilon_Support/kevintest22 0.01s user 1.57s system 84% cpu 1.859 total 10G Isilon-5# time cp /ifs/10GB.out /ifs/data/Isilon_Support/kevintest24 cp -i /ifs/10GB.out /ifs/data/Isilon_Support/kevintest24 0.02s user 15.92s system 85% cpu 18.604 total
Parameter Definitions:
of = name of output file. Confirmed has *nothing* to do with size.
bs = block size 1024 = 1Gb 10024 = 10Gb
Count = copies only this number of blocks. (default is for dd to keep going forever)
if = read from FILE instead of stdin
To summarize, you end up with a file of size = 'bs x count'.