NFS Tuning

Question

Hello All,

We are in the process of building a new Isilon cluster. In the system that the clients are using they are running some code that reads and writes to nfs mounts from linux clients. The code seems to d a lot of what we call small block I/O. This causes a lot of performance problems for NFS.

So I am looking for any tips on tuning to help this when we put the Isilon system in production. I notice it defaults to 8K blocksize. I wonder if increasing to a larger number would help?

crklosterman · Answer

Isilon's OneFS Filesystem uses 8KB Blocks and 128KB Stripes, which is not modifiable, that said the following KB should help.

90041 : OneFS: Best practices for NFS client settings
https://support.emc.com/kb/90041

Thanks,

Chris Klosterman

Senior SA

EMC Isilon Offer & Enablement Team

twitter: @croaking

chris.klosterman@emc.com

peglarr · Answer

The document that Chris refers to is a good start. However, there are some additional tuning items that are useful today (the doc is somewhat dated) especially with the 7.1.1 release. I would suggest working closely with your Isilon SE to determine optimal settings for your hardware/OS combination on your client(s).

In my experience, even today the client OSes are ill-equipped OOTB to provide optimal performance to/from an Isilon cluster. They are designed to provide 'lowest-common-denominator' usage since it is unknown a priori what kind of LAN/WAN connectivity they will encounter. Client-side tuning is a must in nearly every environment, and I would suggest absolutely mandatory for running any 10GE environment optimally with Isilon. The good news there is very little (if anything) to tune in OneFS, since running LAN traffic optimally is what we try to do in every case, unlike a general-purpose client OS. The bad news is that in order to reach optimal ops, throughput and latency on the client side, tuning is required.

HTH

Rob

Peter_Sero · Answer

Flat Lander

in addition to network tuning, you can also experiment with

various options for file layout and access patterns: From what you wrote

it appears worthwhile to try "random access" instead

of the default "concurrent access", and mirroring (e.g. 3x) instead of striping.

https://support.emc.com/docu38187_White-Paper:-Next-Generation-Storage-Tiering-With-EMC-Isilon-SmartPools.pdf?language=e…

-- Peter

FlatLander2 · Answer

Thank All,

Chris, I would like to take a look at that document you provided a link to, but I keep getting an error regarding a bad certificate from salesforce single sign-on. I was able to get the document Peter linked to. I will start trying to find some client side tuning recommendations for 10G networking. Fortunately the clients that are going to be on the 10G network are pretty robust in terms of ram and processor and Intel 10G network adapters.

Phil.Lam · Answer

Here's the pdf.

Isilon

NFS Tuning

Was this post helpful?