OneFS: Best practices for Network File system (NFS) client settings

Supported Protocol Versions

PowerScale OneFS currently supports Network File System (NFS) versions 3 and 4. NFS version 2 is not supported.

NFSv3

NFS version 3 is the most widely used version of the NFS protocol today, and is considered to have the widest client and filer adoption. Here are some key components of this version:

• Stateless - A client does not technically establish a new session if it has the correct information to ask for files and so on. This allows for simple failover between OneFS nodes using dynamic IP pools.
• User and Group info is presented numerically - Client and Server communicate user information by numeric identifiers, allowing the same user to appear as different names between client and server.
• File Locking is out of band - Version 3 of NFS uses a helper protocol called NLM to perform locks. This requires the client to respond to RPC messages from the server to confirm locks have been granted.
• Can run over TCP or UDP - This version of the protocol can run over UDP instead of TCP, leaving handling of loss and retransmission to the software instead of the operating system. Dell Technologies always recommend using TCP.

NFSv4

NFS version 4 is the newest major revision of the NFS protocol, and is increasing in adoption.  Here are some of the key differences between v3 and v4.

• Stateful - NFSv4 uses sessions in order to handle communication, as such both client and server must track session state to continue communicating.
  • Prior to OneFS 8.X this meant that NFSv4 clients required static IP pools on the PowerScale or could encounter issues.
• User and Group info is presented as strings - Both the client and server must resolve the names of the numeric information stored. The server must lookup names to present, while the client must remap those to numbers on it s end.
• File Locking is in band - Version 4 no longer uses a separate protocol for file locking, instead making it a type of call that is compounded with OPENs, CREATES, or WRITES.
• Compound Calls - Version 4 can bundle a series of calls in a single packet, allowing the server to process all of them and reply at the end. This is used to reduce the number of calls involved in common operations.
• Only supports TCP - Version 4 of NFS has left loss and retransmission up to the underlying operating system.

NFSv4.1 and Beyond

NFSv4.1 and v4.2 are available starting in OneFS version 9.3.

Here is the official release information for 9.3:

PowerScale OneFS Info Hubs
 

Mount Options

While Dell Technologies does not have hard requirements for mount options, Dell Technologies make some recommendations on how clients connect. Dell Technologies has not provided specific mount strings, as the syntax used to define these options varies depending on the operating system in use. you must follow distribution maintainers' documentation for specific mount syntax.

PowerScale Support also recommends the following white paper as a primary reference for NFS client configuration with PowerScale, including recommended options for wsize / rize, attribute caching, and more:

PowerScale OneFS NFS Design Considerations and Best Practices
https://infohub.delltechnologies.com/en-us/t/powerscale-onefs-nfs-design-considerations-and-best-practices-3/

Read and Write Size (rsize / wsize)

Regarding the 'wsize/rsize options', PowerScale support recommends a 'wsize' and 'rsize' of at least 128K, which is based on our native block size.

However, for most modern distros of Linux, PowerScale Support actually recommends to not explicitly configure a setting (that is, do not specify a read/write size in the client mount options), and let the client renegotiate the tunings. Modern Linux distributions support NFS read/write block sizes up to 1 MB, and will automatically negotiate the optimal block size with the PowerScale NFS server. The negotiated values are ideal for most properly configured high-performance, low-latency networks. The exception would be unless you have an application or vendor which specifically requires the smaller size.

When not explicitly set, your NFS client uses the PowerScale NFS server FSINFO data, as defined in the NFS export configured on your PowerScale cluster.

The defaults which PowerScale offers are as follows:

NFSv3: 512KB writes / 1MB reads
NFSv4: 1MB writes/ 1MB reads

See pages 12 and 19 in the below white paper for more detailed information regarding 'rsize' and 'wsize':

PowerScale OneFS NFS Design Considerations and Best Practices
https://infohub.delltechnologies.com/en-us/t/powerscale-onefs-nfs-design-considerations-and-best-practices-3/ 

Defining Retries and Timeouts

While the PowerScale generally replies to client communication quickly, during instances when a node has lost power or network connectivity, it might take a few seconds for it s IP addresses to move to a functional node, as such it is important to have correctly defined timeout and retry values. PowerScale generally recommends a timeout of 60 seconds to account for a worst-case failover scenario, set to retry two times before reporting a failure.

Soft vs Hard Mounts

Hard mounts cause the client to retry it s operations indefinitely on timeout or error. This ensures that the client does not disconnect the mount in circumstances where the PowerScale cluster moves IP addresses from one node to another. A soft mount will error out and expire the mount requiring a remount to restore access after the IP address moves.

Allowing Interrupt

By default, most clients do not allow you to interrupt an input/output or I/O wait, meaning you cannot use ctrl+c to end the waiting process if the cluster is stopping responding, including the interrupt mount option allows those signals to pass normally instead.

Local vs. Remote Locking

When mounting an NFS export, you can specify whether a client generates its locks locally, or using the lock coordinator on the cluster. Most clients default to remote locking, and this is generally the best option when multiple clients are accessing the same directory, however there can be performance benefits to performing local locking when a client does not must share access to the directory it is working with. In addition, some databases and software will request that you use local locking, as they have their own coordinator.

Attribute Caching (ac/noac)

Regarding "active cache timeouts," this is considered client-side behavior. As a result, PowerScale Support does not make recommendations on these settings as it depends on what your needs are. However, customers can find some general guidance on these settings on page 22 of the below white paper:

PowerScale OneFS NFS Design Considerations and Best Practices
https://infohub.delltechnologies.com/en-us/t/powerscale-onefs-nfs-design-considerations-and-best-practices-3/ 

Per page 22 from the above:

Attribute caching (ac/noac)

Use the noac mount option to achieve attribute cache coherence among multiple clients. Almost every file system operation checks file attribute information. The client keeps this information cached for a period to reduce network and server load. When noac is in effect, a client's file attribute cache is disabled, so each operation that must check a file's attributes is forced to go back to the server. Besides, the noac option forces application writes to become synchronous so that a client sees changes to a file upon opening, at the cost of many extra network operations. By default, the attribute caching is enabled when mounting the NFS. Enable the attribute caching to improve the attribute checking performance and reduce the NFS operation latency.

Performance of NFSv3 vs. NFSv4

Based on lab testing, PowerScale Support has not found any discernable performance differences between different versions of NFS in the latest, supported versions of OneFS. 

To see the wsize/rsize values of a particular NFS export, you can run the following commands on any PowerScale node:

# isi nfs exports ls -v  --zone <zone name>

 Or, to check for a specific export id, customers can run the following:

# isi nfs export view <export id>

 Example:

Read Transfer Max Size: 1.00M
     Read Transfer Size: 128.00k
Write Transfer Max Size: 1.00M
 Write Transfer Size: 512.00k