VNX7500 NFS performance issue

the 2 NFS are not the only NFS exports we have. We have 50 NFS exports in total but not hear complains yet from other users.

I grant the access to a new box and here is the file inodes.

[root@lx0034nbumed01 test]# df -h –t nfs
Filesystem            Size  Used Avail Use% Mounted on

10.119.10.122:/BusSol_EntEve_theme 50G  1.2G   49G   3% /test2

10.119.10.122:/BusSol_EntEve_webdav 493G  177G  316G  36% /test4

[root@lx0034nbumed01 test2]# df -i /test*
Filesystem                  Inodes    IUsed    IFree  IUse% Mounted on
10.119.10.122:/BusSol_EntEve_theme 6297598  13759  62838391%    1%   /test2  
10.119.10.122:/BusSol_EntEve_webdav 62975998 161952  62814046    1%  /test4

I aslo make dd tests on BusSol_EntEve_theme, producing a 200M files....and the NIC of my server is 1000Mb/s
[root@lx0034nbumed01 test]# time dd if=/dev/zero of=/test2/test/200M_wsize32k_dd1k  bs=1k count=204800         ============>270.473 seconds, 775 kB/s
[root@lx0034nbumed01 test]# time dd if=/dev/zero of=/test2/test/200M_wsize32k_dd2k  bs=2k count=102400         ============>158.294 seconds, 1.3 MB/s
[root@lx0034nbumed01 test]# time dd if=/dev/zero of=/test2/test/200M_wsize32k_dd4k  bs=4k count=51200         ============>90.9329 seconds, 2.3 MB/s
[root@lx0034nbumed01 test]#time dd if=/dev/zero of=/test2/test/200M_wsize32k_dd8k  bs=8k count=25600         ============>80.4733 seconds, 2.6 MB/s
[root@lx0034nbumed01 test]# time dd if=/dev/zero of=/test2/test/200M_wsize32k_dd16k  bs=16k count=12800  ======= =====>78.2397 seconds, 2.7 MB/s
[root@lx0034nbumed01 test]# time dd if=/dev/zero of=/test2/test/200M_wsize32k_dd32k  bs=32k count=6400    ======= =====>78.7956 seconds, 2.7 MB/s

test with a new file system without any snapshots

but I don;t think we set snapshots for the fielsystem...........

sorry to disturbbut the issue is fixed at last.

the user used the improper NFS mount options. if we use the default NFS mount options the write speed(dd) can be enhanced dramatically..................thanks everybody for your response

Thanks for the feedback

Which option was it ?

Rsize/wsize too small?

Hey,

no ,the rsize/wsize are okay and we use the dafult value 32768. The improper option is "noac" .

this is user's NFS mount options and I asked him mount again without "noac" and the speed enhanced from <3Mb/s to around 35Mb/s.

mount storage_IP:/BusSol_EntEve/BusSol_EntEve   /webdav nfs hard,bg,tcp,vers=3,rsize=32768,wsize=32768,timeo=14,intr,noac,nolock

Here is the "noac" options from NFS man pages

noac

To improve performance, NFS clients cache file attributes. Every few seconds, an NFS client checks the server's version of each file's attributes for updates. Changes that occur on the server in those small intervals remain undetected until the client checks the server again. The noac option prevents clients from caching file attributes so that applications can more quickly detect file changes on the server.

In addition to preventing the client from caching file attributes, the noac option forces application writes to become synchronous so that local changes to a file become visible on the server immediately. That way, other clients can quickly detect recent writes when they check the file's attributes.

Using the noac option provides greater cache coherence among NFS clients accessing the same files, but it extracts a significant performance penalty. As such, judicious use of file locking is encouraged instead. The DATA AND METADATA COHERENCE section contains a detailed discussion of these trade-offs.

you want to use noac if you are running Oracle RAC where multiple nodes use the same NFS exports as you don't want any data to be cached, you want it to be consistent and written to disk immediately.