ECS: NFS Write Throws an I/O Error after a Certain Amount of Data

root@client~:/mnt_ecs# dd if=/dev/zero of=/mnt_ecs/test bs=1024 count=80000
dd: write error: I/O error
dd: write error: I/O error
root@client:/mnt_ecs# ll
total 8222
drwx---rwx- 3 2147483647 2147483647 96 Aug 13 10:51 .
drwxr-xr-x- 30 root system 4096 Aug 07 10:46 ..
-rw-r--r--- 1 root system 2102272 Aug 13 10:51 test

# cat /opt/emc/caspian/fabric/agent/services/object/main/log/dataheadsvc.log

admin@node1~>cat /opt/emc/caspian/fabric/agent/services/object/main/log/dataheadsvc.log
2018-08-17T06:09:02,499 [RpcDispatcherWriteIO-5411370]  INFO  RpcProgramNfs3.java (line 241) No sync response, expect an async response for request  WriteRequest : FileHandle:[objectId = e7e6d9327488f5a316420e1f8805fac18d8a4f0773173ff12de2512a384d25de, ownerZoneUUID = 4f7a7b18-4e08-4e25-9bbb-ebeb02b085c3]Export:2 offset:0 count:524288 stablehowUNSTABLE
2018-08-17T06:09:03,261 [RpcDispatcherMD-5323256] ERROR  RpcProgramNfs3.java (line 277) Error processing NfsRequest: xid:1229074411 ip:/<Client IP> SysSecurity:  AuthStatus:AUTH_OK Uid:0 Gid:0 Proc:SETATTR SetAttrRequest : FileHandle:[objectId = e7e6d9327488f5a316420e1f8805fac18d8a4f0773173ff12de2512a384d25de, ownerZoneUUID = 4f7a7b18-4e08-4e25-9bbb-ebeb02b085c3]Export:2 SetAttr3Object : uid:0 gid:0 mode:0 atime:null mtime:null verf:10487808 updateFields:[SIZE] check:true ctime(NfsTime-1534486142s, 467000000ns) NfsResponse:  SetAttr3Response : status:NFS3ERR_NOT_SYNC wccData: WccData: WccAttr: size:10485760 mtime:(NfsTime-1534486142s, 600000000ns) ctime:(NfsTime-1534486143s, 243000000ns) postOpAttr type:NFSREG mode:rw-r--r-- nlink:1 uid:0 gid:0 size:10485760 used:10485760 rdev:(Specdata3: specdata10, specdata2:0) fileid:1019241384186633961 atime:(NfsTime-1534486143s, 243000000ns) mtime:(NfsTime-1534486142s, 600000000ns) ctime:(NfsTime-1534486143s, 243000000ns) RpcError: null

1. Either the client mounted the file system-enabled bucket with the sync option.

# mount -V nfs -o vers=3,proto=tcp,port=2049,sync <ECS IP> :/ns_nfs/nfs /mnt_ecs

2. The Export Host Option in the ECS UI File Menu has been changed to Sync (default is Async)

Run the following checks:

• Check if the client specifies the Sync Mount option, and if so, remove it.
• Check the Write Transfer Policy in the Export Host options (File > Exports), and change it to Async.

NFS v3 allows for safe asynchronous writes which increases performance over synchronous writes that were often a problem in earlier NFS implementations.
 

Note: Increasing the wsize in the mount option allows for more data to be transferred before the I/O error shows up.

From the RFC 1813 NFS Version 3 Protocol :

NFS3ERR_NOT_SYNC:
Update synchronization mismatch was detected during a SETATTR operation.