Isilon: Error during Oracle Data Pump operation "dNFS asynchronous I/O failure"

Summary: Datapump/backup operations between an Oracle client and Isilon performed over a WAN may break with error "dNFS asynchronous I/O failure" while using Smartconnect zone name but works using IP. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

This is as per design.

An Oracle client is performing a Data Pump/Backup operation to an Isilon over a WAN using a Smartconnect zone name.
The operation breaks with the following error:
 
ORA-17516: dNFS asynchronous I/O failure
 
However the same operation works over an IP address.

Cause

This can be caused by two things:
  1. The dNFS functionality of fail over and single-client load balancing.
dNFS or Direct NFS has a functionality of fail over and single-client load balancing and is used for write intensive jobs like Datapump or backup operations. Due to this feature when you take a look at the packet traces you will notice the client will be communicating with multiple nodes over the course of the operation.

While Isilon can handle failover without the need for dNFS, it does not offer single client load balancing; rather, Isilon performs balancing across the entire client base by rotating the IP address that it gives each client when queried. 

So over the WAN each time the Oracle client has to evoke the Smarconnect service to connect to a different node, there will be a delay in getting the Next IP address, during which the Oracle client will presume the NFS server is not reachable and will break the ongoing operation stating "dNFS asynchronous I/O failure".

Oracle's statement for the error is as below which validates our findings :
 
Error code: ORA-17516
Description: dNFS asynchronous I/O failure
Cause: The asynchronous I/O request failed due to storage server reboot.
 
  1. Adaptive Security Appliance [ASA] firewall present between the two devices is discarding incoming TCP packets on port 2049 :
There might be a firewall present between the two devices with a setting to discard TCP packets over the port 2049.The ASA is always expecting the first packet of the TCP connection to be the TCP SYN from the host that tries to open/form the TCP connection. If some other TCP packets are incoming like a "TCP RST ACK" it presumes that this is packets is for an existing connection. It then checks its connection table but fails to find an existing connection and therefore drops the traffic.

You will find a similar log message, when reviewing the logs of ASA device :
 
%ASA-6-106015: Deny TCP (no connection) from x.x.x.x/x to x.x.x.x/x flags RST on interface someinterface

Resolution

Oracle has a separate procedure to configure dNFS.
  1. Check the mount entries from the oranfstab file
  2. Where the different IP addresses of the NFS server are specified
  3. Use one of the IP addresses present in this configuration file to mount the NFS export
  4. This will rule out the use of Smartconnect zone name and instead use only the IP address to contact to the server.
  5. The multiple IP Addresses in the oranfstab file will take care of Load Balancing

Engage Oracle support to get dNFS configured. In case there is a firewall present between the two devices, on the ASA device, you will need to allow incoming TCP traffic on port 2049.

Additional Information

Useful links for dNFS configuration from Oracle :

Also refer

Affected Products

Isilon

Products

Isilon
Article Properties
Article Number: 000056260
Article Type: Solution
Last Modified: 02 Apr 2025
Version:  3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.