Data Domain: Maximum Number of Files per Directory Limits and Performance Impact on DDFS

Summary: Data Domain Restorers (DDRs) run a proprietary file system known as the Data Domain File System (DDFS). Due to how DDFS stores data and corresponding metadata, there are limits on the maximum number of files that can be stored in a single directory. Exceeding these limits can cause degraded performance, timeouts, and system instability. This article explains the directory file limits, the technical reason for the limits, and the risks when limits are not adhered to. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

When the maximum number of files in a single directory is exceeded, the DDR may experience:

  • Degraded performance for regular DDFS operations including backups, restores, and replication.

  • DDFS timeouts causing interruption to regular operations.

  • DDFS panics and restarts resulting in service interruptions.

  • NFS timeout errors in the DDR logs similar to the following:

    06/04 12:35:42.994 (tid 0x2c7a230): ERROR: Async rpc procedure (NFSPROC3_EXTLOOKUP) timed out.
06/04 12:45:27.796 (tid 0x2c7a230): ERROR: Async rpc procedure (NFSPROC3_GETATTR) timed out.

Cause

DDFS stores files in a directory tree hierarchy similar to other file system implementations. Each directory has associated metadata referred to as the directory file. The directory file contains information such as:

  • Names of files and sub-directories in the directory
  • Locations of files and sub-directories in the directory

As files and sub-directories are created, renamed, or deleted, the directory file must be updated to reflect the changes. The directory file can become very large and will constantly be in a state of flux if the directory:

  • Contains a very large number of files.
  • Experiences a large amount of file churn (frequent creates, renames, and deletes).

A number of standard DDFS operations need to read the directory file to discover the contents of a directory. If the directory file is large, these operations take an extended duration to complete, which affects system performance and stability.

Resolution

A number of standard DDFS operations need to read the directory-file to discover the contents of a directory. If the directory-file is large, these operations can take an extended duration to complete which can affect system performance and stability.

As a result of this there are various recommendations for the maximum number of files in a directory as shown below:

  • Maximum 250 thousand files in any single directory
  • Maximum 250 million files in any single MTree
  • Maximum 1 billion (1,000,000,000) files total in any DD filesystem 

 

Note: The above recommendations apply only to the contents of a single directory, and not to an entire directory tree (directory, plus all its sub-directories).  For example, it is ok to have an MTree with 10 top-level directories, with each  top-level directory having 10 sub-directories with 50000 files (so, 500K files in each top-level directory tree, and 5M files in the MTree).  Each directory-file only corresponds to a single directory (and not its sub-directories), and spreading files among multiple sub-directories in this manner will not cause performance degradation.

 

Additional Information





Products

Data Domain, Data Domain Boost – File System, Data Domain Deduplication Storage Systems, Data Domain Virtual Edition
Article Properties
Article Number: 000059265
Article Type: Solution
Last Modified: 21 Aug 2026
Version:  4
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.