Data Domain: EVT-QUOTA-00002: Mtree Quota hard limit reached

Users logged into a Data Domain may see alerts such as "Mtree Quota hard limit reached," for example:

sysadmin@dd# alerts show current
Id     Post Time                  Severity   Class        Object                        Message
----   ------------------------   --------   ----------   ---------------------------   ------------------------------------------------
m0-7   Tue Aug 13 20:05:23 2019   CRITICAL   Filesystem   Mtree=/data/col1/quota_test   EVT-QUOTA-00002: Mtree Quota hard limit reached.
----   ------------------------   --------   ----------   ---------------------------   ------------------------------------------------
There is 1 active alert.

The cause for this alert is that the hard quota is assigned to a MTree in the Data Domain and the amount of data (pre-comp) in the MTree has reached the configured hard limit. Quotas are a useful way to manage growth for data in separate MTrees, with the "soft limit" raising an alert that the amount of pre-comp data in the MTree has exceeded the threshold. Therefore, the backup administrator has advance notice and sufficient time to see if growth is as expected in case there are some corrective actions needed.

However, once the configured MTree quota hard limit is reached, no further writes to the MTree are allowed until the administrator, through the backup application, removes some of the files in the MTree or, alternatively, either increases or removes the hard quota limit for the MTree.

To show the configured quota values for a MTree, use a command similar to the following:

sysadmin@dd# quota capacity show mtrees /data/col1/quota_test
Mtree                   Pre-Comp (MiB)   Soft-Limit (MiB)   Hard-Limit (MiB)
---------------------   --------------   ----------------   ----------------
/data/col1/quota_test                6                  2                  6
---------------------   --------------   ----------------   ----------------

To see a good estimate of how disk space data in particular MTrees is using, refer to KB article 465216: Data Domain Operating System (DDOS) physical capacity measurement/reporting (PCM/PCR) frequently asked questions

When a quota hard limit for a MTree is reached, an alert is generated and further backups fail to that MTree until the administrator either increases or removes the hard quota limit for the MTree (or some of the backups in the MTree are removed). Simply trying to remove the alert does not allow for backups to resume to the MTree. After the hard limit is no longer reached, the alert should go away on its own, if not, manually clear it such as below:

sysadmin@dd# alerts clear alert-id m0-7

Option 1:
If the administrator considers the number of backups in the MTree is right, users may choose to increase the hard quota limit for the MTree to a higher value:

# quota capacity set mtrees /data/col1/quota_test hard-limit 100 MiB

Option 2:
If the administrator sees no need on having a hard quota set on the MTree, users may want to completely remove the hard quota from that MTree:

# quota capacity reset mtrees /data/col1/quota_test hard-limit

If users want to remove both quota setting for the MTree, use instead:

# quota capacity reset mtrees /data/col1/quota_test

Option 3:
The quota may have allowed the customer to realize more than necessary backups existed in the alerted MTree. If that is the case, the next step would be to use the backup application to expire those unnecessary backup files.

When the backup files are expired and the files in the Data Domain are removed from the MTree, the pre-comp reduce, and the hard limit is no longer reached (although you may require to clear the alert yourself). In any case, disk space used by deleted backups is not reclaimed from the Data Domain until the Data Domain clean is run and completed.