Data Domain: DDFS PANIC: dmbt_update_entry_nolock

The affected Data Domain (DD) reports a panic with the following message for systems with Automatic Retention Lock (ARL) enabled mtrees used for replication:

PANIC: ddr/dm/dmbt_iface.c: dmbt_update_entry_nolock

• PowerProtect Data Domains with Automatic Retention Lock (ARL) Enabled.

• Crash of the Data Domain file system
• System instability 
• Logs indicating the above panic message immediately before a system crash

This issue arises when a mtree replication snapshot occurs between the DDBOOST open and close calls.

• The nfsproc3_ddcp_open_file_3_svc() function identifies a Retention lock (RL) file being modified within the ARL cool-off-period (COP) and resets the tag value to 0 (DM_TAG_DEFAULT).
• The subsequent nfsproc3_ddcp_close_file_3_svc() function normally resets the tag to a valid value.
• However, if a file is created with an extended attribute and is modified within the ARL COP, and a snapshot occurs between the DDBOOST open and close calls, this sequence can trigger the PANIC.

This occurs on 7.7.x and 7.10.x, and the trigger is when retention lock (usually ARL) is enabled on the mtrees.

To resolve this issue, do the following: 

1. Break any replication context of the original source and destination mtree.

• This creates a new, clean copy of the mtree without the problematic context.

• This prevents the issue from recurring.

4. Redirect backups to the newly fastcopied mtree on Source the DD.
(Update any backup jobs to target the new mtree.)

• Ensure that the new replication setup is fully synchronized.
• Keep this mtree ARL disabled from preventing the PANIC to recur. 
  • Manage the old mtrees:
  • Leave the original mtrees on both the source and destination systems for the time being.
    • Once the new replication is synchronized and the retention period for the old mtree files has expired, delete the files in the old mtrees.
    • Then delete the old mtree on both the source and destination.

• It is crucial to monitor the system for any further anomalies after performing the above steps.
• Ensure that all backup policies and replication contexts are correctly updated to reflect the changes.