Data Domain：無法刪除由複寫背景關係軟鎖的快照

Summary: 已找到多年前建立的快照，即使已過期仍會保留。無法刪除，因為它已軟鎖定

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

- 幾年前創建的快照，儘管標記為已過期，但仍保留：

ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520
Snapshot Information for MTree: /data/col1/avamar-1493130520
----------------------------------------------
Name                Pre-Comp (GiB)   Create Date         Retain Until        Status
-----------------   --------------   -----------------   -----------------   -------
cp.20250309131143         213612.6   Mar  9 2025 07:13   Mar 10 2025 07:39   expired
cp.20250309133359         214727.3   Mar  9 2025 07:34   Mar 10 2025 07:41   expired
cp.20250310131141         207612.0   Mar 10 2025 07:13
cp.20250310133957         208726.9   Mar 10 2025 07:40
cp.20230919134341         403018.8   Sep 19 2023 07:44   Sep 20 2023 07:34   expired 
-----------------   --------------   -----------------   -----------------   -------

- 快照在紀錄檔中顯示為軟鎖定 ddfs.info
注意：擷取該檔案最簡單的方法是使用 scp 將其從 Data Domain （DD） 複製到另一部 Linux 機器。請確定手頭有登入資料以使用此語法：

admin@anylinuxmachine:~/>: scp dd_username@dd_hostname:/ddr/var/log/debug/ddfs.info .

 

admin@anylinuxmachine:~/>: grep "cp.20230919134341" ddfs.info
03/11 05:54:20.569 (tid 0x7fb71d0e6f10): rmsnapshot: mid=1583172823 s_name=cp.20230919134341.
03/11 05:54:20.569 (tid 0x7fb71d0e6f10): _dm_rmsnapshot: snapshot 'cp.20230919134341' is softlocked.

Data Domain 警示顯示與一個複寫內容相關的警示未同步：

ddboost@dd02# alerts show current
Id      Post Time                Severity Class       Object             Message															
------- ------------------------ -------- ----------- ------------------ ----------------------------------------------------------------
p0-924  Tue Nov 28 09:35:07 2023 CRITICAL Network     Interface Index=11 EVT-NETM-00001: Network interface connectivity is down on ethMc.
m0-1971 Mon Mar 10 06:56:36 2025 WARNING  Replication context=1          EVT-REPL-00006: Sync-as-of time is more than 13109 hours ago.
------- ------------------------ -------- ----------- ------------------ ----------------------------------------------------------------
There are 2 active alerts.

列出 Data Domain 複寫來源中的複寫內容，可取得該背景關係 #1 的詳細資料。
複寫內容 #1 的來源和目的地皆參照包含快照軟鎖定的 mtree。

avamar@dd02# replication show config
CTX Source                                                 	   Destination                                               	 Connection                        
																															 Host and Port                     
--- ---------------------------------------------------------- ------------------------------------------------------------- ------------------------------------- 
1   mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault.yourdomain.com.br/data/col1/avamar-1493130520 ddvault.yourdomain.com.br   (default) 
2   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01               ddvault02.vault         	 (default) 
3   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01repl           ddvault02.vault         	 (default) 
4   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv02     mtree://dd03.yourdomain.com.br/data/col1/SysDR-R_ppmsv02      dd03.yourdomain.com.br      (default) 
5   mtree://dd02.yourdomain.com.br/data/col1/MtreeVault        mtree://ddvault02.vault/data/col1/Mtreevaultdest          	 ddvault02.vault         	 (default) 
6   mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault02.vault/data/col1/avamar-1493130520dest   	 ddvault02.vault         	 (default) 
7   mtree://dd02.yourdomain.com.br/data/col1/CritMtreeVault    mtree://ddvault02.vault/data/col1/CritMtreeVaultDest      	 ddvault02.vault         	 (default) 
8   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01dest       	 ddvault02.vault         	 (default) 
--- ---------------------------------------------------------- --------------------------------------------------------- 	 ------------------------------------- 
DD System default Max-repl-streams per context: 32

 

注意：進行故障診斷時，名稱為 ddvault.yourdomain.com.br 的診斷目的地主機已不存在。它已關閉並被刪除。

Cause

- 在來源 Data Domain 引用快照所在的 mtree 的陳舊且不再使用的複寫背景關係，持有該軟鎖。
已從環境中移除/刪除 Data Domain，但參考已刪除 Data Domain 的複寫內容並未刪除，因此會被遺棄。 

Resolution

我們將需要中斷 （這是刪除複寫內容的程序） 從複寫來源 Data Domain。
登入 Data Domain 並列出複寫組態，以檢閱並確認要中斷/刪除的上下文 （在此範例中，上下文 #1）：

ddboost@dd02# replication show config
CTX Source                                                 	   Destination                                               	 Connection                       
																															 Host and Port                    
--- ---------------------------------------------------------- ------------------------------------------------------------- -------------------------------------
1   mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault.yourdomain.com.br/data/col1/avamar-1493130520 ddvault.yourdomain.com.br   (default)
2   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01           	 ddvault02.vault         	 (default)
3   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01repl       	 ddvault02.vault         	 (default)
4   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv02     mtree://dd03.yourdomain.com.br/data/col1/SysDR-R_ppmsv02      dd03.yourdomain.com.br      (default)
5   mtree://dd02.yourdomain.com.br/data/col1/MtreeVault        mtree://ddvault02.vault/data/col1/Mtreevaultdest          	 ddvault02.vault         	 (default)
6   mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault02.vault/data/col1/avamar-1493130520dest   	 ddvault02.vault         	 (default)
7   mtree://dd02.yourdomain.com.br/data/col1/CritMtreeVault    mtree://ddvault02.vault/data/col1/CritMtreeVaultDest      	 ddvault02.vault         	 (default)
8   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01dest       	 ddvault02.vault         	 (default)
--- ---------------------------------------------------------- ------------------------------------------------------------- -------------------------------------
DD System default Max-repl-streams per context: 32

- 發出命令以中斷該複製上下文：

ddboost@dd02# replication break rctx://1
The 'replication break' command irrevocably turns off logical
replication from this mtree.  To reconfigure the mtree
for replication, the destination mtree must not exist,
or, alternatively, 'replication resync' must be used.
        Are you sure? (yes|no) [no]: yes
ok, proceeding.
ddboost@dd02#
注意：中斷執行期間顯示的消息專門應用於要刪除的複製上下文。不會影響其餘的複寫內容。


- 再次列出複製配置，以確保複製上下文 #1 已消失：

ddboost@dd02# replication show config
CTX Source                                                 	   Destination                                             	Connection                    
																													    Host and Port                 
--- ---------------------------------------------------------- -------------------------------------------------------- ----------------------------------
2   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01         	ddvault02.vault      	 (default)
3   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01repl     	ddvault02.vault      	 (default)
4   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv02     mtree://dd03.yourdomain.com.br/data/col1/SysDR-R_ppmsv02 dd03.yourdomain.com.br   (default)
5   mtree://dd02.yourdomain.com.br/data/col1/MtreeVault        mtree://ddvault02.vault/data/col1/Mtreevaultdest        	ddvault02.vault      	 (default)
6   mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault02.vault/data/col1/avamar-1493130520dest 	ddvault02.vault      	 (default)
7   mtree://dd02.yourdomain.com.br/data/col1/CritMtreeVault    mtree://ddvault02.vault/data/col1/CritMtreeVaultDest    	ddvault02.vault      	 (default)
8   mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01     mtree://ddvault02.vault/data/col1/SysDR_ppmsv01dest     	ddvault02.vault      	 (default)
--- ---------------------------------------------------------- -------------------------------------------------------- ----------------------------------
DD System default Max-repl-streams per context: 32

- 驗證複製上下文 #1 刪除後，該舊快照會自動標記為未過期：

ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520
Snapshot Information for MTree: /data/col1/avamar-1493130520
----------------------------------------------
Name              Pre-Comp (GiB) Create Date        Retain Until      Status
----------------- -------------- -----------------  ----------------- -------
cp.20250317130903       214805.7 Mar 17 2025 07:10  Mar 18 2025 07:32 expired
cp.20250317132705       215924.2 Mar 17 2025 07:28  Mar 18 2025 07:33 expired
cp.20250318131055       215418.1 Mar 18 2025 07:12
cp.20250318133213       216537.6 Mar 18 2025 07:33
cp.20230919134341       403018.8 Sep 19 2023 07:44  Apr  8 2025 13:25			
----------------- -------------- -----------------  ----------------- -------

- 再次將該快照標記為過期：

ddboost@dd02# snapshot expire cp.20230919134341 mtree /data/col1/avamar-1493130520
Snapshot "cp.20230919134341" for mtree "/data/col1/avamar-1493130520" will be retained until Mar 18 2025 13:30.

ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520
Snapshot Information for MTree: /data/col1/avamar-1493130520
----------------------------------------------
Name              Pre-Comp (GiB) Create Date       Retain Until      Status
----------------- -------------- ----------------- ----------------- -------
cp.20250318131055       215418.1 Mar 18 2025 07:12
cp.20250318133213       216537.6 Mar 18 2025 07:33
cp.20230919134341       403018.8 Sep 19 2023 07:44 Mar 18 2025 13:30 expired 
----------------- -------------- ----------------- ----------------- -------

- 在 ddfs.info 的更新副本中查找該快照，捕獲軟鎖的那一刻：

admin@anylinuxmachine:~/>: grep cp.20230919134341 ddfs.info
03/18 13:25:16.663 (tid 0x7fb7969b2b40): updatesnapshot: mid=1583172823 m_name=avamar-1493130520 sid=35512 name=cp.20230919134341 retention is changed from 1741692419 to 1744140316 err :0
03/18 13:25:16.666 (tid 0x7fb7969b2b40): mrepl_snap_deref deref name=cp.20230919134341 (1583172823:35512) 1 -> 0
03/18 13:25:16.669 (tid 0x7fb7969b2b40): repl ctx 1: mrepl_protect_new_base deref'ing base cp.20230919134341 Not Referenced, USER
03/18 13:25:18.680 (tid 0x7fb7969b2b40): repl ctx: mrepl_snap_ref_cleanup Unsoftlocking snapshot name=cp.20230919134341 (1583172823:35512) 
03/18 13:30:41.710 (tid 0x7fb7969b2b40): updatesnapshot: mid=1583172823 m_name=avamar-1493130520 sid=35512 name=cp.20230919134341 retention is changed from 1744140316 to 1742326241 err :0
03/18 13:30:41.710 (tid 0x7fb7969b2b40): FM fms_snapshot_update:1933 - Updated "cp.20230919134341", mtree "/data/col1/avamar-1493130520" 1742326241
03/18 13:32:50.061 (tid 0x7fb79ba174e0): rmsnapshot: mid=1583172823 s_name=cp.20230919134341.
03/18 13:32:50.076 (tid 0x7fb79ba174e0): rmsnapshot: done, mid=1583172823 m_name=avamar-1493130520 sid=35512 s_name=cp.20230919134341 retention=1742326241 ch=<dm_mtree:40000010000,1,0,400,6,0,e98eb815,440357a6,18045d05,d3463134,cadda83,9dfa4504:> cpg_id=1 cp_uuid=cf984be60de2e768:887b264e6a0bd7d6.
admin@anylinuxmachine:~/>:

- 開始新的檔案系統清理

ddboost@dd02# fi clean start
Cleaning started.  Use 'filesys clean watch' to monitor progress.

ddboost@dd02# fi clean status
Cleaning started at 2025/03/18 13:32:49: phase 1 of 6 (pre-merge)
  0.0% complete, 25616 GiB free; time: phase  0:00:14, total  0:00:14

- 驗證舊的快照現在是否已消失，即使清理仍在進行中。
- 問題已解決：

ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520
Snapshot Information for MTree: /data/col1/avamar-1493130520
----------------------------------------------
Name              Pre-Comp (GiB) Create Date       Retain Until      Status
----------------- -------------- ----------------- ----------------- ------
cp.20250318131055       215418.1 Mar 18 2025 07:12
cp.20250318133213       216537.6 Mar 18 2025 07:33
----------------- -------------- ----------------- ----------------- ------

Affected Products

Data Domain Boost – File System
Article Properties
Article Number: 000296945
Article Type: Solution
Last Modified: 24 Mar 2025
Version:  1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.