Data Domain: Replikointikontekstin mukaan ohjelmallisesti lukittua tilannevedosta ei voi poistaa
Summary: Löytyi tilannevedos, luotu vuosia sitten ja säilytetty edelleen, vaikka se on vanhentunut. Ei voi poistaa, koska se on lukittu pehmeästi
Symptoms
- Vuosia sitten luotu tilannevedos säilytetään edelleen, vaikka se on jo merkitty vanhentuneeksi:
ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520 Snapshot Information for MTree: /data/col1/avamar-1493130520 ---------------------------------------------- Name Pre-Comp (GiB) Create Date Retain Until Status ----------------- -------------- ----------------- ----------------- ------- cp.20250309131143 213612.6 Mar 9 2025 07:13 Mar 10 2025 07:39 expired cp.20250309133359 214727.3 Mar 9 2025 07:34 Mar 10 2025 07:41 expired cp.20250310131141 207612.0 Mar 10 2025 07:13 cp.20250310133957 208726.9 Mar 10 2025 07:40 cp.20230919134341 403018.8 Sep 19 2023 07:44 Sep 20 2023 07:34 expired ----------------- -------------- ----------------- ----------------- -------
- Tilannekuva näkyy lokitiedostossa pehmeästi lukittuna ddfs.info
HUOMAUTUS: Helpoin tapa kaapata tiedosto olisi kopioida se SCP: n avulla Data Domainista (DD) toiseen Linux-koneeseen. Varmista, että sinulla on kirjautumistunnukset, jotta voit käyttää tätä syntaksia:
admin@anylinuxmachine:~/>: scp dd_username@dd_hostname:/ddr/var/log/debug/ddfs.info .
admin@anylinuxmachine:~/>: grep "cp.20230919134341" ddfs.info
03/11 05:54:20.569 (tid 0x7fb71d0e6f10): rmsnapshot: mid=1583172823 s_name=cp.20230919134341.
03/11 05:54:20.569 (tid 0x7fb71d0e6f10): _dm_rmsnapshot: snapshot 'cp.20230919134341' is softlocked.
- Data Domain -ilmoituksessa näkyi yksi hälytys, joka liittyi yhteen synkronoimattomaan replikointikontekstiin:
ddboost@dd02# alerts show current Id Post Time Severity Class Object Message ------- ------------------------ -------- ----------- ------------------ ---------------------------------------------------------------- p0-924 Tue Nov 28 09:35:07 2023 CRITICAL Network Interface Index=11 EVT-NETM-00001: Network interface connectivity is down on ethMc. m0-1971 Mon Mar 10 06:56:36 2025 WARNING Replication context=1 EVT-REPL-00006: Sync-as-of time is more than 13109 hours ago. ------- ------------------------ -------- ----------- ------------------ ---------------------------------------------------------------- There are 2 active alerts.
- Luetteloimalla replikointikontekstit Data Domain -replikointilähteessä on mahdollista saada tiedot kyseisestä kontekstista #1.
- Replikointikontekstissa #1 on sekä lähde että kohde, jotka viittaavat mtreeen, jossa tilannevedos on lukittu.
avamar@dd02# replication show config CTX Source Destination Connection Host and Port --- ---------------------------------------------------------- ------------------------------------------------------------- ------------------------------------- 1 mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault.yourdomain.com.br/data/col1/avamar-1493130520 ddvault.yourdomain.com.br (default) 2 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01 ddvault02.vault (default) 3 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01repl ddvault02.vault (default) 4 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv02 mtree://dd03.yourdomain.com.br/data/col1/SysDR-R_ppmsv02 dd03.yourdomain.com.br (default) 5 mtree://dd02.yourdomain.com.br/data/col1/MtreeVault mtree://ddvault02.vault/data/col1/Mtreevaultdest ddvault02.vault (default) 6 mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault02.vault/data/col1/avamar-1493130520dest ddvault02.vault (default) 7 mtree://dd02.yourdomain.com.br/data/col1/CritMtreeVault mtree://ddvault02.vault/data/col1/CritMtreeVaultDest ddvault02.vault (default) 8 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01dest ddvault02.vault (default) --- ---------------------------------------------------------- --------------------------------------------------------- ------------------------------------- DD System default Max-repl-streams per context: 32
Cause
- Data Domain on poistettu/poistettu ympäristöstä, mutta replikointikontekstia, joka viittaa poistettuun Data Domainiin, ei poistettu, joten se jäi taakse.
Resolution
- Meidän on katkaistava (on prosessi replikointikontekstin poistamiseksi) replikointilähteen Data Domainista.
- Kirjaudu Data Domainiin ja luettele replikointimääritykset tarkistaaksesi ja vahvistaaksesi rikottavan/poistettavan kontekstin (tässä esimerkissä konteksti #1):
ddboost@dd02# replication show config CTX Source Destination Connection Host and Port --- ---------------------------------------------------------- ------------------------------------------------------------- ------------------------------------- 1 mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault.yourdomain.com.br/data/col1/avamar-1493130520 ddvault.yourdomain.com.br (default) 2 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01 ddvault02.vault (default) 3 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01repl ddvault02.vault (default) 4 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv02 mtree://dd03.yourdomain.com.br/data/col1/SysDR-R_ppmsv02 dd03.yourdomain.com.br (default) 5 mtree://dd02.yourdomain.com.br/data/col1/MtreeVault mtree://ddvault02.vault/data/col1/Mtreevaultdest ddvault02.vault (default) 6 mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault02.vault/data/col1/avamar-1493130520dest ddvault02.vault (default) 7 mtree://dd02.yourdomain.com.br/data/col1/CritMtreeVault mtree://ddvault02.vault/data/col1/CritMtreeVaultDest ddvault02.vault (default) 8 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01dest ddvault02.vault (default) --- ---------------------------------------------------------- ------------------------------------------------------------- ------------------------------------- DD System default Max-repl-streams per context: 32
- Anna komento replikointikontekstin katkaisemiseksi:
ddboost@dd02# replication break rctx://1
The 'replication break' command irrevocably turns off logical
replication from this mtree. To reconfigure the mtree
for replication, the destination mtree must not exist,
or, alternatively, 'replication resync' must be used.
Are you sure? (yes|no) [no]: yes
ok, proceeding.
ddboost@dd02#
- Listaa replikointikonfiguraatio uudelleen varmistaaksesi, että replikointikonteksti #1 on poissa:
ddboost@dd02# replication show config CTX Source Destination Connection Host and Port --- ---------------------------------------------------------- -------------------------------------------------------- ---------------------------------- 2 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01 ddvault02.vault (default) 3 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01repl ddvault02.vault (default) 4 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv02 mtree://dd03.yourdomain.com.br/data/col1/SysDR-R_ppmsv02 dd03.yourdomain.com.br (default) 5 mtree://dd02.yourdomain.com.br/data/col1/MtreeVault mtree://ddvault02.vault/data/col1/Mtreevaultdest ddvault02.vault (default) 6 mtree://dd02.yourdomain.com.br/data/col1/avamar-1493130520 mtree://ddvault02.vault/data/col1/avamar-1493130520dest ddvault02.vault (default) 7 mtree://dd02.yourdomain.com.br/data/col1/CritMtreeVault mtree://ddvault02.vault/data/col1/CritMtreeVaultDest ddvault02.vault (default) 8 mtree://dd02.yourdomain.com.br/data/col1/SysDR_ppmsv01 mtree://ddvault02.vault/data/col1/SysDR_ppmsv01dest ddvault02.vault (default) --- ---------------------------------------------------------- -------------------------------------------------------- ---------------------------------- DD System default Max-repl-streams per context: 32
- Kun replikointikonteksti #1 on poistettu, tarkista, että vanha tilannekuva merkitään automaattisesti vanhentuneeksi:
ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520 Snapshot Information for MTree: /data/col1/avamar-1493130520 ---------------------------------------------- Name Pre-Comp (GiB) Create Date Retain Until Status ----------------- -------------- ----------------- ----------------- ------- cp.20250317130903 214805.7 Mar 17 2025 07:10 Mar 18 2025 07:32 expired cp.20250317132705 215924.2 Mar 17 2025 07:28 Mar 18 2025 07:33 expired cp.20250318131055 215418.1 Mar 18 2025 07:12 cp.20250318133213 216537.6 Mar 18 2025 07:33 cp.20230919134341 403018.8 Sep 19 2023 07:44 Apr 8 2025 13:25 ----------------- -------------- ----------------- ----------------- -------
- Merkitse tilannekuva jälleen vanhentuneeksi:
ddboost@dd02# snapshot expire cp.20230919134341 mtree /data/col1/avamar-1493130520
Snapshot "cp.20230919134341" for mtree "/data/col1/avamar-1493130520" will be retained until Mar 18 2025 13:30.
ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520
Snapshot Information for MTree: /data/col1/avamar-1493130520
----------------------------------------------
Name Pre-Comp (GiB) Create Date Retain Until Status
----------------- -------------- ----------------- ----------------- -------
cp.20250318131055 215418.1 Mar 18 2025 07:12
cp.20250318133213 216537.6 Mar 18 2025 07:33
cp.20230919134341 403018.8 Sep 19 2023 07:44 Mar 18 2025 13:30 expired
----------------- -------------- ----------------- ----------------- -------
- Etsimällä tätä tilannekuvaa päivitetystä kopiosta ddfs.info, hetki, jolloin pehmeä lukko poistetaan, tallennetaan:
admin@anylinuxmachine:~/>: grep cp.20230919134341 ddfs.info
03/18 13:25:16.663 (tid 0x7fb7969b2b40): updatesnapshot: mid=1583172823 m_name=avamar-1493130520 sid=35512 name=cp.20230919134341 retention is changed from 1741692419 to 1744140316 err :0
03/18 13:25:16.666 (tid 0x7fb7969b2b40): mrepl_snap_deref deref name=cp.20230919134341 (1583172823:35512) 1 -> 0
03/18 13:25:16.669 (tid 0x7fb7969b2b40): repl ctx 1: mrepl_protect_new_base deref'ing base cp.20230919134341 Not Referenced, USER
03/18 13:25:18.680 (tid 0x7fb7969b2b40): repl ctx: mrepl_snap_ref_cleanup Unsoftlocking snapshot name=cp.20230919134341 (1583172823:35512)
03/18 13:30:41.710 (tid 0x7fb7969b2b40): updatesnapshot: mid=1583172823 m_name=avamar-1493130520 sid=35512 name=cp.20230919134341 retention is changed from 1744140316 to 1742326241 err :0
03/18 13:30:41.710 (tid 0x7fb7969b2b40): FM fms_snapshot_update:1933 - Updated "cp.20230919134341", mtree "/data/col1/avamar-1493130520" 1742326241
03/18 13:32:50.061 (tid 0x7fb79ba174e0): rmsnapshot: mid=1583172823 s_name=cp.20230919134341.
03/18 13:32:50.076 (tid 0x7fb79ba174e0): rmsnapshot: done, mid=1583172823 m_name=avamar-1493130520 sid=35512 s_name=cp.20230919134341 retention=1742326241 ch=<dm_mtree:40000010000,1,0,400,6,0,e98eb815,440357a6,18045d05,d3463134,cadda83,9dfa4504:> cpg_id=1 cp_uuid=cf984be60de2e768:887b264e6a0bd7d6.
admin@anylinuxmachine:~/>:
- Aloita uusi tiedostojärjestelmän puhdistus
ddboost@dd02# fi clean start Cleaning started. Use 'filesys clean watch' to monitor progress. ddboost@dd02# fi clean status Cleaning started at 2025/03/18 13:32:49: phase 1 of 6 (pre-merge) 0.0% complete, 25616 GiB free; time: phase 0:00:14, total 0:00:14
- Varmista, että vanha tilannevedos on nyt poissa, vaikka puhdistus on vielä kesken.
- Ongelma on ratkaistu:
ddboost@dd02# snapshot list mtree /data/col1/avamar-1493130520 Snapshot Information for MTree: /data/col1/avamar-1493130520 ---------------------------------------------- Name Pre-Comp (GiB) Create Date Retain Until Status ----------------- -------------- ----------------- ----------------- ------ cp.20250318131055 215418.1 Mar 18 2025 07:12 cp.20250318133213 216537.6 Mar 18 2025 07:33 ----------------- -------------- ----------------- ----------------- ------