Metro-node: Efter opgradering til 8.0.x holder sikkerhedskopieringen af metadata op med at fungere
Summary: Denne artikel omhandler det problem, hvor sikkerhedskopieringen af metadata bliver ikke-operationel efter opgraderingen til 8.0.x-kode. Denne artikel indeholder de praktiske trin til gendannelse af funktionen til sikkerhedskopiering af metadata. ...
Symptoms
Hardware, der er påvirket af Dell:
Metronode mn114
Metronode mn215
Metro Node-Local/Metro
Dell-påvirket software:
Metro-node OS 8.0.0.0.0.267
, Metro-node OS 8.0.0.1.0.21
, Metro-node OS 8.0.1.0.0.220
Påvirkede ændringsaktiviteter:
Efter opgradering til metronode OS 8.0.x
Problem:
-
Ikonet
ndu pre-checkkommando rapporterer nedenstående fejl for hver klynge i en metronodekonfiguration:Eksempel på klynge-1:
VPlexcli:/> ndu pre-check Warning: During the NDU process, multiple directors will be offline for a portion of the time. This is non-disruptive but is dependent on a host-based multipathing solution being installed, configured, and operating on all connected hosts. ================================================================================ Performing NDU pre-checks ================================================================================ Verify NDU is not in progress.. OK Verify that the directors have been running continuously for 15 days.. OK Verify director communication status.. OK . . . Verify meta-volume backup configuration.. ERROR . . . ================================================================================ Errors (x errors found) ================================================================================ cluster-1 Metadata backups are NOT created according to schedule Last backup: Mon Aug 19 00:00:00 UTC 20xx Current time: Fri Dec 13 03:41:33 UTC 20xx There has been no metadata backup for 116 day(s) Run 'metadatabackup local' on cluster-1
Eksempel på klynge 2:
VPlexcli:/> ndu pre-check Warning: During the NDU process, multiple directors will be offline for a portion of the time. This is non-disruptive but is dependent on a host-based multipathing solution being installed, configured, and operating on all connected hosts. ================================================================================ Performing NDU pre-checks ================================================================================ Verify NDU is not in progress.. OK Verify that the directors have been running continuously for 15 days.. OK Verify director communication status.. OK . . . Verify meta-volume backup configuration.. ERROR . . . ================================================================================ Errors (x errors found) ================================================================================ cluster-2 Metadata backups are NOT created according to schedule Last backup: Sat Mar 16 01:30:00 UTC 20xx Current time: Fri Dec 13 03:41:33 UTC 20xx There has been no metadata backup for 272 day(s) Run 'metadatabackup local' on cluster-2
-
Når du beordrer
ll ~system-volumeskøres, afspejler datoen for sikkerhedskopiering af metadata en tidligere dato.I nedenstående eksempel holder sikkerhedskopieringen af metadata op med at fungere på begge klynger i et Metro-miljø:
VPlexcli:/> ll ~system-volumes /clusters/cluster-1/system-volumes: Name Volume Type Operational Health Active Ready Geometry Component Block Block Capacity Slots --------------------------------------- -------------- Status State ------ ----- -------- Count Count Size -------- ----- --------------------------------------- -------------- ----------- ------ ------ ----- -------- --------- -------- ----- -------- ----- meta_C1_xxxxxx meta-volume ok ok true true raid-1 2 20971264 4K 80G 64000 meta_C1_xxxxxxx_backup_20xx-11-21_01-30 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000 \------------/ date and time the last backup was run /clusters/cluster-2/system-volumes: Name Volume Type Operational Health Active Ready Geometry Component Block Block Capacity Slots --------------------------------------- -------------- Status State ------ ----- -------- Count Count Size -------- ----- --------------------------------------- -------------- ----------- ------ ------ ----- -------- --------- -------- ----- -------- ----- meta_C2_xxxxxx meta-volume ok ok true true raid-1 2 20971264 4K 80G 64000 meta_C2_xxxxxxx_backup_20xx-11-20_12-43 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000 \------------/ date and time the last backup was run
Symptomer:
- Sikkerhedskopieringen af metadata holder op med at fungere på begge klynger i et Metro-miljø.
- Sikkerhedskopieringen af metadata holder op med at fungere på en af klyngerne i et Metro-miljø
- Sikkerhedskopieringen af metadata holder op med at fungere i en lokal klynge
Cause
Under den planlagte daglige sikkerhedskopiering af metadata sidder tjenesten "daily_metadata_backup.service" lejlighedsvis fast i aktiveringstilstanden på enten director-1-1-A, director-2-1-A eller begge.
Resolution
Permanent afhjælpning:
Metro node Engineering undersøger dette problem. Når en rettelse er tilgængelig, opdateres denne artikel.
Løsning:
-
For at kontrollere status for tjenesten "daily_metadata_backup.service" ved Shell-prompten skal du køre kommandoen,
sudo systemctl status daily_metadata_backup.servicepå en A-node, f.eks. direktør-1-1-A eller direktør-2-1-A. Kontroller og bekræft, at attributten "Aktiv: aktivering (start)" er til stede, og at den kører længere end et minut. Hvis ja, betyder det, at denne tjeneste sidder fast på den pågældende A-node.Nedenstående eksempel viser, at director-1-1-A og director-2-1-A begge har attributten "service" "daily_metadata_backup.service" "Active: activating (start)" til stede og har kørt længere end et minut, hvilket betyder, at denne tjeneste sidder fast på disse noder som vist nedenfor.
Klynge-1:
service@director-2-1-a:~> sudo systemctl status daily_metadata_backup.service ● daily_metadata_backup.service - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.service; static) Active: activating (start) since Sat 2024-10-xx 01:30:18 UTC; 1 month 3 days ago <--------------------------- TriggeredBy: ● daily_metadata_backup.timer Main PID: 22553 (daily_metadata_) Tasks: 1 CGroup: /system.slice/daily_metadata_backup.service └─22553 /usr/bin/python3 /opt/dell/vplex/sbin/daily_metadata_backup.py Oct xx 01:30:18 director-2-1-a systemd[1]: Starting metronode automated daily metadata backups... . . . <truncated>Klynge-2:
service@director-1-1-a:~> sudo systemctl status daily_metadata_backup.service ● daily_metadata_backup.service - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.service; static) Active: activating (start) since Sat 2024-10-xx 01:30:18 UTC; 1 month 2 days ago <--------------------------- TriggeredBy: ● daily_metadata_backup.timer Main PID: 22553 (daily_metadata_) Tasks: 1 CGroup: /system.slice/daily_metadata_backup.service └─22553 /usr/bin/python3 /opt/dell/vplex/sbin/daily_metadata_backup.py Oct xx 01:30:18 director-1-1-a systemd[1]: Starting metronode automated daily metadata backups... . . . <truncated> -
Ved siden af for at kontrollere status for tjenesten "daily_metadata_backup.timer" på A-node, for eksempel director-1-1-A, director-2-1-A, kør kommandoen
sudo systemctl status daily_metadata_backup.timerog bekræfte, at attributten "Udløser:" vises som "i/t." Hvis ja, betyder det, at denne tjeneste sidder fast på den pågældende A-node.Nedenstående eksempel viser, at director-1-1-A og director-2-1-A begge har attributten "daily_metadata_backup.timer" "Trigger:", der vises som "n/a", hvilket betyder, at denne tjeneste sidder fast på disse noder.
Klynge-1:
service@director-1-1-a:~> sudo systemctl status daily_metadata_backup.timer ● daily_metadata_backup.timer - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.timer; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/daily_metadata_backup.timer.d └─daily_backup.conf Active: active (running) since Wed 2024-11-20 12:46:10 UTC; 18h ago Trigger: n/a <<<<<<<<<<<< Triggers: ● daily_metadata_backup.service Nov 20 12:46:10 director-1-1-a systemd[1]: Started metronode automated daily metadata backups. service@director-1-1-a:~>
Klynge-2:
service@director-2-1-a:~> sudo systemctl status daily_metadata_backup.timer ● daily_metadata_backup.timer - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.timer; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/daily_metadata_backup.timer.d └─daily_backup.conf Active: active (running) since Wed 2024-11-xx 12:46:10 UTC; 18h ago Trigger: n/a >>>>>>>>>>>>>>>>>>>>>>> Triggers: ● daily_metadata_backup.service Nov xx 12:46:10 director-2-1-a systemd[1]: Started metronode automated daily metadata backups. service@director-2-1-a:~>
-
Når det er bekræftet, hvilken node, eller muligvis begge noder, der har de to nævnte tjenester fast, skal du stoppe tjenesterne "daily_metadata_backup.service" og "daily_metadata_backup.timer" og derefter starte tjenesten for "daily_metadata_backup.timer" for at løse denne situation og for at metadatasikkerhedskopieringen begynder at fungere.
BEMÆRK: Brug ikke kommandoindstillingen "genstart".I nedenstående eksempel, da begge A-noder er berørt, stoppes og startes tjenesterne som følger:
sudo systemctl stop daily_metadata_backup.service
sudo systemctl stop daily_metadata_backup.timer
sudo systemctl start daily_metadata_backup.timer
-
Kør nedenstående kommando for at kontrollere status for at bekræfte, at den ikke sidder fast længere som følger:
Nedenstående eksempler viser kørslen af statuskommandoen for "daily_metadata_backup.service" for at kontrollere, om linjen "Aktiv: inaktiv (død)", der angiver, at tjenesten faktisk ikke kører, hvilket, når du venter på den næste sikkerhedskopieringscyklus af metadataene, er "inaktiv (død)":
service@director-2-1-a:~> sudo systemctl status daily_metadata_backup.service ● daily_metadata_backup.service - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.service; static) Active: inactive (dead) since Fri 2024-11-22 21:07:41 UTC; 1min 49s ago >>>>>>>>>>>> TriggeredBy: ● daily_metadata_backup.timer Process: 9183 ExecStart=/opt/dell/vplex/sbin/daily_metadata_backup.py (code=exited, status=0/SUCCESS) Main PID: 9183 (code=exited, status=0/SUCCESS) Nov 22 21:07:36 director-2-1-a systemd[1]: Starting metronode automated daily metadata backups... Nov 22 21:07:41 director-2-1-a systemd[1]: daily_metadata_backup.service: Succeeded. Nov 22 21:07:41 director-2-1-a systemd[1]: Finished metronode automated daily metadata backups. service@director-2-1-a:~>service@director-1-1-a:~> sudo systemctl status daily_metadata_backup.service ● daily_metadata_backup.service - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.service; static) Active: inactive (dead) since Fri 2024-11-22 21:07:41 UTC; 1min 49s ago >>>>>>>>>>>> TriggeredBy: ● daily_metadata_backup.timer Process: 9183 ExecStart=/opt/dell/vplex/sbin/daily_metadata_backup.py (code=exited, status=0/SUCCESS) Main PID: 9183 (code=exited, status=0/SUCCESS) Nov 22 21:07:36 director-1-1-a systemd[1]: Starting metronode automated daily metadata backups... Nov 22 21:07:41 director-1-1-a systemd[1]: daily_metadata_backup.service: Succeeded. Nov 22 21:07:41 director-1-1-a systemd[1]: Finished metronode automated daily metadata backups. service@director-2-1-a:~>Nedenstående eksempel viser, at tjenesten "daily_metadata_backup.timer" skal være "aktiv(venter)", og "Udløser" skal indstilles til i dag eller nu, hvilket betyder, at tjenesten nu fungerer som forventet:
service@director-2-1-a:~> sudo systemctl status daily_metadata_backup.timer ● daily_metadata_backup.timer - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.timer; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/daily_metadata_backup.timer.d └─daily_backup.conf Active: active (waiting) since Fri 2024-11-22 21:09:24 UTC; 14s ago >>>>>>>>>>> Trigger: Sat 2024-11-23 01:30:00 UTC; 4h 20min left >>>>>>>>>>> Triggers: ● daily_metadata_backup.service Nov 22 21:09:24 director-2-1-a systemd[1]: Started metronode automated daily metadata backups. service@director-2-1-a:~>service@director-1-1-a:~> sudo systemctl status daily_metadata_backup.timer ● daily_metadata_backup.timer - metronode automated daily metadata backups Loaded: loaded (/etc/systemd/system/daily_metadata_backup.timer; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/daily_metadata_backup.timer.d └─daily_backup.conf Active: active (waiting) since Fri 2024-11-22 21:09:24 UTC; 14s ago >>>>>>>>>>> Trigger: Sat 2024-11-23 01:30:00 UTC; 4h 20min left >>>>>>>>>>> Triggers: ● daily_metadata_backup.service Nov 22 21:09:24 director-1-1-a systemd[1]: Started metronode automated daily metadata backups. service@director-2-1-a:~> -
Vent, og overvåg, indtil den næste sikkerhedskopiering af metadata fuldføres ved at køre
ll ~system-volumestil at bekræfte, at problemet er blevet løst, og at sikkerhedskopiering af metadata udføres korrekt på følgende måde.Eksempel:
VPlexcli:/> ll ~system-volumes /clusters/cluster-1/system-volumes: Name Volume Type Operational Health Active Ready Geometry Component Block Block Capacity Slots --------------------------------------- -------------- Status State ------ ----- -------- Count Count Size -------- ----- --------------------------------------- -------------- ----------- ------ ------ ----- -------- --------- -------- ----- -------- ----- meta_C1_xxxxxx meta-volume ok ok true true raid-1 2 20971264 4K 80G 64000 meta_C1_xxxxxxx_backup_2024-11-23_01-30 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000 meta_C1_4UQT429_backup_2024-11-24_01-30 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000 /clusters/cluster-2/system-volumes: Name Volume Type Operational Health Active Ready Geometry Component Block Block Capacity Slots --------------------------------------- -------------- Status State ------ ----- -------- Count Count Size -------- ----- --------------------------------------- -------------- ----------- ------ ------ ----- -------- --------- -------- ----- -------- ----- meta_C2_xxxxxx meta-volume ok ok true true raid-1 2 20971264 4K 80G 64000 meta_C2_xxxxxxx_backup_2024-11-23_12-43 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000 meta_C2_xxxxxxx_backup_2024-11-24_12-43 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000