ECS: OBS: 엑스닥터: RAP073/208: 스위치 연결 오류가 감지됨
Summary: 이 기술 자료에서는 스위치 연결 장애가 탐지된 알림을 처리하는 방법에 대해 설명합니다.
Symptoms
ECS xDoctor v4.8-109.0 및 ObjectScale xDoctor v5.1-109.0부터 RAP208(Switch Connection Failure Detected)이 자동 복구로 구현됩니다. 스위치 연결 문제가 구성된 오류 또는 위험 심각도 임계값을 초과하면 xDoctor는 RAP208 알림을 발생시키고 통합 수리 오케스트레이션 워크플로를 자동으로 시작합니다. xDoctor 자동 복구가 활성화된 경우 이 워크플로는 필요한 문제 해결 작업을 수행합니다.
참고: 환경에서 ECS xDoctor v4.8-109.0 또는 ObjectScale xDoctor v5.1-109.0 이전 버전의 xDoctor를 실행 중인 경우 RAP208 자동 복구 기능을 사용할 수 없습니다. 이러한 버전에서는 아래에 설명된 AutoPilot 프로세스를 사용하거나 해결 방법 섹션에 설명된 수동 문제 해결 단계에 따라 문제 해결을 수행해야 합니다.
RAP208 자동 복구를 트리거하는 알림
RAP208 자동 복구 워크플로는 스위치 연결 장애가 구성된 오류 또는 위험 심각도 임계값을 초과할 때 트리거됩니다. 이 임계값을 초과하면 xDoctor는 자동 복구 프로세스를 트리거하는 역할을 하는 RAP208 알림을 생성합니다.
알림 출력 예시
참고: ECS xDoctor v4.8-109.0 및 ObjectScale xDoctor v5.1-109.0 이전의 xDoctor 버전에서는 이 조건으로 인해 알림만 발생합니다. 자동 문제 해결은 수행되지 않습니다.
--------------------------------------------------------
INFO - Auto Healer for dell_switch_connectivity disabled
--------------------------------------------------------
Extra = Auto Healer for dell_switch_connectivity disabled
Timestamp = 2026-04-01_180132
PSNT = CKMXXXXXXXXXXX @ 4.8-109.0
----------------------------------------------------
ERROR - (Cached) Switch Connection Failure detected.
----------------------------------------------------
Node = 169.254.1.1
Extra = {"169.254.1.1": ["hare"]}
RAP = RAP208
Solution = KB 39838
Timestamp = 2026-04-01_180132
PSNT = CKMXXXXXXXXXXX @ 4.8-109.0
Auto Healer 문제 해결(예시)
자동 복구 기능이 활성화되면 xDoctor는 이 기술 자료에 요약된 일반적인 개선 조치를 적용하여 감지된 스위치 연결 문제에 대한 문제 해결을 자동으로 시작합니다.
--------------------------------------------------------
FIXED - Auto Healer fixed Dell switch connectivity issue
--------------------------------------------------------
Node = Nodes
Extra = {"Nodes": ["169.254.1.1"]}
Timestamp = 2026-04-01_180344
PSNT = CKMXXXXXXXXXXX @ 4.8-109.0
자동 치유기 요구 사항
이 문제를 해결하려면 xDoctor 자동 복구 기능을 활성화해야 합니다. 자동 복구는 다음에 설명된 단계에 따라 설치 중 또는 설치 후에 사용하도록 설정할 수 있습니다.
Cause
스위치 교체 후 스위치 인증에 사용되는 SSH 호스트 키가 변경되거나 스위치에 연결된 관리 인터페이스가 관리상 종료될 수 있습니다. 경우에 따라 xDoctor에 구성된 암호가 영향을 받는 스위치의 현재 암호와 일치하지 않으므로 그에 따라 업데이트해야 합니다.
xDoctor 자동화 및 자동 복구 워크플로는 스위치 암호 문제 해결을 수행하지 않습니다. 대신 xDoctor는 인증 관련 오류를 감지하고 적절한 경고를 발생시켜 스위치에 설정된 암호를 사용하도록 xDoctor를 구성하는 방법을 설명하는 관련 기술 자료 문서로 사용자를 안내합니다.
Resolution
xDoctor 자동 치유기: ObjectScale xDoctor v5.1-109.0/ECS xDoctor v4.8-109.0 이상
- 활성화된 자동 복구를 수동으로 트리거하려면
master.rack노드. 그러면 랙 분석기가 시작되며, 노드를 한 번에 하나씩 검증하고 자동 복구합니다.
# sudo xdoctor --rap=RAP208
예:
admin@ecsnode1:~> sudo xdoctor --rap=RAP208 2026-04-01 18:03:45,441: xDoctor_4.8-109.0 - INFO : Initializing xDoctor v4.8-109.0 ... [... Truncated Output ...] 2026-04-01 18:05:01,725: xDoctor_4.8-109.0 - INFO : ANALYZER [ac_dell_switch_connectivity] 2026-04-01 18:05:02,063: xDoctor_4.8-109.0 - INFO : Autohealing switch_connectivity on node 169.254.1.1 ... 2026-04-01 18:08:57,494: xDoctor_4.8-109.0 - INFO : All data analyzed in 0:03:55 2026-04-01 18:08:58,529: xDoctor_4.8-109.0 - INFO : -------------------- 2026-04-01 18:08:58,529: xDoctor_4.8-109.0 - INFO : Diagnosis Summary 2026-04-01 18:08:58,529: xDoctor_4.8-109.0 - INFO : -------------------- 2026-04-01 18:08:58,529: xDoctor_4.8-109.0 - INFO : PSNT: CKMXXXXXXXXXXX 2026-04-01 18:08:58,529: xDoctor_4.8-109.0 - INFO : -------------------- 2026-04-01 18:08:58,529: xDoctor_4.8-109.0 - INFO : FIXED = 1 2026-04-01 18:08:58,530: xDoctor_4.8-109.0 - INFO : CRITICAL = 0 2026-04-01 18:08:58,530: xDoctor_4.8-109.0 - INFO : CRITICAL (CACHED) = 0 2026-04-01 18:08:58,530: xDoctor_4.8-109.0 - INFO : ERROR = 0 2026-04-01 18:08:58,530: xDoctor_4.8-109.0 - INFO : ERROR (CACHED) = 0 2026-04-01 18:08:58,530: xDoctor_4.8-109.0 - INFO : WARNING = 0 2026-04-01 18:08:58,530: xDoctor_4.8-109.0 - INFO : INFO = 0 2026-04-01 18:08:58,530: xDoctor_4.8-109.0 - INFO : VERBOSE = 0 2026-04-01 18:08:58,531: xDoctor_4.8-109.0 - INFO : REPORT = 0 2026-04-01 18:08:58,646: xDoctor_4.8-109.0 - INFO : --------------------- 2026-04-01 18:08:58,646: xDoctor_4.8-109.0 - INFO : xDoctor Post Features 2026-04-01 18:08:58,646: xDoctor_4.8-109.0 - INFO : ---------------- 2026-04-01 18:08:58,646: xDoctor_4.8-109.0 - INFO : Data Combiner 2026-04-01 18:08:58,646: xDoctor_4.8-109.0 - INFO : ------------- 2026-04-01 18:08:58,647: xDoctor_4.8-109.0 - INFO : Created a Data Collection Report (data.xml) 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : ------ 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : SysLog 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : ------ 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : Using Fabric as Syslog Server 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : Not triggered ... no WARNING, ERROR, nor CRITICAL 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : ---- 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : SNMP 2026-04-01 18:08:58,648: xDoctor_4.8-109.0 - INFO : ---- 2026-04-01 18:08:58,649: xDoctor_4.8-109.0 - INFO : Using 10.118.165.48:162 as SNMP server 2026-04-01 18:08:58,649: xDoctor_4.8-109.0 - INFO : Not triggered .. no WARNING, ERROR nor CRITICAL 2026-04-01 18:08:58,649: xDoctor_4.8-109.0 - INFO : ------------ 2026-04-01 18:08:58,649: xDoctor_4.8-109.0 - INFO : ProcComplete 2026-04-01 18:08:58,649: xDoctor_4.8-109.0 - INFO : ------------ 2026-04-01 18:08:58,649: xDoctor_4.8-109.0 - WARNING : ProcComplete is disabled, please re-enable it (xdoctor --config) 2026-04-01 18:08:58,767: xDoctor_4.8-109.0 - INFO : ---------------- 2026-04-01 18:08:58,767: xDoctor_4.8-109.0 - INFO : Session Archiver 2026-04-01 18:08:58,768: xDoctor_4.8-109.0 - INFO : ---------------- 2026-04-01 18:08:58,777: xDoctor_4.8-109.0 - INFO : Session Stored in folder - /usr/local/xdoctor/archive/other/2026-04-01_180344 2026-04-01 18:08:58,777: xDoctor_4.8-109.0 - INFO : Session Archived as tar - /usr/local/xdoctor/archive/other/xDoctor-CKMXXXXXXXXXXX-2026-04-01_180344.tgz 2026-04-01 18:08:58,777: xDoctor_4.8-109.0 - INFO : -------------------------- 2026-04-01 18:08:58,777: xDoctor_4.8-109.0 - INFO : Session Report - sudo xdoctor --report --archive=2026-04-01_180344 2026-04-01 18:08:58,777: xDoctor_4.8-109.0 - INFO : --------------- 2026-04-01 18:08:58,777: xDoctor_4.8-109.0 - INFO : Session Cleaner 2026-04-01 18:08:58,777: xDoctor_4.8-109.0 - INFO : --------------- 2026-04-01 18:08:58,789: xDoctor_4.8-109.0 - INFO : Removing folder (count limit) - /usr/local/xdoctor/archive/other/2026-04-01_170120 2026-04-01 18:08:58,790: xDoctor_4.8-109.0 - INFO : Removing archive (count limit) - /usr/local/xdoctor/archive/other/xDoctor-CKMXXXXXXXXXXX-2026-04-01_170120.tgz 2026-04-01 18:08:58,793: xDoctor_4.8-109.0 - INFO : Cleaned 2 archived session(s) 2026-04-01 18:08:58,793: xDoctor_4.8-109.0 - INFO : ------- 2026-04-01 18:08:58,794: xDoctor_4.8-109.0 - INFO : Emailer 2026-04-01 18:08:58,794: xDoctor_4.8-109.0 - INFO : ------- 2026-04-01 18:08:58,794: xDoctor_4.8-109.0 - INFO : Using Dedicated Server (25:25) as SMTP Server ... 2026-04-01 18:08:58,794: xDoctor_4.8-109.0 - INFO : Email Type = Individual Events 2026-04-01 18:08:58,795: xDoctor_4.8-109.0 - INFO : ------------------------------ 2026-04-01 18:08:58,795: xDoctor_4.8-109.0 - INFO : xDoctor session_1775066624.943 finished in 0:05:13 2026-04-01 18:08:58,813: xDoctor_4.8-109.0 - INFO : Successful Job:1775066624 Exit Code:192
- 세션 보고서를 실행하여 수동 자동 복구 실행 결과를 검토합니다.
# sudo xdoctor --report --archive=<session report>
예:
admin@ecsnode1:~> sudo xdoctor --report --archive=2026-04-01_180344
xDoctor 4.8-109.0
CKMXXXXXXXXXXX - ECS 3.8.1.4
Displaying xDoctor Report (2026-04-01_180344) Filter:[] ...
--------------------------------------------------------
FIXED - Auto Healer fixed Dell switch connectivity issue
--------------------------------------------------------
Node = Nodes
Extra = {"Nodes": ["169.254.1.1"]}
Timestamp = 2026-04-01_180344
PSNT = CKMXXXXXXXXXXX @ 4.8-109.0
- 장애가 발생한 경우 조사를 위해 SRn을 엽니다.
실패의 예:---------------------------------------------------- ERROR - (Cached) Auto fix failed - Switch Connection Failure detected. ---------------------------------------------------- Node = 169.254.1.1 Extra = {"169.254.1.1": ["hare"]} RAP = RAP208 Solution = KB 39838 Timestamp = 2026-04-01_180132 PSNT = CKMXXXXXXXXXXX @ 4.8-109.0
xDoctor 자동 조종 장치:
이 KB(Knowledge Base)는 이제 xDoctor Auto Pilot으로 자동화되어 지원팀의 개입 없이도 대부분의 문제를 해결합니다.
이 기능은 xDoctor 4-8.104.0 이상에 기본적으로 적용되며 구문 및 사용 문제는 ECS를 참조하십시오. ObjectScale: KB 자동화 스크립트(자동 파일럿)를 실행하는 방법
랙의 마스터 노드를 찾으려면 다음을 수행합니다.
명령:
ssh master.rack
NAN IP를 찾으려면 알림에서 식별된 IP를 사용하거나 getrackinfo파일로 교체합니다.
명령:
admin@ecsnode1:~> getrackinfo
Node private Node Public BMC
Ip Address Id Status Mac Ip Address Mac Ip Address Private.4(NAN) Node Name
=============== ====== ====== ================= =============== ================= =============== =============== =========
192.168.219.1 1 MA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.101 169.254.1.1 provo-red
192.168.219.2 2 SA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.102 169.254.1.2 sandy-red
192.168.219.3 3 SA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.103 169.254.1.3 orem-red
192.168.219.4 4 SA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.104 169.254.1.4 ogden-red
192.168.219.5 5 SA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.105 169.254.1.5 layton-red
192.168.219.6 6 SA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.106 169.254.1.6 logan-red
192.168.219.7 7 SA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.107 169.254.1.7 lehi-red
192.168.219.8 8 SA 00:00:00:00:00 0.0.0.0 00:00:00:00:00 192.168.219.108 169.254.1.8 murray-red
- xDoctor 4-8.104.0 이상이 설치된 마스터 노드에서 자동화 명령을 실행합니다.
참고:
--target-rack 이 작업에 대해 지원됩니다.
# sudo xdoctor autopilot --kb 39838 --target-rack <rack_colour>
admin@ecsnode1:~> sudo xdoctor autopilot --kb 39838 --target-rack red
Checking for existing screen sessions...
Starting screen session 'autopilot_kb_39838_20250626_112318'...
Screen session 'autopilot_kb_39838_20250626_112318' started successfully.
Attaching to screen session 'autopilot_kb_39838_20250626_112318'...
Using /etc/ansible/ansible.cfg as config file
VERSION: 3.0
Playbook tasks: 47
Role tasks: 97
Total tasks: 144 across 1 host(s)
PLAY [red] ******************************************************************************************************************************************************************
Detected 8 hosts for this play.
TASK [target_check : set_fact] **********************************************************************************************************************************************
ok: [169.254.1.1 -> localhost] => {"ansible_facts": {"allowed_targets": "Please use: --target-rack", "target_node_check": false, "target_rack_check": true, "target_vdc_check": false}, "changed": false}
TASK [target_check : context] ***********************************************************************************************************************************************
skipping: [169.254.1.1] => {"changed": false, "false_condition": "node_script == false and target_node_check == true or rack_script == false and target_rack_check == true or vdc_script == false and target_vdc_check == true", "skip_reason": "Conditional result was False"}
...truncated
- 요약 검토:
예:
TASK [Print all summaries] **************************************************************************************************************************************************
ok: [169.254.1.1] => {
"msg": [
"*******************************************************************************",
"Switch xDoctor 'RAP073' password and SSH summary:",
"*******************************************************************************",
"Validated Frontend switch(es): FAIL: The passwords for the Dell managed switch(es) are incorrect and need to be configured in the xDoctor settings according to KB 39838.",
"Validated Backend switch(es): FAIL: The passwords for the Dell managed switch(es) are incorrect and need to be configured in the xDoctor settings according to KB 39838.",
"Validated Backend management connections: PASS: Management connections are up and connected to the frontend switches.",
"*******************************************************************************",
"Validated ssh keys to switch(es): PASS: All ssh keys are valid and nothing was corrected.",
"Validated xDoctor alert: PASS: Alert RAP073 was not present in xDoctor.",
"*******************************************************************************"
]
}
TASK [Set fact for context] *************************************************************************************************************************************************
ok: [169.254.1.1 -> localhost] => {"ansible_facts": {"context": " Validated Frontend switch(es): FAIL: The passwords for the Dell managed switch(es) are incorrect and need to be configured in the xDoctor settings according to KB 39838., Validated Backend switch(es): FAIL: The passwords for the Dell managed switch(es) are incorrect and need to be configured in the xDoctor settings according to KB 39838."}, "changed": false}
TASK [Fail if validation fails] *********************************************************************************************************************************************
fatal: [169.254.1.1]: FAILED! => {"changed": false, "msg": "Review the summary above for recommendations."}
NO MORE HOSTS LEFT **********************************************************************************************************************************************************
PLAY RECAP ******************************************************************************************************************************************************************
169.254.1.1 : ok=65 changed=13 unreachable=0 failed=1 skipped=73 rescued=0 ignored=1
169.254.1.2 : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
169.254.1.3 : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
169.254.1.4 : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
169.254.1.5 : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
169.254.1.6 : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
169.254.1.7 : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
169.254.1.8 : ok=4 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
=============================================================================================================================================================================
Status: FAIL
Time Elapsed: 0h 1m 25s
Debug log: /tmp/autopilot/log/autopilot_39838_20250626_113201.log
Message: Validated Frontend switch(es): FAIL: The passwords for the Dell managed switch(es) are incorrect and need to be configured in the xDoctor settings according to KB 39838., Validated Backend switch(es): FAIL: The passwords for the Dell managed switch(es) are incorrect and need to be configured in the xDoctor settings according to KB 39838.
=============================================================================================================================================================================
- xDoctor 암호를 업데이트합니다.
admin@ecsnode7:~> sudo xdoctor -c --expert
xDoctor Configuration Menu
--------------------------
[Expert Mode Active]
(1) Overview
(2) Scheduling
(3) Archiving
(5) Repository
(9) Miscellaneous
(0) Exit
Please make a choice: 9
xDoctor Miscellaneous
---------------------
(3) Switches
(4) Remove Hardware Alerting Timestamp
(0) Main menu
Please make a choice: 3
xDoctor Switch Settings
---------------------
Enable Switch Analysis? [Yes]:
Switches [hare,rabbit,fox,hound]:
Username [admin]:
Password [*****]:
[New Switch Settings]
Enabled = Yes
Switches = hare,rabbit,fox,hound
Username = admin
Password = *****
> Issue new settings? [No]: yes
2024-11-20 16:03:53,702: xDoctor_4.8-100.0 - INFO : Settings saved and distributed ...
xDoctor Miscellaneous
---------------------
(3) Switches
(4) Remove Hardware Alerting Timestamp
(0) Main menu
기본 KB 자동화:
ECS: xDoctor: 랩073: 스위치 연결 오류가 감지됨
이 자동화에 추가 KB 통합:
ECS: xDoctor가 known_hosts의 RSA 키로 인한 스위치 연결 실패를 보고함