IDPA: Nodehendelsestjenesten er i degradert på grunn av en duplikatrute 169.254.0.1
Summary: Det hurtigbufrede svaret med nodehendelsestjenesten er deaktivert. Nodehendelsestjenesten er i degradert på grunn av en duplikatrute 169.254.0.1.
Symptoms
Feil på RUCK-forhåndskontroll firmware_readiness:
[ERROR] Firmware pre-upgrade checks failed. [ 10.100.XX.XXX ]: The cached response with Node Event Service is disabled.
Node Event Service is in a degraded state,iDRAC Service Module is not available/active at this time.
Check iDRAC Service Module/iDRAC status.
Når du kjører under kommandoen, vil idraccache feile eller degraderes.
[root@idpa-esx1:~] /opt/dell/DellPTAgent/tools/pta_call get agent/info
Request sent to DellPTAgent @ https://192.168.100.101:8086
{
"libstorelibit.so": "07.05",
"uptime": "177266 seconds ( 2 days 1 hour 14 minutes 26 seconds )",
"host_pass_thru_ip": "169.254.0.2",
"ism_version": "3.6.0",
"model": "R640 IDPA",
"os_version": "6.7.0 build-17700523",
"process_id": "2101654",
"libstorelibir-3.so": "15.03-0",
"default_server_cert": "true",
"TPM Present": "false",
"MarvellLibraryVersion": "5.0.13.1109",
"servicetag": "D3WCCD3",
"mfr": "Dell Inc.",
"system_uuid": "62dcf7ee-334e-8f96-f507-78ac4426a310",
"status": {
"iSM": "N/A",
"agent": "Error/degraded",
"idracConnection": "OK",
"idraccache": "Error/degraded"
iSM kjører, men kan ikke kommunisere med iDRAC:
PTAgent feilsøkingslogger /scratch/log/pta_debug.log
2023/10/23 14:29:33[UTC] [19394369:246258496] WARN - WSManClient::isValidResponse: Http request to host: 169.254.0.1, failed with status code: -4
2023/10/23 14:29:34[UTC] [19394369:246786880] WARN - WSManClient::isValidResponse: Http request to host: 169.254.0.1, failed with status code: -4
2023/10/23 14:29:34[UTC] [19394369:246258496] WARN - ISMMonitor::isISMServiceRunning: Command to check iSM status failed with error code <2>. <ism is active (running limited functionality)
Cause
Denne ekstra ruten opprettes på verts-OS-laget og er basert på eksisterende nettverkskonfigurasjon. Denne virkemåten oppstår ofte når flere statiske ruter er definert i ESXi-konfigurasjonen.
Mulige årsaker omfatter bruk av en intern SSH-tunnel fra ESXi til iDRAC.
iSM oppretter ikke flere ruter, og oppdager og reparerer heller ikke noen ekstra definerte ruter.
Resolution
To check this ptagent run:
./goidpa esx ptagent check
To fix issues with ptagent run:
./goidpa esx ptagent fix
Additional Information
vmk0 bare. Fjerne en annen rute fra vmk0 eller fjerne en 169.254.xx.xx-rute fra en annen vmk kan forårsake andre rutingsproblemer.
- Bekreft en ugyldig oppføring i ESXi-rutingstabellen, som forhindrer at iSM bruker 169.254.0.2 til å kommunisere med iDRACs 169.254.0.1.
Et eksempel på et nettverksmiljø som opplever dette problemet, vises nedenfor. Informasjonen i vmk0 grensesnittets gateway-adresse for en ekstra rute for nettverk 169.254.0.1/32.
[root@idpa-esxi:~] esxcli network ip route ipv4 list
Network Netmask Gateway Interface Source
-------------- --------------- ------------- --------- ------
default 0.0.0.0 10.100.10.254 vmk0 MANUAL
10.100.10.0 255.255.255.0 0.0.0.0 vmk0 MANUAL
169.254.0.0 255.255.255.0 0.0.0.0 vmk2 MANUAL
169.254.0.1 255.255.255.255 10.100.xx.xxx vmk0 MANUAL
192.168.100.96 255.255.255.224 0.0.0.0 vmk1 MANUAL
- Fjern oppføringen for å løse rutingsproblemet mellom iSM og iDRAC.
esxcli network ip route ipv4 remove -g 10.100.xx.xxx -n 169.254.0.1/32
- Oppføringen er fjernet:
[root@idpa-esxi:~] esxcli network ip route ipv4 list
Network Netmask Gateway Interface Source
-------------- --------------- ------------- --------- ------
default 0.0.0.0 10.100.10.254 vmk0 MANUAL
10.100.10.0 255.255.255.0 0.0.0.0 vmk0 MANUAL
169.254.0.0 255.255.255.0 0.0.0.0 vmk2 MANUAL
192.168.100.96 255.255.255.224 0.0.0.0 vmk1 MANUAL
- Når den ekstra ruten er fjernet, starter du PTAgent og iSM-tjenesten på nytt.
/etc/init.d/DellPTAgent restart
/etc/init.d/dcism-netmon-watchdog restart
- Kjør fastvareberedskap for å bekrefte at VELLYKKET
DP4400:
curl -k -i -H "Content-Type:application/json" -X POST https://localhost:8039/dpatools/api/v1/firmware/readinesscheck -d '{"idpaVersion": "2.7", "isRack": false, "hostList": [{"hostIP": "192.168.100.101", "esxiUser": "root", "esxiPassword": "IDPAPASSWORD"}]}'
IDPAPASSWORD = IDPA common password
Eksempel:
curl -k -i -H "Content-Type:application/json" -X POST https://localhost:8039/dpatools/api/v1/firmware/readinesscheck -d '{"idpaVersion": "2.7", "isRack": false, "hostList": [{"hostIP": "192.168.100.101", "esxiUser": "root", "esxiPassword": "Idpa_12345"}]}'
Ytelse:
HTTP/1.1 200
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, PUT, GET, OPTIONS, DELETE
Access-Control-Max-Age: 3600
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept
Content-Type: application/json
Transfer-Encoding: chunked
Date: Sun, 05 Nov 2023 01:38:41 GMT
[{"id":814402494717874004,"hostIP":"192.168.100.101","result":"SUCCESS","agentIdracDataCached":true,"agentIsOverallReady":true,"agentIsRunning":true,"agentVersionIsOk":true,"idracInRecoveryMode":false,"idracIsAvailable":true,"idracIsReady":true,"idracLcJobQueueIsClear":true,"ismIsEnabled":true,"ismIsReady":true,"firmwareIsValid":true,"twoHopIsRequired":false,"invalidFirmwareList":[],"messages":"The IDPA system is ready for firmware update.","links":[]}]
DP5x00/DP8x00:
for i in {1..3}; do curl -k -i -H "Content-Type:application/json" -X POST https://localhost:8039/dpatools/api/v1/firmware/readinesscheck -d '{"idpaVersion": "2.7", "isRack": true, "hostList": [{"hostIP": "192.168.100.10'${i}'", "esxiUser": "root", "esxiPassword": "IDPAPASSWORD"}]}'; done
IDPAPASSWORD = IDPA common password
Eksempel:
for i in {1..3}; do curl -k -i -H "Content-Type:application/json" -X POST https://localhost:8039/dpatools/api/v1/firmware/readinesscheck -d '{"idpaVersion": "2.7", "isRack": true, "hostList": [{"hostIP": "192.168.100.10'${i}'", "esxiUser": "root", "esxiPassword": "Idpa_12345"}]}'; done
Ytelse:
HTTP/1.1 200
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, PUT, GET, OPTIONS, DELETE
Access-Control-Max-Age: 3600
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept
Content-Type: application/json
Transfer-Encoding: chunked
Date: Sun, 05 Nov 2023 01:39:05 GMT
[{"id":2337625064092707360,"hostIP":"192.168.100.101","result":"SUCCESS","agentIdracDataCached":true,"agentIsOverallReady":true,"agentIsRunning":true,"agentVersionIsOk":true,"idracInRecoveryMode":false,"idracIsAvailable":true,"idracIsReady":true,"idracLcJobQueueIsClear":true,"ismIsEnabled":true,"ismIsReady":true,"firmwareIsValid":false,"twoHopIsRequired":false,"invalidFirmwareList":[{"name":"Integrated Remote Access Controller","currentVersion":"3.36.103.36","minimumRequiredVersion":"4.40.10.00"}],"messages":"The IDPA system is ready for firmware update.","links":[]}]HTTP/1.1 200
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, PUT, GET, OPTIONS, DELETE
Access-Control-Max-Age: 3600
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept
Content-Type: application/json
Transfer-Encoding: chunked
Date: Sun, 05 Nov 2023 01:39:44 GMT
[{"id":1389413431580575643,"hostIP":"192.168.100.102","result":"SUCCESS","agentIdracDataCached":true,"agentIsOverallReady":true,"agentIsRunning":true,"agentVersionIsOk":true,"idracInRecoveryMode":false,"idracIsAvailable":true,"idracIsReady":true,"idracLcJobQueueIsClear":true,"ismIsEnabled":true,"ismIsReady":true,"firmwareIsValid":true,"twoHopIsRequired":false,"invalidFirmwareList":[],"messages":"The IDPA system is ready for firmware update.","links":[]}]HTTP/1.1 200
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, PUT, GET, OPTIONS, DELETE
Access-Control-Max-Age: 3600
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept
Content-Type: application/json
Transfer-Encoding: chunked
Date: Sun, 05 Nov 2023 01:40:19 GMT
[{"id":1184184063038749065,"hostIP":"192.168.100.103","result":"SUCCESS","agentIdracDataCached":true,"agentIsOverallReady":true,"agentIsRunning":true,"agentVersionIsOk":true,"idracInRecoveryMode":false,"idracIsAvailable":true,"idracIsReady":true,"idracLcJobQueueIsClear":true,"ismIsEnabled":true,"ismIsReady":true,"firmwareIsValid":true,"twoHopIsRequired":false,"invalidFirmwareList":[],"messages":"The IDPA system is ready for firmware update.","links":[]}]
Kontroller hurtigbufferstatusen ved å kjøre kommandoen nedenfor:
/opt/dell/DellPTAgent/tools/pta_call get agent/info
Ytelse:
Request sent to DellPTAgent @ https://192.168.100.101:8086
{
"mfr": "Dell Inc.",
"servicetag": "D3WCCD3",
"host_epoch_time": "1699148602.56383 (secs.usecs)",
"uptime": "204355 seconds ( 2 days 8 hours 45 minutes 55 seconds )",
"system_uuid": "62dcf7ee-334e-8f96-f507-78ac4426a310",
"model": "R640 IDPA",
"process_id": "2101654",
"libstorelibir-3.so": "15.03-0",
"domain": "esx1-5800-crk.dp.ce.gslabs.lab.emc.com",
"name": "esx1-5800-crk",
"ptagentversion": "2.4.1-3",
"idrac_ethernet_ip": "192.168.100.110",
"os_version": "6.7.0 build-17700523",
"ism_version": "3.6.0",
"MarvellLibraryVersion": "5.0.13.1109",
"libstorelib.so": "07.07",
"host_pass_thru_ip": "169.254.0.2",
"default_server_cert": "true",
"status": {
"iSM": "N/A",
"agent": "OK",
"idraccache": "OK",
"idracConnection": "OK"
},
"idrac_pass_thru_ip": "169.254.0.1",
"api_blocking_enabled": "false",
"os": "VMWare ESXi",
"rest_endpoints": "https://192.168.100.101:8086",
"TPM Present": "false",
"libstorelibit.so": "07.05"
}
Response: status: 200 [OK], size: 1063 bytes, latency: 0.144 seconds.