Cisco MDS: L'utilizzo dello storage su file di System Manager è inaspettatamente elevato al 100%

• Modifica delle zone (commit di zona)

• L'operazione di commit della zona non è riuscita e sono stati identificati i seguenti errori su un MDS-9513 che esegue NX OS 6.2(5) all'interno della stessa fabric:

  2018 Feb 17 23:27:06 Switch09 %SYSMGR-2-SERVICE_CRASHED: Service "zone" (PID 3632) hasn't caught signal 11 (core will be saved).
  2018 Feb 17 23:27:07 Switch09 %CALLHOME-2-EVENT: SW_CRASH zone in slot 7 crashed with crash type : stateful crash
  2018 Feb 17 23:27:32 Switch09 %SYSMGR-2-CORE_SAVE_FAILED: zip_copy_file_to_logflash: PID 28495 with message Failed to copy 0x701_zone_core.3632 to logflash. Error file error (srcerr 0x0 dsterr 0x807b001c) .
  ...
  2018 Feb 17 23:46:58 Switch09 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 30106 with message non-sysmgr(non-sysmgr) crashed, core will be saved .
  2018 Feb 17 23:46:59 Switch09 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 30119 with message non-sysmgr(non-sysmgr) crashed, core will be saved .
  2018 Feb 17 23:46:59 Switch09 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 30126 with message non-sysmgr(non-sysmgr) crashed, core will be saved .
  2018 Feb 18 00:02:24 Switch09 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 31135 with message non-sysmgr(non-sysmgr) crashed, core will be saved .
  2018 Feb 18 00:02:24 Switch09 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 31144 with message non-sysmgr(non-sysmgr) crashed, core will be saved .
  2018 Feb 18 00:02:24 Switch09 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 31151 with message non-sysmgr(non-sysmgr) crashed, core will be saved .
  .....
  2018 Feb 18 05:08:35 Switch09 %SYSMGR-3-VAR_SYSMGR_FULL: System Manager file storage usage is unexpectedly high at 100%. This may impact system functions.
  2018 Feb 18 05:11:35 Switch09 last message repeated 1 time

• Core dump generato

  `show cores`
  Module  Instance  Process-name     PID       Date(Year-Month-Day Time)
  ------  --------  ---------------  --------  -------------------------
  7       1         zone             3632      2018-02-17 23:27:06

• Il file system /var/sysmgr ha mostrato un utilizzo del 100%:

  `show system internal flash`
  Mount-on 1K-blocks Used Available Use% Filesystem
  ....
  /var/sysmgr 204800 204800 0 100 none

Non è stato possibile determinare la root cause a causa del core dump danneggiato.

È stato copiato il core dump file esistente dallo switch ed è stata eseguita la "cancellazione dei core" che ha cancellato il file system /var/sysmgr.

Esempio:

Switch09# clear cores
Switch09# show cores
Module Instance Process-name PID Date(Year-Month-Day Time)
------ -------- --------------- -------- -------------------------
Switch09#

Switch09# show system internal dir /var/sysmgr
./ 320
../ 220
core_handling.log 2883
ssh/ 40
non-sysmgr/ 40
sysmgr-subproc/ 40
xml/ 40
vsh/ 40
remote/ 100
tmp_logs/ 40
rev-gsync/ 40
startup-cfg/ 240
tmp/ 40
ftp/ 480
work/ 100
logs/ 40

Switch09# show system internal flash
Mount-on 1K-blocks Used Available Use% Filesystem
...
/var/sysmgr 204800 60 204740 1 none