Connectrix MDS: EXT3-fs Error (device sdc1): ext3_readdir: directory #2 contains a hole at offset 0 - kernel

Summary: Connectrix MDS switch streaming the following error: %KERN-2-SYSTEM_MSG: [63661120.723386] EXT3-fs error (device sdc1): ext3_readdir: directory #2 contains a hole at offset 0 - kernel.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.

Symptoms

Connectrix MDS switch streaming the following error:

%KERN-2-SYSTEM_MSG: [63661120.723386] EXT3-fs error (device sdc1): ext3_readdir: directory #2 contains a hole at offset 0 - kernel

The below is seen in the show logging output:

2016 Jul 26 19:54:22 Switch Jul 26 19:54:21 %KERN-2-SYSTEM_MSG: [63661120.680361] EXT3-fs error (device sdc1): ext3_readdir: directory #2 contains a hole at offset 0 - kernel
2016 Jul 26 19:54:22 Switch Jul 26 19:54:21 %KERN-2-SYSTEM_MSG: [63661120.702907] EXT3-fs error (device sdc1): ext3_readdir: directory #2 contains a hole at offset 0 - kernel
2016 Jul 26 19:54:22 Switch Jul 26 19:54:21 %KERN-2-SYSTEM_MSG: [63661120.723386] EXT3-fs error (device sdc1): ext3_readdir: directory #2 contains a hole at offset 0 - kernel

The below is seen in the show system internal flash:

`show system internal flash`
Mount-on                  1K-blocks      Used   Available   Use%  Filesystem
/                            409600     65564      344036     17   /dev/root
/proc                             0         0           0      0   proc
/sys                              0         0           0      0   none
/isan                       1572864    539352     1033512     35   none
/var                          51200       460       50740      1   none
/etc                           5120      1624        3496     32   none
/nxos/tmp                     51200       336       50864      1   none
/var/log                      51200      8992       42208     18   none
/var/home                      5120         0        5120      0   none
/var/tmp                     307200     11628      295572      4   none
/var/sysmgr                 2097152       124     2097028      1   none
/var/sysmgr/ftp              512000     51968      460032     11   none
/var/sysmgr/srv_logs         102400         0      102400      0   none
/var/sysmgr/ftp/debug_logs     10240         0       10240      0   none
/dev/shm                    1572864    234276     1338588     15   none
/volatile                    512000        12      511988      1   none
/debug                         2048        28        2020      2   none
/dev/mqueue                       0         0           0      0   none
/debugfs                          0         0           0      0   nodev
/mnt/plog                    242430      4458      225455      2   /dev/sdc1
/mnt/fwimg                   121219      4127      110833      4   /dev/sdc3
/mnt/cfg/0                    75797      4729       67154      7   /dev/sda5
/mnt/cfg/1                    75797      4729       67154      7   /dev/sda6
/logflash                  15392672     33064    14577684      1   /dev/sdb1
/bootflash                  3653168    244900     3222696      8   /dev/sda3
/cgroup                           0         0           0      0   vdccontrol
/var/sysmgr/startup-cfg      409600     10272      399328      3   none
/dev/pts                          0         0           0      0   devpts
/mnt/pss                      38156      5482       30704     16   /dev/sda4
/bootflash_sup-remote       3653168    244896     3222704      8   127.1.1.6:/bootflash/

`show module`
Mod  Ports  Module-Type                         Model              Status
---  -----  ----------------------------------- ------------------ ----------
2    48     2/4/8/10/16 Gbps Advanced FC Module DS-X9448-768K9     ok
3    48     2/4/8/10/16 Gbps Advanced FC Module DS-X9448-768K9     ok
4    48     2/4/8/10/16 Gbps Advanced FC Module DS-X9448-768K9     ok
5    0      Supervisor module-3                 DS-X97-SF1-K9      active *
6    0      Supervisor module-3                 DS-X97-SF1-K9      ha-standby

`show system redundancy status`
Redundancy mode
---------------
      administrative:   HA
         operational:   HA

This supervisor (sup-1)
-----------------------
    Redundancy state:   Active
    Supervisor state:   Active
      Internal state:   Active with HA standby

Other supervisor (sup-2)
------------------------
    Redundancy state:   Standby
    Supervisor state:   HA standby
      Internal state:   HA standby

 

Cause

The flash storage device is not mounted as per Cisco issue "CSCuv67115."

This issue initially occurs only on supervisors in standby state. The issue is not corrected by an ISSU/D, system switchover, or reload, and is observed on a supervisor in an active state.

Logflash is used only to store the output of debug commands. It is not required for stable system operation.

 

Resolution

Workaround:
Power cycle the affected supervisor.
In dual-supervisor systems, this is done by powering down the supervisor with the out-of-service module command and powering it back up with the no poweroff module command. Ensure that the supervisor is in standby mode so that the recovery does not affect the stability of the system.

It is not sufficient to reboot the module with the reload or system switchover commands.

 

Example:
Commands:

  1. Confirm that the standby supervisor is in HA Standby:

    show system redundancy status
  2. Power down the standby supervisor module:

    out-of-service module <standby sup>
  3. Power up the standby supervisor:

    no poweroff <standby sup>
  4. Wait for HA Standby:

    show system redundancy status

Monitor the switch show logging output to see if the EXT3-fs error message is still logging on the switch. If the error is still logging, switchover to the standby supervisor and repeat steps 1-4 above.

 

Affected Products

Connectrix MDS-Series Hardware

Products

Connectrix MDS-9706, Connectrix MDS-9710, Connectrix MDS-9718, Connectrix MDS-Series Hardware
Article Properties
Article Number: 000064345
Article Type: Solution
Last Modified: 17 Sep 2024
Version:  4
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.