VNX: server_df command fails

The command 'server_df server_2' fails with "Error 2: server_2 : No such file or directory failed to complete command":

[nasadmin@ssgnas03cs0 ~]$ server_df server_2
server_2 :
Error 2: server_2 : No such file or directory
failed to complete command

Pre-upgrade health checks fail with the following errors:

Blades : Check if root filesystem is too large
Error HC_DM_14505148436: Output from command "/nas/bin/server_df
         server_2" has no information on file system root_fs_2

Blades : Check if root filesystem has enough free space
Error HC_DM_14505148440: Output from command "/nas/bin/server_df
         server_2" has no information on file system root_fs_2

Blades : Check if file system usage is under limit
Error HC_DM_14505148563: Could not determine available disk space for
         the mounted file systems on the following Blades:
         * server_2

All other server_df commands work successfully against other data movers:

[nasadmin@ssgnas03cs0 ~]$ server_df server_3
server_3 :
Filesystem          kbytes         used        avail capacity Mounted on
root_fs_common       15368         5280        10088   34%    /.etc_common
root_fs_3           258128         6184       251944    2%    /

The root cause in this instance was that the root file system for server_2 was unmounted.

[nasadmin@ssgnas03cs0 ~]$ server_mount ALL
server_2 :
                       <------ nothing mounted
server_3 :
root_fs_3 on / uxfs,perm,rw
root_fs_common on /.etc_common uxfs,perm,ro

When you list all file systems, it was clear that the root file system 'root_fs_2' for server_2 was listed as not in use because it was unmounted.

[nasadmin@ssgnas03cs0 ~]$ nas_fs -l
id      inuse type acl   volume    name                server
1         n    1   0     10        root_fs_1
2         y    1   0     40        root_fs_common      3,2
3         n    5   0     73        root_fs_ufslog
4         n    5   0     76        root_panic_reserve
5         n    5   0     93        root_fs_d3
6         n    5   0     94        root_fs_d4
7         n    5   0     95        root_fs_d5
8         n    5   0     96        root_fs_d6
9         n    1   0     12        root_fs_2              <------------ inuse = n
10        y    1   0     14        root_fs_3           2

Prior to the customer requesting assistance from Support, a data mover reboot was completed and no improvement was seen. To resolve this issue, login as 'root' user to the Primary Control Station and re-mount the file system 'root_fs_2' to the data mover server_2:

[root@ssgnas03cs0 nasadmin]# server_mount server_2 root_fs_2 /
server_2 : done

Ensure that the root file system is re-mounted:

[root@ssgnas03cs0 nasadmin]# server_mount ALL
server_2 :
root_fs_2 on / uxfs,perm,rw
root_fs_common on /.etc_common uxfs,perm,ro

server_3 :
root_fs_3 on / uxfs,perm,rw
root_fs_common on /.etc_common uxfs,perm,ro

After the root file system is mounted, issue the 'server_df server_2' command to ensure that it completes successfully:

[root@ssgnas03cs0 nasadmin]# server_df server_2
server_2 :
Filesystem          kbytes         used        avail capacity Mounted on
root_fs_common       15368         5280        10088   34%    /.etc_common
root_fs_2           258128         6264       251864    2%    /

In this particular instance, the customer was performing a cleanup of unnecessary file systems to reclaim space for the NAS pool. As part of the cleanup, they permanently unmounted and attempted to delete the root file system 'root_fs_2' for the data mover server_2. The root file system for a data mover should never be deleted.