Dell EMC Unity: Under certain file system out-of-space conditions, an upgrade to 5.1 code may trigger a file system to go offline or rolling SP reboots (User Correctable)

Summary: Under certain out-of-space conditions, an upgrade to 5.1 may trigger file system offline or rolling SP reboots.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

In a user array with version before 5.1 release, and there is a file system which is out of inodes, then does NDU to 5.1 release, user encounters SP rolling panics.

In a user array with version before 5.1 release, and there is a file system whose metadata space is used up, then does NDU to 5.1 release, user encounters file system offline.

Cause

This issue is due to a file system not being able to create the "OrphanDir" during mount operation (file systems are remounted when SP comes up after reboot). OrphanDir creation is a new feature in 5.1, and it uses 13 inodes.

There are two scenarios the OrphanDir creation may fail:
  • If a file system does not have enough inodes available, SP rolling panics.
  • If a file system does not have enough metadata subspace, file system is marked offline in UI or UEMCLI. 

Resolution

Fix:
A fix is being created. Watch this Knowledgebase article for more information.

Workaround:
Before an upgrade, a Unity system can be checked to ensure that there are appropriate inodes and metadata space available. See the Additional Information field below to see the commands to use to check for available inode and metadata space before an upgrade.

If a file system is found in this condition, extending the file system corrects the condition and add enough space to allow for the upgrade. 
  • Extending the file system by at least 10 GB should be sufficient.  
  • Once the file system is extended, create a file in the file system. This ensures that the inode slice is added to the file system.
Once you implement the workaround, check the space again and ensure that there is sufficient inode space (at least 13 free inodes) and metadata space before proceeding with the upgrade.

Additional Information

To check if there is free inode in the file systems, run the following command in a shell session on your Unity:  
svc_storagecheck -s | grep -A1000 inodes

19:21:42 service@Unity spa:~/user# svc_storagecheck -s | grep -A1 inodes
1630610514: ADMIN: 6:  Filesystem      inodes         used         avail   capacity Mounted on
1630610514: ADMIN: 6:   536870913     2331376           93       2331283         0% /
--
1630610514: ADMIN: 6:  Filesystem      inodes         used         avail   capacity Mounted on
1630610514: ADMIN: 6:   536870916     1291504           16       1291488         0% /test1
--
1630610514: ADMIN: 6:  Filesystem     inodes          used         avail   capacity Mounted on
1630610514: ADMIN: 6:   536870920     771568            19        771549         0% /testfs2
--
1630610514: ADMIN: 6:  Filesystem     inodes          used         avail   capacity Mounted on
1630610514: ADMIN: 6:   536870922     771568            82        771486         0% /.etc

A Unity system experiencing this issue shows a file system similar to the below: 
1630610514: ADMIN: 6:  Filesystem      inodes         used         avail   capacity Mounted on
1630610514: ADMIN: 6:   536870916     1291504      1291504             0       100% /test1

There are no available inodes, and the file system is at 100% inode capacity.

To check if there is enough metadata subspace in the file systems, reach out to Unity support to run an internal script with root access.

Affected Products

Dell EMC Unity
Article Properties
Article Number: 000191205
Article Type: Solution
Last Modified: 29 Apr 2022
Version:  8
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.