PowerFlex: Unable to create volumes when a single SDS is in the IMM
Summary: Unable to create volumes when a single SDS is in the Instant Maintenance mode (IMM)
Symptoms
After putting a single SDS in the Maintenance Mode, it is not possible to create a volume, even if there is enough space available. The "add_volume" command fails with "System capacity is unbalanced" error message. Unable to create new volumes.
Unable to add a volume when an SDS is in the IMM even if the "--override_balance_check" flag is used: scli --add_volume --protection_domain_name pd1 --storage_pool_name sp1 --thin_provisioned --size_gb 8 --volume_name test --override_balance_check Allocating volumes while the system is unbalanced will degrade system performance. Press 'y' and then Enter to confirm: y Error: MDM failed command. Status: System capacity is unbalanced
Cause
This is not a bug. Under specific circumstances it may be impossible to create new volumes.
A 'normal' ScaleIO environment must have at least three (3) similar-sized fault units, where the largest fault unit's capacity is less than half of the total available capacity (excluding the MM node capacity).
Sometimes the ScaleIO system is created in an unbalanced manner.
An example of an unbalanced ScaleIO system is when a single fault unit (SDS or Fault Set) is bigger than the other components.
Below are some problematic configurations:
Example1 (FaultSets):
FS0: SDSs 111,112
FS1: SDS 113
FS2: SDSs 114 (MM), 115
Each SDS is the same or similar size. Under this circumstance, when SDS 114 is in MM, FS0 is twice as bigger as the other FaultSets.
Here, volume creation fails, as the system is at its limit (a single fault unit cannot be larger than the rest of the system - and here it is equal to):
Example2 (SDS):
SDS1: 18 TB
SDS2: 18 TB
SDS3: 18 TB
SDS4: 100 GB
In this environment, one of the three remaining fault units (SDS node 3) is 180 times smaller than the other ones. Moreover, since the difference is so big, the SDS at node 4 is set to contain a small amount of storage, so this environment is almost an equivalent of a 2-node environment, which is not sufficient.
Resolution
Make sure the ScaleIO system is balanced, and that every fault unit (FS or SDS) is similar in size.