PowerScale: Isilon: Quota Hard Limit Exceeded Scenarios

The Quota hard limits for any given path may exceed the designated quota for that path. The amount of data exceeding the hard limit may vary from a small to a large amount. 

Scenario 1:  
When multiple users simultaneously write to the same quota path, the network drive may present sufficient free space at the time the writes were started.

Where the data exceeds the hard quota after a pending write has already begun, the Isilon coalescer allows the pending write to complete. This exceeds the hard quota threshold.

Further writes are blocked. 

Example: 
Here is a shared path that has a directory quota with a 10.00M hard limit.

If two separate users with SMB or NFS access to that share begin copying different 8M datafiles at nearly the same time, both users succeed. 

The share contains 16M of data, exceeding the 10M hard quota limit. This is because both clients were presented with 10M of free space at the beginning of their respective copy process.  

Isilon-1# isi quota quotas view --path=/ifs/data/quota_test --type=directory
                        Path: /ifs/data/quota_test
                        Type: directory
                   Snapshots: No
                    Enforced: Yes
                   Container: No
                      Linked: No
                      Labels:
                 Description:
                       Usage
                           Files: 1
         Physical(With Overhead): 2.00k
        FSPhysical(Deduplicated): 2.00k
         FSLogical(W/O Overhead): 0.00
        AppLogical(ApparentSize): 0.00
                   ShadowLogical: -
                    PhysicalData: 0.00
                      Protection: 0.00
     Reduction(Logical/Data): None : 1
Efficiency(Logical/Physical): 0.00 : 1
                        Over: -
               Thresholds On: fslogicalsize
              ReadyToEnforce: Yes
                  Thresholds
                   Hard Threshold: 10.00M
                    Hard Exceeded: No
               Hard Last Exceeded: -
                         Advisory: -
    Advisory Threshold Percentage: -
                Advisory Exceeded: No
           Advisory Last Exceeded: -
                   Soft Threshold: -
        Soft Threshold Percentage: -
                    Soft Exceeded: No
               Soft Last Exceeded: -
                       Soft Grace: -

Scenario 2:  
A quota is placed on a directory path containing a preexisting amount of data which already exceeds the wanted hard Quota limit. 

Further writes activities are blocked once the quota is applied.    

Example: 
Here an 8.0M quota is placed on a directory that already contains 13.00M of data. Since the quota applied is less than the amount of data already in the path, the hard limit is exceeded. All further writes to that path are blocked. 

Isilon-1# isi quota quotas view --path=/ifs/data/quota_test --type=directory
                        Path: /ifs/data/quota_test
                        Type: directory
                   Snapshots: No
                    Enforced: Yes
                   Container: No
                      Linked: No
                       Usage
                           Files: 4
         Physical(With Overhead): 13.00M   <----  Preexisting data.
        FSPhysical(Deduplicated): 13.00M
         FSLogical(W/O Overhead): 10.58M
        AppLogical(ApparentSize): 10.57M
                   ShadowLogical: -
                    PhysicalData: 9.79M
                      Protection: 3.44M
     Reduction(Logical/Data): 1.08 : 1
Efficiency(Logical/Physical): 0.79 : 1
                        Over: 2.58M
               Thresholds On: fslogicalsize
              ReadyToEnforce: Yes
                  Thresholds
                   Hard Threshold: 8.00M  <---- Hard Quota is less then preexisting data
                    Hard Exceeded: Yes
               Hard Last Exceeded: 2023-04-19T03:05:32
                         Advisory: -
    Advisory Threshold Percentage: -
                Advisory Exceeded: No
           Advisory Last Exceeded: -
                   Soft Threshold: -
        Soft Threshold Percentage: -
                    Soft Exceeded: No
               Soft Last Exceeded: -
                       Soft Grace: - 

The scenarios described above are all working as designed.

Cluster Administrators should be aware of these situations and limitations when creating or modifying Quotas. Where the hard threshold is exceed and writes are denied, the Cluster Administrator must either remove data or increase the Hard Threshold.

The coalescer is a process that groups small writes to a file system into larger, more efficient writes. When a write operation occurs, the coalescer may delay and buffer it with other writes which are pending to the same file system. Once enough data has been buffered, the coalescer performs a single, larger write to the file system. It behaves like a cache.

If a write operation exceeds an enforced hard quota, the file system returns an error, typically EDQUOT (quota exceeded). If the coalescer delays the write and the buffered data is acknowledged before the write exceeded the quota, then the coalescer cannot return an EDQUOT error to the user. This is because the data has already been acknowledged.