Node Pools at Capacity

IHAC that inherited a cluster with 2 node pools... One of the node pools is at 93% capacity and the second is at 100% capacity.. I am assuming they are still working due to spillover.. Other than add new nodes is there any tricks to get some ofthat capacity back? They have deleted files, however it appears no jobs that would reclaim that space will run.. I am auming if you have no where to move blocks they stay where they are...

Any help will be greatly appreciated.. So far the only way to get this working properly is to add nodes..

Thanks

Shewby

Responses(2)

A

Anonymous

5 Practitioner

•

274.2K Posts

0

December 26th, 2013 12:00

Bernie,

Let me look into the snapshot angle.. The resolution to this other than add nodes are really limited.

Thanks for the response..

If I can be of any further assistance, please do not hesitate to contact me…

Chris Shewbridge (Shewby)

Mid-market Systems Engineer Specialist

Cell: 410-404-4137

Blog: www.storagewarrior.typepad.com

BernieC

76 Posts

0

December 26th, 2013 12:00

Hi Shewby,

There are a number of files in /ifs/data/Isilon_Support that may consume some space, but probably not all that much.

How many snapshots do you happen to have on this cluster? It's probable that some amount of space you have on the cluster has been consumed by snapshot data. Depending on your version of OneFS, you can run some commands to make that determination:

OneFS 7.x:

isi snapshot snapshot list -v --format=table --sort=path 
isi snapshot snapshot list --state deleting -v --format=table --sort=id

OneFS 6.5 and earlier:

isi snapshot ls -l
isi snap usage

I would suggest taking a look at the Best Practices for Maintaining Enough Free Space guide we have on the support site for some longer-term solutions. Support has a troubleshooting guide internally that they use for resolving these sorts of situations, and it usually involves a combination of data deletions (which you've already tried) and snapshot deletions, before trying to rebalance data between pools or trying to add space. If you aren't able to resolve this on your own, from the hints in this post, feel free to open a support case.

Thanks,
Bernie

View All

No Events found!