This post is more than 5 years old
17 Posts
2
6365
October 9th, 2014 06:00
InsightIQ stopping when low on free disk
Hi
InsightIQ stops when there are less that 10% free disk space on the cluster. Is there a way to tell InsightIQ to continue running?
Thanks and kind regards
Henrik Ravn
No Events found!
peglarr
2 Intern
•
99 Posts
0
October 13th, 2014 07:00
From the IIQ documentation:
InsightIQ begins deleting older data from the datastore when the datastore becomes 92% full. If InsightIQ is unable to delete data from the datastore, and the datastore continues to grow, InsightIQ will stop monitoring clusters when the InsightIQ datastore becomes 98% full. If this happens, you must manually delete data from the datastore before InsightIQ can continue monitoring clusters. InsightIQ is unable to automatically delete the following types of data from the datastore, and you must delete these types of data manually.
Cheers
Rob
dynamox
11 Legend
•
20.4K Posts
•
87.4K Points
0
October 13th, 2014 06:00
my IIQ was having this issue but it was not because of the datastore space utilization but because it was running out of inodes on the local file system. IIQ is buggy and does not properly delete old sessions, there is no fix either ..had to write a script that runs daily and deletes old sessions.
peglarr
2 Intern
•
99 Posts
0
October 13th, 2014 06:00
Hi Henrik,
Not quite sure what you mean...can you elaborate?
1) IIQ doesn't run on OneFS - it uses the results of the FSA job.
2) IIQ may be configured to use an NFS export from OneFS as its database store. Is this the case?
3) There is nothing 'automatic' about FSA shutting down when free space decreases below 10%.
If you could elaborate on what is actually happening, that would be super. Thanks
Cheers
Rob
Henrik_Ravn
17 Posts
0
October 13th, 2014 06:00
Hi Rob
When the datastore used for IIQ data is having less than 10% free Space, IIQ stops with the error:
"Datastore Full
InsightIQ datastore does not have sufficent space to continue monitoring."
There is more that 50 TB free on the datastore (but less than 10% of the total amount).
Best regards
Henrik
dynamox
11 Legend
•
20.4K Posts
•
87.4K Points
1
October 13th, 2014 07:00
nope, for me IIQ would simply halt ..i could not even get to the application web page. I also store IIQ data on an NFS export on Isilon, that NFS export that i created for IIQ i did not set any quota on it.
Henrik_Ravn
17 Posts
0
October 13th, 2014 07:00
Hi Dynamox
Did you also get the error message:
"Datastore Full
InsightIQ datastore does not have sufficent space to continue monitoring."
in IIQ?
Regards
Henrik
Henrik_Ravn
17 Posts
0
October 13th, 2014 23:00
Ok, thanks.
I hoped that there was some configuration where we could tell IIQ to continue above 92%. The amount of data that IIQ writes to disk is small compared to the 50 TB free Space.
Kind regards
Henrik
Peter_Sero
6 Operator
•
1.2K Posts
0
October 14th, 2014 00:00
Henrik
with OneFS SmartQuotas you can hide the actual cluster usage (90%) from any NFS client, such as the IIQ server here.
Say IIQ has actually 1 TB of data on the data space, might be much less. One the Isilon, create a directory quota of 10 TB for the IIQ dataspace and set the "container" flag.
The IIQ server will then see the NFS mount as: 10 TB total size, 1 TB = 10% used, 90% available, making IIQ very
.
Needs a SmartQuota licen$e...
-- Peter
Henrik_Ravn
17 Posts
0
October 14th, 2014 00:00
Great - I will try this, has license for quota.
Henrik
Henrik_Ravn
17 Posts
0
October 14th, 2014 02:00
Hi Peter
Just made the quota configuration, and IIQ is up and running
Quota is 10 TB and the amount of data that IIQ is using is only around 15 GB.
Thanks
Henrik
Peter_Sero
6 Operator
•
1.2K Posts
0
October 14th, 2014 04:00
Super!