This post is more than 5 years old
3 Posts
0
1998
January 13th, 2017 16:00
Isilon OneFS 8.0 - New events have stopped entering the log
Hello All:
I have a 21-node Isilon NL400 Cluster running OneFS 8 (8.0.0.037), joined to an AD domain. All new events on the cluster (except for Heartbeats) have stopped entered the 'Event Groups' log.
An 'isi status' indicates that the cluster is green, and all the nodes are green even though there are nodes that obviously need attention.
As workarounds: I can generate an isi_gather log and evaluate the cluster's health with the Isilon Advisor tool. The actionable issues appear red and orange as expected. I can also manually examine individual node issues (i.e. drives that have Smartfailed, etc.), in the cli and gui.
The 'Event Retention Settings' are set to their defaults (90-days, 1MB per TB).
Thanks in Advance.
Acedude1
3 Posts
0
February 13th, 2017 11:00
This was resolved by applying Patch #170489. Events resumed entering the log again.
mikekozar
2 Posts
0
January 17th, 2017 12:00
It sounds like one or more of your nodes are not updating CELOG as expected. Restarting that service is nondisruptive and easy on 8.0 - try running "isi services -a celog disable", waiting a minute, then reverse it with "isi services -a celog enable". Hopefully this will get all of your nodes on the same page again with no further fuss!
Acedude1
3 Posts
0
January 17th, 2017 13:00
Thanks for the response.
However, I have restarted the CELOG service. After no success, I restarted the cluster. Still no success.
I have seen this issue with the older OneFS 7.x (espcially large clusters), and have the process for reinitializing that CELOG database. But my understanding is that the CELOG has been redesigned with 8.0 and now there are multiple CELOG databases.