In PowerScale OneFS 8.0 and later, a new feature is added that places the Clusterwide event log (CELOG) into maintenance mode to avoid receiving alerts or triggering Dial-Home service requests (SR) while tests or planned activities are being made on the Isilon cluster.
- Run the following command to place the CELOG into maintenance mode and discontinue receiving any alerts during the maintenance window:
# isi event setting modify --maintenance-start <timestamp> --maintenance-duration <integer><time>
For example, to set a two-hour maintenance window that starts on the February 23, 10pm:
# isi event settings modify --maintenance-start 2017-02-23T22:00:00 --maintenance-duration 2H
- If the test or maintenance activity is completed ahead of schedule or if the maintenance activity is canceled, the CELOG can be manually removed from maintenance mode:
# isi event settings modify --clear-maintenance-start
This can also be done in the WebUI:
- Click Cluster Management > Events and Alerts > Alert Management
Placing the CELOG in maintenance mode does not affect client activity or performance. The maintenance or test activity itself may affect client activity or performance, depending on the type of activity.
Upon the expiration of the maintenance window specified, the CELOG is automatically removed from maintenance mode.
For further details and options on placing the CELOG into maintenance mode, see the relevant CLI guide:
Note:
An issue is identified in PowerScaleOneFS 8.x and 9.x where events occurring during the maintenance windows can cause the cluster to send those events based on the alert configuration after the maintenance window ends, regardless if that event is resolved or not.
To work around this issue:
- Manually ignore and resolve the event groups that occurred during the maintenance activity, before the maintenance window ends. For example:
# isi event groups modify --id=<eventgroup ID> --resolved=true --ignore=true
- If you have manually ended the maintenance window or it has expired and you are now being spammed with old alerts, disable the alert channels. CELOG then works through processing the events but alerts are not sent. This may take several minutes to a few hours depending on how long maintenance mode has been enabled.
# isi event channels modify <channel name> --enabled=no
To find the event channel name, use:
# isi event channels list
Disabling the alert channel can also be done in the WebUI, the Enabled checkbox is at the top of the Edit Alert Channel display. See the
OneFS 9.x Administration Guide for more information.