PowerStore: Workload between the nodes is not balanced
Summary: This article describes the PowerStore alert for "Workload between the nodes is not balanced."
Symptoms
| IMPORTANT |
|---|
|
This RxDefinitions check may be ignored in the absence of other factors indicating the existence of an underlying issue.
In the majority of instances where this check is getting triggered, system time is different between the nodes. Follow the steps below to mitigate issue:
In the absence of an NTP/system time issue, please refer to the Resolution section of this knowledge article and review:
|
A health check in the RxDefinitions package detects this issue. The alert 0x03700103 was raised for:
"Workload between the nodes is not balanced."
Cause
Any I/O workload on a PowerStore volume occurs over one of the two PowerStore nodes. When there is a significant difference between the I/O workloads across both these nodes, performance problems or Data Availability issues can occur. This alert is raised when ALL of the following conditions are met:
- Both the nodes are UP for the last 24 hours.
- At least one of the nodes has an average daily total bandwidth of minimum 500 MB/s.
- The difference in the average daily total bandwidth between both the nodes exceeds 300%.
Resolution
To compare system performance by Node from PowerStore Manager, navigate to:
Hardware -> Appliances -> <Appliance Name> -> Performance
From the Performance Tab, select:
- Layout -> Two Columns
- BaseEnclosure-NodeA -> Checked
- BaseEnclosure-NodeB -> Checked
Below is an example view of total bandwidth on each PowerStore Node.
As noted previously, this RxDefinition check compares the average daily total bandwidth. Please note that the example given here is not meant to represent an instance that the check might flag.
Recommendations:
In PowerStore, each Volume is assigned affinity to a Node when it is presented to a Host or Host Group. Each PowerStore Node then is assigned the Active/Optimized paths for handling of the corresponding volume I/O workload.
If the I/O workload between nodes becomes out of balance, Volume Node Affinity is the primary tool that can be leveraged to resolve this, assuming the system does not have other factors that need remediation.
Before considering changes to Volume affinity, we recommend reviewing system health and best practice documentation
- Review Active Alerts
- From PowerStore Manager navigate to Monitoring -> Alerts
- Run System Checks
- From PowerStore Manager navigate to Monitoring -> System Checks -> Run System Checks
- Review PowerStore: Info Hub where Host Connectivity documentation is available
To review and change Volume Node Affinity, refer to KB 000194291: PowerStore: How to change volume node affinity
Contact Technical Support for further assistance. Mention this KB article ID to expedite processing.
Additional Information
See KB 000193518 PowerStore: Landing for Page RxDefinitions Issues for more information of the health check mechanism, the RxDefinitions package, and a complete list of RxDefinitions alerts.