ISILON: Event Notification: NODE_AGGREGATE_LINK_DOWN in Smart Connect environment.
Summary: When a dynamic IP address moves, a NODE_AGGREGATE_LINK_DOWN event is raised. A dynamic IP address moves when node rebooting or interface failure. If Rebalance Policy is manual, Dynamic IP address will not rebalance. In this case, A NODE_AGGREGATE_LINK_DOWN event remains until issue manual rebalance. ...
This article applies to
This article does not apply to
This article is not tied to any specific product.
Not all product versions are identified in this article.
Symptoms
Case of dynamic IP address moving.
Example of Normal state
As shown below, xxx.xxx.xxx.71 to 74 are assigned to each node in groupnet0.subnet_main_service.pool_main_service_dynamic
# isi network interfaces list
LNN Name Status VLAN ID Owners Owner Type IP Addresses
------------------------------------------------------------------------------------------------------------------------
6 10gige-1 Up - - - -
6 10gige-2 Up - - - -
6 10gige-agg-1 Up - - - -
6 10gige-agg-1 Up 310 groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xx.231
6 10gige-agg-1 Up 117 groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic xxx.xxx.xxx.74
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.81
6 mgmt-1 No Carrier - - - -
7 10gige-1 Up - - - -
7 10gige-2 Up - - - -
7 10gige-agg-1 Up - - - -
7 10gige-agg-1 Up 310 groupnet0.subnet_main_backup SSIP xxx.xxx.xxx.250
groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xxx.232
7 10gige-agg-1 Up 117 groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic xxx.xxx.xxx.72
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.82
7 mgmt-1 No Carrier - - - -
8 10gige-1 Up - - - -
8 10gige-2 Up - - - -
8 10gige-agg-1 Up - - - -
8 10gige-agg-1 Up 310 groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xxx.233
8 10gige-agg-1 Up 117 groupnet0.subnet_main_service SSIP xxx.xxx.xxx.100
groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic xxx.xxx.xxx.71
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.83
8 mgmt-1 No Carrier - - - -
9 10gige-1 Up - - - -
9 10gige-2 Up - - - -
9 10gige-agg-1 Up - - - -
9 10gige-agg-1 Up 310 groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xxx.234
9 10gige-agg-1 Up 117 groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic xxx.xxx.xxx.73
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.84
9 mgmt-1 No Carrier - - - -
Example of an abnormal state
When node6 is rebooted for maintenance. The IP address assigned to the node moves to another node.
The IP address xxx.xxx.xxx.74 of node 6 moves to node 7
# isi network interfaces list
LNN Name Status VLAN ID Owners Owner Type IP Addresses
------------------------------------------------------------------------------------------------------------------------
6 10gige-1 Up - - - -
6 10gige-2 Up - - - -
6 10gige-agg-1 Up - - - -
6 10gige-agg-1 Up 310 groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xxx.231
6 10gige-agg-1 Up 117 groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic -
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.81
6 mgmt-1 No Carrier - - - -
7 10gige-1 Up - - - -
7 10gige-2 Up - - - -
7 10gige-agg-1 Up - - - -
7 10gige-agg-1 Up 310 groupnet0.subnet_main_backup SSIP xxx.xxx.xxx.250
groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xxx.232
7 10gige-agg-1 Up 117 groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic xxx.xxx.xxx.72 xxx.xxx.xxx.74
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.82
7 mgmt-1 No Carrier - - - -
8 10gige-1 Up - - - -
8 10gige-2 Up - - - -
8 10gige-agg-1 Up - - - -
8 10gige-agg-1 Up 310 groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xxx.233
8 10gige-agg-1 Up 117 groupnet0.subnet_main_service SSIP xxx.xxx.xxx.100
groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic xxx.xxx.xxx.71
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.83
8 mgmt-1 No Carrier - - - -
9 10gige-1 Up - - - -
9 10gige-2 Up - - - -
9 10gige-agg-1 Up - - - -
9 10gige-agg-1 Up 310 groupnet0.subnet_main_backup.pool_main_backup Static xxx.xxx.xxx.234
9 10gige-agg-1 Up 117 groupnet0.subnet_main_service.pool_main_service_dynamic Dynamic xxx.xxx.xxx.73
groupnet0.subnet_main_service.pool_main_service_static Static xxx.xxx.xxx.84
9 mgmt-1 No Carrier - - - -
In this state, the following event is generated and the Amber LED of node 6 turn on.
# isi event list ID Started Ended Causes Short Lnn Events Severity ---------------------------------------------------------------------------------------- xxxxx 05/22 12:02 NODE_AGGREGATE_LINK_DOWN 6 6603 critical
Cause
Check network pool groupnet0.subnet_main_service.pool_main_service_dynamic
This network pool of Rebalance Policy is manual.
# isi network pools view groupnet0.subnet_main_service.pool_main_service_dynamic
ID: groupnet0.subnet_main_service.pool_main_service_dynamic
Groupnet: groupnet0
Subnet: subnet_main_service
Name: pool_main_service_dynamic
Rules: -
Access Zone: zone-main-service
Allocation Method: dynamic
Aggregation Mode: lacp
Description: Default ext-1 pool
Firewall Policy: default_pools_policy
Ifaces: 6:10gige-agg-1, 7:10gige-agg-1, 8:10gige-agg-1, 9:10gige-agg-1
IP Ranges: xxx.xxx.xxx.71-xxx.xxx.xxx.74
IPv6 Perform DAD: No
Rebalance Policy: manual
SC Failover Policy: conn_count
Static Routes: xxx.xxx.xxx.0/24-xxx.xxx.xxx.252
NFSv3 RDMA RRoCE only: No
SmartConnect DNS Settings:
SC Suspended Nodes: -
SC Connect Policy: round_robin
SC Zone: xxxxx.xxxxx.xx.xx
SC DNS Zone Aliases: -
SC Subnet: subnet_main_service
SC TTL: 0
Resolution
Issue "isi network sc-rebalance-all" command.
Note: This command is OneFS9.1. Check rebalance command each version of OneFS CLI Command Reference
A dynamic IP address moves to an interface that does not have an IP address. Issue "isi network interfaces list" command. Check Dynamic IP address restore. Issue "isi event list" command. Check "NODE_AGGREGATE_LINK_DOWN" is resolved. If the issue persists, contact technical support.
Caution: Do not issue isi event modify --id="EVENTID" --ignore=true
The ignore option clears permanently.
Even if the problem is not solved the event is not re-created.
The resolve option re-creates event when the problem is not solved.
Additional Information
Link down related KB is following.
PowerScale: Event Notification: External Network Link Down - Event ID: 200020005
Affected Products
Isilon, PowerScale, Isilon Gen6.5, Isilon Gen6, Isilon NL-Series, PowerScale OneFS, Isilon S-Series, Isilon SmartConnect, Isilon X-SeriesArticle Properties
Article Number: 000242347
Article Type: Solution
Last Modified: 18 Dec 2024
Version: 6
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.