Dell Unity: FSN performance impact when Ethernet Switch reporting Unity Ethernet MAC address is flapping between ports (Dell Correctable)

Summary: The customer is using Unity’s Fail Safe Networking (FSN). Both the Primary Active and Standby interfaces are configured with a two-port Link Aggregation Control Protocol (LACP) bond. The LACP bonded interfaces for the FSN Primary Active are connected to the Primary Active Access switch. The FSN Standby LACP bonded interfaces are connected to the Standby Access switch. However, the customer’s Cisco Access Switches do not support LACP virtual Port Channel (vPC) across switches. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

Unity Fail Safe Networking (FSN) is experiencing performance degradation whenever the Cisco Access Switches issue these MAC_MOVE alerts.

Sep  2 14:27:52.147 UTC+1: %MAC_MOVE-4-NOTIF: Host xxxx.xxxx.xxa4 in vlan 10 is flapping between port Po1 and port Po2
Sep  2 14:27:52.339 UTC+1: %MAC_MOVE-4-NOTIF: Host xxxx.xxxx.xxa4 in vlan 10 is flapping between port Po2 and port Po1


 

Cause

OpenSLP is Multicasting updates out all active interfaces, including the Unity standby FSN interface.
This causes the switch's management console to see a duplicate MAC address coming from the standby FSN interface on the standby Access switch which causes the switch to post an alert.

Jul 20 17:32:24.624 UTC+1: %MAC_MOVE-4-NOTIF: Host xxxx.xxxx.xxa4 in vlan 10 is flapping between port Po1 and port Po2

The Ethernet switch must converge the network traffic going to the Unity's Ethernet MAC address, that is moved to the standby Access switch.
There is a delay in the network traffic, while the switches reroute the network traffic to the Standby Access switch port.

However, Unity is still using the FSN Primary Active interface to send out its network traffic.
The Switch sees the Duplicate MAC address on Primary Access Switch and posts another alert.

Jul 20 17:32:45.360 UTC+1: %MAC_MOVE-4-NOTIF: Host xxxx.xxxx.xxa4 in vlan 10 is flapping between port Po2 and port Po1

The Ethernet switch must converge the network traffic going to the Unity's Ethernet MAC address, that is moved to the Primary Access switch.
There is a delay in the network traffic, while the switches reroute the network traffic to the Primary Access switch port.

This pattern continues until OpenSLP is configured to only use the Unity Management network interface.

Resolution

Important Steps for Unity’s OpenSLP Configuration:

  1. Network Update using Management Interface: Once these steps are completed, Unity’s OpenSLP uses its Management Interface to update the network that the SMI-S Provider Service is active.

  2. Reapplication Required: If any of Unity’s Storage Processors (SPs) are replaced, re-imaged, or upgraded, these steps must be re-applied.

  3. Configuration Update: Once OpenSLP’s configuration is updated, the OpenSLP daemon on the Primary SP must be stopped and restarted to load the new configuration.

  4. Stopping OpenSLP daemon on Peer: The OpenSLP daemon on the Peer must be stopped. The Peer might still have the OpenSLP daemon running with the old configuration if Unity's ECOM services were previously failed over.


Warning: This task must be executed by a Senior or Technical Lead who has received Unity training.

Configuring OpenSLP to only use the Unity Management network interface will resolve the performance issue.
 
Here are the steps to update the OpenSLP configuration to only use Unity’s Management Interface mgmt_vdev.
 
Using your favorite ssh client, login into Unity as service
 
1. Generate a service shell challenge string from Unity.
svc_diag; svc_inject -k
 
2. Request the service shell challenge response from Unity Engineering.
 
3. Activate the service shell using the service shell challenge response.
svc_inject -k hex_service_shell_challenge_response
 
4. Before the update what is the current status of the OpenSLP daemon.
  
Detail query
lsof -n | grep -i slpd
lsof -n -i :427
echo "";echo slptool findsrvs service:service-agent; slptool findsrvs service:service-agent;echo "";echo slptool findsrvtypes; slptool findsrvtypes;echo "";slptool findsrvtypes | awk '{ print "echo slptool findsrvs " $1" ; slptool findsrvs " $1 ";echo """}' | sh

Quick query
lsof -n -i :427
slptool findsrvs service:service-agent
 
5. Make a backup of /etc/slp.conf
cp -p /etc/slp.conf /etc/slp.conf-bak
 
6. Update OpenSLP's configuration by editing /etc/slp.conf
vi /etc/slp.conf
Find the line that starts with
;net.slp.interfaces = 192.168.0.1
Add a newline with the text
net.slp.interfaces = mgmt_vdev
 
7. Verify the changes. 
diff /etc/slp.conf /etc/slp.conf-bak
192d191
< net.slp.interfaces = mgmt_vdev

egrep -v "^#|^;" /etc/slp.conf |sed '/^\s*$/d'
net.slp.interfaces = mgmt._vdev
 
8. Update Unity's Peer SP /etc/slp.conf files. 
scp -p /etc/slp.conf* peer:/etc/.
 
9. While on the Primary SP, Stop OpenSLP's daemon and start OpenSLP, to load the updated configuration. Verify OpenSLP is using a new Process ID (PID). 
systemctl status slpd
systemctl stop   slpd
systemctl start  slpd
systemctl status slpd

10. Connect to the peer SP. 
ssh peer
 
11. Check OpenSLP daemon is not running. 
smartctl status slpd
12.  If OpenSLP daemon is running, stop the daemon and verify that it is not running 
smartctl stop slpd
smartctl status slpd
13. Return to the Primary SP. 
exit

14. On the Primary SP, verify the update is working by querying OpenSLP daemon status. 
Detail query
lsof -n | grep -i slp
lsof -n -i :427
echo "";echo slptool findsrvs service:service-agent; slptool findsrvs service:service-agent;echo "";echo slptool findsrvtypes; slptool findsrvtypes;echo "";slptool findsrvtypes | awk '{ print "echo slptool findsrvs " $1" ; slptool findsrvs " $1 ";echo """}' | sh

Quick query
lsof -n -i :427
slptool findsrvs service:service-agent

15. The output of: 
slptool findsrvs service:service-agent
lsof -n -i :427
 
Should be clean, with only a few address lines.
  
Loopback
Management interface
Multicast IP address in the range of 224.0.1.0-224.0.1.225
Multicast IP address for Service Location Protocol (SLP) 239.255.255.253
 
16. Ask the Network switch team to verify MAC flapping alerts have stopped for the MAC address xxxx.xxxx.xxa4.
Jul 20 17:32:24.624 UTC+1: %MAC_MOVE-4-NOTIF: Host xxxx.xxxx.xxa4 in vlan 10 is flapping between port Po1 and port Po2
Jul 20 17:32:45.360 UTC+1: %MAC_MOVE-4-NOTIF: Host xxxx.xxxx.xxa4 in vlan 10 is flapping between port Po2 and port Po1
 
17. Exit the Service Shell, and stop the Service shell. 
exit
svc_inject -t -e svc_service_shell

18. Verify the Service Shell has expired. 
svc_inject -l -a
 

Affected Products

Dell EMC Unity, Dell EMC Unity Family |Dell EMC Unity All Flash, Dell EMC Unity Family, Dell EMC Unity Hybrid, Dell Unity Operating Environment (OE)
Article Properties
Article Number: 000228420
Article Type: Solution
Last Modified: 14 Nov 2024
Version:  8
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.