VNX: How to identify Hard Link Down showing for data mover port (User Correctable)
Summary: Troubleshoot and verify Link Down showing for data mover interface - Hard Link Down indicates cables or other end devices have caused loss of connectivity.
Instructions
- IP network suddenly reports loss of connectivity for IP addresses assigned to VNX data mover, though the VNX data mover reports no hardware failure in our case.
- When checking the data mover -pci devices we can see [ Link: Down ] for data mover port.
- If [ Link: Down ] message is shown for data mover device/port, the cables and or other physical end devices (Switch) connected are causing loss of connectivity to IP network.
- The cables and end devices must be confirmed as working as needed as the VNX is not the cause of the [ Link: Down ] message.
- Any CIFS server connected to this IP address may be inaccessible.
- The IP addresses assigned to a specific VNX data mover device/port are not accessible by ping, though other VNX data mover IP addresses are accessible by ping.
- Site networking team must provide the IP address which is inaccessible. Support can confirm whether this interface exists on any VNX data mover server_2 - server_9
Troubleshooting Steps:
1. In a command line session to the control station we will validate which data mover this IP address exists on well as the device the interface is using.
[server_ifconfig server_2 -a ] will present a full listing of interfaces.
For our example we identify the interface name as INT1_PROD assigned to IP X.X.X.X and specify this interface for a short listing in server_ifconfig as below.
[nasadmin@vnx8000~] server_ifconfig server_2 INT1_PROD
INT1_PROD protocol=IP device=fxg-3-0
inet=X.X.X.X netmask=255.255.255.0 broadcast=X.X.X.X
UP, Ethernet, mtu=1500, vlan=0, macaddr=x:xx:xx:xx:xx:xx
Above we confirm INT1_PROD is using device named fxg-3-0.
Below we will confirm this device status of fxg-3-0
[nasadmin@vnx8000~] server_sysconfig server_2 -pci fxg-3-0
Broadcom 10 Gigabit Ethernet Controller
0: fxg-3-0 IRQ: 32
speed=10000 duplex=full txflowctl=disable rxflowctl=disable
Link: Down
2. The VNX interface is also not accessible by ping from the control station or any other devices in the network. -
[nasadmin@vnx8000~] ping X.X.X.X
PING X.X.X.X (X.X.X.X) 56(84) bytes of data.
--- X.X.X.X ping statistics ---
39 packets transmitted, 0 received, 100% packet loss, time 38097ms
3. We can also confirm whether the port shows faulted in the server_log.
In server_2 server_log we see link suddenly drop and show Link DOWN message, but no hardware failures.
[nasadmin@vnx8000 ~]$ server_log server_2 |grep -i fxg-3-0
2017-05-20 17:40:50: DRIVERS: 6: fxg-3-0: Link DOWN
2017-05-20 17:40:50: DRIVERS: 4: 2: fxg-3-0: link down
Additional Information
[Link: down] is caused external to the array, by the cable or other end devices and must be resolved outside of the Array.
The only time [Link: down] will be caused by the array is if the I/O module or physical DM port is faulted and confirmed in the server_log.