Isilon: IP conflict. Error: "arp: <MAC> is using my IP address <IP> on <interface>"
Summary: Issues in accessing SMB shares and mount NFS exports using IP addresses. The IP's on Isilon interface are pingable and also has a valid ARP entry with the mac address in the output of netstat -ni. ...
Symptoms
The following error appears in /var/log/messages:
2018-08-05T23:57:17-05:00 <0.3> isilon-2 /boot/kernel.amd64/kernel: arp: <MAC:58> is using my IP address 10.232.5.51 on ix1!
Issue accessing SMB shares and mount NFS exports using the above IP addresses seen in messages. The respected IP's are on Isilon interface and are pingable. Also has a valid ARP entry with the mac address in netstat -ni:
$ cat isilon-1/ifconfig
ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether <MAC:cd>
inet 10.232.5.90 netmask 0xffffff00 broadcast 10.232.5.255 zone 1
inet 10.232.5.78 netmask 0xffffff00 broadcast 10.232.5.255 zone 1
inet 10.232.5.50 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
inet 10.232.5.66 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
inet 10.232.5.67 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
inet 10.232.5.68 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
inet 10.232.5.69 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause>)
status: active
$ cat isilon-2/ifconfig
ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether <MAC:69>
inet 10.232.5.91 netmask 0xffffff00 broadcast 10.232.5.255 zone 1
inet 10.232.5.79 netmask 0xffffff00 broadcast 10.232.5.255 zone 1
inet 10.232.5.51 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
inet 10.232.5.61 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
inet 10.232.5.64 netmask 0xffffff00 broadcast 10.232.5.255 zone 4
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause>)
status: active
Cause
IP's: 10.232.5.50 and 10.232.5.51
1) Run the following command on the cluster to find a valid ARP entry for the IP which has issue:
Or identify the arp entries on the Isilon for the above IPs in the log file:
? (10.232.5.50) at <MAC:cd> on ix1 permanent [ethernet]
? (10.232.5.51) at <MAC:69> on ix1 permanent [ethernet]
Run the following command on the cluster:
Or look for for the error in messages log file:
2018-08-05T23:58:00-05:00 <0.3> isilon-1 /boot/kernel.amd64/kernel: arp: <MAC:ec> is using my IP address 10.232.5.50 on ix1!
2018-08-05T23:57:17-05:00 <0.3> isilon-2 /boot/kernel.amd64/kernel: arp: <MAC:58> is using my IP address 10.232.5.51 on ix1!
Search for the above MAC address is var/log/messages on Isilon (both nodes):
2018-08-06-001$
2018-08-06-001$
The above IP's are being used on a NetApp Server which led to an IP conflict.
HOST02#show arp|incl 10.232.5.50
10.232.5.50 N/A <MAC:ec> Vlan5, Ethernet8/2
HOST02#
HOST02#show arp|incl 10.232.5.51
10.232.5.51 N/A <MAC:58> Vlan5, Ethernet8/1
HOST02#
Resolution
Find the Server/Machine which has the MAC address as seen in /var/log/messages.
Contact Isilon Support to remove the IP addresses from the Isilon network pool.
or
Contact your Network team to remove the IP from the Client's server/Machine.
Validation:
- Remove the interface from the network pool so the affected IP's are no longer on Isilon interfaces or edit the IP range in the network pool.
- After removing the interface, ping the IP address and check if they are pingable.
- If its pingable, then the IP's are assigned to a different machine/server in the environment.
- A packet capture tells the MAC address of the Server on which the IP's are assigned.