Unsolved
This post is more than 5 years old
11 Posts
0
8435
Adding a FCOE host to VNX 5300
I have run into a wall and am looking for some advice.
I am trying to add my ESX5 servers to my storage group but I can not seem to get the right process.
I have plugged in the fcoe ports to the nexus and from the enxus to the esx servers. (I can pass traffic over nexus from esx)
I went to add an host initiator to unisphere, but after filling out the form, the ok button is not enabled.
I have also tried to add the vcenter server to the virtualzation page and the server shows up fine, but it does not list the hosts under it.
What am I doing wrong?
StevenMurray
11 Posts
0
May 15th, 2012 10:00
CH-NEXUS1# show inter brief
-------------------------------------------------------------------------------
Interface Vsan Admin Admin Status SFP Oper Oper Port
Mode Trunk Mode Speed Channel
Mode (Gbps)
-------------------------------------------------------------------------------
fc2/1 1 auto on sfpAbsent -- -- --
fc2/2 1 auto on sfpAbsent -- -- --
fc2/3 1 auto on sfpAbsent -- -- --
fc2/4 1 auto on sfpAbsent -- -- --
fc2/5 1 auto on sfpAbsent -- -- --
fc2/6 1 auto on sfpAbsent -- -- --
fc2/7 1 auto on sfpAbsent -- -- --
fc2/8 1 auto on sfpAbsent -- -- --
fc2/9 1 auto on sfpAbsent -- -- --
fc2/10 1 auto on sfpAbsent -- -- --
fc2/11 1 auto on sfpAbsent -- -- --
fc2/12 1 auto on sfpAbsent -- -- --
fc2/13 1 auto on sfpAbsent -- -- --
fc2/14 1 auto on sfpAbsent -- -- --
fc2/15 1 auto on sfpAbsent -- -- --
fc2/16 1 auto on sfpAbsent -- -- --
--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Por
t
Interface Ch
#
--------------------------------------------------------------------------------
Eth1/1 1 eth trunk up none 10G(D) 1
Eth1/2 1 eth trunk up none 10G(D) 1
Eth1/3 1 eth trunk up none 1000(D) 10
Eth1/4 1 eth trunk down SFP not inserted 1000(D) 10
Eth1/5 1 eth access down SFP not inserted 10G(D) --
Eth1/6 1 eth access down SFP not inserted 10G(D) --
Eth1/7 1 eth access down SFP not inserted 10G(D) --
Eth1/8 1 eth access down SFP not inserted 10G(D) --
Eth1/9 254 eth trunk up none 10G(D) --
Eth1/10 254 eth trunk down SFP not inserted 10G(D) --
Eth1/11 254 eth trunk down SFP not inserted 10G(D) --
Eth1/12 254 eth trunk down SFP not inserted 10G(D) --
Eth1/13 1 eth access down SFP not inserted 10G(D) --
Eth1/14 1 eth access down SFP not inserted 10G(D) --
Eth1/15 1 eth access down SFP not inserted 10G(D) --
Eth1/16 1 eth access down SFP not inserted 10G(D) --
Eth1/17 1 eth access down SFP not inserted 10G(D) --
Eth1/18 1 eth access down SFP not inserted 10G(D) --
Eth1/19 1 eth access down SFP not inserted 10G(D) --
Eth1/20 1 eth access down SFP not inserted 10G(D) --
Eth1/21 1 eth access down SFP not inserted 10G(D) --
Eth1/22 1 eth access down SFP not inserted 10G(D) --
Eth1/23 1 eth access down SFP not inserted 10G(D) --
Eth1/24 1 eth access down SFP not inserted 10G(D) --
Eth1/25 1 eth access down SFP not inserted 10G(D) --
Eth1/26 1 eth access down SFP not inserted 10G(D) --
Eth1/27 1 eth access down SFP not inserted 10G(D) --
Eth1/28 1 eth access down SFP not inserted 10G(D) --
Eth1/29 1 eth access down SFP not inserted 10G(D) --
Eth1/30 1 eth access down SFP not inserted 10G(D) --
Eth1/31 502 eth trunk up none 10G(D) --
Eth1/32 501 eth trunk up none 10G(D) --
--------------------------------------------------------------------------------
Port-channel VLAN Type Mode Status Reason Speed Proto
col
Interface
--------------------------------------------------------------------------------
Po1 1 eth trunk up none a-10G(D) la
cp
Po10 1 eth trunk up none a-1000(D) la
cp
Po511 501 eth trunk down No operational members auto(D) la
cp
Po512 502 eth trunk down No operational members auto(D) la
cp
--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
mgmt0 -- up 192.168.0.11 1000 1500
-------------------------------------------------------------------------------
Interface Vsan Admin Admin Status SFP Oper Oper Port
Mode Trunk Mode Speed Channel
Mode (Gbps)
-------------------------------------------------------------------------------
vfc9 3210 F on trunking -- TF auto --
vfc10 3210 F on errDisabled -- -- --
vfc11 3210 F on errDisabled -- -- --
vfc12 3210 F on errDisabled -- -- --
vfc31 3210 F on errDisabled -- -- --
vfc32 3210 F on errDisabled -- -- --
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1#
CH-NEXUS1# show vsan 3210
vsan 3210 information
name:VSAN3210 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
dynamox
1 Rookie
1 Rookie
•
20.4K Posts
0
May 15th, 2012 11:00
do you have portchannels on your physical ethernet interfaces ? If yes , why ?
Here are my physical ethernet interfaces
#show running-config interface Ethernet1/33
!Time: Tue May 15 13:50:46 2012
version 5.0(3)N2(2)
interface Ethernet1/33
switchport mode trunk
switchport trunk allowed vlan 50
Vlan 50 is my "FCOE" VLAN
I map VLAN 50 to VSAN 150
1784NDec3B-s3# sh vlan fcoe
Original VLAN ID Translated VSAN ID Association State
---------------- ------------------ -----------------
50 150 Operational
#sh int fcoe
Ethernet1/33 is FCoE UP
vfc33 is Up
FCID is 0xb90001
PWWN is 50:06:01:64:46:e0:4a:8d
MAC addr is 00:60:16:48:f0:86
StevenMurray
11 Posts
0
May 15th, 2012 11:00
The settings are the same as you listed. Could this be a cable compatibility issue?
StevenMurray
11 Posts
0
May 15th, 2012 12:00
Yes there is a port channel. Good question, I will remove to test.
Thank you.
StevenMurray
11 Posts
0
May 16th, 2012 05:00
Resolved the issue by changing the following commands on the nexus.
CH-NEXUS1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
CH-NEXUS1(config)# inter eth1/31
CH-NEXUS1(config-if)# no channel-group 512 mode active
CH-NEXUS1(config-if)# channel-group 512 mode on
CH-NEXUS1(config-if)# interface Ethernet1/32
CH-NEXUS1(config-if)# no channel-group 511 mode active
CH-NEXUS1(config-if)# channel-group 511 mode on
Basically change the mode from active negotiation to just on. Now the vnx shows up in flogi
That you for pointing me in the right direction.
dynamox
1 Rookie
1 Rookie
•
20.4K Posts
0
May 16th, 2012 06:00
this would be correct configuration if you were working with Ethernet, where you want your host to be connected to two different switches. This is not right for FCOE, there is no reason to port-channel FCOE ports, they don't understand LACP anyway. You need to treat FCOE ports as you would treat regular FC ports, the only difference is they ride on top of Ethernet.
dynamox
1 Rookie
1 Rookie
•
20.4K Posts
0
May 16th, 2012 06:00
i am still curious why did they configure port-channel for those ports ? Hopefully ports on Fabric A are not port-channel with ports on Fabric B ( if you have vPC in place that is)
StevenMurray
11 Posts
0
May 16th, 2012 06:00
Is there any documentation I can provide to our network group for this?
StevenMurray
11 Posts
0
May 16th, 2012 06:00
Below is the Port Chan linkage
PC 511
Nexus1 1/32 to VNX A-0
Nexus 2 1/32 to VNX A-1
PC512
Nexus1 1/31 to VNX B-0
Nexus2 1/31 to VNX B-1
dynamox
1 Rookie
1 Rookie
•
20.4K Posts
0
May 16th, 2012 07:00
here is what we used to configure FCOE portion. I am trying to find the topology guide on Cisco website, the one EMC has does not talk about port-channel.
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli_rel_4_1/Cisco_Nexus_5000_Series_Switch_CLI_Software_Configuration_Guide_chapter31.html
Sammy_K
9 Posts
0
November 6th, 2012 21:00
sorry for the late response here...Dynamox is correct.
In fact the EMC FCoE Techbook states this with regards to FCoE and VPC.
here's the snippet...
FCoE does not operate over vPC because a host must use NIC teaming/bonding to operate with vPC, which inherently breaks the native Fibre Channel multipathing software.
However, FCoE runs successfully with CNAs participating in VPCs, but the FCoE raffic will not take advantage of the vPC protocol itself.
christopher_ime
2K Posts
0
January 12th, 2013 02:00
Link aggregation of the FCoE (or iSCSI) ports on the VNX/CLARiiON SP ports are not supported as already noted by Dynamox and as quoted in the Techbook available on support.emc.com by Sammy_K; in fact, we don't provide an interface to even aggregate the SP ports, for instance, in an LACP or EtherChannel configuration as would be required. All end-points (switch-to-array) would have to be configured similiarly so they can negotiate/understand the same protocol.
It is unlikely we would ever offer the ability to aggregate the SP ports on a VNX/CLARiiON that serve up the block protocols as the hosts will benefit much more by presenting multiple paths and allowing host based path management software (MPIO: native or PowerPath) to either load balance and/or failover versus leaving it up to the switch if link aggregation were supported.
Using iSCSI as an example, but the logic is consistent with FCoE as the same base is Ethernet, let's imagine for a moment we did allow you to aggregate all of the SP ports (using an extreme example for emphasis). That would then appear as a single path to the host and you wouldn't need, for instance, an MPIO solution on the host such as PowerPath. However, to leave the decision to manage the paths to the switch at the link layer (versus the upper layer protocols as visible to PowerPath for instance) would not be the right choice. Also, consider the LUN ownership model of the VNX SP in the context of path management.
On the other hand, it makes perfect sense and is fully supported from the data movers which is primarily (I'm ignoring iSCSI which is available from the data movers but only via CLI and only with engineering approval) used to serve up the file sharing protocols: CIFS and NFS. Your CIFS/NFS clients won't have the same multipathing configuration enabled on the hosts. Of course it goes without saying that the link aggregation also provides the necessary resiliency in the back-end as well as the added benefit of a wider pipe (more "lanes").
christopher_ime
2K Posts
0
January 12th, 2013 02:00
StevenMurray,
The fact that you were even able to get it to work in a way with the switch configuration as you had pasted in a way surprises me. I'm going to suggest though that if you were to do any path failure testing, it probably wouldn't behave properly or at all.