MX9116n FCoE-FC configuration?

With some help from Dell ProSupport I've pieced together the following configuration for my environment, and I've successfully established connection with my LUN on my FC SAN and created a VMFS6 datastore for my ESXi servers :BigSmile:

Environment:

• MX7000 chassis with 2 x MX9116n Fabric Switch Engines ("FSE")
• MX740c Servers with QL41262 CNA cards (4 servers so far in my chasses, all 8 will get CNA soon)
• Brocade FC fabrics wiht HUS-VM SAN behind

So the FCoE/FC traffic will be like this:

MX740c w/CNA_FCoE (ports 1-16) <-> MX9116n <-> FC breakout 4x16Gb (port 44) <-> Brocade FC-SW <-> FC-Storage

For ethernet uplink, we use port 41 on each FSE using a breakout cable for 4x10GbE, where we so far only use two of them from each FSE. Using VLT, and a LACP trunk to our core, we have 4x10GbE total uplink for ethernet.
For FC uplink, using port 44, we also have a break out cable, set to 4x16G, and here too we only use 2 of them from each FSE. Best practice dictates we use one uniqe vlan ID in each FSE so we can be sure we won't have FCoE crosstalk over the VLT interconnect, having each FSE as a separate fabric (vfabric) for FC/FCoE. Each FC uplink connects only to one FC brocade fabric.

My configuration for FCoE is so far like this:

FSE-A1:

interface vlan4092
 description "FCoE-01-internal-vlan"
 no shutdown
 exit
!
dcbx enable
feature fc npg
vfabric 1
 name vFabric-1
 vlan 4092
 fcoe fcmap 0efc01
 exit

FSE-A2:

interface vlan4093
 description "FCoE-02-internal-vlan"
 no shutdown
 exit
!
dcbx enable
feature fc npg
vfabric 2
 name vFabric-2
 vlan 4093
 fcoe fcmap 0efc02
 exit

FSE (both switches):

!# Break-out ethernet
port-group 1/1/13
 mode Eth 10g-4x
!
!# Break-out fibrechannel
port-group 1/1/16
 mode FC 16g-4x

class-map type network-qos class-Dot1p-3-FCoE
 match qos-group 3
class-map type queuing map-ETSQueue-0
 match queue 0
class-map type queuing map-ETSQueue-3
 match queue 3
exit
!
trust dot1p-map map-Dot1pToGroups
 qos-group 0 dot1p 0-2,4-7
 qos-group 3 dot1p 3
 exit
!
qos-map traffic-class map-GroupsToQueues
 queue 0 qos-group 0
 queue 3 qos-group 3
 exit
!
policy-map policy-Input-TrustDot1p
 class class-trust
 trust dot1p
 exit
 exit
!
policy-map type network-qos policy-Input-PFC
 class class-Dot1p-3-FCoE
 pause
 pfc-cos 3
 exit
 exit
!
policy-map type queuing policy-Output-BandwidthPercent
 class map-ETSQueue-0
 bandwidth percent 68
 class map-ETSQueue-3
 bandwidth percent 32
 exit
 exit
!
system qos
 ets mode on
 trust-map dot1p map-Dot1pToGroups 
 qos-map traffic-class map-GroupsToQueues
 exit

FSE-A1:

interface range ethernet 1/1/1-1/1/16
service-policy input type qos policy-Input-TrustDot1p
service-policy input type network-qos policy-Input-PFC
service-policy output type queuing policy-Output-BandwidthPercent
priority-flow-control mode on
vfabric 1
exit
!
interface range fibrechannel 1/1/44:1-1/1/44:2
vfabric 1
exit
exit

FSE-A2:

interface range ethernet 1/1/1-1/1/16
 service-policy input type qos policy-Input-TrustDot1p
 service-policy input type network-qos policy-Input-PFC
 service-policy output type queuing policy-Output-BandwidthPercent
 priority-flow-control mode on
 vfabric 2
 exit
!
interface range fibrechannel 1/1/44:1-1/1/44:2
 vfabric 2
 exit

You should now (hopefully) be able to get some data in the different 'show' commands for FCoE/FC:

show fc switch
show vfabric
show npg devices brief
show fcoe enode
show fcoe fcf
show fcoe sessions  
show fc statistics interface fibrechannel 1/1/44:1
show fc statistics interface fibrechannel 1/1/44:2
show fc statistics vfabric 1

Example (i've masked the MACs/WWNs) - shows my 4 servers with CNA card (FCoE enabled with NPAR-configuration):

OS10# show fcoe sessions
Enode MAC Enode Interface FCF MAC FCF interface VLAN FCoE MAC FC-ID PORT WWPN PORT WWNN
---------------------------------------------------------------------------------------------------------------------------------------------------------
66:66:66:15:fe:7f Eth 1/1/15 66:66:66:1f:f6:7d Fc 1/1/44:1 4093 66:66:66:05:8d:02 05:8d:02 66:66:66:66:66:15:fe:7f 66:66:66:66:66:15:fe:7f
66:66:66:15:ff:e7 Eth 1/1/13 66:66:66:1f:f6:7d Fc 1/1/44:1 4093 66:66:66:05:8d:01 05:8d:01 66:66:66:66:66:15:ff:e7 66:66:66:66:66:15:ff:e7
66:66:66:15:ff:0f Eth 1/1/11 66:66:66:1f:f6:7e Fc 1/1/44:2 4093 66:66:66:05:5c:02 05:5c:02 66:66:66:66:66:15:ff:0f 66:66:66:66:66:15:ff:0f
66:66:66:0c:25:a7 Eth 1/1/1  66:66:66:1f:f6:7e Fc 1/1/44:2 4093 66:66:66:05:5c:01 05:5c:01 66:66:66:66:66:0c:25:a7 66:66:66:66:66:0c:25:a7

Hopefully this can help others get FCoE going on MX-infrastructure..