VRTX, R1-2210, and M620 Networking Issues

Question

Hello,

Thanks in advance for any assistance.

I have purchased a used M620 blade for my company's PE VRTX. The VRTX currently has slots 2 and 4 populated, and those are currently carrying my production workloads just fine.

The 'new-used' M620 blade was originally configured for a M100e enclosure. I added the 2 PCIe mezzanine cards so it works in the VRTX as expected (though I don't use the PCIe features as yet) and they are recognized fine.

The blade came with NPAR enabled. I defaulted the 2 NICs on the NDC ( Broadcom 10Gb 2P 57810s bNDC - same as in my working blades) using the BIOS, and that removed the NPAR and any other custom networking settings the previous owners had (I think?).

I can find no other BIOS setting that is not factory default, i.e. no further customizations of the BIOS besides the NPAR have been discovered.

The R1-2210 10Gb card is set to default for the internal ports serving these slots. All verified admin 'up'.

I loaded ESXi 6.5U3 from the Dell OEM distro, which seems to be the last distro supported on my M620's, and that boots fine and sees the internal vSphere datastores on the internal SAN with no issues. The drivers for the bNDC also seem to be there.

The issue is I cannot get the internal ports on the R1-2210 to 'come up" in conjunction with this new blade. All ports have been verified as administratively "up", but no joy. I have tried both open slots 1 and 3.

My use case is vSphere and I will eventually use iSCSI to both the host and the VMs but right now I am trying to keep things very simple and just get the network ports to come up, and the vSphere to get a DHCP address (as a test, not perm of course!).

Should this "just work" using the factory default settings on the nBDC "devices" in the BIOS? I have perused the VRTX networking guide but that doesn't seem to have anything in it about this. That guide starts with "it just works" in the most basic form, and tells you more about 'advanced' items like NPAR.

I have read that there was a factory defect in the original midplane. How would I know if I am affected by that short of moving a 'working' M620 to a 'broken' slot? Is there documentation (I searched but found nothing outside of forum posts where this factory defect and a 'Dell internal kb' s mentioned).

Any help would be greatly appreciated!!

Regards,

BPC

FPBPC · Answer

OP here....One more 'clue' maybe?

When I look at the properties of the ports on the 'new-used' M620, after defaulting the networking device configs in BIOS, I still see a bandwidth 'reservation' of 10% minimum and 100% maximum. On the 'working' M620 it shows a 0% minimum and a 100% max. I cannot find where the 'reservation' might be coming from.

Non-working:

Working:

Thanks again for any help!!

BPC

DELL-Young E · Answer

Hi, from this manual

https://dell.to/2M1ofjx

"The following ports are found on the R1-2210 switch: • 20 x 10G Ethernet Ports. These consist of: – 4 external ports—Connected to network (visible when the switch is in the chassis) – 16 internal ports—Connected to blade servers (not visible when the switch is in the chassis) • 2 x 1G Ethernet Ports • 1 Out-of-Band port (this port is the same as used for CMC)"

DELL-Young E · Answer

This would require knowledge for switch configuration I think... I will look further while we may all wait on other contributors to chip in.

FPBPC · Answer

Young-E,

Thanks for your reply. It is not a question of more ports, so I guess I was unclear. Here is a better screenshot to illustrate the issue. Note that when I took this screenshot there were 3 powered-on blades in the VRTX, in slots 1,2, & 4.

Obviously the 2 ports for slot 1 are not green, and that is the basic problem.

I did some further testing after hours today. I switched the servers in slots 4 and 1. The results were still as shown above, meaning the 'new-used' server was green when in slot 4, and the working unit was red when placed into slot 1. I did learn my 'new-used' M620 is not the issue, as in slot 4 it came up green and DCHP worked so it successfully used the R1-2210 to communicate with the external network. That's good news, at least.

My basic question then is this:

On the Port Configuration 'Edit" page in the R1-2210 I/O Module GUI all ports are marked as administratively up. Here is the page I mention showing Te1/1, the first red port in the pic above:

As noted, the port is admin up.

Is there anywhere else I need to "enable" these ports in that GUI?

After reading what very little can be found about other people's experiences with factory-defective midplanes none of those cases seemed to affect all ports on a slot, much less 2 slots. So I feel that is a rabbit-hole and I am just missing something....somewhere....

Thanks again to all that reply!

BPC

DELL-Young E · Answer

Hi, I hope this information helps: https://dell.to/3bi4P2h

Origin3k · Answer

I have sold a lot of VRTX systems and from a SKU standpoint and from the marketing the blades for the VRTX are named "MXXXp" and for the standard M1000 these are just "MXXX" without the "p".

I have in mind, based on old trainings that they are different in the terms of the PCI/SAS bridge in the MEZZ.

I cannot confirm this because i havent swapped Blades between those systems.

The NPAR is a great feature for us VMware boys and i have use it a lot in the VRTX.

Regards,
Joerg

FPBPC · Answer

OP here..

It was a case of being too close to the subject matter and forgetting 'the basics". I flapped ("shut/no shut") the ports on the R1-2210 and voila it works...somewhat. I get a huge amount of drops when pinging the IP of the OS (VMNIC0) but the ports are up. I think the ping drop is related to either the NDC being bad, or firmware.

I want to thank all who took time to answer this!! Thanks!

BPC

PowerEdge Hardware General

VRTX, R1-2210, and M620 Networking Issues

Was this post helpful?