PowerEdge: Broadcom bnxtnet: How to set RSS Queues per Port in VMware ESXi

Summary: PowerEdge customers using Broadcom NXE adapters in VMware ESXi may experience unexpected results when attempting to tune port parameters. Particularly in configurations with multiple NXE adapters present, understanding the port ordering from both a VMware (vmnicX) and serverPCI-e Slot-Bus-Device-Function (SBDF) may be helpful to select the appropriate commands. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

For this discussion, consider a server with three Broadcom bnxtnet based NIC devices with two ports each device. The following is a basic module parameters set command using the same RSS value for all ports, for example:
 

esxcfg-module -s 'RSS=4,4,4,4,4,4' bnxtnet


This results in correctly setting all ports to RSS=4

However, if attempting to set specific vmnics to different values, the port ordering becomes important.

The module parameters are applied in the order of devices sorted by the PCI SBDF (slot-bus-device-function), NOT the vmnicX ordering.

For example:
 

PCI SBDF vmnicX       RSS Vaule Desired
0000:1a:00.0 vmnic0 4
0000:1a:00.1 vmnic1 4
0000:86:00.0 vmnic4 4
0000:86:00.1 vmnic5 8
0000:af:00.0 vmnic2 8
0000:af:00.1 vmnic3 4


Use the configuration command:
 

esxcfg-module -s 'RSS=4,4,4,8,8,4' bnxtnet


NOTE: A server reboot or module unload/reload is required for the settings to take effect.
The 'disable_tpa' parameter would also work the same way:
 

esxcfg-module -s 'disable_tpa=4,4,4,8,8,4' bnxtnet


Also, configuration parameter commands can be grouped as one command, for example below:
 

esxcfg-module -s "disable_tpa=1,1,1,1,1,1 RSS=8,8,8,8,8,8" bnxtnet

Cause

Note: the Broadcom driver does not have any control of this behavior, it is an artifact of the order in ESXi VMkernel which attaches devices to the driver.

Resolution

For customers reporting this issue, recommend to:

  1. Check the PCI SBDF and the OS vmnic ordering (using "esxcli network nic list" or other methods -TSR, lspci, Vcenter UI, and so on).
  2. Build the server module commands based on the PCI SBDF ordering.
  3. Use the server module 'get' command to ensure the correct settings per port:
esxcfg-module -g bnxtnet

 

vsish -e get /net/pNics/vmnicX/rxqueues/info


Alternatively, VMware allows reordering the vmnic alias mapping which is described in their KB article:
How VMware ESXi determines the order in which names are assigned to devices (2091560)This hyperlink is taking you to a website outside of Dell Technologies.

Additional Information

See the latest release notes for additional VMware commands and info:

Example: release noteThis hyperlink is taking you to a website outside of Dell Technologies.

And for general information about the adapter see the user's manual on Dell.com:
Example: https://dl.dell.com/FOLDER10655246M/1/Bcom_LAN_227.1.111.3_NXE_Manual_227.1.111.3.zip

Affected Products

Broadcom NetXtreme Family of Adapters, PowerEdge FC640, PowerEdge M640 (for PE VRTX), PowerEdge MX740C, PowerEdge MX750c, PowerEdge MX760c, PowerEdge MX840C, PowerEdge R330, PowerEdge R340, PowerEdge R350, PowerEdge R430, PowerEdge R440 , PowerEdge R450, PowerEdge R530, PowerEdge R530xd, PowerEdge R540, PowerEdge R550, PowerEdge R570, PowerEdge R630, PowerEdge R640, PowerEdge R6415, PowerEdge R650, PowerEdge R650xs, PowerEdge R6515, PowerEdge R6525, PowerEdge R660, PowerEdge R660xs, PowerEdge R6615, PowerEdge R6625, PowerEdge R670, PowerEdge R6715, PowerEdge R6725, PowerEdge R730, PowerEdge R730xd, PowerEdge R740, PowerEdge R740XD, PowerEdge R740XD2, PowerEdge R7415, PowerEdge R7425, PowerEdge R750, PowerEdge R750XA, PowerEdge R750xs, PowerEdge R7515, PowerEdge R7525, PowerEdge R760, PowerEdge R760XA, PowerEdge R760xd2, PowerEdge R760xs, PowerEdge R7615, PowerEdge R7625, PowerEdge R770, PowerEdge R7715, PowerEdge R7725, PowerEdge R830, PowerEdge R840, PowerEdge R860, PowerEdge R930, PowerEdge R940, PowerEdge R940xa, PowerEdge R960, PowerEdge T340, PowerEdge T430 ...
Article Properties
Article Number: 000223479
Article Type: Solution
Last Modified: 05 Jun 2025
Version:  2
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.