PowerEdge: How to Measure bandwidth for Intel E810-C 100G PCIe adapter using iPerf 3.1.3

• Intel E810-C PCIe adapter is installed in a x16 (recommended) slot as per Slot priority matrix.
• Compatible QSFP+Cables or Direct-Attached-Cables (DACs) are installed. 
• Link is UP running at 100 Gbps under iDRAC and within the Operating System.
• The system Profile is set to "Performance."

Devices based on the Intel Ethernet Controller E810-C have an expected total throughput for the entire device of 100 Gb/s in each direction if one 100G cable is connected or if two 100G cables are connected. See Intel article Intel Ethernet Adapters and Devices User Guide .

Achieving line rate on a 100G test host requires parallel streams. However, only adding a -P flag while using iperf3 does not work because each iperf3 process is single-threaded. This means all the parallel streams for one test use the same CPU core. If you are core limited, which is often the case for a 100G host, adding parallel streams does not help you unless you add additional iperf3 processes which can use additional cores.

For Red Hat Enterprise Linux or CentOS (SLI):

1. Set port MTU to 9000.
2. CPU set performance mode with command:

cpupower frequency-set -g performance

3. Install iperf3-3.1.3 with download link: 

https://iperf.fr/download/fedora/iperf3-3.1.3-1.fc24.x86_64.rpm 

4. Open multiple threads which require multiple ports. In this scenario, open seven ports: 5201 through 5207

To run multiple iperf3 processes for testing a high-speed host, run the following test commands:

On Server side: 
iperf3 -s -p 5201 & iperf3 -s -p 5202 & iperf3 -s -p 5203 & iperf3 -s -p 5204 & iperf3 -s -p 5205 & iperf3 -s -p 5206 & iperf3 -s -p 5207 &

**The final symbol (&) must be added.

On Client side: 
iperf3 -c 10.10.10.10 -t 100 -p 5201 & iperf3 -c 10.10.10.10 -t 100 -p 5202 & iperf3 -c 10.10.10.10 -t 100 -p 5203 & iperf3 -c 10.10.10.10 -t 100 -p 5204 & iperf3 -c 10.10.10.10 -t 100 -p 5205 & iperf3 -c 10.10.10.10 -t 100 -p 5206 & iperf3 -c 10.10.10.10 -t 100 -p 5207 &

**The final symbol (&) must be added.**

For Windows Server 2019 and 2022:

1. Confirm that the system profile is set to PERFORMANCE.
2. Set port MTU to 9014 for Intel E810.
3. Install iperf 3.13 with download link: 

https://iperf.fr/download/windows/iperf-3.1.3-win64.zip 

4. Extract iperf3-3.1.3-win64.zip file and copy two files (iperf3.exe and cygwin1.dll) to C:\Windows\.  
5. Open multiple threads which require multiple ports. In this scenario, open seven ports: 5201 through 5207

To run multiple iperf3 processes for a testing a high-speed host, run the following test commands:

Run Powershell as an administrator.

On Server side: 
cmd /c 'start iperf3.exe -s -p 5201 & start iperf3.exe -s -p 5202 & start iperf3.exe -s -p 5203 & start iperf3.exe -s -p 5204 & start iperf3.exe -s -p 5205 & start iperf3.exe -s -p 5206 & start iperf3.exe -s -p 5027'

On Client side: 
cmd /c 'start iperf3.exe -c 10.10.10.9 -t 100 -P 30 -l 512k -p 5201 & start iperf3.exe -c 10.10.10.9 -t 100 -P 30 -l 512k -p 5202 & start iperf3.exe -c 10.10.10.9 -t 100 -P 30 -l 512k -p 5203 & start iperf3.exe -c 10.10.10.9 -t 100 -P 30 -l 512k -p 5204 & start iperf3.exe -c 10.10.10.9 -t 100 -P 30 -l 512k -p 5205 & start iperf3.exe -c 10.10.10.9 -t 100 -P 30 -l 512k -p 5206 & start iperf3.exe -c 10.10.10.9 -t 100 -P 30 -l 512k -p 5207'

PowerEdge: How to test network bandwidth using 'iperf'