UNSOLVED

giomoda

updated

7 years ago

G

giomoda

1 Rookie

6 Posts

0

5590

September 20th, 2019 07:00

R640 and Broadcom - Poor performance on SMB Direct (RDMA) configuration

Hello all.

So, I've got a couple new Dell R640 servers with Broadcom NetXtreme E-series Adapter (BCM57416) for a new Hyper-V Cluster  and I've been trying really hard to get SMB Direct working between them so I can leverage the best possible performance during LiveMigration. 

The thing is: all tests go green across the board, both Test-RDMA and Validate-DCB, as per screenshots below:

RDMA1.png

 
 

But when trying to copy something between the servers I get REALLY slow throughput:

Transfer Slow.png

 

The result for Get-SmbMultichannelConnection shows that RDMA is being used:

 

As soon as I disable RDMA on the interface, transfer speed goes back to normal

 

Transfer Normal.png

I've tried bypassing the switch and linking both interfaces directly, but to no avail. The results were the same.

So I guess my question is: has anyone gotten SMB Direct working successfully over Broadcom interfaces? I cannot for the life of me find anything wrong with my setup, but it just won't work.

Best regards,

Giovani