Distributed KV Cache over RoCE: Multi-Node LLM Inference on Dell and Broadcom
Distributed KV-cache transfers for disaggregated LLM inference are evaluated, comparing RoCE with default TCP on two Dell PowerEdge XE9680 nodes and Broadcom 400 GbE networking.