• Download PDF

CPU-GPU Co-Serving Performance Analysis of 
Dell PowerEdge XE9780 with NVIDIA B200 and Intel Xeon 6767P
Dell PowerEdge XE9780LAP with NVIDIA B300 and Intel Xeon 6979P 
Dell PowerEdge XE9780L with NVIDIA B300 and Intel Xeon 6776P

Executive summary

As artificial intelligence workloads scale across enterprises, maximizing inference throughput while maintaining cost efficiency has become a critical business imperative. This white paper presents comprehensive performance benchmarking results that demonstrate the transformative potential of CPU-GPU co-serving architectures on the following Dell AI platforms:

• Dell PowerEdge XE9780 platform equipped with NVIDIA B200 GPUs and Intel Xeon 6767P processors
• Dell PowerEdge XE9780LAP platform equipped with NVIDIA B300 GPUs and Intel Xeon 6979P processors
• Dell PowerEdge XE9780L platform equipped with NVIDIA B300 GPUs and Intel Xeon 6776P processors

Our analysis, conducted across two industry-leading AI models, KIMI K2.6 , a 1-trillion-parameter Mixture-of-Experts (MoE) model with 32B active parameters, and the Llama 405B + Llama 8B combination, reveals that CPU-GPU co-serving delivers exceptional improvements in concurrent user capacity and overall system throughput with negligible impact on GPU performance.

Key findings

• User Capacity Expansion: Co-serving enables up to 2.07X more concurrent users than GPU-only configurations, effectively doubling the system's capacity to serve simultaneous AI inference requests without additional GPU hardware investment.
• Throughput Enhancement: System-wide throughput improved by up to 25.8%, driven by the parallel use of Intel Xeon CPU resources alongside NVIDIA Blackwell GPUs for inference workloads. The peak +25.8% result was recorded on the B300_AP platform (PowerEdge XE9780LAP with Intel Xeon 6979P and NVIDIA B300 GPUs).
• GPU Performance Preservation: GPU inference performance remains virtually unaffected during co-serving operations. GPU performance degradation remained between 1% and 7%.
• Cross-Model Validation: These benefits are consistent across both tested models, the large MoE architecture KIMI K2.6 and the dense transformer model Llama 405B, indicating broad applicability of the co-serving approach.

These results demonstrate that the Dell PowerEdge with Intel Xeon 6 CPUs and NVIDIA GPUs is a compelling platform for enterprise AI inference, enabling organizations to significantly increase their inference serving capacity by leveraging otherwise underutilized CPU resources without compromising GPU performance or requiring additional hardware procurement.

Hardware configuration

Dell PowerEdge XE9780 server (B200_SP platform)

The PowerEdge XE9780 is a 10U, two-socket, air-cooled rack server engineered for large-scale AI model training, fine-tuning, and high-performance inferencing. The PowerEdge XE9780 supports NVIDIA HGX B300 NVL8 and HGX B200 GPUs with high-bandwidth NVLink interconnects, enabling accelerated parallel processing and efficient scaling of complex AI workloads. Designed for integration into rack-scale environments, it provides the performance, memory capacity, and I/O throughput required to support enterprise AI deployment and operational consistency.

 Figure 1. Dell PowerEdge XE9780 air-cooled rack server

Built for AI model training and inference

• GPU fabric built for scale: 8x NVIDIA HGX B300 NVL8 or 8x NVIDIA HGX B200 GPUs, with NVLink/NVSwitch, enable high-bandwidth, low-latency communication for efficient large-model training and distributed workloads.
• Memory designed for large models: High-bandwidth GPU memory, combined with up to 4 TB of DDR5 system memory (32 DIMM slots at up to 6400 MT/s), supports larger models in memory and improves training throughput.
• AI-optimized system architecture: The dual-socket 6th Generation Intel Xeon Scalable platform with up to 86 cores per processor balances compute, memory, and I/O to keep GPUs fully utilized across training and inference pipelines.
• High-speed I/O and data movement: PCIe Gen5 bandwidth, up to 12 full-height half-length PCIe slots (B200 config), and high-performance networking readiness ensure fast data flow to and between GPUs, reducing pipeline bottlenecks.
• Storage for AI pipelines: Flexible NVMe configurations of up to 16x E3.S drives (122.88 TB max) or up to 10x U.2 NVMe SSDs (153.6 TB max) provide high-throughput data ingestion, checkpointing, and dataset access.
• Air-cooled high-density design: A 10U air-cooled platform supports high-wattage GPUs (up to 1000W per GPU), enabling deployment in existing data centers without liquid cooling.
• Built for rack-scale deployment: Engineered for integration into rack-scale environments, enabling consistent performance, scalability, and operational efficiency across AI clusters.

Dell PowerEdge XE9780LAP server (B300_AP platform)

The PowerEdge XE9780LAP is a liquid-cooled variant in the XE9780 family, designed for maximum AI compute density with advanced cooling technology. Unlike the air-cooled XE9780, the XE9780LAP uses liquid cooling for CPUs, GPUs, CX8 networking, and NVLink switches, enabling higher-wattage GPU support (1100W per GPU) within a 3 OU compute node form factor that installs in the Dell IR7000 rack infrastructure.

Figure 2. Dell PowerEdge XE9780LAP liquid-cooled rack server

Key differentiators from XE9780

• Liquid-cooled architecture: Full liquid cooling for CPUs, GPUs, CX8 OSFP networking, and NVLink switches enables sustained operation of 8x 1100W NVIDIA B300 GPUs without thermal throttling.
• Higher-core CPU support: Supports Intel Xeon 6 AP processors with up to 128 cores per processor, compared to up to 86 cores per processor on the air-cooled XE9780.
• Enhanced memory capacity: 24 DDR5 RDIMM slots with up to 6 TB maximum capacity (compared to 4 TB on the air-cooled XE9780), providing more system memory for CPU-side inference workloads.
• Integrated high-speed networking: 8x CX8 OSFP ports with 800 Gbps ConnectX-8 SuperNIC support, enabling direct GPU-to-network connectivity for distributed AI workloads.
• IR7000 rack integration: 3 OU compute node designed for the Dell IR7000 rack infrastructure, supporting 44 or 50 Open Rack Unit configurations.

Dell PowerEdge XE9780L server (B300_SP platform)

The PowerEdge XE9780L is the dual-socket, liquid-cooled variant in the XE9780 family, supporting both NVIDIA B300 and B200 GPU configurations. The B300_SP benchmark configuration pairs two Intel Xeon 6 processors with 8x NVIDIA B300 GPUs. 

Figure 3.Dell PowerEdge XE9780L liquid-cooled rack server   

Key differentiators from XE9780

• Dual-socket flexibility: Two Intel Xeon 6 processor sockets, each supporting up to 86 cores (up to 172 cores per node); the tested B300_SP configuration uses two 64-core Xeon 6776P processors for a total of 128 cores. The dual-socket design doubles available memory channels and PCIe lanes.
• Dual GPU support: Unique in the XE9780 family, the XE9780L supports both NVIDIA B300 (1100W, 270GB) and NVIDIA B200 (1000W, 180GB) GPU configurations, providing deployment flexibility.
• Extended storage: Supports up to 307.2 TB of NVMe storage with 8x U.2 NVMe drives plus 2x additional U.2 NVMe drives using PCIe CEM, or up to 16x E1.S drives (122.88 TB).
• Liquid cooling: Full liquid cooling for CPUs, GPUs, and NVLink switches enables sustained high-performance operation in dense rack configurations.
• 32 DIMM Slots: Supports up to 32x 256GB DDR5 DIMMs for up to 8TB total system memory, or 4TB with standard 128GB DIMMs.

Performance benchmarking results

This section presents detailed performance benchmarking results from two comprehensive evaluation studies: one using the KIMI K2.6 model and another using the Llama 405B + Llama 8B model combination. Both studies compare GPU-only configurations against CPU-GPU co-serving deployments across three hardware platform configurations.

Platform configurations tested

KIMI K2.6 model performance

KIMI K2.6 is a 1-trillion-parameter MoE model with 32 billion active parameters, representing a state-of-the-art architecture for large language model (LLM) inference. The following table presents the key co-serving performance metrics.

KIMI K2.6 - co-serving (GPU128 + CPU128) compared with GPU-only performance

The B200_SP configuration achieves a 1.77X increase in concurrent user capacity, from 199 GPU-only users to 352 total users with co-serving enabled. This represents an additional 153 concurrent users served by the Intel® Xeon® 6767P CPU, with a 19.3% increase in throughput and less than 1% degradation in GPU performance.

Llama 405B + Llama 8B model performance

The Llama 405B model, paired with Llama 8B for CPU-side inference, represents a dense transformer architecture test case. With 405 billion parameters, it places significant demands on GPU memory and compute.

Llama 405B + Llama 8B - co-serving (GPU128 + CPU128) compared with GPU-only performance

The Llama 405B results show even stronger co-serving gains. The B200_SP platform delivers a 1.84X increase in user capacity and a +24.6% improvement in throughput, while maintaining less than 1% GPU performance degradation. The B300_SP platform achieves the highest user gain multiplier of 2.07X.

A critical observation across both models is the consistently minimal impact on GPU performance. With GPU performance dropping by less than 1% across nearly all configurations, organizations can confidently deploy co-serving, knowing that the quality of their primary GPU inference remains intact.

Concurrent user capacity comparison

Figure 4 illustrates the dramatic increase in concurrent user capacity enabled by CPU-GPU co-serving.

Key observations

The chart compares two distinct serving modes, GPU-only and CPU-GPU co-serving, across three PowerEdge platforms (B200_SP, B300_AP, and B300_SP) and two production-grade LLM workloads (KIMI K2.6 and Llama 405B + Llama 8B). In GPU-only mode, all inference requests are handled exclusively by the NVIDIA GPU pool, leaving the CPU cores largely idle. In co-serving mode, the Intel Xeon CPU handles smaller, latency-tolerant requests (served by Llama 8B), freeing GPU capacity for the more compute-intensive large model.

For KIMI K2.6, co-serving increases concurrent user capacity from a baseline of ~195–199 users (GPU-only) to 352–376 users, representing gains of 1.77X-1.93X depending on the platform. The B300_AP platform (XE9780LAP with Intel® Xeon® 6979P, 120 cores) delivers the highest absolute gain at 1.93X, reflecting the larger CPU core count and higher memory bandwidth available for CPU inference offload.

For Llama 405B + Llama 8B, the gains are even more pronounced. The B300_SP platform (XE9780L with dual Intel® Xeon®6776P) achieves the highest multiplier at 2.07X (173 → 358 users), demonstrating that the dual-socket CPU configuration is particularly well-suited to efficiently serving the smaller companion model (Llama 8B). Across all platforms and workloads, CPU-GPU co-serving consistently delivers between 1.77X and 2.07X more concurrent users without any additional GPU hardware investment.

Figure 4. Concurrent user capacity - GPU-only compared with CPU-GPU co-serving

Throughput improvement analysis

Figure 5 quantifies the throughput improvements alongside minimal impact on GPU performance.

Key observations

Figure 5 measures two complementary dimensions of co-serving impact: (1) the percentage improvement in total system throughput (tokens/second) when CPU inference is added, and (2) the residual impact on GPU throughput, confirming that offloading smaller model requests to the CPU does not meaningfully degrade the performance of the GPU-resident large model.

Throughput gains for KIMI K2.6 range from +16.6% (B300_SP) to +19.8% (B300_AP), while Llama 405B + Llama 8B shows even greater improvements, reaching +21.8% on B300_SP and a peak of +25.8% on the B300_AP platform. The higher gains on B300_AP are attributable to the Intel® Xeon® 6979P's 120-core architecture and 614 GB/s memory bandwidth, which enables the CPU to handle a greater share of Llama 8B requests without contention.

Critically, the GPU impact annotations at the bottom of the chart confirm that enabling co-serving introduces minimal overhead on GPU performance: less than 1% throughput reduction in five of the six workload/platform combinations. The sole exception is KIMI K2.6 on B300_AP at under 7%, where the 120-core Xeon® 6979P draws a larger share of memory bandwidth; the same platform stays below 1% on the Llama workload. This result validates that the co-serving architecture is additive: CPU capacity augments total throughput without cannibalizing the GPU's workload.

Figure 5. Throughput improvement with CPU-GPU co-serving

Technical analysis

Co-serving architecture benefits

CPU-GPU co-serving represents a paradigm shift in AI inference deployment strategy. Rather than treating CPUs as passive system management processors, co-serving architectures actively leverage CPU compute capabilities to serve inference requests alongside GPUs.

Workload distribution strategy

In the tested configurations, the GPU serves the primary large model (for example, KIMI K2.6 at 1 trillion total parameters with 32B active, or Llama 405B). At the same time, the CPU handles a complementary model (for example, Llama 8B) or a portion of requests for the same model. Distribution considerations include:

• Model size and computational requirements relative to hardware capability
• Current system load and queue depth across GPU and CPU endpoints
• Latency targets and quality-of-service requirements per request
• Memory availability and bandwidth constraints on each compute resource

Resource utilization efficiency

Traditional GPU-only deployments leave significant CPU resources idle. In the XE9780 with two Intel Xeon 6767P processors, 128 high-performance cores with Intel AMX acceleration sit underutilized during GPU-only operation. Co-serving transforms these idle resources into productive capacity.

The CPU provides 45-50% of the total serving capacity, consistently serving 162-177 additional concurrent users, representing a +77% to +107% uplift over the GPU-only baseline.

• Near-zero resource contention: GPU performance drops below 1% in most cases; CPU and GPU inference paths operate independently.
• Near-ideal scaling at lower concurrency: At GPU32+CPU32 and GPU64+CPU64 configurations, co-serving achieves nearly 2.0X user gains. 

Platform-specific performance characteristics

B200_SP: Dell PowerEdge XE9780 + Intel Xeon 6767P + 8x NVIDIA B200

The B200_SP configuration is the primary focus platform:

• KIMI K2.6: 199 GPU-only users expanded to 352 with co-serving (1.77X gain)
• Llama 405B: 181 GPU-only users expanded to 333 with co-serving (1.84X gain)
• Consistent <1% GPU performance impact across both models
• Throughput gains of +19.3% (KIMI) and +24.6% (Llama)

The B200_SP platform's strength lies in its balance of GPU and CPU capabilities. The Intel® Xeon® 6767P provides sufficient CPU inference capacity, while the 8x NVIDIA B200 GPUs deliver class-leading AI inference performance through innovations in the Blackwell architecture.

Cost-performance analysis

CPU-GPU co-serving delivers significant TCO advantages:

• Zero additional GPU cost: The additional 77-107% user capacity comes from the CPU that is already included in the server.
• Reduced per-inference cost: With throughput improvements of 16.6-25.8%, the cost per inference request falls by approximately 14% to 21% while the system cost remains unchanged.
• Infrastructure efficiency: Co-serving eliminates the need for separate CPU-optimized servers, reducing rack space, power, and cooling requirements.
• Software-defined value: Co-serving is enabled through software configuration, requiring no hardware modifications.

Performance per dollar analysis

Deployment recommendations

The benchmarking results support the following recommendations.

Best practices for co-serving implementation

• Start with a GPU128+CPU128 configuration: This configuration provides the optimal balance of user capacity, throughput, and GPU efficiency.
• Monitor GPU performance metrics: Implement continuous monitoring of GPU inference latency and throughput.
• Model selection for CPU inference: Models up to ~13B parameters have been validated for efficient CPU execution with Intel AMX.
• Memory allocation planning: Ensure adequate DDR5 memory for CPU-side model loading (4 TB maximum capacity on the XE9780).

Hardware optimization guidelines

• Configure DDR5 memory channels for maximum bandwidth (6400 MT/s).
• Configure NVMe storage in high-performance mode for rapid model loading.
• Ensure adequate airflow for sustained co-serving workloads. 
• Utilize Intel Resource Director Technology (RDT) for cache/bandwidth allocation.

Software stack recommendations

• Deploy inference frameworks supporting heterogeneous compute (vLLM, TensorRT-LLM, llama.cpp, Intel Neural Compressor).
• Implement intelligent request routing middleware based on model size, load, and SLA requirements.
• Enable INT8/INT4 quantization for CPU-side models to maximize Intel AMX throughput.
• Configure Kubernetes with resource limits preventing CPU inference from consuming GPU driver resources.
• Implement comprehensive telemetry for both GPU and CPU inference endpoints.

Conclusion

Across two very different production workloads, a trillion-parameter MoE model and a dense 405-billion-parameter transformer, CPU-GPU co-serving delivered between 1.77X and 2.07X more concurrent users on every Dell PowerEdge platform tested, with total system throughput rising by +16.6% to +25.8%.

The decisive result did not change. GPU-side throughput stayed within 1% of the GPU-only baseline in five of the six workload and platform combinations, and within 7% in the sixth. Co-serving is shown to be additive rather than redistributive: the Intel Xeon 6 host processors absorb a second inference stream using capacity that would otherwise be idle, rather than competing with the accelerators for it.

The workload determines the right platform. The B300_AP configuration (PowerEdge XE9780LAP with the 120-core Intel® Xeon® 6979P) produced the largest throughput gains, reflecting its higher core count and 12-channel memory subsystem. The B300_SP configuration (PowerEdge XE9780L with dual Intel® Xeon® 6776P) produced the highest user-capacity multiplier at 2.07X. The air-cooled B200_SP configuration (PowerEdge XE9780 with Intel® Xeon® 6767P) delivered comparable relative gains without requiring liquid-cooling infrastructure, making it the most straightforward retrofit for an existing data center.

For organizations sizing an AI inference estate, the practical implication is that accelerator count is not the only lever available. Enabling co-serving on already-deployed PowerEdge XE9780-family systems recovers a substantial block of serving capacity from hardware that has already been purchased, powered, and cooled —with no additional GPU procurement and no measurable compromise to GPU inference quality.

Summary of key results

For enterprises evaluating AI infrastructure investments, the co-serving capability of the PowerEdge XE9780/XE9780L/XE9780LAP platform represents a significant competitive advantage. By leveraging existing CPU resources for complementary inference workloads, organizations can serve more users, achieve higher throughput, and maximize the value of their hardware investment.

 Contact us with feedback and requests.