KV Cache Offload to Object Storage: GPU-Direct and Upstream

• • Object storage is now a first-class target for high-performance KV cache offload. RDMA lands cache data directly in GPU memory with no host bounce buffer — 837 ms to first token at 235K-token context,^¹ on an S3 endpoint. The performance reason to keep object out of the inference path is gone.
  • An industry first: Dell Technologies, with NVIDIA, contributed an accelerated engine for the NIXL OBJ plugin — merged upstream — so vLLM, LMCache and NIXL can offload KV cache to Dell ObjectScale over RDMA, landing directly in GPU memory via cuObject. No fork, no custom client.
  • Dell measured a 13.4× faster time to first token^¹ at 235K-token context versus recomputing prefill — 837 ms compared with 11,223 ms — and 1.3–1.5× faster than the same offload over S3-HTTP.²
  • Within the Dell AI Data Platform, you pick the engine. PowerScale for file, ObjectScale for object, both GPU-direct over RDMA — the offload target follows your data architecture rather than dictating it.

When inference latency climbs, the reflex is to buy more GPUs. It’s the most expensive lever available, and it treats a symptom rather than the cause: your GPUs aren’t short of compute, they’re repeating work they’ve already done. Reusing cached context instead of recomputing it addresses that directly, on storage priced per terabyte rather than per accelerator.

The scale of the problem is easy to underestimate. In Dell’s testing, a single 235K-token request produced a 43 GB KV cache¹ — large enough that a typical GPU holds only a few such requests in HBM, and far too valuable to discard and recompute.

This is why KV cache offload has moved from an optimization to an architectural decision across enterprises, large CSPs and AI-native builders alike. Tier the cache into fast shared storage, retrieve it when the same context comes back and the prefill bill stops repeating itself.

Which puts the weight on the storage underneath. Object storage earns its place in high-performance AI inference on three things: a GPU-direct data path, native support in the inference stack you already run and scale that holds for your data lake as well as your cache. Dell ObjectScale now meets all three. Both Storage Engines in the Dell AI Data Platform — PowerScale for file, ObjectScale for object — now carry a GPU-direct RDMA path.

Object joins the fast path

Object is the natural fit at scale — a shareable, multi-instance cache maps cleanly onto an object namespace, and it’s where much enterprise AI data already lives. Dell and NVIDIA previously solved the transport with RDMA for S3-compatible storage on ObjectScale, bringing zero-copy, kernel-bypass transfers to the S3 API — data moving directly between GPU memory and the storage server, without the HTTP stack in between.

What the inference stack couldn’t do was use it. LMCache’s route to object storage still ran over S3-HTTP, with the CPU and a host memory copy in the data path, so the fast lane for KV cache offload belonged to file.

What’s new is that this path now runs inside the standard open-source inference stack. Dell contributed an accelerated engine for the NIXL OBJ plugin — RDMA-based asynchronous PUT/GET, GPU memory (VRAM) segment support and runtime engine selection — with every enhancement merged upstream rather than maintained in a private fork.

The practical consequence: if you run vLLM and LMCache today, you point KV cache offload at Dell ObjectScale and get GPU-direct, RDMA-accelerated performance from the upstream stack rather than a vendor-specific one. Dell’s contributions are in the community source today — LMCache v0.4.5 and NVIDIA NIXL v1.1.0 — with official NIXL builds carrying this support to follow. The cuObject client libraries already ship inside the NVIDIA CUDA Toolkit (v13.1 and later), so there is no additional client to install. The core inference stack remains upstream; only the ObjectScale backend configuration is Dell-specific.

Upstream placement is what makes this durable. When an accelerated data path lives in a proprietary client or a forked inference engine, your upgrade cadence is coupled to a vendor’s release schedule. Here it isn’t — the code moves with vLLM, LMCache and NIXL, on the community’s timeline.

Scale is where object compounds. A KV cache is shareable across instances and maps cleanly onto an object namespace — and that namespace is often already holding the corpus you’re retrieving against. One store, one governance model, serving both the data and the cache built from it.

Benchmark results

Dell benchmarked the full open-source stack on a PowerEdge XE9680 server with NVIDIA H100 GPUs and ConnectX-7 NICs, serving Qwen3-Coder-30B-A3B-Instruct at tensor parallelism 4. TTFT was measured as the median of five hot passes on a single request, across context sizes from 4K to 235K tokens.

Three findings matter commercially:

1. 1. At long contexts the gap is decisive. 837 ms versus 11,223 ms^¹ separates an application that feels responsive from one users abandon. Dell expects larger models to benefit further still: the test model’s KV cache footprint per token is modest by current standards, so more of the cache stays offloadable as model size grows.
   2. RDMA consistently beats HTTP, and the margin widens with scale. This comparison is the one to watch, because it isolates what the new work contributes — the baseline comparison shows the value of caching at all; the HTTP comparison shows what GPU-direct RDMA adds on top.
   3. CPU utilization drops roughly 90% against the HTTP path, releasing host resources for work that differentiates your application.

One honest caveat, because it shapes deployment. Below roughly 8K–16K tokens, offload overhead outweighs the benefit and the baseline wins — at 4K context, baseline TTFT was 91 ms versus 113–129 ms with offload. This is a long-context capability. If you serve code assistants, document analysis, RAG over large corpora, or agentic workflows carrying long histories, it’s squarely your case.

File or object: your architecture decides

Within the Dell AI Data Platform, KV cache offload runs on purpose-built Storage Engines — with the flexibility to choose. Your archive tier, training pipeline and RAG corpus have different access patterns and different cost profiles. Forcing them onto a single engine means at least one of them runs on the wrong thing. Dell builds PowerScale and ObjectScale for their own jobs, accelerates both over RDMA and keeps governance and security consistent across them — so an existing PowerScale estate takes on AI work rather than becoming something to migrate off first.

Object no longer costs you the fast path, so the decision returns to where it belongs: your data architecture, your governance model, your scale requirements. High-performance object storage used to be a contradiction in terms for AI inference. It isn’t anymore — and because the work landed upstream, it’s in the community’s hands today.

The trade is worth restating plainly: 11 seconds of GPU time rebuilding context, replaced by 837 ms of storage retrieval — and the accelerator freed for work only it can do.

For the implementation detail — LMCache and cuFile configuration, NIXL buffer sizing and the full benchmark methodology — please read on: Accelerating AI Inference with Open Source and Dell ObjectScale.

If you’re scaling inference and want to put this to work, explore Dell ObjectScale or contact your Dell account team.    

Frequently Asked Questions

What is the best object storage for high-performance AI workloads?

Look for three things: a GPU-direct RDMA data path rather than S3 over HTTP, native support in the upstream inference stack you already run and proven scale for your data lake as well as your cache. Dell ObjectScale delivers all three.

Do I have to change my inference stack to use this?

No fork and no proprietary client. Dell’s contributions are merged upstream in LMCache v0.4.5 and NIXL v1.1.0 source, with official NIXL builds to follow — so the accelerated path arrives through the community stack you already run, not a Dell-specific one.

Can object storage really match file performance for AI?

RDMA removes the copies that made object slower, delivering file-like behavior; note that file and object were not benchmarked against each other here.

Which is more cost-effective — more GPUs or faster storage?

For long-context inference where prefill is being recomputed, storage addresses the cause at a far lower cost per unit. Adding accelerators buys more capacity to redo work; caching removes the work. At 235K tokens that’s 11,223 ms of recomputed prefill replaced by an 837 ms retrieval — on capacity priced per terabyte rather than per accelerator.

Does this help with RAG and agentic workloads?

Those are strong candidates. RAG grounds each query in retrieved context, and agentic workflows carry long histories forward — both produce the long, frequently repeated prompts that KV cache offload is built for. The same ObjectScale namespace holding your RAG corpus can serve the cache, which is where object’s scale advantage compounds.

How does this relate to NVIDIA’s inference stack?

Dell built it with NVIDIA. The accelerated engine sits in NVIDIA’s NIXL data-movement library, transfers run over ConnectX-series NICs and GPU-direct placement uses NVIDIA cuObject — which ships in the CUDA Toolkit from v13.1.

Should I use file or object for KV cache offload?

Both work, and both are now GPU-direct over RDMA, so pick on architecture rather than performance. File suits established file-based AI pipelines and mixed enterprise workloads already running on PowerScale. Object suits exabyte-scale data lakes, RAG corpora and a shareable multi-instance cache — particularly when the same ObjectScale namespace already holds the data you’re retrieving against.

Can I use my existing storage rather than starting over?

That’s the intent of a platform approach. PowerScale deployments already running enterprise workloads take on KV cache offload with 9.11.0, so file estates become AI infrastructure rather than something to migrate off before AI work can begin.

When can I get it?

RDMA for S3-compatible storage is available in ObjectScale 4.3 as a customer preview, with general availability in ObjectScale 4.4.

^1,2 Based on Dell internal testing, Aug. 2026.
Configuration: Dell PowerEdge XE9680 server with NVIDIA H100 GPUs and NVIDIA ConnectX-7 NICs, Dell ObjectScale 4.3, vLLM with LMCache v0.4.5 and NVIDIA NIXL v1.1.0, serving Qwen/Qwen3-Coder-30B-A3B-Instruct at tensor parallelism 4. Time to first token measured as the median of five hot passes at concurrency = 1, single request, at 235K-token context length. Baseline configuration is pure vLLM with no prefix caching or external KV cache. Actual results will vary based on model, context length, concurrency, configuration, and workload.