ROCm AMD Infinity Context: Shared KV Cache for Agentic AI
Abstract
As LLM inference shifts toward long context, multi-turn sessions, and agents, KV cache becomes a dominant scaling constraint that no longer fits in GPU HBM. Existing caching solutions partially solve this using expensive CPU memory, non-optimized remote storage or local non-sharable NVMe SSDs. ROCm AIC proposes a shared across GPUs, low latency storage tier to offload KV cache. This session gives insight into the ROCm AIC architecture and provides measured performance, backed by partner vendors.
July 23, 2026 3:00 PM - 3:30 PM PDT
Speakers
Presented By
Director, Product Management, Infrastructure Software and Ecosystem | AMD
Fellow, AI Storage Software & Architecture | AMD