Skip to main content

ROCm AMD Infinity Context: Shared KV Cache for Agentic AI

abstract background

As LLM inference shifts toward long context, multi-turn sessions, and agents, KV cache becomes a dominant scaling constraint that no longer fits in GPU HBM. Existing caching solutions partially solve this using expensive CPU memory, non-optimized remote storage or local non-sharable NVMe SSDs. ROCm AIC proposes a shared-across-GPUs, low-latency storage tier to offload KV cache. This session gives insight into the ROCm AIC architecture and provides measured performance, backed by partner vendors.

July 23, 2026 3:00 PM - 3:30 PM PDT

Juergen Frick
Juergen Frick

Director, Product Management, Infrastructure Software and Ecosystem | AMD

Stephen Bates
Stephen Bates

Fellow, AI Storage Software & Architecture | AMD

Topic

AI Training & Inference

Enterprise AI

Session Type

Meet the Experts

Related Sessions

View All Sessions
  • Lessons Learned Training ZAYA1-74B End-to-End on AMD

    ZAYA1-74B is a 74B-parameter MoE (4B active) pretrained and post-trained end-to-end on AMD MI300X. We share what it took: a co-designed stack with CCA attention, expert-context-parallel folding, and sliding-window layers that halve KV cache, plus an RL pipeline that sharpens math and code and hardens multi-turn agentic tool use. The result is a competitive long-context, Apache 2.0 model, proving the full train-to-RL loop runs effectively on AMD.;ZAYA1-74B is a 74B-parameter MoE (4B active) pretrained and post-trained end-to-end on AMD MI300X. We share what it took: a co-designed stack with CCA attention, expert-context-parallel folding, and sliding-window layers that halve KV cache, plus an RL pipeline that sharpens math and code and hardens multi-turn agentic tool use. The result is a competitive long-context, Apache 2.0 model, proving the full train-to-RL loop runs effectively on AMD.

    July 23, 2026

  • Right Size Your Memory Footprint to Move IT Refresh Forward

    Memory has rarely been in such short supply and is impeding customer data center refresh plans. In this interactive conversation, we’ll discuss tips and tools for right-sizing memory configurations to help move your data center efficiency initiatives forward and preserve ROI. Bring your questions and our experts will provide answers!;Memory has rarely been in such short supply and is impeding customer data center refresh plans. In this interactive conversation, we’ll discuss tips and tools for right-sizing memory configurations to help move your data center efficiency initiatives forward and preserve ROI. Bring your questions and our experts will provide answers!

    July 23, 2026

  • Scaling AI in Production with Vultr

    Explore how enterprises can scale AI from training to inference using AMD-powered infrastructure on Vultr. Through a deep dive into the University of Cambridge's Tessera model, learn how organizations can accelerate AI deployment, improve operational efficiency, and scale globally. The session also highlights real-world AI initiatives across healthcare, retail, finance, manufacturing, and hospitality.;Explore how enterprises can scale AI from training to inference using AMD-powered infrastructure on Vultr. Through a deep dive into the University of Cambridge's Tessera model, learn how organizations can accelerate AI deployment, improve operational efficiency, and scale globally. The session also highlights real-world AI initiatives across healthcare, retail, finance, manufacturing, and hospitality.

    July 23, 2026

  • Training at Scale with AMD Primus

    Primus makes large-scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. SOTA pre and post training performance with Primus, proven at scales of thousands of GPUs, positions an AMD Instinct GPU as a competitive solution for model development at frontier labs, enterprises, and AI startups.;Primus makes large-scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. SOTA pre and post training performance with Primus, proven at scales of thousands of GPUs, positions an AMD Instinct GPU as a competitive solution for model development at frontier labs, enterprises, and AI startups.

    July 23, 2026

  • Inside AMD IT: Our Enterprise AI Journey

    How is AMD using AI to transform its own enterprise IT? Go inside AMD IT's AI journey – from early experimentation to scaled deployment. Learn about AMD IT’s multi-year AI strategy and its strategic investment in Data platform to enable AI to automate mega process workflows, accelerate delivery, and drive measurable business outcomes. Whether you are starting your AI journey or scaling existing initiatives, this insider view offers practical lessons on strategy, adoption, and change.;How is AMD using AI to transform its own enterprise IT? Go inside AMD IT's AI journey – from early experimentation to scaled deployment. Learn about AMD IT’s multi-year AI strategy and its strategic investment in Data platform to enable AI to automate mega process workflows, accelerate delivery, and drive measurable business outcomes. Whether you are starting your AI journey or scaling existing initiatives, this insider view offers practical lessons on strategy, adoption, and change.

    July 23, 2026

  • Build an MRI Analysis Agent with AMD Blueprints

    Build and deploy an AI-powered MRI analysis agent in minutes using the AMD mri-doc Solution Blueprint. Run a Gradio-based pipeline that accepts DICOM, NIfTI, and standard image formats, applies tissue segmentation and anomaly detection, and generates LLM-drafted clinical reports on AMD Instinct GPUs. Then customize: swap the LLM AIM, reuse an existing model endpoint, or extend the pipeline for your specific clinical workflow.;Build and deploy an AI-powered MRI analysis agent in minutes using the AMD mri-doc Solution Blueprint. Run a Gradio-based pipeline that accepts DICOM, NIfTI, and standard image formats, applies tissue segmentation and anomaly detection, and generates LLM-drafted clinical reports on AMD Instinct GPUs. Then customize: swap the LLM AIM, reuse an existing model endpoint, or extend the pipeline for your specific clinical workflow.

    July 23, 2026

  • Benchmarking AI Systems: from Model Metrics to Real-World Performance

    AI benchmarking is evolving rapidly as enterprises scale from experimentation to deployment. This interactive session explores measuring real world performance across inference and training workloads. We will discuss metrics that matter, throughput vs. latency tradeoffs, memory bandwidth, and open software ecosystems. Gain practical insights into evaluating AI infrastructure for performance, scalability, efficiency, and TCO in modern enterprise and developer environments.;AI benchmarking is evolving rapidly as enterprises scale from experimentation to deployment. This interactive session explores measuring real world performance across inference and training workloads. We will discuss metrics that matter, throughput vs. latency tradeoffs, memory bandwidth, and open software ecosystems. Gain practical insights into evaluating AI infrastructure for performance, scalability, efficiency, and TCO in modern enterprise and developer environments.

    July 23, 2026

  • Creating CPU Inference and Agentic Performance Transparency

    Your finance team doesn't care about tokens per second. They care about predictable costs, compliance risk, and vendor lock-in. With agentic AI, the metrics for tracking success are even more complex. But benchmarks don't answer the question that actually matters: Should you undertake this effort and is it viable for your business? In this interactive technical discussion, we’ll break down the tradeoffs, work through the math, and pressure-test the strategy together.;Your finance team doesn't care about tokens per second. They care about predictable costs, compliance risk, and vendor lock-in. With agentic AI, the metrics for tracking success are even more complex. But benchmarks don't answer the question that actually matters: Should you undertake this effort and is it viable for your business? In this interactive technical discussion, we’ll break down the tradeoffs, work through the math, and pressure-test the strategy together.

    July 23, 2026