Skip to main content

Evolving Comms Libraries in ROCm for Future AI Workloads

abstract background

This talk covers advances in AMD ROCm communication libraries for large-scale AI. RCCL innovations include copy-engine offloading, symmetric memory, GPU-initiated collectives, congestion-aware load balancing, and port failover. rocSHMEM extends an OpenSHMEM-like model with Python and Triton support, while NIXL contributions enable efficient inference. Together, they support diverse NICs like AMD Pensando AINICs and Broadcom Thor.

July 22, 2026 4:30 PM - 4:50 PM PDT

Shashidhar Gandham
Shashidhar Gandham

Corp. VP Software Development | AMD

Maria Garzaran
Maria Garzaran

Fellow Software Development | AMD

Topic

AI Training & Inference

Session Type

Tech Talk

Related Product

Instinct, ROCm

Related Sessions

View All Sessions
  • Scaling AI in Production with Vultr

    Explore how enterprises can scale AI from training to inference using AMD-powered infrastructure on Vultr. Through a deep dive into the University of Cambridge's Tessera model, learn how organizations can accelerate AI deployment, improve operational efficiency, and scale globally. The session also highlights real-world AI initiatives across healthcare, retail, finance, manufacturing, and hospitality.;Explore how enterprises can scale AI from training to inference using AMD-powered infrastructure on Vultr. Through a deep dive into the University of Cambridge's Tessera model, learn how organizations can accelerate AI deployment, improve operational efficiency, and scale globally. The session also highlights real-world AI initiatives across healthcare, retail, finance, manufacturing, and hospitality.

    July 23, 2026

  • Training at Scale with AMD Primus

    Primus makes large-scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. SOTA pre and post training performance with Primus, proven at scales of thousands of GPUs, positions an AMD Instinct GPU as a competitive solution for model development at frontier labs, enterprises, and AI startups.;Primus makes large-scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. SOTA pre and post training performance with Primus, proven at scales of thousands of GPUs, positions an AMD Instinct GPU as a competitive solution for model development at frontier labs, enterprises, and AI startups.

    July 23, 2026

  • Benchmarking AI Systems: from Model Metrics to Real-World Performance

    AI benchmarking is evolving rapidly as enterprises scale from experimentation to deployment. This interactive session explores measuring real world performance across inference and training workloads. We will discuss metrics that matter, throughput vs. latency tradeoffs, memory bandwidth, and open software ecosystems. Gain practical insights into evaluating AI infrastructure for performance, scalability, efficiency, and TCO in modern enterprise and developer environments.;AI benchmarking is evolving rapidly as enterprises scale from experimentation to deployment. This interactive session explores measuring real world performance across inference and training workloads. We will discuss metrics that matter, throughput vs. latency tradeoffs, memory bandwidth, and open software ecosystems. Gain practical insights into evaluating AI infrastructure for performance, scalability, efficiency, and TCO in modern enterprise and developer environments.

    July 23, 2026

  • Agentic Kernel Performance Tuning with AMD ROCm

    This session introduces an agentic kernel development workflow for optimizing AI and HPC workloads on AMD ROCm. Learn how a self-directing optimization loop can profile, analyze, optimize, validate, and generate production-ready kernel improvements with minimal manual tuning. The talk highlights how AMD is accelerating kernel engineering by reducing weeks of performance optimization effort into an automated, scalable workflow for developers and performance engineers.;This session introduces an agentic kernel development workflow for optimizing AI and HPC workloads on AMD ROCm. Learn how a self-directing optimization loop can profile, analyze, optimize, validate, and generate production-ready kernel improvements with minimal manual tuning. The talk highlights how AMD is accelerating kernel engineering by reducing weeks of performance optimization effort into an automated, scalable workflow for developers and performance engineers.

    July 23, 2026

  • Redefining Scalable AI Performance: OCI Supercomputing in the Cloud

    Organizations building frontier AI models need infrastructure designed for performance at scale. This session shows how OCI combines AMD Instinct, AMD EPYC, and Pensando in Oracle Acceleron to enable ultra-low-latency networking for high-throughput distributed workloads, with practical guidance for designing infrastructure for large language, multimodal, and scientific AI models.;Organizations building frontier AI models need infrastructure designed for performance at scale. This session shows how OCI combines AMD Instinct, AMD EPYC, and Pensando in Oracle Acceleron to enable ultra-low-latency networking for high-throughput distributed workloads, with practical guidance for designing infrastructure for large language, multimodal, and scientific AI models.

    July 23, 2026

  • Accelerating Inference at Scale: Crusoe's Experience with AMD

    As a customer and operator of AMD technology, Crusoe’s Managed Inference team has built a production inference stack designed for speed, efficiency, and scale. This session will show how AMD Instinct, including MI355X, helped shape its serverless inference offering and what teams can apply when building production AI services that balance performance, memory bandwidth, and cost.;As a customer and operator of AMD technology, Crusoe’s Managed Inference team has built a production inference stack designed for speed, efficiency, and scale. This session will show how AMD Instinct, including MI355X, helped shape its serverless inference offering and what teams can apply when building production AI services that balance performance, memory bandwidth, and cost.

    July 23, 2026

  • Efficient LLM Serving at Scale with Unified Caching (vLLM+LMCache)

    This is an advanced user hands-on workshop to show TensorMesh and AMD enabling efficient LLM serving through an unified caching layer. You will learn how tiered KV cache management can brings out the benefits of cache-aware inference, improving throughput under interactive latency SLAs, reducing TTFT through KV cache reuse/offload & enabling production-style distributed inference on Instinct GPUs.;This is an advanced user hands-on workshop to show TensorMesh and AMD enabling efficient LLM serving through an unified caching layer. You will learn how tiered KV cache management can brings out the benefits of cache-aware inference, improving throughput under interactive latency SLAs, reducing TTFT through KV cache reuse/offload & enabling production-style distributed inference on Instinct GPUs.

    July 23, 2026

  • Accelerating LLM Inference on AMD ROCm with AITER and ATOM

    This technical talk introduces AITER and ATOM, optimized inference technologies for AMD ROCm software. Learn how AITER accelerates LLM and MoE execution with optimized kernels and distributed inference enhancements, while ATOM integrates these capabilities into familiar vLLM and SGLang workflows through plugin-based acceleration. The session highlights how AMD enables scalable, high-performance open-source LLM serving while preserving existing developer and deployment workflows.;This technical talk introduces AITER and ATOM, optimized inference technologies for AMD ROCm software. Learn how AITER accelerates LLM and MoE execution with optimized kernels and distributed inference enhancements, while ATOM integrates these capabilities into familiar vLLM and SGLang workflows through plugin-based acceleration. The session highlights how AMD enables scalable, high-performance open-source LLM serving while preserving existing developer and deployment workflows.

    July 23, 2026