Skip to main content

Building Next-Gen AI Infrastructure: Scaling Enterprise LLM Serving with RadixArk

abstract background

Deploying generative AI in production requires a robust, scalable, and highly optimized infrastructure. Join Ying Sheng, founder of SGLang and RadixArk, to discover how to build a unified and efficient LLM serving stack. This session will cover the transition from open-source innovations in SGLang (like fast structured generation and advanced scheduling) to enterprise-ready deployments with RadixArk. We will demonstrate how our advanced serving engines maximize compute utilization, minimize latency, and scale AI applications seamlessly across diverse environments—ultimately building the foundations needed to bring the power of frontier AI to everyone.

July 22, 2026 14:30 - 14:50

Ying Sheng
Ying Sheng

Founder & CEO | RadixArk

Topic

AI Training & Inference

Developer Platforms & Open Ecosystems

Session Type

Tech Talk

Related Product

Instinct, ROCm

Related Sessions

View All Sessions
  • Unlocking LLM Inference Performance with ROCm FlyDSL

    This advanced hands-on workshop introduces ROCm FlyDSL, a Python-based domain-specific language (DSL) for developing high-performance GPU kernels with low-level control on AMD GPUs. Attendees will receive a concise introduction to FlyDSL and learn how to implement high-performance kernels using the library. The workshop will also showcase practical optimization techniques for improving end-to-end serving performance of the Kimi K2.5 model using optimized FlyDSL Mixture-of-Experts (MoE) kernels.;This advanced hands-on workshop introduces ROCm FlyDSL, a Python-based domain-specific language (DSL) for developing high-performance GPU kernels with low-level control on AMD GPUs. Attendees will receive a concise introduction to FlyDSL and learn how to implement high-performance kernels using the library. The workshop will also showcase practical optimization techniques for improving end-to-end serving performance of the Kimi K2.5 model using optimized FlyDSL Mixture-of-Experts (MoE) kernels.

    July 23, 2026

  • ElMerFold: Exascale AI for Protein Structure Prediction with El Capitan

    DProtein structure prediction is foundational to modern biology, enabling breakthroughs in drug discovery, enzyme engineering, and AI-driven science. We present ElMerFold, a production-scale synthetic data generation workflow running on the El Capitan system at 11,000 nodes and 44,000 APUs. ElMerFold processes ~41 million proteins at 2,378 structures/s, achieving a 16.3× improvement over prior approaches and reaching 969 PFLOP/s FP32 inference performance.;DProtein structure prediction is foundational to modern biology, enabling breakthroughs in drug discovery, enzyme engineering, and AI-driven science. We present ElMerFold, a production-scale synthetic data generation workflow running on the El Capitan system at 11,000 nodes and 44,000 APUs. ElMerFold processes ~41 million proteins at 2,378 structures/s, achieving a 16.3× improvement over prior approaches and reaching 969 PFLOP/s FP32 inference performance.

    July 23, 2026

  • Accelerating vLLM Inference on AMD Instinct GPUs with AMD ATOM

    This advanced hands-on workshop introduces AMD ATOM an opensource optimized LLM inference backend for ROCm. Learn to serve LLMs with popular workflows using AMD-optimized attention & inference kernels. The Workshop introduces out-of-tree plugins for existing vLLM & SGLang users & aims at demonstrating how ATOM preserves familiarity of the frameworks while accelerating model execution & boosting inference performance, bridging opensource frameworks with the AMD high-performance inference stack.;This advanced hands-on workshop introduces AMD ATOM an opensource optimized LLM inference backend for ROCm. Learn to serve LLMs with popular workflows using AMD-optimized attention & inference kernels. The Workshop introduces out-of-tree plugins for existing vLLM & SGLang users & aims at demonstrating how ATOM preserves familiarity of the frameworks while accelerating model execution & boosting inference performance, bridging opensource frameworks with the AMD high-performance inference stack.

    July 23, 2026

  • Training at Scale with AMD Primus

    Primus makes large scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. Primus’ SOTA pre and post training performance, proven at scales of thousands of GPUs, positions instinct as a competitive solution for model development at frontier labs, enterprises and AI startups.;Primus makes large scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. Primus’ SOTA pre and post training performance, proven at scales of thousands of GPUs, positions instinct as a competitive solution for model development at frontier labs, enterprises and AI startups.

    July 23, 2026

  • Benchmarking AI Systems: from Model Metrics to Real-World Performance

    AI benchmarking is evolving rapidly as enterprises scale from experimentation to deployment. This interactive session explores measuring real world performance across inference and training workloads. We will discuss metrics that matter, throughput vs. latency tradeoffs, memory bandwidth, and open software ecosystems. Gain practical insights into evaluating AI infrastructure for performance, scalability, efficiency, and TCO in modern enterprise and developer environments.;AI benchmarking is evolving rapidly as enterprises scale from experimentation to deployment. This interactive session explores measuring real world performance across inference and training workloads. We will discuss metrics that matter, throughput vs. latency tradeoffs, memory bandwidth, and open software ecosystems. Gain practical insights into evaluating AI infrastructure for performance, scalability, efficiency, and TCO in modern enterprise and developer environments.

    July 23, 2026

  • Agentic Kernel Performance Tuning with AMD ROCm

    This session introduces an agentic kernel development workflow for optimizing AI and HPC workloads on AMD ROCm. Learn how a self-directing optimization loop can profile, analyze, optimize, validate, and generate production-ready kernel improvements with minimal manual tuning. The talk highlights how AMD is accelerating kernel engineering by reducing weeks of performance optimization effort into an automated, scalable workflow for developers and performance engineers.;This session introduces an agentic kernel development workflow for optimizing AI and HPC workloads on AMD ROCm. Learn how a self-directing optimization loop can profile, analyze, optimize, validate, and generate production-ready kernel improvements with minimal manual tuning. The talk highlights how AMD is accelerating kernel engineering by reducing weeks of performance optimization effort into an automated, scalable workflow for developers and performance engineers.

    July 23, 2026

  • Accelerating LLM Inference on AMD ROCm with AITER and ATOM

    This technical talk introduces AITER and ATOM, optimized inference technologies for AMD ROCm software. Learn how AITER accelerates LLM and MoE execution with optimized kernels and distributed inference enhancements, while ATOM integrates these capabilities into familiar vLLM and SGLang workflows through plugin-based acceleration. The session highlights how AMD enables scalable, high-performance open-source LLM serving while preserving existing developer and deployment workflows.;This technical talk introduces AITER and ATOM, optimized inference technologies for AMD ROCm software. Learn how AITER accelerates LLM and MoE execution with optimized kernels and distributed inference enhancements, while ATOM integrates these capabilities into familiar vLLM and SGLang workflows through plugin-based acceleration. The session highlights how AMD enables scalable, high-performance open-source LLM serving while preserving existing developer and deployment workflows.

    July 23, 2026

  • Power is Your Biggest Hidden Cost: How AMD Can Help

    Power is the AI infrastructure cost nobody budgets for until it breaks the business case. In this interactive technical session, an expert from 5C joins AMD to unpack how power consumption impacts total cost of ownership across inference and training deployments. Discuss how intelligent power management, real-world thermal constraints, and silicon-level efficiency shape what your AI infrastructure can sustain. Practical insight for architects and operators making deployment decisions today.;Power is the AI infrastructure cost nobody budgets for until it breaks the business case. In this interactive technical session, an expert from 5C joins AMD to unpack how power consumption impacts total cost of ownership across inference and training deployments. Discuss how intelligent power management, real-world thermal constraints, and silicon-level efficiency shape what your AI infrastructure can sustain. Practical insight for architects and operators making deployment decisions today.

    July 23, 2026