Skip to main content

Building Next-Gen AI Infrastructure: Scaling Enterprise LLM Serving with RadixArk

abstract background

Deploying generative AI at scale requires robust and highly optimized infrastructure. Join Ying Sheng, Co-creator of SGLang and Founder of RadixArk, to learn how innovations such as structured generation and advanced scheduling enable efficient, enterprise-ready AI deployments. Discover techniques for maximizing utilization, reducing latency, and scaling AI applications across diverse environments.

July 22, 2026 2:30 PM - 2:50 PM PDT

Ying Sheng
Ying Sheng

Founder & CEO | RadixArk

Topic

AI Training & Inference

Developer Platforms & Open Ecosystems

Session Type

Tech Talk

Related Product

Instinct, ROCm

Related Sessions

View All Sessions
  • Domain-Specific AI at Scale: Open Models, Post-Training, and AI Infrastructure

    Learn how domain-specific AI moves beyond generic models using post-training, domain evals, and scalable open infrastructure. Using Open Telco Models as a case study, this session covers curated data, reward loops, unified training and serving, and AMD Instinct/ROCm-based stacks for building specialized AI systems at enterprise scale.;Learn how domain-specific AI moves beyond generic models using post-training, domain evals, and scalable open infrastructure. Using Open Telco Models as a case study, this session covers curated data, reward loops, unified training and serving, and AMD Instinct/ROCm-based stacks for building specialized AI systems at enterprise scale.

    July 23, 2026

  • Training at Scale with AMD Primus

    Primus makes large-scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. SOTA pre and post training performance with Primus, proven at scales of thousands of GPUs, positions an AMD Instinct GPU as a competitive solution for model development at frontier labs, enterprises, and AI startups.;Primus makes large-scale training on Instinct reliable, debuggable and highly performant. It supports the latest OSS training frameworks, models, and is expanding support to new, cutting-edge model architectures, training techniques, and datatypes. SOTA pre and post training performance with Primus, proven at scales of thousands of GPUs, positions an AMD Instinct GPU as a competitive solution for model development at frontier labs, enterprises, and AI startups.

    July 23, 2026

  • Scaling AI in Production with Vultr

    Explore how enterprises can scale AI from training to inference using AMD-powered infrastructure on Vultr. Through a deep dive into the University of Cambridge's Tessera model, learn how organizations can accelerate AI deployment, improve operational efficiency, and scale globally. The session also highlights real-world AI initiatives across healthcare, retail, finance, manufacturing, and hospitality.;Explore how enterprises can scale AI from training to inference using AMD-powered infrastructure on Vultr. Through a deep dive into the University of Cambridge's Tessera model, learn how organizations can accelerate AI deployment, improve operational efficiency, and scale globally. The session also highlights real-world AI initiatives across healthcare, retail, finance, manufacturing, and hospitality.

    July 23, 2026

  • Accelerating vLLM Inference on AMD Instinct GPUs with AMD ATOM

    This advanced hands-on workshop introduces AMD ATOM, an open-source optimized LLM inference backend for ROCm. Learn to serve LLMs with popular workflows using AMD-optimized attention & inference kernels. The Workshop introduces out-of-tree plugins for existing vLLM & SGLang users & aims at demonstrating how ATOM preserves familiarity of the frameworks while accelerating model execution & boosting inference performance, bridging opensource frameworks with the AMD high-performance inference stack.;This advanced hands-on workshop introduces AMD ATOM, an open-source optimized LLM inference backend for ROCm. Learn to serve LLMs with popular workflows using AMD-optimized attention & inference kernels. The Workshop introduces out-of-tree plugins for existing vLLM & SGLang users & aims at demonstrating how ATOM preserves familiarity of the frameworks while accelerating model execution & boosting inference performance, bridging opensource frameworks with the AMD high-performance inference stack.

    July 23, 2026

  • Lessons Learned Training ZAYA1-74B End-to-End on AMD

    ZAYA1-74B is a 74B-parameter MoE (4B active) pretrained and post-trained end-to-end on AMD MI300X. We share what it took: a co-designed stack with CCA attention, expert-context-parallel folding, and sliding-window layers that halve KV cache, plus an RL pipeline that sharpens math and code and hardens multi-turn agentic tool use. The result is a competitive long-context, Apache 2.0 model, proving the full train-to-RL loop runs effectively on AMD.;ZAYA1-74B is a 74B-parameter MoE (4B active) pretrained and post-trained end-to-end on AMD MI300X. We share what it took: a co-designed stack with CCA attention, expert-context-parallel folding, and sliding-window layers that halve KV cache, plus an RL pipeline that sharpens math and code and hardens multi-turn agentic tool use. The result is a competitive long-context, Apache 2.0 model, proving the full train-to-RL loop runs effectively on AMD.

    July 23, 2026

  • Agentic Kernel Performance Tuning with AMD ROCm

    This session introduces an agentic kernel development workflow for optimizing AI and HPC workloads on AMD ROCm. Learn how a self-directing optimization loop can profile, analyze, optimize, validate, and generate production-ready kernel improvements with minimal manual tuning. The talk highlights how AMD is accelerating kernel engineering by reducing weeks of performance optimization effort into an automated, scalable workflow for developers and performance engineers.;This session introduces an agentic kernel development workflow for optimizing AI and HPC workloads on AMD ROCm. Learn how a self-directing optimization loop can profile, analyze, optimize, validate, and generate production-ready kernel improvements with minimal manual tuning. The talk highlights how AMD is accelerating kernel engineering by reducing weeks of performance optimization effort into an automated, scalable workflow for developers and performance engineers.

    July 23, 2026

  • Dell Technologies: Balancing CPU and GPU for the Agentic AI Era

    Agentic AI changes the infrastructure equation. These workloads plan, branch, call tools, retrieve data, manage long context, and re-plan in real time, creating far greater CPU demand than traditional GPU-heavy inference. In this Meet the Expert session, learn why balanced Dell PowerEdge platforms with 1:1 vs. 1:8 AMD EPYC CPUs and AMD Instinct MI350P GPUs are critical to improving latency, GPU utilization, and efficiency for production-ready agentic AI.;Agentic AI changes the infrastructure equation. These workloads plan, branch, call tools, retrieve data, manage long context, and re-plan in real time, creating far greater CPU demand than traditional GPU-heavy inference. In this Meet the Expert session, learn why balanced Dell PowerEdge platforms with 1:1 vs. 1:8 AMD EPYC CPUs and AMD Instinct MI350P GPUs are critical to improving latency, GPU utilization, and efficiency for production-ready agentic AI.

    July 23, 2026

  • Building GPU Kernels in Python with ROCm FlyDSL

    This advanced hands-on workshop introduces ROCm FlyDSL, a Python-based domain-specific language (DSL) for developing GPU kernels with low-level control on AMD GPUs. Attendees will receive a concise introduction to FlyDSL and learn how to implement GPU kernels in pure Python using the library. The workshop will also showcase how FlyDSL is used in production to improve end-to-end serving performance of Large Language Models like Kimi K2.5.;This advanced hands-on workshop introduces ROCm FlyDSL, a Python-based domain-specific language (DSL) for developing GPU kernels with low-level control on AMD GPUs. Attendees will receive a concise introduction to FlyDSL and learn how to implement GPU kernels in pure Python using the library. The workshop will also showcase how FlyDSL is used in production to improve end-to-end serving performance of Large Language Models like Kimi K2.5.

    July 23, 2026