Resilient Scale-Up Networking Built for Production AI

Performance Starts with the Scale-Up Fabric

AI infrastructure is becoming increasingly dependent on tightly coordinated systems rather than individual accelerators. As model sizes grow and AI deployments move into production, performance is determined not only by compute but also by how efficiently GPUs communicate, share data, and operate together as a unified system. This shift has elevated scale-up networking from a high-performance rack-level interconnect to a foundational architectural layer for AI infrastructure. At the same time, production AI environments demand more than raw bandwidth and low latency—they require resilience, observability, and operational continuity at rack scale.

At Advancing AI 2026, AMD is introducing new scale-up networking capabilities that extend this focus beyond network performance to include production-grade reliability.

Why Scale-Up Matters Now

The demands on AI infrastructure have changed fundamentally. Frontier models continue to grow in size, increasingly exceeding the memory capacity of a single GPU, requiring GPUs to work together as a unified system. Training workloads require tight synchronization across large numbers of GPUs. Distributed inference depends on rapid communication between accelerators to support larger models, longer context windows, and growing KV-cache requirements. Agentic AI introduces persistent, context-rich workloads that require continuous access to compute and memory resources with minimal interruption.

Together, these trends are increasing the importance of a high-bandwidth, low-latency communication domain within the rack. Scale-up networking enables GPUs to exchange data, coordinate memory access, and synchronize efficiently without traversing the broader cluster network. As AI systems grow, the efficiency of this rack-scale domain increasingly determines how much of the system’s available compute performance can be realized.

AMD Helios™ Rack-Scale Solution -  Scale-Up Platform

AMD Helios was designed as a rack-scale AI system in which compute, memory, networking, and software operate together from the ground up — not assembled from independent components after the fact.

At the center of the Helios scale-up architecture is Ultra Accelerator Link over Ethernet (UALoE), which connects all 72 AMD Instinct™ MI455X GPUs in the rack into a single unified domain. Designed with up to 260 TB/s of aggregate scale-up bandwidth and access to up to 31 TB of HBM4 memory within a single rack, AMD Helios enables the model  and workload sizes that require tightly coupled GPU communication.

The fabric is built on a single-hop topology using standard Ethernet switch ASICs, helping ensure every GPU can reach every other GPU through a single switch — minimizing latency, improving congestion management, and maximizing communication efficiency across the network. UALoE runs on standard Ethernet — the same mature, proven ecosystem the industry already knows — rather than a proprietary interconnect. AMD contributed to ESUN, an open industry workstream advancing Ethernet for scale-up AI, helping shape the interoperable foundation the industry is building on.

Hardware Resilience Built for the Reality of Production AI

Scale-up networking that works under ideal conditions is not enough. At rack scale, certain hardware events are inevitable: links fail, components require maintenance, switches are replaced. Training jobs run for hours. Inference services run continuously. When failures occur in a system of this scale, their impact on productivity is amplified.

The design of the AMD Helios scale-up fabric comes with hardware resilience as a core principle, not a secondary consideration. Each GPU connects through 18 independent UALoE stations distributed across different switch trays. When a hardware event affects one path, the system maintains the unified GPU domain using alternate connections — automatically, without operator intervention, and without terminating the workload.

The result is an architecture that keeps AI systems operational under real-world conditions, not just peak-performance benchmarks.

Software That Makes Resilience Operational

Hardware resilience is only as effective as the software that manages it.

AMD Fabric Manager (AFM) provides the operational layer for the AMD Helios scale-up fabric — handling deployment automation, provisioning, telemetry, observability, and fabric-wide management through a single framework. Rather than managing networking components individually, AFM gives operators visibility across the entire scale-up domain, making it significantly easier to deploy, monitor, and maintain rack-scale AI infrastructure as environments grow.

Running alongside AFM, AMD Fabric OS (AFOS) operates directly on the scale-up switching layer, providing the real-time control and visibility needed to detect and respond to hardware events within the fabric itself. Together, AFM and AFOS transform the AMD Helios scale-up network from a hardware interconnect into a managed, observable operational platform.

Flexible Deployment with vPods

Not all workloads need the full 72-GPU domain simultaneously. AMD introduces vPods — Virtual Pods — which allow customers to partition the AMD Helios scale-up domain into smaller, software-defined GPU environments tailored to specific workloads.

vPods enable organizations to run concurrent workloads on the same infrastructure: separate training runs, testing environments, and inference services can coexist on the same rack while maintaining workload isolation and fault containment. This flexibility helps improve infrastructure utilization and simplifies operational management without sacrificing the performance of the underlying scale-up architecture.

Built for What's Next

The next generation of AI will make greater demands on infrastructure, not fewer. Training clusters will grow. Distributed inference will expand. Agentic AI will require more continuous, low-latency access to large memory pools and sustained system uptime — workloads that expose every weakness in infrastructure that wasn't designed for them.

Success in this environment requires more than faster accelerators. It requires systems that keep those accelerators connected, coordinated, and operational — at scale, under real-world conditions, over extended periods.

With UALoE, AMD Fabric Manager, AFOS, and vPods, AMD is advancing a scale-up networking architecture built for exactly that future: the performance AI demands, and the resilience production AI requires.

Cautionary Statement

This blog may contain forward-looking statements concerning Advanced Micro Devices, Inc. (AMD), which are made pursuant to the Safe Harbor provisions of the Private Securities Litigation Reform Act of 1995. Forward-looking statements are commonly identified by words such as "would," "may," "expects," "believes," "plans," "intends," "projects" and other terms with similar meaning. Investors are cautioned that any forward-looking statements in this blog are based on current beliefs, assumptions and expectations, speak only as of the date of this blog and involve risks and uncertainties that could cause actual results to differ materially from current expectations. Such statements are subject to certain known and unknown risks and uncertainties, many of which are difficult to predict and generally beyond AMD's control, that could cause actual results and other future events to differ materially from those expressed in, or implied or projected by, the forward-looking information and statements. Investors are urged to review in detail the risks and uncertainties in AMD’s Securities and Exchange Commission filings, including but not limited to AMD’s most recent reports on Forms 10-K and 10-Q.  

AMD does not assume, and hereby disclaims, any obligation to update forward-looking statements made in this blog, except as may be required by law.