AI Infrastructure, A System-Level Challenge

The growth of generative and agentic AI is increasing demand across businesses’ infrastructure; inference is now the number one AI workload, training continues to scale up, and agentic AI is driving incredible level of compute across inference, orchestration and data movement. These workloads depend on more than just accelerator performance, important though it may be.

CPUs must orchestrate workloads, networking must keep data moving, software must sustain utilization and the physical rack itself must support everything from power and cooling to availability and serviceability at scale.

As customers’ deployments grow, they are no longer evaluating individual GPUs in isolation; they’re evaluating the output and efficiency of the complete system. As customers increasingly focus on multi-agent workflows, this requirement only increases because interconnected models, applications, tools and data must all operate together seamlessly.

As your customers expand their AI infrastructure, fragmented component-by-component designs can create additional integration, networking and operational complexity. The AMD Helios™ rackscale platform helps address this by bringing compute, memory, networking, software, power, cooling, security and serviceability together into one co-designed rackscale platform.

Purpose-Built Platform for Hyperscale and Sovereign AI

The AMD Helios rackscale platform is fully integrated and purpose-built for high-volume inference, frontier-model training, fine-tuning, sovereign AI and agentic AI. Combining the power of AMD Instinct™ GPUs, AMD EPYC™ server CPUs, AMD Pensando™ NICs and DPUs, AMD ROCm™ software and an open rack architecture aligned with Meta Open Rack-wide standards, AMD Helios rackscale platform delivers an all-in-one solution for customers looking to kick-start or upgrade their AI infrastructure.

Far beyond a simple rack for GPU utilization, it’s a complete blueprint built for industry-leading compute,1 enabling customers to transform hyperscale designs into deployable production platforms.

Rather than having to configure the optimal solution for customers and their AI needs, partners can use the AMD Helios rackscale platform to combine the best technologies AMD offers to produce a cohesive system optimized for high-volume inference, frontier-model training, fine-tuning and agentic AI across hyperscale, AI cloud/NeoCloud, sovereign AI and large enterprise AI infrastructure deployments. 

The Building Blocks Behind the AMD Helios Rackscale Platform

The AMD Helios rackscale platform brings together several core AMD building blocks that address different parts of the server infrastructure challenge, from accelerated compute and workload orchestration to data movement, infrastructure offload and physical serviceability.

The value of the AMD Helios rackscale platform does not come from one component on its own. By utilizing each of these technologies and designing them to operate as one coordinated system – and supported by AMD ROCm software – customers gain easy access to incredible AI performance that can scale as their needs demand.

AMD Instinct™ MI455X GPUs

For customers building AI infrastructure at scale, the value of the AMD Helios rackscale  platform comes from what it can deliver as a complete, repeatable system: 72 AMD Instinct™ MI455X GPUs deliver up to 2.9 exaFLOPS FP4 or 1.4 exaFLOPS FP8 compute and backed by up to 31 TB of HBM4, with up to 1.7 PB/s of HBM bandwidth, up to 260 TB/s scale-up bandwidth and up to 43 TB/s scale-out bandwidth AMD Helios keeps inference, distributed training and large-scale fine-tuning workloads moving efficiently.

Learn more about AMD Instinct MI455X GPUs in our AMD Helios guide   

AMD EPYC™ 9006 Series Server CPUs

AMD EPYC™ 9006 Series server CPUs provide the processing foundation for the AMD Helios rackscale platform, supporting the orchestration, system services and control tasks needed to keep a large accelerator environment operating efficiently.

This is vital for agentic AI, where multiple models, applications, tools and data sources interact continuously.

Read more about AMD EPYC 9006 Series server CPUs 

AMD Pensando™ Vulcano 800 AI NICs

AMD Pensando™ Vulcano 800 AI NICs provide the standards-based Ethernet scale-out connectivity used to extend workloads across racks and larger AI clusters. Inside the rack, UALink over Ethernet connects the 72 GPUs as a coordinated scale-up accelerator domain, while Vulcano NICs carry that performance beyond the rack so customers can build larger, repeatable AI infrastructure without treating networking as a separate redesign.

As inference and training workloads become distributed, networking can become a significant bottleneck on effective system performance. AMD Pensando AI NICs help keep data moving efficiently across infrastructure, reducing the risk of communication becoming a limiting factor.

Read the blog about AMD Pensando Vulcano 800 AI NICs 

AMD Pensando™ Salina DPUs

AMD Pensando™ Salina DPUs offload select networking, storage and security services from the host CPUs, giving the CPUs more capacity for their primary orchestration role while providing a dedicated layer for connecting the AI infrastructure with front-end networks and services. This disaggregated approach can improve infrastructure efficiency by moving software-defined networking functions onto dedicated data processing hardware.

This separation of roles is particularly important for AI cloud and multi-tenant infrastructure where operators need to manage shared resources, infrastructure services and workload isolation without placing every task on the CPUs.

 Read the blog about AMD Pensando Salina DPUs 

An Architecture Built on Open Industry Standards

The final building block of the AMD Helios rackscale platform is the physical architecture surrounding the compute; it’s aligned with a range of open standards and built around repeatable four-GPU compute trays, ensuring a standards-friendly foundation for hyperscale deployment and serviceability for enterprise customers. These standards include:

Open rack-wide / OCP-aligned rack design

UALoE scale-up

UEC-aligned Ether scale-out

AMD ROCm software

Adhering to these standards helps turn dense AI compute into deployable production infrastructure; elements such as power, cooling, maintenance and component replacement are considered at the rack level, meaning customers can move beyond one-off system installations to a repeatable, efficient architecture design.

Read the blog on how the AMD Helios rackscale platform advances open, scalable AI infrastructure 

AMD Helios Solves the Challenges of AI at Scale

The value of the AMD Helios rackscale platform is best understood through the outcomes customers need from AI infrastructure at scale: higher rack-level AI factory output, open rack-to-cluster growth, productive infrastructure, serviceability, security and better economics as deployments move from individual systems into production platforms.

The challenge: GPUs in an AI rack can’t deliver their full value if they’re regularly waiting for instructions or other system elements to catch up. At scale, this level of slow communication between GPUs or other parts of the rack becomes a bottleneck that slows down the entire output of the cluster.

The solution: Each component in the AMD Helios rackscale platform is designed to reduce the risk of one part of the system holding back another, enabling customers to keep more of their accelerator capacity productive and gain more useful output from the infrastructure they’ve invested in.

The challenge: An AI environment that works on a small scale is likely to become increasingly complex as customers add more racks. Without a clear scaling plan in place, every expansion runs the risk of introducing new networking, integration and operational challenges.

The solution: The AMD Helios rackscale platform helps keep accelerator capacity productive by treating the rack as a coordinated system rather than a set of isolated components. Its two-layer network design uses UALoE scale-up connectivity inside the rack to keep the 72 GPUs working as one high-speed accelerator domain, while standards-based Ethernet scale-out carries that performance across racks, helping customers gain more useful AI factory output from the infrastructure they’ve invested in.

The Challenge: Dense AI infrastructure can’t simply be installed like conventional servers. The demands of power, cooling, networking, component replacement and physical serviceability all need to be considered from the very beginning. Designing each deployment from scratch adds valuable time investment, complexity and risk to the installation.

The solution: The AMD Helios rackscale platform is built around 18 repeatable four-GPU compute trays and is aligned with Meta Open Rack Wide standards. Elements like power, cooling and serviceability are considered as part of the rack architecture rather than being left to the customer to resolve after selecting components.

This means customers gain a standardized foundation for their deployment and expansion plans, reducing the amount of custom system design required and making infrastructure easier to plan, deploy and expand over time.

The challenge: AI infrastructure is expensive, but not every model, workload or customer need requires access to an entire GPU or rack at all hours of the day. Without flexible allocation, valuable system capacity can sit unutilized or be assigned inefficiently.

The solution: Hardware-based GPU partitioning lets operators divide available resources into smaller compute segments that can be matched to different workloads and users, offering hyperscalers and AI cloud providers the ability to flexibly allocate and isolate GPU resources across inference, training and fine-tuning workloads, thereby improving utilization, supporting multi-tenant environments and ensuring their AI infrastructure can adapt efficiently as workloads scale from single racks up to large distributed clusters.

The AMD Helios rack scale platform also brings hardware-rooted security across the rack, including device-level identity and attestation, runtime attestation, memory encryption, universal link encryption, secure multi-GPU scaling and partitioning for multi-tenant configurations. Each of these capabilities is designed to help customers protect their AI data and model intellectual property while supporting shared and multi-tenant infrastructure.

The Foundation of AMD ROCm™ Software

Hardware sets the potential performance of a customer’s AI infrastructure. Software is what determines how easily and effectively customers can put that performance to work.

AMD ROCm software provides the open software foundation for the AMD Helios rackscale platform, giving customers the tools they need to deploy, manage and optimize AI inference, training and fine-tuning across rackscale and clustered environments.

AMD ROCm software works with widely used AI models, frameworks and runtimes, including PyTorch, TensorFlow, JAX, ONNX Runtime, vLLM, SGLang and Triton. Recent ecosystem progress includes access to more than 2 million models and expanded support across open-source AI frameworks and inference engines.

Because AMD ROCm is built around open standards, customers can retain greater flexibility over the models, tools and environments they use, while its software optimizations help the hardware running AI workloads remain productive as workloads grow, supporting efficient performance at scale.

For partners, this means the customer conversation should extend beyond the hardware itself. Partners can help customers evaluate whether their preferred models and frameworks are supported, what work may be required to move existing applications, whether teams have the necessary skills, and how the platform will connect with their existing orchestration, monitoring and management tools.

Who the AMD Helios Rackscale Platform Is Built For

The AMD Helios rackscale platform is designed for organizations operating AI infrastructure at scale. While their operations differ, customers seeking rackscale AI all share the need for high-density compute, scalable networking and dependable operations and control over their infrastructure.

Large-Scale AI Operators: Hyperscalers, AI Cloud/NeoCloud Providers, Frontier-Model Developers

This audience is building AI factories to support global-scale services and rapidly growing demand in the AI space. Their workloads include fleet-scale inference, distributed training and large-scale fine-tuning, often operating across extensive clusters rather than isolated racks.

For such customers, their primary concerns focus on output density, utilization, bandwidth and the economics of scaling from single racks to larger clusters; they value repeatability and serviceability across massive fleets.

The AMD Helios rackscale platform combines dense rack-level compute and memory capacity with open scale-out networking and repeatable architecture that’s designed specifically for these types of deployments.

Sovereign and Enterprise AI Deployments

Sovereign AI and data center platform teams provide governments, public institutions and vital industries with control over their AI infrastructure. This type of deployment requires operations to be on-premises or within national borders while meeting requirements centered around security, governance and long-term capacity.

The AMD Helios rackscale platform combines compute, memory, open networking, security, ROCm software and serviceable rack design to help this customer type preserve control, flexibility and long-term infrastructure choice.

OEM/ODM and Ecosystem Builders

These customers play a different role in the AI infrastructure market: they need to turn advanced rackscale designs into systems that can be built, validated, delivered, serviced and supported repeatedly for customers.

For this audience, the value of the AMD Helios rackscale platform is not simply the performance of the rack itself, but the fact that AMD has combined compute, networking, software, security, power, cooling and physical serviceability together into a complete AI factory blueprint.

That gives OEM/ODM partners and ecosystem builders a clearer foundation for developing differentiated systems around AMD technology. AMD Helios helps reduce the amount of custom integration required while still giving partners room to innovate around system design, validation, deployment and customer-specific implementation.

The Open AI Factory Platform Partners Can Deliver

AI infrastructure has moved beyond component-by-component purchasing conversations. Customers are now looking for solutions that span the spectrum of compute, memory, networking, software, security and physical rack architecture – and how they all operate as one coordinated system – to help them move beyond AI experimentation to production at scale.

The AMD Helios rackscale platform gives partners a complete solution to introduce to customers; by combining each of these powerful technologies with open rack standards and AMD ROCm software, it represents an incredible solution for any customer focused on AI output, scalability and operational confidence.

To learn more about the AMD Helios rackscale platform and how you can support your customers either entering or advancing in the AI space, speak to your AMD representative or visit amd.com to learn more.

Visit the AMD Helios rackscale platform landing page 

AMD Arena

Build expertise with training on AMD AI solutions, AMD EPYC™ server CPUs, AMD Instinct™ GPUs and more across the AMD portfolio.

Get Training

Related Articles

Explore More Articles

Related Training Courses

Explore More Courses

Related Webinars

Explore More Webinars
Footnotes
  1. Based on calculations by AMD Performance Labs in June 2026 to determine the peak theoretical precision performance of the AMD Helios Rackscale Solution using peak matrix FP16, BF16, INT8, Open Compute Project MXFP6, MXFP8, FP8 and MXFP4 data types, compared with the NVIDIA Vera Rubin NVL72 rack using dense NVFP4 and FP8/FP6 data types. System manufacturers may vary configurations, yielding different results. MI400-005.