Build a Private Cloud with AMD Ryzen AI Max

Key Takeaways

• A hypervisor with GPU passthrough turns AMD hardware into a private GPU cloud -- on-premises, API-managed, and suitable for air-gapped environments.
• AMD Ryzen™ AI Max processors combine large unified-memory configurations with integrated Radeon™ graphics, enabling local execution of models that traditionally require workstation-class GPU memory. The same passthrough technique applies to discrete Radeon™ AI PRO Graphics.
• The hypervisor layer is what makes GPU hardware operationally useful at scale. VM templates, snapshots, cloning, and REST APIs enable ephemeral CI runners, isolated development environments, multi-tenant compute, and self-service AI labs -- without giving up control of your data or infrastructure.

Introduction

If you have AMD GPU hardware and want to run it as managed infrastructure rather than a collection of standalone workstations, this guide is the starting point.

AMD Ryzen™ AI Max+ 395 (Strix Halo) processors combine Zen 5 CPU cores, a large RDNA 3.5 Radeon GPU, and a high-capacity unified memory architecture in a compact package. Strix Halo systems support up to 128 GB of unified LPDDR5X memory, enabling models up to 128 billion parameters at 4-bit quantization. The newer AMD Ryzen™ AI Max+ PRO 495 (Gorgon Halo) platform raises the memory ceiling to 192 GB and can support models exceeding 300 billion parameters at 4-bit quantization. The same GPU passthrough technique covered in this guide also applies to discrete Radeon AI PRO Graphics, which provide dedicated VRAM and PCIe bandwidth for workloads that exceed what a single APU can address.

Hardware solves the compute problem; it does not solve the management problem. On bare metal, each system is essentially a standalone workstation: workloads share the same operating-system environment, CI jobs can inherit state from previous runs, and managing a fleet requires a separate provisioning and management layer.

Virtualization changes that model. A hypervisor can isolate workloads in virtual machines while VFIO passes a physical GPU directly to the VM. The guest (VM) accesses the physical device directly and runs the AMD ROCm™ software stack against real AMD hardware. The hypervisor provides the infrastructure layer around that GPU:

• clone a VM from a known-good template
• snapshot it before a driver update
• provision an isolated environment for a CI job
• destroy the VM when the job completes
• manage the lifecycle through an API

This guide demonstrates that architecture using Proxmox VE, a KVM/QEMU-based virtualization platform. It walks through the complete path from host configuration to a working VM running ROCm and PyTorch on a passed-through AMD GPU. Instructions cover both integrated Radeon graphics on Ryzen AI Max processors and discrete Radeon GPUs, with callouts where the two paths diverge.

Tested Configurations

This procedure was validated end-to-end on both an integrated and discrete GPU configuration:

PCI device IDs and bus addresses will vary by platform and SKU.

When Virtualization Makes Sense (and When It Doesn't)

Use a hypervisor when:

• CI/CD needs clean state per job. Clone a golden VM template, run the job, destroy the clone. No accumulated state, no flaky builds from leftover artifacts. This is the architecture behind ephemeral CI systems that provision GPU runners on demand.
• Multiple teams share hardware. VMs provide isolation, resource limits, and access control without giving every user root on the host.
• You need rollback. Testing a ROCm update, a kernel upgrade, or a new driver? Snapshot the VM first. If the update breaks the GPU stack, roll back in seconds instead of spending hours reimaging.
• You manage a fleet. Proxmox provides a web UI, REST API, and CLI for VM lifecycle operations across many hosts. At scale, this is significantly easier than bare-metal provisioning.

Use bare metal when:

• You are a single user on a single machine running interactive workloads
• Maximum performance matters more than reproducibility or isolation
• The system has only one GPU and you need host-side GPU access

Note that VFIO passthrough is exclusive: when a GPU is assigned to a VM, the Proxmox host cannot simultaneously use that GPU for its own workloads. The host becomes a control-plane machine and the VM owns the accelerator.

What about containers?

Docker and Podman can access AMD GPUs via --device=/dev/kfd --device=/dev/dri, and for many workloads that is the simpler path. But containers share the host's kernel and GPU driver, and for CI and multi-tenant use cases, that creates real problems:

• No GPU fault isolation. If a workload hangs the GPU or triggers a driver reset inside a container, it takes down every other container using that GPU -- and potentially the host. With a VM, a GPU hang is contained to the guest. Kill the VM, VFIO releases the device, and the host is unaffected.
• No GPU memory isolation. AMD GPUs have no hardware memory partitioning. All containers compete for the same VRAM pool. One container allocating aggressively can OOM another container's workload with no warning.
• Driver state persists between runs. The amdgpu kernel module accumulates state -- firmware contexts, page tables, command queues. A "clean" container still inherits whatever the driver has accumulated from previous workloads. For CI, this means test results can depend on what ran before. With VFIO, the guest loads its own driver from scratch every boot.
• You cannot vary the kernel or driver. Every container uses the host's amdgpu module and ROCm version. If you need to test across ROCm versions, or validate a kernel upgrade, containers cannot help.
• Linux only. Containers are a Linux kernel construct. If you need GPU workloads on Windows or WSL2 -- DirectML inference, Windows driver validation, ONNX Runtime testing -- containers are not an option.

The two approaches are not mutually exclusive. A common pattern is to run containers inside a VFIO-passthrough VM: the VM provides hardware isolation and clean device state, while containers inside it handle application packaging.

What You Will Need

Hardware

Software

Installing Proxmox VE: If Proxmox is not already installed: download the ISO from proxmox.com/en/downloads, write it to a USB drive using a tool like Rufus (Windows) or dd (Linux/macOS), and boot from it. During installation, you will be asked to choose a filesystem (ext4 or ZFS) and a boot method (GRUB or systemd-boot). For most single-disk setups, ext4 with GRUB is the simplest choice. ZFS offers built-in snapshots and data integrity but uses more memory. Select UEFI boot if your system supports it. After installation, access the Proxmox web interface at https://<host-ip>:8006 and log in as root with the password you set during install.

About VFIO Passthrough

VFIO (Virtual Function I/O) binds a PCIe device exclusively to a VM. The guest OS loads native drivers and accesses the hardware directly -- the hypervisor is not in the GPU command/data path during compute operations. Modern AMD GPUs expose at least two PCIe functions: a display/VGA controller and an HDMI/DP audio controller. For the Radeon configurations covered in this guide, pass the GPU's display function and associated audio function together.

A Desktop ISO provides a GUI for initial validation. Server ISO works if you prefer headless guests.

Step 8 -- Create the VM

In the Proxmox web UI (https://<host-ip>:8006), click Create VM with these settings:

Size CPU cores, memory, and disk according to your workload and host capacity. Do not start the VM yet.

Step 9 -- Attach the GPU

1. Select the VM, navigate to Hardware
2. Click Add, then PCI Device, then Raw Device
3. Select the GPU's display function (the one bound to vfio-pci)
4. All Functions: checked (pulls the paired audio function automatically)
5. PCI-Express: checked
6. Primary GPU: leave unchecked during installation (keeps noVNC console usable)
7. Confirm hostpci0 with pcie=1 appears in the hardware list

If the device list is empty or shows the GPU as "in use" by amdgpu, revisit Steps 5-6. Both functions must show vfio-pci as the active driver.

Step 10 -- Install the Guest OS

1. Start the VM and open the Console tab
2. Install Ubuntu using Erase disk and install (this applies to the VM's virtual disk only)
3. After installation, reboot when prompted

Post-install validation inside the VM:

Fleet Management

Across a rack of Strix Halo systems or a mixed fleet of APU and discrete GPU hosts, Proxmox provides centralized management through its cluster features. Each host contributes its GPU(s) to the pool. An orchestrator routes jobs to available hardware based on GPU type, label, or capacity.

For a practical example: the DevLab Dispatch system runs ephemeral CI for open-source projects on AMD hardware using exactly this architecture -- Proxmox VMs with passed-through GPUs, provisioned per job, destroyed after completion.

Troubleshooting

Quick Reference