为大规模 AI 构建网络
Jul 23, 2026
网络正成为决定 AI 性能与经济效益的关键因素
AI 的瓶颈正在转移。尽管算力仍然至关重要,但网络正日益成为决定系统性能的关键因素。如今,数据在 GPU、机架和集群之间传输的速度与效率,会影响模型的训练速度、服务用户的效率,以及交付 AI 的总体成本。随着训练、分布式推理和智能体 AI 扩展至数以万计的同步 GPU,GPU 之间的网络流量对系统性能的重要性与日俱增。这些工作负载相互叠加,每一种都会进一步提高对底层网络的要求。
要满足这些需求,需要采用一种将网络视为三层整体系统的架构:为 AI 基础设施提供数据的前端网络、将多个 GPU 连接为单一计算资源的纵向扩展互连,以及将 AI 扩展至多个机架和数据中心的横向扩展网络。在 Advancing AI 2026 大会上,AMD 推出了覆盖这三个层面的全新网络功能,并将其集成到 AMD Helios™ 中,在统一的可编程软件平台下构成一个机架级系统。其中包括用于前端智能体 AI 加速的 AMD Pensando™ Salina DPU、用于构建开放且高韧性纵向扩展网络的 UALoE,以及面向横向扩展和跨地域扩展的 AMD Pensando™ Vulcano 800 AI NIC。
要跟上这些不断演进的工作负载,仅增加带宽远远不够。随着行业从 400G 迈向 800G,并进一步向 1.6T 演进,网络带宽大约每两年便翻一番,但仅凭速度仍不足以应对挑战。AI 工厂必须同时支持训练、分布式推理和智能体工作负载,而三者对网络的要求截然不同:训练需要数千个 GPU 之间保持紧密同步;推理需要在需求波动时维持稳定的低延迟;智能体 AI 则需要持续访问内存,并同时协调多项服务。如何在同一个数据中心内兼顾这三类需求,正是超大规模云服务提供商当前面临的基础设施挑战。业界的应对方式是进行架构革新:将智能融入网络本身,通过可编程 AI NIC 和 DPU,把感知与决策能力分布到 AI 流量的源端。
前端:释放算力,为 AI 工厂提供数据
前端网络将用户和数据连接到 AI 基础设施,负责数据摄取、身份验证、存储访问和流量安全。通常,这些功能均由主机 CPU 运行,占用原本可用于推理和逻辑推理的核心;随着集群规模扩大,相关开销也会持续增加。
随着 AI 向智能体工作负载演进,更长的上下文窗口、更高的并发度和持续的内存访问需求,使 DPU 的职责远远超出了分担网络负载。如今,DPU 已成为智能体系统访问数据、协调任务并在负载下维持性能的核心,而可编程性则使其从固定功能芯片升级为平台。基础设施服务可通过软件进行更新和差异化,从而在每次部署和每一代产品中持续积累创新。AMD Pensando™ Salina DPU 可提供网络流量安全保护、加速软件定义网络和存储访问,并通过由 DPU 管理的 NVMe 扩展 GPU 显存容量,使 KV 缓存能够支持百万 Token 上下文,同时每台服务器最多可释放 多个 CPU 核心用于 AI 工作负载。
纵向扩展:将多个 GPU 统一为一个系统
超大模型所需的内存和算力超出任何单个 GPU 的能力,因此必须分布到多个 GPU 上运行。为了实现高效运行,这些 GPU 不能像独立计算机之间相互交换消息那样工作,而必须作为一个统一的系统协同运行。张量并行等技术要求 GPU 之间持续交换中间结果,因此对带宽、延迟和一致性提出了严格的要求。互连性能一旦下降,整个 GPU 群组都会受限于其中最薄弱的环节,此时即使增加更多 GPU,也无法继续带来性能提升。
在 AMD Helios 中,UALoE 采用单跳拓扑,将全部 72 个 AMD Instinct GPU 连接为一个统一的计算资源,在单个机架内提供高达 260 TB/s 的聚合带宽,并可访问 31 TB 的 HBM4 内存,同时无需使用专有的互联技术。
AMD 纵向扩展解决方案的核心是高韧性设计,这是纵向扩展投入生产环境的必要条件。每个 GPU 都通过分布在不同交换机托盘中的 18 个独立 UALoE 站点进行连接。因此,当某条路径受到硬件事件影响时,系统可自动维持统一的 GPU 域,而无需终止工作负载。AMD Fabric Manager 与 AMD Fabric OS 共同构成运营层,负责资源预置、工作负载放置、遥测、故障恢复,以及交换层的实时可视性,将纵向扩展网络转变为可管理、可观测的平台。vPods 在此基础上进一步扩展功能,支持客户将该域划分为软件定义的 GPU 环境,使并发工作负载能够在同一机架上彼此隔离地共存,同时不牺牲性能。
横向扩展:将 AI 分布到多个数据中心
单个机架通常不足以承载完整规模的 AI,因此,训练前沿模型以及为数百万用户提供服务,需要将工作负载分布到多个机架和数据中心。在这种规模下,梯度传输不能发生停滞,延迟必须在需求波动时保持在预期之内,而可靠性、可用性和可维护性 (RAS) 与带宽本身同样重要。网络必须能够从拥塞、丢包和路径中断中快速恢复,避免单一事件导致整个作业停滞。
AMD Pensando™ Vulcano 800 AI NIC 专为下一代 AI 基础设施而设计,可为 Helios 提供可编程的高性能横向扩展网络,使每个 GPU 的横向扩展带宽最高可达 2.4 Tbps。额外的带宽有助于加速集合通信,并减少训练瓶颈。下一代 AMD AI NIC™ 还可将 AI 作业完成时间进一步缩短并降低交换成本。AMD Pensando™ Pollara 400 AI NIC 则将这些能力扩展到现有 AI 集群,为已经实现规模化部署的环境带来基于开放以太网的 AI 网络。
开放且可编程的网络,为持续演进而生
AMD 网络技术基于第三代可编程 P4 引擎构建,可通过软件支持新的传输协议和优化,无需进行成本高昂的硬件更换,即可推动基础设施持续演进。最终形成的可编程以太网架构能够与 AI 同步发展,而不会成为制约因素。
专有互连技术可能加深对单一供应商生态系统的依赖,而开放标准旨在提供更多选择,并加速全行业创新。AMD 以开放以太网标准为基础,并依托超以太网联盟 (UEC)、超高速加速器链路 (UAL) 和 ESUN 等生态系统,打造贯穿前端、纵向扩展和横向扩展的一体化系统,为 AI 基础设施的未来演进提供支撑。
Related Blogs
附注
- PEN-017: Testing conducted by AMD Performance Labs as of 15th April 2025 on the AMD Pensando Salina DPU, on a test system comprising of 2x Dual socket Xeon Dell power edge XE9680 function; Cisco 64x400G Switch; IXIA 2x400G tester as a traffic generator from Keysight; 2xAMD Pensando™ Salina DPU; 5th gen Xeon 8568 - 48 core CPU with PCIe Gen-5; Operating System Version : Ubuntu® 22.04.5 LTS; Kernel Version: 5.15.0-139-generic; BIOS version: 1.3.6 [
Mitigation: Off (default)
System profile setting: Performance (default)
SMT: enabled (default)]
AMD Pensando performs an average of 117 MPPS (millions packets per second) in AMD testing while Nvidia Bluefield-3 published performance is 80 MPPS. https://hc33.hotchips.org/assets/program/conference/day1/HC2021.NVIDIA.IdanBurstein.v08.norecording.pdf Slide 6 for ~1.45x the performance with AMD Pensando Salina DPU.
Results may vary based on factors including but not limited to system configuration and software settings.
- 22 CPU Core Savings with Accelerated Connections running on AMD DPU at Microsoft. Core Savings for Customer SLB Appliances running on Sirius Appliance at Microsoft as part of Accelerated Connections Service vs Running SLB on Standard Server
- Based on AMD Engineering silicon modeling and AMD synthetic benchmark simulation, using the MOE_4p5T hi_sparsity_ea9 benchmark test to project time- to- solution speed up in days using the FP8 training datatype on a simulated LLM system modeled with 8000 AMD Instinct MI455X GPUs and (3) versus (2) AMD Pensando Vulcano NICs.
Results reflect analysis of pure training compute (decode layers only) Configuration evaluated with a global batch size of 4096 and a sequence length of 4K, using SwiGLU activation. Results exclude evaluation and checkpointing overhead. FlashAttention v3 with matmul–softmax overlap is assumed. AllReduce and All2All communication costs are fully accounted for and not hidden via tiled compute–communication overlap. Gradient synchronization and FSDP weight prefetch are evaluated across varying levels of overlap to assess scale-out sensitivity. Results assume ideal and not fully optimized real-world behavior and may vary when actual product(s) are released in market.
- PEN-022: AMD comparison and pricing as of May 18, 2026, for network fabric costs to support 32,000 GPUs. Comparison of a Vulcano-based NIC (VULCANO-CUSTOM-2.4T) deployed as part of a Helios rackscale system with a network based on 1.6T Tomahawk 6 switching with 200G SerDes versus using a competitor 800G NIC with 800G Tomahawk 6 switching with 100G SerDes. Both fabrics were fat-tree topologies built on Tomahawk 5 800G switching platforms, with NIC costs considered comparable. The Vulcano-based design is estimated to deliver up to 33% savings in network switching costs by enabling a more cost-effective architecture with fewer switching platforms, more bandwidth per port on the network, and reduced transceiver cables/optics.
TH6-100G Serdes Fat-Tree (Competition):
Switching (TH6C BCM78914 - 128x800G):
• Leaf Units 1,000
• Spine Units 500
• Total Switches 1,500
• TH6 100G Unit Price $79,587
• Total Switching Cost $60M
Cables/Optics:
• NIC Transceivers (800G-DR8) 64,000 @ $500
• Leaf/Spine Transceivers (800G-DR8) 192,000 @ $500
• MPO Cables 256,000 @ $89
• Total Optics/Cables $64M
Total Fabric Cost TH6-100G (Switches + Cables/Optics): $124M
TH6-200G Serdes Fat-Tree (Vulcano-Custom2.4T / AMD Solution):
Switching (TH6P BCM78910 - 64x1.6T):
• Leaf Units 1,000
• Spine Units 500
• Total Switches 1,500
• TH6 200G Unit Price $66,336
• Total Switching Cost $50M
Cables/Optics:
NIC Transceivers (1.6T-DR8) 32,000 @ $900 (50% fewer vs. competition)
Leaf/Spine Transceivers (1.6T-DR8) 64,000 @ $900
MPO Cables 128,000 @ $89 (50% fewer vs. competition)
Total Optics/Cables $43M
Total Fabric Cost TH6-200G (Switches + Cables/Optics): $93M
Capex Savings (Fabric only):
Savings $: $30.7M
Savings %: 33.1%
Pricing sources: SemiAnalysis Hyperscaler Networking Model data used with permission; full analysis available via SemiAnalysis subscription. Edgecore switch pricing as of May 17, 2026. Results may vary based on system configuration.
- PEN-017: Testing conducted by AMD Performance Labs as of 15th April 2025 on the AMD Pensando Salina DPU, on a test system comprising of 2x Dual socket Xeon Dell power edge XE9680 function; Cisco 64x400G Switch; IXIA 2x400G tester as a traffic generator from Keysight; 2xAMD Pensando™ Salina DPU; 5th gen Xeon 8568 - 48 core CPU with PCIe Gen-5; Operating System Version : Ubuntu® 22.04.5 LTS; Kernel Version: 5.15.0-139-generic; BIOS version: 1.3.6 [
Mitigation: Off (default)
System profile setting: Performance (default)
SMT: enabled (default)]
AMD Pensando performs an average of 117 MPPS (millions packets per second) in AMD testing while Nvidia Bluefield-3 published performance is 80 MPPS. https://hc33.hotchips.org/assets/program/conference/day1/HC2021.NVIDIA.IdanBurstein.v08.norecording.pdf Slide 6 for ~1.45x the performance with AMD Pensando Salina DPU.
Results may vary based on factors including but not limited to system configuration and software settings. - 22 CPU Core Savings with Accelerated Connections running on AMD DPU at Microsoft. Core Savings for Customer SLB Appliances running on Sirius Appliance at Microsoft as part of Accelerated Connections Service vs Running SLB on Standard Server
- Based on AMD Engineering silicon modeling and AMD synthetic benchmark simulation, using the MOE_4p5T hi_sparsity_ea9 benchmark test to project time- to- solution speed up in days using the FP8 training datatype on a simulated LLM system modeled with 8000 AMD Instinct MI455X GPUs and (3) versus (2) AMD Pensando Vulcano NICs.
Results reflect analysis of pure training compute (decode layers only) Configuration evaluated with a global batch size of 4096 and a sequence length of 4K, using SwiGLU activation. Results exclude evaluation and checkpointing overhead. FlashAttention v3 with matmul–softmax overlap is assumed. AllReduce and All2All communication costs are fully accounted for and not hidden via tiled compute–communication overlap. Gradient synchronization and FSDP weight prefetch are evaluated across varying levels of overlap to assess scale-out sensitivity. Results assume ideal and not fully optimized real-world behavior and may vary when actual product(s) are released in market. - PEN-022: AMD comparison and pricing as of May 18, 2026, for network fabric costs to support 32,000 GPUs. Comparison of a Vulcano-based NIC (VULCANO-CUSTOM-2.4T) deployed as part of a Helios rackscale system with a network based on 1.6T Tomahawk 6 switching with 200G SerDes versus using a competitor 800G NIC with 800G Tomahawk 6 switching with 100G SerDes. Both fabrics were fat-tree topologies built on Tomahawk 5 800G switching platforms, with NIC costs considered comparable. The Vulcano-based design is estimated to deliver up to 33% savings in network switching costs by enabling a more cost-effective architecture with fewer switching platforms, more bandwidth per port on the network, and reduced transceiver cables/optics.
TH6-100G Serdes Fat-Tree (Competition):
Switching (TH6C BCM78914 - 128x800G):
• Leaf Units 1,000
• Spine Units 500
• Total Switches 1,500
• TH6 100G Unit Price $79,587
• Total Switching Cost $60M
Cables/Optics:
• NIC Transceivers (800G-DR8) 64,000 @ $500
• Leaf/Spine Transceivers (800G-DR8) 192,000 @ $500
• MPO Cables 256,000 @ $89
• Total Optics/Cables $64M
Total Fabric Cost TH6-100G (Switches + Cables/Optics): $124M
TH6-200G Serdes Fat-Tree (Vulcano-Custom2.4T / AMD Solution):
Switching (TH6P BCM78910 - 64x1.6T):
• Leaf Units 1,000
• Spine Units 500
• Total Switches 1,500
• TH6 200G Unit Price $66,336
• Total Switching Cost $50M
Cables/Optics:
NIC Transceivers (1.6T-DR8) 32,000 @ $900 (50% fewer vs. competition)
Leaf/Spine Transceivers (1.6T-DR8) 64,000 @ $900
MPO Cables 128,000 @ $89 (50% fewer vs. competition)
Total Optics/Cables $43M
Total Fabric Cost TH6-200G (Switches + Cables/Optics): $93M
Capex Savings (Fabric only):
Savings $: $30.7M
Savings %: 33.1%
Pricing sources: SemiAnalysis Hyperscaler Networking Model data used with permission; full analysis available via SemiAnalysis subscription. Edgecore switch pricing as of May 17, 2026. Results may vary based on system configuration.