Match AMD Hardware to Your AI Needs
Get an overview of the AMD portfolio for AI, starting with AMD EPYC server CPUs, and explore how to use different types of hardware throughout your AI workflow.
AI Technologies Bring Broad Industry Impact
No matter the size or scale of your AI deployments, AMD EPYC server CPUs give you a high-performance, energy-efficient foundation for enterprise AI and general-purpose workloads.
See Why AMD EPYC Server CPUs Are the Best CPUs for Enterprise AI9
5th Gen AMD EPYC server CPUs stand up to the demands of AI with options that include high core counts or high frequency, plenty of memory and I/O bandwidth, and support for AVX-512 instructions. Built-in security technologies from AMD Infinity Guard help you keep data protected all the way to the silicon layer.7
How to Build Data Centers for AI
To build an AI ready data center, you’ll need a foundation of general-purpose compute designed for security, augmented with GPUs as needed to fit your performance and workload requirements. Here’s how to optimize your next data center as an AI-capable multi-tasking powerhouse.
Step 1
Consolidate Existing Servers
There's a limit to the space and power in your data center. By replacing old servers with new, high-density CPUs, you can consolidate to fewer servers, reduce related energy consumption, and free up space for AI.
Consolidate 8 to 1
Move from 2020-era Intel® “Cascade Lake” servers to 5th Gen AMD EPYC CPU-powered servers.
Up To
86%
Fewer Servers¹
Up To
69%
Less Power¹
Up To
41%
Lower 3-Year TCO¹
Fourteen AMD EPYC 9965 CPU-based servers can deliver the same integer performance as 100 old servers running Intel Xeon Platinum 8280 CPUs.
Get Better Performance Than the Latest Intel® Xeon® 6 CPUs
Up To
29%
More Integer Performance²
Up To
66%
Better Power Efficiency³
5th Generation AMD EPYC 9965 CPUs outperform the latest Intel Xeon 6 6980P CPUs with “performance cores.”
Step 2
Use CPUs to Support New AI Workloads
Many inference workloads run on CPUs and don’t need special accelerator hardware. If you plan to run small or medium models or have occasional AI tasks, high-core count 5th Gen EPYC Server CPUs may meet your performance requirements.
Up To
70%
Better End-to-End AI Performance Than Intel Xeon 6⁴
AMD EPYC 9965 CPUs outperform Intel Xeon 6 6980P CPUs on TPCxAI.
Step 3
Augment with GPUs as Needed
You may need dedicated AI acceleration for training, inference on large models, large-scale deployments, or low latency use cases. Start with high frequency AMD EPYC 9005 server CPUs as a host CPU to take advantage of high core frequency and large memory capacity. Add GPUs like AMD Instinct™ accelerators, available in a PCIe form factor.
Up To
~10x
Better Latency-Constrained Inference on NVIDIA H100 with AMD EPYC 9575F vs. Intel Xeon 8592+⁵
Up To
11%
Better Inference on AMD Instinct MI300X with AMD EPYC 9575F vs. Intel Xeon Platinum 8460Y+⁶
Step 4
Take Advantage of Built-in Security Features
Data protection must be a consideration in every AI deployment. AMD EPYC server CPUs are designed with security in mind to be resistant to many sophisticated attacks. Built-in at the silicon level, AMD Infinity Guard7 helps defend against internal and external threats to keep your data safe.
Step 5
Create the Right Mix of On-Prem and Cloud
Make sure you can quickly scale with a flexible AI infrastructure that has the right combination of on-premises and cloud resources. You can find AMD EPYC server CPUs across hundreds of hardware options and more than a thousand public cloud instances.
Run On Premises
350+
Hardware Platforms
Scale in the Cloud
1,000+
Public Cloud Instances
Start Retooling for AI with AMD EPYC Server CPUs
Frequently Asked Questions
Before investing in AI hardware, data center architects should assess their AI workloads and performance requirements. In some cases, general-purpose AMD EPYC server CPUs may provide enough performance for inference, avoiding the need to purchase GPUs.
In general, AMD EPYC server CPUs deliver enough performance for models up to 20 billion parameters. This includes many popular large language models (LLMs) and other generative AI applications.
AMD EPYC server CPUs are a great fit for many inference use cases. These include classic machine learning, computer vision, memory-intensive graph analysis, recommendation systems, natural language processing, and small to medium generative AI models, like LLMs. They’re also ideal for expertly tuned AI agents and collaborative prompt-based pre-processing, which are popular in retrieval of augmented generation (RAG) models.
5th Gen AMD EPYC server CPUs deliver 70% better end-to-end AI performance than Intel Xeon 6.4 They also offer up to 89% better chatbot performance on DeepSeek with AMD EPYC 9965 vs. Intel Xeon 6980P8 and impressive performance for LLMs.
If you need to comply with data locality or privacy requirements, or have strict requirements for low latency, consider running AI on premises. If you need the flexibility to scale up or down quickly, the cloud is a great choice for on-demand resources.
Choosing the Right CPU for AI
With AMD EPYC server CPUs, you can choose from a range of core, frequency, memory, and power options. You’ll get the best results by matching the CPU to the AI workloads you expect to run the most.
Get Superior Performance for End-to-End AI Workflows
For real-world AI and machine learning applications, AMD EPYC 9965 outperforms the Intel Xeon 6980P.
Up To
70%
Better End-to-End AI Performance⁴
Up To
60%
Better Performance on Facebook AI Similarity Search¹⁰
Up To
93%
Better Performance for Machine Learning¹¹
Impressive CPU Performance for Large Language Models (LLMs)
Confidently deploy chatbots, intelligent search agents, and other generative AI applications with performance for LLMs up to multi-billion parameters. The AMD EPYC 9965 outperforms the Intel Xeon 6980P.
Up To
89%
Better Chatbot Performance on DeepSeek⁴
Up To
33%
Better Performance for Medium Language Models on Llama 3.1 88¹²
Up To
28%
Better Performance for Summary Use CCase Throughput on GPT-J 6B¹³
Up To
36%
Better Performance for Translate Use Case on Llama 3.2 1B¹⁴
AMD Powers the Full Spectrum of AI
Match your infrastructure needs to your AI ambitions. AMD offers the broadest AI portfolio, open standards-based platforms, and a powerful ecosystem—all backed by performance leadership.
Open Software for Flexible AI Development
With AMD ZenDNN and AMD ROCm™ software, developers can optimize their application performance while using their choice of frameworks.
AMD EPYC Deployment Options
Broad Ecosystem for AI On-Premises
Find enterprise AI hardware from our OEM partners, including servers with high core count and high frequency CPUs, a premier line of GPUs, and interoperable networking solutions.
Scale AI in the Cloud
Get the most from your cloud by choosing AMD technology-based virtual machines (VMs) for AI workloads.
Resources
Subscribe to Data Center Insights from AMD
Request Contact from an AMD EPYC Sales Expert
Footnotes
- 9xxTCO-019[DM1] [MK2] : This scenario contains many assumptions and estimates and, while based on AMD internal research and best approximations, should be considered an example for information purposes only, and not used as a basis for decision making over actual testing. The AMD Server & Greenhouse Gas Emissions TCO (total cost of ownership) Estimator Tool - version 1.53, compares the selected AMD EPYC™ and Intel® Xeon® CPU based server solutions required to deliver a TOTAL_PERFORMANCE of 391,000 units of SPECrate2017_int_base performance as of September 30, 2025. This analysis compares a 2P AMD 192 core EPYC_9965 powered server with a SPECrate2017_int_base score of 3230, https://spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47086.pdf; compared to a 2P Intel Xeon 128 core Xeon_6980P based server with a SPECrate2017_int_base score of 2510, https://spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47099.pdf; versus legacy 2P Intel Xeon 28 core Platinum_8280 based server with a SPECrate2017_int_base score of 391, https://spec.org/cpu2017/results/res2020q3/cpu2017-20200915-23984.pdf Environmental impact estimates made leveraging data from the 2025 International Country Specific Electricity Factors and can be found at https://www.carbondi.com/#electricity-factors/ and the US EPA Greenhouse Gas Equivalencies Calculator used in this analysis was sourced on 09/04/2024 and can be found at https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator. For additional details, see https://www.amd.com/claims/9xx5TCO-019.
- 9xx5-128A: SPECrate®2017_int_base comparison based on published scores from www.spec.org as of 5/9/2025. 2P AMD EPYC 9965 (3230 SPECrate®2017_int_base, 384 Total Cores, 500W TDP, $14,813 CPU $), 6.460 SPECrate®2017_int_base/CPU W, 0.218 SPECrate®2017_int_base/CPU $, https://www.spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47086.html ) 2P AMD EPYC 9755 (2840 SPECrate®2017_int_base, 256 Total Cores, 500W TDP, $12,984 CPU $), 5.680 SPECrate®2017_int_base/CPU W, 0.219 SPECrate®2017_int_base/CPU $, https://www.spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47223.html ) 2P Intel Xeon 6980P (2510 SPECrate®2017_int_base, 256 Total Cores, 500W TDP, $12,460 CPU $) 5.020 SPECrate®2017_int_base/CPU W, 0.201 SPECrate®2017_int_base/CPU $, https://www.spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47099.html) SPEC®, SPEC CPU®, and SPECrate® are registered trademarks of the Standard Performance Evaluation Corporation. See www.spec.org for more information. Intel CPU TDP and prices at https://ark.intel.com/ as of 4/17/2025.
- 9xx5-134: SPECpower_ssj® 2008 comparison based on published scores from www.spec.org as of 4/30/2025. 2P AMD EPYC 9965 (35920 ssj_ops/watt, 384 Total Cores, https://spec.org/power_ssj2008/results/res2024q4/power_ssj2008-20241007-01464.html) 2P AMD EPYC 9755 (29950 ssj_ops/watt, 256 Total Cores, https://spec.org/power_ssj2008/results/res2024q4/power_ssj2008-20240924-01460.html) 2P Intel Xeon 6980P (21679 ssj_ops/watt, 256 Total Cores, https://spec.org/power_ssj2008/results/res2025q2/power_ssj2008-20250324-01511.html) SPEC®, SPEC CPU®, and SPECpower® are registered trademarks of the Standard Performance Evaluation Corporation. See www.spec.org for more information.
- 9xx5-151: TPCxAI @SF30 Multi-Instance, 32C Instance Size throughput results based on AMD internal testing as of 04/01/2025 running multiple VM instances. The aggregate end-to-end AI throughput test is derived from the TPCx-AI benchmark and as such is not comparable to published TPCx-AI results, as the end-to-end AI throughput test results do not comply with the TPCx-AI Specification. 2P AMD EPYC 9965 (6067.53 Total AIUCpm, 384 Total Cores, 500W TDP, AMD reference system, 1.5TB 24x64GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910), 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu® 24.04 LTS kernel 6.13, SMT=ON, Determinism=power, Mitigations=on) 2P AMD EPYC 9755 (4073.42 Total AIUCpm, 256 Total Cores, 500W TDP, AMD reference system, 1.5TB 24x64GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910) 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu 24.04 LTS kernel 6.13, SMT=ON, Determinism=power, Mitigations=on) 2P Intel Xeon 6980P (3550.50 Total AIUCpm, 256 Total Cores, 500W TDP, Production system, 1.5TB 24x64GB DDR5-6400, 4 x 1GbE Broadcom NetXtreme BCM5719 Gigabit Ethernet PCIe 3.84TB SAMSUNG MZWLO3T8HCLS-00A07 NVMe, Ubuntu 24.04 LTS kernel 6.13, SMT=ON, Performance Bias, Mitigations=on) Results may vary based on factors including but not limited to system configurations, software versions, and BIOS settings. TPC, TPC Benchmark, and TPC-H are trademarks of the Transaction Processing Performance Council.
- 9xx5-169: Llama-3.3-70B latency constrained throughput (goodput ) results based on AMD internal testing as of 05/14/2025.Configurations: Llama-3.3-70B, vLLM API server v1.0, data set: Sonnet3.5-SlimOrcaDedupCleaned, TP8, 512 max requests (dynamic batching), latency constrained time to first token (300ms, 400ms, 500ms, 600ms), OpenMP 128, results in tokens/s. 2P AMD EPYC 9575F (128 Total Cores, 400W TDP, production system, 1.5TB 24x64GB DDR5-6400 running at 6000 MT/s, 2 x 25 GbE ConnectX-6 Lx MT2894, 4x 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe ; Micron_7450_MTFDKCC800TFS 800GB NVMe for OS, Ubuntu 22.04.3 LTS, kernel=5.15.0-117-generic , BIOS 3.2, SMT=OFF, Determinism=power, mitigations=off) with 8x NVIDIA H100. 2P Intel Xeon 8592+ (128 Total Cores, 350W TDP, production system, 1TB 16x64GB DDR5-5600 , 2 x 25 GbE ConnectX-6 Lx (MT2894), 4x 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Micron_7450_MTFDKBA480TFR 480GB NVMe, Ubuntu 22.04.3 LTS, kernel-5.15.0-118-generic, SMT=OFF, Performance Bias, Mitigations=off) with 8x NVIDIA H100. Results:CPU 300 400 500 600; 8592+ 0 126.43 1565.65 1987.19; 9575F 346.11 2326.21; 2531.38 2572.42; Relative NA 18.40 1.62 1.29. Results may vary due to factors including system configurations, software versions, and BIOS settings. TDP information from ark.intel.com
- 9xx5-013: Official MLPerf™ Inference score v4.1 Llama2-70B-99.9 server tokens/s and offline tokens/s results retrieved from https://mlcommons.org/benchmarks/inference-datacenter/ on 09/01/2024, from the following entries: 4.1-0070 (preview) and 4.1.0022. The MLPerf™ name and logo are trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.
- GD-183A: AMD Infinity Guard features vary by EPYC™ Processor generations and/or series. Infinity Guard security features must be enabled by server OEMs and/or Cloud Service Providers to operate. Check with your OEM or provider to confirm support of these features. Learn more about Infinity Guard at https://www.amd.com/en/products/processors/server/epyc/infinity-guard.html.
- 9xx5-152A: Deepseek-R1-671B throughput results based on AMD internal testing as of 01/28/2025. Configurations: llama.cpp framework, 1.58 bit quantization (UD_IQ1_S, MoE at 1.56 bit), batch sizes 1 and 4, 16C Instances, Use Case Input/Output token configurations: [Chatbot = 128/128, Essay = 128/1024, Summary = 1024/128, Rewrite = 1024/1024]. 2P AMD EPYC 9965 (384 Total Cores, 500W TDP, reference system, 3TB 24x128GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910) 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu® 22.04.3 LTS | 5.15.0-105-generic), SMT=ON, Determinism=power, Mitigations=on) 2P AMD EPYC 9755 (256 Total Cores, 500W TDP, reference system, 3TB 24x128GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910) 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu® 22.04.3 LTS | 5.15.0-105-generic), SMT=ON, Determinism=power, Mitigations=on) 2P Intel Xeon 6980P (256 Total Cores, 500W TDP, production system, 3TB 24x64GB DDR5-6400, 4 x 1GbE Broadcom NetXtreme BCM5719 Gigabit Ethernet PCIe 3.84TB SAMSUNG MZWLO3T8HCLS-00A07 NVMe, Ubuntu 24.04.2 LTS | 6.13.2-061302-generic, SMT=ON, Performance Bias, Mitigations=on) Results: BS=1 6980P 9755 9965 Rel9755 Rel9965 Chatbot 47.31 61.88 70.344 1.308 1.487 Essay 42.97 56.04 61.608 1.304 1.434 Summary 44.99 59.39 62.304 1.32 1.385 Rewrite 41.8 68.44 55.08 1.637 1.318 BS=4 6980P 9755 Rel9755 Rel9965 Chatbot 76.01 104.46 143.496 1.374 1.888 Essay 67.89 93.68 116.064 1.38 1.71 Summary 70.88 103.39 99.96 1.459 1.41 Rewrite 65 87.9 78.12 1.352 1.202 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- Comparison based on thread density, performance, features, process technology and built-in security features of currently shipping servers as of 10/10/2024. EPYC 9005 series CPUs offer the highest thread density [EPYC-025B], leads the industry with 500+ performance world records [EPYC-023F] with performance world record enterprise leadership Java® ops/sec performance [EPYCWR-20241010-260], top HPC leadership with floating-point throughput performance [EPYCWR-2024-1010-381], AI end-to-end performance with TPCx-AI performance [EPYCWR-2024-1010-525] and highest energy efficiency scores [EPYCWR-20241010-326]. The 5th Gen EPYC series also has 50% more DDR5 memory channels [EPYC-033C] with 70% more memory bandwidth [EPYC-032C] and supports 70% more PCIe® Gen5 lanes for I/O throughput [EPYC-035C], has up to 5x the L3 cache/core [EPYC-043C] for faster data access, uses advanced 3-4nm technology, and offers Secure Memory Encryption + Secure Encrypted Virtualization (SEV) + SEV Encrypted State + SEV-Secure Nested Paging security features. See the AMD EPYC Architecture White Paper (https://library.amd.com/l/3f4587d147382e2/) for more information.
- 9xx5-164: FAISS (Runs/Hour) throughput results based on AMD internal testing as of 04/08/2025. FAISS Configurations: v1.8.0, sift1m Data Set, 32 Core Instances, FP32 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P Xeon 6980P (256 Total Cores), 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias) Results: Throughput Relative 2P 6980P 36.63 1 2P 9755 46.86 1.279 2P 9965 58.6 1.600 Results may vary due to factors including system configurations, software versions and BIOS settings.
- 9xx5-162: XGBoost (Runs/Hour) throughput results based on AMD internal testing as of 04/08/2025. XGBoost Configurations: v1.7.2, Higgs Data Set, 32 Core Instances, FP32 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P Xeon 6980P (256 Total Cores), 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias) Results: CPU Throughput Relative 2P 6980P 400 1 2P 9755 436 1.090 2P 9965 771 1.928 Results may vary due to factors including system configurations, software versions and BIOS settings.
- 9xx5-156: Llama3.1-8B throughput results based on AMD internal testing as of 04/08/2025. Llama3.1-8B configurations: BF16, batch size 32, 32C Instances, Use Case Input/Output token configurations: [Summary = 1024/128, Chatbot = 128/128, Translate = 1024/1024, Essay = 128/1024]. 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 6.9.0-060900-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 6.8.0-52-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1 2P Xeon 6980P (256 Total Cores), AMX On, 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on Performance Bias), IPEX 2.6.0 Results: CPU 6980P 9755 9965 Summary 1 n/a1.093 Translate 1 1.062 1.334 Essay 1 n/a 1.14 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- 9xx5-158: GPT-J-6B throughput results based on AMD internal testing as of 04/08/2025. GPT-J-6B configurations: BF16, batch size 32, 32C Instances, Use Case Input/Output token configurations: [Summary = 1024/128, Chatbot = 128/128, Translate = 1024/1024, Essay = 128/1024]. 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 6.9.0-060900-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1, Python 3.10.12 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 6.8.0-52-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1, Python 3.10.12 2P Xeon 6980P (256 Total Cores), AMX On, 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias), IPEX 2.6.0, Python 3.12.3 Results: CPU 6980P 9755 9965 Summary 1 1.034 1.279 Chatbot 1 0.975 1.163 Translate 1 1.021 0.93 Essay 1 0.978 1.108 Caption 1 0.913 1.12 Overall 1 0.983 1.114 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- 9xx5-166: Llama3.2-1B throughput results based on AMD internal testing as of 04/08/2025. Llama3.3-1B configurations: BF16, batch size 32, 32C Instances, Use Case Input/Output token configurations: [Summary = 1024/128, Chatbot = 128/128, Translate = 1024/1024, Essay = 128/1024]. 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 6.9.0-060900-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1, Python 3.10.2 2P Xeon 6980P (256 Total Cores), AMX On, 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias), IPEX 2.6.0, Python 3.12.3 Results: CPU 6980P 9965 Summary 1 1.213 Translation 1 1.364 Essay 1 1.271 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- 9xx5-012: TPCxAI @SF30 Multi-Instance 32C Instance Size throughput results based on AMD internal testing as of 09/05/2024 running multiple VM instances. The aggregate end-to-end AI throughput test is derived from the TPCx-AI benchmark and as such is not comparable to published TPCx-AI results, as the end-to-end AI throughput test results do not comply with the TPCx-AI Specification.
2P AMD EPYC 9965 (384 Total Cores), 12 32C instances, NPS1, 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1DPC, 1.0 Gbps NetXtreme BCM5720 Gigabit Ethernet PCIe, 3.5 TB Samsung MZWLO3T8HCLS-00A07 NVMe®, Ubuntu® 22.04.4 LTS, 6.8.0-40-generic (tuned-adm profile throughput-performance, ulimit -l 198096812, ulimit -n 1024, ulimit -s 8192), BIOS RVOT1000C (SMT=off, Determinism=Power, Turbo Boost=Enabled)
2P AMD EPYC 9755 (256 Total Cores), 8 32C instances, NPS1, 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1DPC, 1.0 Gbps NetXtreme BCM5720 Gigabit Ethernet PCIe, 3.5 TB Samsung MZWLO3T8HCLS-00A07 NVMe®, Ubuntu 22.04.4 LTS, 6.8.0-40-generic (tuned-adm profile throughput-performance, ulimit -l 198096812, ulimit -n 1024, ulimit -s 8192), BIOS RVOT0090F (SMT=off, Determinism=Power, Turbo Boost=Enabled)
2P AMD EPYC 9654 (192 Total cores) 6 32C instances, NPS1, 1.5TB 24x64GB DDR5-4800, 1DPC, 2 x 1.92 TB Samsung MZQL21T9HCJR-00A07 NVMe, Ubuntu 22.04.3 LTS, BIOS 1006C (SMT=off, Determinism=Power)
Versus 2P Xeon Platinum 8592+ (128 Total Cores), 4 32C instances, AMX On, 1TB 16x64GB DDR5-5600, 1DPC, 1.0 Gbps NetXtreme BCM5719 Gigabit Ethernet PCIe, 3.84 TB KIOXIA KCMYXRUG3T84 NVMe, , Ubuntu 22.04.4 LTS, 6.5.0-35 generic (tuned-adm profile throughput-performance, ulimit -l 132065548, ulimit -n 1024, ulimit -s 8192), BIOS ESE122V (SMT=off, Determinism=Power, Turbo Boost = Enabled)
Results:
CPU Median Relative Generational
Turin 192C, 12 Inst 6067.531 3.775 2.278
Turin 128C, 8 Inst 4091.85 2.546 1.536
Genoa 96C, 6 Inst 2663.14 1.657 1
EMR 64C, 4 Inst 1607.417 1 NA
Results may vary due to factors including system configurations, software versions and BIOS settings. TPC, TPC Benchmark and TPC-C are trademarks of the Transaction Processing Performance Council.
Footnotes
- 9xxTCO-019[DM1] [MK2] : This scenario contains many assumptions and estimates and, while based on AMD internal research and best approximations, should be considered an example for information purposes only, and not used as a basis for decision making over actual testing. The AMD Server & Greenhouse Gas Emissions TCO (total cost of ownership) Estimator Tool - version 1.53, compares the selected AMD EPYC™ and Intel® Xeon® CPU based server solutions required to deliver a TOTAL_PERFORMANCE of 391,000 units of SPECrate2017_int_base performance as of September 30, 2025. This analysis compares a 2P AMD 192 core EPYC_9965 powered server with a SPECrate2017_int_base score of 3230, https://spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47086.pdf; compared to a 2P Intel Xeon 128 core Xeon_6980P based server with a SPECrate2017_int_base score of 2510, https://spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47099.pdf; versus legacy 2P Intel Xeon 28 core Platinum_8280 based server with a SPECrate2017_int_base score of 391, https://spec.org/cpu2017/results/res2020q3/cpu2017-20200915-23984.pdf Environmental impact estimates made leveraging data from the 2025 International Country Specific Electricity Factors and can be found at https://www.carbondi.com/#electricity-factors/ and the US EPA Greenhouse Gas Equivalencies Calculator used in this analysis was sourced on 09/04/2024 and can be found at https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator. For additional details, see https://www.amd.com/claims/9xx5TCO-019.
- 9xx5-128A: SPECrate®2017_int_base comparison based on published scores from www.spec.org as of 5/9/2025. 2P AMD EPYC 9965 (3230 SPECrate®2017_int_base, 384 Total Cores, 500W TDP, $14,813 CPU $), 6.460 SPECrate®2017_int_base/CPU W, 0.218 SPECrate®2017_int_base/CPU $, https://www.spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47086.html ) 2P AMD EPYC 9755 (2840 SPECrate®2017_int_base, 256 Total Cores, 500W TDP, $12,984 CPU $), 5.680 SPECrate®2017_int_base/CPU W, 0.219 SPECrate®2017_int_base/CPU $, https://www.spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47223.html ) 2P Intel Xeon 6980P (2510 SPECrate®2017_int_base, 256 Total Cores, 500W TDP, $12,460 CPU $) 5.020 SPECrate®2017_int_base/CPU W, 0.201 SPECrate®2017_int_base/CPU $, https://www.spec.org/cpu2017/results/res2025q2/cpu2017-20250324-47099.html) SPEC®, SPEC CPU®, and SPECrate® are registered trademarks of the Standard Performance Evaluation Corporation. See www.spec.org for more information. Intel CPU TDP and prices at https://ark.intel.com/ as of 4/17/2025.
- 9xx5-134: SPECpower_ssj® 2008 comparison based on published scores from www.spec.org as of 4/30/2025. 2P AMD EPYC 9965 (35920 ssj_ops/watt, 384 Total Cores, https://spec.org/power_ssj2008/results/res2024q4/power_ssj2008-20241007-01464.html) 2P AMD EPYC 9755 (29950 ssj_ops/watt, 256 Total Cores, https://spec.org/power_ssj2008/results/res2024q4/power_ssj2008-20240924-01460.html) 2P Intel Xeon 6980P (21679 ssj_ops/watt, 256 Total Cores, https://spec.org/power_ssj2008/results/res2025q2/power_ssj2008-20250324-01511.html) SPEC®, SPEC CPU®, and SPECpower® are registered trademarks of the Standard Performance Evaluation Corporation. See www.spec.org for more information.
- 9xx5-151: TPCxAI @SF30 Multi-Instance, 32C Instance Size throughput results based on AMD internal testing as of 04/01/2025 running multiple VM instances. The aggregate end-to-end AI throughput test is derived from the TPCx-AI benchmark and as such is not comparable to published TPCx-AI results, as the end-to-end AI throughput test results do not comply with the TPCx-AI Specification. 2P AMD EPYC 9965 (6067.53 Total AIUCpm, 384 Total Cores, 500W TDP, AMD reference system, 1.5TB 24x64GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910), 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu® 24.04 LTS kernel 6.13, SMT=ON, Determinism=power, Mitigations=on) 2P AMD EPYC 9755 (4073.42 Total AIUCpm, 256 Total Cores, 500W TDP, AMD reference system, 1.5TB 24x64GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910) 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu 24.04 LTS kernel 6.13, SMT=ON, Determinism=power, Mitigations=on) 2P Intel Xeon 6980P (3550.50 Total AIUCpm, 256 Total Cores, 500W TDP, Production system, 1.5TB 24x64GB DDR5-6400, 4 x 1GbE Broadcom NetXtreme BCM5719 Gigabit Ethernet PCIe 3.84TB SAMSUNG MZWLO3T8HCLS-00A07 NVMe, Ubuntu 24.04 LTS kernel 6.13, SMT=ON, Performance Bias, Mitigations=on) Results may vary based on factors including but not limited to system configurations, software versions, and BIOS settings. TPC, TPC Benchmark, and TPC-H are trademarks of the Transaction Processing Performance Council.
- 9xx5-169: Llama-3.3-70B latency constrained throughput (goodput ) results based on AMD internal testing as of 05/14/2025.Configurations: Llama-3.3-70B, vLLM API server v1.0, data set: Sonnet3.5-SlimOrcaDedupCleaned, TP8, 512 max requests (dynamic batching), latency constrained time to first token (300ms, 400ms, 500ms, 600ms), OpenMP 128, results in tokens/s. 2P AMD EPYC 9575F (128 Total Cores, 400W TDP, production system, 1.5TB 24x64GB DDR5-6400 running at 6000 MT/s, 2 x 25 GbE ConnectX-6 Lx MT2894, 4x 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe ; Micron_7450_MTFDKCC800TFS 800GB NVMe for OS, Ubuntu 22.04.3 LTS, kernel=5.15.0-117-generic , BIOS 3.2, SMT=OFF, Determinism=power, mitigations=off) with 8x NVIDIA H100. 2P Intel Xeon 8592+ (128 Total Cores, 350W TDP, production system, 1TB 16x64GB DDR5-5600 , 2 x 25 GbE ConnectX-6 Lx (MT2894), 4x 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Micron_7450_MTFDKBA480TFR 480GB NVMe, Ubuntu 22.04.3 LTS, kernel-5.15.0-118-generic, SMT=OFF, Performance Bias, Mitigations=off) with 8x NVIDIA H100. Results:CPU 300 400 500 600; 8592+ 0 126.43 1565.65 1987.19; 9575F 346.11 2326.21; 2531.38 2572.42; Relative NA 18.40 1.62 1.29. Results may vary due to factors including system configurations, software versions, and BIOS settings. TDP information from ark.intel.com
- 9xx5-013: Official MLPerf™ Inference score v4.1 Llama2-70B-99.9 server tokens/s and offline tokens/s results retrieved from https://mlcommons.org/benchmarks/inference-datacenter/ on 09/01/2024, from the following entries: 4.1-0070 (preview) and 4.1.0022. The MLPerf™ name and logo are trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.
- GD-183A: AMD Infinity Guard features vary by EPYC™ Processor generations and/or series. Infinity Guard security features must be enabled by server OEMs and/or Cloud Service Providers to operate. Check with your OEM or provider to confirm support of these features. Learn more about Infinity Guard at https://www.amd.com/en/products/processors/server/epyc/infinity-guard.html.
- 9xx5-152A: Deepseek-R1-671B throughput results based on AMD internal testing as of 01/28/2025. Configurations: llama.cpp framework, 1.58 bit quantization (UD_IQ1_S, MoE at 1.56 bit), batch sizes 1 and 4, 16C Instances, Use Case Input/Output token configurations: [Chatbot = 128/128, Essay = 128/1024, Summary = 1024/128, Rewrite = 1024/1024]. 2P AMD EPYC 9965 (384 Total Cores, 500W TDP, reference system, 3TB 24x128GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910) 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu® 22.04.3 LTS | 5.15.0-105-generic), SMT=ON, Determinism=power, Mitigations=on) 2P AMD EPYC 9755 (256 Total Cores, 500W TDP, reference system, 3TB 24x128GB DDR5-6400, 2 x 40 GbE Mellanox CX-7 (MT2910) 3.84TB Samsung MZWLO3T8HCLS-00A07 NVMe, Ubuntu® 22.04.3 LTS | 5.15.0-105-generic), SMT=ON, Determinism=power, Mitigations=on) 2P Intel Xeon 6980P (256 Total Cores, 500W TDP, production system, 3TB 24x64GB DDR5-6400, 4 x 1GbE Broadcom NetXtreme BCM5719 Gigabit Ethernet PCIe 3.84TB SAMSUNG MZWLO3T8HCLS-00A07 NVMe, Ubuntu 24.04.2 LTS | 6.13.2-061302-generic, SMT=ON, Performance Bias, Mitigations=on) Results: BS=1 6980P 9755 9965 Rel9755 Rel9965 Chatbot 47.31 61.88 70.344 1.308 1.487 Essay 42.97 56.04 61.608 1.304 1.434 Summary 44.99 59.39 62.304 1.32 1.385 Rewrite 41.8 68.44 55.08 1.637 1.318 BS=4 6980P 9755 Rel9755 Rel9965 Chatbot 76.01 104.46 143.496 1.374 1.888 Essay 67.89 93.68 116.064 1.38 1.71 Summary 70.88 103.39 99.96 1.459 1.41 Rewrite 65 87.9 78.12 1.352 1.202 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- Comparison based on thread density, performance, features, process technology and built-in security features of currently shipping servers as of 10/10/2024. EPYC 9005 series CPUs offer the highest thread density [EPYC-025B], leads the industry with 500+ performance world records [EPYC-023F] with performance world record enterprise leadership Java® ops/sec performance [EPYCWR-20241010-260], top HPC leadership with floating-point throughput performance [EPYCWR-2024-1010-381], AI end-to-end performance with TPCx-AI performance [EPYCWR-2024-1010-525] and highest energy efficiency scores [EPYCWR-20241010-326]. The 5th Gen EPYC series also has 50% more DDR5 memory channels [EPYC-033C] with 70% more memory bandwidth [EPYC-032C] and supports 70% more PCIe® Gen5 lanes for I/O throughput [EPYC-035C], has up to 5x the L3 cache/core [EPYC-043C] for faster data access, uses advanced 3-4nm technology, and offers Secure Memory Encryption + Secure Encrypted Virtualization (SEV) + SEV Encrypted State + SEV-Secure Nested Paging security features. See the AMD EPYC Architecture White Paper (https://library.amd.com/l/3f4587d147382e2/) for more information.
- 9xx5-164: FAISS (Runs/Hour) throughput results based on AMD internal testing as of 04/08/2025. FAISS Configurations: v1.8.0, sift1m Data Set, 32 Core Instances, FP32 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P Xeon 6980P (256 Total Cores), 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias) Results: Throughput Relative 2P 6980P 36.63 1 2P 9755 46.86 1.279 2P 9965 58.6 1.600 Results may vary due to factors including system configurations, software versions and BIOS settings.
- 9xx5-162: XGBoost (Runs/Hour) throughput results based on AMD internal testing as of 04/08/2025. XGBoost Configurations: v1.7.2, Higgs Data Set, 32 Core Instances, FP32 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 2P Xeon 6980P (256 Total Cores), 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias) Results: CPU Throughput Relative 2P 6980P 400 1 2P 9755 436 1.090 2P 9965 771 1.928 Results may vary due to factors including system configurations, software versions and BIOS settings.
- 9xx5-156: Llama3.1-8B throughput results based on AMD internal testing as of 04/08/2025. Llama3.1-8B configurations: BF16, batch size 32, 32C Instances, Use Case Input/Output token configurations: [Summary = 1024/128, Chatbot = 128/128, Translate = 1024/1024, Essay = 128/1024]. 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 6.9.0-060900-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 6.8.0-52-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1 2P Xeon 6980P (256 Total Cores), AMX On, 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on Performance Bias), IPEX 2.6.0 Results: CPU 6980P 9755 9965 Summary 1 n/a1.093 Translate 1 1.062 1.334 Essay 1 n/a 1.14 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- 9xx5-158: GPT-J-6B throughput results based on AMD internal testing as of 04/08/2025. GPT-J-6B configurations: BF16, batch size 32, 32C Instances, Use Case Input/Output token configurations: [Summary = 1024/128, Chatbot = 128/128, Translate = 1024/1024, Essay = 128/1024]. 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 6.9.0-060900-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1, Python 3.10.12 2P AMD EPYC 9755 (256 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.4 LTS, Linux 6.8.0-52-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1, Python 3.10.12 2P Xeon 6980P (256 Total Cores), AMX On, 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias), IPEX 2.6.0, Python 3.12.3 Results: CPU 6980P 9755 9965 Summary 1 1.034 1.279 Chatbot 1 0.975 1.163 Translate 1 1.021 0.93 Essay 1 0.978 1.108 Caption 1 0.913 1.12 Overall 1 0.983 1.114 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- 9xx5-166: Llama3.2-1B throughput results based on AMD internal testing as of 04/08/2025. Llama3.3-1B configurations: BF16, batch size 32, 32C Instances, Use Case Input/Output token configurations: [Summary = 1024/128, Chatbot = 128/128, Translate = 1024/1024, Essay = 128/1024]. 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400, 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 6.9.0-060900-generic, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1, ZenDNN 5.0.1, Python 3.10.2 2P Xeon 6980P (256 Total Cores), AMX On, 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias), IPEX 2.6.0, Python 3.12.3 Results: CPU 6980P 9965 Summary 1 1.213 Translation 1 1.364 Essay 1 1.271 Results may vary due to factors including system configurations, software versions, and BIOS settings.
- 9xx5-012: TPCxAI @SF30 Multi-Instance 32C Instance Size throughput results based on AMD internal testing as of 09/05/2024 running multiple VM instances. The aggregate end-to-end AI throughput test is derived from the TPCx-AI benchmark and as such is not comparable to published TPCx-AI results, as the end-to-end AI throughput test results do not comply with the TPCx-AI Specification.
2P AMD EPYC 9965 (384 Total Cores), 12 32C instances, NPS1, 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1DPC, 1.0 Gbps NetXtreme BCM5720 Gigabit Ethernet PCIe, 3.5 TB Samsung MZWLO3T8HCLS-00A07 NVMe®, Ubuntu® 22.04.4 LTS, 6.8.0-40-generic (tuned-adm profile throughput-performance, ulimit -l 198096812, ulimit -n 1024, ulimit -s 8192), BIOS RVOT1000C (SMT=off, Determinism=Power, Turbo Boost=Enabled)
2P AMD EPYC 9755 (256 Total Cores), 8 32C instances, NPS1, 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1DPC, 1.0 Gbps NetXtreme BCM5720 Gigabit Ethernet PCIe, 3.5 TB Samsung MZWLO3T8HCLS-00A07 NVMe®, Ubuntu 22.04.4 LTS, 6.8.0-40-generic (tuned-adm profile throughput-performance, ulimit -l 198096812, ulimit -n 1024, ulimit -s 8192), BIOS RVOT0090F (SMT=off, Determinism=Power, Turbo Boost=Enabled)
2P AMD EPYC 9654 (192 Total cores) 6 32C instances, NPS1, 1.5TB 24x64GB DDR5-4800, 1DPC, 2 x 1.92 TB Samsung MZQL21T9HCJR-00A07 NVMe, Ubuntu 22.04.3 LTS, BIOS 1006C (SMT=off, Determinism=Power)
Versus 2P Xeon Platinum 8592+ (128 Total Cores), 4 32C instances, AMX On, 1TB 16x64GB DDR5-5600, 1DPC, 1.0 Gbps NetXtreme BCM5719 Gigabit Ethernet PCIe, 3.84 TB KIOXIA KCMYXRUG3T84 NVMe, , Ubuntu 22.04.4 LTS, 6.5.0-35 generic (tuned-adm profile throughput-performance, ulimit -l 132065548, ulimit -n 1024, ulimit -s 8192), BIOS ESE122V (SMT=off, Determinism=Power, Turbo Boost = Enabled)
Results:
CPU Median Relative Generational
Turin 192C, 12 Inst 6067.531 3.775 2.278
Turin 128C, 8 Inst 4091.85 2.546 1.536
Genoa 96C, 6 Inst 2663.14 1.657 1
EMR 64C, 4 Inst 1607.417 1 NA
Results may vary due to factors including system configurations, software versions and BIOS settings. TPC, TPC Benchmark and TPC-C are trademarks of the Transaction Processing Performance Council.