Optimizing Data Center Infrastructure in a Constrained Memory Market

Customers Still Want to Refresh Their Infrastructure

The computing landscape is in flux; as AI adoption continues across the enterprise market, its impact is already clear: higher prices and component scarcity. As memory and storage costs remain elevated, customers are reassessing their IT and AI strategy, and with pricing not expected to ease until at least 2027,¹ many are evaluating whether an infrastructure refresh is even possible.

Customer budgets are being squeezed as they look ahead, so the answer you’re probably already hearing is, “Never mind, we’ll just wait it out.”

However, there is a cost of doing nothing.

Delaying the replacement of legacy infrastructure introduces its own risks:

• High operational costs
• Low energy efficiency
• Rising maintenance costs
• Inability to support modern AI workloads
• Security/compliance exposure
• Warranty/support expiration

Facing these issues, customers will look to partners like you to guide them. Use the insights below to support them through new hardware investments, helping them make smart purchasing decisions.

On Track: Providing the Right Guidance

Step 1: Audit the Environment

Historically, it’s been common for customers to overprovision server memory to accommodate maximum needs and add a layer of flexibility to their workloads. Basically, they provision many servers with the maximum resources only needed by a few applications so they can re-locate workloads if needed. As such, many environments are provisioned based on a theoretical peak demand rather than observed utilization.

With memory scarce and costly, this is a luxury fewer businesses can afford. Now is the time to work with your customers to audit their on-premises and cloud environments to determine what resources (CPU, GPU and memory) they’re using and to project realistic, data-driven assessments of needs as workloads or the addition of AI compute grows. 

Step 2: Identify Non-Memory-Sensitive Workloads

Once baseline memory requirements are met, some workloads often see diminishing returns from additional memory capacity. For CPU-bound, I/O-bound or latency-sensitive workloads, additional memory capacity can often deliver limited performance improvements relative to cost.

Help customers explore lower memory configurations that reduce unnecessary costs without materially impacting workload performance.

Step 3: Right-Size Memory Provisioning

Memory is at a premium, which makes it vital for customers to correctly size the amount of memory they require. Work with your customers to explore where memory investment delivers the greatest value and where they can safely reduce memory needs.

Mission-critical and high-priority workloads should receive a healthy buffer for any spikes in utilization or future growth. For workloads with lower memory requirements, consider providing smaller capacity DIMMs or leaving memory slots open (AMD EPYC™ server CPUs support a variety of memory configurations, including open slots).

AMD Can Help

Once customers understand where memory utilization can be optimized, they can begin evaluating platforms that are designed to deliver stronger performance efficiency, such as with systems enabled by AMD EPYC server CPUs.

It’s worth reminding customers looking to refresh their existing infrastructure that AMD EPYC server CPUs can offer great performance with less memory and a compelling upgrade path from aging hardware. Moving from competitor legacy hardware to the latest AMD EPYC server CPUs can achieve up to 29% lower TCO, averaging a payback for systems and software of around 2.5 years.²

For example, representative compute-bound workloads running on the AMD EPYC 9355P server CPUs with 50% reduced memory capacity results in just a 1% performance reduction.³

Memory-capacity-sensitive workloads can scale flexibly with memory constraints; a server running on AMD EPYC 9655 CPUs can operate with just 75% of memory capacity with a minor 6% performance reduction.⁴

Infrastructure modernization also creates an opportunity for consolidation and higher virtualization density. By reducing server count and improving efficiency, your customers can also reduce software licensing, power, cooling and operational costs alongside optimizing their memory usage. Being able to purchase far fewer, higher-performance systems means less of the IT budget will be spent on memory.

For organizations facing space, power or cooling limitations, infrastructure efficiency improvements help extend existing datacenter capacity without requiring immediate expansion. As AI workloads continue to increase demands on infrastructure, these operational efficiencies will continue to become increasingly important alongside raw performance.

Providing Tools for Customer Success

AMD offers a range of tools that can help you support your customers navigating a challenging market. To help right-size memory budgets, use the following tools to estimate memory requirements for both on-prem and cloud environments:

For cloud infrastructure, utilize the AMD EPYC™ Cloud Instance Advisor.

Run a TCO analysis using the AMD EPYC™ TCO Advisory Tool, a web-based tool that lets you work with customers to analyze different TCO scenarios based on current, but also customizable memory prices.

AMD will continue to support partners and their customers, delivering leadership performance and optimized infrastructure. Using hardware platforms like AMD EPYC server CPUs and tools like those listed above help balance performance, scalability and memory efficiency during challenging market cycles. 

While memory pricing pressures continue to impact infrastructure planning, delaying modernization can introduce long-term operational inefficiencies and rising costs. By auditing workloads, right-sizing memory investments and identifying opportunities for consolidation, partners can help customers modernize strategically.

To learn more, speak to your AMD representative.