Meteomatics revolutionizes weather forecasting with AMD EPYC™ CPUs

Meteomatics revolutionizes weather forecasting with AMD EPYC™ CPUs

AMD EPYC CPU performance enables unprecedented weather forecasting resolution and frequency in Europe and beyond.

Accurate weather forecasting has the potential to change the way we live. It can optimize the effectiveness of wind and solar energy and predict extreme environmental events with greater precision. Commercial organizations such as airlines and agriculture can obtain more advanced warning of climate activity affecting their business. Switzerland-based Meteomatics was founded to deliver on this promise. Its central plan is to provide “EURO1k” forecasts with a 1km spatial resolution every hour across the whole of Europe and beyond. AMD EPYC™ processors are helping the company realize this goal.

One component of Meteomatics’ plan is its “Meteodrones”, which replace weather balloons. While balloons tend to be single-use devices that get lost once deployed, drones come back to base for reuse and can obtain a more accurate recording of weather data. Meteomatics supplies its Meteodrones to defense, national weather agencies, and research institutions. But it also uses the data itself to provide forecasts. “Over the past few years, we have become the provider of choice for the energy industry,” says Alexander Stauch, Head of Marketing at Meteomatics. The company wanted to go further, however. “We collect a lot of weather data using our drones, but we also have a lot of weather station data from different providers, including radar and satellite. We decided to build a new weather model.”

Delivering greater weather forecasting accuracy

“There are two main problems in meteorology that we wanted to revolutionize,” says Stauch. “First, our drones provide the data foundation that models need to calculate good forecasts. Second, weather models typically have a resolution of around 10km. Some global models even have lower resolution of 20 or 25km, so a lot of important weather details can be missed, such as hailstorms, local thunderstorms, extreme heavy rain, and wind channeling effects.”

“Our aim was to make weather forecasting more accurate, so we came up with EURO1k,” says Stauch. “This is a big improvement over previous weather data accuracy, especially for hyperlocal events such as hail, fog, and heavy rain.” This is particularly valuable for the renewable energy industry like wind power. “It's all about optimizing power grids and trading strategy. If you have a lot of solar and wind farms in your portfolio you want to know how much you will be producing tomorrow to calculate the right prices.” Demand is also highly dependent on temperature, so accurate weather data helps manage energy resources more effectively.

However, building a weather model as detailed and frequent as EURO1k requires a lot of processing power. Meteomatics found AMD EPYC CPUs delivered the performance the company needed. “We started with the 3^rd Gen around two years ago,” says Christian Schluchter, CTO, Meteomatics. “The decision to use 3^rd Gen AMD EPYC CPUs was based on the benchmarks we ran together with NEC. AMD EPYC processors proved to have the best performance. We had been happy with how 3^rd Gen AMD EPYC CPUs worked, and 4^th Gen gave a significant boost, so it was the obvious choice.” NEC’s role was crucial, supplying the expertise needed to build an HPC cluster, which Meteomatics didn’t have internally.

Doubling performance with 4^th Gen AMD EPYC CPUs

“One of the particular challenges was that we wanted to produce the forecast within one hour, which is a game changer in weather modeling,” says Stauch. “The global European Centre for Medium-Range Weather Forecasts (ECMWF) model takes six hours to calculate, and previous services provided three or four model outputs per day. We wanted our model hourly. This was a huge step, so we needed to find an infrastructure that allowed us to distribute the processing tasks to different CPU cores in parallel.”

Meteomatics completed thorough testing to ensure the AMD EPYC processors would deliver the performance required. “We used generic benchmarks,” says Schluchter. “We also ran some technical workloads to see how the CPUs performed and to quantify how the network scaled. The most crucial test was the Weather Research and Forecasting (WRF) EURO1k model that we use operationally. It was essential to operate the model 24 times a day, meaning we have exactly one hour to run one model. We have a minimal requirement that we need to be able to run, and then we try to get as much done within this hour as we can. Comparing the new 4th Gen AMD EPYC™ CPU cluster with our existing 3rd Gen nodes, we got the same performance for around half the cost. This has allowed us to do significantly more computations within one hour of computation time.

“There are two ways the new infrastructure has helped,” says Stauch. “First, the weather forecast is more up to date because you get a new model output every hour. The other positive aspect is accuracy. Normal weather models cannot deal with rapid local events while EURO1k, based on the frequency that it is running and the granularity of the one-kilometer grid, is able to capture these events in a very reliable way.”

“There is a huge advantage for the energy industry’s trading strategies,” says Stauch. “But also, for everything that is dependent on a very reliable forecast for the next 8 to 24 hours, such as aviation. They want to know if they can keep their airport and runways open. You can also find more efficient airplane routes, saving CO2. Insurance companies can use more granular weather data to produce more accurate risk models.”

Unique weather forecasting frequency and accuracy

 “Ten years ago, it might have been possible to run EURO1K, but for very high costs,” says Schluchter. “Now it’s financially viable with AMD EPYC processors. The performance is the key feature for us. The large L3 cache has been particularly valuable. We are running the X version of AMD EPYC CPUs with more L3, which provides around 20 percent improvement over the non-X version. The core density also helps. Having fewer nodes hosting the same cores means less networking and space, making it less expensive. Also, our bottleneck is usually the interconnects and with more cores within one node, you have less communication between nodes.”

“We already had 300 nodes of 3^rd Gen AMD EPYC CPUs, roughly 40,000 cores,” says Schluchter. “We are now deploying another 280 nodes of the 4^th Gen, adding around 60,000 more cores for a total of 100,000.” This dramatic expansion in compute power will dramatically increase the resolution and regional consistency of forecasts. “Having better forecasts helps evacuate people earlier if there are storms. Additionally, the more accurately you can forecast renewable energy production, the more you can use it. If we can forecast better, we can handle climate change more easily.”

“The performance of AMD EPYC processors has been great,” concludes Stauch. “EURO1k is unique. There is nothing in weather modeling that comes even close to that now, in terms of data capacity that goes into the model and how many runs we do per day. It’s a huge innovation in weather models.”