AI & HPC: Evolving Data Center Cooling
With AI & HPC Trending Hot, How Is Data Center Cooling Evolving to Keep Pace?
Image Source: Евгений Вершинин/stock.adobe.com
By JJ DeLisle for Mouser Electronics
Published April 4, 2025
Out of the approximately 26,787TWh of electricity expected to be generated in 2025, it is estimated that data centers and cryptocurrency will consume approximately 536TWh, roughly 2 percent of total global electricity generation.[1],[2] With present trends, data center consumption will likely reach, and even exceed, 1,000TWh by 2030, approaching 5 percent of total global electricity consumption.[3] Because much of the global electricity supply still relies on coal, natural gas, oil, and other non-renewable sources and the increasing pace of the digital transformation is only trending toward greater electricity consumption, it is imperative to increase renewable energy generation for these technologies while enhancing the efficiency of energy use for these applications.
One area that consumes significant electrical energy is electronics cooling within hyperscale data centers, such as those running artificial intelligence (AI) and high-performance computing (HPC) workloads. The cooling requirements for AI and HPC deployments vary greatly, but they generally have a much higher duty cycle. They can also reach power densities over 10 times those of typical data center server applications. Enhancing the efficiency of data center cooling has been a focus for several years, as energy prices are rising while the electrical energy supply is struggling to catch up. Some new data centers are being built alongside renewable energy generation infrastructure with the intent to power the data centers from these renewables. This approach can be highly cost-effective from an energy use standpoint, but the operating energy demand from cooling technologies can still be substantial enough to raise the initial capital costs of such an endeavor significantly.
Experts have devised many innovative approaches to evolving data centers into more energy-efficient platforms, such as fully immersed data center pods in coastal areas. Still, traditional data centers in the United States are often built in remote and arid climates. The availability of low-cost land, minimal regulations, and favorable tax arrangements help to make these locations desirable for data center sites, with the trade-off of higher cooling demands and limited or costly water availability.
This article examines current methods for cooling data center facilities and future technologies that could disrupt the electronics cooling industry.
Trends in Data Center Cooling
Rack power densities in ultra-high-density data centers have risen from below 10kW to over 100kW in some cases.[4] For these power density levels, air cooling is largely considered inadequate in cooling capacity and too inefficient to compete with liquid cooling methods. Of these methods, single-phase direct-to-chip (DTC) liquid cooling is currently leading in a market rapidly striving for more efficient and effective cooling methods.
DTC technology is both mature and effective for cooling the main processing hardware. However, this approach often leaves the secondary hardware chips, such as networking, RAM, and storage, to be air-cooled. A potential solution to this is single-phase or two-phase immersion cooling. With immersion cooling, an entire data center rack is immersed in dielectric fluids that are thermally conductive, which effectively transfers the thermal energy away from the electronic components to the fluid, which can then be efficiently cooled using a heat exchanger (single-phase) or a vapor and condensation management system (two-phase). Rear-door heat exchangers are becoming commonplace, especially as a retrofit solution for legacy data centers that are predominantly air-cooled.
Air cooling presents the lowest upfront cost, whereas DTC and immersion cooling methods are among the most expensive and complex. For new data center installations with abundant capital and good profit projections, liquid cooling methods are clearly ideal. However, high upfront costs lead some legacy data centers and even new installations to continue relying on inefficient air-cooling systems that use traditional air-conditioning technologies. Some regional regulatory considerations may limit the use of such inefficient cooling technologies, but the cost motive for higher operating costs is more likely to dissuade the use of legacy air-cooling data center methods. There have also been strides in enhancing the efficiency of these systems to reduce costs, such as fan positioning, rack location, and airflow optimization.
Immersion cooling is poised to be the most efficient and sustainable method of cooling technology, especially considering the technology’s ability to handle extremely high heat density. As the latest AI and HPC technologies push for racks with even greater power density, immersion cooling—which some estimates claim can potentially reduce data center cooling energy consumption by as much as 90 percent compared to traditional air cooling—will likely benefit from increased adoption rates.[5] As the technology is new, the infrastructure from widespread immersion cooling is not yet aligned with the pace of new data center rollouts. However, this is likely to change in the next several years as more immersion cooling companies emerge to profit from the massive investments in data centers.
Another potential advantage of liquid cooling methods, especially immersion cooling, is the ability to redirect and repurpose waste heat. Many applications use waste heat, such as heating pools, schools, shopping districts, indoor horticulture installations, waste treatment, chemical processing/refinement, or even thermal energy storage systems, which can convert waste back into electrical energy based on utility demand.
The Future of Data Center Cooling
The previously discussed constraints are now coexisting with the need for highly responsive AI and HPC technologies to be closer to end users. Proximity is necessary to achieve millisecond response times for AI, virtual reality (VR), augmented reality (AR), and automation functions, which are considered necessary for many applications of these technologies. Most end-use applications are in dense urban areas, which means that building additional data center space comes with a much higher real estate cost, less total available space, more local restrictions on pollution and water use, and possible constraints on utility energy capacity. With this additional footprint limitation, achieving the data center performance goals is becoming a density issue.
While data center racks are likely to increase in power density, especially with the adoption of wafer-scale AI processors consuming tens of kilowatts of power each, cooling technology is lagging data center growth and expansion. This trend will only worsen as more AI and HPC technologies are deployed and cloud infrastructure is expanded to meet growing demand.
Overhauling legacy air-cooled systems and incorporating liquid cooling into new data center designs is a monumental task for data center operators. Any live data center environment changes come with significant risks and complex operations. These challenges are why new data center locations will likely be built from the ground up to accommodate these changes instead of repurposing existing data centers—if the capital is available. There are a growing number of partnerships among data center operators and cooling businesses as industry mergers and acquisitions of electronics cooling businesses are poised to facilitate the adoption of advanced cooling technologies.
Catching up will require much more rigorous sensors, analytics, and optimization. Traditional algorithmic cooling control systems are likely to be replaced by more advanced cooling optimization technology, including AI, to handle the potential tens of thousands of sensors and cooling units in a data center. Part of this optimization may be to run data centers hotter than legacy targets, using advanced monitoring and analytics to ensure that no single piece of hardware reaches temperatures that result in degraded performance.
Conclusion
Where air cooling is still dominant today, the next few years will see retrofits and new data centers leveraging hybrid air/liquid cooling with future data centers or extremely high-density data centers using advanced liquid cooling. DTC must become much more reliable to enable this, as even a few seconds of failed DTC could result in extensive server downtime or damage. Immersion cooling meets this reliability need and allows for greater power densities at the cost of greater complexity and specialized installation and maintenance. With future rack power densities exceeding 100kW or even 300kW, engineers must consider heat dissipation beyond the rack. Channeling the excess thermal energy from cooling is going to become increasingly crucial, especially as many governments and agencies are looking to enact legislation that requires data centers to achieve a notable level of sustainability.
Sources
[1]https://www.datacenterdynamics.com/en/opinions/four-key-trends-disrupting-data-centers-in-2025/
[2]https://www2.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2025/genai-power-consumption-creates-need-for-more-sustainable-data-centers.html
[3]https://www.datacenterfrontier.com/cloud/article/55253151/8-trends-that-will-shape-the-data-center-industry-in-2025
[4]https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/ai-power-expanding-data-center-capacity-to-meet-growing-demand
[5]https://www.hpcwire.com/2024/02/01/the-genai-data-center-squeeze-is-here/