Why AI Needs a Different Kind of Data Center

Image Source: AlexGo/Stock.adobe.com

By Nicolette Emmino for Mouser Electronics

Published July 24, 2026

An artificial intelligence (AI)-native data center is designed with AI in mind from the beginning. Many people may associate AI infrastructure with graphics processing units (GPUs) and accelerators, and therefore assume that an AI-native data center simply requires more of them. They might even think it’s just a matter of adding more GPUs to an existing data center. But then they discover that there isn’t enough power, cooling, networking capacity, or storage throughput to support them.

This realization is driving the next possible evolution of data center infrastructure. The challenge is not just whether organizations can acquire enough GPUs, but whether they can build the infrastructure that is needed to support them.

Traditional data centers were designed around enterprise and cloud workloads, including business applications, databases, websites, and storage services. AI-native data centers are being built for AI, training models, running inference workloads, and supporting clusters of accelerators—all operating at the same time.

Just a few years ago, modern AI platforms like ChatGPT and Gemini didn’t exist, and large-scale AI inference had not yet become a mainstream infrastructure requirement. With the explosive growth of AI use, today’s AI workloads behave differently. Training a large language model could require hundreds or thousands of GPUs working continuously.^[1] They must exchange data with one another, access large datasets, consume enormous amounts of power, and consequently generate substantial heat. Supporting this type of workload has created a demand for a new generation of infrastructure.

This article explores the growth of AI-native data centers, the challenges of retrofitting existing data centers for AI, and the requirements for building the next generation of AI infrastructure.

Why AI Data Centers are Expanding So Quickly

There are several factors contributing to the rapid growth of AI data center interest and projects (Figure 1). Companies are training larger AI models, more people are using AI tools every day, and businesses are finding new ways to incorporate AI into their products. All this is happening as cloud providers continue to expand their infrastructure to meet growing demand. The result is a race to deploy AI.^[2]

Figure 1: Multiple factors are contributing to the rapid growth of AI data centers. (Source: Mouser Electronics/Author)

The demand is becoming apparent throughout the semiconductor industry. In June 2026, one major chipmaker raised its data center revenue expectations to roughly US$1 billion for 2026, citing continued strong demand driven by AI infrastructure and indicating that revenue could double again in 2027 if current market conditions continue.^[3]

Organizations don’t want to be left behind as competitors introduce new AI-powered products and services. When combined, these factors are driving the investment in the compute resources needed to support the next wave of AI applications. Supporting these workloads requires a different approach to data center design.

Can a Traditional Data Center Be Retrofitted for AI?

For some organizations expanding AI workloads, the first instinct may be to add GPUs to existing data center infrastructure. For smaller deployments or inference workloads, this instinct may be successful.

Peter Panfil, Distinguished Engineer and Vice President of Technical Business Development at Vertiv, notes that “many existing facilities can be upgraded to support selective AI workloads, but purpose-built designs are usually better suited when scale, density, deployment timelines, and long-term efficiency are the priority” (Table 1).^[4]

Table 1. Different workload requirements for traditional data centers and AI-native data centers

Power is typically the first challenge. AI server racks can consume much more power than traditional server racks, placing greater demand on electrical distribution and backup power systems.

“Traditional facilities often run out of usable power or cooling at the rack or row level long before they run out of physical floor space,” says Panfil.

Power is closely tied to cooling, which becomes another obstacle, especially as rack densities increase and liquid cooling becomes more common. Networking infrastructure may also need to be upgraded to support additional GPU-to-GPU communication for AI training workloads.^[5]

Cisco, for example, encountered similar challenges when building its AI infrastructure and ultimately opted to deploy its AI environment in a separate facility designed to meet AI’s demands.^[6]

As AI workloads continue to grow, some organizations will be able to successfully retrofit an existing data center; many others will find that an AI-native data center is a more practical path forward. The decision will often come down to infrastructure requirements, such as power delivery, cooling, storage, and networking.

Infrastructure Requirements of AI-Native Data Centers

One of the biggest misconceptions about AI data center infrastructure is that simply adding more compute will solve a problem. Many organizations discover that power, cooling, networking, and storage are just as important as the GPUs.

Compute Layer

AI-native data centers typically require specialized processors designed to handle AI workloads, while still relying on CPUs, storage nodes, networking, and control systems to keep the overall environment operating. Traditional data centers have relied on CPUs to run business applications, databases, and web services, but AI workloads are being performed on GPUs, TPUs, and other AI accelerators optimized for parallel processing.

Unlike a CPU, which handles tasks sequentially, a GPU can perform many calculations simultaneously. That capability makes it well-suited for training and running large AI models, where billions of calculations may be performed across massive datasets. As AI models become larger, organizations may deploy hundreds or thousands of accelerators to work together as a single compute cluster.

This strategy is creating a demand for compute capacity. In some cases, organizations have faced extended lead times for GPUs as manufacturers work to increase production capacity.^[7] Industry observers indicate that there are constraints in GPU manufacturing, as well as in advanced packaging technologies and high-bandwidth memory, which are needed to support next-gen AI systems.^[8]

As a result, access to compute resources is an important consideration for companies planning large-scale AI deployments.

Access to more GPUs, TPUs, and other custom AI accelerators solves part of the challenge, but those processors must constantly exchange data and access large datasets, which requires significant power and creates substantial heat. As a result, the surrounding infrastructure will determine how effectively the compute resources can operate.

Networking and Data Movement

Moving the data becomes just as important as processing them. AI workloads create different traffic patterns than traditional applications. During training, hundreds or thousands of GPUs may work together on a single model, continuously exchanging information. This exchange creates significant “east-west” traffic, in which data moves among GPUs, servers, and racks, rather than just between users and applications.^[9] As AI clusters grow, the network becomes a more important part of overall performance, because if data cannot move fast enough between accelerators, costly compute resources can sit idle waiting for information.

The challenge is bigger than just communication with a single server. Today’s AI deployments can span multiple servers, racks, and even data centers, requiring high-bandwidth, low-latency connections to keep everything working efficiently. Industry leaders are pointing to connectivity as one of the next major challenges in AI scaling, arguing that future performance will depend on how efficiently data can move through the infrastructure.^[10]

These factors are driving the adoption of technologies such as the InfiniBand standard and high-speed Ethernet fabrics, which are designed to meet the high-bandwidth and low-latency requirements of AI workloads.^[11] The industry is also working to overcome new challenges that come with increased data rates. Recent PCIe 6.0 and CXL 3.1 retimer products are addressing signal-reach and latency challenges in AI systems, helping improve communication among processors, memory, and storage.^[12]

Some companies have noted that future AI performance will depend more on how efficiently data can move across accelerators, servers, racks, campuses, and even geographically distributed data centers. As a result, networking is not just a matter of connecting equipment together, but a critical part of the AI-native data center infrastructure.

Storage at an AI Scale

GPUs and TPUs can process only data that are available to them. AI models receive enormous amounts of information, are trained on datasets that can reach petabyte scales, and include text, images, video, audio, code, and sensor data.

If storage systems cannot keep pace with accelerator demands, the compute resources can again sit idle waiting for data, making storage throughput another critical factor in overall AI system performance. Training runs may take days or weeks, creating additional storage requirements. AI operators frequently save checkpoints through the training process, so that work can resume in the event of an interruption. These checkpoints also consume storage capacity, especially when large models are involved.

AI workloads have pushed storage requirements beyond what traditional environments were originally designed to handle. Organizations are using everything from cloud platforms such as Amazon S3 and Azure Blob Storage to high-performance, on-premises systems from providers such as VAST Data, Pure Storage, and Dell. In architectures such as NVIDIA’s DGX SuperPOD, storage is designed not just for capacity, but also for throughput, ensuring thousands of GPUs can access data simultaneously without creating bottlenecks.^[13]

Sourcing, Distributing, Converting, and Protecting Power

GPUs, memory, and networking are all important to the infrastructure of an AI-native data center, but power is perhaps the primary challenge.

Traditional data center racks can consume between 5kW and 15kW of power.^[14] Current AI racks can exceed 200kW, with some next-generation designs pushing even higher.^[15] Packing more accelerators into a smaller footprint can make it more difficult to deliver electricity to those systems.

The power challenge begins before a single server is even installed. Brian Smith, Chief Technology Officer, Nuclear Science & Technology, and Director, Nuclear Reactor Development, Idaho National Laboratory, notes that projected load growth from AI infrastructure is increasing at an alarming rate and driving demand for new power generation.^[16] He also points to a growing trend of data center developers pursuing “bring your own power” strategies, in part to avoid interconnection queues that can take years to navigate. This strategy calls for operators to supplement or replace utility-supplied electricity with on-site power generation techniques, such as natural gas generators or small modular reactors (SMRs). In some regions, developers are evaluating sites based on available electrical capacity and utility access before considering other factors, making power availability a primary consideration in AI data center development. Securing sufficient power from the local grid can take years, a bottleneck that has little to do with semiconductors or networking and everything to do with electrical infrastructure.

Vertiv’s Panfil reinforces this point, noting that “today’s constraints are largely utility source capacity and heat rejection real estate. Source capacity can be mitigated by Bring Your Own Power & Cooling (BYOP&C) strategies that add locally generated power to the source mix. BYOP&C is also being applied to use energy more than once by recovering waste heat versus just rejecting it.”

There is also growing interest in alternative energy sources, including advanced nuclear reactors, SMRs, and other long-term energy solutions, to support the growth of future AI infrastructure.

Once power reaches the facility, it must be distributed efficiently to dense AI clusters, prompting changes in power architecture. Traditional data centers relied on multiple conversion stages to move electricity from the grid to the rack, but each stage introduces losses that can be significant when multiplied across thousands of servers.

As a result, the industry is exploring new approaches to power distribution. Some newer AI-focused power designs have explored high-voltage DC architectures, including 800VDC systems to improve efficiency, reduce component count, and free up space in denser server environments.^[17]

Even relatively small efficiency improvements can have a meaningful impact on an AI scale. Beyond the rack, new silicon carbide (SiC)-based power modules are being developed for solid-state transformers (SSTs), which are emerging as a potential way to simplify power delivery from the medium-voltage grid to AI infrastructure.^[18] By reducing the number of conversion stages, SSTs can improve efficiency while supporting higher power densities required by future AI facilities.

Reliable power supply is another important factor in the AI-native data center. AI training can run for long periods, so backup power systems are critical to the infrastructure. Uninterruptible power supplies, redundant power paths, and backup generation help ensure that a brief interruption does not result in lost training progress or halted compute resources.

Power priorities in AI-native facilities differ from those in traditional data centers. AI facilities are consuming more power, forcing designers to rethink how power is sourced, distributed, converted, monitored, and protected.

Everything Becomes Heat

Every watt consumed by a processor will eventually become heat. As AI racks become denser, heat removal is one of the biggest engineering challenges in the AI-native data center. Rising rack densities are prompting engineers to design cooling and power systems together rather than as separate infrastructure layers.

“One example of this convergence is thermal response buffering,” says Panfil. “An AI load steps from about 30 percent load to 100 percent load at the initiation of the compute cycle. This has the potential to cause a temperature increase as the load steps up. A thermal buffer can be added to manage this dynamic. The power system knows what power is being delivered and can signal the thermal system so it will see a thermal gradient.”

Traditional data centers rely on air cooling, using CRAC units, raised floors, and careful airflow management to maintain acceptable operating temperatures. These strategies remain effective for many conventional workloads, but for AI systems, air cooling is not as practical.

As a result, liquid cooling technologies are being viewed as a key solution. This method can remove heat more efficiently than air. Direct-to-chip cooling has become one of the most popular options. Instead of cooling the air around the equipment, the system removes heat from the components that generate it, making it easier to support higher rack densities.

However, liquid cooling introduces new considerations, such as water availability.^[19] While many liquid cooling systems operate in closed-loop configurations to minimize water consumption, operators must still consider long-term access to cooling resources when selecting sites for future facilities.^[20]

Some operators are also exploring immersion cooling, a technique in which servers are submerged in a dielectric fluid that absorbs heat directly from the equipment. While direct-to-chip cooling is currently the more common approach, immersion cooling is attracting interest in certain high-density environments for its ability to efficiently remove large amounts of heat.^[21]

Conclusion

AI growth is impacting data center design. Traditional data centers can be retrofitted under the right circumstances, but larger deployments are increasingly pushing organizations toward AI-native data centers designed for specialized requirements. Networking systems must move data, storage platforms must keep pace with massive amounts of data, power infrastructure must be in place to support the higher rack densities, and cooling systems must remove the additional heat generated.

The future of AI infrastructure will need to better integrate compute, networking, storage, power, cooling, and operational controls, and future challenges will be centered on how effectively the entire infrastructure ecosystem works together. New performance metrics are beginning to emerge, with Panfil pointing to measures such as “tokens per watt” and “speed to token,” which evaluate how efficiently AI systems convert energy into useful output and how quickly those results can be delivered to users.

Looking ahead, infrastructure planning will require a longer-term perspective than organizations have traditionally taken. “Our recommendation is to be future-ready, to not just provision for today’s AI compute,” says Panfil. “Set that future state and work back to today’s state.”

Sources

[1]https://nvidianews.nvidia.com/news/nvidia-brings-large-language-ai-models-to-enterprises-worldwide
[2]https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/ai-power-expanding-data-center-capacity-to-meet-growing-demand
[3]https://newsroom.st.com/media-center/press-item.html/c3396.html
[4] Peter Panfil, interview by Nicolette Emmino
[5]https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/high-performance-ai-infra-amd-so.html
[6]https://www.cisco.com/site/us/en/solutions/cisco-on-cisco/ai-ready-infrastructure.html
[7]https://www.apollo.com/wealth/insights-news/insights/2026/06/growing-compute-shortage
[8]https://www.scientificamerican.com/article/high-bandwidth-memory-is-a-bottleneck-for-ai-chips/
[9]https://community.hpe.com/t5/networking/east-west-data-center-network-traffic-is-becoming-the-new-north/ba-p/7254929
[10]https://www.marvell.com/company/newsroom/marvell-keynote-computex-2026-future-of-scaling-ai-depends-on-connectivity.html
[11]https://www.hpe.com/us/en/what-is/ai-data-center-networking.html
[12]https://www.microchip.com/en-us/about/news-releases/products/xpressconnect-pcie-6-cxl-3-1-retimers
[13]https://docs.nvidia.com/dgx-superpod/reference-architecture-scalable-infrastructure-h100/latest/storage-architecture.html
[14]https://iaeimagazine.org/electrical-fundamentals/how-much-electricity-does-a-data-center-use-complete-2025-analysis/
[15]https://developer.nvidia.com/blog/nvidia-800-v-hvdc-architecture-will-power-the-next-generation-of-ai-factories/
[16] Brian Smith, interview by Nicolette Emmino
[17]https://www.power.com/resources/green-room/blog/1250-v-1750-v-gan-solution-addresses-need-800-v-bus-architectures-power-hungry-ai-data-centers
[18]https://ir.microchip.com/news-events/press-releases/detail/1390/microchip-launches-3-3-kv-hvd3-msic-power-modules-to-enable-solid-state-transformers-for-ai-data-centers
[19]https://www.issa.com/industry-news/ai-data-center-water-consumption-is-creating-an-unprecedented-crisis-in-the-united-states/
[20]https://www.oracle.com/news/announcement/blog/closed-loop-cooling-in-oracle-ai-data-centers-2026-02-09/
[21]https://www.vertiv.com/en-us/insights/articles/blog-posts/advancing-data-center-performance-with-immersion-cooling/