iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
AI data centers are designed around the demands of large training and inference workloads—not just around installing accelerator cards in a conventional facility. The accelerators matter, but they work only if power, cooling, networking, storage and operational controls can keep pace with them. Whether an existing site can be adapted depends on its capacity and the workload; neither retrofits nor purpose-built facilities are right for every case.
Why do AI workloads change the data-center design?
In a conventional facility, adding servers may be possible without redesigning every system around them. Large AI deployments are less forgiving: many accelerators operate together, draw substantial power and produce concentrated heat, while exchanging and processing large volumes of data. That makes the facility a connected system. A constraint in power delivery, heat rejection, data movement or storage can limit the useful output of the compute hardware.
Training depends on coordination across the cluster
Training large models involves accelerators exchanging data with one another as work progresses. The network between them—often called the east-west network—therefore matters alongside the external connections that bring users or data into a facility. Network bandwidth and latency, storage throughput and compute capacity have to be planned together: accelerators can be underused if they wait for data or for other parts of the cluster.
Free tools Windows power users keep installed
One-click scans. No signup required.
Inference can put a premium on latency and location
Inference serves requests from a trained model. Depending on the service, response time and proximity to users may matter more than tightly coordinating a large training cluster. That can affect where capacity is placed and how it connects to users and data sources. A design optimized for one workload should not automatically be assumed to suit the other.
#1 Best Overall
Why are power and cooling central to AI data centers?
More computing capacity requires reliable power delivery, and the electricity used by computing equipment becomes heat that the facility must remove. Higher rack power density concentrates both demands in a smaller space. Facility design must therefore account for utility supply, backup power, distribution within the building, cooling equipment and the ability to reject heat—not just the number of racks that fit on the floor.
McKinsey & Company reported in an October 2024 analysis that average data-center rack power density had more than doubled over the preceding two years, from 8 kW to 17 kW per rack. The analysis projected that it could reach 30 kW by 2027 as AI workloads increased. These are dated estimates and a projection from that 2024 analysis, not current measurements or a guarantee about any particular facility.
Rank #2
Cooling methods suit different deployment conditions
Air-based approaches can become harder to use as heat loads concentrate. Liquid-cooling approaches bring heat away from equipment differently, but they are not interchangeable: the right method depends on rack density, equipment design, facility layout and how heat will ultimately be rejected. McKinsey’s 2024 analysis describes the following density ranges; they are source-reported examples, not universal equipment limits.
| Cooling approach | Density described by McKinsey | What to keep in mind |
|---|---|---|
| Rear-door heat exchangers | 40–60 kW per rack | McKinsey presents this as an option for that range; actual capability depends on implementation. |
| Direct-to-chip liquid cooling | 60–120 kW per rack | McKinsey describes it as commonly deployed in its 2024 account. Performance depends on the specific system and facility. |
| Immersion cooling | 100 kW per rack; above 150 kW for dual-phase use | These are ranges described in the source, not a claim that every immersion installation supports them. |
Cooling is not a standalone equipment choice. A facility needs a compatible path from the components to the heat-rejection system, plus the power, controls and operating procedures to run it. A cooling system that can handle a rack in isolation does not establish that the building can support a whole cluster at that density.
Can a traditional data center be retrofitted for AI?
Sometimes. Vertiv Distinguished Engineer and Vice President of Technical Business Development Peter Panfil said in a statement published by Mouser Electronics on July 24, 2026, that “many existing facilities can be upgraded to support selective AI workloads, but purpose-built designs are usually better suited.” That is an attributed expert view, not a universal engineering standard. The practical answer depends on the site and the intended workload.
Floor space alone is not a useful test. Before choosing a retrofit or new build, assess the whole chain from incoming power to delivered compute:
- Workload: Is the site meant for training, inference or a mix? The balance affects networking, placement and performance priorities.
- Rack density: What power and heat load will each rack create, and can the room and cooling design support it?
- Power: Is there enough utility capacity, distribution and backup power for the planned equipment?
- Heat rejection: Can the facility remove the heat at the intended density, using a cooling approach compatible with the equipment?
- Network: Can the east-west fabric provide the bandwidth and latency required for the cluster?
- Storage: Can storage deliver data at the throughput the workload needs?
- Expansion and timing: Can the site be upgraded in time, and can it scale without repeatedly running into the same constraints?
A retrofit is more plausible when the existing facility has adequate headroom or can be upgraded for a defined, selective workload. A larger or denser cluster may favor purpose-built infrastructure if power, cooling or network limits make adaptation impractical. These are conditional choices, not a rule that all conventional facilities must be replaced.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What is changing in data-center power distribution?
Higher-density AI systems are prompting proposals for different power architectures. NVIDIA’s 800 VDC initiative is a future-oriented plan, not evidence that 800 VDC is already broadly deployed. NVIDIA says the architecture could transmit 85% more power through the same conductor size and reduce copper requirements by 45% compared with 415 VAC distribution. It also claims up to a 5% improvement in end-to-end efficiency. These are vendor-stated benefits, not independently validated results.
NVIDIA says full-scale production is expected to coincide with its Kyber rack-scale systems in 2027. Moving to a new distribution architecture also raises safety, standards and workforce challenges. The figures should therefore be read as claims about a proposed architecture and roadmap, not as settled performance results for operating facilities.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
