How to Plan Power and Cooling for an AI Data Center

iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more

Plan an AI data center as one coupled power, cooling, compute, networking and building-systems project—not as an ordinary server room with more racks. Start with the accelerator workload and its rack-density roadmap, convert synchronized peak and transient loads into utility, UPS and distribution requirements, then select air, hybrid or liquid cooling for each zone and climate. Validate the design in a coupled digital model before construction and continuously commission it after the first workloads arrive. This approach prevents the most expensive failures: insufficient grid capacity, overloaded distribution, thermal runaway during training events, stranded electrical plant and a cooling system that looks efficient on paper but cannot survive a hot day or maintenance outage.

Why does an AI data center need a different power-and-cooling plan?

AI accelerators concentrate far more electrical and thermal power in each rack than conventional CPU systems, and their demand can change quickly. ASHRAE’s 2026 AI Data Center Energy Performance Framework describes rack densities rising from roughly 120 kW toward several hundred kilowatts, with megawatt-class racks anticipated. Those figures describe a trajectory, not a universal specification for every current server, but they explain why average IT load is no longer a safe sizing basis.

#	Preview	Product	Price
1		Thermal Guidelines for Data Processing Environments by TC9.9 MISSION CRITICAL FACILITIES...	$147.83	Buy on Amazon

Large training jobs can start or synchronize across many racks, creating step changes that propagate through transformers, UPS systems, busways, power-distribution units, pumps and control loops. Cooling must respond at the same time. A design that only satisfies a monthly average can still trip protection, exhaust UPS ride-through, or let GPU inlet temperatures exceed the equipment envelope during a synchronized event.

ASHRAE states the design principle plainly: “Power and cooling can no longer be treated as separable domains; decisions in one directly affect the other.” Treat that sentence as the project’s governing requirement. Every workload assumption should have an electrical and heat-rejection consequence, and every cooling choice should be checked against its electrical overhead, water impact, service needs and expansion path.

#1 Best Overall

Thermal Guidelines for Data Processing Environments by TC9.9 MISSION CRITICAL FACILITIES (2004-06-30)

$147.83

Buy on Amazon

How do I define the workload and rack-density envelope?

Build the electrical and thermal model from the equipment and the way it will run. Do not begin with a preferred cooling product or a nominal rack rating.

Inventory the workload and hardware

• Record the GPU or other accelerator type, server configuration, memory, storage and network adapters.
• Create a rack bill of materials, including the number of servers, top-of-rack equipment, power supplies and any rack-level conversion equipment.
• Separate training, fine-tuning, inference and background jobs. Their utilization and synchronization patterns are different.
• Document expected utilization, deployment phases, growth horizon and the proportion of time each phase will operate.
• Include networking traffic and storage activity; both add electrical load and can change the timing of peaks.

Convert the inventory into design quantities

For each phase, calculate per-rack kW, hall MW, annual energy, heat that must be rejected, and the fastest expected step or ramp. Produce separate values for normal operation, synchronized training, inference bursts, startup, shutdown and failure recovery. Record whether a value is measured on the exact reference platform or is a planning estimate that must be verified during acceptance testing.

Model coincidence instead of assuming diversity

Conventional data centers often rely on diversity because not every server peaks together. AI training can defeat that assumption. Model the case in which the planned accelerator fleet starts a synchronized job, then test the electrical and thermal response when networking, storage and cooling auxiliaries rise with it. Include the short transient, the sustained plateau and the return to a lower load. Controls should not overreact to a brief event, but they must not lag a sustained one.

How do I size utility service, UPS and distribution?

Translate the phased IT roadmap into a utility interconnection and an expandable electrical one-line. The service must cover the ultimate credible load and the delivery schedule must match each deployment block; installing all ultimate capacity on day one can strand capital, while delaying the interconnection can leave finished halls without power.

Work through the electrical chain

1. Confirm the utility envelope. Check available interconnection capacity, service voltage, fault levels, protection requirements, tariff structure, renewable-energy constraints and the utility’s delivery date.
2. Map each deployment phase. Assign required medium-voltage switchgear, transformers, UPS capacity, generators or alternate generation, rack PDUs, busways, grounding and power-quality equipment to the phase that needs it.
3. Check coincident and transient demand. Simulate synchronized training, accelerator step loads, power-factor correction, UPS ride-through, generator transitions and credible equipment failures.
4. Coordinate electrical and cooling controls. A load step can start pumps and heat-rejection equipment at the same time that it stresses the electrical system. Test sequencing, ramp limits, shedding and recovery together.
5. Reserve maintainable capacity. Provide isolation, bypasses and fault-selective protection so a component can be serviced without taking an entire AI zone offline.

Should I evaluate higher-voltage rack distribution?

At extreme rack density, current and copper become physical constraints. ASHRAE identifies evaluation of higher-voltage distribution, including 800 VDC, as a possible way to reduce current and conductor requirements. That is an engineering option, not an automatic recommendation: compare conversion stages, protection, arc and shock safety, grounding, maintenance skills, equipment availability and failure containment against conventional distribution. Select the voltage only after the rack reference design and service procedures are known.

Design modular blocks, not a single oversized plant

Use repeatable electrical and cooling blocks that can be energized with each deployment phase. A block should include the upstream distribution, UPS or alternate generation allocation, rack distribution, cooling distribution and controls needed for its planned load. Leave physical and control interfaces for the next block, and verify that partial build-out does not force pumps, fans or chillers to run inefficiently at very low load.

When do AI racks require liquid cooling?

Use air where rack density and heat flux remain within the selected equipment’s allowable envelope; evaluate liquid as rack density approaches and exceeds roughly 50–120 kW. ASHRAE recommends a Technology Cooling System (TCS) for purpose-built AI facilities where compute densities routinely exceed about 50–120 kW per rack. The threshold is guidance for design attention, not a universal cutoff: exact server inlet limits, rack layout, climate and redundancy still govern.

Compare the main cooling architectures

What belongs in a Technology Cooling System?

For a liquid-cooled AI zone, size the complete system rather than just the cold plate. The design includes CDUs, primary and secondary loops, pumps, manifolds, heat exchangers, filtration, leak detection, isolation valves, controls and maintenance bypasses. Establish pressure, temperature and flow limits for every rack and verify balance at minimum and maximum deployment. Provide a safe way to isolate one rack or branch without draining or shutting down the entire hall.

Can a hall mix air and liquid?

Yes. Segment AI/HPC halls from general-purpose computing so each zone can use suitable setpoints, redundancy and controls. A mixed strategy can retain contained air for lower-density racks, use rear-door exchangers for intermediate loads and reserve direct-to-chip or immersion for the densest rows. Keep the zones physically and hydraulically clear; otherwise a failure or maintenance action in one system can compromise the other.

Can warm-water cooling eliminate chillers and reduce water use?

It can in suitable climates and with hardware rated for the required fluid temperatures, but it does not eliminate heat-rejection engineering. NVIDIA states that modern liquid-cooled AI systems can operate with cooling liquid up to 45°C (verify the exact server and rack reference design). That capability can permit dry-cooler-based or chiller-less heat rejection, reducing compressor energy and potentially water consumption.

Check the complete warm-water envelope

• Model supply and return temperatures against the installed hardware’s facility-fluid limits and the site’s design-weather conditions.
• Size dry coolers and heat exchangers for the hottest credible operating period, not only a typical day.
• Include pump energy, control-valve authority, filtration, water treatment and freeze protection where the climate requires it.
• Determine whether adiabatic assistance is needed during extreme heat; ASHRAE’s warm-water, chiller-less concepts note that it may be required.
• Test operation during a hot-weather peak, a pump or fan failure, maintenance isolation and a rapid load increase.

Warm water changes the balance between compressor energy, fan energy, pump energy and water. Compare those terms with PUE and water metrics under the actual climate and operating schedule rather than assuming that a higher fluid temperature is always more efficient.

When does heat recovery make sense?

If a dependable heat sink exists, evaluate recovery for district heating, nearby buildings, domestic-hot-water preheat or process loads. Measure the exported useful heat with energy-reuse metrics such as Energy Reuse Effectiveness (ERE) or Energy Reuse Factor (ERF); a low PUE alone does not show that recovered heat is useful.

How should I design air management and the thermal envelope?

Air management remains important even in a liquid-cooled hall because not every component transfers heat to liquid and many facilities retain mixed-density equipment.

• Use hot-aisle or cold-aisle containment to keep supply and return air separated.
• Control airflow precisely and install blanking panels so bypass air does not dilute cooling capacity.
• Use supply-air reset only within the allowable inlet-temperature and humidity envelope of the installed equipment.
• Apply the appropriate ASHRAE thermal class and document allowable, recommended and excursion conditions for each zone.
• Physically separate AI/HPC zones from general-purpose halls so setpoints, redundancy and controls can be tuned independently.

Raising supply temperature can improve economizer hours, but only when the equipment, humidity-control strategy and monitoring prove that the new setpoint is safe. Record sensor locations and alarm thresholds; a room-average temperature can hide a hot spot at a high-flux rack.

How do I model the design before construction?

Use a coupled digital model—or an equivalent integrated engineering model—that includes compute, networking, power delivery, cooling loops, building systems, operating schedule, climate, utilization and growth. NVIDIA recommends this whole-system modeling before construction because changing a rack density or workload schedule can alter every other subsystem.

Run the cases that expose hidden constraints

• Average operation at each deployment phase.
• Maximum synchronized training and inference events.
• Fast load ramps and short transients.
• Peak outdoor temperature and humidity.
• Loss of a pump, fan, CDU, transformer, UPS module or cooling branch.
• Maintenance with one path isolated.
• Partial occupancy, when auxiliaries may operate at low load.
• Future expansion, including the rack-density trajectory toward several hundred kilowatts.

Use computational-fluid-dynamics analysis or equivalent airflow and thermal analysis for room layouts. For liquid systems, analyze pressure, temperature and flow balance through every branch. Compare air, hybrid and liquid alternatives on the same workload and weather cases; otherwise a favorable result may simply reflect different assumptions.

What commissioning tests prove that the design works?

Commissioning must test controls and failure behavior, not only nameplate capacity. Reconcile measured results with the model and recalibrate it as the fleet changes.

1. Verify sensors, meters, alarms, interlocks and emergency-stop behavior.
2. Apply representative electrical load steps and sustained plateaus while observing UPS, transformers, busways, power quality and protection.
3. Apply corresponding thermal steps and confirm that pumps, CDUs, fans, heat exchangers and heat rejection respond without overshoot.
4. Exercise utility transitions, generator or alternate-generation operation, ride-through and controlled recovery.
5. Isolate cooling branches and electrical components according to the maintenance plan; confirm the remaining path carries the intended load.
6. Test leak detection, automatic isolation, drain or refill procedures and safe service access for liquid systems.
7. Compare measured PUE, water, carbon and utilization data with model predictions, then set a schedule for continuous commissioning.

Continuous commissioning is essential because accelerator generations, software, utilization and schedules evolve. Keep the digital model, alarm limits and operating procedures synchronized with those changes rather than treating acceptance testing as the end of design.

Which design metrics should govern the decision?

Use a metric stack. No single number captures electrical overhead, water, carbon and useful computation.

ASHRAE recommends real-time monitoring, intelligent controls, digital twins and continuous commissioning so setpoints, economizers, pumps, fans and workload schedules can be tuned as the AI fleet evolves.

Interpret illustrative figures correctly

ASHRAE presents an illustrative 50 MW comparison with PUE about 1.40 versus about 1.10. It is an architecture example, not a guaranteed industry benchmark or promise for a particular site. Likewise, the 50–120 kW rack range is a framework threshold for considering a TCS, and the 45°C liquid capability is a modern-system reference that must be checked against the exact hardware.

How should I compare air, hybrid and liquid options?

Score every candidate against the same operational and site constraints. A design that wins on one efficiency metric but cannot obtain grid capacity, survive a hot day or be maintained safely is not viable.

What commonly goes wrong, and how can I prevent it?

• Sizing from average IT load: add synchronized training, ramp and failure cases to the model.
• Choosing cooling before density is known: freeze the workload and rack envelope first, then compare air, rear-door, direct-to-chip and immersion.
• Ignoring residual air heat: even liquid racks have fans, memory, storage and networking; retain an adequate air path.
• Assuming 45°C works for every server: verify the exact facility-fluid envelope and reference design.
• Installing ultimate capacity too early: stage modular electrical and cooling blocks with the deployment plan.
• Optimizing PUE alone: report water, carbon, useful-compute and heat-reuse metrics alongside it.
• Leaving controls until commissioning: model and test electrical and cooling sequences together before occupancy.
• Failing to plan liquid maintenance: include isolation, bypass, filtration, leak detection and safe service procedures in the original design.

What is a practical end-to-end planning sequence?

1. Document accelerator hardware, rack bills of material, workload types, utilization, networking, storage, phases and growth.
2. Convert those assumptions into per-rack kW, hall MW, annual energy, heat rejection and synchronized transient cases.
3. Confirm utility interconnection, voltage, fault, protection, tariff, renewable and delivery constraints.
4. Lay out modular medium-voltage, transformer, UPS, generation, busway, PDU, grounding and power-quality blocks.
5. Zone the building by density and select contained air, rear-door, direct-to-chip or immersion accordingly.
6. For liquid zones, design CDUs, loops, pumps, manifolds, heat exchangers, filtration, detection, isolation and bypasses as one system.
7. Model warm-water and economizer operation against hardware limits and local weather, including extreme heat and freeze conditions.
8. Build the coupled digital model and compare alternatives under average, peak, transient, failure, maintenance and growth cases.
9. Commission electrical, thermal, controls, leak response, failover and utility-transition behavior; reconcile measured data with the model.
10. Operate a metric stack covering PUE, WUE/WUI, CUE, DCRE/ITWC and ERE/ERF, with continuous commissioning as workloads evolve.

Frequently Asked Questions

Can I retrofit an existing air-cooled data hall for AI servers?

Possibly, but first verify structural loading, electrical fault duty, busway and transformer capacity, floor and ceiling clearances, and the ability to route liquid safely. A phased retrofit may keep lower-density rows on contained air while adding a separately controlled liquid zone; the coupled model should prove that shared utilities and failure scenarios remain within limits.

What information should be in a rack reference design before ordering cooling equipment?

Include the exact accelerator and server configuration, rack power supplies, network hardware, facility-fluid limits, allowable inlet conditions, connection sizes, flow and pressure requirements, service clearances and isolation method. Without that reference design, a nominal rack-kW figure is not enough to select CDUs, manifolds or heat exchangers.

Does chiller-less cooling work in every climate?

No. Warm-water and dry-cooler concepts depend on outdoor design conditions and the hardware’s fluid-temperature envelope. Hot-weather peaks may require adiabatic assistance, and the design still needs pump energy, freeze protection where applicable, treatment and resilient heat-rejection capacity.

How often should an AI data-center model be updated?

Update it whenever accelerator generations, rack bills of material, workload schedules, utilization targets, cooling setpoints or expansion phases change. Continuous commissioning then uses measured electrical, thermal, water and utilization data to recalibrate assumptions and adjust controls.

Quick Recap

Bestseller No. 1

Thermal Guidelines for Data Processing Environments by TC9.9 MISSION CRITICAL FACILITIES (2004-06-30)

$147.83

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.