Recommended Free Tools
NVIDIA announced a liquid-cooled A100 80GB PCIe accelerator on May 23, 2022, and outlined plans at the time for liquid-cooled H100 PCIe and HGX H100 offerings in early 2023. Those dates describe the company’s launch-era plans, not confirmation of current stock or availability.
What NVIDIA announced
NVIDIA said its A100 80GB PCIe GPU used direct-chip liquid cooling, calling it the company’s first data-center PCIe GPU with that cooling approach. The cards were sampling when announced, with general availability expected in summer 2022, according to NVIDIA’s May 23, 2022 announcement.
In its COMPUTEX recap, NVIDIA said at least a dozen system builders would support liquid-cooled A100 PCIe GPUs and forecast the first systems shipping in Q3 2022. It also described liquid cooling for HGX H100 servers and an H100 PCIe card as planned for early 2023. These were forecasts made at the time, not a present-day inventory or shipment report. See NVIDIA’s COMPUTEX recap and its launch announcement.
How A100 and H100 differ
| Accelerator | Generation | Liquid-cooled announcement | Configuration context |
|---|---|---|---|
| A100 80GB PCIe | Ampere | NVIDIA announced it in May 2022; sampling was underway, with general availability expected that summer. | PCIe card; compatibility depends on a supported server and cooling design. |
| H100 PCIe and HGX H100 | Hopper, the generation NVIDIA described as succeeding Ampere | NVIDIA described liquid-cooled options for early 2023. | H100 PCIe card and HGX H100 server are distinct configurations, not interchangeable labels. |
NVIDIA’s A100 product brief distinguishes PCIe cards from SXM GPUs and HGX systems. Its H100 materials describe the Hopper-generation accelerator. A PCIe card, an SXM GPU, and an HGX system are not equivalent hardware configurations; a server must support the particular accelerator and cooling arrangement.
#1 Best Overall
- Standard Memory: 40 GB
- Host Interface: PCI Express 4.0
- Cooler Type: Passive Cooler
- Product Type: Graphics Card
What NVIDIA said about efficiency
NVIDIA said the liquid-cooled launch cards would deliver the same performance for less energy, and framed higher performance for the same energy as a future possibility. Those are NVIDIA’s claims, not independent card-specific test results. The cited material does not establish realized power savings or performance gains for liquid-cooled A100 or H100 PCIe cards.
In the announcement, Zac Smith, then head of edge infrastructure at Equinix, said: “This marks the first liquid-cooled GPU introduced to our lab, and that’s exciting for us because our customers are hungry for sustainable ways to harness AI.” Smith also said, “Measuring wattage alone is not relevant, the performance you get for the carbon impact you have is what we need to drive toward.” These comments express Equinix’s perspective; they are not benchmark findings.
Quick Recap
Best Value
- NVIDIA Ampere Architecture-based CUDA Cores - Double-speed processing for single-precision floating point (FP32) operations and improved power efficiency provide significant performance improvements for graphics and simulation workflows, such as complex 3D computer-aided design (CAD) and computer-aided engineering (CAE), on the desktop.
- Second-Generation RT Cores - With up to 2X the throughput over the previous generation and the ability to concurrently run ray tracing with either shading or denoising capabilities, second-generation RT Cores deliver massive speedups for workloads like photorealistic rendering of movie content, architectural design evaluations, and virtual prototyping of product designs. This technology also speeds up the rendering of ray-traced motion blur for faster results with greater visual accuracy.
- Third-Generation Tensor Cores - New Tensor Float 32 (TF32) precision provides up to 5X the training throughput over the previous generation to accelerate AI and data science model training without requiring any code changes. Hardware support for structural sparsity doubles the throughput for inferencing. Tensor Cores also bring AI to graphics with capabilities like DLSS, AI denoising, and enhanced editing for select applications.
- Third-Generation NVIDIA NVLink - Increased GPU-to-GPU interconnect bandwidth provides a single scalable memory to accelerate graphics and compute workloads and tackle larger datasets.
- 48 Gigabytes (GB) of GPU Memory - Ultra-fast GDDR6 memory, scalable up to 96 GB with NVLink, gives data scientists, engineers, and creative professionals the large memory necessary to work with massive datasets and workloads like data science and simulation.
Rank #4
- GPU Memory Size: 16 GB GDDR6 with ECC
- Form Factor: 2.7"(H) x 6.6"(L), dual slot, half height.
- Thermal Solution: Blower Active Fan
Rank #3
- The H100 NVL graphics card is designed to scale the support of large language models, such as GPT3-175B, in mainstream PCIe-based server systems, providing up to 12X the throughput performance of HGX A100 systems when configured with 8 units.
- Equipped with advanced features, including 94GB of high-speed HBM3 memory, NVLink connectivity for enhanced inter-GPU communication, and an impressive memory bandwidth of 3938 GB/sec, the H100 NVL is built for high-performance AI inference tasks.
- The card showcases a robust performance spectrum across various compute types: 68 TFLOPS for FP64, 134 TFLOPS for both FP64 Tensor Core and FP32, escalating up to 7916 TFLOPS/TOPS for FP8 and INT8 Tensor Core operations, all benefiting from sparsity optimizations.
- It enables standard mainstream servers to deliver high-performance capabilities for generative AI inference, simplifying the deployment process for partners and solution providers with fast time to market and ease of scalability.
- The H100 NVL's power efficiency is optimized with a configurable maximum power consumption ranging between 2x 350-400W, supporting extensive computational tasks without excessive power usage.
Rank #2
- Data Center Class Reliability: Designed for 24x7 data center operations, ensuring optimum performance, durability, and longevity to meet demanding real-world conditions in machine learning and AI tasks.
- Ampere Architecture: Employs the world's most powerful data center GPU, offering exceptional AI, data analytics, and high-performance computing capabilities.
- Enhanced Tensor Cores: Accelerate deep learning matrix arithmetic at the heart of neural network training and inferencing, resulting in faster and more efficient AI computations.
- High-Speed HBM2e Memory: Equipped with 80GB of high-bandwidth memory, delivering improved raw bandwidth and higher memory bandwidth efficiency for data-intensive AI applications.
- PCIe Gen 4 Support: Provides double the bandwidth of PCIe Gen 3, improving data-transfer speeds for AI and data science workloads, maximizing performance for machine learning tasks.
What to check before choosing a system
- Form factor: Confirm whether the proposed hardware is a PCIe card, an SXM GPU, or an HGX configuration. The names refer to different implementations.
- Server support: Ask the OEM to confirm the exact accelerator and server combination, including support for the required liquid-cooling design.
- Workload and memory: Match the configuration to the AI, data analytics, or high-performance computing workload and its memory requirements. The announced A100 liquid-cooled PCIe configuration was 80GB.
- Current availability and price: Confirm both with NVIDIA or an OEM. The 2022 and early-2023 announcements do not establish what is in stock, what it costs, or which compatible configurations are currently offered.
- Evidence for efficiency: Request measurements for the specific server and workload before assuming a claimed energy benefit will apply to a deployment.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
