The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Size a multi-GPU AI server in three separate domains: the GPU-to-GPU links inside a server, the scale-out fabric between servers, and the storage/customer path. Then size storage capacity and storage performance independently. GPU count alone is not enough: topology, workload, dataset format, cache behavior, concurrent jobs, and checkpoint policy determine the bill of materials.
Start with the three network domains
A multi-GPU system can have several networks, each with a different job. Treating every link as one pool leads to either an underpowered training fabric or an unnecessarily expensive design.
1. Scale-up: GPU-to-GPU traffic inside a node
The scale-up domain connects GPUs within one server or tightly coupled chassis. NVIDIA describes NVLink as a proprietary scale-up interconnect. Its topology, link count, and switch arrangement determine how quickly GPUs exchange tensors, gradients, and activations without leaving the node. Read the server’s actual topology diagram rather than inferring performance from the number of GPU slots.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
For a single-node workload, this domain is usually the first communication limit to check. Confirm which GPU pairs have direct links, which paths traverse a switch, and whether the CPU and PCIe root complexes introduce a slower route. A server can have a large external NIC while still suffering from an imbalanced internal GPU topology.
#1 Best Overall
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
2. Scale-out: the cluster compute fabric
When training or inference spans multiple servers, the cluster interconnect carries collective operations such as all-reduce, all-gather, and parameter exchange. NVIDIA’s architecture documentation identifies this cluster interconnect (CIN) as configurable for Ethernet or InfiniBand, while NVLink remains the scale-up domain. See NVIDIA’s Data Center Architecture and the HGX AI Factory network architecture.
Scale-out sizing depends on node count, the distributed framework, message sizes, synchronization frequency, and the utilization target. A two-node fine-tuning job and a hundreds-of-GPUs synchronous training job can have very different requirements even when each node contains the same GPUs.
3. Storage, customer, and management paths
Storage traffic is a separate path in NVIDIA’s HGX architecture. Data reads, checkpoint writes, user access, monitoring, and management should not be assumed to share the same bandwidth as the GPU compute fabric. A storage bandwidth number therefore cannot be read as a compute-fabric number. Keep storage/customer connectivity and management traffic isolated or deliberately segmented, with quality-of-service and failure behavior defined for each.
Collect the inputs before choosing links or drives
Write down the following facts for the exact server and workload. If any are unknown, the resulting network or storage number is only a placeholder.
- Platform: GPU model and count, server model, CPU socket count, PCIe generation, NVLink topology, available NIC slots, and supported local NVMe bays.
- Deployment shape: one server or multiple nodes, expected node count over the next expansion, rack and switch layout, and whether the design must survive a failed link, NIC, or switch.
- Workload: pretraining, fine-tuning, inference, batch inference, data preparation, or mixed use; distributed framework; collective communication pattern; sequence length; and data-loader behavior.
- Data path: dataset size, file and record format, average and peak read rate, random versus sequential access, compression, number of workers, and whether the active set fits in RAM or local cache.
- Artifacts: model weights, tokenizer and code packages, intermediate outputs, checkpoint size, checkpoint interval, retention count, and required restart time.
- Operations: number of concurrent jobs, service-level targets, maintenance windows, backup or replication policy, and the performance that must remain available during a component failure.
These inputs prevent a common error: buying a high-rate NIC because the server has many GPUs, while leaving storage, PCIe placement, or checkpoint behavior as an afterthought.
Size a single-node server
Verify the internal GPU topology
For one server, map GPU-to-GPU paths first. Record which links are direct, which traverse an NVLink switch, and which communication paths fall back to PCIe. Also map each GPU, NIC, and NVMe device to a CPU socket and PCIe root complex. The goal is balanced placement: no group of GPUs should contend for one oversubscribed root complex while other devices sit idle.
NVIDIA’s certified-system guidance explicitly calls for balanced PCIe placement of GPUs, NICs, and NVMe devices. Use the platform’s validated placement rules rather than moving cards solely to obtain a convenient physical arrangement. The relevant configuration guide is NVIDIA’s Certified Systems Configuration Guide.
Decide whether an external compute NIC is needed
A single-node job that never exchanges tensors with another server does not automatically need one external NIC per GPU. The NIC is required for storage, user access, or scale-out communication, not simply because the server contains multiple GPUs. If the server will join a cluster later, leave enough PCIe lanes, switch ports, and rack capacity for that growth.
Check the NIC’s sustained usable rate, CPU and PCIe placement, receive and transmit queues, firmware, and support for the selected Ethernet/RoCE or InfiniBand stack. Advertised line rate is not the same as application throughput after protocol overhead and contention.
Size a multi-node compute fabric
Use configuration figures as starting points, not universal rules
NVIDIA’s current certified-system configuration guidance includes at least 200 Gbps of NIC capability for multi-node inference and up to 400 Gbps per GPU. Those are recommendations for the configurations covered by that guide, not a universal requirement for every multi-GPU server, every inference service, or every training job. Apply them only after checking the exact GPU model, node count, topology, and target utilization.
Rank #2
- BREAKTHROUGH PCIe 5.0 PERFORMANCE: Supercharge your workflow and gaming with PCIe 5.0, boasting up to 14,700/13,400 MB/s* sequential read/write speeds. Tackle massive files and power up your gaming with Gen5—twice as fast as the 990 PRO SSD.
- EVERY TASK, TURBOCHARGED: Speed past productivity limits. With random read/write speeds up to 1,850K/2,600K IOPS*, enjoy fast game loads, seamless AI apps, and efficient multitasking. Virtually no lag, no limits—just nonstop performance.
- THINK FAST, CREATE FASTER: With random read/write speeds of up to 1,850K/2,600K IOPS*, the 9100 PRO SSD fuels seamless AI content creation, swift loads, and smooth gameplay. Work, play, and create at lightning speed.
- SPEED, WHENEVER YOU NEED: From laptops to desktop PCs, experience blazing PCIe 5.0 speeds and up to 8TB of storage. Perfect for video editing, gaming, and creative tasks, with the compatibility to match your device.
- STAY COOL, RUN FAST: Push limits, not temperatures. A 5nm controller boosts power efficiency up to 49% over the 990 PRO SSD*, while advanced thermal control keeps performance smooth and reliable.
Do not interpret the upper figure as proof that every GPU needs a dedicated 400-Gbps port. Actual port count and speed depend on how traffic is aggregated, the distributed algorithm, the number of nodes, switch radix, oversubscription, and the amount of communication that remains inside each node.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Compare designs on usable, balanced bandwidth
For each candidate design, document the following rather than comparing port labels alone:
- Aggregate usable bandwidth: total bidirectional capacity available to the job after reserving links for storage, management, and redundancy.
- Per-GPU allocation: the bandwidth each GPU can obtain in the worst placement and during concurrent collectives.
- Latency and message behavior: small-message latency, large-message throughput, and whether the application is sensitive to synchronization delays.
- Topology: GPU-to-NIC affinity, NIC-to-switch paths, rail layout, and the number of switch hops between nodes.
- Oversubscription: uplink ratios, shared switch ports, and what happens when all nodes transmit simultaneously.
- Fabric choice: Ethernet with the required lossless or congestion-control features, or InfiniBand with its supported software and operations model.
- Resilience: dual rails, alternate paths, switch redundancy, and the performance retained after one component fails.
Use the DSX architecture description to keep the scale-up and scale-out terms separate. The final design still needs the server vendor’s validated NIC placement, switch configuration, and software versions.
InfiniBand or 400-Gbps Ethernet?
There is no answer based on GPU count alone. Both fabrics can be valid when the server, switches, drivers, collective libraries, and operations team support them. Choose by measured application behavior and the complete topology:
| Decision factor | Questions to answer |
|---|---|
| Workload communication | How much time is spent in all-reduce or other collectives, and what message sizes dominate? |
| Scale | How many nodes must communicate today, and how many will be added? |
| Topology | Are GPU, NIC, and switch paths balanced, or does traffic cross an oversubscribed uplink? |
| Software | Are the required drivers, collective libraries, congestion controls, and monitoring tools validated for the fabric? |
| Operations | Can the team troubleshoot and maintain the chosen fabric, including failure and firmware procedures? |
| Growth | Can the switch and cabling plan deliver the required rate when the cluster expands? |
Benchmark the real distributed job at the intended node count. A synthetic link test that excludes the data loader, checkpoint traffic, or concurrent users can hide the bottleneck that matters in production.
Free tools Windows power users keep installed
One-click scans. No signup required.
Separate storage capacity from storage performance
Calculate capacity from retained data
Capacity is the space required to keep data and recoverable state, not a conversion of network speed. At minimum, account for:
- Primary datasets and any uncompressed or transformed copies.
- Model weights, tokenizers, indexes, code, containers, and evaluation artifacts.
- Checkpoint files, including every retained version and any temporary file written during an atomic save.
- Local scratch space for shuffling, preprocessing, decompression, and failed-job cleanup.
- Replicas, backups, snapshots, and free space needed to keep the filesystem performant.
The correct capacity depends on retention and recovery policy. A training run that keeps one restorable checkpoint has a different requirement from a run that retains daily versions for several months. Do not label a fixed capacity as sufficient without knowing dataset size, checkpoint size, and retention count.
Estimate the read and write rate
Storage performance depends on data format, dataset size, cache fit, concurrency, and checkpoint behavior. Training often rereads data across epochs, while inference may repeatedly access a smaller hot set. If the active set fits in RAM or local NVMe, shared-storage demand can be much lower after warm-up; if it does not, the shared system must sustain the application’s ongoing read rate.
Measure both sequential throughput and small or random I/O. Record queue depth, latency, and tail latency with the expected number of workers and concurrent jobs. A storage system that reaches its headline GB/s with one large stream may still stall a data loader issuing many small requests.
Apply the 12.5-Gb/s-per-GPU planning reference
NVIDIA’s HGX AI Factory storage guidance uses approximately 12.5 Gb/s per GPU as an initial sizing guideline and illustrates approximately 200 Gb/s aggregate for a 16-GPU cluster. This is a reference-architecture example, not a universal minimum. The source is NVIDIA’s HGX AI Factory NVIDIA-Certified Storage guidance.
Rank #3
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
The arithmetic is straightforward: multiply the GPU count by 12.5 Gb/s, then validate the result against the workload. For example, eight GPUs produce an initial reference of 100 Gb/s, while 16 GPUs produce about 200 Gb/s. These figures describe a planning rate; they do not guarantee that a particular filesystem, network, or server will deliver it.
Convert units before comparing a storage quote with an application metric. Eight bits equal one byte, so 200 Gb/s is 25 GB/s before protocol, filesystem, and hardware overhead. Real application throughput will be lower and may be limited by latency or metadata operations rather than the byte-rate ceiling.
| GPU count | 12.5-Gb/s-per-GPU planning rate | Theoretical byte-rate equivalent before overhead |
|---|---|---|
| 4 | 50 Gb/s | 6.25 GB/s |
| 8 | 100 Gb/s | 12.5 GB/s |
| 16 | 200 Gb/s | 25 GB/s |
| 32 | 400 Gb/s | 50 GB/s |
The 4-, 8-, and 32-GPU rows are direct arithmetic extensions of the 12.5-Gb/s planning rate; they are not additional NVIDIA minimums. Use them to frame a benchmark target, then revise the design using measured read demand, cache hit rate, and checkpoint bursts.
Recommended Free Tools
Design a local-cache and shared-storage tier
Local NVMe for staging and cache
Local NVMe can stage a dataset close to the GPUs, absorb repeated reads, and provide scratch space for preprocessing. NVIDIA’s certified-system guidance describes one local NVMe device per CPU socket and a 1-TB minimum in the configurations it covers. Those are configuration baselines, not proof that 1 TB is adequate for your dataset or that one device is sufficient for every server.
Size local NVMe from the portion of data you intend to stage, the number of simultaneous jobs, and temporary-file growth. Verify endurance, sustained-write behavior, thermal limits, and failure replacement procedures. A cache that silently evicts hot data or throttles during a checkpoint can move the bottleneck back to shared storage.
Shared storage for common data
Shared high-performance storage lets multiple nodes consume the same dataset and provides a central place for checkpoints and artifacts. Its read and write performance must match the aggregate demand of all nodes, not just one server. Include metadata performance when thousands of workers open many files, and test behavior while checkpoints are being written.
NVIDIA’s DGX B300 SuperPOD storage architecture recommends characterizing workloads and organizational needs before finalizing performance and capacity. Treat that advice as a design gate: benchmark representative files, formats, worker counts, and checkpoint operations before committing to a storage system.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallAccount for checkpoint pauses and recovery
Checkpoint files can reach terabytes in large systems. A synchronous checkpoint write can stop forward progress until the write completes, so average storage throughput is not enough. Calculate the write window your job can tolerate:
- Measure the checkpoint size produced by the actual framework and optimizer state.
- Multiply by the number of simultaneous writers or replicas.
- Divide by the sustained end-to-end write rate measured through the production filesystem and network.
- Compare the resulting pause with the job’s target interval and recovery objective.
- Test a full checkpoint while other jobs read data; record throughput, latency, and GPU idle time.
If the pause is unacceptable, evaluate asynchronous or staged checkpointing, local NVMe as a temporary landing tier, fewer retained copies, or a storage system with higher sustained write and metadata performance. Do not assume that changing only the NIC speed removes a checkpoint stall.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Worked sizing examples
Single 8-GPU fine-tuning server
Start by mapping the eight GPUs, CPU sockets, NIC, and NVMe devices. If the job remains on one node, optimize the NVLink and PCIe topology and size the storage path for the dataset and checkpoints. The 12.5-Gb/s reference gives 100 Gb/s, or 12.5 GB/s theoretical, as a storage-bandwidth planning point. Whether that rate is necessary depends on cache fit and data-loader measurements. A local NVMe tier can stage the active dataset, while shared storage holds the source data and checkpoints.
Rank #4
- GROUNDBREAKING READ/WRITE SPEEDS: The 990 EVO Plus features the latest NAND memory, boosting sequential read/write speeds up to 7,250/6,300MB/s. Ideal for huge file transfers and finishing tasks faster than ever.
- LARGE STORAGE CAPACITY: Harness the full power of your drive with Intelligent TurboWrite2.0's enhanced large-file performance—now available in a 4TB capacity.
- EXCEPTIONAL THERMAL CONTROL: Keep your cool as you work—or play—without worrying about overheating or battery life. The efficiency-boosting nickel-coated controller allows the 990 EVO Plus to utilize less power while achieving similar performance.
- OPTIMIZED PERFORMANCE: Optimized to support the latest technology for SSDs—990 EVO Plus is compatible with PCIe 4.0 x4 and PCIe 5.0 x2. This means you get more bandwidth and higher data processing and performance.
- NEVER MISS AN UPDATE: Your 990 EVO Plus SSD performs like new with the always up-to-date Magician Software. Stay up to speed with the latest firmware updates, extra encryption, and continual monitoring of your drive health–it works like a charm.
Two-node distributed training system
For two nodes, document the GPU-to-NIC affinity and the path through the first switch. Use the certified-system guidance’s 200-Gbps multi-node inference figure only as a starting configuration reference; training may need a different balance of ports, latency, and redundancy. Keep storage traffic on its designed path, then benchmark all-reduce together with data loading and a checkpoint write.
Sixteen-GPU cluster with shared datasets
The HGX storage example points to approximately 200 Gb/s aggregate for 16 GPUs, equivalent to 25 GB/s before overhead. Confirm whether the workload rereads the dataset every epoch, whether local cache can hold the active set, and how many jobs share the filesystem. Size capacity separately for model artifacts, checkpoint retention, and scratch space; the 200-Gb/s figure says nothing about required terabytes.
Procurement and validation checklist
- Freeze the topology: obtain the server’s GPU, NVLink, PCIe, NIC, and NVMe map.
- Define the workload: capture data format, worker count, concurrent jobs, collective pattern, and checkpoint schedule.
- Set planning targets: use the 12.5-Gb/s-per-GPU storage reference and the applicable certified-system network guidance as starting points, with their qualifications attached.
- Build separate paths: specify scale-up, scale-out, storage/customer, and management connectivity independently.
- Check balance and redundancy: verify PCIe placement, switch uplinks, alternate paths, and performance after one failure.
- Benchmark end to end: test representative training or inference, cache-cold and cache-warm runs, concurrent jobs, and synchronous checkpoint writes.
- Record acceptance metrics: sustained read/write throughput, latency and tail latency, GPU utilization, checkpoint pause, restart time, and capacity headroom.
- Revisit growth: confirm that additional nodes, datasets, and retained checkpoints will not exceed switch ports, storage controllers, or local cache.
Common sizing mistakes and fixes
Mistake: equating GPU count with NIC count
Fix: derive port speed and count from distributed traffic, topology, and utilization. The 400-Gbps-per-GPU figure is a configuration recommendation, not an automatic one-port-per-GPU rule.
Mistake: treating storage bandwidth as compute-fabric bandwidth
Fix: draw separate paths for NVLink, cluster collectives, storage, customer access, and management. Validate each path under its own failure and contention conditions.
Mistake: buying capacity without a checkpoint policy
Fix: calculate checkpoint size, interval, retention, temporary space, and backup copies before selecting usable terabytes.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesMistake: converting Gb/s and GB/s incorrectly
Fix: divide gigabits per second by eight for the theoretical byte-rate equivalent, then subtract protocol and system overhead when setting an application target.
Mistake: benchmarking an empty cache with one job only
Fix: test cache-cold and cache-warm operation, realistic worker counts, multiple concurrent jobs, metadata-heavy access, and checkpoint writes.
Mistake: ignoring placement and oversubscription
Fix: inspect PCIe affinity and switch uplink ratios. A fast NIC or SSD connected through a congested root complex or uplink cannot deliver its label rate to every GPU.
What a complete design decision should contain
Your final bill of materials should show, for every domain, the required rate, measured rate, topology, and failure behavior. Include the GPU and NVLink map; NIC speed, count, and CPU/PCIe affinity; switch ports and oversubscription; storage protocol and usable throughput; local NVMe capacity and endurance; shared capacity and retention; and the benchmark conditions that produced each number.
Use the NVIDIA Inference Reference Architecture when the target is an inference service, but do not transfer its configuration assumptions to a training cluster without validating workload differences. The same principle applies to every published figure: it is a starting point tied to a specific architecture, not a universal minimum.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
