Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
There is no single storage-bandwidth or latency requirement for AI. Size storage to keep accelerators supplied with the bytes your workload actually reads, complete checkpoint saves and restores within your recovery objective, and meet the latency or throughput target of the serving system. The right figure depends on dataset size and modality, cache hit rate, concurrent jobs, I/O pattern, network topology, and compute configuration.
A useful design starts with measurements—bytes read per step, checkpoint size, request concurrency and deadlines—then validates the complete path under load. Published figures such as NVIDIA’s DGX B300 SuperPOD targets are reference-architecture examples, not universal minimums.
Why AI storage requirements vary so widely
Two systems can use the same GPUs and have radically different storage needs. A language-model fine-tuning run whose training set remains in local memory or NVMe cache may spend little time waiting for shared storage. A multimodal run that repeatedly streams images, video or large feature files from a remote filesystem can be I/O-bound even with identical accelerators.
The important variables are:
- Data volume per step or sample: Include raw records, decoded media, features and preprocessing reads.
- Target step or sample rate: Faster training raises the required byte rate.
- Cache behavior: A warm memory or local-NVMe cache can hide remote latency; first-epoch and cache-miss behavior can dominate startup and recovery.
- Concurrency: Several jobs may read datasets while another saves a checkpoint and a serving cluster loads a model.
- I/O shape: Large sequential transfers stress bandwidth; many small files and random reads stress IOPS, metadata operations and tail latency.
- Network and placement: Storage bandwidth is useful only if the host links, fabric, CPU pipeline and filesystem clients can deliver it to the accelerators.
NVIDIA states in its DGX B300 SuperPOD storage architecture: “DGX SuperPOD is designed to support all workloads, but the storage performance required to maximize training performance can vary depending on the type of model and dataset.” That qualification is the correct starting point for any design.
#1 Best Overall
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
Training workloads: reads, caches and checkpoints
Repeated reads can matter more than a peak benchmark
Deep-learning training usually iterates over data, so the system may read the same corpus many times. A sequential read test that looks impressive in isolation does not prove that a dataloader can sustain the required rate when workers perform shuffling, decoding, augmentation and metadata lookups. Measure the complete input pipeline and distinguish warm-cache from uncached behavior.
Some recommender workloads converge after a fraction of an epoch, while other jobs repeatedly traverse very large corpora. In the first case, startup and random-access behavior may dominate; in the second, sustained read bandwidth and cache capacity are central. NVIDIA describes a “standard” class in which data generally fits in local compute-memory cache and I/O is less likely to limit training, and an “enhanced” class in which multimodal data exceeds that cache and I/O affects end-to-end time.
Checkpoint writes are a separate performance budget
Checkpoints protect against failures and allow a run to resume, but saving can pause or throttle computation. Restore time determines how quickly a failed job returns to useful work. Size both directions: checkpoint write bandwidth for the save window and read bandwidth for the recovery window. Include serialization, many parallel writers, metadata operations, network contention and any replication or object-store staging.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Local cache is not shared storage
Local NVMe can stage a dataset or cache hot shards close to each node. It reduces repeated remote reads, but capacity and contents are tied to individual machines. A distributed job still needs a shared, durable source for datasets and checkpoints, and a cache rebuild after node replacement can create a large burst against that source.
Rank #2
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
Published reference numbers—and how to interpret them
NVIDIA’s current DGX B300 SuperPOD reference architecture publishes the following aggregate targets. “SU” is the system unit used in that architecture. The values are vendor design targets for the stated configuration, not independent measurements or minimums for every AI cluster.
| Workload class | System size | Aggregate read | Aggregate write |
|---|---|---|---|
| Standard | Single SU | 80 GB/s | 40 GB/s |
| Enhanced | Single SU | 250 GB/s | 124 GB/s |
| Standard | Eight SU | 640 GB/s | 320 GB/s |
| Enhanced | Eight SU | 2,000 GB/s | 992 GB/s |
NVIDIA characterizes standard workloads as multiple concurrent large-language-model or fine-tuning jobs with periodic checkpoints where compute generally dominates I/O. Its enhanced class covers concurrent multimodal jobs whose data does not fit local cache. Do not scale these figures directly to a different GPU count, filesystem, topology or model without testing; aggregate numbers can conceal a per-node bottleneck.
Google Cloud gives a different, workload-scoped example. Its TPU VM guidance describes Managed Lustre for synchronous checkpoints and high-speed reinforcement-learning weight propagation where latency below 1 ms is relevant, and Cloud Storage Rapid Bucket for high-throughput asynchronous and multi-tier checkpointing. Those recommendations apply to the specified Google Cloud services and workflows, not as guarantees for another platform.
Calculate a first-pass requirement from your workload
Use measured workload quantities instead of choosing a drive from a headline interface speed. The following sequence produces a planning target that you can validate with a representative load.
Rank #3
- SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
- CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
- IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
- UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
- KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]
- Measure data consumed per unit of work. Record bytes read per training step, sample or batch, including preprocessing and metadata traffic. If a batch reads 2 GB and the job must complete 20 batches per second, the ideal input rate is 40 GB/s before overhead.
- Account for workers and concurrency. Sum simultaneous jobs and dataloader processes. Add model loads, validation passes and cache-warming bursts rather than sizing only for one isolated run.
- Separate cache states. Test a cold start, the first uncached epoch, a warm cache and a cache eviction or node-replacement event. Report where each byte is served from: accelerator memory, host RAM, local NVMe or remote storage.
- Map the path to the network. Check per-node link capacity, oversubscription, filesystem client limits and the bandwidth available to the GPUs after CPU decoding and encryption overhead.
- Set a utilization margin. Leave headroom for concurrent checkpoints, metadata bursts and tail latency. A measured average equal to the theoretical requirement leaves no room for another job or a cache miss.
- Load-test end to end. Run the real framework, data format, worker count and augmentation pipeline. Monitor accelerator utilization, dataloader wait time, storage throughput, IOPS, metadata latency, queue depth and network retransmits.
This calculation is a planning estimate, not a vendor promise. The acceptance criterion is that accelerators remain adequately fed while the system meets checkpoint and serving objectives under the intended concurrency.
Checkpoint sizing: turn recovery goals into bandwidth
For a checkpoint of size S that must be saved in T seconds, the first-order write target is S ÷ T. A 4 TB checkpoint with a 120-second save window implies about 34.1 GB/s of sustained application-visible throughput (4,000 GB divided by 120 seconds), before accounting for serialization, protocol overhead, replication and contention. Restore planning uses the same calculation with the allowed recovery window.
Real systems need more than that quotient:
- Parallelism: Multiple ranks may write separate shards. Confirm that the filesystem and network scale with the number of writers instead of bottlenecking at a single metadata server or gateway.
- Synchronization: A coordinated checkpoint can create a sharp, cluster-wide burst. Storage or network limits may extend the pause even when average daily bandwidth looks adequate.
- Hierarchy: A local or regional staging tier can distribute a checkpoint before copying it to durable object storage. AWS describes hierarchical distribution patterns and services such as S3 Express One Zone and FSx for Lustre for large-scale checkpoint workflows; those are AWS architecture examples, not cross-platform guarantees.
- Retention and restart: Count concurrent saves, reads during recovery, replication and the number of retained versions. A design that writes quickly but restores slowly still misses a recovery objective.
Inference has two distinct storage profiles
Online serving
Online generation is governed by user-facing latency and concurrency. Specify time to first token, time per output token, requests per second, maximum concurrent sessions and the model-load or scale-out time. Storage must deliver model weights and any retrieval or feature data without creating long-tail pauses. Once weights are resident, compute and memory may dominate token generation; when models are frequently loaded, evicted or sharded, storage and network behavior return to the critical path.
Offline and batch inference
Offline scoring can use larger batches and optimize for total completion time. State the batch size, input volume, output volume and deadline. Sequential throughput may matter more than interactive latency, but small files, metadata and random feature lookups can still limit the job. A benchmark result is meaningful only with its model, hardware, scenario, accuracy target, batch or concurrency level and latency constraint attached.
Rank #4
- HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
- BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
- SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
- THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
- SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO
Model and feature-data placement
Keep frequently used weights and feature shards in the fastest practical tier, while placing durable archives and asynchronous outputs in a cheaper tier when the application permits. Validate the time to populate a new worker: a cache that takes longer to warm than the serving SLO is not an effective scale-out strategy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Compare storage options by behavior, not labels
| Evaluation axis | Questions to answer |
|---|---|
| Sustained bandwidth | What read and write rate is available per node and across the cluster during the intended concurrency? |
| Latency and tail latency | What are median, p95 and p99 times for model loads, feature reads and checkpoint operations? |
| IOPS and random I/O | Can the system handle small, parallel reads instead of only large sequential transfers? |
| Metadata concurrency | How many file opens, stats, creates and directory operations can it sustain? |
| Scale-out | Does performance remain predictable as clients, jobs and nodes increase? |
| Cache and staging | Can hot data be staged locally, and how does the system recover after eviction or node loss? |
| Checkpoint behavior | What save and restore times occur while training reads and other jobs are active? |
| Operations and network | What are the placement, bandwidth, failure-domain, security and management constraints? |
Local NVMe
Local NVMe is valuable for per-node dataset caching, temporary shards and staging. It offers low access latency without crossing the network, but it is limited by node capacity and does not by itself provide a shared namespace or durable, cluster-wide copy.
Shared parallel filesystems
A shared parallel filesystem is suited to high-throughput, low-latency, concurrent access when many workers must read the same corpus or write checkpoint shards. Evaluate metadata scaling and client behavior as carefully as headline bandwidth.
Recommended Free Tools
Object storage
Object storage is often attractive for durable datasets and asynchronous or multi-tier checkpoints. It can have different request and latency characteristics from a POSIX filesystem, so applications may need staging, multipart transfers or a cache. Google Cloud’s Rapid Bucket example illustrates this asynchronous, high-throughput role within its environment.
Best Value
- This product has been replaced by our latest generation. Please search for the SANDISK Optimus GX 7100 NVMe SSD
- HIGH-OCTANE GAMING. Experience speeds up to 7,250MB/s read and 6,900MB/s write (1-2TB models), with up to 35% faster performance than previous generation.
- PURPOSE-BUILT. Designed for serious on-the-go gamers, with a PCIe Gen4 interface and SANDISK’s next generation TLC 3D NAND.
- MORE TIME TO CLEAR THAT CHECKPOINT. Built with laptops and handheld gaming devices in mind, with up to 100% more power efficiency over the previous generation.
- DO MORE WITH DASHBOARD. Ensure your drive is optimized for prime performance with the downloadable WD_BLACK Dashboard (Windows only).
Validate the design and diagnose bottlenecks
Benchmarking should reproduce the workload rather than rely on a synthetic maximum. Use representative file sizes, sharding, compression, worker counts, checkpoint cadence and concurrent jobs. Record both aggregate and per-node values.
- Accelerators idle while storage queues rise: Increase effective read bandwidth, improve caching or reduce preprocessing and small-file overhead.
- Average throughput is high but steps stall: Inspect p95/p99 latency, metadata operations, lock contention and straggling workers.
- Only the first epoch is slow: Measure cold-cache behavior and decide whether pre-staging, larger host memory or local NVMe is justified.
- Checkpoint pauses exceed the target: Measure serialized output, writer parallelism, network contention and destination limits; test a staged or hierarchical path.
- Adding nodes does not improve throughput: Check per-node links, fabric oversubscription, filesystem scaling limits and a shared metadata bottleneck.
- Inference tail latency spikes during scale-out: Separate model-load traffic from request traffic and verify that cache warming does not compete with live reads.
Google’s AI/ML performance guidance recommends testing storage read/write performance and latency while tuning network bandwidth and placement. Treat accelerator utilization and application completion time as the final measures of success, not storage throughput alone.
A practical sizing checklist
- Define the training step rate or inference deadline.
- Measure bytes read per step, sample or request, including preprocessing and metadata.
- List cache tiers and test cold, warm and cache-miss cases.
- Record checkpoint size, save interval, restore window and concurrent writers.
- State online serving latency targets or offline batch and completion targets.
- Calculate per-node and aggregate bandwidth, then check IOPS, metadata and tail latency.
- Model simultaneous dataset reads, model loads, checkpoints and recovery traffic.
- Verify network paths, placement and failure-domain behavior.
- Load-test with the real framework and monitor accelerator starvation.
- Revisit the numbers when model size, modality, GPU count, concurrency or retention policy changes.
Bottom line
Choose storage from workload measurements and service objectives, not from a universal “AI bandwidth” number. Cache-resident training may need modest remote throughput, while cache-missing multimodal training can require hundreds of gigabytes per second at cluster scale. Checkpoint deadlines and inference tail latency create separate constraints. Use published architecture figures as context, calculate your own per-node and aggregate targets, and confirm them with an end-to-end test under realistic concurrency.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
