iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To reduce storage I/O in local LLM inference, first make the active model weights and KV cache fit in RAM or VRAM. Use a smaller or lower-bit checkpoint, shorten context or reduce concurrency, then verify whether disk reads continue during steady-state generation. Only after measuring a storage-bound workload should you move model files to a dedicated PCIe NVMe SSD or change llama.cpp loading modes such as mmap, mlock, mmap+mlock or DirectIO. A faster drive mainly improves startup and page-in latency; it cannot overcome compute or memory-bandwidth limits once the data is resident.
What storage I/O means during local inference
A local model is normally loaded from a file into the operating system’s virtual memory and, where applicable, GPU memory. The file may be read quickly during startup and then remain effectively idle while tokens are generated. That pattern is healthy. A different pattern occurs when the working set does not fit: pages containing weights or the KV cache are evicted, and the runtime must fetch them from storage again. Generation then becomes partly a storage-streaming workload.
llama.cpp memory-maps model data by default. Memory mapping does not guarantee that every page is in physical RAM. If available RAM is insufficient, the operating system brings pages in as they are touched and may evict them before they are needed again. An engineering discussion for llama.cpp describes this repeated loading behavior when physical memory cannot hold the mapped file. The visible symptoms are high read throughput, page faults, swap activity, long pauses, or a sharp fall in tokens per second.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →The first question is therefore not whether an SSD is fast. It is whether storage is on the critical path after loading. Separate load-time I/O from generation-time I/O before changing hardware or runtime settings.
#1 Best Overall
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
Diagnose the bottleneck before changing settings
Run the same model, prompt, context length, quantization, GPU offload settings and concurrency for every comparison. Record the following phases independently:
| Measurement | What it tells you | How to interpret it |
|---|---|---|
| Model-load duration | Time from starting the server until the model can accept work | A large one-time read burst points to startup latency, not necessarily slow generation. |
| Prompt-processing throughput | How quickly the runtime evaluates the input prompt | Low throughput can involve compute, memory bandwidth, context size or page faults. |
| Generation tokens per second | Steady-state decode speed after the prompt | Compare only after the model has reached a stable state; this is the key metric for interactive use. |
| RAM and VRAM usage | Whether weights and KV cache have room to remain resident | Usage near the physical limit makes eviction and swap more likely. |
| Disk read throughput and queueing | Whether storage is serving pages while tokens are being generated | Sustained reads during decode are evidence that the working set is not resident. |
| Swap-in activity and page faults | Whether the operating system is reclaiming memory and restoring it from storage | Correlation with pauses and RAM exhaustion indicates memory pressure, not a simple drive-speed problem. |
Recognize a startup-only read burst
Large reads while the server opens a model are expected. A faster storage device can shorten this phase, especially for large checkpoints or repeated restarts. If reads fall to near-idle after loading and tokens per second are stable, changing loading mode or buying a faster SSD is unlikely to produce a meaningful generation-speed increase.
Recognize storage-bound generation
If disk reads remain high while decoding, and those reads coincide with swap-ins, page faults or stalls, the model is not staying resident. This is the case where reducing the working set, preventing eviction or improving storage latency can help. Measure for long enough to exclude the brief page-in activity that occurs immediately after a request begins.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Make the active working set fit
The most reliable fix is to reduce the bytes that must remain available. The working set includes model weights plus the KV cache for the active context and every concurrent sequence. A model that fits by itself may still overflow memory when the context grows or several requests run together.
Rank #2
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
Choose a smaller checkpoint
Moving from a larger parameter count to a smaller one reduces both the model file and the resident weight requirement. This can be a better trade than trying to stream a large model from disk, because repeated page-ins add latency to every generation step.
Use lower-bit quantization
Quantization stores weights with fewer bits, reducing the bytes loaded and the memory required at runtime. It can make a previously marginal model fit in RAM or VRAM and thereby remove storage traffic. The quality and speed trade-off depends on the checkpoint, quantization method and hardware, so compare output quality and tokens per second with the same workload rather than assuming that the lowest-bit option is best.
Understand the scale of full-precision models
Hugging Face documentation gives a useful reference point: loading a 70-billion-parameter Llama 2 model requires 256 GB of memory for full-precision weights and 128 GB for half-precision weights. Those figures describe weights alone and do not include the KV cache, runtime overhead or other applications. Quantization and optimized serving are therefore central techniques for fitting such models on local systems.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallReduce context length when KV cache is the problem
The KV cache grows with the active context and with the number of simultaneous sequences. If weights fit but memory fills during long prompts or multi-user serving, lower the context limit or concurrency and measure again. A smaller cache can prevent the operating system from evicting mapped weight pages.
Rank #3
- BREAKTHROUGH PCIe 5.0 PERFORMANCE: Supercharge your workflow and gaming with PCIe 5.0, boasting up to 14,700/13,400 MB/s¹ sequential read/write speeds. Tackle massive files and power up your gaming with Gen5—twice as fast as the 990 PRO SSD.
- EVERY TASK, TURBOCHARGED: Speed past productivity limits. With random read/write speeds up to 1,850K/2,600K IOPS¹, enjoy fast game loads, seamless AI apps, and efficient multitasking. Virtually no lag, no limits—just nonstop performance.
- THINK FAST, CREATE FASTER: With random read/write speeds of up to 1,850K/2,600K IOPS¹, the 9100 PRO SSD fuels seamless AI content creation, swift loads, and smooth gameplay. Work, play, and create at lightning speed.
- SPEED, WHENEVER YOU NEED: From laptops, desktop PCs, to PS5 with added heatsink devices², experience blazing PCIe 5.0 speeds and up to 8TB of storage. Perfect for video editing, gaming, and creative tasks, with the compatibility to match your device.
- STAY COOL, RUN FAST: Push limits, not temperatures. A 5nm controller boosts power efficiency up to 49% over the 990 PRO SSD³, while advanced thermal control keeps performance smooth and reliable.
Lower concurrency before buying storage
Each concurrent request adds working data. If one request runs normally but several cause swap-in activity, reduce the batch or concurrency limit first. Faster storage can reduce the time spent waiting for pages, but it does not stop thrashing caused by an oversized aggregate working set.
How llama.cpp loading modes change I/O
The official llama.cpp server documentation lists several loading controls. They change residency and read behavior; none is universally fastest. Test them with the same model and workload.
| Mode | What it does | When to test it | Main risk or limitation |
|---|---|---|---|
mmap |
Maps model data into virtual memory and lets the operating system page data as needed. | Use as the baseline because it is the normal default path. | If the working set exceeds RAM, pages can be repeatedly read from storage. |
mlock |
Asks the operating system to keep the model’s memory resident. | Test when you have ample RAM and eviction is the observed problem. | It consumes physical memory that other applications may need; operating-system limits can prevent the request from succeeding fully. |
mmap+mlock |
Combines memory mapping with a request to keep mapped pages resident. | Test when you want the mmap loading path but need stronger residency. | It still cannot create memory that the machine does not have, and locking too much can force other processes into pressure or swap. |
DirectIO (dio) |
Uses direct I/O where the platform and filesystem support it, reducing reliance on the normal page cache. | Benchmark on a system where page-cache behavior is suspected and support is confirmed. | Availability and performance depend on the operating system, filesystem and device; it is not a universal speed setting. |
| Lazy loading | With mmap enabled, reads tensor rows from disk on demand rather than making the full model resident immediately. | Use when lowering resident memory is more valuable than avoiding repeated reads. | If the working set does not fit, on-demand reads can recur and increase generation latency. |
Why mmap alone does not stop paging
Memory mapping describes how the process addresses the file, not where every byte physically resides. The operating system can keep hot pages in RAM and fetch cold pages later. This is efficient when the working set fits and the page cache is useful. It becomes expensive when the access pattern repeatedly touches more pages than physical memory can hold.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsWhen mlock helps
Use mlock or mmap+mlock only after confirming that eviction is hurting generation and that the machine has enough spare RAM. Locking a model while the KV cache, desktop, filesystem cache and other services also need memory can make the overall system less stable. Check whether the runtime reports a partial lock or an operating-system limit rather than assuming every page was pinned.
Rank #4
- HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
- BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
- SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
- THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
- SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO
When DirectIO is worth a controlled test
DirectIO can change how reads bypass or interact with the page cache, but support is platform-specific. Treat it as an experiment, not a default optimization. Keep it only if the measured load time or steady-state behavior improves without increasing stalls or operational complexity.
A practical tuning procedure
- Establish a baseline. Start with the default mmap path. Record model-load time, prompt-processing rate, generation tokens per second, RAM and VRAM use, disk reads and swap activity. Use a fixed prompt and identical runtime settings.
- Classify the symptom. Decide whether reads occur only during startup or continue during steady-state decode. A startup-only problem calls for storage and loading-path work; sustained reads call for a smaller working set or stronger residency.
- Fit weights and cache. Try a smaller checkpoint or lower-bit quantization. If memory pressure appears only with long prompts, reduce context length. If it appears only under load, reduce concurrency. Re-test before changing the drive.
- Test residency controls. With sufficient spare RAM, compare mmap with
mlockandmmap+mlock. Confirm actual resident memory and watch for swap or system-wide pressure. Do not infer success from a setting being accepted; verify that disk reads during generation fall. - Evaluate lazy loading deliberately. If a full resident model is impossible, lazy loading may lower the initial memory requirement. Measure repeated reads and token latency, because on-demand tensor-row reads can be costly when the access pattern exceeds available memory.
- Benchmark DirectIO only when supported. Test it on the same filesystem and device, using repeated runs so that cache warmth does not decide the result. Compare startup and generation separately.
- Move files to appropriate storage. If measurements show storage wait, place model files on a dedicated PCIe NVMe SSD away from the operating system, swap and unrelated workloads. Repeat the original test and keep the change only if the bottleneck metric improves.
- Document the winning configuration. Record model, quantization, context, concurrency, loading mode, storage device and measured results. This prevents a future update or larger prompt from silently recreating the bottleneck.
Choosing storage for local model files
Dedicated PCIe NVMe storage
A dedicated PCIe NVMe SSD is sensible when startup reads are long or measurements show that page-ins are storage-bound. Separating model files from the operating system and swap reduces contention and makes results more predictable. Capacity should cover the checkpoints you actually keep; buying speed without enough RAM does not solve paging.
Why faster storage does not guarantee faster generation
Once weights and KV cache are resident, token generation is governed mainly by compute and memory bandwidth. An SSD is not in that hot path. It can reduce the time to load a model or recover an evicted page, but it cannot raise tokens per second when no storage wait exists.
Recommended Free Tools
RAID0 NVMe
Striping multiple NVMe drives can increase aggregate throughput in an advanced, benchmarked setup. It also increases failure exposure: one failed member can make the striped volume unavailable. RAID0 is not a substitute for enough RAM or VRAM, and it is difficult to justify if the workload is compute-bound or reads only during startup.
Best Value
- SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
- CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
- IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
- UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
- KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]
Runtime choice and checkpoint format
NVIDIA describes llama.cpp as a lightweight local inference backend using GGUF or GGML checkpoints. It positions vLLM and TensorRT for different serving and optimization profiles. The choice of runtime affects loading path, memory management, batching and hardware support, so compare systems using the same outcome metrics rather than assuming that one backend eliminates storage I/O.
Checkpoint format and conversion path also affect startup. A ServerlessLLM study measured approximately 18.5 seconds to load a large LLaMA-2-70B model, compared with more than 80 seconds for Safetensors and nearly 170 seconds for PyTorch in that research system. Those are system-specific measurements, not consumer desktop guarantees. They show why format and loading implementation can dominate startup without proving a universal generation-speed difference.
Troubleshooting common symptoms
| Symptom | Likely cause | Action |
|---|---|---|
| Fast initial generation, then long pauses as the prompt grows | KV cache growth pushes weights or other pages out of RAM. | Reduce context length or concurrency, use a smaller or more heavily quantized model, and verify whether swap-ins decline. |
| High disk reads from the first token onward | Weights are not resident, or lazy loading is fetching rows on demand. | Check available RAM, compare mmap with a residency-tested mode, and fit the working set before upgrading storage. |
| SSD is busy during startup but idle during decode | Normal model loading and page-in behavior. | Use faster dedicated storage only if startup latency matters; do not expect a tokens-per-second gain. |
| Enabling mlock makes the whole system sluggish | Too much memory was reserved, leaving insufficient RAM for the OS, cache or other processes. | Undo the lock, reduce model size or concurrency, or add memory before trying again. |
| DirectIO is unavailable or slower | The platform or filesystem does not support it efficiently, or the workload benefits from the page cache. | Return to mmap, confirm support, and compare repeated runs under identical conditions. |
| A faster SSD changes load time but not tokens per second | Compute or memory bandwidth, rather than storage, limits steady-state decode. | Keep the SSD change only for startup or recovery latency; investigate model size, offload and runtime settings for generation speed. |
| RAID0 improves sequential reads but inference still stalls | Memory pressure is causing random page-ins, or the bottleneck is elsewhere. | Reduce the working set and remove swap thrashing; do not treat throughput numbers as proof that the access pattern improved. |
How to judge an improvement
Use separate acceptance criteria for startup and generation. A storage change is successful for startup if repeated cold or controlled-load runs become shorter without introducing reliability problems. A loading-mode change is successful for generation only if sustained disk reads, page faults or swap-ins fall and tokens per second improve under the same prompt and context.
Do not compare a warmed run with a cold run, or a short prompt with a long-context request. Keep the model version, quantization, context, concurrency, offload settings and background workload constant. If the only improvement is a shorter one-time load, report it as a startup benefit rather than claiming that inference itself became faster.
Bottom line
Storage I/O is usually a symptom of a working set that does not fit, not the primary performance limit of a resident model. Fit weights and KV cache in RAM or VRAM first through model selection, quantization, context and concurrency changes. Then test llama.cpp’s mmap, mlock, mmap+mlock, lazy-loading and supported DirectIO paths with measurements. A dedicated PCIe NVMe SSD is a targeted fix for proven storage-bound loading or paging; it is not a replacement for sufficient memory and will not raise generation speed when compute or memory bandwidth is already the limit.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
