iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Short answer: an SSD stores local AI models and software; system RAM is the computer’s general working memory; and discrete GPU VRAM is the GPU’s high-speed workspace for model weights, the KV cache and other inference data. A larger SSD does not replace RAM or VRAM, and a machine may have enough storage for a model that still cannot load it into memory. System RAM can sometimes extend what runs through CPU inference or supported GPU offload, but it is not interchangeable with dedicated VRAM.
For a reliable fit, size the complete workload: model representation (such as FP16 or a quantized format), context length, batch or concurrent users, runtime backend, operating system and memory already consumed by other applications. The same model can fit in one setup and fail in another.
The three resources at a glance
| Resource | What it does in a local-AI computer | What determines how much you need | What it cannot do |
|---|---|---|---|
| SSD | Stores model files, applications, container images, caches and different model versions when they are not actively loaded. | Number and size of models, software/container footprint, checkpoints, logs and free space needed for updates. | It does not become inference memory merely because it is fast. Once active data is in RAM or VRAM, SSD speed is not established as a universal tokens-per-second improvement. |
| System RAM | Holds the operating system and applications, supports CPU inference, and can be used alongside GPU memory when a backend supports offload or fallback. | Model data assigned to the CPU, operating-system overhead, context and batch size, and other running programs. | It does not provide the same local bandwidth or software behavior as discrete VRAM. A backend may refuse to spill into RAM unless a specific feature is enabled. |
| Discrete GPU VRAM | Provides the GPU’s local working memory for weights, KV cache, activations and other execution allocations. | Model representation, number of layers offloaded, context length, batch/concurrency, KV-cache format and runtime overhead. | It is not permanent storage. Turning off the computer removes its contents, and parameter count alone cannot predict a guaranteed fit. |
| Unified memory | On some platforms, CPU and GPU access a shared address space or memory pool. | Platform architecture, total pool size, bandwidth, operating-system reservations and application use. | It should not be generalized to ordinary discrete-GPU PCs, where system RAM and VRAM remain separate pools. |
NVIDIA’s local-AI overview describes a range of GPU and memory architectures, but the practical rule is consistent: distinguish where data is stored from where it is actively processed.
What the SSD contributes
Persistent model and software storage
An SSD is the durable library for local AI. Downloaded model weights, tokenizer files, inference applications, Python environments, container images and generated outputs remain on it between sessions. A larger drive lets you keep several quantizations or model families without repeatedly deleting and downloading files.
#1 Best Overall
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
Storage capacity is separate from live memory capacity. A 100 GB model file can occupy 100 GB on an SSD while requiring a different amount of RAM or VRAM after it is decoded, mapped, split between devices or accompanied by runtime allocations. Compression and quantization can reduce the file and working-set sizes, but the exact result depends on the format and backend.
When SSD speed matters—and when it does not
A fast NVMe drive can shorten initial downloads, installation, model loading and swapping between models compared with slower storage. The reviewed sources do not establish a universal generation-speed gain after the active model is resident in memory. If generation is waiting on GPU compute, memory bandwidth or CPU work, replacing the SSD does not create additional VRAM.
Planning capacity
- Reserve space for the model files you actually intend to keep, not just the one currently loaded.
- Allow room for duplicate versions, quantized variants, tokenizer files, update downloads and logs.
- Count container and environment footprints separately from model weights.
- Keep meaningful free space so the operating system and applications can update without failing.
NVIDIA’s AI Workbench support matrix gives product-specific examples rather than a universal storage rule. One listed Windows Docker Desktop PyTorch container example uses around 41 GB in total; that is container and software storage, not a 41 GB VRAM requirement. See the AI Workbench Support Matrix for the configuration and platform assumptions.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What system RAM does
Operating-system and CPU workspace
System RAM is shared by the operating system, browser, development tools, databases and any CPU-side part of inference. A CPU-only runtime can keep model weights, KV cache and computations in RAM, so system memory can make a workload feasible even when no suitable GPU is present. The trade-off is that performance depends on the processor, memory bandwidth and implementation; the available sources do not provide a controlled, universal speed penalty for CPU execution.
RAM alongside a GPU
Many runtimes can keep some layers or other data in system RAM while placing the rest in VRAM. This is often called partial offload. It may allow a model that exceeds the GPU’s local capacity to start, but support varies by backend, driver, operating system and model format. Leave enough RAM for the operating system and other applications; allocating nearly all of it to inference can cause paging and instability.
Rank #2
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
Documented fallback in llama.cpp
The llama.cpp build and backend documentation describes a CUDA unified-memory option on Linux. Setting GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 permits swapping to system RAM instead of terminating when VRAM is exhausted. On Windows, the documentation points to the System Memory Fallback setting in NVIDIA Control Panel. These mechanisms are software- and hardware-dependent. They do not make RAM accesses equivalent to VRAM accesses, and another inference backend may have no corresponding feature. Consult the current llama.cpp build and backend documentation for the supported build and platform details.
What GPU VRAM does
Discrete VRAM is the GPU’s local memory budget. A GPU inference runtime may place model weights, intermediate activations, attention data, the KV cache and workspace allocations there. Keeping more of the active workload in VRAM generally gives the GPU the memory locality its software expects, but the exact fit depends on the model representation and runtime rather than parameter count alone.
Free tools Windows power users keep installed
One-click scans. No signup required.
Weights are only the beginning
Two installations of the same model can have different memory requirements. FP16 weights occupy more space than a quantized representation; a longer context creates a larger KV cache; a larger batch or more simultaneous requests increases it further; and different backends reserve different workspaces. Drivers, desktop compositors and other GPU applications also consume part of the advertised VRAM, leaving less for inference.
Before choosing a GPU, identify whether the runtime requires all layers in VRAM or supports a measured split between GPU and CPU memory. A card that barely fits the weight file may still fail when the requested context or batch is applied.
Unified memory is a different architecture
Some systems expose CPU and GPU memory through one address space or a shared pool. NVIDIA describes the GH200 platform as combining 96 GB of GPU memory with 480 GB of CPU LPDDR memory in one address space through NVLink-C2C. That architecture is not a specification for a normal desktop with a discrete graphics card: capacity, bandwidth and coherency behavior differ.
Rank #3
- SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
- CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
- IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
- UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
- KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]
NVIDIA’s current local-AI overview also lists systems it describes as having unified memory, including DGX Spark configurations up to 128 GB. Treat each platform’s stated capacity and software support as a product-specific fact. Do not add ordinary PC RAM and VRAM together and assume the result behaves like a unified high-bandwidth pool.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →How model memory is actually consumed
1. Model representation
Start with the representation that the runtime will load: FP16, another floating-point format or a particular quantization. The same parameter count can produce very different weight sizes. Use the format’s actual file and runtime documentation instead of a generic parameter-to-memory slogan.
2. KV cache
The KV cache stores attention keys and values for the conversation or sequence so the runtime can generate the next token without recomputing the entire history. NVIDIA states that KV-cache use grows with context length and batch size. A 128k-token single-user context can therefore need far more memory than a short chat, even with identical weights.
3. Runtime and platform allocations
Reserve memory for temporary tensors, attention workspaces, tokenizer and server overhead, drivers, the operating system and concurrent applications. Some runtimes use a different KV-cache format or workspace policy, so a number measured in one stack should not be transplanted to another as an exact requirement.
Concrete sizing examples from NVIDIA
The following figures are workload illustrations, not universal minimums. Keep the model, precision, context, batch and product configuration attached to each number.
Recommended Free Tools
Rank #4
- HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
- BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
- SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
- THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
- SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO
| Example | Published figure | How to interpret it |
|---|---|---|
| Llama 3 70B, FP16 weights | About 140 GB for weights | NVIDIA’s 2025 technical article uses this as a model-specific FP16 example. It is not the size of every quantized 70B file. |
| Llama 3 70B, one user at 128k context | About 40 GB KV cache | The same NVIDIA example shows why context memory can be substantial in addition to weights. |
| Llama 4 Scout 109B, FP16 | About 218 GB | A model-specific estimate in NVIDIA’s 2025 article, with the precision and configuration caveat intact. |
| Llama 8B in NVIDIA NIM 1.7.0 guidance | About 15 GB | A rough NIM sizing guideline; NVIDIA warns actual needs may be lower or higher depending on hardware and NIM configuration. |
| Llama 70B in NVIDIA NIM 1.7.0 guidance | About 131 GB | Another product-specific rough guideline, not a universal formula for all 70B runtimes or quantizations. |
Read the full context in NVIDIA’s article on large-scale LLM inference and KV-cache offload and the versioned NVIDIA NIM 1.7.0 getting-started guidance. The different figures are not contradictory: they describe different representations, products or workload assumptions.
A practical method for sizing a local-AI system
- Name the model and representation. Record the exact family, parameter size, precision or quantization, file format and runtime. Do not begin with a vague label such as ‘a 70B model.’
- Set the context and batch target. Write down the maximum prompt length, output length, number of simultaneous users and batch size. These values drive KV-cache and activation use.
- Choose the placement policy. Decide whether all layers must reside in VRAM, whether partial GPU offload is acceptable, or whether CPU inference is the fallback. Verify that the selected backend supports the policy.
- Calculate available, not advertised, memory. Subtract the operating system, desktop, drivers and other applications from system RAM and VRAM. On a shared-memory platform, include the platform’s reservations.
- Leave headroom. A model that starts with no margin can fail when a longer conversation, larger batch or second process appears. Treat a successful load test at your target context as more meaningful than a weight-file comparison.
- Plan the SSD separately. Add model files, alternate versions, containers, environments, caches and update space. A storage upgrade solves a capacity problem only when the bottleneck is stored data.
- Test the exact stack. Use the intended operating system, driver, backend and launch options. Record whether the model loads, how much VRAM and RAM remain, and whether the runtime silently falls back to CPU memory.
Comparing common system designs
| Design | Strength | Constraint to verify | Best question to ask |
|---|---|---|---|
| Large discrete GPU VRAM, moderate system RAM | More of the active model can remain in the GPU’s local memory. | The operating system, model loader and CPU-side allocations still need RAM. | Does the complete model plus KV cache fit in available VRAM at the target context? |
| Moderate VRAM, large system RAM | Partial offload or CPU inference can extend the set of runnable models. | Fallback must be supported and may change responsiveness; no universal speed ratio is established. | Which layers and caches will stay in RAM, and how does the backend report that placement? |
| Large SSD, limited RAM and VRAM | Many model files can be stored locally. | Storage does not make an oversized active workload runnable. | Is the current failure a disk-capacity error or a runtime out-of-memory error? |
| Unified-memory platform | CPU and GPU can address a shared pool according to the platform design. | Bandwidth, reservations and software support are architecture-specific. | What capacity and memory behavior does this exact platform document? |
| CPU-only system with ample RAM | No discrete GPU is required; models can run in system memory when the runtime supports the CPU. | Generation and throughput depend on CPU and memory subsystem; GPU-specific features are unavailable. | Does the chosen backend support the model format and desired context on the CPU? |
Common mistakes and how to correct them
Confusing a model download with its live footprint
Symptom: the file fits on the SSD but loading fails with an out-of-memory message. Fix: check the loaded representation, KV-cache target, workspace reservation and available RAM or VRAM. Storage size is only one input.
Adding RAM and VRAM as if they were interchangeable
Symptom: a specification sheet appears to show enough combined memory, yet the GPU loader rejects the model. Fix: determine whether the backend supports CPU offload or a unified-memory architecture. Otherwise, count each pool separately.
Using parameter count as a complete buying rule
Symptom: two systems with the same nominal model size behave differently. Fix: hold model representation, context, batch, backend and operating system constant before comparing memory capacity.
Ignoring context growth
Symptom: a short test succeeds, but a long document or multi-user server runs out of memory. Fix: test the maximum context and concurrency you will actually serve, because KV-cache use grows with both.
Best Value
- This product has been replaced by our latest generation. Please search for the SANDISK Optimus GX 7100 NVMe SSD
- HIGH-OCTANE GAMING. Experience speeds up to 7,250MB/s read and 6,900MB/s write (1-2TB models), with up to 35% faster performance than previous generation.
- PURPOSE-BUILT. Designed for serious on-the-go gamers, with a PCIe Gen4 interface and SANDISK’s next generation TLC 3D NAND.
- MORE TIME TO CLEAR THAT CHECKPOINT. Built with laptops and handheld gaming devices in mind, with up to 100% more power efficiency over the previous generation.
- DO MORE WITH DASHBOARD. Ensure your drive is optimized for prime performance with the downloadable WD_BLACK Dashboard (Windows only).
Assuming fallback is automatic
Symptom: the runtime crashes as soon as VRAM is full. Fix: check the backend’s documented offload or fallback switches. For llama.cpp, review the CUDA unified-memory option on Linux or the documented Windows System Memory Fallback setting; do not assume another application exposes the same controls.
What the NVIDIA AI Workbench figures mean
NVIDIA lists 16 GB as the minimum and 32 GB as the recommended system RAM for the full-local-install configurations shown for Windows and Ubuntu in its current support matrix. Those numbers apply to that product’s listed installation configuration, not to every local large-language-model setup.
The same matrix includes examples in which application and container storage reaches tens of gigabytes, including the approximately 41 GB Windows Docker Desktop PyTorch example. Use those figures when budgeting the specific Workbench installation, and keep them separate from the VRAM needed by an individual model.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesChoosing an upgrade by the problem you actually have
- Out of disk space: add or replace storage after checking interface, physical format, operating-system support and the capacity needed for model versions and containers.
- Model will not fit in VRAM: first reduce representation size or target context only if that is acceptable; otherwise select a GPU with more usable VRAM or a backend with supported offload.
- System becomes unresponsive during inference: check whether RAM is exhausted and the operating system is paging. Close competing applications, reduce concurrency or add RAM appropriate to the platform.
- GPU is full but the model is otherwise usable: verify partial offload or unified-memory support and measure the exact workload. A fallback path may start the model but does not promise GPU-like responsiveness.
- Loading is slow but generation is fine: investigate SSD throughput, file placement and model format. Storage can affect load time without changing the memory capacity available during generation.
A concise pre-purchase checklist
- Write down the exact model, precision or quantization and file format.
- Set maximum context, output length, batch size and concurrent users.
- Check usable VRAM after the display and other GPU applications reserve memory.
- Check usable system RAM after the operating system and normal applications reserve memory.
- Confirm the backend, driver and operating system support the intended placement or fallback mode.
- Budget SSD capacity for models, containers, environments, caches and updates.
- Run the exact model at the maximum context and concurrency before treating the configuration as proven.
The useful distinction is simple: SSD capacity determines what you can keep, RAM determines what the CPU and supported offload paths can hold, and VRAM determines how much the GPU can work on locally. Unified-memory systems change the boundaries, but only according to their documented architecture. Size the complete workload rather than shopping by a single memory number.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
