iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
System RAM and GPU VRAM are separate resources on a computer with a discrete graphics card. VRAM is the limiting pool for keeping a model on the GPU; system RAM enables CPU inference, holds CPU-resident layers during partial offload, and runs the operating system and applications. You cannot add the two capacities and treat the total as one equally fast memory pool. To size a local AI system, budget for weights, context and KV cache, runtime allocations, and the memory already occupied by the OS and other programs.
The short answer: which memory matters?
The answer depends on where you want inference to run. A model that fits completely in GPU VRAM can generally avoid moving its active layers over the slower system bus and is the usual path to the best GPU throughput. If it does not fit, a compatible runtime may keep some layers in system RAM and execute them on the CPU while the remaining layers run on the GPU. That can make an otherwise impossible model usable, but CPU work and transfers normally reduce generation speed.
System RAM is therefore not a substitute for discrete VRAM. It is a capacity resource for CPU-only inference, mixed placement, the operating system, and other software. VRAM is a fast, dedicated working pool for GPU inference. A larger RAM module does not make a GPU with insufficient VRAM behave as though it had more VRAM.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Unified-memory computers are different: the CPU and GPU draw from one physical memory pool. That changes how capacity is shared, but it does not eliminate the need to reserve memory for model weights, context, runtime overhead, the operating system, and applications. Engine support and the specific machine determine how useful that shared pool is.
#1 Best Overall
- 【High-Performance APU】The MS-S1 MAX features an AMD Ryzen AI Max+ 395 APU, integrating a Zen 5 architecture CPU (up to 5.1GHz, 16C/32T, 64M L3 Cache), an RDNA 3.5 GPU, and an NPU (50 TOPS). The total system output is 126 TOPS. It provides powerful parallel computing capabilities for demanding AI workflows. It is ideal for running local LLMs, multimodal models, and computationally intensive tasks
- 【128GB UMA Memory】Equipped with up to 128GB of LPDDR5x-8000MT/s unified memory, it enables the CPU and GPU to access a shared, high-bandwidth memory pool with extremely low latency. Ideal for large-scale AI inference, 3D workloads, and complex timelines in video editing. It eliminates traditional VRAM bottlenecks, ensuring smoother data transfer during high-intensity computations. The UMA design maximizes performance stability under high loads
- 【Flexible Expansion】The MS-S1 MAX features USB4 V2 (up to 80Gbps), dual 10GbE LAN, HDMI 2.1 (up to 8K60), a full-length PCIe x16 expansion slot, and dual M.2 slots supporting up to 16TB RAID 0/1. Wi-Fi 7 provides stronger signal coverage and a more stable wireless experience. The slide-out design facilitates upgrades and maintenance. It easily adapts to personal, studio, or rack-mount enterprise environments
- 【High-Efficiency Cooling System】Utilizing an aerospace-grade aluminum alloy chassis, copper base plate, six heat pipes, dual turbine fans, and advanced PCM thermal conductive material, it maintains stable cooling performance even under continuous load. This system supports 130W continuous power and 160W peak power operation, with a built-in 320W power supply. It boasts multiple global certifications including CCC, FCC, UL, CE, and UKCA, ensuring stable and reliable operation in various environments
- 【Cluster Design】Two MS-S1 MAX units can be configured as a dual-unit cluster to run a large 235B Q4 model locally, achieving an output speed of 10.87 tok/s. Supporting 2U rack deployment, multiple MS-S1 MAX units can be cascaded into a distributed cluster to create a high-efficiency AI computing center. A cluster of four MS-S1 MAX units successfully ran a DeepSeek-R1 671B Q4 large model. A reserved cluster power-on interface allows for unified start-up and shutdown
RAM and VRAM on a discrete-GPU computer
System RAM
- Stores operating-system services, applications, file caches, and CPU-side runtime data.
- Holds all model data during CPU-only inference.
- Can hold layers that a mixed CPU/GPU runtime leaves off the graphics card.
- Provides staging space for transfers between the CPU and a discrete GPU.
GPU VRAM
- Stores GPU-resident weights, activations, attention data, and other engine allocations while the GPU is processing.
- Determines whether a model and its working state can remain entirely on one GPU.
- Can be pooled across cards only when the runtime supports multi-GPU distribution; separate cards do not automatically become one card.
NVIDIA’s guidance describes VRAM and system RAM in a discrete-GPU system as separate resources. A 16 GB graphics card plus 32 GB of system RAM is not equivalent to a single 48 GB pool of fast GPU memory. The GPU can use system memory through supported transfer mechanisms, but those transfers have different latency and bandwidth characteristics from on-card VRAM.
What must fit: the complete serving footprint
The downloaded model file is only one component of memory use. NVIDIA’s PAIR documentation states that “A model’s download size is not its total memory requirement.” For a realistic estimate, account for every item below at the same time:
| Component | What it contains | Why it changes the requirement |
|---|---|---|
| Model weights | Quantized or full-precision parameters | Quantization and format change the weight footprint; parameter count alone does not establish a capacity number. |
| Context and KV cache | Attention history retained for the prompt and generated tokens | Longer context consumes more memory, and the cache is allocated per active request. |
| Engine/runtime overhead | Buffers, graph/workspace allocations, tokenizer data, and backend bookkeeping | Varies by engine, model format, GPU architecture, and settings. |
| Parallel requests | Separate work and context allocations for simultaneous users or jobs | Concurrency can multiply cache and working-memory demand. |
| System reserve | Operating system, drivers, desktop, monitoring tools, and other applications | Memory already in use is unavailable to the model and can trigger allocation failures. |
Think in terms of a complete serving footprint rather than a file-size match. A model that appears to fit exactly may fail when the engine allocates its KV cache or when the desktop and drivers consume the remaining headroom. Leave practical free capacity instead of planning to use every reported gigabyte.
Free tools Windows power users keep installed
One-click scans. No signup required.
How a local runtime places the model
CPU-only inference
In CPU mode, weights and working data reside in system RAM and the processor performs the inference. This is the fallback when no suitable GPU is available or when the model is too large for the available VRAM and the runtime is configured not to offload. It can run large models on a high-memory workstation, but generation speed is determined by the CPU, memory bandwidth, quantization, context, and implementation—not by the amount of unused VRAM in an unrelated graphics card.
Full GPU residency
When the weights, cache, runtime allocations, and safety margin fit in one GPU, the engine can place the relevant layers on that card. This avoids repeated CPU-to-GPU movement for those layers. “Fits” must include the active context and request count, not only the weight file.
Mixed CPU/GPU offload
Backends such as LocalAI’s llama.cpp integration can run a GGUF model with some layers on the GPU and the remaining layers on the CPU. You choose or adjust the number of GPU layers according to available VRAM. At startup, inspect the log for the number of offloaded layers and the reported CPU and GPU buffer sizes; those messages show what the engine actually placed rather than what you intended to place.
Mixed placement is a capacity/performance compromise. CPU-executed layers and transfers can lower generation speed compared with a model that fits entirely on the GPU. The slowdown is workload- and hardware-dependent, so a model being technically runnable does not guarantee an acceptable interactive experience.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallMulti-GPU distribution
Some runtimes can distribute layers across multiple graphics cards. Ollama, for example, evaluates available VRAM and can place a model on one GPU when it fits or spread it across available GPUs when it does not fit on one card. This requires compatible hardware, drivers, model format, and backend support. The aggregate capacity is not automatically identical to a single card with that capacity, because inter-card communication and placement strategy affect performance.
Rank #2
- [3352 AI TOPS, 5th Gen Tensor Cores, AI Content Creation] Built for AI-assisted photo and video workflows including upscaling, denoise, background removal, masking, and generative AI creation for faster creator productivity.
- [32GB GDDR7 VRAM, Local LLM Inference, Larger Models] Run local LLM inference and on-device AI tools with massive VRAM headroom for larger models, longer context, and heavier multitasking across AI and creator apps.
- [28 Gbps, 512-bit, 21760 CUDA Cores] High-throughput next-gen memory and core resources for demanding creator projects, complex timelines, large assets, and GPU-accelerated ML experimentation and inference pipelines.
- [Quad-Fan Force, Vapor Chamber, Phase-Change Thermal Pad] Designed for sustained performance under heavy loads with quad-fan cooling, a patented vapor chamber, and a phase-change GPU thermal pad to help lower temps and reduce hotspots.
- [DP 2.1b x3, HDMI 2.1b x2, Bundle GPU Holder] Multi-display ready with up to 4 displays and up to 7680 x 4320 max digital resolution, plus an included GPU Holder to help reduce GPU sag and improve long-term build stability.
Capacity and performance are different decisions
Capacity answers, “Can the engine load and serve this configuration?” Performance answers, “How quickly and consistently will it generate?” More system RAM can change a failure into a CPU-only or partially offloaded run, but it does not turn CPU execution into GPU execution. More VRAM can allow full residency and a larger context or more simultaneous requests, but a slower GPU, unsupported architecture, or inefficient backend can still limit throughput.
- Interactive single-user use: prioritize enough VRAM for the weights, desired context, and runtime headroom so the model can stay on the GPU.
- Large models on a budget: use adequate system RAM and a supported offload path, accepting that prompt processing or token generation may be slower.
- Several simultaneous users: budget for multiplied KV-cache and runtime allocations; a card that handles one request may run out of memory under concurrency.
- CPU-first workflows: buy system RAM based on the complete model footprint and leave room for the OS and tools. VRAM is not required for this mode, although a supported accelerator may improve it.
Context length and concurrency can change the result
Context is not a free setting. A larger context retains more attention history and increases KV-cache allocation. Multiple requests each need their own active state. Ollama documents this relationship for its own server: required memory scales with OLLAMA_NUM_PARALLEL multiplied by OLLAMA_CONTEXT_LENGTH. Its example describes a 2K context with four parallel requests producing an 8K total context allocation. That arithmetic and the exact allocator behavior are Ollama-specific, not a universal rule for every engine.
Ollama also documents Flash Attention and KV-cache quantization as options that can reduce memory use in supported configurations. They may change quality, speed, compatibility, or numerical behavior, so evaluate them with the model and backend you actually plan to run. Do not assume that a setting available in one runtime exists or has the same effect elsewhere.
Unified memory: one pool, different trade-offs
Apple-silicon Macs and NVIDIA Spark systems are examples NVIDIA identifies as unified-memory platforms, where CPU and GPU share physical memory capacity. In that design, buying more unified memory increases the pool available to both processors rather than adding a separate VRAM module. The practical question becomes how much of the shared pool the engine, model, context, OS, and applications can use at once.
Unified memory avoids the discrete-GPU rule that VRAM and RAM are separate pools, but it is not automatically equivalent to dedicated high-bandwidth VRAM. Bandwidth, software support, memory pressure, and the engine’s placement strategy still matter. Confirm that the model format and runtime support the machine’s accelerator before assuming that all of the shared capacity is usable for inference.
A repeatable way to size a local-AI computer
- Define the workload. Record the model family and quantization, target context length, expected number of simultaneous requests, whether you need an API server, and whether CPU-only operation is acceptable.
- Identify the memory architecture. Determine whether the computer has a discrete GPU with separate VRAM and system RAM or a unified-memory design. On a discrete system, keep the pools separate in your calculation.
- Check the engine and model format. Verify support for the model’s format, GPU architecture, operating system, and desired features such as layer offload, multi-GPU placement, Flash Attention, or KV-cache quantization.
- Estimate the full footprint. Include weights, context/KV cache, runtime buffers, parallel requests, and a reserve for the OS and applications. Do not use the download size as the estimate.
- Compare the estimate with free memory, not advertised capacity. Check current free VRAM and RAM while the normal desktop and monitoring tools are running.
- Choose a placement plan. Try full GPU residency first when the VRAM budget allows it; otherwise select CPU-only or mixed offload and decide whether the expected speed is acceptable.
- Run a representative test. Use the actual context length and concurrency. Watch startup logs for offloaded layers and buffer sizes, then monitor VRAM, RAM, utilization, and generation speed during a sustained request.
- Adjust one variable at a time. Reduce context or parallel requests, lower the number of GPU layers, enable a supported cache optimization, or move to a smaller quantization. Record the setting that fixed the failure and its effect on speed.
Common failure modes and fixes
“The model file fits, but loading fails”
The missing capacity is often the KV cache, runtime buffers, or memory held by the desktop. Lower the context length, reduce simultaneous requests, close memory-heavy applications, or use a smaller quantization. If the engine supports it, enable a documented cache optimization. Keep enough reserve that the system remains stable during generation rather than merely passing the initial load.
“Adding RAM did not make GPU inference faster”
Additional RAM may have enabled CPU-only or mixed execution, but it does not increase the bandwidth of a discrete GPU’s VRAM. To improve GPU-resident performance, you generally need sufficient VRAM, a faster or better-supported GPU, or a configuration that reduces the active footprint.
“The model runs, but token speed is disappointing”
Check whether layers were offloaded to the CPU and whether the runtime reports frequent transfers. A partially offloaded model can be capacity-feasible yet slower than a smaller model that stays entirely on the GPU. Test a lower context, fewer concurrent requests, or a model/quantization that fits fully in VRAM, and compare under the same prompt and settings.
Rank #3
- System Compatibility Note: This 2-slot card measures 271 x 112 x 39 mm and requires a single 12V-2x6-pin power connector. Please verify chassis and PSU compatibility before purchase.
- Dedicated Support: Please contact us directly through Amazon for any product questions or assistance you may require.
- Professional Intel Arc Pro B70 GPU: Built on the Intel Xe2-HPG architecture, it features 32 Xe cores and 256 XMX engines, designed to accelerate AI, rendering, and complex visualization workloads.
- Massive 32GB GDDR6 VRAM: Equipped with 32GB of high-speed GDDR6 memory on a 256-bit bus, running at 19 Gbps, which allows for handling large AI models and complex datasets locally.
- High-Performance Engine Clock: Delivers an engine clock of 2540 MHz, providing the compute power needed for demanding professional applications and AI inference.
“A second GPU did not give the expected capacity or speed”
Confirm that the engine supports multi-GPU distribution and that both cards are visible with compatible drivers. Placement may leave some memory unused, and inter-card communication can reduce throughput. Treat multi-GPU support as a runtime feature to verify, not as automatic memory pooling.
“A unified-memory machine still runs out of memory”
Shared capacity is consumed by the CPU, GPU, OS, applications, weights, cache, and runtime together. Reduce context or concurrency, close other applications, or choose a smaller model. Also verify that the backend actually supports the machine’s accelerator and shared-memory behavior.
What to upgrade first
| Observed constraint | Most relevant upgrade or change | What it will not guarantee |
|---|---|---|
| Model cannot remain on the GPU because VRAM is full | A GPU with more compatible VRAM, a smaller quantization, or lower context/concurrency | More system RAM alone will not create dedicated VRAM or preserve GPU speed. |
| CPU-only or mixed mode exhausts system RAM | More compatible system RAM, a smaller model, or lower context/concurrency | Extra RAM does not make CPU execution as fast as full GPU residency. |
| One request works but several fail | More free VRAM/RAM, fewer parallel requests, or a runtime cache optimization | The single-request memory measurement does not predict multi-request capacity. |
| GPU is supported but underutilized | Check backend, model format, offload settings, drivers, and CPU transfer bottlenecks before buying hardware | Higher advertised memory capacity alone does not fix an incompatible or inefficient path. |
| Unified-memory pressure | A machine with a larger shared pool or a smaller workload footprint | Shared memory does not guarantee the bandwidth or software behavior of dedicated VRAM. |
When shopping for a GPU for local AI, specify the model, quantization, context, concurrency, operating system, and runtime instead of asking for a universal “best” card. For a RAM upgrade, verify motherboard or laptop limits, module compatibility, and whether the workload is actually RAM-bound. Hardware requirements are workload-specific; there is no general conversion such as “X GB of RAM equals Y GB of VRAM.”
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →How to interpret published performance figures
Vendor examples illustrate configurations, not universal ratios. Ollama’s model-scheduling article reports, for a Gemma 3 12B model at 128K context on one RTX 4090, 52.02 versus 85.54 tokens per second, 19.9 versus 21.4 GiB of VRAM, and 48/49 versus 49/49 layers on the GPU for its old and new scheduling examples. Those are Ollama’s stated results for that model, context, card, and scheduling comparison; they are not a general RAM-versus-VRAM benchmark.
Ollama’s June 5, 2026 performance article says Ollama 0.30 was “up to 20% faster” on NVIDIA hardware and identifies Gemma 4 26B on an RTX 5090 with Q4_K_M quantization as the example. Treat that as a vendor-reported, version-, model-, quantization-, and hardware-specific claim. It does not establish how another engine, model, context, or memory configuration will perform.
No universal speed multiplier can be derived from these examples. For a purchasing decision, measure the exact workload or use figures that state the same model, quantization, context, request count, runtime version, GPU, and placement strategy.
The practical rule
On a discrete-GPU PC, prioritize VRAM when your goal is fast, fully GPU-resident inference; prioritize system RAM when you need CPU inference, mixed offload, larger general workloads, or enough headroom for the OS and applications. Treat partial offload as a way to expand capacity with a possible speed penalty, not as a free extension of VRAM. On unified-memory hardware, evaluate the shared pool and software support together. In every architecture, calculate the complete serving footprint—including weights, context/KV cache, runtime overhead, concurrency, and system reserve—before deciding whether a model will fit.
Recommended Free Tools
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
