Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
There is no single RAM or VRAM minimum for local AI. Start with the exact model file you intend to run, including its quantization format, then add memory for the runtime, context window, key-value cache, other applications and any concurrent requests. System RAM and GPU VRAM are separate pools on most PCs. A model larger than your GPU’s VRAM can sometimes run through CPU/GPU hybrid inference, but usually with lower speed and different trade-offs.
As concrete reference points, a Llama 3.1 8B Q4_K_M file is about 4.9 GB, a 70B Q4_K_M file about 43.1 GB, and a 405B Q4_K_M file about 249.1 GB in ggml-org’s version-tagged quantization documentation. Those are weight-file sizes, not complete runtime requirements.
What actually determines local-AI memory use?
Model parameters are only a rough label
An “8B” or “70B” label describes the model’s parameter count, but it does not tell you exactly how much memory your runtime will allocate. The same family can be distributed in an original format and in several quantized formats. Quantization stores weights with fewer bits, reducing the file and memory footprint, while potentially changing output quality, speed and compatibility.
Use the exact downloadable file, not just the parameter count, as your starting number. Record its format (for example, Q4_K_M), its displayed size and whether the runtime converts or repacks it when loading.
#1 Best Overall
- Requires overclocking/BIOS adjustments. Maximum speed and performance depends on system components, including motherboard and CPU.
- G.SKILL Trident Z5 Neo RGB Series DDR5 U-DIMM Memory Kit, Model: F5-6000J3636F32GX2-TZ5NR
- Non-ECC, DDR5 U-DIMM, 288-pin, for Desktop PC & Gaming
- Includes JEDEC default profile, and AMD EXPO & Intel XMP 3.0 memory overclock profile
- Do not mix memory kits. Memory kits are sold in matched kits that are designed to run together as a set. Mixing memory kits will result in stability issues or system failure.
The file is not the whole workload
Loaded weights share memory with temporary runtime buffers, the context’s key-value (K/V) cache, tokenization and computation buffers, the operating system, display services and other applications. Intel’s OpenVINO guidance gives a deliberately broad rule: system RAM should be at least the model-file size, while GPU VRAM should hold the model file plus padding for other GPU tasks. It also warns that context length, model size and other factors change actual inference use. Treat that as a starting check rather than a universal formula.
Context length and parallel requests add memory
A longer prompt or conversation requires a larger K/V cache. Serving several requests at once can multiply that cache demand. Ollama documents this explicitly: memory for parallel processing scales with OLLAMA_NUM_PARALLEL multiplied by OLLAMA_CONTEXT_LENGTH. A model that loads for one short conversation can therefore run out of memory when you increase the context window or allow multiple users.
Where the runtime places the model matters
System RAM is used for CPU inference and can also hold layers that are not placed on the GPU. VRAM is the GPU’s local memory for weights, caches and other GPU work. The capacity printed on a graphics card is not all available to one model; the operating system, display driver, desktop and other applications consume part of it.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesPublished model-size examples
The following figures illustrate why the exact file matters. The original-size column is the unquantized file listed in ggml-org’s version-tagged documentation; the Q4_K_M column is that documentation’s quantized example. Ollama’s library listing shows rounded values for the same Llama 3.1 family.
| Model | Original file (GB) | Q4_K_M file (source unit) | Ollama library listing |
|---|---|---|---|
| Llama 3.1 8B | 32.1 GB | 4.9 GB; the quantization table also gives 4.58 GiB | 4.9 GB |
| Llama 3.1 70B | 280.9 GB | 43.1 GB | 43 GB |
| Llama 3.1 405B | 1,625.1 GB | 249.1 GB | 243 GB |
GB and GiB are different units, so do not silently convert one into the other when checking a download. These values describe model files or quantized weights. They do not guarantee that a machine with exactly that much RAM or VRAM will load the model, because runtime overhead and context memory still have to fit.
How to estimate system RAM
Begin with the selected file
For a CPU-only run, compare your available RAM with the quantized file size. Intel’s rule of thumb says RAM should be at least as large as the model file, then requires additional room for context, runtime allocations and normal desktop use. If your machine has only enough free RAM to equal the download size, it is already at the edge.
For example, a 4.9 GB 8B Q4_K_M file is a much smaller starting point than a 43.1 GB 70B Q4_K_M file. The comparison does not mean “4.9 GB of total RAM” or “43.1 GB of total RAM” is sufficient; it means those numbers are the weight component before overhead.
Leave headroom instead of relying on swapping
When physical RAM fills, the operating system can move pages to disk. Intel’s guidance notes that this can make inference extremely slow or cause a crash. An SSD improves storage and may make swapping less painful than a hard drive, but it does not turn disk capacity into usable inference RAM.
Close memory-heavy applications before testing, and check free RAM while the model is loaded rather than relying on the computer’s installed-capacity label. A browser with many tabs, a virtual machine, a game launcher or an editor can consume the headroom your model needs.
Account for layers that do not fit on the GPU
In a split CPU/GPU configuration, the CPU-side layers consume system RAM while the GPU-side layers consume VRAM. A model may therefore require substantial amounts of both pools even when neither pool holds the complete file alone. The exact split depends on the runtime and its offload settings.
How to estimate GPU VRAM
VRAM determines how much can be offloaded
A GPU can accelerate the layers and cache that fit in its local memory. Reserve space for the display system, graphics workloads and runtime buffers; a card’s advertised capacity is not a promise that all of it is available to the model. Intel’s recommendation to add padding beyond the model file is especially important on a desktop GPU driving a monitor.
If the quantized weights fit comfortably with that headroom, a mostly or entirely GPU-resident load is possible for a compatible runtime. If they do not, you have three choices: use system RAM, split the model between CPU and GPU, or choose a smaller or more aggressively quantized file.
Rank #2
- Boosts System Performance: 64GB DDR5 RAM laptop memory kit (2x32GB) that operates at 4800MHz to improve multitasking and system responsiveness for smoother performance
- Accelerated gaming performance: Every millisecond gained in fast-paced gameplay counts—power through heavy workloads and benefit from versatile downclocking and higher frame rates
- Optimized DDR5 compatibility: Best for 12th Gen Intel Core and AMD Ryzen 7000 Series processors — Intel XMP 3.0 and AMD EXPO also supported on the same RAM module
- Trusted Micron Quality: Backed by 42 years of memory expertise, this DDR5 RAM is rigorously tested at both component and module levels, ensuring top performance and reliability
- ECC type=non-ECC, Form Factor=SODIMM, Pin count=262-pin, PC speed=PC5-38400, Voltage=1.1V, Rank and Configuration=2Rx8
Hybrid inference makes “larger than VRAM” possible
llama.cpp documents CPU/GPU hybrid inference for models larger than available VRAM. Ollama likewise reports whether a loaded model is entirely on the GPU, entirely in system memory or split across both. Hybrid execution can make a model usable on hardware that cannot hold all weights in VRAM, but moving data between CPU memory and the GPU generally changes throughput and latency. The practical result depends on the model, context, processor, memory bandwidth, GPU and runtime settings; the file size alone cannot predict a speed number.
Check placement instead of guessing
After loading a model in Ollama, run:
ollama psIts Processor column identifies the memory used for the loaded model. Use that report to distinguish a GPU load, a system-memory load and a CPU/GPU split. This is more reliable than inferring placement from the GPU’s specification or from the fact that a model eventually produced an answer.
Context windows and concurrency in Ollama
The default context is not a permanent requirement
Ollama’s FAQ documents a default context window of 4,096 tokens and explains how to change it. A larger context lets the model retain more prompt and conversation text, but it increases K/V-cache memory. If a model loads at the default and fails after you raise the context, the cache—not the weight file—may be the difference.
Parallel requests multiply the cache
With parallel request processing, Ollama describes the effective context allocation as increasing with the number of parallel requests. In practical terms, doubling both the context setting and the parallel-request count can require far more cache memory than either change alone. Set concurrency to the number of simultaneous conversations you actually need, then test with the longest prompts you expect.
Memory-saving controls have trade-offs
Ollama documents Flash Attention and quantized K/V-cache settings as ways to reduce cache memory. They are Ollama-specific controls, not universal requirements for every local-AI runtime. Reduced-precision cache settings can affect numerical behavior and output quality differently across models and tasks, so validate important workloads after enabling them.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.A practical planning method for any PC
- Choose the exact model variant. Record the family, parameter size, quantization format and published file size. Do not plan from “8B” or “70B” alone.
- Decide the intended placement. Will the runtime use CPU RAM, GPU VRAM or a split? If you need GPU acceleration, reserve VRAM for the display and other GPU work.
- Measure available memory. Use free system RAM and usable VRAM, not installed totals. Close competing applications while measuring.
- Add workload requirements. Account for the context window, K/V cache, prompt length, generation length and the number of simultaneous requests.
- Load and inspect. Start with a short prompt, then check runtime status. In Ollama,
ollama psshows the Processor placement. - Stress the expected use. Test your longest normal context and intended concurrency. A model that loads once is not necessarily stable under production-like use.
- Change one variable at a time. Lower context, reduce parallelism, choose a smaller quantization or move more layers to system RAM so you can identify what solved an out-of-memory failure.
What the three Llama 3.1 examples imply
8B class
The 4.9 GB Q4_K_M example is small enough to be a realistic starting point for many desktop configurations, but the complete requirement still includes runtime and context memory. It is the easiest of these examples to test with CPU RAM, a modest-memory GPU or a hybrid split.
70B class
The 43.1 GB Q4_K_M file is an order-of-magnitude larger weight load. A single consumer GPU is unlikely to contain the entire quantized file while retaining comfortable headroom, so system RAM or multi-device/hybrid placement becomes central to planning. Whether the resulting speed is acceptable is workload- and hardware-dependent.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →405B class
The 249.1 GB Q4_K_M example is a multi-hundred-gigabyte weight load before overhead. Running it locally requires planning for very large system memory, multiple GPUs, or both, along with storage for the download. It should not be treated as a normal single-GPU desktop target.
Storage is separate from inference memory
Model downloads occupy disk space. Keep enough free SSD capacity for the file, temporary downloads, alternate quantizations and updates. An SSD can reduce load times compared with slower storage, but it does not increase RAM or VRAM and cannot substitute for either during generation.
For the examples above, plan for at least the displayed file size on disk and additional workspace. The 405B Q4_K_M example alone is about 249.1 GB in the ggml-org listing, while the 70B example is about 43.1 GB; downloading more than one variant multiplies storage use.
Troubleshooting common memory failures
The model will not load
- Verify that the downloaded file is the intended quantization and not an unquantized variant.
- Check free RAM and free VRAM while other applications are closed.
- Reduce context length and parallel requests before changing hardware.
- Use a smaller quantized file or enable CPU/GPU splitting when the runtime supports it.
It loads but is unexpectedly slow
- Run
ollama psand inspect the Processor column for CPU-only or split placement. - Check whether the GPU is sharing memory with the display or another workload.
- Confirm that the larger context or concurrency setting has not increased cache pressure and forced offloading.
- Remember that hybrid execution can be functional while still being slower than an all-GPU load.
The machine freezes or starts swapping
Stop the workload and reduce the model, context or concurrency. Swapping indicates that available physical RAM is insufficient for the current workload; waiting longer will not make the configuration memory-efficient.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What to verify before buying hardware
- RAM compatibility: motherboard-supported memory type, capacity limits, slot availability and upgrade path.
- GPU compatibility: VRAM capacity, physical dimensions, power delivery, display requirements and software/runtime support.
- Workload: one interactive user versus several simultaneous requests, and typical versus maximum context length.
- Placement plan: all-GPU, CPU-only or hybrid. More VRAM is most useful when the runtime can actually offload the model and cache to it.
- Storage: sufficient SSD space for model files and alternate quantizations, kept distinct from the RAM/VRAM calculation.
There is no universally correct RAM or VRAM purchase because the answer changes with the model file, runtime, context and concurrency. Size those variables first, then buy compatible capacity with headroom rather than matching a graphics card’s headline VRAM to a model’s download size.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
