Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
For most local AI users, 16GB of system RAM is the practical starting point: it is comfortable for 7B–8B models and the minimum I would plan for many 13B models. Choose 32GB if you want longer context windows, browser and development multitasking, or partial CPU offload. A 70B-class model generally calls for 64GB or more of system RAM, while full GPU offload for a 4-bit 70B model usually needs roughly 35–40GB of VRAM.
Those are planning ranges, not guarantees. The actual requirement changes with parameter count, quantization, context length, runtime overhead, operating-system memory, and how much of the model runs on the CPU versus the GPU.
How much RAM does a local AI model need by model size?
Parameter count is the first sizing signal: larger models contain more weights, require more memory, and generally run more slowly on the same hardware. The table below combines commonly published guidance from Ollama, NIH Biowulf, NVIDIA and LM Studio. Treat every entry as a planning range rather than a guaranteed minimum.
| Model size | Typical Q4 memory signal | Practical system RAM target | Typical use and constraints |
|---|---|---|---|
| 1–3B | About 2GB of VRAM in the NIH Ollama table | 8GB can work | Lightweight chat, classification and simple extraction; other applications and long prompts still need headroom. |
| 7–8B | About 6–8GB of VRAM in the NIH table | 16GB recommended | The sensible desktop starting point. Ollama lists at least 8GB of RAM, but 16GB leaves room for the operating system and a normal desktop workload. |
| 13–14B | About 12–16GB of VRAM in the NIH table | 16–32GB | Sixteen gigabytes can work with a short context and limited multitasking; 32GB is more forgiving for larger contexts and CPU offload. |
| 30–35B | Roughly 20GB or more of Q4 VRAM, depending on the model and runtime | 32GB or more | Usually a 24GB-plus GPU class or a mixed CPU/GPU setup. Exact fit varies substantially by architecture and context. |
| 70B and larger | About 35–40GB of Q4 VRAM in the NIH table | 64GB or more | Multi-GPU or unified-memory systems are common. Ollama’s general guidance is at least 64GB of RAM. |
The numbers above describe model loading, not a speed guarantee. A model that technically fits may still feel unusable if the GPU is weak, the CPU is doing most of the work, or the context window is large.
#1 Best Overall
- Capacity: 16GB(2 x 8GB)
- Tested Frequency Profile 1: PC5-48000 (6000MT/s)
- Tested Timings: 36-46-46-110
- Feature Overclock: XMP 3.0 & EXPO overclocking supported
- On-Die ECC
Ollama’s Llama 2 documentation, dated July 18, 2023, gives a simpler floor of at least 8GB of RAM for 7B models, 16GB for 13B models and 64GB for 70B models. Those figures remain useful as a baseline, but newer runtimes and longer context settings can push actual usage higher.
What memory does a local AI model actually use?
A local model does not consume RAM only for its downloadable weight file. At runtime, memory is divided among several components:
- Model weights: the learned parameters. Parameter count and numerical precision determine their basic size.
- KV cache: memory used to retain the prompt, conversation history, tool results and retrieved documents that are inside the active context window.
- Runtime overhead: buffers, tokenizer data, computation workspaces and metadata required by the inference engine.
- Operating-system and application memory: the desktop, browser, editors, background services and the runtime itself.
NVIDIA’s guidance defines context as the prompt, chat history, tool outputs and retrieved documents the model can consider at once. Increasing context from a short chat to a long document workflow therefore changes the memory requirement even when the model file is identical.
NVIDIA NIM’s rough sizing guidance reserves 5–10GB for the operating system and other processes. Its examples put an 8B Llama model at about 15GB under that sizing rule and a 70B Llama model at about 131GB. These are deployment estimates, not universal model-file sizes; use them to understand why a computer can fail despite having enough nominal memory for the weights alone.
How does quantization change the RAM requirement?
Quantization stores weights with fewer bits. A Q4 model uses four-bit weights, Q5 uses five-bit weights, Q8 uses eight-bit weights and FP16 uses 16-bit floating-point values. NIH’s Ollama guidance summarizes the effect as: 4-bit quantization reduces memory to ~25% of original.
That reduction is why a quantized 7B or 8B model can run on a mainstream computer instead of requiring workstation-class memory. Higher-bit variants generally consume more memory but preserve more of the original numerical precision. NVIDIA cautions that aggressive quantization can reduce response quality, so the smallest file is not automatically the best choice.
Use this rough calculation to compare weight sizes before accounting for cache and overhead:
Free tools Windows power users keep installed
One-click scans. No signup required.
weight memory in bytes ≈ parameter count × bits per parameter ÷ 8
For example, the raw four-bit weight calculation for a 7-billion-parameter model is about 3.5GB before metadata, alignment, runtime buffers and the KV cache are added. The downloaded file may therefore be larger or smaller than a simple arithmetic estimate, depending on the format and runtime.
Quantization also affects quality unevenly. A model may remain excellent for short factual prompts at Q4 yet lose instruction-following or code accuracy at a more aggressive setting. If output quality matters, move up to Q5 or Q8 only after confirming that your RAM and VRAM have enough headroom.
Rank #2
- Boosts System Performance: 32GB DDR5 RAM laptop memory kit (2x16GB) that operates at 5600MHz, 5200MHz, or 4800MHz to improve multitasking and system responsiveness for smoother performance
- Accelerated gaming performance: Every millisecond gained in fast-paced gameplay counts—power through heavy workloads and benefit from versatile downclocking and higher frame rates
- Optimized DDR5 compatibility: Best for 12th Gen Intel Core and AMD Ryzen 7000 Series processors — Intel XMP 3.0 and AMD EXPO also supported on the same RAM module
- Trusted Micron Quality: Backed by 42 years of memory expertise, this DDR5 RAM is rigorously tested at both component and module levels, ensuring top performance and reliability
- ECC Type = Non-ECC, Form Factor = SODIMM, Pin Count = 262-Pin, PC Speed = PC5-44800, Voltage = 1.1V, Rank And Configuration = 1Rx8
How much RAM do I need for a 7B model?
Plan on 16GB of system RAM for a comfortable 7B–8B desktop setup. Eight gigabytes can load some small Q4 builds, but the operating system, browser, runtime and context leave little room for anything else.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallWhen 8GB is workable
Eight gigabytes is appropriate for a lightweight assistant using a small quantized model, a short context and few background applications. Expect to close memory-heavy programs and avoid large document retrieval or tool pipelines. If the runtime must borrow system RAM because VRAM is unavailable, responsiveness will usually decline.
Why 16GB is the better floor
Sixteen gigabytes lets a 7B model share the computer with a normal desktop session and provides room for a larger context than the minimum configuration. It also gives the runtime somewhere to place layers that do not fit in a dedicated GPU. Keep in mind that “16GB installed” does not mean 16GB is available to the model; the operating system and applications claim part of it first.
Can I run a 13B model on 16GB of RAM?
Yes, often with a Q4 build and a modest context window, but it is a near-floor configuration rather than a roomy one. Ollama’s published guidance places 13B models at a minimum of 16GB of RAM, while the NIH table places a typical Q4 13–14B model around 12–16GB of VRAM. Those figures leave little capacity for the operating system, long conversations or other applications.
Conditions that make 16GB viable
- Use a well-supported Q4 model rather than an FP16 or high-bit file.
- Keep the context window moderate and avoid loading large retrieved documents into every prompt.
- Close browsers, virtual machines and other memory-intensive programs while running inference.
- Use a GPU with enough VRAM to hold as many layers as possible; otherwise the CPU and system RAM carry the shortfall.
When 32GB is the safer choice
Choose 32GB if you routinely keep development tools open, use retrieval-augmented generation, need longer chat history, or expect partial CPU offload. The extra memory does not make a 13B model intrinsically smarter, but it reduces out-of-memory failures and gives the runtime more room to maintain context.
Recommended Free Tools
How much VRAM does a 70B model need?
A 70B model in Q4 typically needs about 35–40GB of VRAM for full GPU offload according to the NIH table. A single 24GB graphics card therefore cannot normally hold the entire model in that format. You would need multiple GPUs, a card or unified-memory system with a larger pool, or a split where some layers remain in system RAM.
NVIDIA’s NIM example estimates about 131GB for an 70B Llama deployment when its broader runtime and system allowances are included. That is why a 70B configuration can require substantially more total memory than the raw Q4 weight estimate. The exact result depends on the model variant, context length, quantization implementation and serving stack.
What happens with partial offload?
Partial offload places the layers that fit in VRAM on the GPU and keeps the remainder in CPU memory. It can make a model usable on hardware that cannot hold the full file, but every operation involving CPU-resident layers crosses the system bus and is slower than full GPU execution. The larger the CPU portion, the more noticeable the latency and the lower the token rate.
Will 4-bit quantization fit on my laptop?
It may. A 4-bit build is the first option to test when a laptop has limited memory, but fit depends on both pools: system RAM and GPU VRAM. A laptop with 16GB of RAM and an 8GB dedicated GPU is a reasonable target for many 7B–8B Q4 models; a 13B model may require CPU offload and a shorter context.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsCheck the runtime’s reported allocation rather than relying only on the file size. Leave room for the operating system, integrated graphics reservations, the desktop and the application that launches the model. If the laptop uses integrated graphics, graphics memory may be taken from the same system-RAM pool, reducing what is available to inference.
Rank #3
- Disclaimer: Maximum Speed requires overclocking/PC BIOS adjustments. Maximum speed and performance depend on system components, including motherboard and CPU
- AMD EXPO & Intel XMP 3.0 Compatible Only: Dual memory profiles allow you to easily select optimized settings for your platform, whether you’re running an AMD or Intel processor
- Dynamic RGB Lighting: Individually addressable RGB lighting delivers vibrant effects through a sleek, understated panoramic diffuser
- Onboard Voltage Regulation: Onboard voltage regulation for reliable power at high frequencies
- Maximum Bandwidth and Tight Response Times: Optimized for peak performance on the latest AMD and Intel DDR5 motherboards
Can I run a local LLM without a GPU?
Yes, but speed is the trade-off. LM Studio describes a dedicated GPU as optional and recommends at least 4GB of dedicated VRAM for Windows users who do have one. NIH’s Ollama guidance says CPU fallback is possible but very slow except for the smallest models.
A CPU-only setup makes sense for small quantized models, offline text processing and experimentation. It is less suitable for interactive work with large models, long contexts or multiple users. More system RAM can let you load a larger model, but it cannot provide the memory bandwidth of a capable GPU.
How do 16GB, 32GB and 64GB systems compare?
The best configuration depends on the model, quantization, available VRAM and context target. This comparison focuses on what each system-RAM tier can realistically support when paired with common GPU-memory classes.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →| System RAM | Typical VRAM pairing | Model range to target | Context and multitasking | CPU/GPU placement | Expected experience |
|---|---|---|---|---|---|
| 16GB | 8GB dedicated VRAM, or integrated graphics | 1–8B Q4 comfortably; 13B Q4 with constraints | Short to moderate context; limited room for heavy browser or development workloads | Prefer mostly GPU-resident 7B–8B models; some CPU offload is possible | Responsive for small models when the GPU holds most layers; CPU-heavy runs can be slow |
| 32GB | 12–16GB VRAM | 7–14B Q4/Q5; some 30B-class experiments with offload | More room for longer conversations, retrieval and normal multitasking | Balanced split for 13B–14B models; partial offload for larger files | More consistent interactive use, subject to GPU bandwidth and model support |
| 64GB or more | 24GB-plus VRAM, multiple GPUs or large unified memory | 13B–35B comfortably; 70B with sufficient total memory and an appropriate split | Long contexts and substantial development workloads are practical, but still consume memory | Full GPU offload for medium models; multi-GPU or mixed placement for 70B-class models | Best flexibility; speed still depends on the GPU, interconnect and runtime |
There is no universal speed number for any row. Two computers with the same RAM can differ dramatically because of GPU architecture, memory bandwidth, CPU generation, thermal limits and the percentage of layers that remain on the CPU.
How should I calculate the memory budget before downloading a model?
Use this sequence to avoid choosing by parameter count alone.
- Identify the exact parameter count and format. Record whether the file is Q4, Q5, Q8 or FP16. “8B” describes scale, not a complete memory specification.
- Estimate raw weight memory. Multiply parameters by bits per parameter and divide by eight. Treat the result as a lower bound because metadata and buffers are not included.
- Set the context target. Decide whether you need a short chat, long conversation, coding session, tool calls or document retrieval. Larger context consumes more memory through the KV cache.
- Reserve system headroom. Keep roughly 5–10GB for the operating system and other processes, following NVIDIA NIM’s rough deployment allowance. Reduce the available budget further if you use an integrated GPU or memory-heavy applications.
- Check VRAM separately from RAM. A scheduler’s
--memvalue refers to system memory, not GPU VRAM. Confirm the GPU allocation shown by your runtime. - Choose a placement plan. Full GPU offload is generally fastest; partial offload can fit a larger model but usually runs slower; CPU-only execution is the fallback for small models.
- Validate with the intended workload. Load the model, set the context you actually plan to use, and run a representative prompt. A successful load at the default context does not prove that a long document workflow will fit.
Which hardware specifications matter besides RAM?
GPU VRAM and memory bandwidth
VRAM is a separate pool from system RAM on a conventional PC. A GPU with enough capacity can keep the model and its cache close to the compute units, while a card with insufficient capacity forces CPU offload. Capacity determines whether the model fits; bandwidth and compute determine how quickly it generates tokens.
Apple unified memory
Systems with unified memory share one pool between CPU and GPU rather than exposing separate user-upgradeable VRAM. That can simplify large-model loading, but the operating system and graphics also draw from the same pool. Treat the advertised unified-memory capacity as a total budget, not as memory dedicated entirely to the model.
CPU and interconnect
CPU-only and mixed placements depend on memory bandwidth and the path between CPU and GPU. A model that fits through PCIe transfers may still feel sluggish if many layers move across that link for every token. Multi-GPU systems also depend on how efficiently cards exchange data.
Storage
Model weights should live on an SSD when possible. An SSD does not replace RAM, but it shortens downloads, model switching and reloads. Keep free space beyond the model file itself for alternate quantizations, temporary downloads and runtime caches.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How do I upgrade RAM for local AI use?
If your computer is below the 16GB baseline, a correctly matched memory kit is often the simplest upgrade. Before buying a generic 16GB DDR5 RAM kit or any other capacity, verify all of the following:
Rank #4
- Disclaimer: Maximum Speed requires overclocking/PC BIOS adjustments. Maximum speed and performance depend on system components, including motherboard and CPU
- Intel XMP 3.0 Compatible Only: Customize and save your own XMP profiles via iCUE to tailor performance by app or task for greater efficiency
- Dynamic Ten-Zone RGB Lighting: Illuminate your system with ten individually addressable, ultra-bright RGB LEDs per module, encased in a panoramic light bar for vivid RGB lighting from any viewing angle
- Onboard Voltage Regulation: Enables easier, more finely-tuned, and more stable overclocking through CORSAIR iCUE software than previous generation motherboard control
- Maximum Bandwidth and Tight Response Times: Optimized for peak performance on the latest Intel DDR5 motherboards
- Whether the motherboard or laptop uses DDR4 or DDR5; the generations are not interchangeable.
- Whether the machine accepts desktop DIMMs or laptop SO-DIMMs.
- The maximum supported capacity and speed in the system or motherboard manual.
- How many slots are present and whether any are already occupied.
- Whether the firmware supports the planned module density and whether a matched pair is recommended.
On a laptop with soldered memory, check the service manual before ordering anything. If the computer cannot be upgraded, reduce model size or context, use a more aggressive quantization, or move inference to hardware with a larger memory pool.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What should I do when a model does not fit?
The runtime reports out of memory while loading
- Choose a lower-bit quantization, such as Q4 instead of Q8 or FP16.
- Lower the context-window setting before opening a large chat or document.
- Close other applications and restart the runtime to clear fragmented allocations.
- Move additional layers to system RAM if the software supports CPU offload.
- Use a smaller parameter count when the model still exceeds the combined budget.
The model loads but generation is painfully slow
- Check whether most layers landed in system RAM rather than VRAM.
- Use a smaller model that fits fully on the GPU.
- Reduce context length and remove unnecessary retrieved text or tool output.
- Confirm that the runtime is using the intended GPU and an appropriate acceleration backend.
Quality drops after quantization
Move up one quantization level and compare outputs on the tasks you care about. Aggressive quantization can preserve fluency while weakening coding, arithmetic, instruction following or factual detail; the acceptable trade-off depends on your workload.
The desktop becomes unstable during inference
Monitor system RAM while the model is running, not just at load time. If memory is exhausted, the operating system may page aggressively or terminate the runtime. Lower the model size or context and keep additional headroom for the desktop instead of treating every installed gigabyte as available to inference.
What is the safest rule for choosing a local model?
NVIDIA’s practical rule is: In general, use the most powerful model that fits comfortably in your GPU’s memory.
“Comfortably” means the model loads with your intended quantization and context while leaving room for the operating system and the applications you will keep open. A smaller model that stays on the GPU is often more useful than a larger model that constantly spills into CPU memory.
For a first purchase or upgrade, prioritize 16GB of system RAM for 7B–8B use, 32GB for broader 13B–14B experimentation and multitasking, and 64GB or more only when your workloads justify 30B–70B-class models or large unified-memory configurations. Recheck the model card and runtime requirements whenever you change architecture, quantization or context length.
Frequently Asked Questions
Does virtual memory or swap let me run a larger local model?
Swap can prevent an immediate crash by moving inactive pages to storage, but it does not provide RAM-like bandwidth. Once active model data is repeatedly paged, generation can become unusably slow, so swap is an emergency cushion rather than a sizing strategy.
Why does the same model use different amounts of memory in different runtimes?
Runtimes may store metadata differently, choose different KV-cache formats, reserve different workspaces or support different GPU-offload strategies. The quantization label alone therefore cannot predict the exact allocation shown by every application.
How much SSD space should I reserve for local models?
Reserve space for the weight file plus alternate quantizations, temporary downloads and runtime caches. Keeping models on an SSD improves loading and switching times, but it cannot compensate for insufficient system RAM or VRAM during inference.
Can I add more VRAM to a laptop if the model does not fit?
Dedicated laptop VRAM is normally soldered to the graphics hardware and is not upgradeable like system memory. Practical alternatives are a smaller quantized model, CPU offload, an external or desktop GPU where supported, or a computer with more unified or dedicated memory.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
