The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Short answer: Nemotron 3.5 Super VL is not a small single-GPU model in its BF16 form. NVIDIA documents BF16 profiles ranging from two GPUs with at least 119 GB per GPU to eight H100 80 GB GPUs with a 32 GB-per-GPU startup floor. The official NVFP4 profiles reduce the requirement to 72 GB on one supported GPU or 38 GB per GPU across two supported GPUs. Those figures are startup floors for particular profiles, not guarantees for a 1-million-token context, high concurrency, or heavy image and video traffic.
Out-of-memory (OOM) errors usually come from the KV cache, media-decoding reservations, an incorrect NIM profile, or insufficient host RAM for agent workloads—not simply from the model weights. The reliable fix is to select a supported precision and GPU layout, choose the matching profile explicitly, leave utilization headroom, and size context, concurrency, media, and system memory together.
What Nemotron 3.5 Super VL is and why its memory footprint is unusual
NVIDIA describes Nemotron 3.5 Super VL as a 120-billion-parameter-total, 12-billion-active hybrid Mamba-Transformer mixture-of-experts (MoE) vision-language model. It combines interleaved Mamba-2, MoE and attention layers with a C-RADIO vision encoder and a vision-language connector. The model accepts text, images and video and returns text, with a maximum context of up to 1 million tokens.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
“Active” parameters describe the MoE computation used for a token; they do not mean the complete 120B parameter set disappears from memory. The complete model, runtime buffers, communication storage and cache allocations still have to be placed across the GPUs selected by the deployment profile. Vision inputs add another allocation path because images and video must be decoded and represented before generation.
#1 Best Overall
- AI Performance: 767 AI TOPS
- OC mode: 2632 MHz (OC mode)/ 2602 MHz (Default mode)
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- A 2.5-slot design maximizes compatibility and cooling efficiency for superior performance in small chassis
The model card lists deployment on 4× H100 or 2× GB200 systems. NIM profile tables show additional verified layouts, including an eight-H100 BF16 configuration and one- or two-GPU NVFP4 configurations. These are profile-specific runtime choices, not interchangeable GPU-count recommendations.
Published VRAM requirements by precision and profile
The following values are NVIDIA-published minimum per-GPU startup floors for named profiles. They cover model weights and runtime allocations, but not the extra KV-cache capacity needed for long contexts or many simultaneous sequences.
| Precision and profile | Published layout | Minimum memory floor | What it means in practice |
|---|---|---|---|
| BF16, TP2 | 2× B200, B300, GB200 or GB300 | 119 GB per GPU | Requires high-memory Blackwell or Grace Blackwell hardware; reserve additional capacity for KV cache and media. |
| BF16, TP4 | 4× H200 | 61 GB per GPU | A four-GPU H200 profile; usable context and concurrency determine how much memory remains. |
| BF16, TP8 | 8× H100 80GB | 32 GB per GPU | A verified eight-GPU H100 profile. The 32 GB figure is not a promise that one H100 can run BF16 alone. |
| NVFP4, TP1 | 1× B200, B300, GB10, GB200, GB300 or H200 | 72 GB | Quantized weights fit one supported high-memory GPU, but cache, image/video work and framework overhead still need room. |
| NVFP4, TP2 | 2× A100 80GB, H100 80GB or RTX PRO 6000 Blackwell Server Edition | 38 GB per GPU | A verified two-GPU quantized layout with a lower per-card floor; it still is not a 38 GB universal guarantee for every workload. |
“TP” means tensor parallelism: model tensors are sharded across that many GPUs. A floor such as 32 GB or 38 GB is calculated at startup for the exact profile. It does not include headroom for the KV cache at long context length or high concurrency. Treat every value in the table as a minimum to load the profile, not as the amount of VRAM your production service can safely consume.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070 Ti
- Integrated with 16GB GDDR7 256bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Can Nemotron 3.5 Super VL run on one GPU?
Yes, but only with a supported NVFP4 single-GPU profile and a GPU that meets its requirement. NVIDIA lists a 72 GB minimum for the TP1 NVFP4 profiles on B200, B300, GB10, GB200, GB300 and H200. A card with less available memory cannot be made compatible by changing tensor-parallel settings.
BF16 is a different situation. The published BF16 layouts are multi-GPU: TP2 on selected 119-GB-per-GPU systems, TP4 on H200, and TP8 on H100 80GB. The model card’s 4× H100 example and NIM’s verified eight-H100 BF16 profile refer to different deployment profiles and should not be conflated. There is no documented one-H100 BF16 profile in the figures above.
NVFP4 also changes the trade-off. Quantization lowers weight memory, but it may not suit a workflow that requires BF16 customization, LoRA training or full-precision evaluation. Choose it when the official profile supports your hardware and quantized inference meets your quality and tooling requirements.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Why the GPU can show free memory and the process still OOMs
The KV cache expands with tokens and concurrency
Every active sequence needs cache storage for its attention history. The context window includes the user prompt, image and video representations, conversation history, tool results and generated tokens. A 1-million-token maximum is a capability ceiling, not a workload recommendation. Running several long sequences multiplies cache demand even when the model weights fit comfortably.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsNVIDIA’s deployment recipes may select a smaller context window to fit available memory and concurrency. If OOM appears only after requests begin, rather than during model loading, cache growth is a prime suspect.
Image and video decoding reserve GPU memory
On deployments other than DGX Spark, NIM reserves 10 GiB of GPU memory for image and video decoding through the mm_ipc_gpu_memory_gb setting. Media requests wait for that reservation instead of immediately colliding with other allocations. High-resolution images, many images in one request, or multiple videos can still increase pressure and queue requests.
Rank #4
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5060
- Integrated with 8GB GDDR7 128bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Reducing the reservation gives the KV cache more space but makes large or concurrent media requests serialize sooner. It is a throughput trade-off, not free memory.
AgentX profiles can exhaust system RAM first
NIM’s agentX profiles offload the KV cache to CPU memory. They use 200 GiB of host memory per GPU, or 400 GiB for the two-GPU H100 agentX profile, in addition to the memory consumed by the container, operating system and other services. A host can therefore fail or begin swapping while its GPUs still appear to have capacity.
Automatic profile selection may choose incorrectly
For this release, NVIDIA says automatic selection does not filter profiles by GPU model or per-GPU memory. A machine can therefore attempt a profile that does not match its cards or objective and fail at startup. List the profiles and set NIM_MODEL_PROFILE explicitly to the profile whose GPU type, precision and GPU count match your installation.
Best Value
- Powered by the NVIDIA Blackwell architecture and DLSS 4 OC mode: 2640MHz/Default mode: 2610MHz (Boost Clock)
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.125-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Using all visible memory leaves no runtime margin
Frameworks, communication buffers, CUDA allocations and media processing need space beyond the weights. NVIDIA’s BF16 vLLM example uses --gpu-memory-utilization=0.85. This is a documented starting point that leaves a margin; it is not a universal setting for every context or concurrency level. A higher value can cause startup or runtime collisions when cache allocations arrive.
A practical OOM-avoidance procedure
- Choose precision before buying or assigning GPUs. Select BF16 when you need BF16 behavior, LoRA or customization, or full-precision evaluation. Select the official NVFP4 profile when supported hardware and quantized inference are acceptable.
- Match tensor parallelism to a published profile. Use the exact GPU model and count named by NVIDIA. Adding an arbitrary number of cards does not create a supported configuration, and mixing cards can leave the profile unmatched.
- Enumerate available profiles. In the NIM environment, run
list-model-profiles. Record each profile’s precision, tensor-parallel size, GPU requirement and any agentX or media settings. - Set the profile explicitly. Assign
NIM_MODEL_PROFILEto the profile name or ID that matches your GPUs and workload. Do not rely on automatic selection when multiple profiles are present. - Start with utilization headroom. For a BF16 vLLM deployment, begin with
--gpu-memory-utilization=0.85, then lower it if startup allocations or cache growth collide. Raising it should be a measured change after confirming the actual workload. - Set a realistic maximum context. If requests do not need the full 1 million tokens, lower the maximum context length. Shorten accumulated tool traces and conversation history, and cap maximum output tokens where the application allows.
- Limit simultaneous sequences. Reduce batching or concurrency when OOM occurs only under load. Test the longest expected prompt together with the target number of concurrent requests, rather than testing each variable separately.
- Control media inputs. Limit image count, image resolution and video frame count. Keep media requests from sharing a queue with unusually long text requests when latency or memory is tight.
- Budget host RAM for agentX. Provide 200 GiB per GPU, or 400 GiB for the two-GPU H100 agentX profile, in addition to normal operating-system and container memory. Monitor system memory and swap, not only GPU telemetry.
- Separate storage from VRAM planning. Model-cache disk capacity only determines whether files can be stored. It does not prove that a GPU can load or serve the model.
How to choose between BF16 and NVFP4
| Decision factor | BF16 | NVFP4 |
|---|---|---|
| Memory approach | Requires the published multi-GPU layouts. | Lower weight footprint; NVIDIA lists one- and two-GPU profiles. |
| Customization | Preferable when BF16 behavior, LoRA or full-precision evaluation is required. | Use only when quantized inference and the supported profile meet your quality and tooling needs. |
| Typical published examples | TP2 at 119 GB per GPU, TP4 at 61 GB, or TP8 at 32 GB per GPU. | TP1 at 72 GB or TP2 at 38 GB per GPU on named hardware. |
| Operational caveat | Still needs cache and media headroom beyond the floor. | Quantization does not remove cache, decoding or host-RAM requirements. |
The right comparison is not simply “which number is smaller.” Include the GPU model and count, context length, concurrent sequences, image/video mix, host memory, and whether NVIDIA verifies the exact profile. A lower per-GPU floor can still be the wrong choice if your application needs a larger cache or unsupported customization.
Diagnosing common failure patterns
OOM during startup
- Confirm that
NIM_MODEL_PROFILEnames a profile for the installed GPU and precision. - Check tensor-parallel size against the physical GPU count.
- Compare available VRAM with the profile’s startup floor, not the card’s advertised capacity.
- Lower
--gpu-memory-utilizationfrom the 0.85 starting point if framework allocations collide with loading.
OOM after a long prompt or large batch
- Lower maximum context length and output-token limits.
- Reduce concurrent sequences or batch size.
- Trim tool results and old conversation turns before they enter the context.
- Measure cache growth while reproducing the longest realistic request.
Failures only on image or video requests
- Verify that the 10-GiB media reservation is available on non-DGX-Spark systems.
- Reduce media count, resolution or video frames.
- Queue media-heavy requests separately if simultaneous decoding exhausts available capacity.
- Do not assume that text-only tests validate a vision workload.
The host becomes unresponsive while GPUs look healthy
- Check whether an agentX profile is active.
- Reserve 200 GiB of host RAM per GPU, or 400 GiB for the two-GPU H100 profile, plus container and operating-system needs.
- Watch for swapping; disk-backed swap can make an apparently successful service unusably slow.
Deployment checklist
- GPU model and count exactly match a documented profile.
- Precision is intentional: BF16 for the required customization or evaluation path, NVFP4 for supported quantized inference.
list-model-profileswas checked andNIM_MODEL_PROFILEwas set explicitly.- Startup VRAM floor is treated as a minimum, with additional space reserved for KV cache.
- Maximum context, output tokens and concurrent sequences reflect the real workload rather than the 1-million-token ceiling.
- Image/video reservation and media concurrency are included in capacity planning.
- AgentX host-memory requirements are provisioned when CPU KV-cache offload is enabled.
- Model-cache disk capacity is not being mistaken for GPU memory.
Bottom line
For BF16, plan on a multi-GPU deployment: NVIDIA’s published floors are 119 GB per GPU for TP2 on selected B200/B300/GB200/GB300 systems, 61 GB per GPU for TP4 on H200, and 32 GB per GPU for TP8 on H100 80GB. For NVFP4, the documented floors are 72 GB on one supported high-memory GPU or 38 GB per GPU across two supported A100 80GB, H100 80GB or RTX PRO 6000 Blackwell Server Edition GPUs. None of these figures includes unlimited context, concurrency or media. Explicit profile selection, conservative utilization, controlled cache demand and adequate host RAM are what turn a nominally compatible machine into a stable deployment.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
