Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
The most dependable answer comes from the inference runtime itself. Start a real prompt, then inspect the runtime’s device-placement report while the model is loaded. Confirm that result with live GPU monitoring during generation. A model that is merely installed, or a GPU graph that happens to be quiet, cannot by itself prove where inference is running.
For Ollama, run ollama ps: its PROCESSOR column distinguishes GPU-only, CPU-only and split CPU/GPU placement. On Windows, Task Manager identifies available GPUs and shows activity; NVIDIA users can add nvidia-smi and, where supported, nvidia-smi pmon for process-level evidence.
Use two kinds of evidence, in this order
There are two different questions hidden in “Is my AI model using the GPU?”
- Where did the runtime place the model? The runtime’s status output answers this most directly.
- Is the device doing work right now? Operating-system and vendor monitors show activity during generation.
Use the placement report first, then monitor the device while a prompt is actively generating. This avoids two common errors: assuming that any visible GPU activity belongs to the model, and assuming that a quiet graph means the model is running on the CPU.
#1 Best Overall
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- 0dB technology lets you enjoy light gaming in relative silence
- Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
- Dual ball fan bearings last up to twice as long as sleeve bearing designs
Run an actual inference request
- Open the application or command-line runtime that loads your model.
- Load the model completely.
- Submit a prompt that takes long enough to observe, rather than checking while the model is idle.
- Keep the runtime’s status command and your operating-system or vendor monitor open during token generation.
Inference can be bursty. A monitor sampled between GPU operations may show little or no activity even though the model is using the GPU. Conversely, the GPU may be busy with another application, so activity alone does not identify your model.
What “running on the GPU” actually means
A local model can use system memory, GPU memory, or both. “GPU inference” is therefore not always a binary condition. A runtime may load every model component into GPU memory, keep it in system memory and execute on the CPU, or divide the model between the two. That last arrangement is commonly called partial offload or split placement.
Placement and utilization are different measurements. A placement report tells you where the model was loaded; a utilization graph tells you whether a device is busy during a particular instant. A model can be GPU-placed but show brief bursts of activity, and a GPU can show activity from a browser, game or display process even when the model is CPU-only. Use both measurements, but give the runtime’s explicit placement report priority when the question is where the model resides.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Check Ollama placement with ollama ps
Ollama’s process listing is the clearest first check when Ollama is the runtime. While the model is loaded, open a terminal and run:
ollama psOllama documents that “The Processor column will show which memory the model was loaded into.” Read the value for the model you just queried.
| Ollama processor value | What it establishes | How to describe it |
|---|---|---|
100% GPU |
The model is placed entirely in GPU memory according to Ollama’s status report. | GPU-only placement. |
100% CPU |
The model is placed in system memory according to Ollama’s status report. | CPU-only placement. |
| A mixed CPU/GPU percentage | Ollama has divided the model between system memory and GPU memory. | Partial GPU offload; it is neither fully GPU-resident nor CPU-only. |
The mixed result matters. Treating the answer as a simple yes or no hides how the model is actually loaded. A split model is using the GPU for part of its work, while another part remains in CPU memory. Ollama’s documentation defines the percentages; it does not establish a universal percentage that counts as “enough” GPU use.
Make sure the listing describes the active model
Run ollama ps after the prompt has loaded the model and while it remains resident. If several models are listed, match the model name to the one generating your response. An installed model that is not loaded does not demonstrate active GPU inference.
The percentages in this column should not be read as a GPU-utilization percentage or a speed score. They describe the reported memory placement. A model showing 40% GPU is not necessarily delivering 40% of its possible performance, and there is no universal cutoff at which partial offload becomes “real” GPU inference. Describe exactly what Ollama reports.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070 Ti
- Integrated with 16GB GDDR7 256bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Watch GPU activity in Windows Task Manager
Windows can identify the GPUs available to the operating system and provide a live view while inference runs. Microsoft’s Task Manager guidance also covers GPU and NPU entries; the exact engines and graphs depend on the device and driver.
- Press Ctrl+Shift+Esc to open Task Manager.
- Select Performance.
- Look for entries labeled GPU. On systems with more than one adapter, note which GPU is the discrete device and which is integrated.
- With the model generating text, observe the relevant GPU graphs.
- Where Task Manager offers engine choices, select the compute-related engine rather than relying only on a default 3D graph.
Task Manager is useful for answering “which GPU is present?” and “is a GPU doing work during this prompt?” It is not a substitute for the runtime’s placement report. A graph can be quiet between short bursts, and a busy GPU graph may represent a different process. Engine labels and available graphs vary by driver and hardware.
GPU versus NPU entries
Some newer Windows PCs show both GPU and NPU entries. An NPU is a separate accelerator category, so activity on an NPU graph does not prove that the model is using the GPU. Identify the device named by the runtime and watch that device’s compute activity. If the runtime reports CPU placement, seeing an NPU or GPU listed in Task Manager does not change that placement result.
Free tools Windows power users keep installed
One-click scans. No signup required.
Use process information as corroboration
If Task Manager’s Processes view exposes GPU columns, use them to see whether the application hosting your model appears during generation. Treat that as supporting evidence, not as a definitive placement statement: a process can use the GPU for display or other operations without running model inference there.
Confirm NVIDIA activity with nvidia-smi
On an NVIDIA system, the vendor command-line utility adds device and process information. Start generation, then run:
nvidia-smiUse the output to identify the GPU, current utilization and processes associated with the device. Run the command during active generation; checking after the response has finished can miss short work periods.
Inspect per-process monitoring when available
NVIDIA documents nvidia-smi pmon for reporting process IDs, process names and utilization averages where the platform supports those metrics:
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallnvidia-smi pmonMatch the listed process to the application or service that runs your model. On multi-GPU systems, verify that you are looking at the GPU on which the runtime placed the model.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5060
- Integrated with 8GB GDDR7 128bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Support is not universal. NVIDIA notes that some metrics can be unavailable and that per-process monitoring varies by environment. An unsupported or missing metric is therefore not proof that the model is doing no GPU work. Use the runtime’s placement output and a live observation together.
How the checks compare
| Check | Best evidence for | Scope | Important limitation |
|---|---|---|---|
| Runtime placement report | Whether the model is assigned to GPU memory, CPU memory or both | The target model and runtime | Commands and labels differ by runtime; a report may not show moment-to-moment utilization. |
| Windows Task Manager | Which GPUs Windows sees and whether a device shows activity | Whole system, with process views where available | Activity can be intermittent; graph names and engines vary by driver and device. |
nvidia-smi |
NVIDIA device status, utilization and visible processes | An NVIDIA GPU and its processes | It is vendor-specific and may not expose every metric on every platform. |
nvidia-smi pmon |
Per-process utilization averages where supported | Processes on an NVIDIA GPU | Unsupported metrics or environments can leave the output incomplete. |
If you are not using Ollama
Do not apply ollama ps to another runtime. Each inference engine has its own device-selection setting, status screen or log format. The general method remains the same:
- Identify the exact runtime and backend that launched the model.
- Start a prompt and keep it generating.
- Read that runtime’s device, backend or placement status while the model is loaded.
- Check the operating system or vendor monitor at the same time.
- Compare the model name and process ID, if shown, so you do not attribute another application’s GPU work to the inference process.
If the runtime reports CPU placement while Windows shows a GPU, believe that these are different facts: Windows can see a GPU, but the selected backend may not support it, may not be configured to use it, or may have rejected it during initialization.
Do not infer support merely from the model format or the fact that another application uses the same GPU. GPU compatibility depends on the runtime, backend, hardware, driver, operating system and configuration. Read the current support notes for the specific application before changing a driver or buying hardware.
Interpret the result correctly
“100% GPU” or an equivalent GPU-only report
The runtime says the model is entirely placed in GPU memory. A live GPU observation during generation should normally provide corroborating activity, although a single low reading is not conclusive because work may occur in bursts.
“100% CPU” or an equivalent CPU-only report
The runtime says the model is in system memory and is not using GPU placement for that model. Check backend support, driver status and runtime configuration before changing hardware. A GPU entry in Task Manager does not override the runtime’s placement decision.
A mixed CPU/GPU report
The model is partially offloaded. It is accurate to say that the GPU is being used, but inaccurate to call this full GPU inference. The percentages describe placement as reported by the runtime; they do not provide a universal performance target.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →GPU activity appears, but no model process is listed
Check that monitoring was running during generation, that you selected the correct GPU in a multi-GPU system, and that the monitor supports process reporting in your environment. Other applications can account for the activity.
Rank #4
- Powered by Radeon RX 9070 XT
- WINDFORCE Cooling System
- Hawk Fan
- Server-grade Thermal Conductive Gel
- RGB Lighting
No visible GPU activity
First repeat the check during a longer prompt. Then read the runtime placement report. A short or idle observation is weak evidence; an explicit CPU-only placement report is much stronger.
Troubleshooting by symptom
The runtime reports CPU placement
- Confirm that the exact runtime and selected backend support your GPU model and operating system.
- Review the runtime’s GPU-selection settings and initialization or backend logs.
- Check that the installed driver is suitable for the runtime’s supported configuration.
- Do not assume that a GPU visible to Windows is automatically enabled for inference.
Ollama’s GPU support and troubleshooting documentation specifically discusses GPU initialization, device visibility and configuration. Those notes apply to Ollama’s supported paths; another runtime may require different settings.
The runtime reports split CPU/GPU placement
Report it as partial GPU placement. If you need to understand why the split occurred, inspect the runtime’s memory and device-selection messages rather than inventing a threshold for acceptable GPU use. The documented percentage tells you how placement was divided, not whether the arrangement meets a particular speed goal.
nvidia-smi shows no process
- Run it while tokens are actively generating.
- Check every GPU in a multi-GPU machine and identify the one selected by the runtime.
- Verify that the relevant process-monitoring metrics are supported.
- Use the runtime’s own placement report as the primary answer.
The model runs in Docker or WSL2
Check GPU visibility inside the environment that actually runs inference. Host-level visibility does not prove that a container or subsystem can access the device. Ollama’s documented Docker acceleration path calls for the NVIDIA Container Toolkit on Linux or Windows with WSL2. Follow the runtime’s platform-specific instructions for device passthrough and permissions.
Ollama also documents that Docker Desktop GPU acceleration is unavailable on macOS. This is an Ollama-specific support note, not a rule that applies to every containerized inference application.
Windows status output is inconclusive
For Ollama on Windows, its documentation lists server.log under %LOCALAPPDATA%Ollama, along with other application log locations. Review those logs for GPU initialization, device visibility and backend errors when the GUI and live graphs do not explain the selection.
Do you need to buy anything?
No. Checking placement requires no new graphics card, external GPU, accessory or paid monitoring product. Windows Task Manager, NVIDIA’s utility where applicable, and the runtime’s own status output are sufficient for this diagnostic. A hardware upgrade is a separate performance decision, not a requirement for answering whether the installed model currently uses the GPU.
Common mistakes to avoid
- Checking an idle model: installation or an unloaded model says nothing about active compute.
- Using utilization alone: a quiet graph can miss intermittent work, while a busy graph can belong to another process.
- Calling split placement “GPU-only”: mixed CPU/GPU percentages mean the model is divided across memory types.
- Assuming OS visibility equals runtime support: the operating system can list a GPU that the selected inference backend cannot use.
- Ignoring the selected device: on multi-GPU systems, inspect the adapter where the runtime placed the model.
- Treating missing vendor metrics as a negative result: unsupported process metrics are an instrumentation limitation, not proof of CPU inference.
- Applying one runtime’s command to another: use the status and logs documented for the runtime that launched your model.
- Confusing memory placement with utilization: a percentage in a placement column is not a benchmark result or a promise of a particular speed.
- Reading only a default graphics graph: inference may appear on a compute engine rather than the 3D graph shown first.
A practical verification checklist
- Identify the runtime, backend and operating system.
- Load the model and submit a prompt.
- While generation is active, read the runtime’s placement status.
- Record whether it says GPU-only, CPU-only or split.
- Open Windows Task Manager, if applicable, and identify the GPU entries.
- Watch the relevant compute activity during another prompt.
- On NVIDIA hardware, run
nvidia-smi; usenvidia-smi pmonif supported. - Match the observed process and device to the runtime that launched the model.
- If the results conflict, inspect backend, driver, initialization and container-access logs before drawing a conclusion.
The strongest conclusion combines an explicit placement report with live, correctly targeted monitoring. For Ollama, ollama ps supplies the placement answer; Task Manager and NVIDIA utilities provide corroboration and help identify which device is active.
Best Value
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- Phase-change GPU thermal pad helps ensure optimal heat transfer, lowering GPU temperatures for enhanced performance and reliability
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- Dual-ball fan bearings last up to twice as long as standard conventional sleeve bearings designs
- 0dB technology lets you enjoy light gaming in relative silence
Frequently Asked Questions
Can a model use both the CPU and GPU at the same time?
Yes. A split or mixed placement means part of the model is in GPU memory and part is in system memory. Describe it as partial GPU offload, not GPU-only or CPU-only execution.
Does a low GPU-utilization reading mean the model is running on the CPU?
No. Inference can be intermittent, and a single sample may fall between GPU operations. Check the runtime’s placement report while a prompt is generating, then use live monitoring as corroboration.
Why does Windows show a GPU when my runtime says CPU?
Task Manager only establishes that Windows can see the device. The selected inference backend may lack support, be misconfigured, or have rejected the GPU during initialization. Check the runtime’s compatibility notes, settings, driver status and logs.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteWhat does 100% GPU in Ollama mean?
Ollama’s PROCESSOR column is reporting that the model was loaded entirely into GPU memory. It is a placement statement, not a GPU-utilization percentage or a guarantee of a particular response speed.
Why does nvidia-smi show no model process?
Run it during active token generation, check the GPU selected in a multi-GPU system, and confirm that per-process metrics are supported in your environment. Missing process output can be a monitoring limitation; it is not conclusive proof of CPU inference.
Can I verify GPU use inside Docker or WSL2 from the host?
No. Host visibility does not prove that the container or subsystem can access the device. Check visibility inside the environment running inference and follow the runtime’s device-passthrough requirements. Ollama’s documented Docker path calls for NVIDIA Container Toolkit on Linux or Windows with WSL2.
Do I need a new graphics card or paid tool to perform this check?
No. The runtime’s placement command, Windows Task Manager and vendor utilities such as NVIDIA’s nvidia-smi provide the necessary evidence. Hardware purchases are unrelated to this basic diagnostic.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →The Bottom Line
To check whether an AI model runs on your PC’s GPU, start a real prompt and read the inference runtime’s placement report while the model is loaded. In Ollama, ollama ps distinguishes 100% GPU, 100% CPU and mixed placement. Confirm the result with Task Manager or NVIDIA monitoring during generation, remembering that bursty activity, unsupported metrics and container boundaries can make a single graph misleading.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
