Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Yes—you can run a capable local LLM on Proxmox without turning the hypervisor into an experimental desktop. The most dependable design is a Linux virtual machine with a whole GPU passed through to it, then Ollama serving the model and Open WebUI providing a browser interface. A lighter LXC container can share a GPU, but it requires tighter control of host drivers, device permissions and runtime compatibility. A documented build uses Ubuntu 24.04, four vCPUs, 12 GB of RAM, a GTX 1050 Ti and Qwen2.5 3B; another uses a dedicated GTX 1080 Ti VM for larger model families. Your useful model ceiling is set mainly by VRAM and context headroom, not the GPU’s product name.
Why run a local LLM in Proxmox instead of using a cloud service?
A local model keeps prompts, documents and generated responses inside infrastructure you control. It also gives a home lab a reusable service: other machines can call one private API instead of each device running its own model client. Proxmox remains responsible for virtual-machine lifecycle, storage, snapshots and networking while the guest operating system owns the AI stack.
That does not automatically make local inference cheaper, faster or more capable. No controlled measurement in the available documentation establishes latency, tokens per second, electricity use or cloud-cost savings for these builds. Treat those as values to measure on your own hardware, not as guaranteed benefits.
- Privacy boundary: prompts and model files stay on your server unless you deliberately expose the service.
- Operational control: you choose the model version, storage location, update schedule and network access.
- Shared service: a browser interface and API can serve several clients on your LAN.
- Trade-off: you must maintain GPU drivers, kernels, model files, backups and access controls.
What Proxmox architecture works best for a local LLM?
Use a VM when you want the clearest isolation and the fewest surprises. Use an LXC container when conserving overhead matters and you are prepared to manage shared device access yourself.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
| Design | Documented configuration | Strength | Primary complication |
|---|---|---|---|
| Dedicated VM with whole-GPU passthrough | Ubuntu 24.04, 4 vCPUs, 12 GB RAM, GTX 1050 Ti, Ollama and Qwen2.5 3B | Guest owns the complete PCIe device; driver and runtime are isolated from the host | The card is unavailable to the Proxmox host while assigned to the VM |
| Dedicated VM for larger models | GTX 1080 Ti, Ollama models including Llama 3.1, Mistral, Gemma 2 and Qwen 2.5, Open WebUI in Docker | Simple mental model for a persistent AI appliance | Requires a compatible passthrough setup and enough VRAM for the selected model |
| Shared-GPU LXC | llama.cpp with Qwen2.5-7B-Instruct Q4_K_M, approximately 4.7 GB, OpenAI-compatible local API | Lower virtualization overhead and convenient sharing of a host GPU | Host drivers, device permissions and container runtime details are coupled |
The VM pattern is the safer starting point for a first build. LXC is attractive when you already understand Linux device nodes, group permissions and the exact accelerator runtime your card needs.
How do I choose between a whole-GPU VM and a shared-GPU LXC?
Choose the VM if reliability, rollback and isolation matter more than squeezing out a small amount of overhead. Choose LXC only when you have a specific reason to share the device and are willing to troubleshoot host-container integration.
Choose a VM when
- You want the guest to install and update its own CUDA, ROCm or other accelerator stack.
- You need a clean boundary between Proxmox services and AI workloads.
- You expect to rebuild or snapshot the AI environment independently.
- You want the documented Ubuntu 24.04 and Ollama workflow.
Choose LXC when
- The GPU must be shared by several lightweight services.
- You understand which host device files must be exposed and which user or group IDs need access.
- You are comfortable using llama.cpp or another runtime whose container requirements you can verify.
Do not treat LXC as a shortcut around compatibility. A container shares the host kernel, so a host update can affect the inference stack even when the container itself has not changed.
How do I pass a GPU through to an Ubuntu VM in Proxmox?
For inference, pass through the entire PCIe card and let the guest drive it. Proxmox documentation recommends a q35 machine type, OVMF firmware and PCIe passthrough for best compatibility; it also notes that a passed-through GPU’s framebuffer cannot be displayed through the Proxmox web console’s NoVNC or SPICE.
- Check the platform: enable the CPU’s IOMMU feature in firmware (Intel VT-d or AMD-Vi) and confirm that the motherboard places the GPU and its audio function in usable IOMMU groups.
- Enable IOMMU on the host: add the appropriate Intel or AMD IOMMU kernel parameter, load VFIO modules, rebuild the initramfs and reboot. Confirm that the card is bound for passthrough rather than being used by the host display stack.
- Create the VM: in the Proxmox wizard, select OVMF (UEFI) firmware and the q35 chipset. Allocate the planned vCPUs, memory and a virtual disk for the guest.
- Install Ubuntu 24.04: use SSH or the VM’s virtual display for installation. The virtual display is useful before passthrough is attached; it is not a replacement for a physical monitor once the guest owns the GPU.
- Add the PCI device: open the VM’s Hardware tab, choose Add > PCI Device, select the GPU and, when presented, its associated audio function. Enable the PCIe option and the full-device assignment rather than selecting a mediated slice.
- Boot and verify: connect by SSH, install the guest’s matching driver/runtime, and check that the operating system sees the card. If the accelerator is missing, stop and fix IOMMU grouping, VFIO binding or guest driver compatibility before installing model software.
Because the web console cannot show the passed-through framebuffer, a black or empty NoVNC window does not prove that inference failed. Use the guest API, SSH or a remote-desktop solution when you need graphical access.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
How do I install Ollama and serve a model in the VM?
Install Ollama inside the Linux guest, not on the Proxmox host. Then pull a model that fits the available VRAM and test the service locally before adding a browser front end.
- Prepare the guest: update Ubuntu, install the guest GPU driver/runtime required by your card, and reboot if the driver package requests it.
- Install Ollama: follow Ollama’s current Linux installation procedure inside the VM. Keep the service enabled so it starts after a guest reboot.
- Run a first model: use
ollama run qwen2.5:3bfor the documented GTX 1050 Ti example. The first run downloads model data into the guest, so allow enough disk space in addition to RAM and VRAM. - Confirm acceleration: restart the Ollama service after the GPU driver is installed. In the documented VM, Ollama detected CUDA after a restart. If it falls back to CPU, inspect the guest driver and service logs before changing model settings.
- Test the API: send requests to the Ollama service on the guest’s private address. Keep the listener on loopback or the private LAN unless you have deliberately designed authentication and an encrypted gateway.
Model names are not interchangeable in resource use. A 3B model is a sensible smoke test, while a quantized 7B model can require several gigabytes before context and runtime overhead are included.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11How do I add Open WebUI without exposing the model?
Run Open WebUI in the same guest or a trusted internal container and point it at the guest’s Ollama endpoint. A documented homelab setup runs Open WebUI in Docker alongside Ollama, giving users a browser workflow while the model server remains on the private network.
- Install Docker in the Linux guest or place the interface in a container that can reach the Ollama service.
- Configure the interface’s Ollama connection using the guest’s private hostname or address, not a public internet address.
- Create user accounts and restrict administrative access to the LAN or a private overlay network.
- Test a prompt through the browser, then test the API separately so you know whether a failure is in the interface or the model server.
A Tailscale walkthrough published April 2, 2025 combines Proxmox PCIe passthrough, Ollama, Open WebUI and Tailscale for remote access. Use that pattern as a network design, not as permission to publish an unauthenticated inference endpoint. Remote access is a separate decision from GPU passthrough: keep Ollama private unless an authenticated gateway is in front of it.
What GPU do I need for a home-lab LLM?
Start with the model’s memory requirement, then add headroom for context length, runtime overhead, concurrent users and the operating systems. VRAM is the practical ceiling; a familiar product name does not tell you how large a model will fit.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
| Workload example | Published memory guidance | Hardware pairing in the documentation | How to interpret it |
|---|---|---|---|
| Qwen2.5-7B-Instruct Q4_K_M | Approximately 4.7 GB for the quantized model | Shared-GPU LXC with llama.cpp | Leave additional capacity for context, runtime and other processes |
| 14B-class models | Approximately 10–12 GB | RTX 3090 with 24 GB; indicative $600–800 range in a 2026 passthrough guide | Guidance, not a benchmark or a guaranteed street price; 24 GB leaves more operating room |
| Qwen2.5 3B | Used in the documented VM example | GTX 1050 Ti passed through to Ubuntu 24.04 | A practical starter workload for validating the complete passthrough and serving path |
The 10–12 GB figure for 14B models is an indicative configuration range, not a promise that every 14B checkpoint or context length will fit. Quantization reduces memory use but can affect quality and does not eliminate overhead. If several users will generate simultaneously, reserve more capacity or expect requests to queue.
Do Intel and AMD GPUs work the same way?
No. Accelerator support is vendor- and version-specific, so verify the exact card, host kernel, guest driver and inference runtime before buying hardware.
Intel Arc
An Intel guide from June 2024 uses IPEX-LLM for Arc GPU serving and documents environment variables for the runtime. An Ollama maintainer issue dated September 24, 2024 stated that Intel GPU support was not officially part of Ollama at that time. That combination means an Arc build may require a different serving stack or a later compatibility change than a CUDA-based example.
AMD Radeon
An AMD article dated October 19, 2026 documents ROCm-based Proxmox passthrough on current AMD hardware. “Current” is important: ROCm support depends on the card generation, kernel and userspace packages. Confirm the exact matrix for your chosen card and distribution instead of assuming that a working NVIDIA configuration translates directly.
NVIDIA CUDA
The documented GTX 1050 Ti VM picked up CUDA after an Ollama restart, and the GTX 1080 Ti homelab uses Ollama with several model families. These examples show a workable path, not a universal guarantee for every driver release or GPU variant.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
What usually breaks, and how do I recover?
The VM boots but the GPU is missing
Check that IOMMU is enabled, the card is bound to VFIO on the host, and both the graphics and audio functions were assigned. Compare the guest driver version with the runtime expected by Ollama or llama.cpp.
Ollama uses the CPU after passthrough
Confirm the guest can see the device before troubleshooting the model. Restart Ollama after installing or changing the GPU driver, then inspect service logs for runtime initialization errors. A successful VM boot alone does not prove CUDA or ROCm is available.
NoVNC shows no GPU output
This is expected for a passed-through framebuffer. Use SSH, the local API or a remote desktop instead of treating the Proxmox console as a physical monitor.
The model runs out of memory
Reduce context length or choose a smaller or more aggressively quantized model, then retest with one user. If the workload is expected to be concurrent, increase VRAM or move to a GPU with more headroom rather than relying on swap.
Recommended Free Tools
The LXC container cannot open the device
Review the host device paths, container permissions and group ownership. Shared-GPU containers inherit more of the host’s driver assumptions, so a host kernel or driver update can require corresponding container changes.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Remote clients cannot connect
Check the guest listener address, firewall rules and private-network routing separately. Do not solve a connectivity problem by opening the API to the public internet without authentication and encryption.
Is a local LLM on Proxmox worth the effort?
It is worth it when you value private, always-available inference and enjoy maintaining a small service. The strongest first project is a dedicated Ubuntu VM with whole-GPU passthrough, a modest model and local-only access. Once that path is stable, add Open WebUI and remote access as separate changes.
- Good fit: summarizing private notes, experimenting with models, coding assistance on a LAN, and providing one internal API to several devices.
- Less suitable: workloads that require a very large model, guaranteed cloud-scale uptime, or a proven latency target you have not measured on your hardware.
- Best first milestone: Qwen2.5 3B on the documented Ubuntu 24.04 VM pattern, followed by a measured test with your own prompts and context lengths.
Keep Proxmox focused on virtualization, keep the model stack inside the guest, and treat GPU compatibility and network exposure as independent engineering decisions. That separation is what makes the home lab understandable instead of turning every update into a host-wide experiment.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Frequently Asked Questions
Can I use the passed-through GPU for Proxmox services at the same time?
A whole-device passthrough assignment gives the card to the guest, so the Proxmox host should not rely on that GPU simultaneously. If you need host graphics, plan for another display adapter or run the host headlessly.
Do I need Open WebUI to use Ollama?
No. Ollama can serve models through its local API without a browser interface. Open WebUI is an optional client layer for accounts, chat history and easier multi-device access.
Can I run a 14B model on a 12 GB GPU?
The cited guidance places 14B models around 10–12 GB before extra headroom. Whether a specific checkpoint fits depends on quantization, context length and runtime overhead, so a 12 GB card leaves little margin for larger contexts or multiple users.
How should I measure whether my build is fast enough?
Record prompt and generation timing for a fixed model, context length and number of concurrent requests, and note whether the runtime used the GPU. Repeat after driver or model changes; the available documentation does not provide a controlled latency or tokens-per-second result.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
