Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
In one reported RTX 4090 test, Llama 3 8B Instruct Q4_K_M generated at 115.2 tokens per second in a virtio-nvgpu KVM guest—95.6% of the 120.5 tokens per second reported for bare metal. The same test reported 119.8 tokens per second with PCI passthrough. These are results for one workload and software configuration, not a general performance guarantee.
What the benchmark reports
A September 24, 2026 article by Umair Bilal at BuildZn reports average throughput and average time to the first generated token for three RTX 4090 configurations. The article’s 95.6% figure is the ratio of its virtio-nvgpu throughput average to its bare-metal average: 115.2 ÷ 120.5.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
MINISFORUM MS-02 Ultra Workstation Mini PC, Intel Core Ultra 9 285HX (24C/24T, up to 5.5GHz), PCIe... | $1,659.00 | Buy on Amazon |
| 2 |
|
GMKtec EVO-X2 AI Mini PC Ryzen Al Max+ 395 Superchip 128GB LPDDR5X 2TB SSD | $3,649.99 | Buy on Amazon |
| Configuration | Average throughput | Average first-token latency |
|---|---|---|
| Bare metal | 120.5 tokens/s | 150 ms |
| PCI passthrough | 119.8 tokens/s | 155 ms |
| virtio-nvgpu KVM guest | 115.2 tokens/s | 168 ms |
In this set of averages, PCI passthrough was 0.7 tokens/s below bare metal, while virtio-nvgpu was 5.3 tokens/s below bare metal and 4.6 tokens/s below passthrough. First-token latency tells a different part of the story: the guest’s reported 168 ms was 18 ms higher than bare metal and 13 ms higher than passthrough. Throughput and first-token latency are separate measures; neither alone describes every aspect of an inference session.
Which workload and system produced those numbers?
The BuildZn article describes a single-host setup and an LLM inference workload. Its reported figures should be read in that context rather than as results for virtio-nvgpu generally.
#1 Best Overall
- High-Performance AI Processor：The MS-02 Ultra features an Intel Core Ultra 9 285HX (24C/24T, up to 5.5 GHz, 13 TOPS NPU), delivering fast and efficient performance for AI inference, algorithm development, and media workloads. A PCIe x16 expansion slot supports desktop-class GPU upgrades for advanced model training and accelerated computing tasks. It's ideal for creators, engineers, and teams handling intensive parallel workloads.
- 4 × M.2 PCIe 4.0 + 4 × DDR5 SODIMM slots:Four DDR5 SODIMM slots support up to 256 GB of memory, while ECC helps maintain data integrity in mission-critical environments. Four PCIe 4.0 M.2 slots support up to 24 TB of storage, supporting RAID 0/1/5/10, combining high-speed performance with data protection. It allows for the creation of independent scratch disks, media libraries, and project drives, providing high-throughput for production workflows.
- PCIe & USB 4.0 v2: Up to three PCIe slots can be equipped, including a dual-slot x16 GPU. The main slot supports PCIe 5.0, meeting the needs of high-bandwidth creative and computing workloads. USB 4.0 v2 (80Gbps) supports high-bandwidth external storage and displays.
- Ultra-fast Networking: Wi-Fi 7 further enhances wireless performance with next-generation speeds and low-latency stability. Intelligent bandwidth switching optimizes throughput in different network environments, ensuring optimal performance for enterprise or local networks. Dual 25GbE ports (providing up to approximately 3.125 GB/s bandwidth, about 25 times faster than traditional 1GbE), enabling seamless large-scale file transfers and parallel computing. 10GbE and 2.5GbE ports, with support for Intel vPro technology, ensure enterprise-grade remote management and deployment flexibility.
- Server-grade thermal architecture: Utilizing a dedicated CPU/GPU airflow design, equipped with a 6-pipe dual-fan cooler, it maintains stable performance even under sustained loads, delivering up to 140W Turbo power while maintaining a 100W TDP, and operating with noise levels as low as 36 dB. An integrated 350W power supply ensures stable and reliable output for demanding computing tasks and fully loaded extended configurations.
Host and guest configuration
- Host: custom system with an AMD Ryzen 9 7950X, 64 GB of DDR5 memory, an NVIDIA GeForce RTX 4090, and Ubuntu 22.04.
- Guest: KVM virtual machine with 16 vCPUs, 32 GB of RAM, Ubuntu 22.04, and a custom Linux 6.8 kernel with a virtio-nvgpu driver.
- GPU software path: the article describes a custom QEMU build and NVIDIA user-mode libraries in the guest.
Model and request pattern
The workload used Llama 3 8B Instruct quantized as Q4_K_M through Ollama/llama.cpp. The article says it sent 100 requests, each with an approximately 100-token input and a target of approximately 200 output tokens. It describes measuring tokens per second and first-token latency.
The published text does not provide raw observations for individual runs or enough detail about token accounting to reproduce the exact throughput calculation. It also gives averages without run-to-run distributions or confidence intervals. The reported precision should not be mistaken for evidence that every run, prompt, or deployment will achieve the same result.
What virtio-nvgpu is doing in this comparison
The virtio-nvgpu project architecture document describes a virtio device and guest kernel driver that forward NVIDIA kernel-driver ioctls between a KVM guest and its host. Its stated aim is to let the guest use NVIDIA user-mode drivers without translating graphics or compute calls at a higher API layer. The project documentation describes the approach as “Near-native NVIDIA GPU access inside a KVM guest.”
That architecture explanation is not an independent validation of the LLM result. The project document discusses GPU rendering performance, including a separate 95–99% bare-metal statement in its rendering context. That figure concerns a different workload and should not be treated as corroboration of the BuildZn LLM throughput averages.
Rank #2
- EVOLUTION RYZEN AI MAX+ 395 MINI PC - GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series. Thanks to AMD Simultaneous Multithreading (SMT) the core-count is effectively doubled, to 32 threads. Ryzen AI Max+ 395 has 64 MB of L3 cache and can boost up to 5.1 GHz, depending on the workload. The Ryzen AI Max+ 395 is currently rated as the "most powerful x86 APU" on the market for AI computing.
- AI NPU with XDNA 2 ARCHITECTURE - Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive integrated GPU driven by 40 AMD RDNA 3.5 CUs, the Ryzen AI MAX+ 395 is a transformative upgrade and delivers a significant performance boost over the competition. The Ryzen AI Max+ 395 excels in consumer AI workloads like the llama.cpp-powered application: LM Studio. Shaping up to be the must-have app for client LLM workloads, LM Studio allows users to locally run the latest language model without any technical knowledge required and unleash their creativity and productivity.
- AMD RADEON 8090S iGPU GAMING PC - The AMD Radeon RX 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the new RDNA 3.5 architecture. The powerful iGPU is positioned between an RTX 4060 and 4070 laptop GPU and therefore enables gaming in FHD at maximum details in most demanding games. The 8060S can also utilize the full 128GB pool, which is perfect for running LLMs such as Deepseek 70B Q8, which runs comfortably on this machine.
- EIGHT CHANNEL LPDDR5X - LPDDR5X is a new ground breaking memory small form factor installed on-board. With blazing speeds up to to 8000MT/s, it runs 1.5x faster than the DDR5 SODIMMs; 90% better performance over DDR5 SODIMMs in video conferencing and photo editing; 30% better performance in productivity apps; 12% better performance in digital content workloads.
- QUAD SCREEN 8K DISPLAY SUPPORT - EVO-X2 AI Mini PC support 4-screen 4K/8K output via HDMI 2.1 (8K@60Hz), DisplayPort 1.4 (4K@60Hz), and dual USB 4 40Gbps Transfer speed (supporting PD3.0/DP1.4/DATA). Ideal for gaming, video editing, and multitasking, it provides expansive and crisp multi-display support.
What the benchmark does—and does not—establish
- It reports a result for one defined combination: RTX 4090, Llama 3 8B Instruct Q4_K_M, Ollama/llama.cpp, and the described custom KVM, QEMU, kernel, and driver setup.
- It does not establish a universal “95%” performance level. Different GPUs, driver interfaces, kernels, QEMU revisions, model settings, and inference workloads could produce different results.
- It is not an independent replication. The available project architecture material covers rendering rather than independently rerunning this LLM benchmark.
- It does not supply a public benchmark harness, run-level results, or confidence intervals in the retrieved article text, limiting how precisely another reader can reproduce or assess the averages.
The BuildZn article characterizes its setup as experimental and describes reliance on custom QEMU and patched guest-kernel components. The available material does not establish an official production-support status or a complete, current compatibility matrix for GPU models, NVIDIA drivers, QEMU revisions, and kernel revisions. Treat setup details as the article’s experience, not universal installation instructions.
How to interpret the separate operational-latency claim
The BuildZn article also claims a 30% reduction in “operational latency” for a multi-tenant workflow, attributing it to faster virtual-machine instantiation and teardown. The text does not clearly define that latency, explain its measurement method, or provide raw data for the percentage. It is a separate, insufficiently documented claim—not a result that can be inferred from the tokens-per-second or first-token figures above.
What to check before relying on the result
For a deployment decision, the useful question is not only whether one reported guest run approached bare-metal throughput, but whether the same software path works reliably for your system and workload. Confirm current compatibility information for the exact GPU, driver, kernel, and QEMU versions you intend to use; then compare against a controlled baseline using the same model, quantization, prompt and output lengths, and measurement method. Record repeated runs and distributions rather than relying only on averages. The supplied reports do not settle those broader compatibility or reproducibility questions.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
