The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
It is possible to exceed 100 generated tokens per second with Qwen3.8-Flash-Next on an RTX 4090 in a published community benchmark—but that result came from a specific quantized setup, not from the 24 GB graphics card alone. The reported machine paired the card with a Ryzen 7900 and 192 GB of system RAM, and used an unpruned GSQ IQ3_S model configuration. NVIDIA’s documented deployment recipes target much larger Blackwell GPUs instead.
What “100T/s” means for this model
Here, “100T/s” means about 100 generated tokens per second, usually written as tok/s. It is a decode-speed target, not a claim about the model’s parameter count or the GPU’s theoretical computing rate.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
VIPERA NVIDIA GeForce RTX 4090 Founders Edition Graphic Card | $4,425.00 | Buy on Amazon |
| 2 |
|
ASUS TUF Gaming GeForce RTX™ 5080 16GB GDDR7 OC Edition Graphics Card | $1,830.91 | Buy on Amazon |
Decode is the stage where the model generates output one token at a time. Prompt processing, or prefill, is a separate workload. The community project reports roughly 4,700–4,800 tok/s for uncached 32K-token prefill; that figure cannot be compared directly with a decode result. Time to first token and aggregate throughput across multiple users are also different measures.
What Qwen3.8-Flash-Next is
Qwen3.8-Flash-Next is a multimodal sparse mixture-of-experts model. Qwen and NVIDIA documentation describe it as having 125 billion total parameters, with 6 billion active per token, plus a 51-billion-parameter n-gram embedding table. The 125B label therefore does not mean that all 125 billion parameters are activated for every generated token.
#1 Best Overall
- 16,384 NVIDIA CUDA Cores
- Supports 4K 120Hz HDR, 8K 60Hz HDR and variable refresh rate as indicated in HDMI 2.1A
- New streaming multiprocessors: up to 2x power and power efficiency
- Fourth generation tensor cores: up to 2x AI power
- Third-generation RT cores: up to 2x ray tracing performance
Its attention design combines Gated DeltaNet (GDN) with Qwen Sparse Attention (QSA): three out of every four layers use GDN, while the remaining layer uses QSA. This model-specific design is relevant to deployment because the official NVIDIA recipe calls for GDN/QSA kernels; ordinary memory estimates or results for a different model do not establish that this model will run at the same speed.
Context length
The documented native context length is 262,144 tokens. Qwen documentation describes extending context to 1 million tokens with YaRN. The model card recommends changing YaRN configuration for long-context use; a static YaRN configuration can affect shorter text as well. A supported context limit is not a promise that every prompt of that length will fit in a particular setup or retain the same generation speed.
What the RTX 4090 benchmark actually reports
A community project reports results from one system with a 24 GiB RTX 4090, a Ryzen 7900, and 192 GB of RAM. Its baseline used an unpruned GSQ IQ3_S configuration. In six diversified, low-reasoning runs capped at 512 generated tokens, the project reports median decode speeds of 110.8 tok/s for its text profile and 97.65 tok/s for its GPU-vision profile.
Those are profile-specific results from that project, not a general benchmark standard or an independently reproduced guarantee. The text-profile median is above 100 tok/s; the GPU-vision baseline median is below it. The repository also cautions that decode counts reasoning and answer tokens, each run stops at 512 tokens, first-token latency is not a completed-answer measurement, and the compared columns have different cache histories.
Other reported measurements are not interchangeable
The same project reports a later matched A/B comparison of 114.85 tok/s for its text profile and 104.25 tok/s for its GPU-vision profile. It separately reports matched-context probes of 120.10 tok/s at 32K and 112.55 tok/s at 128K. These are distinct measurements; they should not be substituted for the baseline medians or treated as proof that every 4090 configuration will sustain those speeds.
For context, NVIDIA reports more than 16,000 tokens/s per GPU and more than 200 tokens/s per user on GB300 NVL72. That is a rack-scale Blackwell result, not an RTX 4090 result, and it is not a like-for-like comparison with the single-system community decode measurements.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Why NVIDIA’s documented path is different
The NVIDIA deployment guides reviewed for this model describe Blackwell-based configurations with far more GPU memory than an RTX 4090 provides. NVIDIA’s Dynamo Qwen3.8-Flash-Next recipe describes its Inferact NVFP4 quantization as requiring approximately 130 GB of GPU memory minimum, with 51 GB of host RAM per worker. NVIDIA’s TensorRT-LLM guide lists one Blackwell GPU with 100 GB of GPU memory and 96 GiB of host memory for its NVFP4 configuration.
Recommended Free Tools
| Deployment path | GPU memory and hardware | Host memory | What the figures describe |
|---|---|---|---|
| NVIDIA Dynamo recipe | Approximately 130 GB GPU memory minimum | 51 GB per worker | NVIDIA’s recipe-specific Inferact NVFP4 path with GDN/QSA kernels |
| NVIDIA TensorRT-LLM guide | One Blackwell GPU with 100 GB GPU memory | 96 GiB | The guide’s NVFP4 configuration |
| Community RTX 4090 setup | One 24 GiB RTX 4090 | 192 GB system RAM | The project’s unpruned GSQ IQ3_S benchmark setup |
NVIDIA’s cited minimums are specific to those official recipes; they are not universal requirements for every third-party quantized engine. The community 4090 result is a different deployment approach, with substantially more host memory and a different quantization path. NVIDIA specifies 24 GB of GDDR6X memory and 1,008 GB/s memory bandwidth for the RTX 4090.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What a 4090-class local setup needs to reproduce the example
The benchmark is evidence that one carefully configured 4090 system can cross the target in a particular text-generation profile. It does not establish a turnkey NVIDIA-supported 4090 recipe. A reasonable replication effort should account for the whole system and match the benchmark conditions as closely as possible.
- GPU: The reported machine used a 24 GiB RTX 4090. The published RTX 4090 specification lists 24 GB GDDR6X memory; do not assume a smaller-memory card will behave identically.
- Host: The reported system had a Ryzen 7900 and 192 GB of system RAM. The GPU model alone is not a complete bill of materials.
- Quantization: The benchmark used unpruned GSQ IQ3_S. Results from another quantization, pruning method, or model build are not directly comparable; quantization can change both memory use and output quality.
- Model-aware software: Check that the selected inference engine supports this model’s GDN/QSA architecture and the intended vision path. The official NVIDIA recipe explicitly names GDN/QSA kernels, but its stated memory requirements are for its Blackwell/NVFP4 deployment path.
- Profile and prompt: Record whether vision is enabled, the prompt/context length, cache state, and whether the number is decode, prefill, or time to first token.
- Benchmark procedure: Record the number of runs, output-length cap, reasoning setting, and GPU power limit. The community baseline used six diversified low-reasoning runs capped at 512 output tokens, so a different test protocol may produce a different result.
The available benchmark description does not provide a universal installation command or a guaranteed set of engine settings for every RTX 4090 system. Use the community project’s own setup instructions and verify model, quantization, engine, and profile details before treating a local result as a reproduction.
How to judge whether your result is comparable
A single speed number is useful only when its conditions are visible. When evaluating a local run or another published result, compare these variables:
- Memory and architecture: usable GPU memory, GPU generation, and how much model state is kept on the GPU versus handled through host memory.
- Model representation: exact quantization and whether the model is pruned.
- Software path: inference engine and support for the model’s GDN/QSA kernels and vision mode.
- Workload: prompt length, configured context, output length, reasoning behavior, and whether vision is active.
- Metric: decode tokens per second, prefill tokens per second, first-token latency, or multi-user aggregate throughput.
- Test conditions: cache state, run count, power limit, and whether the result is a median or a single run.
If those details differ, a higher tok/s figure does not necessarily mean the system is faster for your use case. In particular, a short capped decode test does not establish the time required to process a long prompt or finish a long answer.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
