Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
AMD and Zyphra say an AMD-powered cluster trained ZAYA1-base, a mixture-of-experts (MoE) language model with 8.3 billion total parameters and 760 million active parameters. Their published comparison names Llama-3-8B—not Llama 3.1—so the claim that ZAYA1 “smokes Llama 3.1” is not established by the cited results.
What the AMD–Zyphra project produced
ZAYA1-base is a pretrained MoE model introduced in Zyphra’s technical report, Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design, dated November 21, 2025. The report gives it 8.3 billion total parameters and 760 million active parameters. The live Zyphra/ZAYA1-base model card rounds the active count to 800 million; the two figures reflect different source presentations, not a basis for treating them as exactly identical.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
AMD Radeon Instinct MI210 64GB HBM2 300W PCIe Dual Slot Full Height Graphics Accelerator | $5,249.99 | Buy on Amazon |
In an MoE model, a router activates only a subset of its expert parameters for a given token. That means active parameter count is not the same as total model size, nor does either number alone establish model quality. The report describes architectural choices including a ZAYA1 router, Compressed Convolutional Attention, and residual scaling.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesWhat hardware and software trained it
AMD’s November 24, 2025 announcement describes a 128-node cluster. Each node had eight AMD Instinct MI300X GPUs and eight AMD Pensando Pollara 400 interconnects. AMD says ROCm was part of the stack; Zyphra describes IBM Cloud’s high-performance fabric and storage architecture as supporting the jointly engineered infrastructure in its project announcement.
#1 Best Overall
AMD states that the MI300X’s 192 GB of high-bandwidth memory helped reduce the need for expert or tensor sharding. AMD also reports that Zyphra achieved more than 10× faster model save times with AMD-optimized distributed I/O. These are vendor- and project-reported infrastructure claims, not independent cross-platform measurements. AMD’s release says Zyphra performed the cluster testing using a proprietary stack and ROCm 6.4.
What the benchmark comparison actually establishes
The report says ZAYA1-base outperformed Llama-3-8B and OLMoE across its reported reasoning, mathematics, and coding benchmarks, and performed comparably to Qwen3-4B and Gemma3-12B. Its table gives these ZAYA1-base scores:
| Benchmark | ZAYA1-base score reported in the technical report | What it evaluates |
|---|---|---|
| MMLU | 67.01 | Broad knowledge and reasoning across multiple subjects |
| MMLU-Pro | 40.43 | A more challenging, expanded knowledge and reasoning evaluation |
| GPQA | 30.70 | Graduate-level, science-focused questions |
| MATH-hard | 54.15 | Mathematical problem solving |
| MBPP+ | 75.40 | Python programming problems |
These are the report’s published results, not an independent replication. The comparison should be read task by task and checkpoint by checkpoint; the report’s summary does not prove that ZAYA1-base is better on every benchmark or in everyday use. In particular, AMD’s release and the report identify the comparison model as Llama-3-8B. They do not establish a head-to-head result against Llama 3.1 8B, so “smokes Llama 3.1” overstates what these sources show.
Why base-model scores do not equal chat-assistant performance
ZAYA1-base is a base model, not automatically a polished instruction-following assistant. The model card provides inference examples, while the technical report distinguishes base and reasoning-focused checkpoints. A score for the base checkpoint on a benchmark cannot, by itself, tell you how well a finished chat product follows instructions, handles long conversations, or behaves safely. Those questions require results for the particular tuned checkpoint and evaluation that match the intended use.
The model card documents example use through Transformers, vLLM, and SGLang, and says the model uses the Gemma3 tokenizer. It also describes a Zyphra Transformers fork based on Transformers v4.57.1. Since model-card instructions and software dependencies can change, check the live model card for current installation and serving details before deploying.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What this result says about AMD infrastructure
The project is evidence that Zyphra trained this model on a coordinated AMD compute, networking, and software stack, with IBM Cloud infrastructure support. It is not a like-for-like comparison proving AMD is faster or cheaper than another accelerator platform: the cited material does not provide matched cross-vendor throughput, cost, or availability measurements. For infrastructure decisions, the relevant comparison would need the same model workload, cluster scale, network and storage setup, software support, and cost basis.
Zyphra’s announcement characterizes the work as a production-ready alternative and frames it as co-design across models, silicon, and systems. Those are Zyphra’s claims and perspective; the technical report is a case study, not a guarantee that another deployment will reproduce the same performance.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
