iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
On the reported four-core server, one question took 580 ms at p50 with Laya’s English checkpoint, 193 ms with its multilingual checkpoint, and 584 ms with typed-decisions. Those are benchmark results for a specific machine and software version—not a performance guarantee for other CPUs or deployments.
What the four-core benchmark measured
The Laya project benchmark ran in-process on an AWS m7a.xlarge with an AMD EPYC 9R14, four physical cores, no simultaneous multithreading (SMT), and 16 GiB of RAM. It used Laya v0.3.20, fp32, and OMP_NUM_THREADS=4. Each call alternated between a three-option choice question and a noul question. The reported times are p50; the benchmark guide says p95 was within 2% of p50 for every row.
The project’s BENCHMARKS.md says, “Every checkpoint answered byte-identical questions in each run (fixed seed).”
| Checkpoint | 1 question | 5 questions | 10 questions | 50 questions | Cold load |
|---|---|---|---|---|---|
laya (English) |
580 ms | 3,072 ms | 6,244 ms | 35,969 ms | 4.4 s |
laya-multilingual |
193 ms | 912 ms | 1,842 ms | 11,157 ms | 2.5 s |
laya-typed-decisions |
584 ms | 2,819 ms | 6,031 ms | 35,653 ms | 0.5 s |
Timings and cold-load values are transcribed in the Laya benchmark guide. Cold loads are approximate and depend on the operating system’s file cache.
#1 Best Overall
- For AMD EPYC 9754 128 Core Bergamo 2.25GHz (100-000001234) EPYC 9004 Series Socket SP5 ZEN4 256MB L3 Bulk / Tray Pack (Unlocked) Server Processor
How to interpret the latency
Checkpoint choice changes the result
In this run, the multilingual checkpoint answered one question in about one third the time of the English checkpoint. The guide connects the difference to encoder size: mmBERT-base has 322 million parameters, compared with 421 million for ModernBERT-large. The benchmark does not break down how much of the timing gap comes from model size or other factors. Treat the roughly threefold difference as a result from this machine and test, not a general rule.
More questions mean more elapsed time
For calls with one to ten questions, elapsed time grows roughly with the number of questions. Ten questions took 6.244 seconds for English, 6.031 seconds for typed-decisions, and 1.842 seconds for multilingual. At 50 questions, the reported total works out to about 15–20% more time per question than in the smaller batches. Batching therefore offered little per-question saving in this test.
Rank #2
- Powerful Hardware: The F6-424 NAS storage features an N95 quad-core 4-thread CPU 3.4GHz (turbo), integrated UHD GPU at 0.75GHz, 8GB DDR5 4800MHz memory (non-ECC, upgradable up to 32GB), dual 2.5G Ethernet ports, and dual M.2 NVMe slots for SSD caching.
- Enhanced Application Speed: The F6-424 NAS enclosure, equipped with a powerful CPU and DDR5 memory, achieves a remarkable 140% performance boost over its predecessors. Experience 90% faster application load times, a 40% increase in file and photo retrieval speed, a 50% improvement in database response speed, and a outstanding 60% faster PHP response for web pages. Elevate your workflow efficiency with seamless high-load performance.
- High-Speed Connectivity: The F6-424 NAS case features dual 2.5 GbE interfaces and can achieve a linear data transmission speed of 283 MB/s (tested with NAS dedicated HDDs 18TB x 4, RAID 0), Link Aggregation offers a network bandwidth up to 5 Gb, ideal for multi-user and high-concurrent file access. Featuring dual USB 3.2 10Gbps interfaces, data transfer between your NAS and external devices is made swift. The F6-424 supports various RAID types, including RAID 0/1/5/6/10/JBOD/Single, TRAID and TRAID+.
- Effortless M.2 SSD Installation: TerraMaster streamlines M.2 SSD handling in the F6-424 NAS server with a side-sliding cover, allowing quick and simple SSD installation. The inclusion of hand-tightened screws ensures a hassle-free installation process. The F6-424 NAS adopts a tool-free hard disk tray, eliminating the need for additional tools during hard disk installation. Additionally, TerraMaster's innovative Push-Lock design automatically secures inserted hard disks, preventing accidental disconnection or offline issues.
- Comprehensive Business Backup Solution: The F6-424 backup server comes with TerraMaster Business Backup Suite (BBS) which is an enterprise-grade solution that includes Centralized Backup for data consolidation, TerraSync for server and PC synchronization, Duple Backup for off-site recovery, CloudSync for cloud recovery, and Snapshot for ransomware protection. BBS offers flexible, high-performance backup strategies tailored for small and medium-sized businesses.
Cold loading is separate from answering
The cold-load figures are startup costs, not steady-state question latency. A service that loads a checkpoint when it starts can avoid making its first user wait for that load, provided the checkpoint remains available in memory.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →CPU timings are not HTTP response times
The four-core figures are in-process measurements. They do not include the added effects of HTTP handling, request queues, or deployment concurrency, so a user-facing service can behave differently. The benchmark guide also reports a separate GPU comparison: one-question p50 was 39.5 ms for laya and 32.8 ms for laya-multilingual on a Tesla T4. These are GPU results, not CPU timings, and the hardware differs from the four-core server.
Rank #3
- 4 cores (4 P-cores + 0 E-cores) and 8 threads. Integrated Intel UHD Graphics 730 included.
- Performance two core microarchitecture, prioritizing and distributing workloads to optimize performance
- Up to 4.7 GHz unlocked. 12MB Cache
- Compatible with Intel 600-series (with potential BIOS update) and 700-series chipset-based motherboards
- PCIe 5.0 & 4.0 support. DDR4 and DDR5 Memory support. RM1 thermal solution included.
A separate laptop test illustrates why thread configuration matters. On a Ryzen 9 6900HX under WSL2, a three-question HTTP call took 9,396 ms at p50 with the stated default thread settings, then 783 ms after PyTorch intra-op threads were set to 8 and inter-op threads to 1. In an in-process one-question test on that laptop, latency was 910 ms with one thread, 374 ms with four, 329 ms with eight, and 388 ms when every vCPU was used. These laptop measurements are not directly comparable to the four-core server benchmark, but they show that thread settings can materially affect observed performance.
What the results imply for CPU deployment
The benchmark guide’s interpretation is that CPU deployment may suit background classification and some interactive multilingual routing, while English calls containing several questions may feel slow. For high-throughput, low-latency serving, it recommends evaluating a GPU. These are deployment judgments based on the reported results, not outcomes guaranteed by the benchmark.
Rank #4
- Game Without Compromise. Play harder and work smarter with Intel Core 14th Gen processors
- 20 cores (8 P-cores plus 12 E-cores) and 28 threads. Integrated Intel UHD Graphics 770 included
- Up to 5.6 GHz with Turbo Boost Max Technology 3.0 gives you smooth game play, high frame rates, and rapid responsiveness
- Compatible with Intel 600-series (with potential BIOS update) or 700-series chipset-based motherboards
- DDR4 and DDR5 platform support cuts your load times and gives you the space to run the most demanding games
For a meaningful estimate, run the intended checkpoint and request mix on the target machine. Keep the measurement method consistent: in-process latency will not account for HTTP overhead, queuing, or the concurrency patterns of a live service. The exact four-core run is documented for Laya v0.3.20; it should not be assumed to represent later versions without a fresh benchmark.
Recommended Free Tools
Memory context for serving checkpoints
The benchmark script peaked at 9.3 GiB while as many as five checkpoints were loaded. That is a whole-run peak, not a measured per-checkpoint memory figure. The separate Laya requirements guide estimates about 3 GB of free RAM for one checkpoint, recommends 8 GB for the default Router and 16 GB for a server preloading all three checkpoints, and lists download sizes of about 808 MB for the English and typed-decisions checkpoints and 647 MB for multilingual. These are estimates and recommendations from the guide, not measurements of each checkpoint’s runtime memory.
Quick Recap
Best Value
- [CPU] Intel Core Ultra 7 265 Processor (20 Cores, 20 Threads, 3.9 GHz Base Clock Speed up to 5.5 GHz Max Boost Clock Speed) for Elite Gaming and Content Creation | [STORAGE] 1TB PCIe NVMe M.2 SSD - Experience Hyper-Fast Bootup and Data Transfer thats up to 30x Faster Performance than a Traditional Hard Drive.
- [GPU] Integrated Intel UHD Graphics: Get All the Power You Need for Fast, Smooth, Power-Efficient Performance | [RAM] 24GB DDR5 RAM 5600 Gaming Memory for Seamless Multitasking from Multiple Web Pages to Playing Games Online Simultaneously | [OS] Windows 11 Pro x64
- 2x 3.5" Drive Bays | 4x Expansion Slots | mATX Motherboard | ATX PSU
- [BUY WITH CONFIDENCE] Empowered PCs are Assembled in the USA, Rigorously Stress-Tested Before Shipping, and Supported with Lifetime Technical and Diagnostic Support and 3-Year Limited Hardware Warranty.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
