If an inference request hangs as llama-server goes to sleep, first check GET /props for is_sleeping, then compare the sleep, request, wake/reload, and client-timeout timestamps in the server logs. A request that arrives exactly as sleep begins may be affected by a timing race reported upstream, even though normal inference work is documented to wake the model.
What sleep mode does to llama-server
The --sleep-idle-seconds option sets how long the server waits without incoming tasks before sleeping. The reviewed Debian unstable llama-server(1) manual documents a default of -1, which disables the timeout; that manual is for llama.cpp-tools 1:0.5.0+dfsg-2, dated 2026-09-24. Defaults and options can vary by build, so record your own version and launch arguments. Debian llama-server(1) manual
When the idle timeout expires, sleep mode unloads the model and associated memory, including the KV cache, from RAM. The llama.cpp server README says that new incoming work ordinarily triggers the model to reload. It also documents GET /props as the way to inspect sleeping status; in router mode, use /props?model=(model_name). llama.cpp server README
What a lost-request symptom can mean
An upstream issue opened on 2026-09-30 reports that an inference request arriving as the server enters sleep may be queued but never processed. In the reporter’s example, the logs showed the server entering sleep, then the client cancelling about ten seconds later, with no “exiting sleeping state” message. The reporter said the behavior occurred in about 1 in 8 runs on their setup; that is an anecdotal rate for one reproduction, not a general failure rate. Upstream issue #29689
#1 Best Overall
- EVOLUTION AMD RYZEN AI MAX+ 395 MINI PC - GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series. Thanks to AMD Simultaneous Multithreading (SMT) the core-count is effectively doubled, to 32 threads. Ryzen AI Max+ 395 has 64 MB of L3 cache and can boost up to 5.1 GHz, depending on the workload. The Ryzen AI Max+ 395 is currently rated as the "most powerful x86 APU" on the market for AI computing.
- AI NPU with XDNA 2 ARCHITECTURE - Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive integrated GPU driven by 40 AMD RDNA 3.5 CUs, the Ryzen AI MAX+ 395 is a transformative upgrade and delivers a significant performance boost over the competition. The Ryzen AI Max+ 395 excels in consumer AI workloads like the llama.cpp-powered application: LM Studio. Shaping up to be the must-have app for client LLM workloads, LM Studio allows users to locally run the latest language model without any technical knowledge required and unleash their creativity and productivity.
- AMD RADEON 8090S iGPU GAMING PC - The AMD Radeon RX 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the new RDNA 3.5 architecture. The powerful iGPU is positioned between an RTX 4060 and 4070 laptop GPU and therefore enables gaming in FHD at maximum details in most demanding games. The 8060S can also utilize the full 64GB pool, which is perfect for running LLMs such as Deepseek 32B, which runs comfortably on this machine.
- EIGHT CHANNEL LPDDR5X - LPDDR5X is a new ground breaking memory small form factor installed on-board. With blazing speeds up to to 8000MT/s, it runs 1.5x faster than the DDR5 SODIMMs; 90% better performance over DDR5 SODIMMs in video conferencing and photo editing; 30% better performance in productivity apps; 4% better performance in digital content workloads.
- QUAD SCREEN 8K DISPLAY SUPPORT - EVO-X2 AI Mini PC support 4-screen 4K/8K output via HDMI 2.1 (8K@60Hz), DisplayPort 1.4 (4K@60Hz), and dual USB 4 40Gbps Transfer speed (supporting PD3.0/DP1.4/DATA). Ideal for gaming, video editing, and multitasking, it provides expansive and crisp multi-display support.
The reported setup was llama.cpp 0.5.0-dev, build 11160, commit 70c4e1582, built with Clang 20.1.8 for Windows x86_64, with --sleep-idle-seconds 1. The client sequence was /health, /props, /tokenize, then POST /completion, with completion sent about a second after readiness. This describes the reporter’s environment and timing; it is not a universal reproduction recipe.
The issue author proposes a queue-timing explanation: an HTTP worker passes a wake check while the server is awake, but sleep begins before that worker posts its task. In the reported analysis, the sleeping queue loop then does not react to the newly posted task, leaving the client to time out and cancel. This is the reporter’s code-reading theory, not a maintainer-confirmed root cause.
Rank #2
- 3.5 Inch Hot Plug Hard Drive PowerEdge T340 Tower Server Chassis
- Microsoft Windows Server 2019 Standard Operating System
- Processors: Intel Xeon E-2124 Quad-Core 3.3GHz 8MB CPU, Up To 4.3GHz Turbo
- Memory: 32GB (2 x 16GB) DDR4 PC4-21300 2666MHz Unbuffered Memory
- Hard Drive: 8TB (4 x 2TB) 7.2K RPM 6Gb/s SATA 3.5 Inch HDDs in RAID
Check whether the server is asleep
- Ask the server for its state. Send
GET /props; if router mode is enabled, useGET /props?model=(model_name). Look foris_sleepingin the response, following the server README’s documented status check. - Capture the exact runtime configuration. Save the output of
llama-server --version, the full command line or service configuration, the model, router-mode setting, and the value of--sleep-idle-seconds. This makes comparisons meaningful across builds. - Align server and client timestamps. Find the log entries for entering sleep, request arrival, exiting sleep or model reload, and client cancellation or timeout. A request at the transition followed by no wake/reload message is consistent with the upstream report, but does not alone prove that the same issue caused your failure.
- Check which endpoints ran before inference. The README identifies
/health,/props,/models, and/metricsas endpoints that may use cached responses while asleep and do not count as incoming tasks that wake the model or reset the idle timer. The issue report specifically says/tokenizedid not reset its timer in the reported scenario. Therefore, a successful status or tokenization request does not establish that the model stayed awake.
Compare the two timing conditions
| Test condition | What to do | What the result can tell you |
|---|---|---|
| Sleep disabled | Set --sleep-idle-seconds -1, the disabled value documented in the reviewed Debian manual, and run the same request sequence. |
If the hang disappears, that supports a sleep-path cause, but does not establish the specific queue race. |
| Request while already asleep | Wait until GET /props reports is_sleeping: true, then send the inference request. |
The issue author reports this path worked in their tests because the request found the server already asleep and followed the wake-request path. It is a reported workaround, not a guarantee across builds. |
| Request near sleep transition | Repeat the same request timing with sleep enabled and inspect whether the request coincides with the sleep-entry log. | A hang isolated to this timing window resembles the reported condition. Preserve logs and exact timing rather than treating similarity as proof. |
For a useful comparison, keep the build, model, request body, endpoint sequence, client timeout, and sleep setting consistent except for the condition being tested. Record whether a wake or reload appears before the client gives up.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What is known about a fix
Issue #29689 proposes waking when a task is queued or preventing sleep while an HTTP request is between its wake check and task posting. Those are proposals in the issue, not confirmed shipped fixes. The issue page showed no linked branch or pull request when checked on 2026-10-03; verify the issue and release notes for current status before changing or upgrading a deployment.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Quick Recap
Best Value
- [ Maximum AI Compute Power ] Dominate complex workloads with the ASUS ESC8000A-E13. This 4U rack server is a powerhouse engineered for mass-scale AI, machine learning, and deep training. Featuring support for dual AMD EPYC 9005/9004 processors and up to eight dual-slot GPUs, it delivers the raw computational muscle required to train LLMs and run complex simulations effortlessly. Accelerate your data science pipeline and transform raw data into actionable intelligence faster than ever.
- [ Advanced Thermal Efficiency ] High performance demands elite cooling. The ESC8000A-E13 features a cutting-edge aerodynamic design with independent CPU and GPU airflow tunnels. Equipped with redundant hot-swap fans and optimized for liquid cooling integrations, this 4U server ensures maximum uptime under heavy, sustained workloads. Keep your data center running cool, quiet, and highly efficient while preventing thermal throttling during mission-critical enterprise operations.
- [ Scale with Flexible Storage ] Future-proof your infrastructure with unmatched storage and expansion flexibility. This offers comprehensive front-panel drive bays supporting Gen5 NVMe, SAS, or SATA drives alongside multiple PCIe 5.0 slots. Designed as a high-density 4U server capable of housing eight dual-slot GPUs: NVD H200, RTX PRO 6000 Blackwell, RTX PRO 4500 Blackwell or AMD Instinct MI350P PCIe Card, each supporting up to 600 watts.
- [ Enterprise-Grade Reliability ] Minimize downtime and secure your ecosystem with server-grade redundancy. The ESC8000A-E13 is built for 24/7 continuous operation, boasting 2+2 redundant (3200W total) 80 PLUS Titanium power supplies and integrated ASUS ASMB11-iKVM for comprehensive out-of-band management. Ideal for cloud service providers, rendering farms, and large enterprise infrastructure, it combines robust physical hardware with smart remote monitoring to safeguard your digital assets.
- [Reliability Guaranteed] Shop with total peace of mind knowing that every new computer component we sell is backed by our EPC 3-year warranty. Whether you are investing in high-speed DDR5 RAM or a powerhouse GPU, we protect your build against defects and performance failures. We stand firmly behind the quality of our hardware, ensuring that your setup remains fast, stable, and secure for years to come.
Rank #3
- Dell PowerEdge R730xd 24B SFF 2U Server
- 2x Intel Xeon E5-2690 v4 2.6Ghz 14-Core (28-cores Total)
- 128GB DDR4 RAM – 4x 1.2TB 10K SAS 2.5” 12Gb/s
- Dell H730P mini 2GB 12Gb/s RAID
- 2x 750W PSU - 2x 10Gb SFP+ 2x 1Gb (RJ45) NIC
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
