Apple researchers demonstrated a way for a phone to run a language model whose parameters exceed available DRAM: keep the parameters in flash storage and load them as needed. Their method, called LLM in a flash, uses windowing and row-column bundling to reduce slow flash reads. It is a research technique, not proof that every iPhone can run any large model. Apple’s separate Apple Intelligence system uses a roughly 3-billion-parameter on-device model and can route more demanding requests to Apple’s Private Cloud Compute.
How can a model be larger than a phone’s memory?
A model’s parameters are the values it learned during training. In ordinary inference, those values need to be available to the processor, usually in DRAM. If the complete model cannot fit there, a device can instead keep parameters in flash storage and move the needed portions into DRAM as inference proceeds.
Flash is persistent storage, not a substitute for working memory: it is slower to access than DRAM, and repeatedly fetching small pieces can become a bottleneck. Apple researchers’ 2023 paper, revised in July 2024, addresses that bottleneck with two techniques. The authors report that their method enabled models up to twice the available DRAM size in their evaluation; the result depends on the model, device, implementation, and workload.
What does “LLM in a flash” do?
Windowing reuses activated neurons
During inference, only some neurons are activated for a given input. Windowing takes advantage of that sparsity by reusing previously activated neurons, reducing how much parameter data has to be transferred from flash. Less data movement can make a flash-backed model more practical, though it does not eliminate the need to compute the output.
#1 Best Overall
- AN AMAZING MAC AT A SURPRISING PRICE — With an incredibly portable and durable aluminum design, up to 16 hours of battery life,* and the A18 Pro chip, MacBook Neo is ready to go wherever school takes you.
- FOUR STUNNING COLORS. ONE DURABLE DESIGN — Choose from four beautiful colors — Silver, Blush, Citrus, or Indigo — each with a color-coordinated keyboard. And MacBook Neo is made with a durable recycled aluminum enclosure that helps it reach 60 percent recycled content by weight — the most ever in any Apple product.*
- FLY THROUGH EVERYDAY ASSIGNMENTS — Whether you’re cramming for finals, using Apple Intelligence* to summarize class notes, creating presentations, or even playing the latest Apple Arcade game,* MacBook Neo delivers the performance and AI capabilities you need to get things done.
- UP TO 16 HOURS OF BATTERY LIFE — MacBook Neo delivers all day battery life, so you can power through from early morning classes to late night study sessions without worrying about plugging in.
- A VIBRANT 13-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Neo supports 1 billion colors, so photos and videos pop and text is crisp for easy reading.
Row-column bundling reads larger chunks
Flash storage performs better when data is read in larger, contiguous chunks than when many tiny reads are scattered across storage. Row-column bundling organizes parameter access to take advantage of those sequential reads. It complements windowing: one technique reduces unnecessary transfers, while the other makes the transfers that remain more efficient.
How to interpret the reported speedups
In the paper’s evaluation, the authors report inference-speed increases of 4–5× on CPU and 20–25× on GPU compared with naive loading approaches. Those are comparisons against the paper’s baseline, not claims that a phone will run a model at a particular absolute speed or outperform another device. The results also do not mean flash has DRAM-like bandwidth.
Rank #2
- AN AMAZING MAC AT A SURPRISING PRICE — With an incredibly portable and durable aluminum design, up to 16 hours of battery life,* and the A18 Pro chip, MacBook Neo is ready to go wherever school takes you.
- FOUR STUNNING COLORS. ONE DURABLE DESIGN — Choose from four beautiful colors — Silver, Blush, Citrus, or Indigo — each with a color-coordinated keyboard. And MacBook Neo is made with a durable recycled aluminum enclosure that helps it reach 60 percent recycled content by weight — the most ever in any Apple product.*
- FLY THROUGH EVERYDAY ASSIGNMENTS — Whether you’re cramming for finals, using Apple Intelligence* to summarize class notes, creating presentations, or even playing the latest Apple Arcade game,* MacBook Neo delivers the performance and AI capabilities you need to get things done.
- UP TO 16 HOURS OF BATTERY LIFE — MacBook Neo delivers all day battery life, so you can power through from early morning classes to late night study sessions without worrying about plugging in.
- A VIBRANT 13-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Neo supports 1 billion colors, so photos and videos pop and text is crisp for easy reading.
Is this the technique Apple Intelligence uses?
The flash-backed method is a research result and should not be conflated with Apple’s described production stack. Apple’s 2024 foundation-model report described an on-device model of approximately 3 billion parameters, alongside a larger server model for Private Cloud Compute. At WWDC24, Apple said quantization reduced a 16-bit-per-parameter model to an average below 4 bits per parameter so it would fit on supported devices, while maintaining model quality.
Apple also described other production optimizations: speculative decoding, context pruning, group-query attention, adapters, Core ML execution, and acceleration across CPU, GPU, and Neural Engine. Its 2025 technical report again described a roughly 3-billion-parameter on-device model, this time noting KV-cache sharing and 2-bit quantization-aware training. That report also describes a server model based on a Parallel-Track Mixture-of-Experts transformer and a Swift-centric Foundation Models framework with guided generation, constrained tool calling, and LoRA adapter fine-tuning.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsRank #3
- BUILT FOR COLLEGE. AND BEYOND — MacBook Air with the M5 chip packs blazing speed and powerful AI capabilities into an incredibly portable design. And with up to 18 hours of battery life,* this thin and light powerhouse is ready to take on almost any major, just about anywhere.
- TEAR THROUGH TOUGH ASSIGNMENTS — With its faster CPU and unified memory, the M5 chip delivers even more performance and fluidity across apps, making multitasking and creative workflows smooth and responsive. A powerful Neural Engine and next-generation GPU with Neural Accelerators give you a powerful platform for AI.
- MAKE QUICK WORK OF YOUR TO-DO LIST — Apple Intelligence helps you write, express yourself, and get things done effortlessly — whether it’s for school or everyday life. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data — not even Apple.*
- UP TO 18 HOURS OF BATTERY LIFE — MacBook Air delivers incredible battery life with amazing performance, so you can power through a full day of classes without worrying about plugging in.
- A BRILLIANT 13.6-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Air supports 1 billion colors, making photos and videos pop with rich contrast and sharp detail, and text appears supercrisp. So everything — from class presentations to movies to games — looks truly stunning.
Quantization reduces the number of bits used to represent model weights, which can lower their memory footprint. It is distinct from moving weights from flash on demand. The published descriptions therefore support two different ideas: flash-backed inference as a research approach to models larger than DRAM, and a production system that combines a compact on-device model with several memory and inference optimizations.
Does Apple Intelligence run entirely on the phone?
No. Apple says it aims to handle as much as possible on-device for responsiveness, low latency, and privacy. Requests that need greater capability can be sent to Private Cloud Compute, which Apple describes as an Apple-silicon cloud system with attestation, end-to-end encryption, no retention after the response, and publicly inspectable production builds. So an Apple Intelligence feature may run locally, but “on-device” does not describe every request or every model involved.
Rank #4
- SUPERCHARGED BY M5 — The 14-inch MacBook Pro with M5 brings next-generation speed and powerful on-device AI to personal, professional, and creative tasks. Featuring all-day battery life and a breathtaking Liquid Retina XDR display with up to 1600 nits peak brightness, it’s pro in every way.*
- HAPPILY EVER FASTER — Along with its faster CPU and unified memory, M5 features a more powerful GPU with a Neural Accelerator built into each core, delivering faster AI performance. So you can blaze through demanding workloads at mind-bending speeds.
- BUILT FOR APPLE INTELLIGENCE — Apple Intelligence is the personal intelligence system that helps you write, express yourself, and get things done effortlessly. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data — not even Apple.*
- ALL-DAY BATTERY LIFE — MacBook Pro delivers the same exceptional performance whether it’s running on battery or plugged in.
- APPS FLY WITH APPLE SILICON — All your favorites, including Microsoft 365 and Adobe Creative Cloud, run lightning fast in macOS.*
What limits a phone running a large local model?
Making model parameters fit is only one part of inference. A larger-than-DRAM model still has to move data from storage, perform the required computations, and manage runtime memory such as the KV cache. Actual performance depends on the model and workload as well as device memory, storage and memory bandwidth, and the implementation.
There are also practical trade-offs: more storage reads and computation can affect responsiveness and power use, and sustained workloads are subject to device thermal limits. The cited Apple publications do not establish a general battery-life figure or sustained thermal result for running these models on phones, so no single battery or temperature expectation follows from the reported speedups.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteBest Value
- AN AMAZING MAC AT A SURPRISING PRICE — With an incredibly portable and durable aluminum design, up to 16 hours of battery life,* and the A18 Pro chip, MacBook Neo is ready to go wherever school takes you.
- FOUR STUNNING COLORS. ONE DURABLE DESIGN — Choose from four beautiful colors — Silver, Blush, Citrus, or Indigo — each with a color-coordinated keyboard. And MacBook Neo is made with a durable recycled aluminum enclosure that helps it reach 60 percent recycled content by weight — the most ever in any Apple product.*
- FLY THROUGH EVERYDAY ASSIGNMENTS — Whether you’re cramming for finals, using Apple Intelligence* to summarize class notes, creating presentations, or even playing the latest Apple Arcade game,* MacBook Neo delivers the performance and AI capabilities you need to get things done.
- UP TO 16 HOURS OF BATTERY LIFE — MacBook Neo delivers all day battery life, so you can power through from early morning classes to late night study sessions without worrying about plugging in.
- A VIBRANT 13-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Neo supports 1 billion colors, so photos and videos pop and text is crisp for easy reading.
- Model size versus available memory: flash can hold parameters that do not fit in DRAM, but the device still needs working memory for active computation and runtime state.
- Quantization and quality: using fewer bits can shrink weights, but quality depends on the model and quantization method; Apple’s cited statement is specific to its supported-device model and its own quality assessment.
- Speed and data movement: reading from flash on demand introduces a storage-bandwidth constraint that windowing and bundling seek to reduce.
- Privacy and connectivity: local execution and Private Cloud Compute are different execution paths; whether a request is sent to the cloud depends on the task.
Which iPhone can run local AI models?
There is no sound basis here for naming a universal iPhone model that can run any local large language model. The flash paper’s result is tied to its tested setup, while Apple’s production availability is device- and feature-specific. Check Apple’s current compatibility information for the particular Apple Intelligence feature and device you intend to use; support for one feature does not establish that the phone can run an arbitrary model locally.
For a local model beyond Apple’s built-in features, compatibility also depends on the model’s memory needs, quantization, software implementation, and the device’s storage, DRAM, and compute resources. A headline model size alone is not enough to predict usable speed or whether the full workload fits.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
