Built for machines smaller than the model.

The idea

Treat the model as a file and memory as a cache. The engine keeps only what every token needs resident, and pages the rest in on demand.

• A small resident core. The pieces every token touches are kept in memory. Everything else is loaded when a token needs it and released when it does not.
• Fixed buffers, no surprises. Working memory is allocated once at load. Nothing grows mid-generation, so a model that opens will keep running.
• The operating system does the caching. Weights that were used recently stay warm; weights that were not stay on disk. No copy of the model is ever made in memory.

The .qpx bundle

One self-contained file per model, and the same file on every platform.

• Everything in one file. Weights, tokenizer, configuration and provenance travel together. There is nothing to attach and nothing to lose.
• Portable by design. A bundle prepared on one machine opens identically on another. The layout was chosen so that the same file works on Apple silicon today and on other platforms as the engine is ported.
• Prepared for paging. The bundle is laid out for the way the engine reads it, which is what makes bigger-than-memory models possible at all.
• Stamped. Every bundle records what it is and where it came from, and the Studio trusts the stamp over the filename.

Why mixture-of-experts is the whole story

A dense 30B model streamed from SSD is 9.5 seconds per token. Nobody ships that. Tokens on a small machine come from sparsity.

• Sparse models touch a fraction of their weights per token. That fraction stays warm in memory, which is how a 30B coder ran on the very first day, on the CPU alone, and reached 14.73 tok/s on the GPU the same day.
• The frontier models are bounded by the SSD. On DeepSeek V4 Flash, every token still pulls gigabytes of experts from disk, which sets the ceiling on the mini at roughly 2 tok/s. The remaining levers are all about reading less and reading smarter.
• Dense models do not get out this way. A dense 27B streams its whole feed-forward every token no matter how it is arranged. That lane was closed as bandwidth-bound; the 27B ships fully resident instead.

Paging that earns its keep

• A bounded cache of hot experts on the GPU took DeepSeek V4 Flash from 0.13 to 1.03 tok/s on its own.
• Reading ahead. The engine anticipates what the next layer will need and has it warm before the layer asks. On Qwen 3.6 that cut the wait per token by about a quarter; on Flash Next it took 3.18 to 3.95 tok/s.
• Nothing blocks the compute path. Disk work happens beside the generation, never in front of it.
• Only the core is wired down. The rest of the model is free to come and go, so the machine stays responsive.

Speed beyond bandwidth

• Self-drafting. The model drafts tokens ahead of itself and the main pass verifies them, so the output is unchanged. Qwen 3.8 27B: 11.9 tok/s. No separate drafter model is ever loaded beside another model.
• A lighter output stage. The last step of every token used to read half a gigabyte; it now reads a small fraction of that with the same answer. Qwen 3.6 gained 7 percent from this alone.
• Prefix reuse and chunked prefill. Prompts that share a prefix skip to the new tokens. Long-prompt prefill on Flash Next was cut nearly in half through five measured steps, each verified before the next was built.
• Attention that stays flat. Qwen 3.6 decodes at the same speed past 6k tokens of context as it does at the start.

Hard-won rules

Each of these cost a session. They are the engine laws now.

• Read the model before writing the engine. Every lane is written fresh from the model's own specification, never copied from an older one.
• Every failure is loud. A halt surfaces in the app; no silent fallback ever holds.
• Verification runs on the raw output. Sampling never touches it.
• Per-model configs, never shared. That applies to sampling parameters and to UI controls.
• Switching models fully unloads the previous one and clears the cache.
• Verify before you delete. Stamp before you archive.
• Swift only, on every platform. The core has no AppKit in it, so it builds on Linux and Windows with the swift.org toolchains.

Where it goes next

The standing plan after the Mac mini era: one bundle format, one paging doctrine, one curated catalog, everywhere.

1. Curated download-and-slice inside the Studio, reusing the verified-queue core.
2. A new system picked from the hardware tier table. 512 GB-class makes the flagship conversational; a commodity tower with a modern GPU is the price-to-performance play at roughly 18 tok/s cold on DeepSeek-scale models.
3. Downloader core to Linux and Windows shells.
4. The engine port to Vulkan and CUDA, which takes Quadpoint cross-platform for customers.