This page has two parts. The table immediately below is estimates; the section after it is measured results from an independent owner, sourced and labeled separately. Neither set was produced on maccloud hardware. The figures in the next table are third-party estimates, not measurements taken on maccloud's own hardware. They come from ModelFit's published bandwidth-and-model-size modeling for a 256GB M5 Ultra Mac Studio, whose own methodology note states plainly: "Speeds are ModelFit estimates from chip bandwidth and model size, not measured benchmarks." We're citing them because they're a reasonable, transparent starting point — not presenting them as in-house benchmark results. As we run our own measured benchmarks on maccloud hardware, we'll publish those separately and label them accordingly.
Estimated throughput by model
All figures below assume the 256GB configuration and the quantization level shown. Memory footprint includes ModelFit's KV-cache assumptions (FP16 cache, the llama.cpp/Ollama default) on top of model weights.
| Model | Params | Quantization | Memory footprint | Est. tokens/sec | Est. first token |
|---|---|---|---|---|---|
| Qwen3-Next 80B-A3B | 80B | Q4_K_M | ~50.4 GB | ~64 tok/s | ~1.4s |
| GPT-OSS 120B | 117B | MXFP4 | ~65.4 GB | ~43 tok/s | ~1.5s |
| Qwen3-Next 80B-A3B | 80B | Q8_0 | ~84.8 GB | ~35 tok/s | ~1.5s |
| Qwen3.8-Flash-Next | 125B | IQ1_S | ~123 GB | ~32 tok/s | ~1.6s |
| Laguna S 2.1 | 118B | Q4_K_M | ~96 GB | ~32 tok/s | ~1.6s |
| Qwen3.5 122B-A10B Instruct | 122B | Q4_K_M | ~72 GB | ~28 tok/s | ~1.6s |
| Llama 4 Scout | 109B | Q4_K_M | ~67 GB | ~23 tok/s | ~1.7s |
| Qwen3 235B A22B | 235B | Q4_K_M | ~130 GB | ~14 tok/s | ~2.0s |
Measured results from a 256GB M5 Ultra
Separately from the estimates above, an independent owner published a detailed benchmark run on the exact configuration we offer — a maxed-out M5 Ultra Mac Studio, 256GB, 80-core GPU, macOS 27.0.1 — with the scripts, raw results and test harness posted publicly so anyone can reproduce them.
A brand-new 500,000-token prompt took about 22 minutes before the first word came out. The very next turn in that same conversation started in about one second, because the processed prompt stays cached. That is the single most important thing to understand about running long-context work on Apple silicon: the first read of a large codebase or document set is genuinely slow, and after that it is fast. An agent that keeps appending to one conversation pays that cost once. A harness that rewrites or re-summarizes its context on every step pays it every single time.
Decode throughput, measured
Qwen3.8-Flash-Next at 8-bit on the oMLX engine, with multi-token prediction enabled. Note how little decode speed degrades as context grows — on summary prompts, which are MTP's weakest case, it holds most of its speed from 10k all the way to 250k tokens.
| Workload | Model / engine | Measured decode |
|---|---|---|
| Short code / JSON prompts | Qwen3.8-Flash-Next 8-bit, oMLX | 118–139 tok/s |
| Sustained overnight coding session | Qwen3.8-Flash-Next 8-bit, oMLX | 90–97 tok/s |
| Summary prompt at 10k context | Qwen3.8-Flash-Next 8-bit, oMLX | 88 tok/s |
| Summary prompt at 250k context | Qwen3.8-Flash-Next 8-bit, oMLX | 68 tok/s |
| Code generation | DeepSeek-V4-Flash, oMLX, DSpark on | 63 tok/s |
| 1,000,000-token context | DeepSeek-V4-Flash, oMLX custom kernels | ~21 tok/s |
Your inference engine matters as much as your hardware
The same machine and the same model produced a 2.2x spread in prefill throughput depending purely on software choices. DeepSeek-V4-Flash prefill, measured four ways:
| Setup | Prefill throughput |
|---|---|
| llama.cpp, default settings | 628 tok/s |
llama.cpp with -b 2048 -ub 2048 | 827 tok/s |
| oMLX, Homebrew build | 1,204 tok/s |
| oMLX, built from source with custom Metal kernels | 1,368 tok/s |
The Homebrew build of oMLX ships without the custom Metal kernels for DeepSeek's sparse attention, which costs you roughly 14% of prefill. Building from source with OMLX_WITH_CUSTOM_KERNEL=1 requires Xcode's Metal toolchain. On a dedicated single-tenant machine you have full root access to make exactly these choices — which is the point.
The one-million-token run
The same tester fed DeepSeek-V4-Flash a million tokens of classic literature in a single prompt, with three "vault codes" hidden at the 10%, 50% and 90% marks — each one planted next to a deliberately similar decoy to make the retrieval test harder than a plain needle-in-a-haystack.
| Measurement | Result |
|---|---|
| First read of 1M tokens (cold prefill) | ~57 minutes |
| Every subsequent cached turn | ~44 seconds to first token |
| Decode at 1M context | ~21 tok/s |
| Retrieval accuracy to 750k | Perfect |
| Retrieval accuracy at 1M | 2 of 3 (third answer blended two similar facts) |
| Chip power draw, sustained | ~135 W |
That power figure deserves a second look. 135 watts — CPU and GPU combined — while processing a million-token context. A multi-GPU rig with enough VRAM to attempt the same workload draws well over a kilowatt. This is the reason Mac Studio hosting economics work at all: the machine is cheap to keep running, and that shows up in what we can charge you.
What this means in practice
Across a full overnight agentic coding run — a deliberately oversized TypeScript project, 30 milestones, tests and a build after each one — the prompt cache held with zero cache breaks for the entire night. The "pay prefill once" behavior isn't a benchmark artifact; it survives a real multi-hour agent session. Engine choice mattered here too: replaying one recorded 181-request agent session against both engines, oMLX completed it roughly 1.5x faster than llama.cpp.
Two operational notes worth knowing before you start, both learned the hard way in that write-up: large models won't fit until you raise the GPU memory ceiling (sudo sysctl iogpu.wired_limit_mb=245760, which resets on reboot), and the first request after loading a model is always slow while macOS pages it in — warm up before you measure anything.
Where Llama 70B fits in
Llama 3.3 70B isn't in ModelFit's top-8 list for the 256GB config (it's tuned toward the larger MoE models that make better use of 256GB of headroom), but its own published GGUF sizes give a good proxy: at Q4_K_M (42.5GB) it sits in roughly the same memory class as GPT-OSS 120B above, and the same bandwidth-based reasoning suggests throughput in a broadly comparable range — likely faster than the 235B-class models on this list and slower than smaller 70–80B MoE models, given Llama 70B is a dense (non-MoE) architecture. See our Llama 70B sizing guide for the full quantization breakdown.
Why this matters for configuration choice
Every model on this list fits inside the 256GB plan with memory to spare. The 512GB plan exists for the cases that don't fit here: running the largest MoE models (like Qwen3 235B) at higher quantization for better quality, running two of these models concurrently, or leaving headroom for a long-context workload that pushes the KV cache well past what these estimates assume.