Machine-generated. Surfaced by a sourcing pipeline (HuggingFace trending + newest, plus curated X signals) and written by an AI assistant under human curation. Speeds are measured where a benchmark is cited, and flagged as estimates where they aren’t.
A smiling cartoon brain wearing a bib sits at a tiny table inside a cross-sectioned desktop Mac while a glowing conveyor belt from an SSD drive labeled EXPERTS slides tiny glowing books onto the table; a cheerful dwarf engineer in a hard hat works the belt lever beside a gauge reading 800.
Dinner is served one routed expert at a time. The belt, not the table, is the bottleneck.

The most useful local-AI number this week came from one builder, one card, and one variable. Same 198 GPQA-diamond questions, same RTX 5090, same llama.cpp server, greedy decoding — and the only thing moved was the ceiling on thinking tokens, the private deliberation a reasoning model emits before it answers: 4k, then 8k, then 16k.

  • Qwen3.8-27B (Q6_K): 60.1% → 70.2% → 79.3%
  • Ornith-1.5-35B-A3B (Q4_K_M): 70.2% → 77.3% → 81.8%

Nothing about the first model changed between runs. At a 4k thinking cap it was simply running 19 points below its own 16k score because the operator capped its deliberation. A tight reasoning budget doesn’t make a model cheaper and equally good; it hides quality you already paid for. And no model card states the thinking budget its benchmark numbers were produced at — a score quoted at a 4k cap is a different model than the same checkpoint at 16k, and nobody wants to say that in a press release. Honest caveats on the sweep itself: one run per configuration, one quant per model, 198 questions, one card. A direction, not a leaderboard — but it’s a direction about your dial, which is more than any vendor table gives you.

The second row is the engineering interest: the sparse MoE Ornith started about 10 points higher at 4k but gained less from quadrupling the ceiling (+11.6 vs +19.2) — it reaches for the answer sooner and wastes fewer tokens getting lost. Architectures want different budgets, so one default fits none.

Why this matters more at home than in the cloud: on an API, wasted thinking tokens cost fractions of a cent; on your box, they bill at your decode speed. 8,000 tokens of deliberation at 40 tok/s is 200 seconds per answer, before the answer starts. The whole ecosystem is orbiting this dial — DeepSeek-V4.1-Flash ships a reasoning_effort 1–100 control; a Sep 8 measurement counted ~4,229 tokens per correct GPQA answer on Qwopus3.8-27B-Flash; DavidAU’s Turbo fine-tunes sell “1/2–1/10 the thinking tokens” as the headline (vendor claim, flagged as always). The move: take a 50-prompt sample of your actual workload, cap the budget until accuracy starts bending down, then stop one notch earlier. It’s the cheapest quality knob you own.

The quants nobody else shipped

On Sep 11, antirez — the Redis author who has been spending his weekends making absurdly large MoEs run on Apple Silicon — published calibrated Q2/Q4 GGUFs of DeepSeek-V4.1-Flash for his own DwarfStar engine. Not a re-quant of someone else’s file: the engine author’s own pass, both cuts from a single 8,192-token activation imatrix, documented like a lab notebook. The Q2 is a 340.6 GiB file of which only 151.8 GiB are quantized weights — the rest is 188.8 GiB of native FP8 Engram tables that DwarfStar reads row-by-row off disk and never makes resident — with Q8 kept on the attention projections, shared experts and output, and IQ2/Q2_K on the routed experts.

The deployment math comes straight off the card: one 128 GB Mac runs Q2 with SSD streaming; full residency wants two 128 GB Macs over RDMA or a bigger machine; Q4’s resident target is a 512 GB Mac. The weights are DwarfStar-specific — this buys you into that engine, not into llama.cpp. The repo’s 1,086 downloads in ~2 days is a small number carrying a lot of meaning: it’s where people who read last week’s “V4.1 streams on a 128 GB Mac” posts go to actually run the model. For the record, unsloth — the quant factory you’d normally wait for — still hadn’t shipped a V4.1-Flash build; the calibrated one came from the engine instead. If you own a 128 GB+ Mac and accept the engine lock-in, these are the most trustworthy way to run an open frontier model at home right now: known recipe, known memory math, author-maintained.

Behind it, the wave keeps building without anyone steering: the base repo nearly doubled in a day (75,774 → 140,636 downloads) and at least 24 community builds now clear 200 downloads across GGUF, MLX, EXL3 and NVFP4. If you live in llama.cpp/Ollama on 16–32 GB, that generic wave is first-week access with unmeasured quality gaps — wait.

Two things compounding quietly

MiniCPM5-2B went from 2,879 to 102,334 downloads in five days — a 35× adoption curve, with its official GGUF at 87,316. The permissive 2B edge slot has an owner. And DavidAU’s llama.cpp build of the uncensored Turbo fusion line passed 665,911 downloads, selling the thinking-token cut as its headline. The download count is the real data; the “735 ARC-C” quality board is his own metric, same flag as every previous entry for this family.

Sources: HuggingFace model cards and the HF API (figures verified 13 Sep 2026), public X posts for the GPQA thinking-budget sweep (198 questions, one card — a direction, not a leaderboard), and the antirez quant-card README for the file sizes and residency math.