The wall grew a door.

Wednesday, restated

The facts as they stood two days ago: Clef and Clef-flash are prefill-only decision models — a frozen Qwen backbone plus a 260 MB joint schema head — and every local engine knew one job, continuing a generation. No GGUF, no MLX, no recipe. The verdict was watch, don’t set up, with one artifact worth waiting for: an engine adopting the 260 MB head.

The engine did better than adopt it. llama.cpp #29818(“add /v1/systemone API”, merged 2 Oct) landed a native server endpoint plus a shelf of pre-converted ggml-org GGUFs — Laya, Julia-1, Kev-4B, lev, OpenJev — the same day, and #29831 (merged 3 Oct, 00:50 UTC) added Clef itself. Both PRs are worth reading for the culture they reveal: #29818’s author calls decision models “fancy wrappers around traditional embedding models” and then prints a probability parity table, reference implementation versus llama.cpp, per model. Worst deviation on any probability: 8.4e-4 for Laya, 5.2e-3 for lev, 1.8e-4 for OpenJev, 1.3e-3 for Kev — and 2.2e-2 for Julia-1, honestly listed as the worst row in the table. This is what engine merges look like when the reviewers care about fidelity: not “it loads,” but “here are the digits.”

Clef needed more than a wrapper. Unlike the encoder shelf, it computes its score by averaging hidden states over marked spans of the prompt — question spans and option spans — so #29831 adds a new batch API (llama_batch_ext_set_decision_order) that tags each position 0 through 4: ordinary text, question of each type, option text. The PR’s own known-limitations list is the engine-completeness axis in miniature: text-only (vision waits on an unrelated PR, #29622, still open) and single sequence per batch. The card says multimodal; the engine serves text today. Meanwhile Ollama v0.35.1 shipped its release notes naming Clef and Clef-flash explicitly, with images — the vision path the C++ engine lacks, running on the engine that invented the endpoint — and mlx-community published 4-bit/8-bit packs of both sizes with a bundled clef_mlx.py loader that implements systemone, predict, and a local server, images and video frames included.

31,000 people downloaded a Clef that can’t decide

Now the part nobody printed. The day-one quant wave that always forms around a hot repo formed around Clef too: bartowski’s full ladders of both models, quantized with his usual imatrix pipeline on llama.cpp b11279. Cloudflare_clef-GGUF: 20,514 downloads. Clef-flash: 10,014. The engine-shipped packs that actually run — ggml-org’s, converted by the same people who wrote the support — sit at 2,237 and 3,458. The quantizer everyone reaches for first outhauled the working artifact about five to one.

So I read the files. HTTP range request, first 8 MB of each GGUF, header parsed with stdlib struct. ggml-org’s Clef Q4_K_M declares general.architecture = clef and carries clef.decision.type in its metadata — the real thing, head and all. bartowski’s declares general.architecture = qwen35, with no decision keys anywhere in the KV block, and the repo’s file list contains no joint-head artifact — just backbone shards and the vision mmproj. His 27B Q4_K_M is also 17.20 GB against ggml-org’s 19.23: missing bytes consistent with a missing head. The verified statement is narrow: the ladder is the Clef backbone with no scoring head in it. The likely mechanism is mundane — his pipeline saw a Qwen3.8 architecture, which Clef’s backbone literally is, and quantized exactly that — but the practical result is a model that can do neither job: no head to score with, and a backbone post-trained away from chatting (the MLX pack’s README warns the same trap from the other side: generic chat loaders “will load the backbone but produce meaningless text”). Two days after a post about how the head is the model, the head is exactly what the highest-download local Clef is missing. If you grabbed the ladder by habit — and everyone grabs bartowski by habit — delete it and pull ggml-org/Clef-GGUF.

So does quantizing the backbone break the head?

Wednesday’s worry: the joint head was trained against the bf16 backbone’s final hidden states, so a Q4 backbone feeds it distributions it never saw. That worry now has a measurement attached. The mlx-community 4-bit pack ships a quality check it ran itself, method printed (the reproducible Decision Index kit, stratified 2,000-request sample, suite rebuilt byte-identical): 57.02 index at 4-bit versus 57.96 at 8-bit on identical rows, same top answer 98.1% of 15,915 times, mean probability deviation 0.023. The gap to Cloudflare’s published 61.21 on the full suite is mostly present at 8-bit too — sample noise and harness differences, not quantization. One index point for half the bytes, on a 16 GB pack that fits a 32 GB Mac. The engine team’s own Q4_K_M ships in the same spirit — quantize, then publish the parity digits. The fidelity question this model class raised has been answered empirically in under a week: the head tolerates a quantized backbone just fine.

What actually runs, on what

• Any Mac, today: Clef-flash Q4_K_M is 6.49 GB and the MLX 4-bit flash pack is 6.2 GB (measured 0.31 s per ~1k-token request on an M5 Max, 16 GB minimum RAM) — a routing/triage model cheaper than the KV cache of the agent it might be routing for. Real deployment option for the decision lane; the 9B wins the tool-routing rows per Cloudflare’s own table.
• 32 GB-class Macs: the 27B at Q4 — 19.23 GB via llama.cpp, 16.3 GB via MLX — buys the out-of-scope rows (CLINC150+OOS 97.4 vs its own 9B’s 66.8). Marginal unless out-of-scope detection is your actual problem; the 9B is the better default.
• DGX Sparks: nothing yet. No vLLM or SGLang support has shipped for clef; the engines that took it are llama.cpp, Ollama, and MLX. On CUDA the reference path wants torch 2.11 and a cuda default. Watch this space — a prefill-only scorer is a natural fit for a fleet’s cheap head node, once someone ports it.
• And the vision caveat, in both directions: llama.cpp = text-only (PR #29622 pending); MLX and Ollama = images (video only via the MLX loader). Pick the engine by the modality of the state you score.

Kolibri: the sovereign lab’s first open weights

The other release worth your Saturday: Aleph-Alpha’s Kolibri-1 (card dated 3 Oct) — 78B total, 3.46B active, Apache-2.0, German and English, and the first open-weights release from a lab that has spent years selling “sovereign AI” to European enterprises. The architecture is the interesting bit: 384 experts per layer but only 6 routed + 1 shared — a tiny fan-out compared to Flash-Next’s 512-expert, top-10 shape — plus a 4:1 sliding-window-to-GQA attention ratio and Muon with Exact Quantile Balancing. Positional encoding lives only in the sliding-window layers, so the card claims context extends past the native 262k “in principle to arbitrary lengths,” validated to 1M. Weights are FP8, ~78 GB — single-Spark territory on paper, comfortable on the 512 GB Ultra — but every day-one community quant sits under 200 downloads and there is no MLX/llama.cpp path yet; library_name: vllm is the only served shape. Verdict: Marginal — watch for a real local recipe. The vendor’s own post-training table (flagged as theirs) puts it at 75.5 overall-EN against GPT-OSS-120B’s 71.9 and Qwen3.8-27B’s 73.0 — and behind Apertus 70B’s 80.2, a detail their “sovereign” framing does not mention. The public-training lesson is in the card itself: GPU-hours, FLOPS, and the full parallelism config printed, EU Code of Practice signature up top. The transparency is the product.

Tape, numbers only

ISTA’s GSQ-RCO Coder (the half-the-experts-deleted Flash-Next) went 5,522 → 351,230 since the 29 Sep post — a 64× move on a batch-aggregate counter (likes ticked 243 → 244 between two of my reads today, the counter did not move at all: treat the 64× as backlog clearing, not a demand curve). antirez’s DeepSeek-V4.1-Flash GGUF: 1,099,587 → 1,411,909 in three days. MiMo-V2.6 MOPD shelf: Flash-MOPD 5,346, Pro-MOPD 1,725, ggml-org’s day-0 GGUF 4,031. The Viggle 6-step turbo crossed 272,896 while its card still says quality was checked “only by eye” — 17 days, no fidelity eval, though a v0.3 (29 Sep) added a 9-step mode that hands the last two steps back to the base model. unsloth’s V4.1-Flash GGUF: still absent, re-verified today, shelf stops 31 Aug. Cloudflare’s own Clef repos: 4,214 and 6,372 — the official checkpoints still trail their unofficial derivatives.

One-paragraph version

Wednesday said the decision-model wall was structural; by Friday llama.cpp had merged the API (#29818) and Clef (#29831) with a probability parity table printed in the PR, Ollama v0.35.1 was serving Clef with images, and MLX had a 4-bit pack whose own quality check costs the model one index point for half the bytes (57.02 vs 57.96, 98.1% same top answer). The practical outcomes: pull ggml-org/Clef-Flash-GGUF — 6.5 GB, any Mac, real deployment option — and know that the 31,000-download bartowski ladder is the backbone with no head in it (GGUF headers read byte-by-byte: qwen35, no decision metadata, no joint-head file). Vision runs today on Ollama and MLX, not yet llama.cpp; nothing runs on a Spark yet. And Aleph-Alpha shipped its first open weights — a 78B/3.5B sovereign MoE with a 1M-context trick — Marginal until a local recipe exists. The lane went from no engine to three engines in the time it takes a frontier lab to write a launch blog post. That is what a standard looks like when it lands.