Everyone ships a decider.

Models

vllm-sr/d3 is Decision 3.0, the decision model family of the vLLM Semantic Router project — six sizes released 10 October under Apache-2.0, all built on Qwen backbones: d3 at 26.09B (Qwen3.8-27B, including a 0.46B vision encoder), d3-flash (Qwen3.5-9B), d3-mini (Qwen3.5-4B), d3-nano (Qwen3.5-2B), d3-lite and d3-edge (both Qwen3.5-0.8B). Give it a state — text, JSON, plus images and videos — and a set of typed questions (choice, noul for true/false, score) and it returns a probability for every option, generating no text at all.

Evidence. This is the first entrant in the lane to publish the official kit’s own number run on its own released weights with a request count attached: the card claims Jev Decision Index 0.3.1 = 64.7 (public suite 65.5), with all 140,178 public requests answered and none unsupported, and +8.5 on the public suite over its own Decision 2.0 27B (57.0 public / 55.9 index) — ahead on all five areas. The vision board reads 71.6 against 70.6 for Perplexity’s Decider v1.1 and 69.6 for JEV-27B-VL. Text answers take a median 55 ms, image requests 273 ms and a 10-second video 553 ms, one request at a time on a single Instinct MI325X. Adoption is thin but real and spread across the sizes rather than clustered on the flagship: 420 / 407 / 338 / 336 / 334 / 329 downloads for d3 down to d3-nano, with the 27B taking the most likes (106). The honesty row worth noting: video is read at 2 frames per second, capped at 32 frames of 0.2 megapixels each, and the card says Engine mode ships in the router’s development channel until a stable release includes it.

Critical read. A project publishing its own score with the public kit is not the same as a third-party board, but it is checkable in the way that matters: the number, the item count and the kit version are printed together, and the previous generation of the same family is on the same table losing by 8.5 points. Whether the six sizes are six real models or one recipe scaled is the open question — the download counters look suspiciously uniform, which is normally a sign of one release being fetched as a set.

Verdict: Real deployment option for the router/gate/judge slot — Apache-2.0, a 0.8B member for the edge, and the first family to bring video into the typed-question interface.

h2oai/h2o-lightning-4b (1,578 downloads / 62 likes) is a 4B decision model from H2O.ai, built on Qwen/Qwen3.5-4B, answering typed questions about a record and the images attached to it. What makes it worth the section is the delivery: it runs on unmodified vLLM 0.30.0 with a small standard-library shim in front, one forward pass and one output token per decision, so the cost is input tokens only.

Evidence. Measured by its own authors on an RTX PRO 4500 Blackwell (32 GB) and cross-checked on an H100 80 GB SXM5: 205 of 231 (88.7%) on JevBench’s public text items via JevBench’s own CLI — 48/48 easy, 71/72 standard, 86/111 hard — plus 8 of 8 on the published ImageJevBench examples and 87.4% over 1,877 image questions across 16 public sets. Client latency p50 is 29.6 ms on the 32 GB card and 28.8 ms on the H100 (raw LoRA run: 57.3 ms, which is the argument for merging). The card also prints a number most decision models hide: 0 of 74 yes/no answers landed between P=0.2 and 0.8 — the model commits. The headline is #1 on the third-party JevBench composite board from 7 to 12 October, 72.5 against Jev’s own 71.5, ahead of every open 12B, 26B and 31B on it; that is a leaderboard claim, read as one.

Verdict: Real deployment option — a 4B that reports matching a hosted reference’s accuracy profile at roughly 29 ms and a fraction of the cost per thousand decisions is the shape a router wants. The demos (a 1,000-claim operations inbox decided in 63 seconds at 32 concurrent, eight browsers checking 192 vendors, a Freedoom level played to the exit) are the authors’ own runs on their own harness.

chinhnc/Quyet-1.0-Large (1,026 / 39) is the 31B member of a five-size family from an individual author: google/gemma-4-31B-it with a merged rank-16 LoRA, a letter-readout decision prompt, 62.5 GB of bf16 weights, running on one 80 GB card or several with device_map="auto". The real delta is where the decision machinery lives: not in the weights. The prompt and the fitted calibration temperatures sit in the quyet Python package and a config file, and the card is explicit that loading it as a plain Gemma-4 checkpoint in transformers or vLLM gives you the architecture without the readout. Answers follow TypeSafe’s /v1/systemone shape, up to ten options per question, and the family’s smaller members are released and unadopted — Medium at 321 downloads (4 likes), Small, Small-EN and Tiny at 0.

Verdict: Marginal — adopt only if you want a 31B decider on a single 80 GB card. The wider evidence for the family is currently cross-quoted from a competitor’s comparison page, which is not where you want your numbers to come from.

neuphonic/neudecide (460 / 72, gated with a short form) throws out the transcript entirely: audio plus a list of tool definitions in, a tool call out. It is 43 MB of weights in three ONNX graphs — a 28.7M streaming English speech encoder (22.5 MB), an 11.8M tool encoder (10.1 MB) and a 15.0M decoder step (10.0 MB) with a cached self-attention state — initialised from cactus-compute’s Needle and trained end to end, quantised to q4, running on a single CPU thread with no GPU and no network. Tools are JSON Schema passed at inference time, so you change what it can do by editing JSON rather than retraining. "clean the bathroom" against [cleanRoom, goToBase, getBattery] returns {"name":"cleanRoom","arguments":{"room":"bathroom"}}.

Verdict: Real deployment option for the voice→action slot on a device, where transcribing first is wasted work. Limits are printed rather than discovered: 30 seconds of audio, a 128-token output cap, WAV natively and everything else via your own decode, and the arXiv paper is still “coming soon”.

cloud0day3/antalia-mini (1,369 / 36) is not a decider but belongs in the same week’s pattern of small purpose-built models: a 7.62 million parameter Turkish text-to-speech model (3.69M acoustic + 3.93M vocoder) at 48 kHz, with the text normaliser inside the wheel so numbers, dates, amounts and English brand words are read the way a Turkish speaker says them. It streams sentence by sentence, resamples to 8/16/24 kHz for telephony and ASR pipelines, takes a seed for reproducible audio, and runs well faster than real time on a CPU. The browser demo is the same release compiled to ONNX Runtime Web — no server, the audio never leaves the page — and the voice is explicitly synthetic with no real-speaker recordings in the training data.

Verdict: Real deployment option for Turkish TTS on CPU or in-browser, Apache-2.0 both ways.

Under the gate: openbmb/MiniCPM-V-4.7-35B-A3B (166 downloads / 208 likes) and its 1B sibling (14 / 126) shipped on 6–7 October and do not clear the adoption bar here, so they get one line — but the engine support arrived anyway (see below), which is the more interesting half. openbmb/MiniCPM-V-4.7-1B under a gigabyte with vision is worth watching on downloads rather than on likes.

Quantizations

The decision lane got a quant shelf, and one member of it out-downloads the model it came from by a factor of eight. mradermacher/Quyet-1.0-Medium-GGUF stands at 8,228 downloads against 321 for the Medium checkpoint itself, and Quyet-1.0-Large-GGUF at 2,110 against the Large model’s 1,026 — both posted 5 October, the same day the family shipped. The honest caveat is structural rather than about the quantiser: Quyet’s readout is a pinned prompt plus fitted temperatures that live in the Python package, and the decision support merged into llama.cpp this weekend targets the vllm and Perplexity decision architectures, not a letter-readout Gemma — so a generic GGUF of it is currently the weights without the calibrated readout. That is an inference from two cards, flagged as one, and it is exactly the kind of thing that should be measured rather than assumed before anyone trusts a number out of those files.

The files that are engine-backed landed the same day as the models. ggml-org published a GGUF for all six d3 sizes on 12 October — vllm-d3-GGUF, -flash, -mini, -lite, -nano, -edge — and they read 0 downloads at first sight with 2–5 likes, which is the batch-update caveat, not a verdict: they are hours old and the support that makes them run merged within a day of the upload (below). The older vllm-sr/Decision-2.0-Vega-27B-GGUF is sitting at227 downloads, which is a fair reading of how much appetite exists for the previous generation of the family once a real engine path appears.

The Apple path arrived within 48 hours of the checkpoints, which has been the pattern all month: mlx-community/Qwen3.5-0.8B-decision (260 / 5), FluidInference/d3-coreml (163 / 10) with a nano CoreML sibling (46), and ahmedandaloes’ d3-mlx / d3-flash-mlx / d3-flash-CoreAI builds (19–22 downloads each). All of those are under the gate and stay on this line rather than getting a verdict — they are evidence of who is trying, not of who is running. MiniCPM-V-4.7 is showing the same day-two pattern at sub-gate counts: AtomicChat GGUF and an abliterated variant, two APEX builds, MLX-Serve packs, an NVFP4 build for one DGX Spark and four OpenVINO conversions, every one of them at zero downloads so far.

Verdict: Real deployment option for the part of the shelf that has an engine behind it — the ggml-org d3 files and the MLX/CoreML packs. Marginal for the high-volume Quyet GGUFs until somebody prints a coverage or calibration number measured out of the quantised file.

Fine-tunes

nandakishorm/vega-08b-public-intents (278 / 35, created 9 October, Apache-2.0) is the most interesting architecture in the lane: a frozen, SHA-256-pinned Qwen3.5-0.8B backbone with everything learned living in a 57 MB engine (14,342,512 parameters, 4B twin 124 MB / 30.9M) plus a 835,937-parameter task adapter. The engine has no attention anywhere: it is an explicit damped Hamiltonian integrated with symplectic Euler against a written-down potential, and the answer is the lowest-energy well. The residual kinetic energy at the end sets the temperature, which is the part that makes the probabilities mean something, and it is predicted per decision from the physical state rather than fitted once.

The real delta is three things the lane mostly doesn’t ship: a conformal set at a chosen risk level α, abstain and unbound flags, and a 73,728-token context that is read once with a KV cache and then answered against twelve times, with refusal instead of truncation when an input does not fit. Answers run 22 ms per decision (0.8B) and 28 ms (4B) on a T4, and the images path goes through the frozen backbone’s own vision encoder.

Evidence, and the card argues with itself in public. On the whole menu the hosted reference is more accurate — 0.836 against 0.717 macro accuracy, and paraphrase consistency 0.972 against 0.722 — while the local models win the probability block: on DecisionBench’s 1,075 rows the calibration error is 0.103 (4B) and 0.098 against the reference’s 0.122, the negative log-likelihood 1.628 / 1.420 against 3.196, and the KL distance to the gold distribution 1.012 / 0.854 against 6.099, with 1.000 of rows answered. The phishing screening row is the sharpest:75.38 accuracy and 252 of 400 phishing caught against 61.88 and 99 of 400, McNemar p = 3e-11 — the reference is rarely wrong when it calls phishing, and misses three quarters of it. And then the row that keeps the rest honest: prompting the same 4B backbone directly reaches 0.700 on the menu, within noise of the 4B engine’s 0.717, so on this menu the engine’s contribution is coverage, probabilities and latency rather than raw accuracy — the authors’ own sentence. The headline win table is a selected set, and the card says so, printing that eight of twelve task wins are three rows or fewer and two of them do not beat answering the same thing every time.

Verdict: Marginal — adopt only if you need calibrated probabilities, conformal sets and an abstain flag inside your own process. It is not the most accurate decider and it says so; it is the only one shipping a coverage guarantee and a refusal path as first-class outputs.

akhaliq/Qwen-Image-2.1-Next-Scene-LoRA (1,769 / 52) and Multiple-Angles-LoRA (1,278 / 77) are two new adapters on the Qwen-Image-2.1 base that this log covered through September and October: scene continuation for storyboards where each shot picks up where the last ended, and camera-angle control. Both are Apache-2.0, both have adoption, and neither ships a fidelity evaluation — the same gap this log flagged on the video adapters on 10 October. Verdict: Marginal.

akatz-ai/MiniMax-H3-Person-Remover-LoRA moved 749 to 941 downloads since yesterday’s entry and still carries no fidelity evaluation. Verdict: Marginal — the demand is real, the receipts are not.

Technical term of the day: the conformal set

A calibrated probability tells you how often a model is right when it says 0.62. A conformal set tells you something different and, for anything that makes a decision you have to live with, more useful: which answers you cannot rule out at a risk level you chose in advance. Vega’s API takes an alpha and returns, alongside the probabilities, a set of options that covers the truth with probability at least 1 − α — plus an abstain flag for the cases it will not stand behind.

How it works, without the maths. You hold back a slice of labelled data you never trained on. You score every candidate answer with a “how wrong does this look” measure — for a choice question, usually one minus the probability the model gave the correct label. Sort those scores and take the one at the 1 − α quantile; that number is your threshold. At inference, you keep every option whose score is under the threshold. That is the whole method. The promise it buys is distribution-free: no assumption about the shape of the probabilities, no normal distribution, no Gaussian error term — only that your calibration rows and your live rows are exchangeable.

Why it matters for local AI. The lane this log has followed since 21 September is entirely made of small models answering typed questions in one forward pass, which means the set is nearly free — you already computed the probability for every option, so the conformal layer is a threshold and a comparison, not a second pass. It also gives a local deployment something a hosted API structurally cannot: aper-question risk dial. Route a ticket automatically at α=0.10 and send the leftovers to a human, or set α=0.01 and let the set grow until you would rather look yourself. And set size is the honest difficulty meter: a model that returns five of eight options on a question is telling you it cannot tell them apart, which is more information than a confident wrong answer will ever give you.

The trade-offs, stated plainly. The guarantee is marginal and average — coverage holds over the whole distribution of questions, not for the one in front of you, and a question it misses is missed completely. It is silently void under distribution shift: run it on a customer segment or a document layout that was not in the calibration slice and the guarantee evaporates with no error and no warning, which is the same trap a fitted temperature falls into. It costs a calibration split — a few hundred labelled rows per question type — and if you cannot spare them you cannot have the guarantee. And bigger sets are the failure mode of a weak model: a 0.8B engine that returns most of the option list is technically compliant and operationally useless, which is why the paired metric is risk–coverage (Vega prints AUROC on the risk–coverage curve: 0.169 for the 4B and 0.362 for the 0.8B, both worse than the hosted model’s 0.077) and not coverage alone.

The three questions to ask any decider that advertises this. What α, and what was coverage on held-out data at that α? Does the set survive the options being shuffled, since option-order shortcuts are the standard way a decision model passes a test it does not understand? And what is your policy when the set contains every option — because "abstain" only counts if something downstream is listening.

Harness improvements

llama.cpp merged the decision lane into the engine, twice, in one day. PR #30378 “model: support vllm decision 2 model” landed 12 October, followed hours later by #30383 “model: support vllm d3 models” — whose own body points at the collection and notes the implementation is minimal because it is “quite similar to perplexity decider” — and then #30408, a fix for d3’s multi-sequence case, that evening. That is the weekend’s whole argument in three commits: the wire format converged first, so the engine work became a copy of the previous decision model instead of a new architecture. The window also brought #29416 support for MiniCPM-V 4.7 (10 October, on a model still under the download gate), #30416 padding the KV dimension of no-cache attention to 256, #30014 making ggml symbols hidden by default (an ABI promise for anyone linking it), #30373 fixing Metal mul_mat_id with duplicate ids and #28748 a Metal tensor kernel error on large tensors, #29877 packing FMAs for f16 accumulation on Vulkan with #30164 avoiding switch lowering for CM2 quant block size and #30326 fixing buffer logging on 32-bit platforms, more Hexagon work (CPY/CONCAT round three, scalar DMA broadcasting for binary ops, IM2COL_3D), #30329 reporting compute-graph reallocations during inference — the number that explains a stall nobody could previously see — and #30324/#30401 turning the OpenVINO builds into releases with GGML_NATIVE=OFF and CPU backend variants.

Ollama v0.40.4-rc1 (12 October) is a one-line confirmation of where this is going: its llama.cpp bump is described as being for decision models and EmbeddingGemma 2, which is the first time the decision lane appears in a general local-runtime release note. It also compacts restored recurrent state after a speculative rollback in the MLX path — a correctness fix for hybrid attention models — and lets ollama launch codex see installed models in Codex’s /model picker.

Pi v1.2.0 (12 October) publishes configuration schemas: add $schema to settings.json, models.json, keybindings and themes and editors validate and complete them, with the theme schema moved to pi.dev/schemas/theme.schema.json — a breaking change for anyone who had the old reference, and startup themes now get the same validation as reloads. With an OpenRouter key it also lists only the models that key may use, with that key’s context size, output limit and prices, including regional endpoints; PI_CLIPBOARD_OSC52=1 pushes copied text to the clipboard of a multiplexer attached from another machine, where Pi cannot otherwise detect the remote client; and codemode rows now show input constraints and elapsed versus total time.

oh-my-pi v18.10.2 → v18.10.3 (12 October, repo at 35,122 stars) is a token-accounting release, which matters more than it sounds: replayed Claude thinking turns were being counted about 55% over the input actually billed, and screenshots and non-frame images were priced at the OpenAI rate for every model — Claude billed them roughly 16% higher — both now counted against the reading model’s own catalog rules, so compaction and context sizing match the bill. OpenAI Responses and Codex estimates now include replayed reasoning and none while a cold session rebuilds it. It also added CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS, an escape hatch for gateways that reject pre-release fields, and changed the task tool so subagents return their final message as plain text by default, with task.structured opting back into schemas.

mlx-serve v26.10.2 was promoted from pre-release to stable on 12 October (the release covered here on 11 October, now final): storyboards that turn a sentence into a run of shots joined into a clip up to two minutes long, YuE2 songs with an editable score, Stable Audio 3 effects, 1024×1024 images in about eight seconds on an M5 Ultra, and chat that keeps answering while the same server renders. Hermes v0.21.7 and Blaizzy/nativ v0.4.0 are unchanged since their entries this week.

The through-line: the decision lane stopped being a curiosity and became a wire format. Five releases from five unconnected directions, all speaking TypeSafe’s /v1/systemone shape, all answering without generating a token, and all shipping the number next to the claim — an index with the kit version attached, a request count, a coverage figure, a phishing row. When the engine work becomes a copy-paste of the previous decision model, the standard has already won.