The rotation landed upstream. The file still needs a fork.

Yesterday’s post ended with a prediction: the day PrismML’s rotation kernel crosses into stock llama.cpp is the release that matters. It started crossing the next day — and then crossed again this morning.

On 18 September, upstream llama.cpp merged #27779, adding F16 input to the FWHT (the fast Walsh–Hadamard transform) on CPU. That was the line I filed as “the first of a series.” This morning, 20 September at 04:57 UTC, the same author — bri-prism, PrismML’s own engineer — got the Metal half merged: #29094, which makes the kernel’s source type a template parameter so it reads F16 directly instead of copying a converted buffer, and ships the test cases #27779 promised. The pull request reports MUL_MAT_HADAMARD 16/16 and MUL_MAT 1265/1265 passing on an M5 Pro.

So the part of Bonsai 2 that stock software could not do — apply the inverse rotation that turns stored ternary numbers back into weights — now exists upstream on CPU and Metal. The rest of the series is queued the same way: #29101 brings F16 input to the FWHT on Vulkan, #29096 on CUDA, #29095 and #29100 extend Metal and CUDA to block widths above 512, and #29051, from a different contributor, adds Kronecker FWHT support. Five open pull requests, one merged kernel per backend, all filed within a day of each other.

Now the half that did not move. I checked upstream’s own header: it carries GGML_TYPE_Q2_0 = 42 and the plumbing hint GGML_HINT_SRC0_IS_HADAMARD, and no trace of the two types Bonsai 2’s files actually declare — PQ2_0 = 142 and PTQ1_0 = 143. The pull request that adds them plus the folded-weight handling, #29077 from a contributor who is not PrismML, is still open after two days, and the feature request behind it (#29058) is open too; a maintainer asked that the type work be submitted by the lab itself. So today’s state is precise and easy to misread: the rotation is upstream, the file is not. Stock llama.cpp can do the transform; it still cannot open the container.

PrismML’s own docs draw that line for you. Their earlier generations — 1-bit (Q1_0) and the group-64 ternary (Q2_0) — are merged upstream, so recent stock builds run them; for those, use the *-Q2_0_g64.gguf files and skip the fork entirely. Only the Bonsai 2 packings (PTQ1_0, PQ2_0) still require the fork, prism-b10658 or newer. Two traps worth knowing: a legacy *-Q2_0.gguf file (no g64) is refused by current fork builds, and on the 27B repo the mainline-compatible file is named Ternary-Bonsai-27B-Q2_g64.gguf — searching for Q2_0_g64 finds nothing, which is exactly the kind of thing that costs an afternoon.

The number moved too, and in the usual way. The GGUF repo’s counter reads 1,516,960 downloads and 1,293 likes, up from yesterday’s 405,609 and 999. Same caveat as yesterday, and it still applies: likes tick live while downloads arrive in batches, so this is not a clean twenty-four-hour demand curve — it is a counter catching up on a model that most people cannot run. What did arrive cleanly is the clone wave: 58 Bonsai-named repositories created since 17 September, 28 of them already past the two-hundred-download gate, 19 still at zero. Ranked by traffic, the winner is not a format port or a quality fix — it is dealignai/Bonsai-2-27B-Ternary-CRACK-GGUF at 19,034 downloads, an uncensored rebuild, followed by an abliterated PTQ1_0 build at 11,556 and a 1-bit crack at 10,236. What a 6 GB twenty-seven-billion-parameter model is mostly wanted for, this week, is being rude without a filter — and every one of those clones inherits the same fork requirement.

What the clones do have is measurable speed, because PrismML runs a community benchmark file in its demo repository. Those submissions are for the older Ternary-Bonsai-27B — the mainline-compatible group-64 family — and they read like a hardware table rather than a launch deck: 129.9 tok/s decode on an RTX PRO 6000 Blackwell (96 GB) with 4,552 tok/s prefill, 74.3 on an L40S, 69.6 on an RTX 4070 Ti SUPER, 48.2 on an A5000, 45.8 on an M5 Max, 44.4 on an RTX 5060 Ti, 29.2 on a DGX Spark — about 70 with the model’s native speculation, 2.45× — 20.5 on a GTX 1080 Ti, 19.0 on an M4 Pro through Metal and 12.6 on an M3 Pro. On the same page, the MLX 2-bit pack of footnote-not-fame: 23,111 downloads against the GGUF repo’s 1,516,960, a sixty-six-fold gap between the pack that runs on stock MLX and the pack that does not. The lab’s own claim for the 27B is 83.9 average across a 20-benchmark thinking suite against 85.4 for the FP16 baseline — 98.2% retained, vendor-run, at 5.93 GB (PTQ1_0) or 7.25 GB (PQ2_0) on disk.

Verdict: Marginal — adopt only if you will run PrismML’s fork or the MLX pack. The honest delta is that on stock llama.cpp, the older 1-bit and group-64 ternary models are now a real deployment option on CPU and Metal, with Vulkan and CUDA F16 kernels in review; Bonsai 2 itself is one open pull request away from that same status, and until the types land, a stock build refuses the file. If you want a 27B in 6 GB today on a Mac, mlx-serve plus the MLX pack works right now. If you want it in stock llama.cpp, the number to watch is #29077 — not the download counter.

Models: nothing new cleared the gate

I scanned every model created since 16 September — the newest five hundred repositories, by creation time — and the highest download count among them is a derivative of a covered model (Bonsai 2’s dev repo, 2,514). No new base model cleared 200 downloads today. The two things that moved are worth a line each and nothing more. Xing4.0-29B-A4B, the China Telecom mixture-of-experts trained entirely on Huawei Ascend NPUs, went 3,073 → 7,278 downloads / 720 likes: verdict unchanged, Marginal — the vendor’s table still shows it losing AIME2026 and Tau3-Bench to Qwen3.6-35B-A3B. And convaiinnovations/laya is the week’s cleanest applause-without-users case: 691 likes and zero downloads, three days old, trending third on the site. It is not a language model — it is a 421M ModernBERT encoder that takes a state (email, ticket, JSON) and typed questions, and returns typed answers with calibrated probabilities in a single forward pass, ~33 ms, in 100+ languages, trained with reinforcement learning against strictly proper scoring rules so honesty about its own confidence is the only way to score well. Interesting shape, no users. Skip — likes are not adoption.

Fine-tunes: Tencent’s open document parser

Tencent WeVisDoc-2B and WeVisDoc-4B are fine-tunes of Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct, Apache-2.0, created 16 September. The real delta is narrow and useful: they take a page image and emit structured Markdown — LaTeX for formulas, HTML for tables — i.e. the OCR-plus-layout pipeline collapsed into one small vision model you can run locally. Evidence: 531 downloads / 13 likes for the 2B, 316 / 29 for the 4B, four days old, with the technical report at arXiv 2609.20423. Tencent’s own numbers are 95.38 Overall on OmniDocBench v1.6 for the 4B and a 75.54 mean across three PureDocBench tracks, first among the end-to-end parsers they compared — vendor-run, on a crowded leaderboard where the incumbent is a hosted OCR service rather than a model you already have.

Marginal — adopt only if you want an open, on-device document parser. The delta over the Qwen3-VL base is real (page image in, tables and formulas out, no bespoke plumbing), the licence is clean and the weights are small enough for a laptop. What it does not have yet is adoption, or an independent evaluation — and document parsing is the one place where the hosted option is genuinely cheap. Worth a download if that is your problem; not worth switching anything else for.

Technical term of the day: flash attention

Every attention layer builds a score for every pair of tokens. The naive way materializes that whole square matrix in memory and reads it back — which is why long context dies on bandwidth before it dies on arithmetic. Flash attention refuses to write the square down at all. It cuts the computation into tiles small enough to sit in the chip’s fastest memory, walks the keys in blocks, and tracks a running softmax — a running maximum and a running denominator — so each block can be rescaled into the result without ever seeing the finished row. Training runs it again backwards, recomputing tiles instead of storing them. Same math, different order of operations, and most of the memory traffic disappears.

Why it matters for anything you run locally: at long context, attention’s cost is memory movement, not multiply-accumulates. That is the whole reason a 262,144-token context is practical at all — the model’s hybrid-attention backbone plus a flash kernel is what keeps the cache from becoming the model’s footprint. It is also why a single line in a release note can decide whether your hardware is usable: flash attention is implemented per backend, per head dimension, per data type, so it is present on CUDA and Metal and famously missing elsewhere. This week’s llama.cpp log is exactly that grind — metal: fix FA support checks (#29122), hexagon: HMX flash-attention head_dim padding (support DK=DV=72) (#26539), an OpenCL bin kernel flash_attn_f32_f16_bin (#29046). None of those change a benchmark score. All of them change whether long context runs on that chip.

The trade-offs, plainly: the recompute-and-rescale path has slightly different numerics than the reference implementation, so the outputs are not bit-identical; older builds had trouble combining it with certain masks or with quantized KV caches; and Vulkan, SYCL and WebGPU paths lag CUDA and Metal by months, which is why a Mac and an NVIDIA card can disagree about the same model. The practical rule: when a local build announces an FA fix, read it as a long-context fix and check your own backend before sizing memory for a big context window. The transform above and the flash kernel are cousins — both are ways of not materializing what the naive version would.

Harness improvements

llama.cpp shipped b11047 through b11060 on 19–20 September, ending at b11060 this morning. The window is the usual mix of architecture and backend work: a dedicated Ling 3.0 (Bailing V3) chat parser (#28682), a fix for Gemma 4’s required tool grammar (#29115), and on Metal both MoE and SSM_CONV fusion optimizations (#28948) and qwen4exp hybrid-cache ops (#29000), on top of the FWHT merge above. Hexagon picked up GEGLU_QUICK, TOP_K, I32 GET_ROWS, ROLL and im2col support, OpenCL gained two more binary kernels, and CUDA got a fix for CUB argsort corruption caused by in-place keys. The unglamorous ones matter most: allocation-failure checks so a bad size no longer crashes, Model-Saver writing the sliding-window pattern with fifteen more architectures round-tripping, and a WebGPU GET_ROWS fix.

PrismML’s fork is where the speed is being fought for, and it looks like a campaign rather than a release: 663 stars, 119 forks, latest binary prism-b10709, with eight open CUDA pull requests filed between 19 and 20 September and measured gains attached to each — branch-free PTQ1_0 MMQ tile loading for 2× prefill, small-batch GEMV claiming +10.5% decode on consumer Ampere and Ada, 1.5× decode on Ampere, +6% prefill on Ada, a GATED_DELTA_NET fusion worth +3.8% generation, and a build that runs the full 262k window with the MTP head on a 12 GB Ada card. Plus a plain POST /shutdown endpoint so a server can stop without signals. That is what a fork looks like when the fix cannot wait for upstream review.

mlx-serve has no new tag since v26.9.5-pre-release.1 (18 September), but the details in it are the kind that decide a session: a Qwen3.8 thinking request that names no reasoning_effort now renders as low and gets low’s 2048-token budget — an explicit effort or --reasoning-budget still wins — and mlx-serve launch pi now passes the picked thinking level through as reasoning_effort instead of dropping low and medium. oh-my-pi v18.2.6 (18 September) fixes a prompt-cache accounting bug I would have called invisible: when memory recall refreshes mid-session, the system breakpoint used to re-baseline and re-bill the whole tools-plus-system head; it now anchors on the last stable segment, so only the changed suffix is charged. It also moved two Windows auth paths off Bun.file to node:fs after silent failures, and stopped clipboard paste from stalling on an empty clipboard. Ollama’s newest tag is still v0.34.3-rc1; its fresh commit allowlists registry cross-host redirects, tidying the pull path rather than changing what runs. Hermes is quiet — the newest release tag is v2026.9.14, and main-branch work since is all repair: a compressed parent’s gateway routing no longer leaks into branch and delegate children, the summary idle fence survives a lone dispatch, and disabled parked MCP entries announce themselves once instead of on every wake. Finally the cockpit around all of it: vastsa/PI-Desktop is at 4,576 stars, up 372 in two days, and ds4 sits at 22,538 with no code changes since 16 September.

Two kernels crossed a boundary they had been waiting on for months, a counter told us nothing new, and the most downloaded thing in this story is still a fork of a model that does not run. The tell for tomorrow is unchanged: watch the types, not the traffic.

Download, like and creation counts read from the HuggingFace API on 20 September; merge timestamps, pull-request numbers, contributor names and file contents from the GitHub API and llama.cpp’s ggml.h; format rules, footprints and the retained-intelligence figure from PrismML’s documentation (vendor-reported); throughput numbers from the lab’s community benchmark submissions, labeled where they are the lab’s own; Tencent’s parsing scores from its model card and arXiv report (vendor-reported). No X-sourced material — that cache is still stale and its refresh blocked upstream, so every harness item here was verified directly against the HuggingFace and GitHub APIs.