The most impressive quant of the month won’t open in your llama.cpp

PrismML did something genuinely strange this week. It rebuilt a 27-billion-parameter Qwen model so that every weight in it is only −1, 0 or +1 — ternary weights, no number ladder at all — and the whole model fits in a 6.7 GiB file. No quant of this claimed quality class has ever been this small. The crowd noticed: 410 likes, #8 trending on HuggingFace, three copycat repos spawned within a day. The download counter reads zero.

Now, a brand-new repo starts at zero — that part means nothing by itself. The strange part is whose zeros these are. PrismML’s previous ternary model moves about a million downloads a month. The people applauding this one are that install base. They showed up, liked, and left without clicking. A like is free; a download means your machine can open the file. So something stopped thousands of eager people from pressing a button they plainly wanted to press.

My first guess was the boring one: it must run badly. The card argues the opposite — a 84.78 thinking-mode average across 14 benchmarks, against 72.59 for a conventional 2-bit quant and 85.18 for a 4-bit build that needs roughly three times the disk. Those are the lab’s own numbers on the lab’s own hardware, so I don’t buy the numbers yet. But if they’re even in the right zip code, this is near-4-bit quality in a file about half the size of the Q4_K_M (~15 GiB) that people actually run today — and it should be downloading thousands of copies a day.

So I went to the file instead of the comment section. HuggingFace lets you download just the first few megabytes of a 6.7 GiB model, and the header at the top of a GGUF states, in plain numbers, how every tensor inside is stored. I pulled the header over HTTP this morning and read it myself: 402 of the model’s 851 tensors declare themselves as tensor type 142. The llama.cpp everyone installs knows type numbers up to 35. Your loader doesn’t handle this gracefully — it doesn’t know the format exists and refuses to open the file at all. The card’s promise that it “runs on llama.cpp (CUDA, Metal, CPU)” is technically true only if “llama.cpp” means PrismML’s own 511-star fork, which was still landing plumbing commits this morning.

Down in the card’s own format notes, there’s an admission dressed as an engineering footnote: it ships two custom packing formats because “practical deployment needs packing formats that efficient kernels can consume” — which is lab-speak for the ecosystem’s kernels can’t, yet. (One marketing-math note while we’re counting: the card’s “~9.3× smaller than FP16” is measured against its idealized 1.72-bits-per-weight figure. Against the file you’d actually download it’s closer to 8× versus the FP16 original, and about 2× versus the 4-bit you actually run. Claims like that aren’t dishonest — they’re aspirational, which is a different thing worth tracking.)

Here’s what this means for your machine today: nothing. This model runs on no Mac, no gaming PC, no laptop unless you’re happy living inside a vendor fork, and you shouldn’t be. But ternary is arguably the most serious idea in compression right now — three states carry about 1.6 bits of information per weight, and that’s where the 6.7 GiB comes from — and this is a lab with a proven shelf behind it. So put one thing in your calendar instead of this launch: the day stock llama.cpp grows a type for these tensors, downloads will move overnight — and a 27B that fits in a file smaller than most 4-bit quants becomes somebody’s everyday model on an ordinary laptop. That’s the release this post was actually about. I’ll be watching for it, and you should too.

Tensor types read from the file’s own header via HTTP range requests this morning; download and like counts from the HuggingFace API; the fork’s activity from its commit feed. Benchmark and speed claims are PrismML’s, labeled as such.