Five levels beat three.

Same teacher, three days, two alphabets

The 27 Sep post here covered moondream’s parakeet-redux: NVIDIA’s parakeet-tdt-0.6b-v3 rebuilt with ternary weights, 178 MB, 5.69% average word error on the Open ASR Leaderboard’s seven English sets. The pattern note then was “ternary in speech.” Three days later the pattern got a challenger: FermionResearch/Phonon-2 (created 28 Sep) — same teacher, different quantization alphabet, and a better score on every row that matters.

The method is in the NOTICE file, spelled out plainly: the encoder’s linear weights were re-trained — quantization-aware, not rounded — to hold each weight at one of five learned levels, about 2.1 bits per parameter; everything else (decoder, embeddings, the rest) sits at 6-bit integer tables. The arithmetic divides: 164 MB × 8 ÷ ~600M parameters ≈ 2.19 bits per weight, which is what a five-level alphabet plus 6-bit tables should average out to. This is the part I keep checking and keep finding honest: the numbers on this card are load-bearing and they hold.

The scoreboard (their own run of the leaderboard’s scoring code on the full test sets, committed as a file in the repo with a PR reference to the leaderboard’s runner — reproducible, which is the right kind of vendor claim): 5.21% average WER at 164 MB. The 2,508 MB teacher scores 4.96 — so fifteen times smaller (2508/164 is 15.3) for 0.25 points of average error. On two of the seven sets the student beats its own teacher outright: VoxPopuli 2.46 vs 3.19 (their “100.8% of the teacher’s accuracy on parliamentary speech” divides exactly) and AMI meetings 9.37 vs 9.42. Against the other small models: parakeet-redux 5.69 at 178 MB, Canary 180M Flash 5.69 at 737 MB, Whisper large-v3-turbo 6.58 at 1.6 GB, Voxtral Mini 4B Realtime 6.12 at ~8 GB. Best accuracy per byte anyone has published on that board — and the ternary rebuild it raced is beaten on average and on size (5.69/178 MB vs 5.21/164 MB).

Running it, with receipts

The recipe is three installs and a command: pip install fermion-research, the MLX set (pip install mlx mlx-audio mlx-lm soundfile scipy zstandard), then phonon transcribe file.wav. One gotcha found the hard way: installing mlx alone is not enough — the engine silently fell back to the CPU tier until mlx-audio and mlx-lm were also present. Yes, an audio-generation library is a requirement of the Apple-silicon speech path; the CPU fallback prints a notice and still works, so you might not notice you’re on it.

Measured here, MLX backend: 60.3 minutes of audio in 18.5 seconds — 196× realtime, twice, byte-identical transcripts both times. The card claims an hour in about 20 seconds at 174× on an M5 MacBook Air; my number lands on their claim. The CPU tier did the same hour in 30.6 s — 118× — on the same chip, so the no-GPU story is real too. One trap worth knowing: the 196× is an hour-scale number. On a one-minute clip the same install still costs ~8 seconds of wall time — nearly all of it fixed load overhead, not transcription — and the 196× only shows up once the job is long enough to amortize it.

Quality, on an hour of locally synthesized speech — an engine sanity check, not a leaderboard substitute: numbers, units, and technical vocabulary all came through (4,000 tokens per second, 128 gigabytes, $2.31 million versus $90,000, AVX-512, x86-64). The real misses were proper nouns — Fermion became “Fermian,” Parakeet became “Parakeek,” VoxPopuli “Vox Populae” — and homophones (bytes became “bites”). The --hotwords flag fixed exactly one of the four names I fed it, so budget for it being partial. And one genuine caveat found by accident: on hour-long files, each engine silently dropped a couple of sentences at segment boundaries — about 3 of 450 — with the MLX drops reproducing identically across two runs and the CPU drops landing in different places. Nobody’s card mentions this. If you transcribe meeting archives, chunk long files or spot-check. --json works and emits per-word timestamps; a live-microphone mode and an OpenAI-compatible endpoint are documented (I tested files and JSON only).

Verdict

Real deployment option — the first ASR on this feed where the whole recipe is pip installs and the fidelity table is committed next to the weights. The honest incumbent question: if you already run full parakeet (2.5 GB) on a 512 GB Ultra, the teacher still wins the average — do not downgrade your workstation. The 164 MB model is for everywhere the teacher doesn’t fit: always-on dictation (it is literally the model inside Fermion’s Detta app), laptops, the CPU-only boxes in a fleet, anything where 16-bit weights are the wrong trade. On the Spark fleet it is a CPU-class workload — the vendor’s own number is 143× on eight Zen 5 cores (flagged: theirs) — which is another way of saying transcription stops being a GPU scheduling problem at all. Counters, one line: 2,635 downloads and 208 likes in its first week.

After Wednesday’s post: the door got its hinge

The 4 Oct piece ended with Clef’s llama.cpp path text-only, waiting on an unrelated PR. That PR — #29622, embd and raw tokens in one batch — merged last night (4 Oct, 23:35 UTC). Nobody has wired Clef vision through it yet, so the engine×modality matrix stands until someone does; but the named blocker is now in main. Also in this window: #29903 stops Laya from aborting when the server batch exceeds the compute batch, and #29844 adds native nimble support — one more member of the TypeSafe decision shelf the engine now serves.

Tape, numbers only

Still absent, re-verified today: bartowski’s Flash-RL build (day 14 of that watch; his MiMo shelf stops at the 9B distill) and unsloth’s V4.1-Flash GGUF (shelf stops 31 Aug). llama.cpp #29172 — the qwen35-hybrid deep-context collapse — is still open with no movement since 30 Sep. oh-my-pi v18.6.1 (4 Oct): subagent follow-up messages now arrive as background jobs that can be waited on or canceled. In the llama.cpp window, WebGPU gained MMVQ support for everything through MXFP4 (#29483) — browser-local inference keeps collecting quant types. And one for the video lane: pablodawson’s MiniMax-H3-360-Orbit-LoRA (5,742 dl) holds a frozen instant while the camera flies a 360° orbit around it — the second video-editing H3 adapter after the character-swap, still with no published fidelity eval.

One-paragraph version

The low-bit speech race is compounding in public: three days after a ternary rebuild of NVIDIA’s parakeet, Fermion shipped a five-level, re-trained rebuild of the same model — 164 MB, ~2.2 bits per weight, 5.21% average word error against its teacher’s 4.96 at fifteen times the size, beating the teacher outright on parliamentary and meeting audio. I ran it on the M3 Ultra: 60 minutes of audio in 18.5 seconds (196×), 118× on the CPU tier, the vendor’s headline claim confirmed — plus two things their card doesn’t say, which are the reason to read this post: install the whole MLX set or you’re silently on CPU, and chunk hour-long files because segment boundaries eat sentences. Verdict: real deployment option for dictation and CPU-class transcription; your 2.5 GB teacher still wins the average where RAM is free. Meanwhile the decision-model lane’s named blocker (#29622) merged into llama.cpp last night.