Open mic.

Models

fishaudio/drama-3 (839 downloads / 346 likes; repo created 1 October, the launch wave — blog, ComfyUI node, community GGUFs — landed 13–14 October) is Fish Audio’s new-generation TTS, and the pitch is direction rather than narration. It is a 35B-parameter MoE with about 3B active per token: a text backbone with hybrid full-and-linear attention and 120 routed experts stored in FP8, everything else bf16, wired to a Codebook-MTP audio decoder that decodes ten codebooks causally across the whole sequence. A script carries up to 16 speaker tags (<|speaker:0|> …), so one generation can be a whole scene; you write stage directions inline where they should land ([sighs], [laughs], sounds, pauses); it clones a voice from one short reference clip plus its transcript; long text is chunked and streamed automatically; and the tag list claims roughly ninety languages. SGLang-Omni shipped a serving cookbook on launch day.

The feature with no local equivalent is editing as continuation: send the previous takes’ audio codes back as context, change one word, and the model re-renders the line so that only the words you named move — the blog’s demo chains a second edit onto the first with both earlier takes in context. That is take-level, word-local audio editing from a generative model, and it falls straight out of the architecture: audio lives in the context as tokens, so revising it is just conditional generation.

Evidence. The checkpoint’s counter is early — 839 downloads against 346 likes, the high like ratio this log keeps reading as people bookmarking a release they have not run — but the build shelf is unusually fast and unusually official: Jmica/Drama-3-GGUF, tagged as built with Fish Audio from a pinned checkpoint revision, hit 1,212 downloads in about two days — already out-pulling the checkpoint it quantizes. Fish’s own INT4 export followed on 14 October, the mlx-community published bf16/8/4/3-bit packs within hours, and there is a ComfyUI node. For pedigree: the lab’s previous flagship, S2 Pro, sits at 34,148 downloads / 1,428 likes since March.

Vendor numbers, once: on one H200 with SGLang-Omni the card prints time-to-first-audio of 88.9 ms at p50 on a single stream (RTF 0.131), an RTF of 0.373 at 16 concurrent streams — audio generated about 2.7× faster than it plays back — and a +72% throughput gain from moving the vocoder to a second GPU at 256 streams. Server-class, vendor-measured; no independent evals exist yet, and the license is a real decision point: Fish Audio Community License v1.0 is free below US$500,000 annual revenue (their blog’s number), negotiate above.

Critical read. As a plain narrator it is a 35B heavyweight, and the tiny models win every resource count that matters for “read this aloud.” What nobody else ships — open or closed — is directed multi-speaker scenes with word-local editing, and the download race between the GGUF and the checkpoint says the people arriving want it on their own hardware. Verdict: Marginal — adopt only if directed scenes and take-level editing are the job; worth a re-check once independent quality evals show up, because the counter is moving now that the builds exist.

Quantizations

Jmica/Drama-3-GGUF (1,212 downloads / 9 likes, created 13 October, built with Fish Audio from pinned revision dd64a85) is how the 35B actually runs locally: 15.88 GiB at Q2_K, 18.93 at Q3_K_S and 22.91 GiB at Q4_K_S with Q8_0 backbone matrices, plus a 21.78 GiB variant that quantizes the backbone to 4-bit as well. The routed expert banks take the Q2/Q3/Q4 storage and — note this part — the audio tower and the waveform codec stay FP16 in every package. The packages are tagged for audio.cpp, the pure-C++ ggml audio engine this section has watched grow all month. Verdict: Real deployment option — a 24 GB GPU or a 32 GB-class machine runs the scene; the caveat is that these are experimental conversions with no fidelity table printed, so the first thing a serious user should do is A/B a Q4_K_S take against the INT4 official export (0 downloads so far) and the bf16 MLX packs (also hours old, batch caveat).

The other shelf move worth printing: LiquidAI/LFM2.5-24B-A2B-GGUF, the official llama.cpp build of yesterday’s lead model, reached 17,807 downloads in three days — 4.4× the bf16 checkpoint (4,067) — confirming the pattern the entry named: the official shelf got there first, and it is the distribution channel.

Tape (covered builds, counters moved): ConwayResearch’s Underdog-Saluki 27B IQ2-mix 269,829 → 311,969; AtomicChat’s Qwen-Image-2.1-Turbo-Uncensored encoder GGUF 208,575 → 258,320; autotrust’s JEV-35B-NVFP4 → 33,343, still ahead of its bf16 parent; whistle → 23,589; clef-flash 26,636 and clef 18,304.

Fine-tunes

Nothing new cleared the gate. Tape: nerkyor’s Coder390 (Qwen3.8-27B SFT/RLOO) 83,887 → 114,351, about +30,000 in a day as the GGUF and NInfer shelves compound; autotrust’s JEV-35B 17,426 → 27,970. Sub-gate: DavidAU’s Qwen3.8-Flash-Next GSQ-RCO abliterated rider read zero downloads — the SC117 parent it clones has 906,582; the rider is a repost, not a release.

Technical term of the day: residual vector quantization (RVQ)

A language model emits tokens, but sound is a waveform — tens of thousands of numbers per second. Something has to turn the waveform into a vocabulary, and that something is a neural audio codec; drama-3’s repo ships one as codec.pth. The core trick inside most of them is residual vector quantization. An encoder turns a short frame of audio into a vector. A first codebook — a learned dictionary of, say, 1,024 typical frame-shapes — stores the single nearest entry, which becomes one token. That entry is an approximation, so the error is what remains: a second codebook is trained to describe exactly what the first one got wrong, a third describes what the second missed, and so on. Each stage quantizes the residual — the leftover — which is the name. Stack the entries and you get the original frame back, close enough to sound right.

Why local AI cares: this is the contract between audio and every transformer technique this log covers. Once audio is tokens, a TTS model is just a language model over them — which is why drama-3’s decoder is literally named Codebook-MTP(ten codebooks predicted causally, multi-token prediction doing the spec-decoding), why editing a take means sending previous codes back as context, and why the token economics are brutal compared to text: a text model spends three or four tokens per word, an audio model spends tens of tokens per second of sound, forever. Drama 3’s throughput table counts acoustic frames per second because frames are the tokens. It is also why the GGUF builds leave the codec in FP16: quantize the codebooks and you hear it — the same lesson MiniMax-Music3’s community packagers printed in September (“audio-code LMs degrade audibly below Q5”), and the reason that project had to build its own 169M-parameter RVQ encoder when the vendor didn’t ship the tokenizer: codes are specific to the codebook that produced them.

The trade-offs are a bitrate ladder: more codebooks at higher rates sound better but multiply the tokens the model must generate, which multiplies the compute per second of audio; too few, and you get the classic codec artifacts — metallic sibilants, smeared music, hiss under the floor. RVQ’s known failure mode is codebook collapse, where later stages stop being used and the ladder silently narrows. The practical rule for anyone shopping the audio shelf: look at frames-per-second times codebooks — that is the real token bill — and never let anyone quantize the codec.

Harness improvements

Ollama v0.40.5-rc0 (14 October) ported the MLX backend’s custom Metal kernels to CUDA: the hand-tuned kernels that made Ollama’s MLX-by-default bet fast on Apple silicon now run on NVIDIA GPUs, plus custom tool calls accepted in OpenAI Responses history and an MLX version bump. The backend that was a Mac strategy is going cross-vendor.

llama.cpp reached b11641 (15 October) and the headline is #30506: LiliCorr — NVIDIA’s speculative-decoding refinement (arXiv 2608.20530) merged with conversion support for LiliCorr draft checkpoints. Instead of always taking DFlash’s top token per slot, a small transformer computes an in and an out vector for every candidate and picks the one most compatible with the previous position’s choice — buying acceptance rate with extra compute, and the PR prints a llama.cpp-vs-vLLM dot-product parity table to prove the port is faithful. Around it: the plain KV cache honors PARTIAL_ONLY (#30514), the server saves only restorable slot states to the prompt cache (#30498), Metal gained a BF16 tensor-API flash-attention kernel (#30491), CUDA/HIP got a chunked GDN kernel (#29353), Vulkan schedules cm2 matmuls L2-friendly (#30215), and gguf-py detects NFC-normalized vocabularies (#30540).

oh-my-pi v18.11.0 (14 October) fixes a compaction bug with a nice mechanism: the remote-compaction preflight counted the JSON-escaped request, inflating quote-, backslash- and newline-heavy tool output — disassembly listings — by 10–20% and refusing sessions that actually fit; it now counts text the way the model reads it, and V1/V2 compaction retries on the history before the kept tail instead of failing when a late tool result overflows the window. Fitted output caps no longer overflow strict llama.cpp-family hosts; extensions get AgentTool.delegationGuidance; breaking: ApprovalDetailsOptions removed.

Hermes v0.22.1 (published overnight, one day after “The Threshold Release”) is a patch: Windows Store fixes, a curated model list pinned at the top of the model picker, Nemotron Super 3.5 default parameters, and plugin-catalog additions including Korean and Brazilian-Portuguese language packs plus lightningai, author-feed, wait-ledger and openinvest connectors.

audio.cpp (0xShug0’s pure-C++ ggml audio engine, 3,534 stars, still committing daily) keeps widening the local audio lane this post’s lead depends on: opt-in RKNN NPU sidecars now offload the kokoro and piper vocoders and a qwen3-tts speech decoder to NPUs over shared runtime plumbing, Granite Speech 5.0 TurboCTC gained CTC word timestamps, and a shared Voice Library page landed in the Studio. Its supported-models table now spans Breeze TTS 2, Chatterbox, CosyVoice3, DotTTS, Fish Audio S2 Pro and more. mlx-serve (v26.10.2) and Pi (v1.2.0) are unchanged since their last entries.

The through-line: the audio lane got what the LLM lane got in September — an open flagship, a same-week build shelf, and an engine layer maturing under it — while the harness layer quietly stopped caring which GPU you own: MLX kernels on CUDA, NVIDIA’s own speculation trick in stock llama.cpp, and a compaction bug fixed by counting tokens the way the model does. The codec is the contract; everything else is plumbing.