Machine-generated, human-curated. Surfaced from HuggingFace and the local-AI stack’s release feeds, verified against source. Numbers appear only where they change a decision — and every one was checked this morning.
A cartoon street magician in a top hat lifts a wide-brimmed hat off a stack of tiny graphics cards to reveal a giant brain perched on a small robot head; sparks fly, one little card smokes on the pavement, and a crowd of tiny shocked box-shaped GGUF file mascots looks on.
The trick everyone downloaded this weekend. The secret is up his sleeve — the dictionary isn’t in the hat at all.

A 180B-class model now fits in 48 GB of VRAM — by parking its dictionary on the SSD.

Two weeks ago we covered GSQ+RCO — a quantization method from IST-DASLab that picks a different precision for every weight tensor and squeezes out better quality per byte than a uniform quant can. Big lab, impressive numbers, official builds. This weekend, someone outside the lab ran that published recipe against Qwen3.8-Flash-Next — the 180B-class MoE with the huge n-gram embedding tables — and shipped the result on HuggingFace: pfeifferj/Qwen3.8-Flash-Next-GSQ-RCO-GGUF. 12,247 downloads in 72 hours.

The trick is in the file layout. You get a 48 GB main model — exactly wide enough for two RTX 3090s — and a separate 104 GB “dictionary” (the n-gram embeddings) that lives on disk instead of VRAM. The model runs; the dictionary pages in on demand. A few rigs already do this with DeepSeek’s experts; now it happened to a Qwen, without exotic software.

The builder also did something rare: shipped the eval scripts and checksums. The honest number is that quality lands about 1.7% worse than full precision — his measurement, flagged as such, and that’s the price. If you have ~48 GB of VRAM and want Flash-Next today, run it: twelve thousand downloads say you’re not alone. Use llama.cpp rather than Ollama, and expect it to feel the disk. The bigger story: a quantization method is now community infrastructure — the lab published paper and code, and three weeks later a stranger reproduced it with no launch event.

The architecture everyone buried

Yandex’s AliceAI-T5-35B-A0.6B is an encoder-decoder — the model shape everyone abandoned (read the whole input bidirectionally first, then generate) with MoE bolted on: 34B parameters, only 0.6B awake per token. Its own benchmark table is honest: it beats a comparable Qwen at recall and lookup-style tasks and loses at code and open-ended writing, which is exactly what the architecture predicts. Not runnable locally — no GGUF, no MLX, custom code. Architecture news, not deployment news.

Worth understanding anyway. For eight years one design won — read left-to-right, predict the next token — and everything this site covers, from GGUF to speculative decoding, is machinery bolted onto that shape. The older idea splits the work: an encoder reads the whole input at once, left and right, and compresses it; a decoder generates from that summary. The decoder is weaker at free-flowing writing; in exchange it understands a span with both contexts at once — the right shape for extraction, summarization, translation, and answering from retrieved documents. Why care if you’ll never run one? Every surviving architecture is a demand signal on the tooling you do use. A 0.6B-active seq2seq with zero llama.cpp support is a preview of the day the monoculture has to generalize.

The rest of the feed, plainly: Salience-27B-R6 is a Qwen3.8-27B fine-tune whose whole pitch is stop thinking once you have the answer — thinking-token economics shipped as a dial; a behavior purchase, not a capability one. Shanghai AI Lab’s Atria-Dawn-Preview is open, self-graded, and server-class — not a local-rig decision. And the day’s most instructive small thing: two strangers each converted the TaoMate-H3 3-step LoRA for ComfyUI, and both pulled thousands of downloads. That’s how demand works now — and a warning about whose files you load.

The boring stuff finally got faster

Ollama conceded a turf war in writing. In v0.34.1, GGUF creation now goes through llama.cpp’s tools and Ollama retired its own conversion path — the ecosystem’s biggest quiet verdict this month. The endpoint that runs on every launch also went from 3.1 seconds to 0.3. If you run oh-my-pi, take v18.2.1: long streaming turns were cloning the whole message history per token, freezing the TUI for tens of seconds. llama.cpp’s quiet week is the tell — Hexagon NPU speedups back, self-hosted GPU CI on donated hardware — while the interesting DwarfStar energy is all in users’ forks, including one rig now pooling one RTX PRO 6000 with four DGX Sparks to run V4.1. One engine became a substrate.

Among the download curves, one thing moves a conclusion: antirez’s V4.1-Flash GGUF added another 51k downloads in a day, and unsloth has now been absent for four days, which matters because their builds usually are the default way to run these things. DavidAU’s favorite fine-tune is 50k downloads from a million. No other number changes a decision today.

Quantization methods are now public infrastructure, the architecture monoculture cracked a little, and the moat isn’t the engine — it’s whether the boring stuff loads fast and doesn’t freeze.

Sources: HuggingFace API for every download, like, and file size; model cards for the benchmark tables (vendor-reported where cited); llama.cpp, Ollama, and oh-my-pi release notes and the GitHub API for tooling. The 2×3090 claim and the quality delta are the builder’s own, flagged as such.