A quantization method got 100,000 downloads in two days

Quantization normally asks one brutal question of a model: do all of your weights fit in N bits? A new release from ISTA-DASLab stops asking it. Their method — GSQ (Gumbel-Softmax Quantization) does the low-bit work, RCO hands out the budget — assigns a quantization type to every individual tensor: layers that earn precision keep it, lazy ones get pushed to 2-bit, all under one total-size budget. Crucially, what comes out is a plain GGUF — it loads unmodified in llama.cpp, Ollama and LM Studio, with MTP speculative-decoding support included. The adoption is the headline: 100,110 downloads and 256 likes in about two days — roughly 50,000 pulls a day, for a method rather than a model. I’d take that number over any benchmark table the same release could print.

The reason the method exists is a specific trade-off: at 2–3 bits, vector quantization has historically beaten scalar quantization on quality — but vector methods need a bespoke runtime. GSQ is designed to close most of that gap while staying in standard scalar formats like GGUF: vector-style quality that your existing loader can already read. The group behind it is the DASLab that built QuIP-style compression, so it knows which fight it’s picking. The card’s quality scores — task averages holding up as bit-width drops — are the lab’s own, so they’re claims; the download counter is the part that isn’t.

What you can do with it today: the release is Qwen3.8-27B at four sizes with the vision projector (mmproj) included. If you already run a uniform quant of that model, this is a swap-the-file upgrade — same per-token compute, same tooling, more quality per GB — and a 2–3-bit 27B is single-GPU or 96 GB Mac territory (a footprint estimate, not a measured speed). If you’re not on Qwen3.8-27B, the thing to notice is that the method is base-agnostic: the interesting calendar item isn’t this file, it’s the builds that follow on the models you actually run.

The day’s other two entries earn less enthusiasm. K2-Horizon-MoVA-36B-A4B is a genuinely new architecture from IFM — a 36B MoE with Mixture-of-Values attention, 4B active per token, native 512K context — and 4B active alongside 512K context is the standout combo for long-context local work, in roughly 20–24 GB at 4-bit (estimate), single-GPU or big-Mac territory. But the counter reads 0 downloads and 92 likes — released yesterday, which means brand-new, not rejected — and the entire benchmark table is the lab’s own. One flag on that table: it claims to beat models “up to 15× its size,” yet the named rivals it lists — G9v3-39A5B, Qwen3.6-35B-A3B, Muse-Glimmer-30B, Gemma 4 31B — are all about its own size. Community quants already exist and the training data and code are “coming,” but serving support is unproven. Worth watching, not moving to.

The second is DavidAU’s Qwen3.8-27B TURBO NEO-CODER-MAX, a fine-tune of an abliterated Unsloth build of the same 27B that claims to cut thinking tokens to between a half and a tenth — think less, answer faster — with the adoption to be taken seriously (39,646 downloads, 144 likes) and a benchmark problem to be ignored: its headline “735 ARC-C / 882 ARC-E” is DavidAU’s own private metric and compares to nothing. Use it as a fast, uncensored writer model; skip its score sheet.

The pattern underneath, and the reason a method out-downloaded every model on the same day: value is moving from “bigger model” to “same model, smarter bytes.”

Downloads, likes and community-build counts read directly from HuggingFace; the GSQ/RCO quality figures (arXiv 2604.18556, 2605.00649) and the K2-Horizon benchmarks are the labs’ own, labeled as such.