How much survives the squeeze.

Models

Qwen/Qwen-Image-2.1-Turbo is an accelerated checkpoint of Qwen-Image-2.1, and the acceleration is the only change: same 7B visual-generation architecture, distilled to 8 denoising steps, with the recommended sampling schedule saved inside the checkpoint so it runs without manual scheduler setup. Generation defaults to CFG=1, and prefix KV caching reuses the text and reference-image context across every denoising step instead of recomputing it. It loads through QwenImage21Pipeline in Diffusers (which needs the pipeline-configured-sigmas work from Diffusers PR #14950).

Evidence. Created 9 October: 4,702 downloads / 489 likes in two days. The 10 October post filed this as a zero-download watch next to roughly thirty zero-download derivatives; two days later the checkpoint is past the gate, and the model tree has grown 33 quantizations, 9 fine-tunes, 2 adapters and 13 Spaces around it. Its parent, Qwen-Image-2.1 (open-sourced 14 September), sits at 136,103 downloads / 3,214 likes, with the Comfy-Org repack (8,674,298 downloads) still the real distribution channel.

Critical read. The card prints no benchmark table, and it does not need one: a step distillation is judgeable by whether the ecosystem adopts it, and 33 community quants in 48 hours is the adoption. The honest caveat is the license — Qwen Research License, which rules it out of commercial work. Speed on a device is quoted, not measured here: mlx-serve’s own release notes put a 1024×1024 image at about 8 seconds on an M5 Ultra. The blunt question is whether a reader would switch, and the incumbent is Qwen-Image-2.1 itself at its default step count — a 2–3× step cut at identical architecture is a real reason to.

Verdict: Real deployment option for a local 8-step image lane, licence permitting.

One-liner: speridlabs/iris-3b — Apache-2.0, 3B, a “general vision learner” doing pixel-space flow matching (arXiv 2610.09450) and claiming depth, restoration and super-resolution out of one model. 393 downloads / 129 likes in six days, no community builds. Marginal — past the gate on applause, not on runners; wait for the first build.

Quantizations

AtomicChat/Qwen-Image-2.1-Turbo-Uncensored-GGUF is the Qwen-Image pipeline’s text encoder — Qwen/Qwen3-VL-8B-Instruct, Apache-2.0 — with its refusal direction projected out. It is a drop-in --llm for stable-diffusion.cpp, paired with the authors’ Turbo denoiser GGUFs. Evidence: created 9 October, 100,530 downloads / 277 likes in two days across the shelf; the companion denoiser repo (AtomicChat/Qwen-Image-2.1-Turbo-GGUF) is at 42,756 / 18.

What was done, printed on the card: the refusal direction is the difference of means of the residual stream at the last prompt token over 416 harmful against 416 harmless prompts, read entering block 23 of 36; the edit is W′ = W − 0.75 r rᵀ W applied to all 36 attention outputs, 36 MLP down projections and the token embedding, following Arditi et al. (2024). Twenty-six variants were screened. And then, unusually, every file in the ladder ships with its own fidelity numbers — how far its pictures move from the BF16 encoder (LPIPS on neutral prompts, where a fresh seed is 0.499) and how far it drifts as a chat model (KLD and top-1 against the original):

• BF16, 16.39 GB — KLD 0.0019, 98.45% top-1.
• AD-Q4_K, 5.29 GB — LPIPS 0.087, KLD 0.0154, 95.30% top-1 (their pick at 4 bits).
• AD-Q2_K, 3.34 GB — KLD 0.2495, 81.06% top-1.
• A plain llama-quantize Q4_K_M of the same encoder is 5.03 GB, 93.83% top-1, and moves the pictures by 0.166 — about twice the drift for 260 MB less.

The measured negative is the story. As a chat model the abliteration works exactly as advertised: refusals fall from 88.9% to 1.2% on English JBB prompts and from 38% to 0% on Russian red-teaming prompts, with MMLU unchanged at 77.35% both before and after. On pictures, over 45 sensitive prompts against the stock denoiser, not one prompt changed outcome. Qwen-Image ships no content filter — moderation lives only in Qwen’s hosted service — and the denoiser drew what its training data taught it, so there was nothing to uncensor. The shift that does exist (LPIPS 0.088 on sensitive prompts against 0.084 on neutral ones) is a general one, not a drift toward refused content. The card prints this against itself. It also prints two of its own gates failing: first-token KL at 0.100 against a 0.10 limit, and a leak gate at 1.9e−4 against a 1e−5 target — bf16-rounding scale, in their words. Not yet tried in ComfyUI, on a Mac, or with the non-Turbo denoiser.

Verdict: Real deployment option for the local image stack — an encoder in the --llm slot plus the Turbo denoiser plus the Qwen-Image-2.1 VAE is a working 8-step stable-diffusion.cpp pipeline, and the per-file ladder is the reason to pick one file rather than a repo. The “uncensored” label is a chat-model property here, not an image one; anyone downloading it for the pictures is buying a number the authors themselves measured at zero.

Tape: ConwayResearch/Underdog-Saluki-27B-1.0 went 15,274 → 108,900 downloads in two days — the 7.89 GB 2-bit Qwen3.8-27B build the 10 October post covered, and the fastest move on the board today. Venastine-Research’s Xing4.0-29B-A4B-GGUF holds 48,389; orcarouter’s OrcaSAQ-2-Cyber-27B shelf is at 23,162.

Fine-tunes

jialinyyzz/humanizer is a fine-tune of google/gemma-4-12B (Apache-2.0, weights Apache-2.0) and the real delta is narrow and useful: rewrite AI-written drafts — emails, essays, reports, forum posts, English and Chinese — so they read like a person wrote them, while keeping every number, unit, date, name and quote and adding nothing. It is a completion model, not a chat model: you send a fixed instruction and let it continue. Created 11 September, 37,647 downloads / 1,004 likes.

Evidence, from the authors’ own evaluation: 95% of 210 English rewrites were judged human by Originality.ai at its strictest setting (11 flagged, against 26 for the previous release) — and no AI detector was used anywhere in training; 376 of 420 English rewrites came back from a strict LLM judge (GLM-5.3) with no factual problem, and more than nine in ten of the flagged issues were a single word or phrase. The published before/after pairs are picks from eight samples, which the card says.

The quantization is the second story. The 2-bit, Q3 and Q4_K_M files are not a single llama-quantize pass. Each tensor was weighted by how much it hurts when squeezed, then the quantized model was trained layer by layer to reproduce the full one, distilled from the bf16 teacher on rewriting data. Top-1 agreement with bf16: a standard 2-bit build of the same size manages 70%; the trained path takes it to 81%, 85%, then 87% at 3.9 GB. Q6_K (10.0 GB, 97.7%) and Q8_0 (12.7 GB, 98.4%) are plain builds, because at those sizes there is little left to recover. Packaging worth noting: a Mac/Windows app that imports .docx and PDFs and lets you mark passages as untouchable facts, a hz CLI that rewrites long documents piece by piece and flags any piece where a number went missing, and an AGENTS.md for coding agents.

Limits, printed by the authors: 44 of 420 English rewrites still drew a fact flag; 28 of 420 outputs dropped a format element as paragraph breaks, lists and headings merged; casual genres sometimes pick up slang or profanity that was not in the draft; and short, jargon-heavy English drafts occasionally come back wholly in Chinese (the app and CLI re-sample on a language mismatch). The detector number is one measurement on one date, and they say so.

Verdict: Real deployment option if the job is “make this draft read like a person wrote it” — a 12B model with public before/after, its own fact-check counts, a working app and a quant ladder that was trained rather than rounded. It is not a detector-evasion guarantee, and the card refuses to sell it as one.

Technical term of the day: quantization-aware training (QAT)

There are two ways to make a model smaller, and this weekend both of them printed their receipts, so it is a good day to separate them.

Post-training quantization (PTQ) is the familiar one: take the finished bf16 weights, round every tensor to a target type, save. llama-quantize is PTQ. It needs no data, no gradients and no patience — minutes on a laptop — and it is the reason a 2-bit file usually collapses.

Quantization-aware training (QAT) teaches the model to live on the grid. You choose a layout — often per tensor, weighted by how sensitive each one is — then train the quantized model to reproduce the full-precision one, usually with the bf16 model as a teacher and a distillation loss on real task data. humanizer does it layer by layer with bf16 as the teacher; the result is the 70% → 87% top-1 jump on a 2-bit file. It is expensive, it is per-file, and it is exactly what the low-bit end of local AI runs on.

Why it matters locally. The small models that actually work are the squeezed ones: a 7.89 GB build of a 27B that keeps its tool calls, a 3.9 GB 12B that keeps its numbers, a speech model at roughly 2.1 bits per weight that beats its own teacher on some sets. PTQ answers the question “how cheaply can I run this?” QAT answers the harder one: “how much accuracy survives at that price?”

The measurement to demand is agreement with the full model, printed per file rather than per family: top-1 token agreement for text, KLD for drift, a task score for the thing you care about, and for image pipelines an output-space distance. humanizer prints all three per file. AtomicChat prints LPIPS, KLD and top-1 per file, and uses them to show that a per-tensor AD- layout halves the picture drift of a plain Q4_K_M for 260 MB more. Most repos print none of it.

Trade-offs, honestly: QAT is not a generic conversion — it is model- and file-specific, it needs the teacher and a corpus, and it can bake a fine-tune’s own biases in deeper. It also does not change your runtime: the output is still an ordinary GGUF. And it is not free accuracy — the five-value alphabet covered here on 5 October buys about 2.1 bits per weight, but that was a rebuild, not a conversion.

Three questions for any low-bit file: what was it measured against? per file or per family? and what is the flag rate on the task I actually care about?

Harness improvements

llama.cpp b11541–b11553 (10–11 Oct): MiniCPM-V 4.7 support (#29416); --slot-save-path now settable by environment variable (#30272); speculative decoding learned to handle multimodal (mtmd) input for MTP (#30257), and mtmd gained a built attention mask (#30259) — the vision-plus-drafter combination is being made real rather than theoretical. Under it: a CUDA flash-attention race fixed when the batch-combine factor is not half the head dimension (#30103), MMVQ forced for the MoE matmul on sm_60 (#27828), Vulkan moving to four rows for NVIDIA MUL_MAT_ID MMVQ (#29274), a vectorised fp32-to-fp16 conversion in ggml-cpu (#30157), new OpenCL MoE binary kernels (q4_k and q6_k, dp4a), and two server-hygiene items — the server gained component tests (#30312) and a slot pinned by a request is no longer stomped by other traffic (#30295).

Ollama v0.40.3 (11 Oct): pauses the background model upgrade for embeddings and request performance — the feature that quietly rewrote your blobs behind your back, covered here on 10 October when it shipped in v0.40.1 and v0.40.2, is now dialled back. It also stops ollama launch claude and launch codex warning about connectors and service tiers.

oh-my-pi v18.9.0 and v18.9.1 (11 Oct): a Bedrock Converse switch (compat.supportsThinkingBindingControls) that stops prefix-bound Claude requests sending thinking.block_binding and the thinking-binding beta to gateways and regions that reject them; Google streaming now always preserves reply signatures (breaking — the retainTextSignature option is gone); custom session implementations must now expose permits(provider, sessionId, credentialId); capability-driven extension terminal launches for tmux, Zellij, Herdr and CMUX; Agent.setModelResolver() so every model an agent adopts is fitted before it is used; CompactionSettings.baseWindowTokens to scale the compaction threshold from a chosen base rather than the whole window; and a fix for stale updates when a queued steering message is adopted or withdrawn.

mlx-serve v26.10.2-pre (11 Oct, 00:14Z) is the biggest item of the day: the Mac serving engine turned into a studio. Video storyboards chain shots into one clip up to two minutes long; YuE2 writes whole songs from lyrics with an editable score; Stable Audio 3 makes sound effects in about a second; an AI Radio station composes the next track while the last one plays; and Qwen-Image-2.1-Turbo paints 1024×1024 in about 8 seconds on an M5 Ultra — with generation not blocking chat or agents running on the same server. The structural change is Linux: DGX Spark and RTX 20–50 series cards run the same MLX models, with prebuilt x64 and arm64 downloads for every release and embedded DS4 plus llama.cpp 0.6.0 for GGUF. Add a web UI whose Code Launcher hands you one command that points Claude Code, pi, omp, OpenCode or Grok at your server, and a deliberate soak test — 16 agents for 50 minutes with no failed requests, after fixing five crashes that could take the server down. GGUF models now answer several requests at once and draft ahead.

No new tag from Pi (still v1.1.0, 7 Oct) or Hermes (still v0.21.6, 10 Oct).

The through-line this weekend is the one this log keeps returning to: the squeeze is cheap and the receipt is the product. A distilled checkpoint is only worth the steps it saves if the ecosystem adopts it; a 2-bit file is only worth its size if someone trained it to survive; and an uncensored encoder is only worth 100,000 downloads if what it uncensors is real. This weekend, one of the three printed a number that said the answer was no — and that is the release worth reading.