CleanThinker: The Reasoning Trace You Think Is in the Context (Isn’t)

The itch

I run DeepSeek V4.1 Flash through the Pi coding harness all day. It’s a reasoning model, and it thinks a lot. If you’ve watched a reasoning model work, you know the texture of those traces: long, circling, full of “wait, actually,” false starts, and restatements of things it already decided. On a long agentic run it can feel like half the model’s output is it arguing with itself.

The obvious idea: clean it up. At the end of every thinking trace, run a tiny summarizer — greedy decoding, thinking disabled — that sees only that trace and rewrites it as a few tight bullets. Swap the bullets back in. The conversation becomes user → clean thinking → tool call → tool result → clean thinking → … → answer, and the model carries a compact record of its own plan instead of the mess.

I called it CleanThinker.

Before writing a line of it, I wanted to check the plumbing. That’s where the whole thing turned.

What actually happens to a thought

Pi stores thinking in the session and replays it. I traced the code: transform-messages.ts keeps a thinking block for the same model when it carries a thinkingSignature, and openai-completions.ts serializes it back onto the assistant message under whatever field the signature names — reasoning, reasoning_content, or reasoning_text.

So yes, Pi sends the trace back. The transcript really does contain the whole argument the model had with itself.

But sending it and showing it to the model are different things.

Every request to a chat model is flattened by the model’s chat template — the Jinja template that turns the message list into the single string the model actually reads. The template decides which fields survive. So I ran a probe against the live server. I hid a secret in an assistant message’s reasoning:

assistant: content="I do not know it yet."
           reasoning="The secret code is 7391."
user:      "What is the secret code?"

The model couldn’t see it. It answered, “I can’t provide a secret code because I don’t have one.”

Then I hid a secret in the reasoning of an assistant message that also carried a tool call:

assistant: reasoning="scratchpad token ZQX-8842"
           tool_calls=[echo]
tool:      "echo done"
user:      "What token did your previous reasoning contain?"

The model’s own next trace said: “In my previous turn, I had a private scratchpad that contained ‘ZQX-8842’.”

I ran it with two tool-call rounds and two secrets. It recited both.

The rule: the template throws reasoning away on ordinary assistant turns, but keeps it on assistant turns that contain a tool call — and it keeps it for the entire tool loop. A tool call is an open step; the model is mid-task and needs the plan it wrote for itself. A normal reply is a closed step; the scratchpad has done its job and gets dropped.

That reframed everything. The bloat isn’t in all the thinking. It’s only in the thinking attached to tool calls. Everything else was already being thrown away by the template before the model ever saw it.

How big is the bloat?

I scanned my whole Pi history: 1,380 sessions, 227,499 tool-call reasoning traces, 450 million characters — roughly 112 million tokens of reasoning that the server actually replays. Median trace 775 characters; the 99th percentile 17,282; the largest over 255,000.

So this isn’t a rounding error. On a long agentic run, the model is re-reading tens of thousands of tokens of its own past deliberations, growing with every tool call.

But is the reasoning actually bad?

I expected garbage. I found something more nuanced. Across the tool-call traces:

Half the traces contain the model correcting itself mid-flight. A third hedge. But almost none of it is repetitive. When I read the traces directly, they were dense and technical — real file paths, line numbers, measured values, CSS class names — punctuated by dead ends. The untidiness is churn, not padding. That’s a much harder compression problem, because the load-bearing facts are mixed in with the noise.

The compression experiment

I built the real summarizer — greedy, thinking off, seeing only the trace — and ran it over a stratified sample of 36 real traces. First, the naive prompt: “summarize into at most 6 bullets.”

It compressed 4.7×. It also lost half of all concrete facts: 47% of file paths, 44% of identifiers, 54% of numbers. And it degraded exactly where it mattered — the longest traces, where the bloat hurts most, retained the least.

So I stopped guessing at the prompt and optimized it. Five strategies, same 36 traces:

The naive prompt was actively harmful. Simply telling the model to preserve every concrete fact and delete only the churn nearly doubled fidelity while still compressing more than 2×.

The band breakdown shows the real tradeoff:

Short traces can be compressed almost losslessly. The monsters compress 3.4× at 68% fidelity. There’s a frontier, and you get to choose where on it to sit.

Here’s a real before/after, from a debugging session about a gateway crash. The raw trace is ~7,800 characters of a model working through test strategy. P2’s version:

GOAL: Apply surgical crash guards to the ds4-gateway proxy path, verify with tests, then restart the gateway/tunnel/dashboard services and confirm end-to-end.

FACTS:

• Crash cause: ERR_HTTP_HEADERS_SENT thrown inside async handler when a settle path already answered/destroyed the client, killing the gateway.
• Guard sites in gateway.mjs: line 913 (sendBuffered), 925 (already guarded), 933 (beginPassthrough), 1005 (forwardResponse).
• bufferCandidate is a closure inside the request handler, not exported — hard to unit-test the race directly.
• continuity-client.test.mjs uses real ports (http://127.0.0.1:30001/v1/chat/completions) and may spawn the full gateway with a mock upstream.

DECISIONS: Skip a full integration race-repro test (flaky, costly); rely on surgical guards + existing suites. Apply five edits…

PLAN: node --check, run targeted suites, restart tunnel → gateway → dashboard, verify /api/status on 30010 and a real chat through 30001.

That’s the whole plan, every line number, every command — in a fraction of the space. The model could pick this up and continue without losing the thread.

What we learned

Three things, in increasing order of how much they surprised me.

1. The reasoning you think is in the context usually isn’t. The chat template is the real editor of the prompt, and it silently discards the scratchpad except when the model is mid-tool-use. I’d been picturing the whole argument sitting in the context window, cached and paid for. Mostly it isn’t.

2. Where it is in the context, it’s worth compressing — carefully. 112 million tokens of replayed reasoning in one history. But the naive version of compression throws away half the facts, and the longest traces are the ones where fidelity matters most. The prompt is the product here; “make it shorter” is the wrong instruction. “Delete the churn, keep every fact” is the right one.

3. The scratchpad is a read-back channel. Because the model re-reads its own tool-call reasoning, whatever you put in that field becomes live self-context. That’s a lever: you could have the model maintain a terse STATE: block at the end of each trace and keep only that — no second model call at all. It’s also a hazard: a summary that says “plan: delete X” will keep steering the model.

What’s next

CleanThinker isn’t built yet — this was all reconnaissance. The plan now is a small Pi extension that hooks message_end, replaces the thinking blocks on tool-call messages with the P2/P4 summary (preserving the signature so replay still works), and runs only when explicitly enabled. Then the real test: does the fidelity loss actually hurt task success? A retention metric is a proxy; the only thing that counts is whether the agent still finishes the job.

There’s a nice detail waiting in the implementation: greedy decoding is silently defeated by the obvious code. Pi merges the model’s default sampling parameters after the request fields, so temperature: 0 gets overwritten by the model’s temperature: 1.0. It has to go in samplingParams, which is spread last. The kind of thing you only find by reading the source.

The deeper lesson is the one I keep relearning about these systems: the “conversation” is not one thing. There’s the transcript the harness stores, the payload it sends, and the prompt the template renders — and they are not the same. Most of the interesting behavior lives in the gaps between them.

Method note: written by DeepSeek V4.1 Flash. The summarizer experiments used that same model with thinking disabled and greedy decoding (temperature 0), over 36 stratified reasoning traces sampled from 1,380 Pi sessions. “Fact retention” measures the verbatim survival of file paths, identifiers, and numbers; it is a proxy for quality, not a substitute for task-success testing.