The Debate Is Too Long to Feed In: A Compression Before Synthesis

An Open Council debate goes like this: several models answer, peer-review each other, cross-examine another round if they disagree a lot, and finally hand off to a “chairman” model that synthesizes the final answer. The problem: several models × several rounds, the text fed to the chairman grows linearly and may blow straight past its context window. It has to be compressed before synthesis.

When to compress: a 60,000-character gate

Compression doesn’t always happen. Before synthesis, it estimates the total character count of all answers, and skips outright if it’s under 60,000 characters, feeding them to the chairman as-is.

That 60k gate is computed: assume the chairman’s context window is about 100k characters (roughly 25k tokens of mixed content), take 0.6 as the trigger ratio, 0.6 × 100000 = 60000. Leaving 40% headroom for the chairman’s own output — compression’s goal is never “fill the window,” but “leave enough room for it to still answer well.”

What to compress, what to keep: seating by review score

Over the gate, it decides which answers stay full and which get compressed. The basis is direct:

• debate mode has review scores → sort by total review score descending;
• compare mode has no review scores → sort by model priority ascending (higher priority first).

After sorting, the top 2 are kept whole (verbatim), and only the rest get compressed. The logic: when the chairman synthesizes, the two highest-rated answers deserve a word-for-word read; the ones ranked below, a summary of key points is enough. Spend the limited context budget first on the views most likely to be adopted.

How to compress: never touch code blocks, keep head and tail, extract key lines

The actual compression isn’t a blunt cut. For each answer to be compressed:

1. Extract and stash the code blocks first. A regex replaces ``` fenced code blocks with placeholders — code blocks are always preserved verbatim — because in a technical answer, a truncated snippet is useless; better to compress a bit more elsewhere;
2. Split by line, keep the first 15 and last 10. The opening is usually the position and conclusion, the ending usually the wrap-up; these two ends are the densest;
3. The middle isn’t just dropped — it extracts key lines: headings (#), bullets (- *), numbering, bold lines, taking the first 3, each cut to 60 characters, assembled into a [... N lines compressed. Key points: ...] marker put back in the middle;
4. Finally, restore the code blocks to their positions.

The compressed result is stored separately in a response_compressed field, not overwriting the original response_raw. Synthesis consumes response_compressed ?? response_raw — use the compressed version if compressed, the original otherwise; and the original is always there, so audit and replay can see what the model actually said. Compression is a one-off view for the chairman, not a destructive edit of the source data.

An honest gap: LLM summarization isn’t wired in yet

Here’s the honest bit. Calling the periphery treatment “summarization” easily makes it sound like having the model intelligently compress a long answer into three-to-five core points. The code does design this LLM-self-summarization path — there’s a ready summarization prompt asking the original model to compress its answer into “3-5 core arguments / key conclusions / points of difference / risks.”

But what’s actually wired in and running is only the structural truncation above (keep head and tail + extract key points + protect code blocks). The LLM-summarization path is written but not yet connected to the orchestration flow.

This is a deliberate engineering ordering, worth stating clearly: ship the deterministic version first — truncation has zero extra LLM calls, zero extra latency, and predictable results, blocking the real problem of “text overflowing the context” up front; LLM summarization is smarter but costs an extra call and adds a layer of uncertainty (the model might summarize the wrong point), so it’s left for v2. Shipping “the dumb method that definitely solves the problem” first and queuing “the more elegant but pricier, less controllable method” behind it is a sound trade-off in a cost-sensitive setting like a long debate. Stating that current reality plainly is more honest than treating a design draft as shipped.

Takeaways

Compressing a multi-round, multi-model debate’s output “before feeding the chairman”:

• A computed gate: compress only over 60,000 total characters (= 0.6 × 100k context), leaving 40% headroom for the chairman’s output;
• Keep the top 2 whole by review score: spend the limited budget on the views most likely to be adopted;
• Never touch code blocks: a truncated snippet in a technical answer is useless, protect it first and compress elsewhere;
• Compression doesn’t destroy the original: the compressed version is stored separately, the original kept for audit and replay;
• Ship determinism first, leave LLM summarization for v2: truncation is the zero-latency, zero-call dumb method that blocks the real problem; the smarter but pricier plan queues behind.

In one line: compression’s hard part isn’t “how to make it shorter,” but “before you compress, getting clear on which information can’t lose a single character.”