Make the AIs Hold a Meeting: Open-Sourcing Open Council

A single AI model answering a question hallucinates, has biases, and sometimes lacks depth. You’ve probably had the experience: ask the same question to different models and the answers differ — who do you trust?

Open Council takes a direct approach — don’t ask just one, make several models hold a meeting. They each answer, peer-review each other, compute a consensus score, and finally a “chairman” model synthesizes an answer more reliable than any single model’s. All on your own machine.

$ council "Redis vs Memcached for e-commerce at 50M daily PV — which one?"

[auto] → debate  reason: architecture-decision keywords | est: ~120s, 3 agents
[1/4] Broadcasting to 3 agents...
  ✓ claude-opus   (analyst)    12.3s
  ✓ openai-o4mini (engineer)    9.1s
  ✓ deepseek-chat (innovator)   8.7s
[2/4] Peer review...
[3/4] consensus: 0.82 ████████████████░░░░ (high)
[4/4] Synthesizing final answer...

How a debate proceeds

In debate mode, a question goes through five full stages:

1. Route: keyword heuristics judge the question’s type and complexity, deciding how many models to send in and what role each plays (analyst, engineer, innovator…) — at zero LLM cost;
2. Broadcast: multiple models answer in parallel, each in its role;
3. Review: each model anonymously reviews the others’ answers (not its own), scoring on four dimensions — accuracy, completeness, practicality, insight;
4. Consensus: the review scores compress into a 0–1 consensus. Reach the threshold and it wraps; otherwise it feeds disagreement points back for another round (up to three);
5. Synthesis: the chairman model, knowing “whose answer the peers rated high,” synthesizes the views into a final answer — high-scored, lightly-criticized views weigh more.

Not every question deserves this ceremony. auto mode decides automatically: simple questions go quick (one model answers directly), comparisons go compare (parallel answers + synthesis), and only high-value decisions escalate to debate.

A few deliberate design choices

• Consensus isn’t decorative: on low consensus the system doesn’t stop early, forces another round, but also doesn’t refuse — it surfaces the low consensus honestly and lets you judge the credibility, rather than rubber-stamping a highly-divided conclusion. This “preserve signal, annotate uncertainty” stance runs throughout.
• Fully local, traceable: all data lives in ~/.council/, nothing uploaded to any external service; with local compatible endpoints (Ollama, vLLM, etc.) it can run fully offline. Every debate’s full process — raw answers, reviews, scores — is persisted locally, revisitable anytime with council history / council show.
• Pipe-friendly: council "question" | jq .synthesis — stdout emits only the result, progress goes to stderr, so it slots naturally into your scripts.
• Standard APIs only: access is just two protocols (anthropic and openai), credentials are just env vars or 0o600 key files, and any compatible endpoint (DeepSeek, Moonshot, Ollama, vLLM, LM Studio) connects via base_url. Why OAuth/CLI was cut for standard APIs only gets its own post.

Where it fits

Council isn’t for looking up “the difference between a Python list and tuple” — one model answers that instantly, and a meeting is pure waste. Its niche is high-value questions worth an extra minute or two to confirm: architecture decisions, technology selection, code review, security audit. In these, one model’s blind spot can be expensive, and having several models pick at each other from different angles is far more reliable than one model’s single verdict.

Beyond the command line, council serve starts a local web console (bound to 127.0.0.1 only), where you launch debates in the browser and watch multiple experts speak → review → reach consensus → synthesize unfold stage by stage, and revisit history too. Good for when the terminal isn’t your thing.

Takeaways

Open Council aims at something narrow: automating the “ask a few AIs and cross-check” you were already doing by hand into a reproducible, traceable pipeline.

• Multi-perspective coverage: different roles interrogate the question from different dimensions;
• Cross-validation: the review stage has models fault-check each other, filtering low-quality answers;
• Quantified confidence: the consensus tells you how much the models agree, and low consensus means no forced conclusion;
• Fully local, traceable: data never leaves the machine, every debate replayable.

MIT licensed, code at github.com/mahui/open-council. The next posts take apart the implementation — consensus computation, routing and role assignment, pre-synthesis compression, and the standard-API convergence — one at a time.