One OpenAI Endpoint, Three Very Different Processes Behind It

LLM-Bridge has exactly one outward shape: OpenAI Chat Completions. But inward, nothing the three backends emit looks alike —

• claude is a stream of async message objects from the Agent SDK (AssistantMessage, StreamEvent, ResultMessage);
• codex is line-delimited JSON on a subprocess’s stdout, one event per line;
• agy is plain text on a subprocess’s stdout, with no structure at all.

Converging these three beasts into one tidy output is the adapter layer’s only job. This post is about where, exactly, that normalization happens.

One base class, three implementations

Each provider is a file under providers/, all extending BaseProvider, exposing just three methods:

class BaseProvider:
    async def complete(self, request) -> ChatCompletionResponse: ...
    async def stream(self, request) -> AsyncIterator[ChatCompletionChunk]: ...
    async def list_models(self) -> list[ModelInfo]: ...

Whether the backend is an SDK or a subprocess, what comes in is an OpenAI ChatCompletionRequest and what goes out is an OpenAI Response / Chunk. The routing layer only talks to these three methods; it never knows whether an SDK message object or a blob of plain text is underneath. The differences must get eaten in this layer — not one byte is allowed to leak upward.

Inbound: three ways to assemble a prompt

OpenAI’s messages is a structured array with roles. Of the three backends, only claude’s SDK has a separate system-prompt channel; the other two eat a single blob of text. So normalization’s first step is flattening the structured messages into each one’s required shape:

• claude: system is pulled out to the SDK’s system_prompt; the rest is assembled as user / assistant into one block;
• codex / agy: everything flattened into one block, system prefixed [System Instructions], assistant prefixed [Previous Assistant Response], so the model can still tell who said what.

There’s an honest drop here: because it’s chat-only, messages with role="tool" get flattened away rather than pretend-handled. The drop is a deliberate design decision, not a bug — the docs say so.

Outbound: collapsing three streams into one chunk

On the output side, the three streams’ events look completely different, but they all end up calling three constructors: make_role_chunk (the opening role frame), make_content_chunk (a text delta), make_final_chunk (the tail, with finish_reason and usage). A single StreamState threads one stream’s id and model name.

The only difference is how you extract that text from each native event:

• claude: in the SDK’s StreamEvent, a content_block_delta event’s delta.text_delta is the incremental text;
• codex: in the line-delimited JSON, the text field of a type=="item.completed" with item.type=="agent_message"; turn.completed means done;
• agy: no event stream — just stdout.read(4096) block by block, forwarding whatever it reads.

So there’s an honest difference worth documenting: claude and codex are token-level streaming, agy is chunk-level — its CLI only gives plain text, and the gateway can’t conjure token boundaries out of nothing, so it forwards the blocks as they arrive. The granularity you can hit depends on what the underlying tool gives you; you can’t fake it.

Even “list the models” has no unified answer

You’d think “list available models” is the simplest step. It turns out all three answer in a different shape:

• claude: the CLI has no list-models command at all. With an Anthropic API key set, it uses the free Models API (listing only, never inference), with a one-hour cache; without one, it falls back to a hardcoded list;
• codex: reads the CLI’s own ~/.codex/models_cache.json — a cache the CLI refreshes on its own runs, the closest thing Codex has to a “list-models API,” and you still have to filter internal models by visibility=="list";
• agy: the tidiest, fully dynamic via agy models, then a slug↔display-name map (Claude Sonnet 4.6 (Thinking) ↔ claude-sonnet-4.6-thinking).

Outward, those differences get smoothed away again: unified into provider/model format, plus a layer of aliases (fable, opus, sonnet, haiku, gemini-pro, flash) so common models are one word away. Half the adapter layer’s job is forwarding; the other half is shouldering “each vendor has its own quirks” on the upper layer’s behalf.

The honest blank: agy has no token count

usage (input/output token counts) is also uneven across the three: claude reads it from ResultMessage.usage, codex from the turn.completed event’s usage, and agy — plain-text output, no token count at all.

The choice at that point is: leave it blank, return an empty UsageInfo(), rather than smear on a fake number estimated from string length. A made-up token count is worse than no token count — it looks real and gets treated as real downstream. The adapter layer can smooth over format; it can’t smooth over information that simply doesn’t exist underneath. That, you can only leave honestly blank.

Takeaways

One outward OpenAI shape, three inward processes, normalization all pressed into the adapter layer:

• One base class, three implementations: complete / stream / list_models, differences not allowed to leak to the routing layer;
• Inbound flattens structure: structured messages → each vendor’s prompt; chat-only means explicitly dropping role="tool";
• Outbound collapses into chunks: three native streams → three unified constructors, granularity set by whether the backend gives tokens or plain text;
• Even listing models isn’t unified: Models API / local cache / dynamic command, unified outward into provider/model + aliases;
• Leave blank what you can’t smooth: agy has no token count, so return empty — never invent a fake one.

In one line: the adapter layer’s value isn’t making three vendors look the same — it’s handling honestly the places they genuinely aren’t.