Three Subprocess Invariants: Don't Turn a CLI Into a Zombie

In LLM-Bridge, the codex and agy backends both come down to one thing: turn a request into a codex exec / agy -p - subprocess call, then convert its output back to OpenAI format.

Sounds like a one-liner with asyncio.create_subprocess_exec. The hard part isn’t starting — it’s cleanup. In a concurrent gateway where the client can disconnect at any moment, one slip leaks zombie processes or deadlocks the child. This post is the three invariants I stepped my way into; editing these two providers, you can’t break a single one.

Invariant 1: prompt via stdin, not argv

The first instinct is to splice the prompt into the command-line args: codex exec -m gpt-5.5 "the whole conversation…". Fine for one sentence, it blows up on real chats — a prompt assembled from multiple turns easily runs to tens of thousands of characters and slams straight into the OS argument length limit (ARG_MAX); the child dies with E2BIG before it even runs.

So the prompt always goes through stdin:

args = [self.cli_path, "exec", "--json", "--skip-git-repo-check",
        "--ephemeral", "-m", model, "-"]  # trailing "-" means read from stdin
proc = await asyncio.create_subprocess_exec(
    *args,
    stdin=asyncio.subprocess.PIPE,
    stdout=asyncio.subprocess.PIPE,
    stderr=...,
)
proc.stdin.write(prompt.encode())
await proc.stdin.drain()
proc.stdin.close()

The command-line args carry only fixed flags; all variable-length content is fed through the pipe. close() after drain() so the child sees EOF and knows the input ended. Argument length has a ceiling, a pipe doesn’t — anything of uncontrollable length should never go into argv.

Invariant 2: an undrained stderr must be DEVNULL

This is the sneakiest one, and it shows up as “occasional hangs.”

For low latency, the streaming path reads stdout and forwards as it goes, and never reads stderr. If stderr is wired to a PIPE here, the trap is set: the OS pipe buffer is finite (typically 64KB); the child writes log lines to stderr, fills the buffer, and with nobody reading, it blocks on the write to stderr — stdout stops producing too, and the whole stream hangs. You think the model is thinking; really the subprocess is stuck on a pipe nobody’s reading.

The fix is to handle stderr differently on the two paths:

async def _run_cli(self, prompt, model, capture_stderr):
    ...
    stderr=asyncio.subprocess.PIPE if capture_stderr else asyncio.subprocess.DEVNULL,

• Streaming path (capture_stderr=False): stderr goes straight to DEVNULL, never fills, never deadlocks;
• Non-streaming path (capture_stderr=True): uses communicate(), which drains stdout and stderr together, so it’s safe to capture stderr with a PIPE and use it for error messages.

In one line: any pipe you don’t intend to read, set to DEVNULL; a full PIPE nobody reads is a scheduled deadlock.

Invariant 3: kill in finally, clean up the instant the client disconnects

The gateway is an SSE streaming response, and the client (a browser tab, a dropped network, a user hitting stop) can disconnect mid-stream at any time. When that happens, FastAPI calls aclose() on the async generator, and Python raises a GeneratorExit on the current yield line.

If you only clean up the subprocess on the “read to completion” path, then when the client disconnects halfway, the codex / agy process keeps running — burning CPU, holding your subscription quota, turning into an orphan process. A day later, ps shows dozens of zombies.

The fix is to put cleanup in finally, so it treats normal completion, exceptions, and client disconnect alike:

async with self._semaphore:
    proc = await self._run_cli(prompt, model, capture_stderr=False)
    try:
        yield make_role_chunk(state)
        async for raw_line in proc.stdout:
            ...  # parse, convert to chunk, yield
    finally:
        # Runs on normal exit, errors, and client disconnect (GeneratorExit)
        # alike: never leave an orphan CLI process.
        if proc.returncode is None:
            proc.kill()
            await proc.wait()

GeneratorExit triggers finally, finally checks whether the process is still alive (returncode is None), and if so kill()s it then wait()s. The wait() isn’t optional — skip it and you get a defunct zombie still holding a slot in the process table. The non-streaming path is the same: after a communicate() timeout it’s also proc.kill() + await proc.wait().

On the SDK side: use aclosing to propagate the disconnect

claude goes through the official Agent SDK, which owns the process lifecycle itself — but only if you pass it the “client disconnected” signal. The way is wrapping the SDK’s query() in aclosing:

from contextlib import aclosing

async with aclosing(query(prompt=prompt, options=options)) as messages:
    async for message in messages:
        ...

aclosing guarantees the generator’s aclose() is called on exit (including the GeneratorExit from a client disconnect), and that close propagates into the SDK, which then tears down the CLI process it started. Managing your own subprocess needs a hand-written finally kill; with the SDK it becomes aclosing — different mechanism, same thing to guard: no exit path may skip process cleanup.

Takeaways

The gateway’s hard part isn’t starting a subprocess; it’s cleaning up safely in an environment where the client can disconnect at any time:

• Prompt via stdin: argv has a length ceiling, a pipe doesn’t — keep uncontrollable-length content out of the command line;
• Set an unread stderr to DEVNULL: a full PIPE nobody reads blocks the child on its stderr write and silently deadlocks the stream;
• kill + wait in finally: GeneratorExit fires on client disconnect, finally unifies cleanup, wait() reaps to prevent zombies;
• Use aclosing for the SDK: propagate the generator close into the SDK so it tears down its own process.

In one line: making “someone else’s CLI” into a reliable backend isn’t about getting it to run — it’s about sealing every single exit path.