Letting an AI Type Commands on Your Server: Shellby's Three-Tier Gate and Five Guardrails

Shellby has an AI Agent mode: you state an ops goal in one sentence — “install nginx and reverse-proxy to :3000” — and it plans the steps, runs commands one at a time, reads the output, and decides what to do next, until it’s done.

This is the highest-value feature, and the highest-risk. Letting a language model type commands on your production server is one careless step away from rm -rf. So the core of the design isn’t “how do I get it to do work” — it’s “how do I stop it from wrecking things while it works.” This post covers two layers: a per-command three-tier approval gate, and a per-loop set of five guardrails.

Why a tool loop, not plan-then-execute

Start with the architecture, because it decides where the safety boundary lives.

The tempting candidate is plan-then-execute: have the model lay out a full plan, the user approves the whole plan, then it runs step by step. Sounds clean, but the plan is stale by step two: the port might be taken, the distro might not be what you assumed, the last command’s output can invalidate everything that follows. Ops work is inherently “look, then re-plan.”

The final choice is a single agent + client-side tool loop: the model emits a tool call → the client runs it through an approval gate → the result is fed back → the model looks at it and decides the next step → until it says it’s done. The plan’s value isn’t lost, just demoted to a first-round “task understanding + expected steps” so the user has a mental model.

One decision is a prerequisite for safety: the loop must run on the client. Tools execute over your SSH connection, and both the connection and the credentials live on your device. A server-hosted agent can’t touch your server at all — that’s not a tradeoff, it’s physics. And precisely because the loop runs locally, there’s a place to insert an approval gate before each step.

Layer one: the three-tier gate — the classifier is the boundary, not the model

Before each command runs, a pure-logic CommandClassifier classifies it locally:

Tier	Rule	Behavior
Read-only	Whitelist regex: ls / cat / grep / ps / df / systemctl status / journalctl / uname…	Auto-approved, collapsed
Mutating	Not on the read-only whitelist, not destructive	Tap to approve
Destructive	rm -rf / dd / mkfs / shutdown / reboot / pipe to sh/bash / redirect to system paths / chmod -R…	Red-boxed hard confirm, double tap

There’s one iron rule here, and it’s the foundation of the whole safety model: the model’s self-reported rationale is for the user to read only — it never participates in the safety decision.

Why hammer on this? Because if you trust the model’s self-description to decide whether to allow a command, your attack surface becomes “can you trick the model into calling this command read-only” — and a language model can be prompt-injected, or just wrong. The classifier is local, deterministic, unit-testable code that looks at the command itself. The model can lie or err; the classifier won’t. The safety boundary has to live in deterministic code you control, not in the model’s judgment.

Two supporting details:

• Rejections carry a reason back. When the user taps “reject and explain,” the reason goes back to the model as a tool_result (is_error: true), so it adjusts its approach instead of dumbly re-sending the same command.
• Agent commands never broadcast. Shellby has a “command broadcast” feature (one input sent to multiple sessions). Agent commands go to the current session only, never through the broadcast path — you don’t want the AI’s one rm flying to ten machines at once.

Layer two: five loop guardrails — LLMs drift, so tie a few nets

The per-command gate handles “is this one command dangerous.” But an autonomous loop has another class of risk: it drifts — spins, fakes completion, circles in place, burns money. These aren’t security holes, they’re the normal state of “the model is unreliable,” and they need guardrails.

Guardrail 1: max iterations (default ~15, later raised to 30). The crudest backstop, but it’s only the last one — the real drift fixes are below.

Guardrail 2: no-progress. Three consecutive rounds of nothing but “unknown tool calls” means this provider’s tool format is incompatible; stop early and say why, instead of spinning to the iteration cap. Unknown calls are also surfaced as visible steps for diagnosis — a silent failure is harder to debug than a loud one.

Guardrail 3: unfinished-plan nudge. The model has a classic drift: it announces “next I’ll do X,” then stops (end_turn) without any tool call. If its own plan still has unfinished leaf steps, we don’t trust its “done” — we auto-inject a “please continue the remaining steps” nudge, up to twice. It tracks the latest plan and computes real completion by leaf nodes.

Guardrail 4: stall detection. Given a plan, six consecutive rounds with no growth in completion means it’s circling in place. The classic case is the model repeatedly running read-only probes to “understand the whole picture” but never actually changing anything. Then stop early and give a clear diagnosis (suggest telling it to start acting in a follow-up, or switching to a stronger model), instead of burning to the iteration cap before declaring Done.

The stall guardrail has a counterintuitive trap, learned the hard way: running a mutating/destructive command or writing a file resets the stall counter. Because it once misjudged a session that was busy editing a config with sed — just too lazy to update its plan — as “circling” and cut it off. Actually doing work (even without updating the plan) shouldn’t be punished; only “multiple rounds of no progress AND only read-only probing” counts as a stall. The guardrail is there to catch spinning, not to punish it for heads-down work.

Guardrail 5: redaction. Command output is always run through a Redactor before going back to the model. The agent is more dangerous than read-only diagnostics — it actively runs commands to pull more output, so it’s likelier to hit keys and tokens. This step can’t be skipped. Output truncation is also careful: before feeding back, keep the head and tail, and explicitly mark “N bytes omitted in the middle” — never truncate silently, or the model will make wrong calls on incomplete information.

write_file: why read the old content and diff first

Of the four tools, write_file’s design deserves its own note, because it embodies the same philosophy.

It doesn’t use shell heredoc; it uses SFTP. Before writing, it reads back the old content over SFTP, computes a diff, renders the diff as an approval card so you can see exactly which lines this change touches, and only writes once you approve.

Why go to this trouble? Because “let the AI edit a config file” is a high-risk action, and a diff is the most natural interface for a human to review a change. Letting the model just echo ... > /etc/nginx/nginx.conf gives you no visibility into what it changed. Read-then-diff makes “review” a first-class part of the product.

Writes run with the login user’s permissions; when root is needed, it suggests writing to /tmp first then sudo mv — no sneaky privilege escalation. Writes to critical system paths are classified as destructive by path and go through hard confirmation.

Takeaways

The biggest lesson from building this: when you wire an unreliable executor (an LLM) into a high-risk system (a production server), safety can’t rely on the executor’s good behavior — only on the deterministic structure you build around it.

• The dangerous-or-not decision lives in a local, unit-testable classifier, not in the model’s self-description;
• Loop drift is caught by several orthogonal guardrails — each handles one failure mode, and each allows a “genuinely working” exemption to avoid false positives;
• High-risk actions (file edits, destructive commands) are made into first-class interactions that require a human nod, not buried in an automatic flow.

Models will keep getting stronger, but the principle — distrust the executor, keep the safety boundary in your own hands — won’t age. An agent’s value is that it can act autonomously; an agent’s safety is that its autonomy is always framed by you.