No Agent, One Command for All Server Metrics: Shellby's Monitoring

Shellby shows live CPU, memory, disk, load, and top processes once you connect to a server. Building this has a hard constraint: you can’t install any collector agent on the server. Users connect to other people’s, temporary, no-installing-random-stuff machines, so monitoring can only rely on what SSH itself can do — run commands, read output. This post is about pulling all metrics with one command, and how the parser has to be written to survive every distro.

One command, not five round trips

The obvious way to pull five metric classes is to send five commands: one for CPU, one for memory… but each command is one SSH round trip, and on mobile networks five round trips’ latency stacks up ugly — and it refreshes every 3 seconds.

So pack the five classes into one command, pulled in a single exec, separated by a delimiter that won’t appear in normal output:

cat /proc/loadavg; echo '@@SEC@@'; cat /proc/stat; echo '@@SEC@@'; \
free -b; echo '@@SEC@@'; df -kP; echo '@@SEC@@'; \
ps -eo pid,pcpu,pmem,comm --sort=-pcpu | head -11

One round trip brings back all five sections, and the client splits on @@SEC@@ and parses each. Five round trips compressed into one, cutting latency and overhead.

Prefer reading /proc — it’s a stable, parseable kernel interface, far easier to parse than the human-readable output of various top/vmstat. BSD/macOS remote command differences are left for later adaptation (they have no /proc).

CPU usage needs two samples and a delta

There’s an easy-to-get-wrong point: CPU usage can’t be read just once.

/proc/stat gives cumulative values of various CPU times since boot (user, system, idle, iowait…), not “current usage.” To compute usage, you have to sample twice and take a delta:

idle  = idle + iowait
total = sum of all fields
usage = 1 - Δidle / Δtotal

That is, remember the last idle and total, subtract this time from last time, and the ratio of the idle increment to the total increment, inverted, is the usage. On the first sample there’s no “last time,” so you only get a number on the second — which means the CPU cell has a one-beat delay when monitoring first opens.

The rest parse directly: memory from free -b (total/used), disk from df -kP (1K blocks need ×1024), load from the first three values of /proc/loadavg, processes from ps — note comm (command name) may contain spaces, so join the trailing fields back.

The parser is a pure function, degrading when fields are short

Parsing is all written as pure functions — take a text string, output structured metrics, touching no IO or state. This has two benefits:

First, easy to unit test. Real-machine Linux integration needs an environment, but the parsing logic can be tested with text fixtures — feed it all kinds of real/malformed /proc/stat, free, df output and assert the parse results. That’s also why, when the local macOS sshd has no /proc and can’t be tested end to end, the parser can still be thoroughly covered with Linux text fixtures.

Second, easy to degrade. Servers vary wildly — some distro’s ps might lack a field, some free format might differ slightly. The parser’s principle: if fields are short, continue or return nil, and never crash; if a whole section is missing (say, fewer than five sections split out), return directly and don’t proceed. Monitoring showing one fewer cell beats the whole feature crashing. When parsing untrusted data, a parser’s robustness matters more than its precision — you can’t guarantee every server’s output fits your format.

Overhead: throttle polling, keep history in memory

Live monitoring samples every few seconds, so overhead must be controlled:

• The polling interval defaults to a few seconds, not too dense — each is a real SSH command, and too frequent both wastes bandwidth and burdens the server;
• The CPU chart’s history keeps only the last 60 points in memory, not persisted — monitoring is a glance-at-it thing, no need to store it;
• Polling stops when the monitor panel closes — it’s a pop-up-to-look thing, stopping collection by the panel’s lifecycle, not looking means not collecting.

These are the general disciplines of any “live feature”: refresh rate, history length, when to stop — each must be set deliberately, or a polling loop running forever in the background is continuous bandwidth and battery drain.

Takeaways

Building server monitoring without an agent, a few takeaways:

• Pack into one command: splice the five metric classes into one exec with a delimiter, pull all in one round trip, don’t stack five round trips’ latency;
• Prefer reading /proc: the kernel interface is stable and parseable, easier to handle than human-readable tool output;
• Cumulative values need a delta: CPU usage takes two samples and an increment ratio, not one read;
• Write the parser as a pure function: easy to unit test (text fixtures, no real machine needed), easy to degrade (skip on short fields, never crash);
• Throttle live features: set the polling interval, history length, and stop timing deliberately, don’t let it silently drain bandwidth and battery.

Monitoring other people’s servers, all you can rely on is SSH and the bit of text the remote kernel is willing to tell you. Squeezing that text dry and parsing it robustly is the whole job.