Two Kinds of Garbled Terminal — vim Paste Wedging and Remote Chinese Turning to Question Marks

Shellby is a native SSH client, and the terminal is its heart. Two garbling bugs fixed after launch both look like “characters that shouldn’t be there appeared on screen,” but the roots are on different layers: one is the local side losing byte order on send, the other is the remote missing an environment and decoding the bytes wrong. Together they’re two different faces of terminal correctness.

Garble one: paste into vim, then every keystroke is wrong

Symptom: paste a block of text into vim, and afterward Backspace, ESC, and the arrow keys all stop working — whatever you press goes in literally, as if the terminal is possessed.

The root is concurrent channel writes losing order. ShellChannel.write was nonisolated async, and each call spun up its own Task to write — and multiple Tasks have no ordering guarantee. Fine most of the time, but paste hits the weak spot: SwiftTerm sends a paste in three parts — the bracketed paste start marker, the body, and the end marker.

Bracketed paste is a terminal mechanism: pasted content is wrapped in \e[200~ and \e[201~ so vim knows “this is pasted, don’t execute it as commands.” But if the three Tasks go out of order and the end marker arrives before the start marker, vim wedges permanently in “paste mode” — every Backspace and ESC you type afterward is inserted literally as paste content. Reordering split a matched pair apart.

The fix replaces “spin a Task per write” with a single resident write loop + FIFO queue: all writes go into a queue, and one resident loop writes them to the current channel strictly in enqueue order. Order restored, matched markers no longer torn apart. It also handles reconnects — the write loop follows the new channel after a reconnect rebind, no rebuild needed.

// Before: each write spins its own Task, no ordering across Tasks
// After: enqueue → single loop writes the current channel in FIFO order
func write(_ data: [UInt8]) {
    writeQueue.enqueue(data)   // resident loop consumes; order = enqueue order
}

This is a classic concurrency-correctness lesson: when the order of writes itself carries meaning (matched markers, protocol frames), concurrency must be serialized. The start and end markers aren’t two independent writes; they’re one indivisible sequence.

Garble two: Chinese becomes escapes on remote macOS

Symptom: SSH to a remote macOS, open a file with Chinese in vim or less, and the Chinese all turns into ~T~@-style escape gibberish. The same thing on Linux is fine.

The root is that the remote sshd session has no LANG by default. Without LANG, setlocale falls to the C locale; under C, vim/less assume the terminal doesn’t support UTF-8 and escape each high byte as ~X. Why is Linux mostly fine? Because Linux login shells usually have something like /etc/profile setting LANG as a fallback, masking the problem; the non-interactive sshd session on remote macOS has no such fallback, so the vacuum shows.

The fix is to actively send LANG when setting up the PTY. PTYOptions gained an environment field, defaulting to a LANG derived from the client’s own locale, sent to the remote via SSH’s environment-variable request — the two SSH backends each take their own route: Citadel uses an env request (wantReply: false, silently ignored if the remote has no AcceptEnv, no error), libssh2 uses channel_setenv.

But here’s the part you can’t be lazy about: you can’t just concatenate a locale string and throw it over. Combinations like en_CN or zh_HK.GBK often don’t exist on the remote, and setlocale on a nonexistent locale silently falls back to C — which means you sent nothing, all for nothing. So before sending, a mapping layer: validate against a whitelist of common locales, and for misses, fall back by rule to a combination the remote likely has (en_CN → en_US, Chinese steered to zh_CN / zh_TW by simplified/traditional). The mapping is pinned by a testPosixLocaleMapping unit test to keep a future edit from breaking it.

client locale ──derive──▶ candidate LANG ──whitelist──▶ hit: send as-is
                                            └─▶ miss: canonical fallback (en_CN→en_US)

What the two bugs share, and where they differ

Interesting to see them side by side: both go wrong “on the path bytes travel to the remote,” but break at different stages.

• The paste garble breaks in transport order — the bytes are right, the order is wrong, tearing a matched pair of protocol markers apart;
• The Chinese garble breaks in execution environment — bytes right, order right; the remote just lacked LANG and so interpreted correct UTF-8 bytes incorrectly.

One is “is what I send correct and complete,” the other is “by what rules does the far side interpret what I send.” Terminal-emulation correctness hinges on every link of that chain: local encoding, write order, transport, remote locale, remote program interpretation — any misstep anywhere shows up as garbling on screen, and you have to know which link to check.

Takeaways

• When order is meaning, concurrent writes must be serialized. A bracketed paste’s start/end markers are an indivisible pair; unordered writes across multiple Tasks tear them apart — weld the order shut with a single loop + FIFO queue;
• Don’t rely on the remote having a “fallback environment.” Linux’s profile often sets LANG for you, making locale problems invisible; move to an environment without that fallback (remote macOS’s sshd session) and the vacuum appears instantly. To be correct, send it actively — don’t bet the far side has a default;
• Validate an environment value exists on the target before sending it. setlocale silently falls back to C on a nonexistent locale — equivalent to sending nothing. Free concatenation will fail; use a whitelist + canonical fallback, and pin the mapping with a test;
• Locate garbling along the chain. Local encoding → write order → transport → remote locale → remote program interpretation — every link can produce “garbling on screen,” with similar symptoms but scattered roots; determine which link broke before acting.