# Design: Responsive Web Terminal ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ BROWSER │ │ ┌──────────────┐ ┌─────────────────┐ ┌──────────────────────────┐ │ │ │ TerminalPage │ │ TerminalComponent │ │ TerminalConnection │ │ │ │ (router) │◄──│ (xterm.js + UI) │◄──│ (WS + heartbeat + echo) │ │ │ └──────────────┘ └─────────────────┘ └──────────────────────────┘ │ │ │ │ │ │ ┌─────┴─────┐ ┌──────┴──────┐ │ │ │ xterm.js │ │ sessionStorage│ │ │ │ + addons │ │ (scrollback) │ │ │ └───────────┘ └───────────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ │ WebSocket ▼ ┌─────────────────────────────────────────────────────────────────────────┐ │ FASTAPI │ │ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ │ │ │ terminal.py │ │ TerminalManager │ │ TerminalSession │ │ │ │ (WS endpoint) │◄──│ (session mgmt) │◄──│ (PTY + docker exec) │ │ │ └──────────────────┘ └──────────────────┘ └─────────────────────┘ │ │ │ │ │ ┌────┴────┐ │ │ │ docker │ │ │ │ exec │ │ │ └─────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ ``` ## Connection State Machine ### Client State Machine ``` ┌─────────────┐ │ IDLE │ └──────┬──────┘ │ mount ▼ ┌─────────────┐ │ CONNECTING │◄────────────────────────┐ └──────┬──────┘ │ │ onopen │ ▼ │ ┌─────────────────────────┐ │ │ CONNECTED │ │ │ (heartbeat active) │ │ └──────┬──────────┬───────┘ │ │ │ │ onclose/ │ │ ping timeout │ onerror │ │ │ ▼ ▼ │ ┌─────────────────────────┐ │ │ RECONNECTING │───────────────────┘ │ (backoff: 1→2→4→8→30s) │ onopen (success) └──────┬──────────────────┘ │ max retries (10) ▼ ┌─────────────────────────┐ │ DISCONNECTED │ │ (manual reconnect │ │ or navigate away) │ └─────────────────────────┘ ``` ### Server State Machine (per session) ``` ┌─────────────┐ │ PENDING │ └──────┬──────┘ │ ws.accept() ▼ ┌─────────────┐ ┌────►│ ACTIVE │◄────┐ │ │ (I/O loops │ │ │ │ + heartbeat) │ │ └──────┬──────┘ │ │ │ │ │ ws close│ new ws │ │ ▼ │ │ ┌─────────────┐ │ └─────┤ CLOSED ├──────┘ │ (cleanup) │ └─────────────┘ ``` ## Protocol Specification ### Message Types All control messages are JSON text frames. Raw terminal I/O uses binary frames. #### Client → Server | Type | Payload | When | |------|---------|------| | `ping` | `{ id: number }` | Every 15s of inactivity | | `pong` | `{ id: number }` | Response to server ping | | `resize` | `{ cols: number, rows: number }` | Terminal size changes (debounced) | | `input` | `{ data: string }` | User keystrokes (base64-encoded) | #### Server → Client | Type | Payload | When | |------|---------|------| | `pong` | `{ id: number }` | Response to client ping | | `status` | `{ status: "connected" \| "reconnected" }` | After auth + session ready | | `set_echo_state` | `{ enabled: boolean }` | When PTY echo flag changes | | `session_ended` | `{ reason: string }` | When container process exits | ### Binary Frame Convention - **Client → Server:** Raw UTF-8 bytes of user input. No wrapping. - **Server → Client:** Raw bytes from PTY master read. No wrapping. This avoids the current Blob→ArrayBuffer async conversion and JSON parsing overhead for the hot path. ## Frontend Design ### New Files ``` apps/web/src/ ├── components/ │ └── terminal.tsx (rewrite: state machine + reconnect) ├── hooks/ │ └── use-terminal-connection.ts (NEW: WS lifecycle, heartbeat, reconnect) ├── utils/ │ └── terminal-protocol.ts (NEW: message encoding/decoding) └── types/ └── terminal.ts (NEW: protocol types) ``` ### `useTerminalConnection` Hook Responsibilities: 1. **WebSocket lifecycle:** Open, close, reconnect with backoff 2. **Heartbeat:** Send ping every 15s, expect pong within 5s 3. **Local echo:** Write printable chars to xterm immediately, deduplicate server echo 4. **Resize:** Debounce resize events, send JSON control message 5. **Scrollback:** Serialize on disconnect, restore on reconnect 6. **State reporting:** Expose `status`, `latency`, `attempt` to UI ```typescript interface TerminalConnectionState { status: "connecting" | "connected" | "reconnecting" | "disconnected"; attempt: number; latency: number | null; // last RTT in ms error: string | null; } interface TerminalConnection { state: TerminalConnectionState; sendInput: (data: string) => void; sendResize: (cols: number, rows: number) => void; reconnect: () => void; // manual, bypasses backoff onData: (callback: (data: Uint8Array) => void) => void; onControl: (callback: (msg: ServerControlMessage) => void) => void; } ``` ### Local Echo Algorithm ``` 1. User types character c 2. IF c is printable ASCII AND echo is enabled: a. Write c to xterm immediately b. Add c to "pending echo" buffer c. Send c to server via WebSocket 3. ELSE (control char, arrow, escape sequence): a. Send c to server only b. Do NOT write to xterm 4. When server sends data: a. For each char in server data: - IF char matches head of "pending echo" buffer: → Pop from buffer (deduplication) - ELSE: → Write char to xterm b. If "pending echo" buffer grows > 100 chars (stale): → Flush buffer to xterm (server echo was lost) ``` ### Scrollback Serialization ``` ON disconnect: 1. buffer = xterm.serialize({ scrollback: 10000 }) 2. sessionStorage.setItem(`hq-terminal-${instanceId}`, buffer) ON reconnect: 1. buffer = sessionStorage.getItem(`hq-terminal-${instanceId}`) 2. IF buffer: xterm.write(buffer) xterm.write("\r\n\x1b[90m--- Reconnected ---\x1b[0m\r\n") 3. sessionStorage.removeItem(`hq-terminal-${instanceId}`) ``` ### Resize Debouncing Use `ResizeObserver` on the terminal container instead of `window.resize`: ```typescript const resizeObserver = new ResizeObserver( debounce((entries) => { fitAddon.fit(); sendResize(term.cols, term.rows); }, 200) ); ``` Rate limit: max 1 resize message per 500ms. ## Backend Design ### Modified Files ``` apps/api/src/ ├── api/terminal.py (modify: ping/pong, session_ended) ├── services/terminal_manager.py (rewrite: heartbeat tracking, batching) └── services/terminal_session.py (modify: batching read, echo detection) ``` ### TerminalManager Changes **Heartbeat tracking:** - Track `last_ping_at` per session - Background task: if `last_ping_at` is older than 60s, close the WebSocket **Message batching in read_loop:** ```python async def _read_loop(self, session, websocket): buffer = bytearray() last_flush = time.monotonic() while session.is_alive() and not session._closed: data = await session.read_output() if data: buffer.extend(data) now = time.monotonic() if buffer and (now - last_flush >= 0.016 or not data): await websocket.send_bytes(bytes(buffer)) buffer.clear() last_flush = now elif not data: await asyncio.sleep(0.001) ``` **Reconnect support:** - When a new WebSocket connects for the same instance, terminate the old session and spawn a new one - This is the docker exec limitation — we cannot resume a PTY, only replace it ### TerminalSession Changes **Echo state detection:** ```python import termios def _detect_echo_state(self) -> bool: if self._master_fd is None: return True try: attrs = termios.tcgetattr(self._master_fd) return bool(attrs[3] & termios.ECHO) except: return True ``` Call `_detect_echo_state()` after each resize and periodically (every 1s) during active I/O. Send `set_echo_state` to client when it changes. **Batch-friendly read:** - Change `read_output()` to use `asyncio.wait_for(select, timeout)` instead of blocking `select.select` with 0.1s timeout - Return immediately when data is available, sleep briefly when not ### Terminal Endpoint Changes - Accept `ping` messages, respond with `pong` - On session end (process exit), send `session_ended` before closing with code 1000 - Distinguish between container exit (friendly) and error (unexpected) ## Data Flow: Typing with Local Echo ``` User presses 'a' │ ▼ ┌─────────────────┐ │ onData handler │──► xterm.write('a') [instant feedback] │ │──► pendingEcho.push('a') │ │──► ws.send(binary 'a') └─────────────────┘ │ ▼ (network) ┌─────────────────┐ │ TerminalSession │──► os.write(master_fd, b'a') │ │──► docker exec PTY echoes 'a' back │ │──► os.read(master_fd) → b'a' └─────────────────┘ │ ▼ (WebSocket) ┌─────────────────┐ │ onMessage │──► data = b'a' │ (binary frame) │──► IF data[0] == pendingEcho[0]: │ │ pendingEcho.shift() // dedup │ │ ELSE: │ │ xterm.write(data) └─────────────────┘ ``` ## Data Flow: Reconnection ``` WebSocket closes (code 1006) │ ▼ ┌─────────────────┐ │ ConnectionState │──► status = "reconnecting" │ │──► attempt = 1 │ │──► scrollback = xterm.serialize() │ │──► sessionStorage.setItem(key, scrollback) │ │──► schedule reconnect in 1s └─────────────────┘ │ ▼ (1s later) ┌─────────────────┐ │ Reconnect │──► new WebSocket(url) │ │──► onopen: send scrollback from storage │ │──► xterm.write(restored + divider) │ │──► status = "connected" └─────────────────┘ ``` ## Component Responsibilities | Component | Responsibilities | |-----------|-----------------| | `TerminalPage` | Routing, layout, back button | | `TerminalComponent` | xterm.js lifecycle, addons, theme, status bar UI | | `useTerminalConnection` | WebSocket, heartbeat, reconnect, local echo, resize | | `terminal-protocol` | Encode/decode control messages, base64 helper | | `terminal.py` (API) | Auth, WebSocket accept, route control messages | | `TerminalManager` | Session lifecycle, heartbeat tracking, read/write loops | | `TerminalSession` | PTY + docker exec, echo detection, batching read | ## Tradeoffs | Decision | Option A (Chosen) | Option B | Why A | |----------|-------------------|----------|-------| | **Reconnect strategy** | Exponential backoff, max 30s | Instant reconnect with no backoff | Backoff prevents server overload during outages | | **Local echo scope** | Printable ASCII only | All characters | Control chars/escapes need server-side processing (shell state) | | **Scrollback storage** | `sessionStorage` (tab-scoped) | `localStorage` (persistent) | Privacy: terminal may contain secrets | | **Scrollback cap** | 10,000 lines | Unlimited | Memory safety; 10K lines covers typical session | | **Heartbeat interval** | 15s client → server | 5s | Balance between detection speed and server load | | **Binary vs text I/O** | Binary frames for raw data | JSON-wrapped base64 | Binary is ~33% more efficient, zero parse overhead | | **Resize trigger** | ResizeObserver on container | window.resize | Container-level is more accurate for flex layouts | | **Echo detection** | Server inspects PTY termios | Client guesses from input | Server is authoritative; client cannot know shell state | | **New docker exec on reconnect** | Accept limitation | Implement persistent session | PTY resumption across connections is extremely complex; scrollback continuity is the pragmatic fix | ## Quality Gates - `cd apps/web && npm run typecheck` — TypeScript compiles - `cd apps/web && npm run lint` — ESLint passes - `cd apps/web && npm test` — Vitest passes (new tests for protocol + hook) - `make test` — Backend pytest passes - Manual test: disconnect/reconnect, type latency, resize, container exit