alex
6c8cfe9157
Implements a resilient, responsive web terminal that survives network blips, provides instant typing feedback, and restores scrollback on reconnect. Backend changes: - Add heartbeat tracking (15s ping interval, 60s idle timeout) - Add message batching (16ms flush window) for efficient I/O - Add termios echo detection and set_echo_state control messages - Add graceful session_ended notification before close - Add ping/pong protocol support Frontend changes: - Rewrite TerminalComponent with status bar, connection indicator, session-ended overlay, reconnect banner, and ResizeObserver - Add useTerminalConnection hook with: - Exponential backoff auto-reconnect (1s → 30s max, 10 attempts) - Heartbeat/ping-pong with latency tracking - Local echo for printable ASCII with server deduplication - Resize debounce (200ms) + throttle (500ms) - Scrollback serialization via xterm-addon-serialize - Ctrl+Shift+R manual reconnect shortcut - Add WebSocket protocol types and encoding utilities - Add xterm-addon-serialize dependency Tests: - 16 backend unit tests (TerminalSession + TerminalManager) - 13 frontend hook tests (connection lifecycle, reconnect, resize, scrollback, callbacks) Quality gates: - Frontend typecheck: clean - Frontend lint: clean - Frontend tests: 48 passed - Backend unit tests: 101 passed - Backend ruff: clean SDD artifacts: openspec/changes/responsive-terminal/
372 lines
16 KiB
Markdown
372 lines
16 KiB
Markdown
# Design: Responsive Web Terminal
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────┐
|
|
│ BROWSER │
|
|
│ ┌──────────────┐ ┌─────────────────┐ ┌──────────────────────────┐ │
|
|
│ │ TerminalPage │ │ TerminalComponent │ │ TerminalConnection │ │
|
|
│ │ (router) │◄──│ (xterm.js + UI) │◄──│ (WS + heartbeat + echo) │ │
|
|
│ └──────────────┘ └─────────────────┘ └──────────────────────────┘ │
|
|
│ │ │ │
|
|
│ ┌─────┴─────┐ ┌──────┴──────┐ │
|
|
│ │ xterm.js │ │ sessionStorage│ │
|
|
│ │ + addons │ │ (scrollback) │ │
|
|
│ └───────────┘ └───────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────┘
|
|
│ WebSocket
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────┐
|
|
│ FASTAPI │
|
|
│ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ │
|
|
│ │ terminal.py │ │ TerminalManager │ │ TerminalSession │ │
|
|
│ │ (WS endpoint) │◄──│ (session mgmt) │◄──│ (PTY + docker exec) │ │
|
|
│ └──────────────────┘ └──────────────────┘ └─────────────────────┘ │
|
|
│ │ │
|
|
│ ┌────┴────┐ │
|
|
│ │ docker │ │
|
|
│ │ exec │ │
|
|
│ └─────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Connection State Machine
|
|
|
|
### Client State Machine
|
|
|
|
```
|
|
┌─────────────┐
|
|
│ IDLE │
|
|
└──────┬──────┘
|
|
│ mount
|
|
▼
|
|
┌─────────────┐
|
|
│ CONNECTING │◄────────────────────────┐
|
|
└──────┬──────┘ │
|
|
│ onopen │
|
|
▼ │
|
|
┌─────────────────────────┐ │
|
|
│ CONNECTED │ │
|
|
│ (heartbeat active) │ │
|
|
└──────┬──────────┬───────┘ │
|
|
│ │ │
|
|
onclose/ │ │ ping timeout │
|
|
onerror │ │ │
|
|
▼ ▼ │
|
|
┌─────────────────────────┐ │
|
|
│ RECONNECTING │───────────────────┘
|
|
│ (backoff: 1→2→4→8→30s) │ onopen (success)
|
|
└──────┬──────────────────┘
|
|
│ max retries (10)
|
|
▼
|
|
┌─────────────────────────┐
|
|
│ DISCONNECTED │
|
|
│ (manual reconnect │
|
|
│ or navigate away) │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
### Server State Machine (per session)
|
|
|
|
```
|
|
┌─────────────┐
|
|
│ PENDING │
|
|
└──────┬──────┘
|
|
│ ws.accept()
|
|
▼
|
|
┌─────────────┐
|
|
┌────►│ ACTIVE │◄────┐
|
|
│ │ (I/O loops │ │
|
|
│ │ + heartbeat) │
|
|
│ └──────┬──────┘ │
|
|
│ │ │
|
|
│ ws close│ new ws │
|
|
│ ▼ │
|
|
│ ┌─────────────┐ │
|
|
└─────┤ CLOSED ├──────┘
|
|
│ (cleanup) │
|
|
└─────────────┘
|
|
```
|
|
|
|
## Protocol Specification
|
|
|
|
### Message Types
|
|
|
|
All control messages are JSON text frames. Raw terminal I/O uses binary frames.
|
|
|
|
#### Client → Server
|
|
|
|
| Type | Payload | When |
|
|
|------|---------|------|
|
|
| `ping` | `{ id: number }` | Every 15s of inactivity |
|
|
| `pong` | `{ id: number }` | Response to server ping |
|
|
| `resize` | `{ cols: number, rows: number }` | Terminal size changes (debounced) |
|
|
| `input` | `{ data: string }` | User keystrokes (base64-encoded) |
|
|
|
|
#### Server → Client
|
|
|
|
| Type | Payload | When |
|
|
|------|---------|------|
|
|
| `pong` | `{ id: number }` | Response to client ping |
|
|
| `status` | `{ status: "connected" \| "reconnected" }` | After auth + session ready |
|
|
| `set_echo_state` | `{ enabled: boolean }` | When PTY echo flag changes |
|
|
| `session_ended` | `{ reason: string }` | When container process exits |
|
|
|
|
### Binary Frame Convention
|
|
|
|
- **Client → Server:** Raw UTF-8 bytes of user input. No wrapping.
|
|
- **Server → Client:** Raw bytes from PTY master read. No wrapping.
|
|
|
|
This avoids the current Blob→ArrayBuffer async conversion and JSON parsing overhead for the hot path.
|
|
|
|
## Frontend Design
|
|
|
|
### New Files
|
|
|
|
```
|
|
apps/web/src/
|
|
├── components/
|
|
│ └── terminal.tsx (rewrite: state machine + reconnect)
|
|
├── hooks/
|
|
│ └── use-terminal-connection.ts (NEW: WS lifecycle, heartbeat, reconnect)
|
|
├── utils/
|
|
│ └── terminal-protocol.ts (NEW: message encoding/decoding)
|
|
└── types/
|
|
└── terminal.ts (NEW: protocol types)
|
|
```
|
|
|
|
### `useTerminalConnection` Hook
|
|
|
|
Responsibilities:
|
|
1. **WebSocket lifecycle:** Open, close, reconnect with backoff
|
|
2. **Heartbeat:** Send ping every 15s, expect pong within 5s
|
|
3. **Local echo:** Write printable chars to xterm immediately, deduplicate server echo
|
|
4. **Resize:** Debounce resize events, send JSON control message
|
|
5. **Scrollback:** Serialize on disconnect, restore on reconnect
|
|
6. **State reporting:** Expose `status`, `latency`, `attempt` to UI
|
|
|
|
```typescript
|
|
interface TerminalConnectionState {
|
|
status: "connecting" | "connected" | "reconnecting" | "disconnected";
|
|
attempt: number;
|
|
latency: number | null; // last RTT in ms
|
|
error: string | null;
|
|
}
|
|
|
|
interface TerminalConnection {
|
|
state: TerminalConnectionState;
|
|
sendInput: (data: string) => void;
|
|
sendResize: (cols: number, rows: number) => void;
|
|
reconnect: () => void; // manual, bypasses backoff
|
|
onData: (callback: (data: Uint8Array) => void) => void;
|
|
onControl: (callback: (msg: ServerControlMessage) => void) => void;
|
|
}
|
|
```
|
|
|
|
### Local Echo Algorithm
|
|
|
|
```
|
|
1. User types character c
|
|
2. IF c is printable ASCII AND echo is enabled:
|
|
a. Write c to xterm immediately
|
|
b. Add c to "pending echo" buffer
|
|
c. Send c to server via WebSocket
|
|
3. ELSE (control char, arrow, escape sequence):
|
|
a. Send c to server only
|
|
b. Do NOT write to xterm
|
|
4. When server sends data:
|
|
a. For each char in server data:
|
|
- IF char matches head of "pending echo" buffer:
|
|
→ Pop from buffer (deduplication)
|
|
- ELSE:
|
|
→ Write char to xterm
|
|
b. If "pending echo" buffer grows > 100 chars (stale):
|
|
→ Flush buffer to xterm (server echo was lost)
|
|
```
|
|
|
|
### Scrollback Serialization
|
|
|
|
```
|
|
ON disconnect:
|
|
1. buffer = xterm.serialize({ scrollback: 10000 })
|
|
2. sessionStorage.setItem(`hq-terminal-${instanceId}`, buffer)
|
|
|
|
ON reconnect:
|
|
1. buffer = sessionStorage.getItem(`hq-terminal-${instanceId}`)
|
|
2. IF buffer:
|
|
xterm.write(buffer)
|
|
xterm.write("\r\n\x1b[90m--- Reconnected ---\x1b[0m\r\n")
|
|
3. sessionStorage.removeItem(`hq-terminal-${instanceId}`)
|
|
```
|
|
|
|
### Resize Debouncing
|
|
|
|
Use `ResizeObserver` on the terminal container instead of `window.resize`:
|
|
|
|
```typescript
|
|
const resizeObserver = new ResizeObserver(
|
|
debounce((entries) => {
|
|
fitAddon.fit();
|
|
sendResize(term.cols, term.rows);
|
|
}, 200)
|
|
);
|
|
```
|
|
|
|
Rate limit: max 1 resize message per 500ms.
|
|
|
|
## Backend Design
|
|
|
|
### Modified Files
|
|
|
|
```
|
|
apps/api/src/
|
|
├── api/terminal.py (modify: ping/pong, session_ended)
|
|
├── services/terminal_manager.py (rewrite: heartbeat tracking, batching)
|
|
└── services/terminal_session.py (modify: batching read, echo detection)
|
|
```
|
|
|
|
### TerminalManager Changes
|
|
|
|
**Heartbeat tracking:**
|
|
- Track `last_ping_at` per session
|
|
- Background task: if `last_ping_at` is older than 60s, close the WebSocket
|
|
|
|
**Message batching in read_loop:**
|
|
```python
|
|
async def _read_loop(self, session, websocket):
|
|
buffer = bytearray()
|
|
last_flush = time.monotonic()
|
|
|
|
while session.is_alive() and not session._closed:
|
|
data = await session.read_output()
|
|
if data:
|
|
buffer.extend(data)
|
|
|
|
now = time.monotonic()
|
|
if buffer and (now - last_flush >= 0.016 or not data):
|
|
await websocket.send_bytes(bytes(buffer))
|
|
buffer.clear()
|
|
last_flush = now
|
|
elif not data:
|
|
await asyncio.sleep(0.001)
|
|
```
|
|
|
|
**Reconnect support:**
|
|
- When a new WebSocket connects for the same instance, terminate the old session and spawn a new one
|
|
- This is the docker exec limitation — we cannot resume a PTY, only replace it
|
|
|
|
### TerminalSession Changes
|
|
|
|
**Echo state detection:**
|
|
```python
|
|
import termios
|
|
|
|
def _detect_echo_state(self) -> bool:
|
|
if self._master_fd is None:
|
|
return True
|
|
try:
|
|
attrs = termios.tcgetattr(self._master_fd)
|
|
return bool(attrs[3] & termios.ECHO)
|
|
except:
|
|
return True
|
|
```
|
|
|
|
Call `_detect_echo_state()` after each resize and periodically (every 1s) during active I/O. Send `set_echo_state` to client when it changes.
|
|
|
|
**Batch-friendly read:**
|
|
- Change `read_output()` to use `asyncio.wait_for(select, timeout)` instead of blocking `select.select` with 0.1s timeout
|
|
- Return immediately when data is available, sleep briefly when not
|
|
|
|
### Terminal Endpoint Changes
|
|
|
|
- Accept `ping` messages, respond with `pong`
|
|
- On session end (process exit), send `session_ended` before closing with code 1000
|
|
- Distinguish between container exit (friendly) and error (unexpected)
|
|
|
|
## Data Flow: Typing with Local Echo
|
|
|
|
```
|
|
User presses 'a'
|
|
│
|
|
▼
|
|
┌─────────────────┐
|
|
│ onData handler │──► xterm.write('a') [instant feedback]
|
|
│ │──► pendingEcho.push('a')
|
|
│ │──► ws.send(binary 'a')
|
|
└─────────────────┘
|
|
│
|
|
▼ (network)
|
|
┌─────────────────┐
|
|
│ TerminalSession │──► os.write(master_fd, b'a')
|
|
│ │──► docker exec PTY echoes 'a' back
|
|
│ │──► os.read(master_fd) → b'a'
|
|
└─────────────────┘
|
|
│
|
|
▼ (WebSocket)
|
|
┌─────────────────┐
|
|
│ onMessage │──► data = b'a'
|
|
│ (binary frame) │──► IF data[0] == pendingEcho[0]:
|
|
│ │ pendingEcho.shift() // dedup
|
|
│ │ ELSE:
|
|
│ │ xterm.write(data)
|
|
└─────────────────┘
|
|
```
|
|
|
|
## Data Flow: Reconnection
|
|
|
|
```
|
|
WebSocket closes (code 1006)
|
|
│
|
|
▼
|
|
┌─────────────────┐
|
|
│ ConnectionState │──► status = "reconnecting"
|
|
│ │──► attempt = 1
|
|
│ │──► scrollback = xterm.serialize()
|
|
│ │──► sessionStorage.setItem(key, scrollback)
|
|
│ │──► schedule reconnect in 1s
|
|
└─────────────────┘
|
|
│
|
|
▼ (1s later)
|
|
┌─────────────────┐
|
|
│ Reconnect │──► new WebSocket(url)
|
|
│ │──► onopen: send scrollback from storage
|
|
│ │──► xterm.write(restored + divider)
|
|
│ │──► status = "connected"
|
|
└─────────────────┘
|
|
```
|
|
|
|
## Component Responsibilities
|
|
|
|
| Component | Responsibilities |
|
|
|-----------|-----------------|
|
|
| `TerminalPage` | Routing, layout, back button |
|
|
| `TerminalComponent` | xterm.js lifecycle, addons, theme, status bar UI |
|
|
| `useTerminalConnection` | WebSocket, heartbeat, reconnect, local echo, resize |
|
|
| `terminal-protocol` | Encode/decode control messages, base64 helper |
|
|
| `terminal.py` (API) | Auth, WebSocket accept, route control messages |
|
|
| `TerminalManager` | Session lifecycle, heartbeat tracking, read/write loops |
|
|
| `TerminalSession` | PTY + docker exec, echo detection, batching read |
|
|
|
|
## Tradeoffs
|
|
|
|
| Decision | Option A (Chosen) | Option B | Why A |
|
|
|----------|-------------------|----------|-------|
|
|
| **Reconnect strategy** | Exponential backoff, max 30s | Instant reconnect with no backoff | Backoff prevents server overload during outages |
|
|
| **Local echo scope** | Printable ASCII only | All characters | Control chars/escapes need server-side processing (shell state) |
|
|
| **Scrollback storage** | `sessionStorage` (tab-scoped) | `localStorage` (persistent) | Privacy: terminal may contain secrets |
|
|
| **Scrollback cap** | 10,000 lines | Unlimited | Memory safety; 10K lines covers typical session |
|
|
| **Heartbeat interval** | 15s client → server | 5s | Balance between detection speed and server load |
|
|
| **Binary vs text I/O** | Binary frames for raw data | JSON-wrapped base64 | Binary is ~33% more efficient, zero parse overhead |
|
|
| **Resize trigger** | ResizeObserver on container | window.resize | Container-level is more accurate for flex layouts |
|
|
| **Echo detection** | Server inspects PTY termios | Client guesses from input | Server is authoritative; client cannot know shell state |
|
|
| **New docker exec on reconnect** | Accept limitation | Implement persistent session | PTY resumption across connections is extremely complex; scrollback continuity is the pragmatic fix |
|
|
|
|
## Quality Gates
|
|
|
|
- `cd apps/web && npm run typecheck` — TypeScript compiles
|
|
- `cd apps/web && npm run lint` — ESLint passes
|
|
- `cd apps/web && npm test` — Vitest passes (new tests for protocol + hook)
|
|
- `make test` — Backend pytest passes
|
|
- Manual test: disconnect/reconnect, type latency, resize, container exit
|