feat: high-performance web terminal with asyncio-native I/O
Complete rewrite of the terminal pipeline for VS Code Server-level responsiveness. Key improvements: Backend: - Replace blocking select.select(0.1) with asyncio.add_reader() for event-driven PTY reading (eliminates ~110ms polling latency) - Add output batching (2ms window) to reduce WebSocket frame overhead - Add flow control: client acks processed bytes, server pauses PTY reads at 64KB threshold, resumes at 32KB - Add 5s ack timeout fallback to prevent stuck sessions Frontend: - Switch WebSocket to binary mode (binaryType = 'arraybuffer') - Eliminate Blob -> arrayBuffer async conversion overhead - Add flow control ack messages (every 4096 bytes or 100ms) - Add xterm-addon-webgl with graceful DOM fallback - Add performance tuning (scrollback=10000, fastScrollSensitivity) SDD artifacts: - openspec/explorations/terminal-responsiveness.md - openspec/proposals/terminal-responsiveness.md - openspec/specs/terminal-responsiveness.md - openspec/designs/terminal-responsiveness.md - openspec/tasks/terminal-responsiveness.md Quality gates: pytest (19 passed, 1 skipped), tsc --noEmit clean
This commit is contained in:
Alex Blank
@@ -0,0 +1,227 @@
|
||||
# Design: High-Performance Web Terminal
|
||||
|
||||
## Component Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Browser │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
|
||||
│ │ Terminal │ │ WebSocket │ │ xterm.js │ │
|
||||
│ │ Component │──│ Client │──│ + WebGL addon │ │
|
||||
│ │ │ │ (binary) │ │ + DOM fallback │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ Flow control ack │ │
|
||||
│ └──────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│ WebSocket
|
||||
│ (binary frames)
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ API Container │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
|
||||
│ │ Terminal │ │ WebSocket │ │ TerminalSession │ │
|
||||
│ │ Manager │──│ Endpoint │──│ (new) │ │
|
||||
│ │ (lifecycle) │ │ (router) │ │ - asyncio fd reader │ │
|
||||
│ └─────────────┘ └─────────────┘ │ - output batcher │ │
|
||||
│ │ - flow control │ │
|
||||
│ └─────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ docker exec -it │ │
|
||||
│ │ (subprocess) │ │
|
||||
│ └────────┬────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ Tool Container │ │
|
||||
│ │ (bash shell) │ │
|
||||
│ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## File Changes
|
||||
|
||||
### New Files
|
||||
- `apps/api/src/services/terminal_session_v2.py` — New TerminalSession implementation
|
||||
|
||||
### Modified Files
|
||||
- `apps/api/src/services/terminal_session.py` — Delete (or keep as legacy, user said no legacy needed)
|
||||
- `apps/api/src/services/terminal_manager.py` — Update to use new TerminalSession
|
||||
- `apps/api/src/api/terminal.py` — Update WebSocket handler for binary frames + flow control
|
||||
- `apps/web/src/components/terminal.tsx` — Binary mode, WebGL, flow control ack
|
||||
- `apps/web/package.json` — Add `xterm-addon-webgl`
|
||||
|
||||
## TerminalSession Implementation
|
||||
|
||||
```python
|
||||
class TerminalSession:
|
||||
"""High-performance terminal session with asyncio-native I/O."""
|
||||
|
||||
BUFFER_SIZE = 10 * 1024
|
||||
IDLE_TIMEOUT = 30 * 60
|
||||
BATCH_WINDOW_MS = 2
|
||||
FLOW_CONTROL_THRESHOLD = 64 * 1024
|
||||
FLOW_CONTROL_RESUME = 32 * 1024
|
||||
|
||||
def __init__(self, session_id, instance_id, container_id, ...):
|
||||
self._master_fd: int | None = None
|
||||
self._process: asyncio.subprocess.Process | None = None
|
||||
self._output_buffer: deque[bytes] = deque(maxlen=self.BUFFER_SIZE)
|
||||
self._websockets: set[WebSocket] = set()
|
||||
self._batch_buffer = bytearray()
|
||||
self._batch_timer: asyncio.TimerHandle | None = None
|
||||
self._unacknowledged_bytes = 0
|
||||
self._paused = False
|
||||
self._read_handler_set = False
|
||||
|
||||
async def start(self):
|
||||
self._master_fd, slave_fd = pty.openpty()
|
||||
self._set_terminal_size(80, 24)
|
||||
|
||||
self._process = await asyncio.create_subprocess_exec(
|
||||
"docker", "exec", "-it", "-e", "TERM=xterm-256color",
|
||||
self.container_id, "bash", "-il",
|
||||
stdin=slave_fd, stdout=slave_fd, stderr=slave_fd,
|
||||
)
|
||||
os.close(slave_fd)
|
||||
|
||||
def _start_reading(self):
|
||||
"""Register fd with asyncio event loop for event-driven reading."""
|
||||
if self._read_handler_set or self._master_fd is None:
|
||||
return
|
||||
loop = asyncio.get_event_loop()
|
||||
loop.add_reader(self._master_fd, self._on_fd_readable)
|
||||
self._read_handler_set = True
|
||||
|
||||
def _on_fd_readable(self):
|
||||
"""Callback when PTY fd has data available."""
|
||||
if self._master_fd is None or self._paused:
|
||||
return
|
||||
try:
|
||||
data = os.read(self._master_fd, 4096)
|
||||
if data:
|
||||
self._add_to_buffer(data)
|
||||
self._queue_output(data)
|
||||
self.last_activity = time.time()
|
||||
except (OSError, IOError):
|
||||
pass
|
||||
|
||||
def _queue_output(self, data: bytes):
|
||||
"""Add to batch buffer, schedule flush."""
|
||||
self._batch_buffer.extend(data)
|
||||
self._unacknowledged_bytes += len(data)
|
||||
|
||||
if self._unacknowledged_bytes > self.FLOW_CONTROL_THRESHOLD:
|
||||
self._pause_output()
|
||||
|
||||
if self._batch_timer is None:
|
||||
loop = asyncio.get_event_loop()
|
||||
self._batch_timer = loop.call_later(
|
||||
self.BATCH_WINDOW_MS / 1000, self._flush_batch
|
||||
)
|
||||
|
||||
def _flush_batch(self):
|
||||
"""Flush batched output to all WebSockets."""
|
||||
self._batch_timer = None
|
||||
if not self._batch_buffer:
|
||||
return
|
||||
|
||||
payload = bytes(self._batch_buffer)
|
||||
self._batch_buffer.clear()
|
||||
|
||||
dead = set()
|
||||
for ws in self._websockets:
|
||||
try:
|
||||
asyncio.create_task(ws.send_bytes(payload))
|
||||
except Exception:
|
||||
dead.add(ws)
|
||||
|
||||
self._websockets -= dead
|
||||
|
||||
def acknowledge_data(self, char_count: int):
|
||||
"""Client acknowledges processed bytes."""
|
||||
self._unacknowledged_bytes = max(0, self._unacknowledged_bytes - char_count)
|
||||
if self._paused and self._unacknowledged_bytes < self.FLOW_CONTROL_RESUME:
|
||||
self._resume_output()
|
||||
|
||||
def _pause_output(self):
|
||||
"""Pause reading from PTY."""
|
||||
if self._read_handler_set and self._master_fd is not None:
|
||||
loop = asyncio.get_event_loop()
|
||||
loop.remove_reader(self._master_fd)
|
||||
self._read_handler_set = False
|
||||
self._paused = True
|
||||
|
||||
def _resume_output(self):
|
||||
"""Resume reading from PTY."""
|
||||
self._paused = False
|
||||
self._start_reading()
|
||||
```
|
||||
|
||||
## Frontend Changes
|
||||
|
||||
### WebSocket Binary Mode
|
||||
```typescript
|
||||
const ws = new WebSocket(wsUrl);
|
||||
ws.binaryType = "arraybuffer"; // Receive ArrayBuffer directly
|
||||
|
||||
ws.onmessage = (event) => {
|
||||
if (event.data instanceof ArrayBuffer) {
|
||||
const data = new Uint8Array(event.data);
|
||||
termRef.current?.write(data);
|
||||
|
||||
// Flow control: acknowledge processed bytes
|
||||
ackAccumulator += data.length;
|
||||
if (ackAccumulator >= 4096) {
|
||||
ws.send(JSON.stringify({ type: "ack", chars: ackAccumulator }));
|
||||
ackAccumulator = 0;
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### WebGL Renderer
|
||||
```typescript
|
||||
import { WebglAddon } from "xterm-addon-webgl";
|
||||
|
||||
const webglAddon = new WebglAddon();
|
||||
try {
|
||||
term.loadAddon(webglAddon);
|
||||
} catch (e) {
|
||||
console.warn("WebGL failed, using DOM renderer", e);
|
||||
}
|
||||
```
|
||||
|
||||
## WebSocket Protocol
|
||||
|
||||
### Message Types
|
||||
|
||||
**Client → Server:**
|
||||
- `{"type": "input", "data": "base64_encoded"}` — Keystrokes
|
||||
- `{"type": "resize", "cols": 80, "rows": 24}` — Resize
|
||||
- `{"type": "ack", "chars": 4096}` — Flow control acknowledgment
|
||||
- `{"type": "reset"}` — Reset session
|
||||
|
||||
**Server → Client:**
|
||||
- Binary frame: raw terminal output bytes
|
||||
- `{"type": "status", "status": "connected"}` — Connection status
|
||||
- `{"type": "ping"}` — Heartbeat (server → client)
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
- `test_terminal_session_v2.py` — Test batching, flow control, resize, reset
|
||||
- `test_terminal_manager.py` — Test session lifecycle with new session class
|
||||
|
||||
### Integration Tests
|
||||
- `test_terminal_websocket.py` — Full WebSocket round-trip
|
||||
|
||||
### Performance Tests
|
||||
- `benchmark_terminal_latency.py` — Measure input/output latency
|
||||
- `benchmark_terminal_throughput.py` — Measure max throughput
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Since this is a full rewrite with no legacy support:
|
||||
- Keep a backup branch of the old terminal code
|
||||
- Feature flag in frontend: `?terminal=v2` to test before full rollout
|
||||
- Monitor error rates after deployment
|
||||
Reference in New Issue
Block a user