docs: add responsive terminal documentation

- Add docs/features/terminal.md with user guide, connection states, keyboard shortcuts, protocol details, and troubleshooting - Update docs/architecture/frontend.md with terminal component stack, connection hook behavior, and data flow diagrams - Update docs/architecture/backend.md with terminal system architecture, protocol reference, message batching, and reconnect behavior - Update docs/README.md to include terminal in feature list
2026-05-27 21:46:57 +02:00
parent 6c8cfe9157
commit a01e6252f5
4 changed files with 351 additions and 12 deletions
@@ -26,6 +26,7 @@ User guides for each feature:
 - [Repositories](features/repositories.md) - Git repository management
 - [Workspace](features/workspace.md) - Repository workspace
 - [Git History](features/git-history.md) - History visualization
+- [Web Terminal](features/terminal.md) - Interactive terminal for tool instances
 - [Authentication](features/auth.md) - Login and user management
 - [Settings](features/settings.md) - User preferences
 - [Tool Types](features/tool-types.md) - Development tool management
@@ -13,16 +13,16 @@ The Headquarter backend is built with **FastAPI** and follows a layered architec
 │  Middleware: CORS → Request Logging → Exception Logging      │
 ├─────────────────────────────────────────────────────────────┤
 │  API Layer (src/api/)                                        │
-│  ┌─────────┐ ┌──────────┐ ┌────────┐ ┌──────────┐          │
-│  │  Auth   │ │ Projects │ │ Users  │ │   Git    │          │
-│  │ Routes  │ │ Routes   │ │ Routes │ │ Repos    │          │
-│  └────┬────┘ └────┬─────┘ └───┬────┘ └────┬─────┘          │
-├───────┼───────────┼───────────┼───────────┼─────────────────┤
-│       │           │           │           │                  │
-│  Auth │  Project  │   User    │   Git     │                  │
-│ Layer │  Service  │  Service  │ Service   │                  │
-│       │           │           │           │                  │
-├───────┴───────────┴───────────┴───────────┴─────────────────┤
+│  ┌─────────┐ ┌─────────┐ ┌────────┐ ┌──────────┐           │
+│  │  Auth   │ │Terminal │ │Projects│ │   Git    │           │
+│  │ Routes  │ │   WS    │ │ Routes │ │ Repos    │           │
+│  └────┬────┘ └────┬────┘ └───┬────┘ └────┬─────┘           │
+├───────┼───────────┼──────────┼───────────┼──────────────────┤
+│       │           │          │           │                  │
+│  Auth │  Terminal │ Project  │   Git     │                  │
+│ Layer │  Manager  │ Service  │ Service   │                  │
+│       │  + Session│          │           │                  │
+├───────┴───────────┴──────────┴───────────┴──────────────────┤
 │  Data Layer                                                  │
 │  ┌──────────┐  ┌──────────┐  ┌──────────┐                   │
 │  │  Models  │  │ Database │  │   Config │                   │
@@ -37,6 +37,7 @@ The Headquarter backend is built with **FastAPI** and follows a layered architec
 src/
 ├── api/                    # API Routes
 │   ├── auth.py            # Authentication endpoints
+│   ├── terminal.py        # WebSocket terminal endpoint
 │   ├── projects.py        # Project endpoints
 │   ├── git_repositories.py # Repository endpoints
 │   ├── users.py           # User endpoints
@@ -55,6 +56,11 @@ src/
 │   ├── tool_type.py       # Tool type model
 │   ├── ssh_key.py         # SSH key model
 │   └── user_config.py     # User config model
+├── services/              # Business Logic
+│   ├── terminal_manager.py # Terminal session manager
+│   ├── terminal_session.py # PTY + docker exec session
+│   ├── docker.py          # Docker operations
+│   └── profile_resolver.py # Profile resolution
 ├── utils/                  # Utilities
 │   ├── git_url_parser.py  # URL parsing
 │   ├── git_files.py       # Git file operations
@@ -64,6 +70,65 @@ src/
 └── main.py                # Application entry point
 ```

+## Terminal System
+
+The terminal system provides interactive shell access to running tool instances via WebSocket.
+
+### Architecture
+
+```
+Client (WebSocket)
+  ↕
+terminal.py (FastAPI WS endpoint)
+  ├─ Auth validation (session cookie)
+  ├─ Instance ownership check
+  ├─ Session lifecycle (create / monitor / cleanup)
+  └─ Echo state detection (termios)
+  ↕
+TerminalManager
+  ├─ create_session() → spawns TerminalSession
+  ├─ _read_loop() → batches PTY output → WebSocket
+  ├─ _write_loop() → WebSocket input → PTY
+  └─ _heartbeat_loop() → closes idle connections (60s)
+  ↕
+TerminalSession
+  ├─ start() → pty.openpty() + docker exec
+  ├─ read_output() → select.select() + os.read()
+  ├─ write_input() → os.write() to PTY master
+  ├─ resize() → TIOCSWINSZ ioctl
+  └─ check_echo_state() → termios.ECHO flag
+```
+
+### Protocol
+
+**Binary frames**: Raw terminal I/O (hot path)
+**Text (JSON) frames**: Control messages
+
+**Control messages:**
+
+| Direction | Type | Purpose |
+|-----------|------|---------|
+| Client → Server | `ping` | Heartbeat (every 15s idle) |
+| Server → Client | `pong` | Heartbeat response |
+| Client → Server | `resize` | Terminal dimensions changed |
+| Server → Client | `set_echo_state` | Enable/disable local echo |
+| Server → Client | `session_ended` | Container process exited |
+
+### Message Batching
+
+The read loop batches small PTY reads into single WebSocket frames:
+- Buffer accumulates data for up to 16ms
+- Flushed immediately when no new data is available
+- Reduces WebSocket frame overhead for rapid output
+
+### Reconnect Behavior
+
+The server cannot resume a `docker exec` PTY across connections. On reconnect:
+1. Old session is terminated
+2. New `docker exec` is spawned
+3. Client restores scrollback from `sessionStorage`
+4. New shell appears seamlessly to the user
+
 ## Layers

 ### 1. API Layer (`src/api/`)
@@ -26,22 +26,26 @@ apps/web/src/
 │   ├── ssh_keys.ts      # SSH key API
 │   ├── tool_types.ts    # Tool type API
 │   ├── users.ts         # User API
+│   ├── sessions.ts      # Tool instance sessions API
 │   └── settings.ts      # Settings API
 ├── components/           # Reusable components
 │   ├── app-shell.tsx    # Main app layout
+│   ├── terminal.tsx     # xterm.js terminal component
 │   ├── protected-route.tsx  # Auth guard
 │   └── [more...]
 ├── context/             # React contexts
 │   └── auth.tsx         # Auth state management
 ├── hooks/               # Custom hooks
 │   ├── use-auth.ts      # Auth hook
-│   └── use-theme.ts     # Theme hook
+│   ├── use-theme.ts     # Theme hook
+│   └── use-terminal-connection.ts  # Terminal WebSocket lifecycle
 ├── pages/               # Page components (routes)
 │   ├── dashboard.tsx    # Dashboard
 │   ├── projects.tsx     # Project list
 │   ├── repo-workspace.tsx   # Repository workspace
 │   ├── git-history.tsx  # Git history
 │   ├── git-repositories.tsx # Repository management
+│   ├── terminal.tsx     # Web terminal
 │   ├── profile.tsx      # User profile
 │   ├── settings.tsx     # User settings
 │   ├── tool-types.tsx   # Tool types
@@ -164,6 +168,7 @@ interface AuthState {
 <Route path="/projects/:projectId" element={<RepoWorkspace />} />
 <Route path="/projects/:projectId/repositories" element={<GitRepositories />} />
 <Route path="/projects/:projectId/repositories/:repoId/history" element={<GitHistory />} />
+<Route path="/terminal/:instanceId" element={<TerminalPage />} />
 <Route path="/profile" element={<ProfilePage />} />
 <Route path="/settings" element={<SettingsPage />} />
 <Route path="/ssh-keys" element={<SSHKeysPage />} />
@@ -269,12 +274,64 @@ test('renders file list', () => {
 4. **Caching**: Browser caches API responses (ETags)
 5. **Optimistic UI**: Immediate feedback before API response

+## Terminal Architecture
+
+The web terminal is the most complex component in the frontend. It bridges a browser-based terminal emulator with a server-side PTY session.
+
+### Component Stack
+
+```
+TerminalPage (route)
+└── TerminalComponent
+    ├── Status bar (connection state, latency, actions)
+    ├── Session-ended overlay (reconnect / go back)
+    ├── Reconnect banner (spinner + countdown)
+    └── xterm.js (terminal emulator)
+        ├── FitAddon (auto-resize to container)
+        ├── SerializeAddon (scrollback serialization)
+        └── WebLinksAddon (clickable URLs)
+```
+
+### Connection Hook
+
+`useTerminalConnection` manages the full WebSocket lifecycle:
+
+```
+CONNECTING
+  → onopen → CONNECTED → heartbeat every 15s
+  → onclose (unexpected) → RECONNECTING
+    → backoff: 1s → 2s → 4s → 8s → 16s → 30s max
+    → up to 10 attempts
+    → onopen → restore scrollback → CONNECTED
+  → onclose (expected) → DISCONNECTED
+```
+
+**Key behaviors:**
+- **Local echo**: Printable ASCII chars appear instantly; server echo is deduplicated
+- **Resize**: Debounced 200ms, throttled to 1 message per 500ms
+- **Scrollback**: Serialized to `sessionStorage` on disconnect, restored on reconnect
+- **Keyboard**: `Ctrl+Shift+R` triggers manual reconnect
+
+### Data Flow
+
+```
+User types 'a'
+  → xterm onData event
+  → useTerminalConnection.sendInput('a')
+  → local echo writes 'a' to xterm immediately
+  → WebSocket sends 'a' to server
+  → server PTY echoes 'a' back
+  → client receives 'a' via binary frame
+  → deduplicates against pending echo buffer
+  → (no-op if matched, or writes remaining chars)
+```
+
 ## Future Improvements

 - [ ] Add React Query for server state management
 - [ ] Implement virtual scrolling for large file trees
 - [ ] Add service worker for offline support
- [ ] Implement real-time updates (WebSocket)
+- [x] Implement real-time updates (WebSocket) — Terminal done
 - [ ] Add error boundary components

 ## Development Workflow
@@ -0,0 +1,216 @@
+# Web Terminal
+
+## Overview
+
+The web terminal provides an interactive shell session inside running tool instances directly from your browser. It uses xterm.js to render a full terminal emulator connected via WebSocket to a PTY-backed docker exec session.
+
+The terminal is designed to feel as close to a local terminal as possible, with features for network resilience, low-latency typing, and session continuity.
+
+## How to Use
+
+### Opening a Terminal
+
+1. Navigate to a **project** and select a **repository**
+2. Go to the repository **workspace**
+3. Start or select a **tool instance** that supports the terminal interface
+4. Click the **"Open Terminal"** button
+
+The terminal opens in full-page mode with a status bar at the top.
+
+### Terminal Layout
+
+```
+┌─────────────────────────────────────────────┐
+│  ● Connected                [Reconnect] [×] │
+├─────────────────────────────────────────────┤
+│                                             │
+│  user@container:~$ ls -la                   │
+│  total 128                                  │
+│  drwxr-xr-x  5 user user 4096 May 27 10:00 │
+│  ...                                        │
+│                                             │
+└─────────────────────────────────────────────┘
+```
+
+**Status bar (top):**
+- **Connection dot** — color indicates connection health
+- **Status text** — shows current state and latency
+- **Reconnect button** — appears when disconnected
+- **Close button** — returns to the previous page
+
+### Connection States
+
+| Indicator | Meaning | Action |
+|-----------|---------|--------|
+| 🟡 **Yellow dot** + "Connecting..." | Opening WebSocket | Wait or check network |
+| 🟢 **Green dot** + "Connected" | Healthy connection (<100ms) | Ready to use |
+| 🟡 **Yellow dot** + "Slow (150ms)" | Elevated latency | Connection usable but laggy |
+| 🟡 **Yellow dot** + "Reconnecting (2)" | Connection lost, retrying | Wait for auto-reconnect |
+| ⚪ **Gray dot** + "Disconnected" | Max retries exceeded | Click Reconnect or refresh |
+
+**Hover the status dot** to see the current round-trip latency in milliseconds.
+
+### Typing
+
+Type normally as you would in a local terminal. The terminal supports:
+
+- **Printable characters** appear instantly (local echo)
+- **Special keys** (Tab, Enter, Ctrl+C, arrow keys) are sent to the server
+- **Password prompts** automatically suppress local echo
+- **Unicode** input and output
+
+### Reconnecting
+
+The terminal **automatically reconnects** if the WebSocket drops:
+
+- Brief disconnects (WiFi hiccups, proxy timeouts) are recovered within 1–5 seconds
+- Up to **10 reconnection attempts** with exponential backoff
+- **Scrollback is preserved** across reconnects
+- A visual divider (`--- Reconnected ---`) separates old and new output
+
+**Manual reconnect:**
+- Click the **Reconnect** button in the status bar
+- Or press **Ctrl+Shift+R** anywhere in the terminal page
+
+### Session Ended
+
+When the container process exits (e.g., you run `exit` or the container stops), the terminal shows an overlay:
+
+```
+┌─────────────────────────┐
+│    Session Ended        │
+│ The container process   │
+│ has exited.             │
+│                         │
+│ [Reconnect]  [Go Back]  │
+└─────────────────────────┘
+```
+
+- **Reconnect** — spawns a new shell session in the same container
+- **Go Back** — returns to the workspace page
+
+## Keyboard Shortcuts
+
+| Shortcut | Action |
+|----------|--------|
+| `Ctrl+Shift+R` | Force reconnect (bypasses backoff) |
+| Standard terminal shortcuts | `Ctrl+C`, `Ctrl+D`, `Ctrl+L`, Tab completion, etc. |
+
+## Technical Details
+
+### WebSocket Protocol
+
+The terminal communicates over a binary WebSocket with mixed JSON control messages.
+
+**Connection:**
+```
+ws://api.example.com/ws/tool-instances/{instance_id}/terminal
+```
+
+**Binary frames** carry raw terminal I/O. **Text (JSON) frames** carry control messages:
+
+**Client → Server:**
+- `{"type":"ping","id":n}` — heartbeat ping
+- `{"type":"resize","cols":120,"rows":40}` — terminal resize
+- Raw bytes — keystroke input
+
+**Server → Client:**
+- `{"type":"pong","id":n}` — heartbeat response
+- `{"type":"status","status":"connected"}` — session ready
+- `{"type":"set_echo_state","enabled":false}` — disable local echo
+- `{"type":"session_ended","reason":"process_exit"}` — session ended
+- Raw bytes — terminal output
+
+### Architecture
+
+```
+Browser                          Backend
+┌──────────────────────┐         ┌─────────────────────────────┐
+│ TerminalComponent    │         │ terminal.py (WS endpoint)   │
+│ ├─ xterm.js          │◄───────►│ ├─ auth + session mgmt      │
+│ ├─ FitAddon          │  WS     │ └─ echo state detection     │
+│ ├─ SerializeAddon    │         │                             │
+│ └─ useTerminalConn.  │         │ TerminalManager             │
+│    ├─ heartbeat      │         │ ├─ read_loop (batching)     │
+│    ├─ reconnect      │         │ ├─ write_loop               │
+│    ├─ local echo     │         │ └─ heartbeat_loop           │
+│    └─ resize throttle│         │                             │
+│                      │         │ TerminalSession             │
+│ sessionStorage       │         │ ├─ PTY + docker exec        │
+│ (scrollback backup)  │         │ └─ termios echo detection   │
+└──────────────────────┘         └─────────────────────────────┘
+```
+
+### Reconnect Behavior
+
+On disconnect:
+1. The client serializes terminal scrollback to `sessionStorage`
+2. Backoff timer starts (1s, 2s, 4s, 8s, 16s, then caps at 30s)
+3. On reconnect, scrollback is restored + divider line
+4. A new `docker exec` session is spawned transparently
+
+**Note:** The underlying docker exec PTY is not resumable. Reconnect creates a new shell, but scrollback continuity makes this transparent.
+
+### Performance
+
+- **Local echo** makes printable characters appear in <1ms
+- **Message batching** on the backend reduces WebSocket frame overhead
+- **Resize debouncing** (200ms) + throttling (500ms) prevents server spam
+- **Heartbeat interval** is 15s to balance detection speed with server load
+
+## Troubleshooting
+
+### "Connecting..." stays yellow
+
+**Issue:** WebSocket cannot open
+**Check:**
+1. Is the API server running?
+2. Is the tool instance in "running" status?
+3. Check browser console for connection errors
+4. Verify the `VITE_API_BASE_URL` points to the correct API
+
+### "Reconnecting" loops forever
+
+**Issue:** Max reconnection attempts exceeded
+**Check:**
+1. Is the container still running? (`docker ps`)
+2. Did the container crash or get stopped?
+3. Check server logs for `Terminal session error`
+
+### Typing feels slow
+
+**Issue:** High latency or no local echo
+**Check:**
+1. Hover the status dot — latency >100ms is shown as "Slow"
+2. Local echo only works for printable ASCII characters
+3. Password prompts intentionally disable echo
+4. Very high latency may indicate a congested network
+
+### Terminal is blank after reconnect
+
+**Issue:** Scrollback not restored
+**Check:**
+1. `sessionStorage` may have been cleared (new browser session)
+2. The scrollback cap is 10,000 lines — very long sessions may truncate
+3. Browser privacy settings may block `sessionStorage`
+
+### "Session Ended" immediately
+
+**Issue:** Container process exits right away
+**Check:**
+1. The container's default command may have finished
+2. Check the tool type's Docker Compose template
+3. Some tools (like one-off scripts) are not meant for persistent terminal sessions
+
+## Configuration
+
+No additional configuration is required. The terminal adapts automatically to:
+- Browser window size (via ResizeObserver)
+- System light/dark theme preference
+- Network conditions (reconnect backoff)
+
+## Related Features
+
+- [Workspace](workspace.md) — Open the terminal from the repository workspace
+- [Tool Types](tool-types.md) — Configure which tools expose a terminal interface
+- [SSH Keys](ssh-keys.md) — Manage SSH keys for repository access from within the terminal