merge: align dev branch with main
This commit is contained in:
Developer
@@ -13,20 +13,16 @@ The Headquarter backend is built with **FastAPI** and follows a layered architec
|
||||
│ Middleware: CORS → Request Logging → Exception Logging │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ API Layer (src/api/) │
|
||||
│ ┌─────────┐ ┌──────────┐ ┌────────┐ ┌──────────┐ │
|
||||
│ │ Auth │ │ Projects │ │ Users │ │ Git │ │
|
||||
│ │ Routes │ │ Routes │ │ Routes │ │ Repos │ │
|
||||
│ └────┬────┘ └────┬─────┘ └───┬────┘ └────┬─────┘ │
|
||||
│ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ ToolInst │ │ Events │ │
|
||||
│ │ Routes │ │ Routes │ │
|
||||
│ └────┬─────┘ └────┬─────┘ │
|
||||
├───────┼───────────┼───────────┼───────────┼─────────────────┤
|
||||
│ │ │ │ │ │
|
||||
│ Auth │ Project │ User │ Git │ │
|
||||
│ Layer │ Service │ Service │ Service │ │
|
||||
│ │ │ │ │ │
|
||||
├───────┴───────────┴───────────┴───────────┴─────────────────┤
|
||||
│ ┌─────────┐ ┌─────────┐ ┌────────┐ ┌──────────┐ │
|
||||
│ │ Auth │ │Terminal │ │Projects│ │ Git │ │
|
||||
│ │ Routes │ │ WS │ │ Routes │ │ Repos │ │
|
||||
│ └────┬────┘ └────┬────┘ └───┬────┘ └────┬─────┘ │
|
||||
├───────┼───────────┼──────────┼───────────┼──────────────────┤
|
||||
│ │ │ │ │ │
|
||||
│ Auth │ Terminal │ Project │ Git │ │
|
||||
│ Layer │ Manager │ Service │ Service │ │
|
||||
│ │ + Session│ │ │ │
|
||||
├───────┴───────────┴──────────┴───────────┴──────────────────┤
|
||||
│ Data Layer │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Models │ │ Database │ │ Config │ │
|
||||
@@ -41,13 +37,12 @@ The Headquarter backend is built with **FastAPI** and follows a layered architec
|
||||
src/
|
||||
├── api/ # API Routes
|
||||
│ ├── auth.py # Authentication endpoints
|
||||
│ ├── terminal.py # WebSocket terminal endpoint
|
||||
│ ├── projects.py # Project endpoints
|
||||
│ ├── git_repositories.py # Repository endpoints
|
||||
│ ├── users.py # User endpoints
|
||||
│ ├── tool_types.py # Tool type endpoints
|
||||
│ ├── tool_instances.py # Tool instance endpoints
|
||||
│ ├── ssh_keys.py # SSH key endpoints
|
||||
│ ├── events.py # SSE streaming endpoint
|
||||
│ └── dashboard.py # Dashboard endpoints
|
||||
├── auth/ # Authentication
|
||||
│ ├── session.py # Session management
|
||||
@@ -60,15 +55,12 @@ src/
|
||||
│ ├── git_repository.py # Repository model
|
||||
│ ├── tool_type.py # Tool type model
|
||||
│ ├── ssh_key.py # SSH key model
|
||||
│ ├── instance_event.py # Instance event audit model
|
||||
│ ├── health_check.py # Health check snapshot model
|
||||
│ └── user_config.py # User config model
|
||||
├── services/ # Services
|
||||
│ ├── docker.py # Docker operations
|
||||
├── services/ # Business Logic
|
||||
│ ├── terminal_manager.py # Terminal session manager
|
||||
│ ├── event_bus.py # Instance event bus (pub/sub)
|
||||
│ ├── health_monitor.py # Background health monitoring
|
||||
│ └── lifecycle_hooks.py # Instance lifecycle events
|
||||
│ ├── terminal_session.py # PTY + docker exec session
|
||||
│ ├── docker.py # Docker operations
|
||||
│ └── profile_resolver.py # Profile resolution
|
||||
├── utils/ # Utilities
|
||||
│ ├── git_url_parser.py # URL parsing
|
||||
│ ├── git_files.py # Git file operations
|
||||
@@ -78,6 +70,65 @@ src/
|
||||
└── main.py # Application entry point
|
||||
```
|
||||
|
||||
## Terminal System
|
||||
|
||||
The terminal system provides interactive shell access to running tool instances via WebSocket.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Client (WebSocket)
|
||||
↕
|
||||
terminal.py (FastAPI WS endpoint)
|
||||
├─ Auth validation (session cookie)
|
||||
├─ Instance ownership check
|
||||
├─ Session lifecycle (create / monitor / cleanup)
|
||||
└─ Echo state detection (termios)
|
||||
↕
|
||||
TerminalManager
|
||||
├─ create_session() → spawns TerminalSession
|
||||
├─ _read_loop() → batches PTY output → WebSocket
|
||||
├─ _write_loop() → WebSocket input → PTY
|
||||
└─ _heartbeat_loop() → closes idle connections (60s)
|
||||
↕
|
||||
TerminalSession
|
||||
├─ start() → pty.openpty() + docker exec
|
||||
├─ read_output() → select.select() + os.read()
|
||||
├─ write_input() → os.write() to PTY master
|
||||
├─ resize() → TIOCSWINSZ ioctl
|
||||
└─ check_echo_state() → termios.ECHO flag
|
||||
```
|
||||
|
||||
### Protocol
|
||||
|
||||
**Binary frames**: Raw terminal I/O (hot path)
|
||||
**Text (JSON) frames**: Control messages
|
||||
|
||||
**Control messages:**
|
||||
|
||||
| Direction | Type | Purpose |
|
||||
|-----------|------|---------|
|
||||
| Client → Server | `ping` | Heartbeat (every 15s idle) |
|
||||
| Server → Client | `pong` | Heartbeat response |
|
||||
| Client → Server | `resize` | Terminal dimensions changed |
|
||||
| Server → Client | `set_echo_state` | Enable/disable local echo |
|
||||
| Server → Client | `session_ended` | Container process exited |
|
||||
|
||||
### Message Batching
|
||||
|
||||
The read loop batches small PTY reads into single WebSocket frames:
|
||||
- Buffer accumulates data for up to 16ms
|
||||
- Flushed immediately when no new data is available
|
||||
- Reduces WebSocket frame overhead for rapid output
|
||||
|
||||
### Reconnect Behavior
|
||||
|
||||
The server cannot resume a `docker exec` PTY across connections. On reconnect:
|
||||
1. Old session is terminated
|
||||
2. New `docker exec` is spawned
|
||||
3. Client restores scrollback from `sessionStorage`
|
||||
4. New shell appears seamlessly to the user
|
||||
|
||||
## Layers
|
||||
|
||||
### 1. API Layer (`src/api/`)
|
||||
@@ -207,33 +258,6 @@ Errors are handled at multiple levels:
|
||||
- **Integration tests**: PostgreSQL with transaction rollback
|
||||
- **Fixtures**: Shared in `conftest.py`
|
||||
|
||||
## Monitoring & Notifications
|
||||
|
||||
The backend includes a real-time monitoring system:
|
||||
|
||||
### Components
|
||||
|
||||
- **InstanceEventBus** (`services/event_bus.py`): Typed pub/sub singleton for instance lifecycle events
|
||||
- **HealthMonitor** (`services/health_monitor.py`): Asyncio background task polling container health every 15s
|
||||
- **SSE Endpoint** (`api/events.py`): Server-Sent Events streaming for real-time frontend updates
|
||||
- **Lifecycle Hooks** (`services/lifecycle_hooks.py`): Publishes events on create/start/stop/restart/delete
|
||||
|
||||
### Event Flow
|
||||
|
||||
```
|
||||
Container Action → Lifecycle Hook → EventBus → SSE Stream → Frontend Toast
|
||||
```
|
||||
|
||||
### Event Types
|
||||
|
||||
| Event | When Fired |
|
||||
|-------|-----------|
|
||||
| `instance.created` | After DB insert |
|
||||
| `instance.starting` | Before docker compose up |
|
||||
| `instance.running` | After readiness probe succeeds |
|
||||
| `instance.error` | Build fail, crash, or probe fail |
|
||||
| `instance.stopped` | After docker compose stop |
|
||||
|
||||
## Technology Stack
|
||||
|
||||
| Component | Technology | Version |
|
||||
|
||||
@@ -26,27 +26,26 @@ apps/web/src/
|
||||
│ ├── ssh_keys.ts # SSH key API
|
||||
│ ├── tool_types.ts # Tool type API
|
||||
│ ├── users.ts # User API
|
||||
│ ├── events.ts # SSE events API
|
||||
│ ├── sessions.ts # Tool instance sessions API
|
||||
│ └── settings.ts # Settings API
|
||||
├── components/ # Reusable components
|
||||
│ ├── app-shell.tsx # Main app layout
|
||||
│ ├── terminal.tsx # xterm.js terminal component
|
||||
│ ├── protected-route.tsx # Auth guard
|
||||
│ ├── event-toast-bridge.tsx # Events → toasts
|
||||
│ └── [more...]
|
||||
├── state/ # Global state
|
||||
│ ├── auth.tsx # Auth state management
|
||||
│ ├── events.tsx # Event provider (SSE)
|
||||
│ └── toast.tsx # Toast notifications
|
||||
├── context/ # React contexts
|
||||
│ └── auth.tsx # Auth state management
|
||||
├── hooks/ # Custom hooks
|
||||
│ ├── use-auth.ts # Auth hook
|
||||
│ ├── use-theme.ts # Theme hook
|
||||
│ └── use-events.ts # SSE events hook
|
||||
│ └── use-terminal-connection.ts # Terminal WebSocket lifecycle
|
||||
├── pages/ # Page components (routes)
|
||||
│ ├── dashboard.tsx # Dashboard
|
||||
│ ├── projects.tsx # Project list
|
||||
│ ├── repo-workspace.tsx # Repository workspace
|
||||
│ ├── git-history.tsx # Git history
|
||||
│ ├── git-repositories.tsx # Repository management
|
||||
│ ├── terminal.tsx # Web terminal
|
||||
│ ├── profile.tsx # User profile
|
||||
│ ├── settings.tsx # User settings
|
||||
│ ├── tool-types.tsx # Tool types
|
||||
@@ -160,28 +159,6 @@ interface AuthState {
|
||||
}
|
||||
```
|
||||
|
||||
### Real-Time Events (SSE)
|
||||
|
||||
The frontend receives real-time instance events via Server-Sent Events:
|
||||
|
||||
```
|
||||
EventSource → useEvents() hook → EventProvider → EventToastBridge → ToastContainer
|
||||
```
|
||||
|
||||
**Components:**
|
||||
- `useEvents()`: Manages SSE connection with auto-reconnect
|
||||
- `EventProvider`: Shares event stream across components
|
||||
- `EventToastBridge`: Maps events to toast notifications
|
||||
- `ToastContainer`: Displays and manages toast stack
|
||||
|
||||
**Event-to-Toast Mapping:**
|
||||
| Event | Toast Severity | Auto-dismiss |
|
||||
|-------|---------------|--------------|
|
||||
| `instance.starting` | Info | 3s |
|
||||
| `instance.running` | Success | 3s |
|
||||
| `instance.error` | Error | Persistent |
|
||||
| `instance.stopped` | Info | 3s |
|
||||
|
||||
### 5. Routing Structure
|
||||
|
||||
```typescript
|
||||
@@ -191,6 +168,7 @@ EventSource → useEvents() hook → EventProvider → EventToastBridge → Toas
|
||||
<Route path="/projects/:projectId" element={<RepoWorkspace />} />
|
||||
<Route path="/projects/:projectId/repositories" element={<GitRepositories />} />
|
||||
<Route path="/projects/:projectId/repositories/:repoId/history" element={<GitHistory />} />
|
||||
<Route path="/terminal/:instanceId" element={<TerminalPage />} />
|
||||
<Route path="/profile" element={<ProfilePage />} />
|
||||
<Route path="/settings" element={<SettingsPage />} />
|
||||
<Route path="/ssh-keys" element={<SSHKeysPage />} />
|
||||
@@ -296,12 +274,64 @@ test('renders file list', () => {
|
||||
4. **Caching**: Browser caches API responses (ETags)
|
||||
5. **Optimistic UI**: Immediate feedback before API response
|
||||
|
||||
## Terminal Architecture
|
||||
|
||||
The web terminal is the most complex component in the frontend. It bridges a browser-based terminal emulator with a server-side PTY session.
|
||||
|
||||
### Component Stack
|
||||
|
||||
```
|
||||
TerminalPage (route)
|
||||
└── TerminalComponent
|
||||
├── Status bar (connection state, latency, actions)
|
||||
├── Session-ended overlay (reconnect / go back)
|
||||
├── Reconnect banner (spinner + countdown)
|
||||
└── xterm.js (terminal emulator)
|
||||
├── FitAddon (auto-resize to container)
|
||||
├── SerializeAddon (scrollback serialization)
|
||||
└── WebLinksAddon (clickable URLs)
|
||||
```
|
||||
|
||||
### Connection Hook
|
||||
|
||||
`useTerminalConnection` manages the full WebSocket lifecycle:
|
||||
|
||||
```
|
||||
CONNECTING
|
||||
→ onopen → CONNECTED → heartbeat every 15s
|
||||
→ onclose (unexpected) → RECONNECTING
|
||||
→ backoff: 1s → 2s → 4s → 8s → 16s → 30s max
|
||||
→ up to 10 attempts
|
||||
→ onopen → restore scrollback → CONNECTED
|
||||
→ onclose (expected) → DISCONNECTED
|
||||
```
|
||||
|
||||
**Key behaviors:**
|
||||
- **Local echo**: Printable ASCII chars appear instantly; server echo is deduplicated
|
||||
- **Resize**: Debounced 200ms, throttled to 1 message per 500ms
|
||||
- **Scrollback**: Serialized to `sessionStorage` on disconnect, restored on reconnect
|
||||
- **Keyboard**: `Ctrl+Shift+R` triggers manual reconnect
|
||||
|
||||
### Data Flow
|
||||
|
||||
```
|
||||
User types 'a'
|
||||
→ xterm onData event
|
||||
→ useTerminalConnection.sendInput('a')
|
||||
→ local echo writes 'a' to xterm immediately
|
||||
→ WebSocket sends 'a' to server
|
||||
→ server PTY echoes 'a' back
|
||||
→ client receives 'a' via binary frame
|
||||
→ deduplicates against pending echo buffer
|
||||
→ (no-op if matched, or writes remaining chars)
|
||||
```
|
||||
|
||||
## Future Improvements
|
||||
|
||||
- [x] Implement real-time updates (SSE)
|
||||
- [ ] Add React Query for server state management
|
||||
- [ ] Implement virtual scrolling for large file trees
|
||||
- [ ] Add service worker for offline support
|
||||
- [x] Implement real-time updates (WebSocket) — Terminal done
|
||||
- [ ] Add error boundary components
|
||||
|
||||
## Development Workflow
|
||||
|
||||
Reference in New Issue
Block a user