headquarter/docs/architecture/backend.md

# Backend Architecture

## Overview

The Headquarter backend is built with **FastAPI** and follows a layered architecture pattern.

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────────┐
│                         FastAPI App                          │
├─────────────────────────────────────────────────────────────┤
│  Middleware: CORS → Request Logging → Exception Logging      │
├─────────────────────────────────────────────────────────────┤
│  API Layer (src/api/)                                        │
│  ┌─────────┐ ┌─────────┐ ┌────────┐ ┌──────────┐           │
│  │  Auth   │ │Terminal │ │Projects│ │   Git    │           │
│  │ Routes  │ │   WS    │ │ Routes │ │ Repos    │           │
│  └────┬────┘ └────┬────┘ └───┬────┘ └────┬─────┘           │
├───────┼───────────┼──────────┼───────────┼──────────────────┤
│       │           │          │           │                  │
│  Auth │  Terminal │ Project  │   Git     │                  │
│ Layer │  Manager  │ Service  │ Service   │                  │
│       │  + Session│          │           │                  │
├───────┴───────────┴──────────┴───────────┴──────────────────┤
│  Data Layer                                                  │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐                   │
│  │  Models  │  │ Database │  │   Config │                   │
│  │(SQLAlch) │  │(AsyncPG) │  │(Pydantic)│                   │
│  └──────────┘  └──────────┘  └──────────┘                   │
└─────────────────────────────────────────────────────────────┘
```

## Directory Structure

```
src/
├── api/                    # API Routes
│   ├── auth.py            # Authentication endpoints
│   ├── terminal.py        # WebSocket terminal endpoint
│   ├── projects.py        # Project endpoints
│   ├── git_repositories.py # Repository endpoints
│   ├── users.py           # User endpoints
│   ├── tool_types.py      # Tool type endpoints
│   ├── ssh_keys.py        # SSH key endpoints
│   └── dashboard.py       # Dashboard endpoints
├── auth/                   # Authentication
│   ├── session.py         # Session management
│   ├── oidc.py            # OAuth2 client
│   ├── dependencies.py    # Auth dependencies
│   └── cookies.py         # Cookie utilities
├── models/                 # Database Models
│   ├── user.py            # User model
│   ├── project.py         # Project model
│   ├── git_repository.py  # Repository model
│   ├── tool_type.py       # Tool type model
│   ├── ssh_key.py         # SSH key model
│   └── user_config.py     # User config model
├── services/              # Business Logic
│   ├── terminal_manager.py # Terminal session manager
│   ├── terminal_session.py # PTY + docker exec session
│   ├── docker.py          # Docker operations
│   └── profile_resolver.py # Profile resolution
├── utils/                  # Utilities
│   ├── git_url_parser.py  # URL parsing
│   ├── git_files.py       # Git file operations
│   └── git_history.py     # Git history operations
├── config.py              # Configuration
├── database.py            # Database setup
└── main.py                # Application entry point
```

## Terminal System

The terminal system provides interactive shell access to running tool instances via WebSocket.

### Architecture

```
Client (WebSocket)
  ↕
terminal.py (FastAPI WS endpoint)
  ├─ Auth validation (session cookie)
  ├─ Instance ownership check
  ├─ Session lifecycle (create / monitor / cleanup)
  └─ Echo state detection (termios)
  ↕
TerminalManager
  ├─ create_session() → spawns TerminalSession
  ├─ _read_loop() → batches PTY output → WebSocket
  ├─ _write_loop() → WebSocket input → PTY
  └─ _heartbeat_loop() → closes idle connections (60s)
  ↕
TerminalSession
  ├─ start() → pty.openpty() + docker exec
  ├─ read_output() → select.select() + os.read()
  ├─ write_input() → os.write() to PTY master
  ├─ resize() → TIOCSWINSZ ioctl
  └─ check_echo_state() → termios.ECHO flag
```

### Protocol

**Binary frames**: Raw terminal I/O (hot path)
**Text (JSON) frames**: Control messages

**Control messages:**

| Direction | Type | Purpose |
|-----------|------|---------|
| Client → Server | `ping` | Heartbeat (every 15s idle) |
| Server → Client | `pong` | Heartbeat response |
| Client → Server | `resize` | Terminal dimensions changed |
| Server → Client | `set_echo_state` | Enable/disable local echo |
| Server → Client | `session_ended` | Container process exited |

### Message Batching

The read loop batches small PTY reads into single WebSocket frames:
- Buffer accumulates data for up to 16ms
- Flushed immediately when no new data is available
- Reduces WebSocket frame overhead for rapid output

### Reconnect Behavior

The server cannot resume a `docker exec` PTY across connections. On reconnect:
1. Old session is terminated
2. New `docker exec` is spawned
3. Client restores scrollback from `sessionStorage`
4. New shell appears seamlessly to the user

## Layers

### 1. API Layer (`src/api/`)

**Responsibilities:**
- Define HTTP endpoints
- Parse request parameters
- Return HTTP responses
- Use dependencies for auth and DB

**Pattern:**
```python
@router.get("/projects")
async def list_projects(
    session: AsyncSession = Depends(get_db_session),
    user_id: str = Depends(get_current_user_id),
):
    # Call service layer
    projects = await project_service.list(session, user_id)
    return projects
```

### 2. Auth Layer (`src/auth/`)

**Responsibilities:**
- Session management (create, verify, expire)
- OAuth2 flow (login, callback)
- User authentication dependencies

**Key Components:**
- `session.py`: HMAC-SHA256 signed cookies
- `oidc.py`: OAuth2 token exchange
- `dependencies.py`: FastAPI dependencies for auth

### 3. Data Layer (`src/models/`, `src/database.py`)

**Responsibilities:**
- Database schema definition
- Async database sessions
- Connection management

**Technology:**
- SQLAlchemy 2.0 with async support
- asyncpg driver for PostgreSQL
- Alembic for migrations

### 4. Utility Layer (`src/utils/`)

**Responsibilities:**
- Git operations (file browsing, history)
- URL parsing
- Helper functions

## Data Flow

### Request Lifecycle

```
1. Request arrives at FastAPI
2. Middleware processes (CORS, logging)
3. Auth dependency verifies session
4. Route handler processes request
5. Database session executes queries
6. Response returned to client
```

### Authentication Flow

```
1. User clicks login
2. Backend redirects to Authentik
3. User authenticates
4. Authentik redirects with code
5. Backend exchanges code for token
6. Backend fetches user info
7. Backend creates session cookie
8. User is authenticated
```

## Dependencies

### Database Session

```python
async def get_db_session() -> AsyncGenerator[AsyncSession, None]:
    async with SessionLocal() as session:
        yield session
```

### Current User

```python
async def get_current_user_id(
    request: Request,
    settings: Settings = Depends(get_settings),
) -> str:
    # Verify session cookie
    # Return user_id
```

## Configuration

Configuration is managed via Pydantic Settings:

```python
class Settings(BaseSettings):
    app_env: str = "development"
    database_url: str = "..."
    authentik_domain: str = "..."
    # ...
```

Environment variables are automatically loaded from `.env` files.

## Error Handling

Errors are handled at multiple levels:

1. **Validation**: Pydantic validates request bodies
2. **HTTP Exceptions**: FastAPI HTTPException for client errors
3. **Middleware**: ExceptionLoggingMiddleware logs server errors
4. **Database**: SQLAlchemy errors converted to HTTP responses

## Testing

- **Unit tests**: SQLite in-memory database
- **Integration tests**: PostgreSQL with transaction rollback
- **Fixtures**: Shared in `conftest.py`

## Technology Stack

| Component | Technology | Version |
|-----------|-----------|---------|
| Web Framework | FastAPI | ^0.104 |
| ORM | SQLAlchemy | ^2.0 |
| Database Driver | asyncpg | ^0.29 |
| Validation | Pydantic | ^2.0 |
| Migrations | Alembic | ^1.12 |
| HTTP Client | httpx | ^0.25 |
| Testing | pytest | ^7.4 |

## Related Documentation

- [Database Schema](database.md)
- [Frontend Architecture](../frontend.md)
- [API Documentation](../api/)