62640daf36
- Update architecture.md with spawn service and auth proxy sections - Update deployment.md with production stack details - Update development.md with spawn workflow documentation - Update mvp-scope.md, project-brief.md, tool-manifest-spec.md - Update conversation-handoff.md with current status - Replace all RunFusion references with OpenCode
1174 lines
50 KiB
Markdown
1174 lines
50 KiB
Markdown
# Headquarter Architecture
|
|
|
|
> Canonical architecture specification for the hosted workspace and tool-orchestration platform.
|
|
> Decisions in this document override ad-hoc choices in implementation tasks.
|
|
|
|
## 1. Overview & Goals
|
|
|
|
Headquarter is a hosted control plane where authenticated users create projects, connect Git repositories, and spawn containerized tools (OpenCode, code-server, and future tools). The platform is manifest-driven and provider-abstracted so new tools, Git providers, runtimes, and access providers can be added without rewriting core orchestration logic.
|
|
|
|
**Product purpose:** Give individual developers and small teams a self-hosted alternative to cloud IDEs and CI dashboards by combining Git-backed project workspaces with on-demand tool containers, all routed through a unified subdomain scheme.
|
|
|
|
**MVP scope:** Single-user projects, Authentik OIDC auth, Docker runtime, Traefik subdomain routing, Portainer-managed Docker Compose deployment. The MVP supports two built-in tools (OpenCode and code-server) and provides extension points for additional tools, Git providers, and runtimes.
|
|
|
|
**Non-goals (explicitly out of MVP scope):**
|
|
- Multi-user teams or shared projects
|
|
- Real-time collaboration inside tools
|
|
- Advanced CI/CD pipelines
|
|
- Kubernetes or non-Docker runtimes
|
|
- Automatic Git provider webhooks
|
|
- Built-in GitHub/GitLab UI integrations
|
|
- Backup and disaster recovery automation
|
|
- High availability, replicas, or horizontal scaling
|
|
- Rate limiting or audit logging
|
|
|
|
---
|
|
|
|
## 2. System Context Diagram
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph External
|
|
U[User/Browser]
|
|
A[Authentik OIDC]
|
|
G[Git Providers<br/>GitHub / GitLab / Gitea / Forgejo]
|
|
end
|
|
|
|
subgraph Platform
|
|
T[Traefik Reverse Proxy]
|
|
W[Web Frontend<br/>apps/web — React + Vite]
|
|
B[FastAPI Backend<br/>apps/api — Python]
|
|
D[(PostgreSQL)]
|
|
end
|
|
|
|
subgraph Runtime
|
|
R[Docker Compose Stack<br/>Portainer-managed]
|
|
TC[Tool Containers<br/>OpenCode / code-server]
|
|
end
|
|
|
|
U -->|HTTPS| T
|
|
T -->|subdomain routing| W
|
|
T -->|subdomain routing| B
|
|
T -->|subdomain routing| TC
|
|
W -->|API calls /api/v1| B
|
|
B -->|SQLAlchemy async| D
|
|
B -->|Docker API / Compose| R
|
|
R -->|spawn / stop / health| TC
|
|
B -->|GitProvider adapter| G
|
|
A -->|OIDC login / callback / JWKS| W
|
|
A -->|OIDC login / callback / JWKS| B
|
|
TC -->|Git clone/fetch/push| G
|
|
```
|
|
|
|
**Data flows:**
|
|
1. **User → Traefik → Web frontend → FastAPI backend → PostgreSQL** — Standard request/response for all platform operations.
|
|
2. **FastAPI backend → Docker API / Compose → Spawned tool containers** — Orchestration commands to create, stop, and inspect containers.
|
|
3. **Tool containers → Traefik (subdomain routing)** — Exposes running tools to users on unique subdomains.
|
|
4. **FastAPI backend → Git providers (via provider adapters)** — Remote API operations such as deploy-key management and repository listing.
|
|
5. **Authentik → Web frontend + FastAPI backend (OIDC flow)** — Authentication via authorization-code grant with PKCE.
|
|
|
|
---
|
|
|
|
## 3. Component Boundaries
|
|
|
|
### 3.1 Frontend (`apps/web/`)
|
|
|
|
**Responsibilities:**
|
|
- Render the authenticated dashboard shell (header, navigation, main content area)
|
|
- Manage auth state (loading, authenticated, unauthenticated, error)
|
|
- Provide navigation for Dashboard, Projects, Repositories, Tools, and Settings
|
|
- Implement placeholder and active product screens for project creation, tool spawning, and tool access URLs
|
|
- Consume the FastAPI backend via a typed API client layer
|
|
- Handle Vite environment variables for runtime configuration
|
|
|
|
**Technology Stack:**
|
|
- React 19 + TypeScript with strict mode
|
|
- Vite for build tooling
|
|
- React Router v7 for client-side routing
|
|
- TanStack Query v5 for server state management
|
|
- Zustand for client state management (auth store)
|
|
- Tailwind CSS v4 for styling
|
|
- Headless UI for accessible components
|
|
- Heroicons for iconography
|
|
|
|
**Project Structure:**
|
|
```
|
|
apps/web/src/
|
|
├── api/ # API client and error handling
|
|
├── auth/ # OIDC utilities and AuthProvider
|
|
├── components/ # Reusable UI components (Header, Sidebar, RouteGuard, DashboardLayout)
|
|
├── pages/ # Page components (Dashboard, Projects, Tools, Settings, etc.)
|
|
├── stores/ # Zustand stores (auth store)
|
|
├── types/ # TypeScript type definitions matching backend schemas
|
|
└── router.tsx # React Router configuration
|
|
```
|
|
|
|
**Routing:**
|
|
- `/` — Dashboard (protected)
|
|
- `/projects` — Project list (protected)
|
|
- `/projects/new` — Create project (protected)
|
|
- `/projects/:id` — Project detail (protected)
|
|
- `/projects/:id/edit` — Edit project (protected)
|
|
- `/repositories` — Repository management (protected)
|
|
- `/tools` — Tool registry and spawn surface (protected)
|
|
- `/settings` — User and platform settings (protected)
|
|
- `/login` — Login redirect (public)
|
|
- `/callback` — OIDC callback handler (public)
|
|
|
|
**Auth state:**
|
|
- Managed via Zustand auth store (`useAuthStore`)
|
|
- States: `loading`, `authenticated`, `unauthenticated`, `error`
|
|
- Access token stored in `localStorage` (MVP simplification; httpOnly cookie recommended for production)
|
|
- PKCE flow for OIDC authentication
|
|
- On 401 from API, redirect to login
|
|
|
|
**API client conventions:**
|
|
- Base URL from `VITE_API_URL`
|
|
- JSON request/response with standard HTTP status codes
|
|
- `ApiError` class for consistent error handling
|
|
- Bearer token injected via `Authorization` header
|
|
- Debug mode request/response logging
|
|
- Typed API methods for all endpoints
|
|
|
|
### 3.2 Backend (`apps/api/`)
|
|
|
|
**Module boundaries:**
|
|
|
|
| Module | Responsibility |
|
|
|--------|---------------|
|
|
| `app/config.py` | Pydantic Settings — env var validation and defaults |
|
|
| `app/db.py` | Async SQLAlchemy engine, session factory, `get_db_session` dependency |
|
|
| `app/models/` | SQLAlchemy 2.0 domain models (ORM) |
|
|
| `app/schemas/` | Pydantic v2 request/response schemas |
|
|
| `app/auth/` | JWT/OIDC decoding, `get_current_user` / `get_current_active_user` dependencies |
|
|
| `app/encryption.py` | Fernet-based secret encryption/decryption |
|
|
| `app/routers/` | FastAPI APIRouter instances per domain area |
|
|
| `app/git/` | Provider-independent Git abstraction (types, provider ABC, credentials, SSH keys, connection manager, operations) |
|
|
| `app/infrastructure/` | Deployment utilities (Traefik label generation, runtime adapters) |
|
|
|
|
**Provider interface layer:**
|
|
- `app/git/provider.py` — `GitProvider` ABC for remote provider API operations
|
|
- `app/git/operations.py` — `GitOperations` ABC for local Git CLI orchestration
|
|
- `app/infrastructure/traefik_labels.py` — `generate_traefik_labels()`, `generate_tool_compose_service()`
|
|
- Future: `app/runtime/` — `RuntimeProvider` ABC for container orchestration
|
|
- Future: `app/access/` — `AccessProvider` ABC for routing configuration
|
|
|
|
**Domain model layer:**
|
|
- All models inherit `Base`, `UUIDMixin`, `TimestampMixin`
|
|
- UUID primary keys for all entities
|
|
- Ownership-based authorization: users own projects; resources belong to projects
|
|
- No shared projects in MVP (schema leaves room for `ProjectMember` later)
|
|
|
|
**API layer:**
|
|
- Versioned under `/api/v1`
|
|
- All write endpoints require `get_current_active_user`
|
|
- Read endpoints require authentication by default
|
|
- Thin router handlers: direct DB session + model queries, no service layer in MVP
|
|
|
|
### 3.3 Database
|
|
|
|
**Role:** PostgreSQL 17+ stores all relational data: users, projects, repositories, tool definitions, tool instances, configs, secrets, access routes, and repository connections.
|
|
|
|
**Migration strategy:**
|
|
- Alembic manages schema migrations
|
|
- Migrations are auto-generated from SQLAlchemy models during development
|
|
- Migrations are reviewed and committed; never edit existing migration files after they have been applied in production
|
|
- Test migrations with `alembic upgrade head` and `alembic downgrade -1` before committing
|
|
|
|
### 3.4 Reverse Proxy
|
|
|
|
**Role:** Traefik (v2.10+ or v3) terminates TLS, routes subdomains, and applies middleware.
|
|
|
|
**Label generation:**
|
|
- Backend generates Traefik Docker labels dynamically at spawn time via `app/infrastructure/traefik_labels.py`
|
|
- Labels include `traefik.enable`, `Host`, `entrypoints`, `tls.certresolver`, `service`, `loadbalancer.server.port`, and optional middleware references
|
|
- Labels are applied to tool containers as Docker container labels
|
|
|
|
**Network attachment:**
|
|
- Tool containers must attach to the external Traefik Docker network (default name: `traefik`)
|
|
- The platform stack also attaches to this network so Traefik can route to both the platform and spawned tools
|
|
|
|
### 3.5 Runtime
|
|
|
|
**Role:** Docker Compose stack managed by Portainer.
|
|
|
|
**Management model:**
|
|
- Portainer deploys the platform stack from `docker-compose.prod.yml` (or `deploy/portainer-stack.yml`)
|
|
- The backend spawns tool containers by generating Compose service definitions or using the Docker API directly
|
|
- Each tool instance is a separate Docker service/container
|
|
- Resource limits declared in the tool manifest are mapped to Docker Compose `deploy.resources`
|
|
|
|
---
|
|
|
|
## 4. PostgreSQL Domain Model (MVP)
|
|
|
|
All tables use `uuid` primary keys. All models include `created_at` and `updated_at` timestamps with `server_default=func.now()` and `onupdate=func.now()`.
|
|
|
|
### 4.1 `user`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `authentik_sub` | `VARCHAR(255)` | `UNIQUE`, `NOT NULL`, `INDEX` | | OIDC `sub` claim |
|
|
| `email` | `VARCHAR(255)` | `UNIQUE`, `NOT NULL`, `INDEX` | | |
|
|
| `display_name` | `VARCHAR(255)` | `NULLABLE` | `NULL` | From OIDC `name` claim |
|
|
| `is_active` | `BOOLEAN` | `NOT NULL` | `TRUE` | |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
### 4.2 `project`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `owner_id` | `UUID` | `FOREIGN KEY (user.id)`, `INDEX`, `NOT NULL` | | `ON DELETE CASCADE` |
|
|
| `name` | `VARCHAR(255)` | `NOT NULL` | | Display name |
|
|
| `slug` | `VARCHAR(255)` | `NOT NULL` | | URL-friendly identifier |
|
|
| `description` | `TEXT` | `NULLABLE` | `NULL` | |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
**Indexes:**
|
|
- `UNIQUE (owner_id, slug)` — slugs are unique per user
|
|
|
|
### 4.3 `repository`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `project_id` | `UUID` | `FOREIGN KEY (project.id)`, `INDEX`, `NOT NULL` | | `ON DELETE CASCADE` |
|
|
| `name` | `VARCHAR(255)` | `NOT NULL` | | |
|
|
| `git_url` | `TEXT` | `NOT NULL` | | Full clone URL |
|
|
| `provider_type` | `VARCHAR(50)` | `NOT NULL` | `'generic'` | `github`, `gitlab`, `gitea`, `forgejo` |
|
|
| `default_branch` | `VARCHAR(100)` | `NOT NULL` | `'main'` | |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
### 4.4 `workspace`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `project_id` | `UUID` | `FOREIGN KEY (project.id)`, `INDEX`, `NOT NULL` | | `ON DELETE CASCADE` |
|
|
| `name` | `VARCHAR(255)` | `NOT NULL` | | |
|
|
| `mount_path` | `TEXT` | `NULLABLE` | `NULL` | Optional override path |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
### 4.5 `tool_definition`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `key` | `VARCHAR(100)` | `UNIQUE`, `NOT NULL`, `INDEX` | | Machine identifier (e.g., `opencode`, `code-server`) |
|
|
| `name` | `VARCHAR(255)` | `NOT NULL` | | Human-readable name |
|
|
| `version` | `VARCHAR(50)` | `NOT NULL` | `'1.0.0'` | |
|
|
| `description` | `TEXT` | `NULLABLE` | `NULL` | |
|
|
| `image` | `TEXT` | `NOT NULL` | | Docker image reference |
|
|
| `manifest_data` | `JSONB` | `NULLABLE` | `NULL` | Full manifest as JSON |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
### 4.6 `tool_instance`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `project_id` | `UUID` | `FOREIGN KEY (project.id)`, `INDEX`, `NOT NULL` | | `ON DELETE CASCADE` |
|
|
| `tool_definition_id` | `UUID` | `FOREIGN KEY (tool_definition.id)`, `INDEX`, `NOT NULL` | | `ON DELETE RESTRICT` |
|
|
| `name` | `VARCHAR(255)` | `NOT NULL` | | User-defined instance name |
|
|
| `status` | `VARCHAR(50)` | `NOT NULL` | `'pending'` | See Section 8 for enum values |
|
|
| `container_id` | `VARCHAR(255)` | `NULLABLE`, `UNIQUE` | `NULL` | Docker container/service ID |
|
|
| `subdomain` | `VARCHAR(255)` | `NULLABLE`, `UNIQUE` | `NULL` | Generated subdomain |
|
|
| `config_override` | `JSONB` | `NULLABLE` | `NULL` | Instance-level config overrides |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
### 4.7 `config`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `scope_type` | `VARCHAR(50)` | `NOT NULL` | | `global`, `user`, `project`, `tool_instance` |
|
|
| `scope_id` | `UUID` | `INDEX`, `NOT NULL` | | Entity UUID matching `scope_type` |
|
|
| `tool_definition_id` | `UUID` | `FOREIGN KEY (tool_definition.id)`, `INDEX`, `NULLABLE` | `NULL` | `ON DELETE CASCADE` |
|
|
| `key` | `VARCHAR(255)` | `NOT NULL` | | Config key name |
|
|
| `value` | `JSONB` | `NOT NULL` | | Config value (any JSON) |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
**Indexes:**
|
|
- `UNIQUE (scope_type, scope_id, tool_definition_id, key)`
|
|
- **Note:** PostgreSQL treats `NULL != NULL`, so rows with `tool_definition_id=NULL` and the same `(scope_type, scope_id, key)` are technically allowed duplicates. This is acceptable for MVP because global/user/project configs naturally do not need a `tool_definition_id`.
|
|
|
|
### 4.8 `secret`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `scope_type` | `VARCHAR(50)` | `NOT NULL` | | `global`, `user`, `project`, `tool_instance` |
|
|
| `scope_id` | `UUID` | `INDEX`, `NOT NULL` | | Entity UUID matching `scope_type` |
|
|
| `key` | `VARCHAR(255)` | `NOT NULL` | | Secret key name |
|
|
| `encrypted_value` | `TEXT` | `NOT NULL` | | Fernet-encrypted ciphertext |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
**Indexes:**
|
|
- `UNIQUE (scope_type, scope_id, key)`
|
|
|
|
### 4.9 `access_route`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `tool_instance_id` | `UUID` | `FOREIGN KEY (tool_instance.id)`, `INDEX`, `NOT NULL` | | `ON DELETE CASCADE` |
|
|
| `domain` | `TEXT` | `NOT NULL` | | Full routing domain |
|
|
| `path_prefix` | `VARCHAR(255)` | `NOT NULL` | `'/'` | |
|
|
| `provider_type` | `VARCHAR(50)` | `NOT NULL` | `'traefik'` | |
|
|
| `provider_config` | `JSONB` | `NULLABLE` | `NULL` | Provider-specific routing config |
|
|
| `is_active` | `BOOLEAN` | `NOT NULL` | `TRUE` | |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
### 4.10 `repository_connection`
|
|
|
|
| Column | Type | Constraints | Default | Notes |
|
|
|--------|------|-------------|---------|-------|
|
|
| `id` | `UUID` | `PRIMARY KEY` | `uuid_generate_v4()` | |
|
|
| `project_id` | `UUID` | `FOREIGN KEY (project.id)`, `INDEX`, `NOT NULL` | | `ON DELETE CASCADE` |
|
|
| `repository_id` | `UUID` | `FOREIGN KEY (repository.id)`, `INDEX`, `NULLABLE` | `NULL` | `ON DELETE SET NULL` — nullable because connection may be created before repository row exists |
|
|
| `provider_kind` | `VARCHAR(50)` | `NOT NULL` | `'generic'` | |
|
|
| `credential_id` | `UUID` | `INDEX`, `NULLABLE` | `NULL` | Opaque UUID referencing the encrypted credential stored in the `secret` table (via `CredentialStorage` abstraction) |
|
|
| `connection_status` | `VARCHAR(50)` | `NOT NULL` | `'pending'` | |
|
|
| `default_branch` | `VARCHAR(100)` | `NULLABLE` | `NULL` | |
|
|
| `created_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
| `updated_at` | `TIMESTAMP WITH TIME ZONE` | `NOT NULL` | `now()` | |
|
|
|
|
---
|
|
|
|
## 5. Git Provider Abstraction
|
|
|
|
The backend defines two complementary abstractions so that remote provider API logic and local Git CLI orchestration evolve independently.
|
|
|
|
### 5.1 GitProvider Protocol (Remote Operations)
|
|
|
|
```python
|
|
from abc import ABC, abstractmethod
|
|
from typing import Any
|
|
|
|
class GitProvider(ABC):
|
|
@abstractmethod
|
|
def get_kind(self) -> ProviderKind: ...
|
|
|
|
@abstractmethod
|
|
def validate_connection(self, git_url: str, credential_id: str) -> ConnectionStatus: ...
|
|
|
|
@abstractmethod
|
|
def list_repositories(self, credential_id: str) -> list[dict[str, Any]]: ...
|
|
|
|
@abstractmethod
|
|
def create_deploy_key(self, git_url: str, public_key: str) -> str:
|
|
"""Returns the provider-side deploy key ID."""
|
|
...
|
|
|
|
@abstractmethod
|
|
def delete_deploy_key(self, git_url: str, deploy_key_id: str) -> None: ...
|
|
|
|
@abstractmethod
|
|
def get_default_branch(self, git_url: str, credential_id: str) -> str: ...
|
|
```
|
|
|
|
- **Adapters:** GitHub, GitLab, Gitea, Forgejo, etc.
|
|
- **Credential types:** SSH key pairs (per-repository) or access tokens.
|
|
- **SSH keys must be scoped per repository connection** for clean revocation and auditability.
|
|
|
|
### 5.2 GitOperations Protocol (Local CLI)
|
|
|
|
```python
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
class GitOperations(ABC):
|
|
@abstractmethod
|
|
def clone(self, repo_url: str, dest: Path, credential_id: str) -> None: ...
|
|
|
|
@abstractmethod
|
|
def fetch(self, repo_path: Path, credential_id: str) -> None: ...
|
|
|
|
@abstractmethod
|
|
def push(self, repo_path: Path, credential_id: str) -> None: ...
|
|
|
|
@abstractmethod
|
|
def get_status(self, repo_path: Path) -> dict[str, Any]: ...
|
|
```
|
|
|
|
### 5.3 GitCredentials Model
|
|
|
|
```python
|
|
class GitCredential(BaseModel):
|
|
model_config = ConfigDict(extra="forbid")
|
|
|
|
id: uuid.UUID = Field(default_factory=uuid.uuid4)
|
|
kind: CredentialKind # ssh_key | access_token
|
|
encrypted_payload: str = Field(repr=False) # Never plaintext
|
|
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
|
updated_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
|
```
|
|
|
|
- **No plaintext fields:** `token` and `private_key` must never appear as model fields.
|
|
- **Storage:** Encrypted payload is opaque bytes encoded as a string.
|
|
- **Scope:** Credentials are associated with a specific `repository_connection`, not global.
|
|
|
|
### 5.4 SSH Key Generation Lifecycle
|
|
|
|
1. **Generation:** When a user connects a repository, `SshKeyLifecycle.generate(connection_id)` creates an Ed25519 key pair via `cryptography`.
|
|
2. **Storage:** The private key is serialized to PEM (PKCS8) and immediately encrypted via `encrypt_private_key(raw: bytes) -> str`. The public key is stored as plaintext OpenSSH format.
|
|
3. **Registration:** The public key is sent to the Git provider via `GitProvider.create_deploy_key()`.
|
|
4. **Status transitions:** `generated` → `registered` → `rotating` → `revoked`.
|
|
5. **Injection:** At runtime, the encrypted private key is decrypted and mounted into tool containers as a file (e.g., `/root/.ssh/id_ed25519`) or injected via SSH agent.
|
|
6. **Revocation:** On repository disconnect, the deploy key is deleted from the provider and the local key status is set to `revoked`.
|
|
|
|
### 5.5 Access Token Lifecycle
|
|
|
|
1. **Creation:** User provides an access token through the UI. The backend encrypts it immediately and stores only the ciphertext.
|
|
2. **Refresh:** Not supported in MVP. Future: implement provider-specific refresh logic.
|
|
3. **Revocation:** The user can delete the credential from the platform. The backend does **not** attempt to revoke tokens at the provider in MVP (documented limitation).
|
|
4. **Storage:** Same encrypted storage as SSH keys, scoped to the repository connection.
|
|
|
|
### 5.6 Provider Type Registry and Discovery
|
|
|
|
- Enum `ProviderKind` defines supported providers: `github`, `gitlab`, `gitea`, `forgejo`, `generic`.
|
|
- A factory function maps `ProviderKind` → concrete `GitProvider` subclass.
|
|
- New providers are added by implementing `GitProvider` and registering in the factory.
|
|
|
|
---
|
|
|
|
## 6. Repository Credential Model
|
|
|
|
### 6.1 Per-Repository Credential Association
|
|
|
|
Credentials are **never global**. Every credential is linked to a `repository_connection` row. This ensures:
|
|
- Clean revocation when a repository is disconnected
|
|
- No accidental reuse of credentials across projects
|
|
- Auditability: each connection has exactly one credential
|
|
|
|
### 6.2 Credential Rotation Strategy
|
|
|
|
**MVP:** Manual rotation only.
|
|
1. User deletes the existing credential.
|
|
2. User generates a new SSH key or provides a new access token.
|
|
3. The old deploy key is removed from the provider.
|
|
4. A new deploy key is registered.
|
|
|
|
**Future:** Automatic refresh for access tokens with expiry tracking.
|
|
|
|
### 6.3 Storage
|
|
|
|
Credentials are stored as `secret` rows with `scope_type='repository'` and `scope_id=<repository_connection_id>`. The `encrypted_value` field contains the Fernet-encrypted payload. The encryption key is the environment variable `SECRET_ENCRYPTION_KEY`.
|
|
|
|
---
|
|
|
|
## 7. Tool Manifest Model
|
|
|
|
Tools are defined by manifests that declare runtime behavior, resource needs, and routing requirements.
|
|
|
|
### 7.1 Manifest Schema
|
|
|
|
| Field | Type | Required | Description |
|
|
|-------|------|----------|-------------|
|
|
| `key` | `str` | Yes | Machine identifier (`opencode`, `code-server`) |
|
|
| `name` | `str` | Yes | Human-readable name |
|
|
| `version` | `str` | No | SemVer string; default `1.0.0` |
|
|
| `description` | `str` | No | Markdown-friendly description |
|
|
| `image` | `str` | Yes | Docker image reference (`codercom/code-server:4.89`) |
|
|
| `ports` | `list[Port]` | Yes | Exposed ports with protocol |
|
|
| `workspace_mounts` | `list[Mount]` | No | Paths to mount the project workspace |
|
|
| `config_mounts` | `list[Mount]` | No | Paths to mount persistent tool config |
|
|
| `env` | `dict[str, str]` | No | Default environment variables |
|
|
| `secrets` | `list[str]` | No | Secret key names required at runtime |
|
|
| `health_check` | `HealthCheck` | No | HTTP or command health check definition |
|
|
| `resource_limits` | `ResourceLimits` | No | CPU and memory constraints |
|
|
| `routing` | `Routing` | Yes | Traefik routing configuration |
|
|
| `bootstrap_commands` | `list[str]` | No | Commands to run on first start |
|
|
| `runtime_requirements` | `RuntimeRequirements` | No | Node/npm/pnpm version expectations |
|
|
|
|
#### Sub-schemas
|
|
|
|
**Port:**
|
|
```json
|
|
{
|
|
"port": 8080,
|
|
"protocol": "tcp",
|
|
"name": "web"
|
|
}
|
|
```
|
|
|
|
**Mount:**
|
|
```json
|
|
{
|
|
"source": "{workspace}",
|
|
"target": "/home/coder/project",
|
|
"read_only": false
|
|
}
|
|
```
|
|
|
|
**HealthCheck:**
|
|
```json
|
|
{
|
|
"type": "http",
|
|
"path": "/health",
|
|
"port": 8080,
|
|
"interval_seconds": 10,
|
|
"timeout_seconds": 5,
|
|
"retries": 3
|
|
}
|
|
```
|
|
|
|
**ResourceLimits:**
|
|
```json
|
|
{
|
|
"cpus": "1.0",
|
|
"memory": "1g",
|
|
"swap": "512m"
|
|
}
|
|
```
|
|
|
|
**Routing:**
|
|
```json
|
|
{
|
|
"subdomain_pattern": "{tool}-{project}-{user}.{tool_domain}",
|
|
"entrypoint": "websecure",
|
|
"cert_resolver": "letsencrypt",
|
|
"middlewares": ["security-headers"],
|
|
"tls": true
|
|
}
|
|
```
|
|
|
|
**RuntimeRequirements:**
|
|
```json
|
|
{
|
|
"node_version": ">=20",
|
|
"npm_version": ">=10",
|
|
"package_manager": "pnpm"
|
|
}
|
|
```
|
|
|
|
### 7.2 Manifest Validation Rules
|
|
|
|
1. `key` must match `^[a-z0-9-]+$` and be unique across the registry.
|
|
2. `image` must be a valid Docker image reference (registry optional, tag optional).
|
|
3. `ports` must contain at least one port.
|
|
4. `routing.subdomain_pattern` must include `{tool}`, `{project}`, and `{user}` placeholders or be documented as a custom pattern.
|
|
5. `health_check.port` must reference a port defined in `ports`.
|
|
6. `resource_limits.memory` and `resource_limits.swap` must match Docker Compose memory-string format (`<number><unit>`).
|
|
7. Duplicate `env` keys are forbidden.
|
|
8. `secrets` entries must be non-empty strings.
|
|
|
|
---
|
|
|
|
## 8. Tool Spawn Lifecycle
|
|
|
|
```
|
|
pending → provisioning → running → healthy
|
|
↘
|
|
unhealthy
|
|
|
|
healthy / unhealthy → stopping → stopped
|
|
```
|
|
|
|
### 8.1 Phase Definitions
|
|
|
|
| Phase | Backend Action | Docker/Compose Artifacts | Labels Applied |
|
|
|-------|---------------|--------------------------|----------------|
|
|
| `pending` | User requests spawn; backend validates manifest, checks project ownership, allocates subdomain. | None | None |
|
|
| `provisioning` | Backend generates Docker Compose service definition; creates volumes; injects secrets and config; calls Docker API to create container. | Service name, volume mounts, env vars, secret files. | `traefik.enable=true`, router rule, service, TLS, network. |
|
|
| `running` | Container is started; backend polls Docker for container state. | Container ID recorded in `tool_instance.container_id`. | Same as provisioning. |
|
|
| `healthy` | Health check (HTTP or command) passes for N consecutive intervals. | Health check configuration from manifest. | Same as provisioning. |
|
|
| `unhealthy` | Health check fails for N consecutive intervals. | Container continues running; backend may alert user. | Same as provisioning. |
|
|
| `stopping` | User requests stop; backend calls Docker stop. | Container enters `exited` state. | Labels remain but Traefik will remove route when container stops. |
|
|
| `stopped` | Container is fully stopped; backend updates status. | Container may be removed after a grace period in MVP. | Labels removed if container is removed. |
|
|
|
|
### 8.2 Error Handling and Rollback
|
|
|
|
- **Provisioning failure:** If Docker API returns an error, the backend deletes any partially created resources (volumes, containers), updates `tool_instance.status` to `stopped`, and returns a 500 with a sanitized error message.
|
|
- **Health check failure:** The container remains running. The backend records `unhealthy` status. User can manually stop and restart.
|
|
- **Subdomain collision:** If the generated subdomain already exists, the backend appends a short random suffix and retries once.
|
|
- **Secret injection failure:** If a required secret is missing, spawn is rejected with 400 before provisioning begins.
|
|
|
|
---
|
|
|
|
## 9. Docker Runtime Abstraction
|
|
|
|
### 9.1 RuntimeProvider Protocol
|
|
|
|
```python
|
|
from typing import Any
|
|
|
|
class RuntimeProvider(ABC):
|
|
@abstractmethod
|
|
async def spawn(
|
|
self,
|
|
manifest: ToolManifest,
|
|
project: Project,
|
|
config: SpawnConfig,
|
|
) -> ToolInstance: ...
|
|
|
|
@abstractmethod
|
|
async def stop(self, instance: ToolInstance) -> None: ...
|
|
|
|
@abstractmethod
|
|
async def health(self, instance: ToolInstance) -> HealthStatus: ...
|
|
```
|
|
|
|
### 9.2 AccessProvider Protocol
|
|
|
|
```python
|
|
from typing import Any
|
|
|
|
class AccessProvider(ABC):
|
|
@abstractmethod
|
|
def route(self, instance: ToolInstance, domain: str) -> RoutingConfig: ...
|
|
|
|
class RoutingConfig(BaseModel):
|
|
subdomain: str
|
|
entrypoint: str = "websecure"
|
|
cert_resolver: str = "letsencrypt"
|
|
middlewares: list[str] = Field(default_factory=list)
|
|
tls: bool = True
|
|
```
|
|
|
|
- **MVP adapter:** Traefik labels on Docker containers (`app/infrastructure/traefik_labels.py`).
|
|
- **Future adapters:** Cloudflare Tunnel, custom reverse proxy, etc.
|
|
|
|
### 9.3 Type Stubs
|
|
|
|
Minimal Pydantic stubs referenced by protocols above:
|
|
|
|
```python
|
|
class ToolManifest(BaseModel):
|
|
key: str
|
|
name: str
|
|
version: str = "1.0.0"
|
|
description: str | None = None
|
|
image: str
|
|
ports: list[Port]
|
|
workspace_mounts: list[Mount] = Field(default_factory=list)
|
|
config_mounts: list[Mount] = Field(default_factory=list)
|
|
env: dict[str, str] = Field(default_factory=dict)
|
|
secrets: list[str] = Field(default_factory=list)
|
|
health_check: HealthCheck | None = None
|
|
resource_limits: ResourceLimits | None = None
|
|
routing: Routing
|
|
bootstrap_commands: list[str] = Field(default_factory=list)
|
|
runtime_requirements: RuntimeRequirements | None = None
|
|
|
|
class SpawnConfig(BaseModel):
|
|
environment: dict[str, str] = Field(default_factory=dict)
|
|
secret_values: dict[str, str] = Field(default_factory=dict)
|
|
config_overrides: dict[str, Any] = Field(default_factory=dict)
|
|
|
|
class HealthStatus(BaseModel):
|
|
status: Literal["healthy", "unhealthy", "unknown"]
|
|
last_checked: datetime | None = None
|
|
message: str | None = None
|
|
```
|
|
|
|
### 9.4 MVP Adapter: Docker Compose Service Generation
|
|
|
|
The MVP runtime adapter generates a Docker Compose service definition dict and applies it via the Docker API or `docker compose` CLI.
|
|
|
|
### 9.5 Service Naming Convention
|
|
|
|
Service names must be DNS-friendly and unique:
|
|
```
|
|
{tool_key}-{project_slug}-{user_slug}-{instance_short_id}
|
|
```
|
|
|
|
- All lowercase.
|
|
- Non-alphanumeric characters replaced with `-`.
|
|
- Max 63 characters (Docker Compose service name limit).
|
|
- The `instance_short_id` is the first 8 characters of the `tool_instance.id` UUID.
|
|
|
|
Example: `code-server-myapp-alice-a1b2c3d4`
|
|
|
|
### 9.6 Network Attachment
|
|
|
|
Every tool container attaches to:
|
|
1. **Default app network** — internal communication between platform services.
|
|
2. **External Traefik network** — required for Traefik to discover and route to the container.
|
|
|
|
```yaml
|
|
networks:
|
|
- default
|
|
- traefik
|
|
```
|
|
|
|
### 9.7 Volume Mounts
|
|
|
|
| Mount Type | Source | Target | Notes |
|
|
|------------|--------|--------|-------|
|
|
| Workspace | `/data/projects/{project_id}/repo/` | Manifest-defined target | Read-write by default |
|
|
| Tool config (project) | `/data/projects/{project_id}/tool-configs/{tool_key}/` | Manifest-defined target | Read-write |
|
|
| Tool config (user) | `/data/users/{user_id}/tool-configs/{tool_key}/` | Manifest-defined target | Read-write |
|
|
| SSH key | In-memory secret file | `/root/.ssh/id_ed25519` or agent socket | Injected at spawn time |
|
|
|
|
Volumes are named Docker volumes or bind mounts depending on deployment configuration.
|
|
|
|
### 9.8 Resource Limits Mapping
|
|
|
|
Manifest `resource_limits` map directly to Docker Compose `deploy.resources`:
|
|
|
|
```yaml
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
cpus: "1.0"
|
|
memory: 1g
|
|
reservations:
|
|
cpus: "0.5"
|
|
memory: 512m
|
|
```
|
|
|
|
### 9.9 Container Isolation Assumptions
|
|
|
|
- **One container per tool instance.** No shared containers.
|
|
- **No privileged containers in MVP.** All tools run unprivileged.
|
|
- **No host network mode.** Containers use Docker bridge networks only.
|
|
- **Read-only root filesystem** is recommended but not enforced in MVP (future hardening).
|
|
|
|
---
|
|
|
|
## 10. Traefik Subdomain Routing Model
|
|
|
|
### 10.1 Subdomain Pattern Specification
|
|
|
|
Default pattern:
|
|
```
|
|
{tool}-{project}-{user}.{tool_domain}
|
|
```
|
|
|
|
Variable interpolation rules:
|
|
- `{tool}` → `tool_definition.key`
|
|
- `{project}` → `project.slug`
|
|
- `{user}` → User slug derived from `user.display_name` or `user.email` local-part, slugified
|
|
- `{tool_domain}` → `settings.tool_domain` (e.g., `tools.example.com`)
|
|
|
|
**Relationship to `ROOT_DOMAIN`:** `settings.tool_domain` is typically a subdomain of `settings.root_domain` (e.g., `tools.localhost` when `root_domain=localhost`). The backend `config.py` exposes both `root_domain` and `tool_subdomain_pattern` for flexibility; implementers should use `settings.tool_domain` for tool routing and `settings.root_domain` for platform routing.
|
|
|
|
Example:
|
|
```
|
|
https://opencode-myapp-alice.tools.example.com
|
|
https://code-server-myapp-alice.tools.example.com
|
|
```
|
|
|
|
**DNS limits:** Each label (dot-separated segment) must be ≤ 63 bytes. The total FQDN must be ≤ 253 bytes. If interpolation would exceed these limits, the backend truncates `project` and `user` segments preferentially and appends a hash suffix to maintain uniqueness.
|
|
|
|
### 10.2 Traefik Label Template for Tool Instances
|
|
|
|
```python
|
|
labels = {
|
|
"traefik.enable": "true",
|
|
"traefik.http.routers.{router_name}.rule": f"Host(`{subdomain}`)",
|
|
"traefik.http.routers.{router_name}.entrypoints": traefik_entrypoint,
|
|
"traefik.http.routers.{router_name}.tls.certresolver": cert_resolver,
|
|
"traefik.http.routers.{router_name}.tls": "true",
|
|
"traefik.http.services.{router_name}.loadbalancer.server.port": str(port),
|
|
"traefik.docker.network": network,
|
|
}
|
|
|
|
if middlewares:
|
|
labels["traefik.http.routers.{router_name}.middlewares"] = ",".join(middlewares)
|
|
```
|
|
|
|
Where:
|
|
- `router_name` = sanitized service name (alphanumeric + hyphens, max 64 chars)
|
|
- `subdomain` = fully interpolated subdomain string
|
|
- `traefik_entrypoint` = `websecure` (default) or `web`
|
|
- `cert_resolver` = `letsencrypt` (default) or custom resolver name
|
|
- `port` = primary port from manifest `ports`
|
|
- `network` = `traefik` (default external network name)
|
|
- `middlewares` = optional list of middleware names (e.g., `security-headers`)
|
|
|
|
### 10.3 Dynamic Label Generation at Spawn Time
|
|
|
|
1. Backend reads `tool_definition.manifest_data["routing"]`.
|
|
2. Backend calls `build_tool_subdomain(tool_name, project_slug, user_slug, tool_domain)`.
|
|
3. Backend calls `generate_traefik_labels(...)` to produce the label dict.
|
|
4. Labels are passed to the runtime adapter and applied to the container/service.
|
|
|
|
### 10.4 Network Requirement
|
|
|
|
Tool containers **must** attach to the external Traefik Docker network. If the network is missing, spawn fails with a clear error message.
|
|
|
|
### 10.5 HTTPS/TLS Assumptions
|
|
|
|
- **Default entrypoint:** `websecure` (HTTPS)
|
|
- **Cert resolver:** `letsencrypt` (default) or a custom Traefik resolver
|
|
- **TLS is enabled by default** for all tool instances.
|
|
- HTTP-only mode (`web` entrypoint, no TLS) is supported for local development via environment configuration.
|
|
|
|
### 10.6 Auth Proxy for Tool Instances
|
|
|
|
Spawned tools are protected behind the platform's authentication via Traefik forwardAuth middleware:
|
|
|
|
1. **Middleware Configuration:**
|
|
- Traefik forwards incoming requests to `/api/v1/auth/validate`
|
|
- The endpoint validates the Bearer token and returns 200 for authenticated users
|
|
- Unauthenticated requests receive 401 and are blocked
|
|
|
|
2. **Tool Configuration:**
|
|
- code-server built-in auth is disabled (`PASSWORD: ""`)
|
|
- OpenCode relies entirely on the platform auth layer
|
|
- Tools run on internal networks only, inaccessible directly
|
|
|
|
3. **Security Model:**
|
|
- Only platform-authenticated users can access spawned tools
|
|
- Each tool instance has its own subdomain with isolated routing
|
|
- No shared containers between users or projects
|
|
|
|
---
|
|
|
|
## 11. Storage Layout
|
|
|
|
### 11.1 Filesystem Paths
|
|
|
|
All paths are absolute inside the backend container or on the Docker host.
|
|
|
|
```
|
|
/data/
|
|
projects/
|
|
{project_id}/
|
|
repo/ # Cloned repository workspace
|
|
tool-configs/
|
|
{tool_definition_key}/ # Project-level tool configuration
|
|
users/
|
|
{user_id}/
|
|
tool-configs/
|
|
{tool_definition_key}/ # User-level tool configuration
|
|
global/
|
|
defaults/ # Global default configurations (future)
|
|
```
|
|
|
|
### 11.2 Exact Mount Paths
|
|
|
|
| Purpose | Host Path | Container Target | Notes |
|
|
|---------|-----------|------------------|-------|
|
|
| Repository workspace | `/data/projects/{project_id}/repo/` | Manifest `workspace_mounts[].target` | Persistent across restarts |
|
|
| Project tool config | `/data/projects/{project_id}/tool-configs/{tool_key}/` | Manifest `config_mounts[].target` | Persistent across restarts |
|
|
| User tool config | `/data/users/{user_id}/tool-configs/{tool_key}/` | Manifest `config_mounts[].target` | Persistent across restarts |
|
|
| SSH private key | In-memory temp file | `/root/.ssh/id_ed25519` | Created at spawn, deleted on stop |
|
|
|
|
### 11.3 Volume Naming Conventions
|
|
|
|
Named Docker volumes (used in Docker Compose):
|
|
- `postgres-data` — PostgreSQL data
|
|
- `api-data` — Backend data root (`/data`)
|
|
- `{project_id}-workspace` — Project workspace (future granularity)
|
|
|
|
For MVP, a single `api-data` volume is mounted at `/data` on the backend container. The backend manages subdirectories.
|
|
|
|
### 11.4 Persistence Guarantees
|
|
|
|
| Data | Survives Container Restart | Survives Full Stack Redeploy | Notes |
|
|
|------|---------------------------|------------------------------|-------|
|
|
| PostgreSQL data | Yes | Yes | Named volume `postgres-data` |
|
|
| Repository workspaces | Yes | Yes | Named volume `api-data` |
|
|
| Tool configs | Yes | Yes | Named volume `api-data` |
|
|
| Tool instance state (DB) | Yes | Yes | PostgreSQL |
|
|
| Running containers | No | No | Must be re-spawned after redeploy |
|
|
| In-memory secrets | No | No | Re-injected on re-spawn |
|
|
|
|
---
|
|
|
|
## 12. Authentik OIDC Auth Flow
|
|
|
|
### 12.1 Login Flow
|
|
|
|
1. User clicks "Sign In" on the frontend.
|
|
2. Frontend redirects browser to Authentik authorize endpoint:
|
|
```
|
|
GET {AUTHENTIK_ISSUER_URL}/authorize?
|
|
response_type=code
|
|
&client_id={AUTHENTIK_CLIENT_ID}
|
|
&redirect_uri={WEB_URL}/auth/callback
|
|
&scope=openid+email+profile
|
|
&state={csrf_state}
|
|
```
|
|
3. User authenticates with Authentik.
|
|
4. Authentik redirects back to `{WEB_URL}/auth/callback?code={auth_code}&state={csrf_state}`.
|
|
|
|
### 12.2 Callback Handling
|
|
|
|
**Backend-handled callback (recommended for MVP):**
|
|
1. The `redirect_uri` registered with Authentik points directly to a backend endpoint (e.g., `GET /api/v1/auth/callback`).
|
|
2. Authentik redirects the browser to:
|
|
```
|
|
GET {API_URL}/api/v1/auth/callback?code={auth_code}&state={csrf_state}
|
|
```
|
|
3. Backend validates `state` against session storage.
|
|
4. Backend exchanges `code` for tokens via Authentik token endpoint:
|
|
```
|
|
POST {AUTHENTIK_ISSUER_URL}/token
|
|
grant_type=authorization_code
|
|
code={code}
|
|
redirect_uri={API_URL}/api/v1/auth/callback
|
|
client_id={AUTHENTIK_CLIENT_ID}
|
|
client_secret={AUTHENTIK_CLIENT_SECRET}
|
|
```
|
|
5. Backend receives `id_token` and `access_token`.
|
|
6. Backend validates `id_token` via JWKS (see 12.3).
|
|
7. Backend creates/updates the `User` row from token claims.
|
|
8. Backend sets an **httpOnly cookie** containing a session token or the Authentik access token.
|
|
9. Backend returns an HTTP 302 redirect to the frontend dashboard (`{WEB_URL}/`).
|
|
|
|
**Alternative (not recommended):** The frontend could extract the code and POST it to the backend, receiving JSON with a `redirect_url`. The frontend then navigates via `window.location.href`. This pattern is acceptable only if the backend callback endpoint cannot be exposed as a browser-accessible URL.
|
|
|
|
**Alternative (frontend-handled):** The frontend exchanges the code directly. This requires exposing `client_secret` to the frontend and is **not recommended** for MVP.
|
|
|
|
### 12.3 Token Validation
|
|
|
|
1. Backend fetches the OIDC discovery document:
|
|
```
|
|
GET {AUTHENTIK_ISSUER_URL}/.well-known/openid-configuration
|
|
```
|
|
2. Backend extracts `jwks_uri` and fetches the JWKS.
|
|
3. Backend validates the `id_token` signature using `jwt.decode()` with:
|
|
- `algorithms=["RS256"]`
|
|
- `audience=AUTHENTIK_CLIENT_ID`
|
|
- `issuer=AUTHENTIK_ISSUER_URL`
|
|
4. If validation fails, return 401.
|
|
|
|
### 12.4 User Identity Mapping
|
|
|
|
| Token Claim | User Column | Fallback |
|
|
|-------------|-------------|----------|
|
|
| `sub` | `authentik_sub` | — |
|
|
| `email` | `email` | — |
|
|
| `name` | `display_name` | `email` local-part |
|
|
| `preferred_username` | — | Future use |
|
|
|
|
- If `sub` is not found in the database, a new `User` row is created.
|
|
- If `sub` exists, `email` and `display_name` are updated from the token claims.
|
|
|
|
### 12.5 Session/Token Storage Strategy
|
|
|
|
**Recommended (MVP):** httpOnly cookie
|
|
- Backend sets `Set-Cookie: session={encrypted_token}; HttpOnly; Secure; SameSite=Lax; Path=/`
|
|
- Frontend does not touch the token directly.
|
|
- Backend reads the cookie on every request and validates it.
|
|
|
|
**Not recommended:** localStorage
|
|
- Vulnerable to XSS extraction.
|
|
- Only acceptable if the token has very short expiry and the frontend implements PKCE without a backend callback handler.
|
|
|
|
### 12.6 Logout Flow
|
|
|
|
1. User clicks "Sign Out".
|
|
2. Frontend calls `POST /api/v1/auth/logout`.
|
|
3. Backend clears the httpOnly cookie.
|
|
4. Backend optionally redirects to Authentik end-session endpoint:
|
|
```
|
|
GET {AUTHENTIK_ISSUER_URL}/end-session?
|
|
id_token_hint={id_token}
|
|
&post_logout_redirect_uri={WEB_URL}
|
|
```
|
|
5. Frontend redirects to the login page.
|
|
|
|
### 12.7 Dev Bypass Mechanism
|
|
|
|
For local development, set `AUTH_DEV_BYPASS=true` and `DEBUG=true`.
|
|
|
|
- When enabled, `get_current_user` returns a fixed development user (`authentik_sub="dev-user"`, `email="dev@localhost"`) without requiring a token.
|
|
- **This bypasses ALL authentication and must never be enabled in production.**
|
|
- The backend must reject requests when `settings.auth_dev_bypass` is `True` but `settings.debug` is `False`.
|
|
|
|
---
|
|
|
|
## 13. Security Considerations
|
|
|
|
### 13.1 Threat Model for MVP
|
|
|
|
**What we are protecting against:**
|
|
- Unauthorized access to user projects, repositories, and running tools
|
|
- Secret leakage (env vars, credentials, SSH keys)
|
|
- Container escape or privilege escalation
|
|
- Cross-user data access (one user reading another user's projects)
|
|
- Path traversal via config mount paths
|
|
|
|
**What we are accepting as risk (MVP limitations):**
|
|
- No rate limiting (vulnerable to brute-force and DoS)
|
|
- No audit logging (cannot trace who did what after the fact)
|
|
- No automatic backup or disaster recovery
|
|
- No Web Application Firewall (WAF) or intrusion detection
|
|
- Single-tenant deployment (no hard multi-tenant isolation)
|
|
- Manual secret rotation only
|
|
- No automatic provider token revocation on delete
|
|
- No container image vulnerability scanning
|
|
|
|
### 13.2 Secret Encryption at Rest
|
|
|
|
- Algorithm: Fernet (symmetric encryption) from `cryptography.fernet`
|
|
- Key derivation: SHA-256 hash of `SECRET_ENCRYPTION_KEY`, base64-urlsafe encoded
|
|
- All secrets stored in the `secret` table as `encrypted_value` (ciphertext)
|
|
- No plaintext secrets in database, logs, or API responses (except the trusted API boundary decrypting for authorized users)
|
|
|
|
### 13.3 Secret Injection at Runtime
|
|
|
|
| Secret Type | Injection Method | When to Use |
|
|
|-------------|------------------|-------------|
|
|
| Generic secrets | Environment variables | Simple key-value pairs |
|
|
| SSH private keys | Mounted file (`/run/secrets/...` or `~/.ssh/`) | Key material that tools expect as files |
|
|
| Access tokens | Environment variables | API tokens consumed by tools |
|
|
| Multi-line secrets | Mounted file | Certificates, PEM blocks |
|
|
|
|
Secret files are written to temporary in-memory files (tmpfs if available) and mounted read-only into containers.
|
|
|
|
### 13.4 SSH Key Isolation
|
|
|
|
- **Per-repository only.** SSH keys are never shared across projects or repositories.
|
|
- Private keys are encrypted at rest and decrypted only at spawn time.
|
|
- Public keys are registered as deploy keys (read-only where possible) on the Git provider.
|
|
- On repository disconnect, the deploy key is deleted from the provider and the private key is marked `revoked`.
|
|
|
|
### 13.5 Container Isolation
|
|
|
|
- Each tool instance runs in its own container.
|
|
- No privileged containers in MVP.
|
|
- Resource limits are enforced via Docker Compose `deploy.resources`.
|
|
- Containers do not share network namespaces.
|
|
- Workspace mounts are scoped to `project_id`.
|
|
|
|
### 13.6 Network Isolation
|
|
|
|
- **Internal app network:** Platform services (API, web, DB) communicate on the default Compose network. This network is not attached to Traefik.
|
|
- **External Traefik network:** Only services that need external routing (web, API, tool containers) attach to this network.
|
|
- No direct container port exposure in production (no `ports:` mappings in `docker-compose.prod.yml`).
|
|
|
|
### 13.7 AuthZ Model
|
|
|
|
- **Ownership-based:** Users own projects. All resources under a project inherit the owner's access rights.
|
|
- **No shared projects in MVP.** A user cannot access another user's projects, repositories, tool instances, configs, or secrets.
|
|
- **Global config:** Readable by all authenticated users; write restricted to future admin role (for MVP, either allow all authenticated users or reject with 403 — implementer must choose and document).
|
|
|
|
### 13.8 Input Validation
|
|
|
|
- **Manifest validation:** All tool manifests are validated against the Pydantic schema before registration. Invalid manifests are rejected with 422.
|
|
- **User input sanitization:** All user-provided strings (project names, slugs, descriptions) are sanitized for HTML injection and path traversal.
|
|
- **Slug validation:** `slug` must match `^[a-z0-9-]+$`, max 255 chars.
|
|
- **Path sanitization:** All mount paths are resolved to absolute paths and verified to be within `/data/projects/{project_id}/` or `/data/users/{user_id}/`.
|
|
|
|
---
|
|
|
|
## 14. Extension Points
|
|
|
|
### 14.1 New Git Provider
|
|
|
|
1. Create a new module under `app/git/providers/` (e.g., `github.py`).
|
|
2. Implement `GitProvider` ABC with all abstract methods.
|
|
3. Register in the factory:
|
|
```python
|
|
def get_git_provider(kind: ProviderKind) -> GitProvider:
|
|
registry = {
|
|
ProviderKind.github: GitHubProvider,
|
|
ProviderKind.gitlab: GitLabProvider,
|
|
# ...
|
|
}
|
|
return registry[kind]()
|
|
```
|
|
4. Add tests in `tests/test_git_provider.py` pattern.
|
|
|
|
### 14.2 New Tool
|
|
|
|
1. Write a manifest YAML file conforming to the schema in Section 7.
|
|
2. Register the manifest in the tool registry (`app/tools/registry.py` or equivalent).
|
|
3. No backend code changes required for standard containers.
|
|
|
|
### 14.3 New Runtime
|
|
|
|
1. Create a new module under `app/runtime/` (e.g., `kubernetes.py`).
|
|
2. Implement `RuntimeProvider` ABC.
|
|
3. Configure the backend to use the new adapter via environment variable or settings.
|
|
|
|
### 14.4 New Access Provider
|
|
|
|
1. Create a new module under `app/access/` (e.g., `cloudflare_tunnel.py`).
|
|
2. Implement `AccessProvider` ABC.
|
|
3. Configure the backend to use the new adapter.
|
|
|
|
### 14.5 Teams/Organizations
|
|
|
|
1. Add `Organization` and `ProjectMember` entities via Alembic migration.
|
|
2. Update `Project` to optionally reference `organization_id` instead of `owner_id`.
|
|
3. Add `ProjectMember` join table with roles (`owner`, `admin`, `member`).
|
|
4. Update authZ layer to check `ProjectMember` permissions in addition to ownership.
|
|
|
|
---
|
|
|
|
## 15. Environment Assumptions
|
|
|
|
| Variable | Purpose | Required | Safe Local Default |
|
|
|----------|---------|----------|-------------------|
|
|
| `APP_NAME` | Application display name | No | `Headquarter` |
|
|
| `ROOT_DOMAIN` | Base domain for platform | No | `localhost` |
|
|
| `TOOL_DOMAIN` | Subdomain suffix for tools | No | `tools.localhost` |
|
|
| `API_URL` | Public backend URL | No | `http://localhost:8000` |
|
|
| `WEB_URL` | Public frontend URL | No | `http://localhost:5173` |
|
|
| `DATABASE_URL` | PostgreSQL connection string | Yes (production) | `postgresql://postgres:postgres@localhost:5432/headquarter` |
|
|
| `AUTHENTIK_ISSUER_URL` | Authentik OIDC issuer | Yes (production) | Empty string (disables OIDC) |
|
|
| `AUTHENTIK_CLIENT_ID` | Authentik OIDC client ID | Yes (production) | Empty string |
|
|
| `AUTHENTIK_CLIENT_SECRET` | Authentik OIDC client secret | Yes (production) | Empty string |
|
|
| `TRAEFIK_NETWORK` | External Docker network name | No | `traefik` |
|
|
| `TRAEFIK_ENTRYPOINT` | Traefik entrypoint name | No | `websecure` |
|
|
| `TRAEFIK_CERT_RESOLVER` | Traefik certificate resolver | No | `letsencrypt` |
|
|
| `SECRET_ENCRYPTION_KEY` | Fernet encryption key | Yes (production) | `change-me-in-production` |
|
|
| `AUTH_DEV_BYPASS` | Skip auth in debug mode | No | `false` |
|
|
| `ACCESS_TOKEN_EXPIRE_MINUTES` | Local token expiry | No | `60` |
|
|
|
|
**Security note:** `AUTH_DEV_BYPASS` must be `false` in all production-like environments. `SECRET_ENCRYPTION_KEY` must be a cryptographically random string of at least 32 bytes in production.
|
|
|
|
---
|
|
|
|
## 16. Technology Boundaries
|
|
|
|
| Layer | Choice | Migration Path |
|
|
|-------|--------|----------------|
|
|
| Frontend | React + Vite | Next.js, Vue, etc. if needed |
|
|
| Backend | FastAPI | Any ASGI framework |
|
|
| Database | PostgreSQL | Managed Postgres, CockroachDB |
|
|
| Runtime | Docker Compose | Kubernetes, Nomad |
|
|
| Access | Traefik | Cloudflare Tunnel, custom proxy |
|
|
| Auth | Authentik OIDC | Any OIDC provider (Keycloak, Okta, etc.) |
|
|
|
|
---
|
|
|
|
## 17. Acceptance Criteria for Architecture Compliance
|
|
|
|
Any implementation task must:
|
|
|
|
1. Respect provider interfaces (no hardcoded GitHub/Traefik logic in core orchestration).
|
|
2. Keep secrets out of committed files and plaintext logs.
|
|
3. Use environment variables for deployment-specific values.
|
|
4. Leave schema room for multi-user teams without rewriting ownership models.
|
|
5. Support adding a new tool via manifest + registry entry alone (no new backend code for standard containers).
|
|
6. Use UUID primary keys for all new entities.
|
|
7. Validate all user inputs at API boundaries before database insertion.
|
|
8. Encrypt secrets at rest using Fernet via `SECRET_ENCRYPTION_KEY`.
|
|
9. Generate per-repository SSH keys; never share credentials across projects.
|
|
10. Attach spawned tool containers to the external Traefik network.
|
|
11. Use subdomain-based routing for all tool instances; avoid path-based routing.
|
|
12. Document any deviation from this architecture in the task's delivery documentation.
|
|
|
|
---
|
|
|
|
## 18. Deferred Decisions / Open Questions
|
|
|
|
1. **User slug format:** The subdomain pattern uses `{user}` but the `user` table has no `slug` column. Should we derive the slug from `display_name` or `email`, or add a dedicated `slug` column? This affects URL stability and DNS length limits.
|
|
2. **Global config write permissions:** In MVP, global config writes are either allowed for all authenticated users or rejected with 403. A stakeholder decision is needed on whether MVP needs an admin role.
|
|
3. **Subdomain truncation strategy:** DNS labels are limited to 63 bytes. We need a deterministic truncation/hashing strategy for long project or user names.
|
|
4. **Container image trust:** Should the platform restrict tool images to an allow-list or registry in production? Deferred to post-MVP security hardening.
|
|
5. **Background task queue:** Tasks like health polling, log aggregation, and async Git operations may need a task queue (Celery, RQ, or similar). Not in MVP.
|
|
6. **Multi-tenancy:** MVP is single-tenant deployment. Multi-tenant routing and isolation are future concerns.
|
|
7. **High availability:** No replicas or load balancing in MVP.
|
|
8. **Backup strategy:** Out of MVP scope; rely on host-level volume backups.
|
|
9. **Rate limiting:** Not in MVP; add at Traefik or API gateway layer later.
|