headquarter/docs/architecture.md

# Headquarter Architecture

> Canonical architecture specification for the hosted workspace and tool-orchestration platform.
> Decisions in this document override ad-hoc choices in implementation tasks.

## 1. Overview & Goals

Headquarter is a hosted control plane where authenticated users create projects, connect Git repositories, and spawn containerized tools (RunFusion, code-server, and future tools). The platform is manifest-driven and provider-abstracted so new tools, Git providers, runtimes, and access providers can be added without rewriting core orchestration logic.

**MVP scope:** Single-user projects, Authentik OIDC auth, Docker runtime, Traefik subdomain routing, Portainer-managed Docker Compose deployment.

## 2. Domain Model

```text
User ──< Project ──< Repository
                └──< ToolInstance

Tool (manifest-driven, global registry)
```

- **User:** Authentik-managed identity. MVP assumes individual users; schema leaves room for teams/shared projects later.
- **Project:** Owned by a user. Contains repositories and spawned tool instances.
- **Repository:** Git-backed workspace. Clone, fetch, and push via provider-independent adapters.
- **Tool:** Manifest-driven definition (image, ports, mounts, env, health checks, routing rules). Defined in FN-003.
- **ToolInstance:** A running container spawned from a Tool manifest for a specific project. Receives workspace mounts, config mounts, secrets, and Traefik routing labels.

## 3. Provider Interfaces

The backend must define provider contracts before implementing any concrete adapter.

### 3.1 GitProvider

> **Architecture divergence note (FN-011):** The original spec defined a single
> `GitProvider(Protocol)` with `clone/fetch/push` methods. The implementation
> intentionally splits this responsibility into two abstractions:
>
> - `GitProvider` (`app/git/provider.py`) — provider API adapter for remote
>   operations (`validate_connection`, `list_repositories`, `create_deploy_key`,
>   etc.).
> - `GitOperations` (`app/git/operations.py`) — local Git subprocess interface
>   (`clone`, `fetch`, `push`, `get_status`).
>
> This separation keeps provider-specific API logic distinct from local Git CLI
> orchestration.

```python
class GitProvider(Protocol):
    def validate_connection(self, repo_url: str, credential_id: str) -> ConnectionStatus: ...
    def list_repositories(self, credential_id: str) -> list[dict[str, Any]]: ...
    def create_deploy_key(self, repo_url: str, public_key: str) -> str: ...
    def delete_deploy_key(self, repo_url: str, deploy_key_id: str) -> None: ...
    def get_default_branch(self, repo_url: str, credential_id: str) -> str: ...
```

- Adapters: GitHub, GitLab, Gitea, Forgejo, etc.
- Credentials: generated SSH keys (per-repository) or access tokens.
- SSH keys must be scoped per repository connection for clean revocation.

```python
class GitOperations(Protocol):
    def clone(self, repo_url: str, dest: Path, credential_id: str) -> None: ...
    def fetch(self, repo_path: Path, credential_id: str) -> None: ...
    def push(self, repo_path: Path, credential_id: str) -> None: ...
    def get_status(self, repo_path: Path) -> dict[str, Any]: ...
```

### 3.2 RuntimeProvider

```python
class RuntimeProvider(Protocol):
    def spawn(self, manifest: ToolManifest, project: Project, config: SpawnConfig) -> ToolInstance: ...
    def stop(self, instance: ToolInstance) -> None: ...
    def health(self, instance: ToolInstance) -> HealthStatus: ...
```

- MVP adapter: Docker Compose service generation + Docker API.
- Future adapters: Kubernetes, Nomad, etc.

### 3.3 AccessProvider

```python
class AccessProvider(Protocol):
    def route(self, instance: ToolInstance, domain: str) -> RoutingConfig: ...
```

- MVP adapter: Traefik labels on Docker containers.
- Future adapter: Cloudflare Tunnel, etc.

## 4. Deployment Architecture

### 4.1 MVP Target

- **Orchestration:** Portainer-managed Docker Compose stack.
- **Reverse Proxy:** Existing Traefik instance (external to the app stack).
- **Network:** Shared Traefik Docker network; app stack attaches to it.
- **Certificate Resolution:** Let's Encrypt or internal CA via Traefik cert resolver.

### 4.2 Subdomain Routing

Path-based routing is avoided because many tools expect to run at `/`.

Pattern:
```
https://{tool}-{project}-{user}.{tool_domain}
```

Examples:
```
https://runfusion-myapp-alice.tools.example.com
https://code-myapp-alice.tools.example.com
```

### 4.3 Compose Skeleton

- `docker-compose.yml`: local development (backend, frontend, PostgreSQL).
- `docker-compose.traefik.yml`: deployment overlay with Traefik labels and external network.
- Environment-driven; no secrets committed to repository.

## 5. Security Boundaries

### 5.1 Authentication

- Authentik OIDC for user login.
- FastAPI backend validates JWT/id tokens at API boundaries.
- Frontend stores tokens securely (httpOnly cookie or secure storage pattern).

### 5.2 Secrets

- Never treat secrets as plaintext config.
- Support encrypted storage at rest and runtime injection as:
  - Environment variables
  - Mounted secret files
- Encryption key is an environment secret (`SECRET_ENCRYPTION_KEY`).

### 5.3 SSH Keys

- Generated per repository connection.
- Stored encrypted.
- Injected into tool containers at runtime for Git operations.

### 5.4 Container Isolation

- Each tool instance runs in its own container.
- Resource limits declared in tool manifest.
- Workspace and config mounts are scoped to user/project.

## 6. Data & Storage

### 6.1 Database

- PostgreSQL for relational data (users, projects, repositories, tool instances, manifests).
- Schema migrations managed by backend (Alembic or equivalent).

### 6.2 Filesystem Layout (Conceptual)

```text
/data/
  users/{userId}/tool-configs/{toolId}/
  projects/{projectId}/repo/
  projects/{projectId}/tool-configs/{toolId}/
```

- Repository workspace storage: Docker volumes or local bind mounts.
- Tool config storage: persistent host mounts, separate from repository workspaces.
- Config scopes: global default → user-level → project-level → tool-instance override.

## 7. Tool Manifest & Orchestration

Tools are defined by manifest files validated against the canonical Pydantic v2 schema.
The full schema reference, validation rules, and extension guide are documented in
[`docs/tool-manifest-spec.md`](./tool-manifest-spec.md).

**Summary:** Each manifest declares the runtime image, ports, workspace/config mounts,
environment variables, secrets, health checks, resource limits, and Traefik routing rules.
The orchestration backend reads these manifests and generates Docker Compose service
definitions, Traefik labels, and runtime volume/secret bindings.

Built-in manifests for RunFusion and code-server are shipped in
`apps/api/app/tools/manifests/` and loaded automatically on API startup.
New standard-container tools can be added by creating a YAML manifest—no backend code
changes are required.

## 8. MVP Phases

| Phase | Task | Deliverable |
|-------|------|-------------|
| Foundation | FN-002 | Monorepo scaffold, build/test/lint pipelines |
| Registry | FN-003 | Manifest schema, RunFusion and code-server manifests |
| Backend | FN-004 | FastAPI app, domain models, API endpoints, DB migrations |
| Frontend | FN-005 | Auth-ready shell, navigation, placeholder screens |
| Deployment | FN-006 | Docker Compose overlays, Traefik labels, Portainer config |
| Git Model | FN-007 | Provider interface, SSH key generation, credential storage |
| RunFusion POC | FN-008 | Executable environment proof of concept |
| Secrets & Config | FN-009 | Encrypted secrets, persistent config mounts |
| code-server Spawn | FN-010 | code-server manifest, spawn script, runtime integration |

## 9. Extension Points

- **New Git providers:** Implement `GitProvider` protocol.
- **New tools:** Add a manifest to the registry (no code changes required for standard containers).
- **New runtimes:** Implement `RuntimeProvider` protocol.
- **New access providers:** Implement `AccessProvider` protocol.
- **Teams/organizations:** Add `Organization` and `ProjectMember` entities later.

## 10. Environment Assumptions

Required environment variables (no defaults in production):

```env
APP_NAME=
ROOT_DOMAIN=
TOOL_DOMAIN=
API_URL=
TRAEFIK_NETWORK=
TRAEFIK_ENTRYPOINT=
TRAEFIK_CERT_RESOLVER=
AUTHENTIK_ISSUER_URL=
AUTHENTIK_CLIENT_ID=
AUTHENTIK_CLIENT_SECRET=
DATABASE_URL=
SECRET_ENCRYPTION_KEY=
```

Local development uses `.env.example` and safe defaults.

## 11. Technology Boundaries

| Layer | Choice | Migration Path |
|-------|--------|----------------|
| Frontend | React + Vite | Next.js, Vue, etc. if needed |
| Backend | FastAPI | Any ASGI framework |
| Database | PostgreSQL | Managed Postgres, CockroachDB |
| Runtime | Docker Compose | Kubernetes, Nomad |
| Access | Traefik | Cloudflare Tunnel, custom proxy |
| Auth | Authentik OIDC | Any OIDC provider |

## 12. Acceptance Criteria for Architecture Compliance

Any implementation task must:

1. Respect provider interfaces (no hardcoded GitHub/Traefik logic in core orchestration).
2. Keep secrets out of committed files and plaintext logs.
3. Use environment variables for deployment-specific values.
4. Leave schema room for multi-user teams without rewriting ownership models.
5. Support adding a new tool via manifest + registry entry alone (no new backend code for standard containers).

## 13. Deferred Decisions

- **Multi-tenancy:** MVP is single-tenant deployment. Multi-tenant routing and isolation are future concerns.
- **High availability:** No replicas or load balancing in MVP.
- **Backup strategy:** Out of MVP scope; rely on host-level volume backups.
- **Rate limiting:** Not in MVP; add at Traefik or API gateway layer later.