headquarter/docs/architecture.md

Headquarter Architecture

Canonical architecture specification for the hosted workspace and tool-orchestration platform. Decisions in this document override ad-hoc choices in implementation tasks.

1. Overview & Goals

Headquarter is a hosted control plane where authenticated users create projects, connect Git repositories, and spawn containerized tools (RunFusion, code-server, and future tools). The platform is manifest-driven and provider-abstracted so new tools, Git providers, runtimes, and access providers can be added without rewriting core orchestration logic.

MVP scope: Single-user projects, Authentik OIDC auth, Docker runtime, Traefik subdomain routing, Portainer-managed Docker Compose deployment.

2. Domain Model

User ──< Project ──< Repository
                └──< ToolInstance

Tool (manifest-driven, global registry)

• User: Authentik-managed identity. MVP assumes individual users; schema leaves room for teams/shared projects later.
• Project: Owned by a user. Contains repositories and spawned tool instances.
• Repository: Git-backed workspace. Clone, fetch, and push via provider-independent adapters.
• Tool: Manifest-driven definition (image, ports, mounts, env, health checks, routing rules). Defined in FN-003.
• ToolInstance: A running container spawned from a Tool manifest for a specific project. Receives workspace mounts, config mounts, secrets, and Traefik routing labels.

3. Provider Interfaces

The backend must define provider contracts before implementing any concrete adapter.

3.1 GitProvider

Architecture divergence note (FN-011): The original spec defined a single GitProvider(Protocol) with clone/fetch/push methods. The implementation intentionally splits this responsibility into two abstractions:

• GitProvider (app/git/provider.py) — provider API adapter for remote operations (validate_connection, list_repositories, create_deploy_key, etc.).
• GitOperations (app/git/operations.py) — local Git subprocess interface (clone, fetch, push, get_status).

This separation keeps provider-specific API logic distinct from local Git CLI orchestration.

class GitProvider(Protocol):
    def validate_connection(self, repo_url: str, credential_id: str) -> ConnectionStatus: ...
    def list_repositories(self, credential_id: str) -> list[dict[str, Any]]: ...
    def create_deploy_key(self, repo_url: str, public_key: str) -> str: ...
    def delete_deploy_key(self, repo_url: str, deploy_key_id: str) -> None: ...
    def get_default_branch(self, repo_url: str, credential_id: str) -> str: ...

• Adapters: GitHub, GitLab, Gitea, Forgejo, etc.
• Credentials: generated SSH keys (per-repository) or access tokens.
• SSH keys must be scoped per repository connection for clean revocation.

class GitOperations(Protocol):
    def clone(self, repo_url: str, dest: Path, credential_id: str) -> None: ...
    def fetch(self, repo_path: Path, credential_id: str) -> None: ...
    def push(self, repo_path: Path, credential_id: str) -> None: ...
    def get_status(self, repo_path: Path) -> dict[str, Any]: ...

3.2 RuntimeProvider

class RuntimeProvider(Protocol):
    def spawn(self, manifest: ToolManifest, project: Project, config: SpawnConfig) -> ToolInstance: ...
    def stop(self, instance: ToolInstance) -> None: ...
    def health(self, instance: ToolInstance) -> HealthStatus: ...

• MVP adapter: Docker Compose service generation + Docker API.
• Future adapters: Kubernetes, Nomad, etc.

3.3 AccessProvider

class AccessProvider(Protocol):
    def route(self, instance: ToolInstance, domain: str) -> RoutingConfig: ...

• MVP adapter: Traefik labels on Docker containers.
• Future adapter: Cloudflare Tunnel, etc.

4. Deployment Architecture

4.1 MVP Target

• Orchestration: Portainer-managed Docker Compose stack.
• Reverse Proxy: Existing Traefik instance (external to the app stack).
• Network: Shared Traefik Docker network; app stack attaches to it.
• Certificate Resolution: Let's Encrypt or internal CA via Traefik cert resolver.

4.2 Subdomain Routing

Path-based routing is avoided because many tools expect to run at /.

Pattern:

https://{tool}-{project}-{user}.{tool_domain}

Examples:

https://runfusion-myapp-alice.tools.example.com
https://code-myapp-alice.tools.example.com

4.3 Compose Skeleton

• docker-compose.yml: local development (backend, frontend, PostgreSQL).
• docker-compose.traefik.yml: deployment overlay with Traefik labels and external network.
• Environment-driven; no secrets committed to repository.

5. Security Boundaries

5.1 Authentication

• Authentik OIDC for user login.
• FastAPI backend validates JWT/id tokens at API boundaries.
• Frontend stores tokens securely (httpOnly cookie or secure storage pattern).

5.2 Secrets

• Never treat secrets as plaintext config.
• Support encrypted storage at rest and runtime injection as:
  • Environment variables
  • Mounted secret files
• Encryption key is an environment secret (SECRET_ENCRYPTION_KEY).

5.3 SSH Keys

• Generated per repository connection.
• Stored encrypted.
• Injected into tool containers at runtime for Git operations.

5.4 Container Isolation

• Each tool instance runs in its own container.
• Resource limits declared in tool manifest.
• Workspace and config mounts are scoped to user/project.

6. Data & Storage

6.1 Database

• PostgreSQL for relational data (users, projects, repositories, tool instances, manifests).
• Schema migrations managed by backend (Alembic or equivalent).

6.2 Filesystem Layout (Conceptual)

/data/
  users/{userId}/tool-configs/{toolId}/
  projects/{projectId}/repo/
  projects/{projectId}/tool-configs/{toolId}/

• Repository workspace storage: Docker volumes or local bind mounts.
• Tool config storage: persistent host mounts, separate from repository workspaces.
• Config scopes: global default → user-level → project-level → tool-instance override.

7. Tool Manifest & Orchestration

Tools are defined by manifest files validated against the canonical Pydantic v2 schema. The full schema reference, validation rules, and extension guide are documented in docs/tool-manifest-spec.md.

Summary: Each manifest declares the runtime image, ports, workspace/config mounts, environment variables, secrets, health checks, resource limits, and Traefik routing rules. The orchestration backend reads these manifests and generates Docker Compose service definitions, Traefik labels, and runtime volume/secret bindings.

Built-in manifests for RunFusion and code-server are shipped in apps/api/app/tools/manifests/ and loaded automatically on API startup. New standard-container tools can be added by creating a YAML manifest—no backend code changes are required.

8. MVP Phases

Phase	Task	Deliverable
Foundation	FN-002	Monorepo scaffold, build/test/lint pipelines
Registry	FN-003	Manifest schema, RunFusion and code-server manifests
Backend	FN-004	FastAPI app, domain models, API endpoints, DB migrations
Frontend	FN-005	Auth-ready shell, navigation, placeholder screens
Deployment	FN-006	Docker Compose overlays, Traefik labels, Portainer config
Git Model	FN-007	Provider interface, SSH key generation, credential storage
RunFusion POC	FN-008	Executable environment proof of concept
Secrets & Config	FN-009	Encrypted secrets, persistent config mounts
code-server Spawn	FN-010	code-server manifest, spawn script, runtime integration

9. Extension Points

• New Git providers: Implement GitProvider protocol.
• New tools: Add a manifest to the registry (no code changes required for standard containers).
• New runtimes: Implement RuntimeProvider protocol.
• New access providers: Implement AccessProvider protocol.
• Teams/organizations: Add Organization and ProjectMember entities later.

10. Environment Assumptions

Required environment variables (no defaults in production):

APP_NAME=
ROOT_DOMAIN=
TOOL_DOMAIN=
API_URL=
TRAEFIK_NETWORK=
TRAEFIK_ENTRYPOINT=
TRAEFIK_CERT_RESOLVER=
AUTHENTIK_ISSUER_URL=
AUTHENTIK_CLIENT_ID=
AUTHENTIK_CLIENT_SECRET=
DATABASE_URL=
SECRET_ENCRYPTION_KEY=

Local development uses .env.example and safe defaults.

11. Technology Boundaries

Layer	Choice	Migration Path
Frontend	React + Vite	Next.js, Vue, etc. if needed
Backend	FastAPI	Any ASGI framework
Database	PostgreSQL	Managed Postgres, CockroachDB
Runtime	Docker Compose	Kubernetes, Nomad
Access	Traefik	Cloudflare Tunnel, custom proxy
Auth	Authentik OIDC	Any OIDC provider

12. Acceptance Criteria for Architecture Compliance

Any implementation task must:

1. Respect provider interfaces (no hardcoded GitHub/Traefik logic in core orchestration).
2. Keep secrets out of committed files and plaintext logs.
3. Use environment variables for deployment-specific values.
4. Leave schema room for multi-user teams without rewriting ownership models.
5. Support adding a new tool via manifest + registry entry alone (no new backend code for standard containers).

13. Deferred Decisions

• Multi-tenancy: MVP is single-tenant deployment. Multi-tenant routing and isolation are future concerns.
• High availability: No replicas or load balancing in MVP.
• Backup strategy: Out of MVP scope; rely on host-level volume backups.
• Rate limiting: Not in MVP; add at Traefik or API gateway layer later.