diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..eaee669 --- /dev/null +++ b/.env.example @@ -0,0 +1,28 @@ +# App identity +APP_NAME=Headquarter +ROOT_DOMAIN=localhost +TOOL_DOMAIN=tools.localhost + +# API / Web URLs +API_URL=http://localhost:8000 +WEB_URL=http://localhost:5173 + +# Database (local development) +POSTGRES_USER=postgres +POSTGRES_PASSWORD=postgres +POSTGRES_DB=headquarter +DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5432/${POSTGRES_DB} + +# Authentik OIDC placeholders (wire in FN-004) +AUTHENTIK_ISSUER_URL=https://auth.example.com/application/o/headquarter/ +AUTHENTIK_CLIENT_ID=your-client-id +AUTHENTIK_CLIENT_SECRET=your-client-secret + +# Traefik / deployment placeholders (wire in FN-006) +TRAEFIK_NETWORK=traefik +TRAEFIK_ENTRYPOINT=websecure +TRAEFIK_CERT_RESOLVER=letsencrypt +TOOL_SUBDOMAIN_PATTERN={tool}-{project}-{user}.tools.${ROOT_DOMAIN} + +# Secrets (generate strong random values for production) +SECRET_ENCRYPTION_KEY=change-me-in-production diff --git a/apps/api/headquarter_api.egg-info/PKG-INFO b/apps/api/headquarter_api.egg-info/PKG-INFO deleted file mode 100644 index ff539f4..0000000 --- a/apps/api/headquarter_api.egg-info/PKG-INFO +++ /dev/null @@ -1,13 +0,0 @@ -Metadata-Version: 2.4 -Name: headquarter-api -Version: 0.0.1 -Summary: Headquarter FastAPI backend -Requires-Python: >=3.11 -Requires-Dist: fastapi>=0.115.0 -Requires-Dist: uvicorn[standard]>=0.34.0 -Requires-Dist: pydantic-settings>=2.8.0 -Provides-Extra: dev -Requires-Dist: pytest>=8.3.0; extra == "dev" -Requires-Dist: httpx>=0.28.0; extra == "dev" -Requires-Dist: ruff>=0.11.0; extra == "dev" -Requires-Dist: mypy>=1.15.0; extra == "dev" diff --git a/apps/api/headquarter_api.egg-info/SOURCES.txt b/apps/api/headquarter_api.egg-info/SOURCES.txt deleted file mode 100644 index 7591d30..0000000 --- a/apps/api/headquarter_api.egg-info/SOURCES.txt +++ /dev/null @@ -1,10 +0,0 @@ -pyproject.toml -app/__init__.py -app/config.py -app/main.py -headquarter_api.egg-info/PKG-INFO -headquarter_api.egg-info/SOURCES.txt -headquarter_api.egg-info/dependency_links.txt -headquarter_api.egg-info/requires.txt -headquarter_api.egg-info/top_level.txt -tests/test_health.py \ No newline at end of file diff --git a/apps/api/headquarter_api.egg-info/dependency_links.txt b/apps/api/headquarter_api.egg-info/dependency_links.txt deleted file mode 100644 index 8b13789..0000000 --- a/apps/api/headquarter_api.egg-info/dependency_links.txt +++ /dev/null @@ -1 +0,0 @@ - diff --git a/apps/api/headquarter_api.egg-info/requires.txt b/apps/api/headquarter_api.egg-info/requires.txt deleted file mode 100644 index 723d1de..0000000 --- a/apps/api/headquarter_api.egg-info/requires.txt +++ /dev/null @@ -1,9 +0,0 @@ -fastapi>=0.115.0 -uvicorn[standard]>=0.34.0 -pydantic-settings>=2.8.0 - -[dev] -pytest>=8.3.0 -httpx>=0.28.0 -ruff>=0.11.0 -mypy>=1.15.0 diff --git a/apps/api/headquarter_api.egg-info/top_level.txt b/apps/api/headquarter_api.egg-info/top_level.txt deleted file mode 100644 index b80f0bd..0000000 --- a/apps/api/headquarter_api.egg-info/top_level.txt +++ /dev/null @@ -1 +0,0 @@ -app diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..2e905ed --- /dev/null +++ b/docs/README.md @@ -0,0 +1,15 @@ +# Headquarter Documentation + +This directory contains architecture, development, and deployment documentation for the Headquarter platform. + +## Index + +- [Architecture](architecture.md) — System architecture, stack decisions, and MVP phases *(FN-001)* +- [Development](development.md) — Local setup, prerequisites, and day-to-day commands *(FN-002)* +- [Deployment](deployment.md) — Deployment assumptions, Portainer/Traefik skeleton, and follow-up scope *(FN-002)* + +## Quick Links + +- [Root README](../README.md) +- [Deploy Skeleton](../deploy/README.md) +- [Project Brief](project-brief.md) — Original product brief and confirmed stack diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..8282bc2 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,236 @@ +# Headquarter Architecture + +> Canonical architecture specification for the hosted workspace and tool-orchestration platform. +> Decisions in this document override ad-hoc choices in implementation tasks. + +## 1. Overview & Goals + +Headquarter is a hosted control plane where authenticated users create projects, connect Git repositories, and spawn containerized tools (RunFusion, code-server, and future tools). The platform is manifest-driven and provider-abstracted so new tools, Git providers, runtimes, and access providers can be added without rewriting core orchestration logic. + +**MVP scope:** Single-user projects, Authentik OIDC auth, Docker runtime, Traefik subdomain routing, Portainer-managed Docker Compose deployment. + +## 2. Domain Model + +```text +User ──< Project ──< Repository + └──< ToolInstance + +Tool (manifest-driven, global registry) +``` + +- **User:** Authentik-managed identity. MVP assumes individual users; schema leaves room for teams/shared projects later. +- **Project:** Owned by a user. Contains repositories and spawned tool instances. +- **Repository:** Git-backed workspace. Clone, fetch, and push via provider-independent adapters. +- **Tool:** Manifest-driven definition (image, ports, mounts, env, health checks, routing rules). Defined in FN-003. +- **ToolInstance:** A running container spawned from a Tool manifest for a specific project. Receives workspace mounts, config mounts, secrets, and Traefik routing labels. + +## 3. Provider Interfaces + +The backend must define provider contracts before implementing any concrete adapter. + +### 3.1 GitProvider + +```python +class GitProvider(Protocol): + def clone(self, repo_url: str, dest: Path, credentials: GitCredentials) -> None: ... + def fetch(self, repo_path: Path, credentials: GitCredentials) -> None: ... + def push(self, repo_path: Path, credentials: GitCredentials) -> None: ... +``` + +- Adapters: GitHub, GitLab, Gitea, Forgejo, etc. +- Credentials: generated SSH keys (per-repository) or access tokens. +- SSH keys must be scoped per repository connection for clean revocation. + +### 3.2 RuntimeProvider + +```python +class RuntimeProvider(Protocol): + def spawn(self, manifest: ToolManifest, project: Project, config: SpawnConfig) -> ToolInstance: ... + def stop(self, instance: ToolInstance) -> None: ... + def health(self, instance: ToolInstance) -> HealthStatus: ... +``` + +- MVP adapter: Docker Compose service generation + Docker API. +- Future adapters: Kubernetes, Nomad, etc. + +### 3.3 AccessProvider + +```python +class AccessProvider(Protocol): + def route(self, instance: ToolInstance, domain: str) -> RoutingConfig: ... +``` + +- MVP adapter: Traefik labels on Docker containers. +- Future adapter: Cloudflare Tunnel, etc. + +## 4. Deployment Architecture + +### 4.1 MVP Target + +- **Orchestration:** Portainer-managed Docker Compose stack. +- **Reverse Proxy:** Existing Traefik instance (external to the app stack). +- **Network:** Shared Traefik Docker network; app stack attaches to it. +- **Certificate Resolution:** Let's Encrypt or internal CA via Traefik cert resolver. + +### 4.2 Subdomain Routing + +Path-based routing is avoided because many tools expect to run at `/`. + +Pattern: +``` +https://{tool}-{project}-{user}.{tool_domain} +``` + +Examples: +``` +https://runfusion-myapp-alice.tools.example.com +https://code-myapp-alice.tools.example.com +``` + +### 4.3 Compose Skeleton + +- `docker-compose.yml`: local development (backend, frontend, PostgreSQL). +- `docker-compose.traefik.yml`: deployment overlay with Traefik labels and external network. +- Environment-driven; no secrets committed to repository. + +## 5. Security Boundaries + +### 5.1 Authentication + +- Authentik OIDC for user login. +- FastAPI backend validates JWT/id tokens at API boundaries. +- Frontend stores tokens securely (httpOnly cookie or secure storage pattern). + +### 5.2 Secrets + +- Never treat secrets as plaintext config. +- Support encrypted storage at rest and runtime injection as: + - Environment variables + - Mounted secret files +- Encryption key is an environment secret (`SECRET_ENCRYPTION_KEY`). + +### 5.3 SSH Keys + +- Generated per repository connection. +- Stored encrypted. +- Injected into tool containers at runtime for Git operations. + +### 5.4 Container Isolation + +- Each tool instance runs in its own container. +- Resource limits declared in tool manifest. +- Workspace and config mounts are scoped to user/project. + +## 6. Data & Storage + +### 6.1 Database + +- PostgreSQL for relational data (users, projects, repositories, tool instances, manifests). +- Schema migrations managed by backend (Alembic or equivalent). + +### 6.2 Filesystem Layout (Conceptual) + +```text +/data/ + users/{userId}/tool-configs/{toolId}/ + projects/{projectId}/repo/ + projects/{projectId}/tool-configs/{toolId}/ +``` + +- Repository workspace storage: Docker volumes or local bind mounts. +- Tool config storage: persistent host mounts, separate from repository workspaces. +- Config scopes: global default → user-level → project-level → tool-instance override. + +## 7. Tool Manifest & Orchestration + +Tools are defined by manifests (FN-003) that declare: + +- Runtime image / image tag +- Node/npm version expectations (for executable environments) +- Bootstrap / install commands +- Command execution needs +- Workspace mounts +- Config mounts +- Environment variables +- Secrets +- Ports +- Health checks +- Resource limits +- Traefik routing needs (subdomain pattern, middleware) + +The platform reads manifests and generates: +- Docker Compose service definitions +- Traefik labels for routing +- Volume mounts for workspace and config +- Secret injection at runtime + +## 8. MVP Phases + +| Phase | Task | Deliverable | +|-------|------|-------------| +| Foundation | FN-002 | Monorepo scaffold, build/test/lint pipelines | +| Registry | FN-003 | Manifest schema, RunFusion and code-server manifests | +| Backend | FN-004 | FastAPI app, domain models, API endpoints, DB migrations | +| Frontend | FN-005 | Auth-ready shell, navigation, placeholder screens | +| Deployment | FN-006 | Docker Compose overlays, Traefik labels, Portainer config | +| Git Model | FN-007 | Provider interface, SSH key generation, credential storage | +| RunFusion POC | FN-008 | Executable environment proof of concept | +| Secrets & Config | FN-009 | Encrypted secrets, persistent config mounts | +| code-server Spawn | FN-010 | code-server manifest, spawn script, runtime integration | + +## 9. Extension Points + +- **New Git providers:** Implement `GitProvider` protocol. +- **New tools:** Add a manifest to the registry (no code changes required for standard containers). +- **New runtimes:** Implement `RuntimeProvider` protocol. +- **New access providers:** Implement `AccessProvider` protocol. +- **Teams/organizations:** Add `Organization` and `ProjectMember` entities later. + +## 10. Environment Assumptions + +Required environment variables (no defaults in production): + +```env +APP_NAME= +ROOT_DOMAIN= +TOOL_DOMAIN= +API_URL= +TRAEFIK_NETWORK= +TRAEFIK_ENTRYPOINT= +TRAEFIK_CERT_RESOLVER= +AUTHENTIK_ISSUER_URL= +AUTHENTIK_CLIENT_ID= +AUTHENTIK_CLIENT_SECRET= +DATABASE_URL= +SECRET_ENCRYPTION_KEY= +``` + +Local development uses `.env.example` and safe defaults. + +## 11. Technology Boundaries + +| Layer | Choice | Migration Path | +|-------|--------|----------------| +| Frontend | React + Vite | Next.js, Vue, etc. if needed | +| Backend | FastAPI | Any ASGI framework | +| Database | PostgreSQL | Managed Postgres, CockroachDB | +| Runtime | Docker Compose | Kubernetes, Nomad | +| Access | Traefik | Cloudflare Tunnel, custom proxy | +| Auth | Authentik OIDC | Any OIDC provider | + +## 12. Acceptance Criteria for Architecture Compliance + +Any implementation task must: + +1. Respect provider interfaces (no hardcoded GitHub/Traefik logic in core orchestration). +2. Keep secrets out of committed files and plaintext logs. +3. Use environment variables for deployment-specific values. +4. Leave schema room for multi-user teams without rewriting ownership models. +5. Support adding a new tool via manifest + registry entry alone (no new backend code for standard containers). + +## 13. Deferred Decisions + +- **Multi-tenancy:** MVP is single-tenant deployment. Multi-tenant routing and isolation are future concerns. +- **High availability:** No replicas or load balancing in MVP. +- **Backup strategy:** Out of MVP scope; rely on host-level volume backups. +- **Rate limiting:** Not in MVP; add at Traefik or API gateway layer later. diff --git a/docs/deployment.md b/docs/deployment.md new file mode 100644 index 0000000..eb59e09 --- /dev/null +++ b/docs/deployment.md @@ -0,0 +1,60 @@ +# Deployment Guide + +## Overview + +The MVP deployment target is a **Portainer-managed Docker Compose stack** with an existing **Traefik** reverse proxy. + +This document covers the scaffold-level deployment assumptions created in FN-002. Detailed deployment automation (dynamic labels for spawned tool containers, secret rotation, CI/CD pipelines) is follow-up scope for **FN-006**. + +## Stack Assumptions + +- **Reverse proxy**: Traefik (already running on the target host) +- **Orchestration**: Portainer managing Docker Compose stacks +- **Network**: External Traefik network named `traefik` (or as configured) +- **Routing**: Subdomain-based (`{tool}-{project}-{user}.tools.{ROOT_DOMAIN}`) +- **TLS**: Traefik cert resolver (e.g., `letsencrypt` or Cloudflare) + +## Deployment Files + +| File | Purpose | +|------|---------| +| `docker-compose.yml` | Local development (API, web, Postgres) | +| `docker-compose.traefik.yml` | Deployment overlay with Traefik labels | +| `deploy/portainer.env.example` | Deployment environment variables | +| `deploy/traefik-labels.example.yml` | Example Traefik labels for services | +| `deploy/README.md` | Deploy skeleton usage notes | + +## Environment Variables + +See `.env.example` for the full variable list. Key deployment variables: + +```env +APP_NAME=Headquarter +ROOT_DOMAIN=example.com +TOOL_DOMAIN=tools.example.com +TRAEFIK_NETWORK=traefik +TRAEFIK_ENTRYPOINT=websecure +TRAEFIK_CERT_RESOLVER=letsencrypt +AUTHENTIK_ISSUER_URL=https://auth.example.com/application/o/headquarter/ +AUTHENTIK_CLIENT_ID= +AUTHENTIK_CLIENT_SECRET= +POSTGRES_PASSWORD= +SECRET_ENCRYPTION_KEY= +``` + +## Local vs Production + +- **Local**: `docker compose up --build -d` uses `docker-compose.yml` only. +- **Production**: Portainer deploys the stack using the main compose file plus the Traefik overlay. + +## Scoped Secrets + +Do not commit real secrets. Use: + +- Portainer environment variables (stored in Portainer, not in Git) +- `.env` files (ignored by Git, documented in `.env.example`) +- Docker secrets (to be evaluated in FN-006) + +## Follow-up Work + +- **FN-006**: Full deployment automation, dynamic Traefik labels for spawned tool containers, Portainer stack definitions, and CI/CD integration. diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..010964d --- /dev/null +++ b/docs/development.md @@ -0,0 +1,139 @@ +# Development Guide + +## Prerequisites + +- **Node.js** ≥ 20 and **pnpm** ≥ 9 +- **Python** ≥ 3.11 with `venv` support +- **Docker** and **Docker Compose** (for local services) + +## Installation + +```bash +# Install Node dependencies and Python virtualenv + packages +make install + +# Or manually: +pnpm install +cd apps/api && python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" +``` + +## Environment Setup + +Copy the root environment example and fill in local values: + +```bash +cp .env.example .env +``` + +Copy the frontend environment example: + +```bash +cp apps/web/.env.example apps/web/.env +``` + +## Running Locally + +### Frontend only + +```bash +cd apps/web +pnpm dev # Vite dev server on http://localhost:5173 +``` + +### Backend only + +```bash +cd apps/api +.venv/bin/uvicorn app.main:app --reload --port 8000 +``` + +### Both (via root script) + +```bash +pnpm dev # Runs frontend and backend in parallel +``` + +### With Docker Compose + +```bash +docker compose up --build -d +``` + +## Testing + +### Frontend + +```bash +pnpm --filter @headquarter/web test +``` + +Uses **Vitest** + **Testing Library** + **jsdom**. + +### Backend + +```bash +pnpm --filter @headquarter/api test +``` + +Or directly with pytest: + +```bash +cd apps/api && .venv/bin/pytest +``` + +### All tests + +```bash +make test +# or +pnpm test +``` + +## Linting and Type Checking + +### Frontend + +```bash +pnpm --filter @headquarter/web lint +pnpm --filter @headquarter/web typecheck +``` + +### Backend + +```bash +pnpm --filter @headquarter/api lint +pnpm --filter @headquarter/api typecheck +``` + +### All + +```bash +make lint +make typecheck +``` + +## Building + +```bash +make build +# or +pnpm build +``` + +## Project Layout + +```text +├── apps/ +│ ├── web/ # Vite React TypeScript frontend +│ └── api/ # FastAPI Python backend +├── docs/ # Documentation +├── deploy/ # Deployment skeleton files +├── docker-compose.yml +└── package.json # Root monorepo scripts +``` + +## Conventions + +- **Frontend**: React functional components, TypeScript strict mode, ESLint + Ruff-like rules. +- **Backend**: FastAPI, Pydantic settings, pytest, ruff, mypy. +- **Commits**: Conventional commits with task ID prefix, e.g. `feat(FN-002): description`.