feat(FN-002): complete Step 4 — Documentation Structure and Environment Examples

2026-05-14 01:40:58 +02:00
parent 9eaf4cfa36
commit 1f1c95cdde
10 changed files with 478 additions and 34 deletions
@@ -0,0 +1,28 @@
+# App identity
+APP_NAME=Headquarter
+ROOT_DOMAIN=localhost
+TOOL_DOMAIN=tools.localhost
+
+# API / Web URLs
+API_URL=http://localhost:8000
+WEB_URL=http://localhost:5173
+
+# Database (local development)
+POSTGRES_USER=postgres
+POSTGRES_PASSWORD=postgres
+POSTGRES_DB=headquarter
+DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5432/${POSTGRES_DB}
+
+# Authentik OIDC placeholders (wire in FN-004)
+AUTHENTIK_ISSUER_URL=https://auth.example.com/application/o/headquarter/
+AUTHENTIK_CLIENT_ID=your-client-id
+AUTHENTIK_CLIENT_SECRET=your-client-secret
+
+# Traefik / deployment placeholders (wire in FN-006)
+TRAEFIK_NETWORK=traefik
+TRAEFIK_ENTRYPOINT=websecure
+TRAEFIK_CERT_RESOLVER=letsencrypt
+TOOL_SUBDOMAIN_PATTERN={tool}-{project}-{user}.tools.${ROOT_DOMAIN}
+
+# Secrets (generate strong random values for production)
+SECRET_ENCRYPTION_KEY=change-me-in-production
@@ -1,13 +0,0 @@
-Metadata-Version: 2.4
-Name: headquarter-api
-Version: 0.0.1
-Summary: Headquarter FastAPI backend
-Requires-Python: >=3.11
-Requires-Dist: fastapi>=0.115.0
-Requires-Dist: uvicorn[standard]>=0.34.0
-Requires-Dist: pydantic-settings>=2.8.0
-Provides-Extra: dev
-Requires-Dist: pytest>=8.3.0; extra == "dev"
-Requires-Dist: httpx>=0.28.0; extra == "dev"
-Requires-Dist: ruff>=0.11.0; extra == "dev"
-Requires-Dist: mypy>=1.15.0; extra == "dev"
@@ -1,10 +0,0 @@
-pyproject.toml
-app/__init__.py
-app/config.py
-app/main.py
-headquarter_api.egg-info/PKG-INFO
-headquarter_api.egg-info/SOURCES.txt
-headquarter_api.egg-info/dependency_links.txt
-headquarter_api.egg-info/requires.txt
-headquarter_api.egg-info/top_level.txt
-tests/test_health.py
@@ -1 +0,0 @@
-
@@ -1,9 +0,0 @@
-fastapi>=0.115.0
-uvicorn[standard]>=0.34.0
-pydantic-settings>=2.8.0
-
-[dev]
-pytest>=8.3.0
-httpx>=0.28.0
-ruff>=0.11.0
-mypy>=1.15.0
@@ -1 +0,0 @@
-app
@@ -0,0 +1,15 @@
+# Headquarter Documentation
+
+This directory contains architecture, development, and deployment documentation for the Headquarter platform.
+
+## Index
+
+- [Architecture](architecture.md) — System architecture, stack decisions, and MVP phases *(FN-001)*
+- [Development](development.md) — Local setup, prerequisites, and day-to-day commands *(FN-002)*
+- [Deployment](deployment.md) — Deployment assumptions, Portainer/Traefik skeleton, and follow-up scope *(FN-002)*
+
+## Quick Links
+
+- [Root README](../README.md)
+- [Deploy Skeleton](../deploy/README.md)
+- [Project Brief](project-brief.md) — Original product brief and confirmed stack
@@ -0,0 +1,236 @@
+# Headquarter Architecture
+
+> Canonical architecture specification for the hosted workspace and tool-orchestration platform.
+> Decisions in this document override ad-hoc choices in implementation tasks.
+
+## 1. Overview & Goals
+
+Headquarter is a hosted control plane where authenticated users create projects, connect Git repositories, and spawn containerized tools (RunFusion, code-server, and future tools). The platform is manifest-driven and provider-abstracted so new tools, Git providers, runtimes, and access providers can be added without rewriting core orchestration logic.
+
+**MVP scope:** Single-user projects, Authentik OIDC auth, Docker runtime, Traefik subdomain routing, Portainer-managed Docker Compose deployment.
+
+## 2. Domain Model
+
+```text
+User ──< Project ──< Repository
+                └──< ToolInstance
+
+Tool (manifest-driven, global registry)
+```
+
+- **User:** Authentik-managed identity. MVP assumes individual users; schema leaves room for teams/shared projects later.
+- **Project:** Owned by a user. Contains repositories and spawned tool instances.
+- **Repository:** Git-backed workspace. Clone, fetch, and push via provider-independent adapters.
+- **Tool:** Manifest-driven definition (image, ports, mounts, env, health checks, routing rules). Defined in FN-003.
+- **ToolInstance:** A running container spawned from a Tool manifest for a specific project. Receives workspace mounts, config mounts, secrets, and Traefik routing labels.
+
+## 3. Provider Interfaces
+
+The backend must define provider contracts before implementing any concrete adapter.
+
+### 3.1 GitProvider
+
+```python
+class GitProvider(Protocol):
+    def clone(self, repo_url: str, dest: Path, credentials: GitCredentials) -> None: ...
+    def fetch(self, repo_path: Path, credentials: GitCredentials) -> None: ...
+    def push(self, repo_path: Path, credentials: GitCredentials) -> None: ...
+```
+
+- Adapters: GitHub, GitLab, Gitea, Forgejo, etc.
+- Credentials: generated SSH keys (per-repository) or access tokens.
+- SSH keys must be scoped per repository connection for clean revocation.
+
+### 3.2 RuntimeProvider
+
+```python
+class RuntimeProvider(Protocol):
+    def spawn(self, manifest: ToolManifest, project: Project, config: SpawnConfig) -> ToolInstance: ...
+    def stop(self, instance: ToolInstance) -> None: ...
+    def health(self, instance: ToolInstance) -> HealthStatus: ...
+```
+
+- MVP adapter: Docker Compose service generation + Docker API.
+- Future adapters: Kubernetes, Nomad, etc.
+
+### 3.3 AccessProvider
+
+```python
+class AccessProvider(Protocol):
+    def route(self, instance: ToolInstance, domain: str) -> RoutingConfig: ...
+```
+
+- MVP adapter: Traefik labels on Docker containers.
+- Future adapter: Cloudflare Tunnel, etc.
+
+## 4. Deployment Architecture
+
+### 4.1 MVP Target
+
+- **Orchestration:** Portainer-managed Docker Compose stack.
+- **Reverse Proxy:** Existing Traefik instance (external to the app stack).
+- **Network:** Shared Traefik Docker network; app stack attaches to it.
+- **Certificate Resolution:** Let's Encrypt or internal CA via Traefik cert resolver.
+
+### 4.2 Subdomain Routing
+
+Path-based routing is avoided because many tools expect to run at `/`.
+
+Pattern:
+```
+https://{tool}-{project}-{user}.{tool_domain}
+```
+
+Examples:
+```
+https://runfusion-myapp-alice.tools.example.com
+https://code-myapp-alice.tools.example.com
+```
+
+### 4.3 Compose Skeleton
+
+- `docker-compose.yml`: local development (backend, frontend, PostgreSQL).
+- `docker-compose.traefik.yml`: deployment overlay with Traefik labels and external network.
+- Environment-driven; no secrets committed to repository.
+
+## 5. Security Boundaries
+
+### 5.1 Authentication
+
+- Authentik OIDC for user login.
+- FastAPI backend validates JWT/id tokens at API boundaries.
+- Frontend stores tokens securely (httpOnly cookie or secure storage pattern).
+
+### 5.2 Secrets
+
+- Never treat secrets as plaintext config.
+- Support encrypted storage at rest and runtime injection as:
+  - Environment variables
+  - Mounted secret files
+- Encryption key is an environment secret (`SECRET_ENCRYPTION_KEY`).
+
+### 5.3 SSH Keys
+
+- Generated per repository connection.
+- Stored encrypted.
+- Injected into tool containers at runtime for Git operations.
+
+### 5.4 Container Isolation
+
+- Each tool instance runs in its own container.
+- Resource limits declared in tool manifest.
+- Workspace and config mounts are scoped to user/project.
+
+## 6. Data & Storage
+
+### 6.1 Database
+
+- PostgreSQL for relational data (users, projects, repositories, tool instances, manifests).
+- Schema migrations managed by backend (Alembic or equivalent).
+
+### 6.2 Filesystem Layout (Conceptual)
+
+```text
+/data/
+  users/{userId}/tool-configs/{toolId}/
+  projects/{projectId}/repo/
+  projects/{projectId}/tool-configs/{toolId}/
+```
+
+- Repository workspace storage: Docker volumes or local bind mounts.
+- Tool config storage: persistent host mounts, separate from repository workspaces.
+- Config scopes: global default → user-level → project-level → tool-instance override.
+
+## 7. Tool Manifest & Orchestration
+
+Tools are defined by manifests (FN-003) that declare:
+
+- Runtime image / image tag
+- Node/npm version expectations (for executable environments)
+- Bootstrap / install commands
+- Command execution needs
+- Workspace mounts
+- Config mounts
+- Environment variables
+- Secrets
+- Ports
+- Health checks
+- Resource limits
+- Traefik routing needs (subdomain pattern, middleware)
+
+The platform reads manifests and generates:
+- Docker Compose service definitions
+- Traefik labels for routing
+- Volume mounts for workspace and config
+- Secret injection at runtime
+
+## 8. MVP Phases
+
+| Phase | Task | Deliverable |
+|-------|------|-------------|
+| Foundation | FN-002 | Monorepo scaffold, build/test/lint pipelines |
+| Registry | FN-003 | Manifest schema, RunFusion and code-server manifests |
+| Backend | FN-004 | FastAPI app, domain models, API endpoints, DB migrations |
+| Frontend | FN-005 | Auth-ready shell, navigation, placeholder screens |
+| Deployment | FN-006 | Docker Compose overlays, Traefik labels, Portainer config |
+| Git Model | FN-007 | Provider interface, SSH key generation, credential storage |
+| RunFusion POC | FN-008 | Executable environment proof of concept |
+| Secrets & Config | FN-009 | Encrypted secrets, persistent config mounts |
+| code-server Spawn | FN-010 | code-server manifest, spawn script, runtime integration |
+
+## 9. Extension Points
+
+- **New Git providers:** Implement `GitProvider` protocol.
+- **New tools:** Add a manifest to the registry (no code changes required for standard containers).
+- **New runtimes:** Implement `RuntimeProvider` protocol.
+- **New access providers:** Implement `AccessProvider` protocol.
+- **Teams/organizations:** Add `Organization` and `ProjectMember` entities later.
+
+## 10. Environment Assumptions
+
+Required environment variables (no defaults in production):
+
+```env
+APP_NAME=
+ROOT_DOMAIN=
+TOOL_DOMAIN=
+API_URL=
+TRAEFIK_NETWORK=
+TRAEFIK_ENTRYPOINT=
+TRAEFIK_CERT_RESOLVER=
+AUTHENTIK_ISSUER_URL=
+AUTHENTIK_CLIENT_ID=
+AUTHENTIK_CLIENT_SECRET=
+DATABASE_URL=
+SECRET_ENCRYPTION_KEY=
+```
+
+Local development uses `.env.example` and safe defaults.
+
+## 11. Technology Boundaries
+
+| Layer | Choice | Migration Path |
+|-------|--------|----------------|
+| Frontend | React + Vite | Next.js, Vue, etc. if needed |
+| Backend | FastAPI | Any ASGI framework |
+| Database | PostgreSQL | Managed Postgres, CockroachDB |
+| Runtime | Docker Compose | Kubernetes, Nomad |
+| Access | Traefik | Cloudflare Tunnel, custom proxy |
+| Auth | Authentik OIDC | Any OIDC provider |
+
+## 12. Acceptance Criteria for Architecture Compliance
+
+Any implementation task must:
+
+1. Respect provider interfaces (no hardcoded GitHub/Traefik logic in core orchestration).
+2. Keep secrets out of committed files and plaintext logs.
+3. Use environment variables for deployment-specific values.
+4. Leave schema room for multi-user teams without rewriting ownership models.
+5. Support adding a new tool via manifest + registry entry alone (no new backend code for standard containers).
+
+## 13. Deferred Decisions
+
+- **Multi-tenancy:** MVP is single-tenant deployment. Multi-tenant routing and isolation are future concerns.
+- **High availability:** No replicas or load balancing in MVP.
+- **Backup strategy:** Out of MVP scope; rely on host-level volume backups.
+- **Rate limiting:** Not in MVP; add at Traefik or API gateway layer later.
@@ -0,0 +1,60 @@
+# Deployment Guide
+
+## Overview
+
+The MVP deployment target is a **Portainer-managed Docker Compose stack** with an existing **Traefik** reverse proxy.
+
+This document covers the scaffold-level deployment assumptions created in FN-002. Detailed deployment automation (dynamic labels for spawned tool containers, secret rotation, CI/CD pipelines) is follow-up scope for **FN-006**.
+
+## Stack Assumptions
+
+- **Reverse proxy**: Traefik (already running on the target host)
+- **Orchestration**: Portainer managing Docker Compose stacks
+- **Network**: External Traefik network named `traefik` (or as configured)
+- **Routing**: Subdomain-based (`{tool}-{project}-{user}.tools.{ROOT_DOMAIN}`)
+- **TLS**: Traefik cert resolver (e.g., `letsencrypt` or Cloudflare)
+
+## Deployment Files
+
+| File | Purpose |
+|------|---------|
+| `docker-compose.yml` | Local development (API, web, Postgres) |
+| `docker-compose.traefik.yml` | Deployment overlay with Traefik labels |
+| `deploy/portainer.env.example` | Deployment environment variables |
+| `deploy/traefik-labels.example.yml` | Example Traefik labels for services |
+| `deploy/README.md` | Deploy skeleton usage notes |
+
+## Environment Variables
+
+See `.env.example` for the full variable list. Key deployment variables:
+
+```env
+APP_NAME=Headquarter
+ROOT_DOMAIN=example.com
+TOOL_DOMAIN=tools.example.com
+TRAEFIK_NETWORK=traefik
+TRAEFIK_ENTRYPOINT=websecure
+TRAEFIK_CERT_RESOLVER=letsencrypt
+AUTHENTIK_ISSUER_URL=https://auth.example.com/application/o/headquarter/
+AUTHENTIK_CLIENT_ID=
+AUTHENTIK_CLIENT_SECRET=
+POSTGRES_PASSWORD=
+SECRET_ENCRYPTION_KEY=
+```
+
+## Local vs Production
+
+- **Local**: `docker compose up --build -d` uses `docker-compose.yml` only.
+- **Production**: Portainer deploys the stack using the main compose file plus the Traefik overlay.
+
+## Scoped Secrets
+
+Do not commit real secrets. Use:
+
+- Portainer environment variables (stored in Portainer, not in Git)
+- `.env` files (ignored by Git, documented in `.env.example`)
+- Docker secrets (to be evaluated in FN-006)
+
+## Follow-up Work
+
+- **FN-006**: Full deployment automation, dynamic Traefik labels for spawned tool containers, Portainer stack definitions, and CI/CD integration.
@@ -0,0 +1,139 @@
+# Development Guide
+
+## Prerequisites
+
+- **Node.js** ≥ 20 and **pnpm** ≥ 9
+- **Python** ≥ 3.11 with `venv` support
+- **Docker** and **Docker Compose** (for local services)
+
+## Installation
+
+```bash
+# Install Node dependencies and Python virtualenv + packages
+make install
+
+# Or manually:
+pnpm install
+cd apps/api && python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"
+```
+
+## Environment Setup
+
+Copy the root environment example and fill in local values:
+
+```bash
+cp .env.example .env
+```
+
+Copy the frontend environment example:
+
+```bash
+cp apps/web/.env.example apps/web/.env
+```
+
+## Running Locally
+
+### Frontend only
+
+```bash
+cd apps/web
+pnpm dev        # Vite dev server on http://localhost:5173
+```
+
+### Backend only
+
+```bash
+cd apps/api
+.venv/bin/uvicorn app.main:app --reload --port 8000
+```
+
+### Both (via root script)
+
+```bash
+pnpm dev        # Runs frontend and backend in parallel
+```
+
+### With Docker Compose
+
+```bash
+docker compose up --build -d
+```
+
+## Testing
+
+### Frontend
+
+```bash
+pnpm --filter @headquarter/web test
+```
+
+Uses **Vitest** + **Testing Library** + **jsdom**.
+
+### Backend
+
+```bash
+pnpm --filter @headquarter/api test
+```
+
+Or directly with pytest:
+
+```bash
+cd apps/api && .venv/bin/pytest
+```
+
+### All tests
+
+```bash
+make test
+# or
+pnpm test
+```
+
+## Linting and Type Checking
+
+### Frontend
+
+```bash
+pnpm --filter @headquarter/web lint
+pnpm --filter @headquarter/web typecheck
+```
+
+### Backend
+
+```bash
+pnpm --filter @headquarter/api lint
+pnpm --filter @headquarter/api typecheck
+```
+
+### All
+
+```bash
+make lint
+make typecheck
+```
+
+## Building
+
+```bash
+make build
+# or
+pnpm build
+```
+
+## Project Layout
+
+```text
+├── apps/
+│   ├── web/          # Vite React TypeScript frontend
+│   └── api/          # FastAPI Python backend
+├── docs/             # Documentation
+├── deploy/           # Deployment skeleton files
+├── docker-compose.yml
+└── package.json      # Root monorepo scripts
+```
+
+## Conventions
+
+- **Frontend**: React functional components, TypeScript strict mode, ESLint + Ruff-like rules.
+- **Backend**: FastAPI, Pydantic settings, pytest, ruff, mypy.
+- **Commits**: Conventional commits with task ID prefix, e.g. `feat(FN-002): description`.