feat: implement docker infrastructure (US-001)
- Add docker-compose.yml with postgres, redis, api, and web services - Add multi-stage Dockerfile for API (Python 3.11) - Add multi-stage Dockerfile for web (Node.js 20 + nginx) - Add Makefile with common development commands - Add .env.example with all required environment variables - Add placeholder pyproject.toml and package.json for builds - Configure health checks for all services - Setup persistent volumes for postgres, redis, and repos - Run services as non-root users
This commit is contained in:
@@ -1,20 +0,0 @@
|
||||
# Headquarter Documentation
|
||||
|
||||
This directory contains architecture, development, and deployment documentation for the Headquarter platform.
|
||||
|
||||
## Index
|
||||
|
||||
- [Architecture](architecture.md) — Canonical system architecture, domain model, provider contracts, and security boundaries *(FN-019)*
|
||||
- [MVP Scope](mvp-scope.md) — MVP boundaries, user journeys, milestones, and dependency order *(FN-019)*
|
||||
- [Project Brief](project-brief.md) — High-level product context: what, why, who, and confirmed stack *(FN-019)*
|
||||
- [Development](development.md) — Local setup, prerequisites, and day-to-day commands *(FN-002)*
|
||||
- [Deployment](deployment.md) — Deployment assumptions, Portainer/Traefik skeleton, and operator guide *(FN-002)*
|
||||
|
||||
## Reference
|
||||
|
||||
- [Conversation Handoff](conversation-handoff.md) — Key architectural decisions, assumptions, and open loops from planning *(FN-019)*
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Root README](../README.md)
|
||||
- [Deployment Guide](deployment.md)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,68 +0,0 @@
|
||||
# Conversation Handoff: FN-019
|
||||
|
||||
## Task Context
|
||||
|
||||
This document captures the key architectural decisions, assumptions, and open loops produced during **FN-019 — Architecture and MVP Specification for Headquarter**.
|
||||
|
||||
## Key Decisions Made
|
||||
|
||||
### 1. Auth Callback Flow (Section 12.2)
|
||||
- **Decision:** Backend-handled callback is the recommended MVP pattern.
|
||||
- **Rationale:** The frontend receives the Authentik redirect at a backend endpoint (`GET /api/v1/auth/callback`), the backend exchanges the code for tokens, sets an httpOnly cookie, and returns an HTTP 302 redirect to the frontend dashboard. This avoids exposing the client secret to the frontend and avoids the `fetch()` + redirect ambiguity.
|
||||
|
||||
### 2. GitProvider Split (Section 5.1)
|
||||
- **Decision:** Two complementary abstractions: `GitProvider` (remote API operations) and `GitOperations` (local Git CLI).
|
||||
- **Rationale:** This separation was already implemented in FN-011. The architecture doc preserves and formalizes it.
|
||||
|
||||
### 3. AccessProvider Protocol (Section 9.2)
|
||||
- **Decision:** Added an explicit `AccessProvider` ABC with `RoutingConfig` Pydantic model.
|
||||
- **Rationale:** The original architecture doc mentioned `AccessProvider` as an extension point but never defined method signatures. The enhanced doc makes it as concrete as `GitProvider` and `RuntimeProvider`.
|
||||
|
||||
### 4. Subdomain Pattern (Section 10.1)
|
||||
- **Decision:** Default pattern is `{tool}-{project}-{user}.{tool_domain}`.
|
||||
- **Rationale:** Aligns with existing `config.py` (`tool_subdomain_pattern`) and downstream FN-006 label generator. The relationship between `ROOT_DOMAIN` and `TOOL_DOMAIN` is now explicitly documented.
|
||||
|
||||
### 5. Dev Bypass Security Model (Section 12.7)
|
||||
- **Decision:** `AUTH_DEV_BYPASS` is an environment variable, not a query parameter.
|
||||
- **Rationale:** The backend must reject the bypass when `settings.debug` is `False`, even if the env var is set. This prevents accidental production exposure.
|
||||
|
||||
### 6. Credential Storage Abstraction (Section 4.10, 5.3, 6.3)
|
||||
- **Decision:** Credentials are stored as encrypted `secret` rows with `scope_type='repository'`.
|
||||
- **Rationale:** The `CredentialStorage` ABC (from FN-011) is the interface, but the canonical storage is the `secret` table. The architecture doc now cross-references correctly.
|
||||
|
||||
## Assumptions
|
||||
|
||||
1. **Single-tenant MVP:** The platform runs as a single deployment with no hard multi-tenant isolation.
|
||||
2. **Portainer-managed stacks:** Production deployment assumes an existing Portainer instance.
|
||||
3. **Existing Traefik:** The reverse proxy is already running and attached to an external Docker network named `traefik`.
|
||||
4. **Authentik pre-configured:** The OIDC application is created in Authentik before deployment.
|
||||
5. **PostgreSQL 17+:** The database schema uses features compatible with PostgreSQL 17.
|
||||
|
||||
## Open Loops for Future Tasks
|
||||
|
||||
1. **User slug derivation (architecture.md Section 18, Open Question 2):** Should the user slug for subdomain generation be derived from `display_name`, `email` local-part, or a new `slug` column? Decision needed before FN-006/FN-010 implementation.
|
||||
2. **Admin role in MVP (architecture.md Section 18, Open Question 1):** Do we need a basic admin role for global config management? Decision needed before FN-009 API implementation.
|
||||
3. **Auto-deploy-key registration (architecture.md Section 18, Open Question 4):** Should the platform auto-register deploy keys via provider APIs, or is manual copy-paste acceptable for MVP? Decision needed before FN-011 UI work.
|
||||
4. **Container image trust (architecture.md Section 18, Open Question 5):** Should the platform restrict tool images to an allow-list? Decision needed before FN-010/FN-008 spawn implementation.
|
||||
5. **Subdomain truncation strategy (architecture.md Section 10.1):** DNS labels have a 63-byte limit. A deterministic truncation/hashing strategy for long project or user names is needed before FN-006 label generation is finalized.
|
||||
6. **Global config write permissions (architecture.md Section 13.7):** The architecture doc documents two possible MVP behaviors (allow all authenticated users, or reject with 403). A stakeholder must choose before FN-009 router implementation.
|
||||
|
||||
## Files Modified / Created
|
||||
|
||||
- `docs/architecture.md` — Rewritten with 18 required sections
|
||||
- `docs/mvp-scope.md` — New file
|
||||
- `docs/project-brief.md` — New file
|
||||
- `docs/conversation-handoff.md` — New file (this document)
|
||||
- `docs/README.md` — Updated index
|
||||
- `tests/docs/test_architecture.py` — New automated validation suite
|
||||
- `tests/docs/__init__.py` — New empty init
|
||||
|
||||
## Downstream Dependencies
|
||||
|
||||
- **FN-004:** Backend Foundation — depends on the PostgreSQL domain model in Section 4
|
||||
- **FN-005:** Frontend Foundation — depends on component boundaries in Section 3.1
|
||||
- **FN-006:** Deployment Config — depends on Traefik routing model in Section 10
|
||||
- **FN-008:** OpenCode POC — depends on spawn lifecycle in Section 8 and Docker runtime in Section 9
|
||||
- **FN-009:** Config & Secrets — depends on storage layout in Section 11 and security in Section 13
|
||||
- **FN-010:** code-server Spawn — depends on tool manifest in Section 7 and Docker runtime in Section 9
|
||||
- **FN-011:** Git Provider — depends on Git provider abstraction in Section 5
|
||||
@@ -1,133 +0,0 @@
|
||||
# Deployment Guide
|
||||
|
||||
## Overview
|
||||
|
||||
The MVP deployment target is a **Portainer-managed Docker Compose stack** with **Traefik** as the reverse proxy. All services run in Docker containers with automatic TLS via Let's Encrypt.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Internet
|
||||
|
|
||||
v
|
||||
Traefik (443/80) ──► Let's Encrypt TLS
|
||||
|
|
||||
├──► api.example.com ──► FastAPI backend
|
||||
├──► example.com ──► React frontend
|
||||
├──► traefik.example.com ──► Traefik dashboard
|
||||
└──► {tool}-{project}-{user}.tools.example.com ──► Spawned tool containers
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker and Docker Compose
|
||||
- A domain with DNS A/AAAA records pointing to your server
|
||||
- Ports 80 and 443 open
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Configure Environment
|
||||
|
||||
Copy the production environment example and fill in all values:
|
||||
|
||||
```bash
|
||||
cp deploy/.env.example deploy/.env
|
||||
```
|
||||
|
||||
Required variables:
|
||||
|
||||
| Variable | Description | Example |
|
||||
|----------|-------------|---------|
|
||||
| `ROOT_DOMAIN` | Your domain | `example.com` |
|
||||
| `ACME_EMAIL` | Let's Encrypt contact email | `admin@example.com` |
|
||||
| `POSTGRES_PASSWORD` | Database password | (strong random) |
|
||||
| `AUTHENTIK_CLIENT_SECRET` | OIDC client secret | (from Authentik) |
|
||||
| `SECRET_ENCRYPTION_KEY` | Fernet encryption key | (32-byte base64) |
|
||||
|
||||
### 2. Deploy Locally (Testing)
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.prod.yml up --build -d
|
||||
```
|
||||
|
||||
This starts: Traefik, API, web frontend, and PostgreSQL.
|
||||
|
||||
### 3. Deploy to Production (Portainer)
|
||||
|
||||
1. In Portainer, create a new stack
|
||||
2. Upload `deploy/portainer-stack.yml`
|
||||
3. Set environment variables from `deploy/portainer.env.example`
|
||||
4. Deploy the stack
|
||||
|
||||
## Deployment Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `docker-compose.yml` | Local development (API, web, Postgres) |
|
||||
| `docker-compose.prod.yml` | Production compose with Traefik |
|
||||
| `deploy/portainer-stack.yml` | Portainer stack definition |
|
||||
| `deploy/portainer.env.example` | Portainer environment variables |
|
||||
| `deploy/.env.example` | Production environment variables |
|
||||
|
||||
## Traefik Configuration
|
||||
|
||||
Traefik handles all routing and TLS:
|
||||
|
||||
- **Entrypoints**: `web` (80) → redirects to `websecure` (443)
|
||||
- **Certificates**: Let's Encrypt via TLS challenge
|
||||
- **Dashboard**: Available at `traefik.${ROOT_DOMAIN}` (protected by middleware)
|
||||
- **Metrics**: Prometheus metrics exposed on `/metrics`
|
||||
|
||||
### Subdomain Routing
|
||||
|
||||
Tool containers are routed via subdomains:
|
||||
|
||||
```
|
||||
{tool}-{project}-{user}.tools.{ROOT_DOMAIN}
|
||||
```
|
||||
|
||||
Example: `vscode-myproject-john.tools.example.com`
|
||||
|
||||
## DNS Requirements
|
||||
|
||||
Create DNS A records for:
|
||||
|
||||
- `example.com` → your server IP
|
||||
- `*.example.com` → your server IP (wildcard for subdomains)
|
||||
- `*.tools.example.com` → your server IP (tool subdomains)
|
||||
|
||||
## Security
|
||||
|
||||
- All services communicate over HTTPS
|
||||
- Traefik adds security headers (HSTS, XSS protection, etc.)
|
||||
- Database is not exposed externally
|
||||
- Secrets are injected via environment variables
|
||||
|
||||
## Updating
|
||||
|
||||
To update the deployment:
|
||||
|
||||
```bash
|
||||
# Pull new images
|
||||
docker compose -f docker-compose.prod.yml pull
|
||||
|
||||
# Restart services
|
||||
docker compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
Check Traefik logs:
|
||||
```bash
|
||||
docker logs traefik
|
||||
```
|
||||
|
||||
Check service health:
|
||||
```bash
|
||||
docker compose -f docker-compose.prod.yml ps
|
||||
```
|
||||
|
||||
Verify certificates:
|
||||
```bash
|
||||
curl -v https://api.example.com
|
||||
```
|
||||
@@ -1,453 +0,0 @@
|
||||
# Development Guide
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node.js** ≥ 20 and **pnpm** ≥ 9
|
||||
- **Python** ≥ 3.11 with `venv` support
|
||||
- **Docker** and **Docker Compose** (for local services)
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Install Node dependencies and Python virtualenv + packages
|
||||
make install
|
||||
|
||||
# Or manually:
|
||||
pnpm install
|
||||
cd apps/api && python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
## Environment Setup
|
||||
|
||||
Copy the root environment example and fill in local values:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Copy the frontend environment example:
|
||||
|
||||
```bash
|
||||
cp apps/web/.env.example apps/web/.env
|
||||
```
|
||||
|
||||
### Frontend Authentication (OIDC)
|
||||
|
||||
The frontend uses OpenID Connect (OIDC) with PKCE for authentication. Configure the following environment variables in `apps/web/.env`:
|
||||
|
||||
| Variable | Description | Example |
|
||||
|----------|-------------|---------|
|
||||
| `VITE_API_URL` | Backend API base URL | `http://localhost:8000` |
|
||||
| `VITE_OIDC_ISSUER` | OIDC provider issuer URL | `https://authentik.example.com/application/o/headquarter` |
|
||||
| `VITE_OIDC_CLIENT_ID` | OIDC client ID | `headquarter-web` |
|
||||
| `VITE_OIDC_REDIRECT_URI` | Post-login redirect URL | `http://localhost:5173/callback` |
|
||||
|
||||
**Authentication Flow:**
|
||||
1. User clicks login → redirected to OIDC provider authorize endpoint
|
||||
2. User authenticates with provider
|
||||
3. Provider redirects to `/callback` with authorization code
|
||||
4. Frontend exchanges code for access token (PKCE)
|
||||
5. Token stored in `localStorage`, user info fetched from `/api/v1/users/me`
|
||||
|
||||
**Logout:**
|
||||
- Clears local token
|
||||
- Redirects to login page
|
||||
- User can re-authenticate via OIDC flow
|
||||
|
||||
## Running Locally
|
||||
|
||||
### Frontend only
|
||||
|
||||
```bash
|
||||
cd apps/web
|
||||
pnpm dev # Vite dev server on http://localhost:5173
|
||||
```
|
||||
|
||||
### Backend only
|
||||
|
||||
```bash
|
||||
cd apps/api
|
||||
.venv/bin/uvicorn app.main:app --reload --port 8000
|
||||
```
|
||||
|
||||
### Both (via root script)
|
||||
|
||||
```bash
|
||||
pnpm dev # Runs frontend and backend in parallel
|
||||
```
|
||||
|
||||
### With Docker Compose
|
||||
|
||||
```bash
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
### Alembic Migrations
|
||||
|
||||
Generate a new migration after modifying models:
|
||||
```bash
|
||||
cd apps/api
|
||||
.venv/bin/alembic revision --autogenerate -m "description"
|
||||
```
|
||||
|
||||
Apply migrations:
|
||||
```bash
|
||||
cd apps/api
|
||||
.venv/bin/alembic upgrade head
|
||||
```
|
||||
|
||||
Downgrade one revision:
|
||||
```bash
|
||||
cd apps/api
|
||||
.venv/bin/alembic downgrade -1
|
||||
```
|
||||
|
||||
Or use the Makefile targets:
|
||||
```bash
|
||||
cd apps/api
|
||||
make revision msg="description"
|
||||
make upgrade
|
||||
make downgrade
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Frontend
|
||||
|
||||
```bash
|
||||
pnpm --filter @headquarter/web test
|
||||
```
|
||||
|
||||
Uses **Vitest** + **Testing Library** + **jsdom**.
|
||||
|
||||
### Backend
|
||||
|
||||
```bash
|
||||
pnpm --filter @headquarter/api test
|
||||
```
|
||||
|
||||
Or directly with pytest:
|
||||
|
||||
```bash
|
||||
cd apps/api && .venv/bin/pytest
|
||||
```
|
||||
|
||||
### All tests
|
||||
|
||||
```bash
|
||||
make test
|
||||
# or
|
||||
pnpm test
|
||||
```
|
||||
|
||||
## CI / Testing
|
||||
|
||||
A GitHub Actions workflow (`.github/workflows/ci.yml`) runs on every push and pull request to `main`. It executes two jobs in parallel:
|
||||
|
||||
- **web-ci** — checks out the repo, installs Node.js ≥ 20 and pnpm, then runs `lint`, `typecheck`, and `test` for `@headquarter/web`.
|
||||
- **api-ci** — checks out the repo, sets up Python 3.11, installs API dev dependencies (`pytest`, `ruff`, `mypy`, `httpx`), starts a PostgreSQL service container, then runs `ruff check`, `mypy`, and `pytest` for the API.
|
||||
|
||||
The workflow reports pass/fail status directly on pull requests as required status checks. All commands must pass before a PR can be merged.
|
||||
|
||||
## Linting and Type Checking
|
||||
|
||||
### Frontend
|
||||
|
||||
```bash
|
||||
pnpm --filter @headquarter/web lint
|
||||
pnpm --filter @headquarter/web typecheck
|
||||
```
|
||||
|
||||
### Backend
|
||||
|
||||
```bash
|
||||
pnpm --filter @headquarter/api lint
|
||||
pnpm --filter @headquarter/api typecheck
|
||||
```
|
||||
|
||||
### All
|
||||
|
||||
```bash
|
||||
make lint
|
||||
make typecheck
|
||||
```
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
make build
|
||||
# or
|
||||
pnpm build
|
||||
```
|
||||
|
||||
## Project Layout
|
||||
|
||||
```text
|
||||
├── apps/
|
||||
│ ├── web/ # Vite React TypeScript frontend
|
||||
│ └── api/ # FastAPI Python backend
|
||||
├── packages/ # Shared packages (future)
|
||||
├── docs/ # Documentation
|
||||
├── deploy/ # Deployment skeleton files
|
||||
├── docker-compose.yml
|
||||
└── package.json # Root monorepo scripts
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Frontend**: React functional components, TypeScript strict mode, ESLint + Ruff-like rules.
|
||||
- **Backend**: FastAPI, Pydantic settings, pytest, ruff, mypy.
|
||||
- **Commits**: Conventional commits with task ID prefix, e.g. `feat(FN-002): description`.
|
||||
|
||||
## Git Abstraction
|
||||
|
||||
The `app/git/` package in the backend provides provider-independent Git
|
||||
orchestration. It is split into two layers so that remote provider API logic
|
||||
and local CLI operations evolve independently:
|
||||
|
||||
| Module | Responsibility |
|
||||
|--------|--------------|
|
||||
| `types` | Enumerations (`ProviderKind`, `CredentialKind`, `ConnectionStatus`, `SshKeyStatus`) |
|
||||
| `provider` | Abstract `GitProvider` — remote operations (`validate_connection`, `list_repositories`, `create_deploy_key`, …) |
|
||||
| `credentials` | `GitCredential` / `AccessTokenCredential` models and `CredentialStorage` ABC |
|
||||
| `ssh_key` | `SshKeyPair` model and `SshKeyLifecycle` (Ed25519 generation via `cryptography`) |
|
||||
| `connection` | `RepositoryConnection` ORM mapping and `ConnectionManager` orchestration |
|
||||
| `operations` | Abstract `GitOperations` and concrete `LocalGitOperations` (subprocess-based `get_status`) |
|
||||
|
||||
Security rules for the package:
|
||||
- Credential models store **only** `encrypted_payload` — no plaintext `token` or `private_key` fields.
|
||||
- SSH private keys are encrypted before storage; the field uses `repr=False`.
|
||||
- SSH private keys are encrypted with Fernet before storage.
|
||||
|
||||
## Tool Spawn Workflow
|
||||
|
||||
The platform supports spawning development tools (e.g., code-server) as Docker containers via Docker Compose.
|
||||
|
||||
### Architecture
|
||||
|
||||
1. **Tool Manifest** (`apps/api/app/tools/manifests/*.yml`):
|
||||
- Defines Docker image, ports, volumes, environment variables, health checks
|
||||
- Loaded into in-memory registry at application startup
|
||||
|
||||
2. **Spawn Service** (`apps/api/app/services/spawn.py`):
|
||||
- Generates Docker Compose service definitions from manifests
|
||||
- Handles container lifecycle: spawn, stop, status polling
|
||||
- Integrates Traefik label generation for subdomain routing
|
||||
|
||||
3. **API Endpoints** (`apps/api/app/routers/tool_instances.py`):
|
||||
- `POST /projects/{id}/tool-instances` — Spawn a new tool instance
|
||||
- `POST /projects/{id}/tool-instances/{id}/stop` — Stop a running instance
|
||||
- `POST /projects/{id}/tool-instances/{id}/start` — Restart a stopped instance
|
||||
- `GET /projects/{id}/tool-instances/{id}/status` — Get container status
|
||||
|
||||
4. **Frontend UI**:
|
||||
- `/tools/spawn` — Form to select tool, project, and spawn
|
||||
- `/projects/{id}/instances/{id}` — Instance detail with status, controls, and "Open Tool" link
|
||||
|
||||
### Auth Proxy
|
||||
|
||||
Spawned tools are protected behind Traefik forwardAuth middleware:
|
||||
- Traefik forwards requests to `/api/v1/auth/validate` for session validation
|
||||
- code-server built-in auth is disabled (`PASSWORD: ""`)
|
||||
- Only authenticated platform users can access spawned tools
|
||||
|
||||
### Local Development
|
||||
|
||||
Ensure Docker socket is accessible and the `tools` network exists:
|
||||
```bash
|
||||
docker network create tools # One-time setup
|
||||
```
|
||||
|
||||
Spawned containers use the `tools` network for Traefik routing.
|
||||
|
||||
## Config & Secrets
|
||||
|
||||
### Overview
|
||||
|
||||
The platform supports scoped configuration values and encrypted secrets that are injected into tool containers at spawn time.
|
||||
|
||||
### Scopes
|
||||
|
||||
Configs and secrets support four scope levels (closest match wins):
|
||||
|
||||
1. **Global** — Available to all users and projects
|
||||
2. **User** — Available to a specific user across all projects
|
||||
3. **Project** — Available within a specific project
|
||||
4. **Instance** — Available to a specific tool instance
|
||||
|
||||
### Configs
|
||||
|
||||
Configs are plaintext JSON values mounted as files into containers:
|
||||
|
||||
- Mount path: `/app/config/<key>.json`
|
||||
- Permissions: `0400` (read-only, owner-only)
|
||||
- Scope resolution: instance > project > user > global
|
||||
|
||||
**API Endpoints:**
|
||||
- `POST /configs` — Create config
|
||||
- `GET /configs` — List configs (filter by scope_type, scope_id)
|
||||
- `PUT /configs/{id}` — Update config value
|
||||
- `DELETE /configs/{id}` — Delete config
|
||||
|
||||
**Frontend:**
|
||||
- `/projects/{id}/configs` — Config management UI
|
||||
|
||||
### Secrets
|
||||
|
||||
Secrets are encrypted with Fernet and injected as environment variables:
|
||||
|
||||
- Env var format: `<UPPERCASE_KEY>=<decrypted_value>`
|
||||
- Values are never sent to the frontend decrypted (displayed as `••••••`)
|
||||
- Scope resolution: instance > project > user > global
|
||||
|
||||
**API Endpoints:**
|
||||
- `POST /secrets` — Create secret
|
||||
- `GET /secrets` — List secrets (filter by scope_type, scope_id)
|
||||
- `PUT /secrets/{id}` — Update secret value
|
||||
- `DELETE /secrets/{id}` — Delete secret
|
||||
|
||||
**Frontend:**
|
||||
- `/projects/{id}/secrets` — Secret management UI
|
||||
|
||||
### Runtime Injection
|
||||
|
||||
When a tool instance is spawned:
|
||||
|
||||
1. Configs are resolved from all applicable scopes
|
||||
2. Secrets are resolved and decrypted
|
||||
3. Config files are generated in `/tmp/headquarter-configs/{instance_id}/`
|
||||
4. Config files are mounted as read-only volumes
|
||||
5. Secrets are injected as environment variables
|
||||
6. Missing required secrets will fail the spawn with a clear error
|
||||
|
||||
### Validation
|
||||
|
||||
Before spawning, the system validates that all required secrets exist. If any are missing, the spawn fails with an error message listing the missing secrets.
|
||||
|
||||
## Repository Connections
|
||||
|
||||
### Overview
|
||||
|
||||
The platform supports connecting Git repositories to projects with provider-independent authentication.
|
||||
|
||||
### Architecture
|
||||
|
||||
1. **Repository** (`apps/api/app/models/repository.py`):
|
||||
- Stores repository metadata (name, git_url, provider_type, default_branch)
|
||||
- Belongs to a project
|
||||
|
||||
2. **Repository Connection** (`apps/api/app/models/repository_connection.py`):
|
||||
- Links a repository to a Git provider with credentials
|
||||
- Tracks connection status (pending, connected, error, disconnected)
|
||||
- Supports SSH key authentication
|
||||
|
||||
3. **Credential Storage** (`apps/api/app/git/credential_storage.py`):
|
||||
- Database-backed storage for encrypted credentials
|
||||
- Uses Fernet encryption for payload
|
||||
- Supports access tokens and SSH keys
|
||||
|
||||
4. **Provider Adapters** (`apps/api/app/git/providers/`):
|
||||
- GitHubAdapter and GitLabAdapter with URL parsing
|
||||
- Extensible for other providers (Gitea, Forgejo)
|
||||
|
||||
5. **SSH Key Lifecycle** (`apps/api/app/git/ssh_key.py`):
|
||||
- Ed25519 key pair generation
|
||||
- Fernet-encrypted private key storage
|
||||
- Public key available for deploy key registration
|
||||
|
||||
### API Endpoints
|
||||
|
||||
- `POST /projects/{id}/repositories` — Add repository
|
||||
- `GET /projects/{id}/repositories` — List repositories
|
||||
- `DELETE /projects/{id}/repositories/{id}` — Remove repository
|
||||
- `POST /projects/{id}/repository-connections` — Create connection
|
||||
- `GET /projects/{id}/repository-connections` — List connections
|
||||
- `DELETE /projects/{id}/repository-connections/{id}` — Remove connection
|
||||
- `POST /projects/{id}/repository-connections/{id}/ssh-key` — Generate SSH key
|
||||
- `POST /projects/{id}/repository-connections/{id}/validate` — Validate connection
|
||||
|
||||
### Frontend
|
||||
|
||||
- `/repositories` — Repository list and creation
|
||||
- `/projects/{id}/repositories/{id}` — Repository detail with connections
|
||||
|
||||
### Git Operations
|
||||
|
||||
Local Git operations are supported via subprocess:
|
||||
- Clone, fetch, push with credential-aware subprocess
|
||||
- Working tree status (branch, clean, untracked, modified, staged, deleted)
|
||||
|
||||
## OpenCode Tool
|
||||
|
||||
### Overview
|
||||
|
||||
OpenCode is an AI-powered terminal-based development environment accessible via web browser. It is included as a built-in tool manifest alongside code-server.
|
||||
|
||||
### Manifest
|
||||
|
||||
**File:** `apps/api/app/tools/manifests/opencode.yml`
|
||||
|
||||
```yaml
|
||||
id: opencode
|
||||
name: OpenCode
|
||||
image: ghcr.io/opencode-ai/opencode:latest
|
||||
ports:
|
||||
- container_port: 3000
|
||||
primary: true
|
||||
```
|
||||
|
||||
### Web Terminal Access
|
||||
|
||||
OpenCode exposes a terminal interface on port 3000:
|
||||
- **Subdomain:** `opencode-{project}-{user}.{domain}`
|
||||
- **Health Check:** `GET /` on port 3000
|
||||
- **Terminal:** Full xterm-256color support with color output
|
||||
|
||||
### Environment Variables
|
||||
|
||||
The following environment variables are configured for terminal support:
|
||||
|
||||
| Variable | Value | Description |
|
||||
|----------|-------|-------------|
|
||||
| `TERM` | `xterm-256color` | Terminal type with color support |
|
||||
| `FORCE_COLOR` | `"1"` | Force color output |
|
||||
|
||||
### Workspace Mount
|
||||
|
||||
OpenCode mounts the project workspace at `/workspace` for persistent file access.
|
||||
|
||||
### Config Mount
|
||||
|
||||
User-specific OpenCode configuration is mounted at `/root/.config/opencode`.
|
||||
|
||||
### Usage
|
||||
|
||||
1. Navigate to `/tools/spawn` in the frontend
|
||||
2. Select "OpenCode" from the tool dropdown
|
||||
3. Choose a project
|
||||
4. Enter an instance name
|
||||
5. Click "Spawn Tool"
|
||||
6. Once running, click "Open Tool" to access the web terminal
|
||||
|
||||
### Differences from code-server
|
||||
|
||||
| Feature | OpenCode | code-server |
|
||||
|---------|----------|-------------|
|
||||
| Interface | Terminal (web-based) | VS Code (web-based) |
|
||||
| Port | 3000 | 8080 |
|
||||
| Primary Use | Terminal/CLI tasks | Code editing/IDE |
|
||||
| AI Features | Built-in AI assistance | Extensions required |
|
||||
|
||||
### Local Testing
|
||||
|
||||
To test OpenCode locally without the full platform:
|
||||
|
||||
```bash
|
||||
docker run -it --rm \
|
||||
-p 3000:3000 \
|
||||
-e TERM=xterm-256color \
|
||||
-e FORCE_COLOR=1 \
|
||||
ghcr.io/opencode-ai/opencode:latest
|
||||
```
|
||||
|
||||
Then open `http://localhost:3000` in your browser.
|
||||
@@ -1,184 +0,0 @@
|
||||
# Headquarter MVP Scope
|
||||
|
||||
> Canonical definition of what is in, out, and deferred for the Minimum Viable Product.
|
||||
> This document is the scope boundary for all downstream implementation tasks.
|
||||
|
||||
---
|
||||
|
||||
## 1. Product Vision
|
||||
|
||||
Headquarter is a hosted workspace and tool-orchestration platform for developers who want self-hosted control over their development environments. It gives authenticated users a single dashboard to create Git-backed projects, connect repositories from any provider, and spawn containerized tools—starting with OpenCode and code-server—on demand, each accessible via its own HTTPS subdomain. Headquarter is for individual developers and small teams who outgrow cloud IDEs but do not want to build their own orchestration layer from scratch.
|
||||
|
||||
---
|
||||
|
||||
## 2. MVP User Journeys
|
||||
|
||||
An MVP user can complete the following end-to-end flows without assistance:
|
||||
|
||||
### 2.1 Sign Up / Log In via Authentik
|
||||
- User clicks "Sign In" and is redirected to the organization's Authentik instance.
|
||||
- After OIDC authentication, the user is redirected back to the Headquarter dashboard.
|
||||
- A `User` row is created automatically on first login.
|
||||
|
||||
### 2.2 Create a Project
|
||||
- User clicks "New Project" and provides a name and optional description.
|
||||
- The backend generates a URL-friendly `slug` from the name.
|
||||
- The project appears in the user's project list.
|
||||
|
||||
### 2.3 Connect a Git Repository
|
||||
- User selects a project and chooses "Connect Repository."
|
||||
- User provides the Git clone URL and selects the provider type (GitHub, GitLab, Gitea, Forgejo, or generic).
|
||||
- The backend creates a `Repository` row and a `RepositoryConnection` row.
|
||||
|
||||
### 2.4 Generate Per-Repository SSH Credentials
|
||||
- User clicks "Generate SSH Key" for a repository connection.
|
||||
- The backend generates an Ed25519 key pair, encrypts the private key, and stores it.
|
||||
- The public key is displayed to the user for manual registration at the provider, or registered automatically via the provider adapter when available.
|
||||
|
||||
### 2.5 Spawn a Tool Instance
|
||||
- User navigates to "Tools" and selects a tool (OpenCode or code-server).
|
||||
- User chooses a project and optional config overrides.
|
||||
- The backend generates a Docker Compose service definition, Traefik labels, and starts the container.
|
||||
- The tool instance receives workspace mounts, config mounts, and secret injection.
|
||||
|
||||
### 2.6 Access the Running Tool via Subdomain
|
||||
- After the tool instance reaches `running` or `healthy` status, the user sees a link.
|
||||
- The link follows the subdomain pattern: `https://{tool}-{project}-{user}.{tool_domain}`.
|
||||
- Traefik routes the subdomain to the container's exposed port over HTTPS.
|
||||
|
||||
### 2.7 Stop and Restart a Tool Instance
|
||||
- User clicks "Stop" on a running tool instance.
|
||||
- The backend calls Docker to stop the container and updates the status to `stopped`.
|
||||
- User can click "Start" to re-provision the container with the same configuration.
|
||||
|
||||
### 2.8 Configure Tool Settings
|
||||
- User navigates to "Settings" for a project or their user profile.
|
||||
- User can create, update, or delete config values at project or user scope.
|
||||
- Config values are stored as JSON and mounted into tool containers at runtime.
|
||||
|
||||
### 2.9 Store and Inject Secrets
|
||||
- User navigates to "Secrets" for a project.
|
||||
- User creates a secret by providing a key name and value.
|
||||
- The backend encrypts the value with Fernet before storage.
|
||||
- At spawn time, the backend decrypts the secret and injects it as an environment variable or mounted file.
|
||||
|
||||
---
|
||||
|
||||
## 3. In-Scope Features
|
||||
|
||||
- **Authentik OIDC authentication** with automatic user provisioning
|
||||
- **Project management** (CRUD, ownership-based)
|
||||
- **Repository connections** with provider-agnostic Git URL storage
|
||||
- **Per-repository SSH key generation** (Ed25519) with encrypted private-key storage
|
||||
- **Tool registry** with manifest-driven definitions for OpenCode and code-server
|
||||
- **Tool instance spawning** via Docker Compose with Traefik subdomain routing
|
||||
- **Tool instance lifecycle** (start, stop, health checks, status tracking)
|
||||
- **Persistent config storage** at global, user, project, and tool-instance scopes
|
||||
- **Encrypted secret storage** at user, project, and tool-instance scopes
|
||||
- **Traefik label generation** for dynamic subdomain routing
|
||||
- **Local development stack** via Docker Compose (API, web, PostgreSQL)
|
||||
- **Deployment skeleton** for Portainer-managed production stacks
|
||||
|
||||
---
|
||||
|
||||
## 4. Out-of-Scope Features (Non-Goals)
|
||||
|
||||
The following are explicitly excluded from MVP to prevent scope creep:
|
||||
|
||||
- **Multi-user teams / shared projects** — Schema leaves room for `ProjectMember`, but no UI or API in MVP
|
||||
- **Real-time collaboration** — No shared cursors, simultaneous editing, or presence
|
||||
- **Advanced CI/CD pipelines** — No build orchestration, test runners, or deployment stages
|
||||
- **Kubernetes runtime** — Docker Compose only; Kubernetes adapter is a future extension point
|
||||
- **Non-Docker runtimes** — No Podman, LXC, or VM runtimes in MVP
|
||||
- **Automatic Git provider webhooks** — No push-triggered actions or webhook receivers
|
||||
- **Built-in GitHub/GitLab UI integrations** — No issue trackers, PR viewers, or code review UI
|
||||
- **Backup and disaster recovery automation** — Rely on host-level volume backups
|
||||
- **High availability / replicas** — Single-instance deployment only
|
||||
- **Rate limiting** — No API or Traefik rate limits in MVP
|
||||
- **Audit logging** — No immutable audit trail of user actions
|
||||
- **Automatic credential rotation** — Manual rotation only
|
||||
- **Container image vulnerability scanning** — No image trust enforcement
|
||||
|
||||
---
|
||||
|
||||
## 5. MVP Milestones / Slices
|
||||
|
||||
Slices are ordered by dependency. Each slice corresponds to a task on the Fusion board.
|
||||
|
||||
| Slice | Task ID | Title | Deliverable |
|
||||
|-------|---------|-------|-------------|
|
||||
| 1 | **FN-002** | Monorepo Scaffold | Root tooling, frontend/backend skeletons, Docker Compose, deployment skeleton |
|
||||
| 2 | **FN-019** | Architecture & Specification | Enhanced `docs/architecture.md`, `docs/mvp-scope.md`, doc validation tests |
|
||||
| 3 | **FN-004** | Backend Foundation | Domain models, Alembic migrations, auth boundaries, secret encryption, API routers |
|
||||
| 4 | **FN-005** | Frontend Foundation | Auth shell, navigation, placeholder pages, API client, config layer |
|
||||
| 5 | **FN-003** | Tool Registry | Manifest schema, in-memory registry, built-in OpenCode/code-server manifests, FastAPI routes |
|
||||
| 6 | **FN-006** | Deployment Config | Traefik label generator, production Compose stacks, Portainer stack definition |
|
||||
| 7 | **FN-011** | Git Provider Model | Provider abstraction, SSH key lifecycle, credential models, repository connection |
|
||||
| 8 | **FN-009** | Config & Secrets | Encrypted storage, runtime injection, frontend config/secrets UI |
|
||||
| 9 | **FN-010** | code-server Spawn | code-server manifest, spawn flow, runtime integration, auth layer |
|
||||
| 10 | **FN-008** | OpenCode POC | AI-powered terminal environment, web interface, health reporting |
|
||||
|
||||
**Dependency notes:**
|
||||
- FN-004 and FN-005 can proceed in parallel once FN-019 is complete.
|
||||
- FN-003 depends on FN-004 (backend models exist).
|
||||
- FN-006 depends on FN-002 (scaffold exists) and benefits from FN-003 (manifest routing fields).
|
||||
- FN-011 depends on FN-004 (models and test infrastructure).
|
||||
- FN-009 depends on FN-004 and FN-005.
|
||||
- FN-010 depends on FN-003, FN-006, and FN-009.
|
||||
- FN-008 depends on FN-003, FN-006, and FN-009.
|
||||
|
||||
---
|
||||
|
||||
## 6. Dependency Order for Downstream Implementation
|
||||
|
||||
```
|
||||
FN-002 (Scaffold)
|
||||
│
|
||||
├──> FN-019 (Architecture) ──> FN-004 (Backend)
|
||||
│ │
|
||||
│ ├──> FN-003 (Tool Registry)
|
||||
│ │ │
|
||||
│ │ ├──> FN-010 (code-server Spawn)
|
||||
│ │ └──> FN-008 (OpenCode POC)
|
||||
│ │
|
||||
│ ├──> FN-011 (Git Provider)
|
||||
│ │
|
||||
│ └──> FN-009 (Config/Secrets)
|
||||
│ │
|
||||
│ └──> FN-010, FN-008 (runtime injection)
|
||||
│
|
||||
└──> FN-005 (Frontend) ───────> FN-009 (Config/Secrets UI)
|
||||
|
||||
FN-006 (Deployment) runs in parallel with FN-004/FN-005
|
||||
after FN-002 is complete.
|
||||
```
|
||||
|
||||
**Critical path:** FN-002 → FN-019 → FN-004 → FN-003 → FN-010/FN-008
|
||||
|
||||
---
|
||||
|
||||
## 7. Definition of MVP Done
|
||||
|
||||
MVP is complete and shippable when **all** of the following are true:
|
||||
|
||||
1. A user can sign up, create a project, connect a Git repository, and spawn code-server from a single dashboard.
|
||||
2. Spawned tools are accessible via HTTPS subdomains routed through Traefik.
|
||||
3. Secrets and configs are encrypted at rest and injected correctly at runtime.
|
||||
4. SSH keys are generated per repository and used for Git operations inside containers.
|
||||
5. All backend tests pass (`pytest`), all frontend tests pass (`vitest`), and all lint/typecheck gates pass.
|
||||
6. The production stack (`docker-compose.prod.yml`) deploys cleanly via Portainer.
|
||||
7. Documentation (`architecture.md`, `mvp-scope.md`, `deployment.md`, `development.md`) is accurate and consistent with the implementation.
|
||||
8. No incomplete placeholders or task markers remain in committed code or documentation.
|
||||
|
||||
---
|
||||
|
||||
## 8. Open Questions
|
||||
|
||||
The following scope decisions are pending stakeholder input. Implementers should not choose defaults for these without explicit approval:
|
||||
|
||||
1. **Admin role in MVP:** Do we need a basic admin role for global config management, or can all authenticated users write global config in MVP?
|
||||
2. **User slug derivation:** Should the user slug for subdomain generation be derived from `display_name`, `email` local-part, or a new dedicated `slug` column?
|
||||
3. **Provider adapter coverage:** Which Git providers get concrete adapters in MVP? GitHub and GitLab are assumed; Gitea and Forgejo may be deferred.
|
||||
4. **Auto-deploy-key registration:** Should the platform attempt to register deploy keys automatically via provider APIs, or is manual copy-paste acceptable for MVP?
|
||||
5. **Container image trust:** Should the platform restrict tool images to an allow-list in production, or is any image reference acceptable in MVP?
|
||||
6. **Billing or resource quotas:** Is any form of usage limiting or project quota needed in MVP, or is it strictly single-user-unlimited?
|
||||
@@ -1,31 +0,0 @@
|
||||
# Project Brief: Headquarter
|
||||
|
||||
## What
|
||||
|
||||
Headquarter is a hosted workspace and tool-orchestration platform. Authenticated users create Git-backed projects and launch containerized development tools—starting with OpenCode and code-server—each exposed via its own HTTPS subdomain.
|
||||
|
||||
## Why
|
||||
|
||||
Cloud IDEs and CI dashboards are convenient but lock users into proprietary platforms. Headquarter gives developers the same convenience with full control over their runtime environments, source code, and routing.
|
||||
|
||||
## Who
|
||||
|
||||
- Individual developers who want self-hosted workspaces
|
||||
- Small teams that outgrow cloud IDEs but do not want to build orchestration from scratch
|
||||
- Operators who prefer Docker Compose and Traefik over Kubernetes for simple deployments
|
||||
|
||||
## Confirmed Stack
|
||||
|
||||
- **Frontend:** React + Vite + TypeScript
|
||||
- **Backend:** FastAPI + SQLAlchemy 2.0 + Pydantic v2
|
||||
- **Database:** PostgreSQL 17
|
||||
- **Auth:** Authentik OIDC
|
||||
- **Runtime:** Docker Compose (Portainer-managed)
|
||||
- **Routing:** Traefik subdomain-based
|
||||
|
||||
## Canonical Documentation
|
||||
|
||||
- [Architecture](architecture.md) — System design, domain model, and provider contracts
|
||||
- [MVP Scope](mvp-scope.md) — In-scope features, non-goals, milestones, and dependency order
|
||||
- [Development](development.md) — Local setup and day-to-day commands
|
||||
- [Deployment](deployment.md) — Production deployment assumptions and operator guide
|
||||
@@ -0,0 +1,726 @@
|
||||
# Docker Infrastructure Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Set up complete Docker-based development environment with PostgreSQL, Redis, Traefik, Authentik, FastAPI backend, and React frontend.
|
||||
|
||||
**Architecture:** Multi-service Docker Compose setup with Traefik as reverse proxy, PostgreSQL for data, Redis for caching, Authentik for auth, FastAPI backend, and Vite React frontend. All services include health checks and persistent volumes.
|
||||
|
||||
**Tech Stack:** Docker 24.0+, Docker Compose 2.20+, Make, Python 3.11+, Node.js 20+
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
/
|
||||
├── docker-compose.yml # All services orchestration
|
||||
├── .env.example # Required environment variables
|
||||
├── Makefile # Common commands
|
||||
├── apps/
|
||||
│ ├── api/
|
||||
│ │ ├── Dockerfile # Multi-stage Python build
|
||||
│ │ └── pyproject.toml # Python dependencies (placeholder)
|
||||
│ └── web/
|
||||
│ ├── Dockerfile # Multi-stage Node build
|
||||
│ └── package.json # Node dependencies (placeholder)
|
||||
└── data/
|
||||
└── repos/ # Git repository storage volume
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Create Project Directory Structure
|
||||
|
||||
**Files:**
|
||||
- Create: `apps/api/`
|
||||
- Create: `apps/web/`
|
||||
- Create: `data/repos/`
|
||||
|
||||
- [ ] **Step 1: Create directory structure**
|
||||
|
||||
```bash
|
||||
mkdir -p apps/api apps/web data/repos
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create placeholder files for Docker build context**
|
||||
|
||||
Create `apps/api/pyproject.toml`:
|
||||
```toml
|
||||
[project]
|
||||
name = "headquarter-api"
|
||||
version = "0.1.0"
|
||||
description = "Headquarter platform API"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"fastapi>=0.104.0",
|
||||
"uvicorn[standard]>=0.24.0",
|
||||
"sqlalchemy>=2.0.0",
|
||||
"asyncpg>=0.29.0",
|
||||
"alembic>=1.12.0",
|
||||
"pydantic>=2.5.0",
|
||||
"pydantic-settings>=2.1.0",
|
||||
"python-jose[cryptography]>=3.3.0",
|
||||
"python-multipart>=0.0.6",
|
||||
"httpx>=0.25.0",
|
||||
"structlog>=23.2.0",
|
||||
"cryptography>=41.0.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=7.4.0",
|
||||
"pytest-asyncio>=0.21.0",
|
||||
"mypy>=1.7.0",
|
||||
"ruff>=0.1.0",
|
||||
"httpx>=0.25.0",
|
||||
]
|
||||
```
|
||||
|
||||
Create `apps/web/package.json`:
|
||||
```json
|
||||
{
|
||||
"name": "headquarter-web",
|
||||
"version": "0.1.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "tsc && vite build",
|
||||
"preview": "vite preview",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0"
|
||||
},
|
||||
"dependencies": {
|
||||
"react": "^18.2.0",
|
||||
"react-dom": "^18.2.0",
|
||||
"react-router-dom": "^6.20.0",
|
||||
"axios": "^1.6.0",
|
||||
"tailwindcss": "^3.3.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/react": "^18.2.0",
|
||||
"@types/react-dom": "^18.2.0",
|
||||
"@vitejs/plugin-react": "^4.2.0",
|
||||
"typescript": "^5.3.0",
|
||||
"vite": "^5.0.0",
|
||||
"eslint": "^8.55.0",
|
||||
"@typescript-eslint/eslint-plugin": "^6.14.0",
|
||||
"@typescript-eslint/parser": "^6.14.0",
|
||||
"autoprefixer": "^10.4.16",
|
||||
"postcss": "^8.4.32"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add apps/ data/
|
||||
git commit -m "chore: create project directory structure"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Create API Dockerfile
|
||||
|
||||
**Files:**
|
||||
- Create: `apps/api/Dockerfile`
|
||||
|
||||
- [ ] **Step 1: Write multi-stage API Dockerfile**
|
||||
|
||||
```dockerfile
|
||||
# Build stage
|
||||
FROM python:3.11-slim as builder
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
# Install build dependencies
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
gcc \
|
||||
libpq-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install Python dependencies
|
||||
COPY pyproject.toml .
|
||||
RUN pip install --no-cache-dir --user -e ".[dev]"
|
||||
|
||||
# Production stage
|
||||
FROM python:3.11-slim
|
||||
|
||||
# Create non-root user
|
||||
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install runtime dependencies
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
libpq5 \
|
||||
git \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Copy dependencies from builder
|
||||
COPY --from=builder /root/.local /home/appuser/.local
|
||||
ENV PATH=/home/appuser/.local/bin:$PATH
|
||||
|
||||
# Copy application code
|
||||
COPY --chown=appuser:appgroup . .
|
||||
|
||||
# Create directories for repo storage
|
||||
RUN mkdir -p /data/repos && chown -R appuser:appgroup /data/repos
|
||||
|
||||
# Switch to non-root user
|
||||
USER appuser
|
||||
|
||||
# Expose port
|
||||
EXPOSE 8000
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1
|
||||
|
||||
# Run the application
|
||||
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add apps/api/Dockerfile
|
||||
git commit -m "feat: add multi-stage API Dockerfile"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Create Web Frontend Dockerfile
|
||||
|
||||
**Files:**
|
||||
- Create: `apps/web/Dockerfile`
|
||||
- Create: `apps/web/nginx.conf`
|
||||
|
||||
- [ ] **Step 1: Write multi-stage Web Dockerfile**
|
||||
|
||||
```dockerfile
|
||||
# Build stage
|
||||
FROM node:20-alpine as builder
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
# Copy package files
|
||||
COPY package.json package-lock.json* ./
|
||||
|
||||
# Install dependencies
|
||||
RUN npm ci
|
||||
|
||||
# Copy source code
|
||||
COPY . .
|
||||
|
||||
# Build production bundle
|
||||
RUN npm run build
|
||||
|
||||
# Production stage
|
||||
FROM nginx:alpine
|
||||
|
||||
# Create non-root user
|
||||
RUN addgroup -g 1001 -S nodejs && adduser -S nextjs -u 1001
|
||||
|
||||
# Copy custom nginx config
|
||||
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
||||
|
||||
# Copy built assets from builder
|
||||
COPY --from=builder --chown=nextjs:nodejs /build/dist /usr/share/nginx/html
|
||||
|
||||
# Create required directories
|
||||
RUN mkdir -p /var/cache/nginx /var/run && \
|
||||
chown -R nextjs:nodejs /var/cache/nginx /var/run /usr/share/nginx/html
|
||||
|
||||
# Switch to non-root user
|
||||
USER nextjs
|
||||
|
||||
# Expose port
|
||||
EXPOSE 80
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD wget --quiet --tries=1 --spider http://localhost/ || exit 1
|
||||
|
||||
# Start nginx
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write nginx configuration**
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name localhost;
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
|
||||
# Enable gzip compression
|
||||
gzip on;
|
||||
gzip_vary on;
|
||||
gzip_min_length 1024;
|
||||
gzip_types text/plain text/css application/json application/javascript text/xml application/xml;
|
||||
|
||||
# Handle client-side routing
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
||||
# Cache static assets
|
||||
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2)$ {
|
||||
expires 1y;
|
||||
add_header Cache-Control "public, immutable";
|
||||
}
|
||||
|
||||
# Health check endpoint
|
||||
location /health {
|
||||
access_log off;
|
||||
return 200 "healthy\n";
|
||||
add_header Content-Type text/plain;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add apps/web/Dockerfile apps/web/nginx.conf
|
||||
git commit -m "feat: add multi-stage web frontend Dockerfile with nginx"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Create Docker Compose Configuration
|
||||
|
||||
**Files:**
|
||||
- Create: `docker-compose.yml`
|
||||
|
||||
- [ ] **Step 1: Write Docker Compose file**
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
# PostgreSQL Database
|
||||
postgres:
|
||||
image: postgres:15-alpine
|
||||
container_name: hq-postgres
|
||||
environment:
|
||||
POSTGRES_USER: ${POSTGRES_USER:-headquarter}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-headquarter}
|
||||
POSTGRES_DB: ${POSTGRES_DB:-headquarter}
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
- ./init-scripts:/docker-entrypoint-initdb.d:ro
|
||||
ports:
|
||||
- "5432:5432"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-headquarter} -d ${POSTGRES_DB:-headquarter}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 10s
|
||||
networks:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
|
||||
# Redis Cache
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
container_name: hq-redis
|
||||
command: redis-server --appendonly yes
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
ports:
|
||||
- "6379:6379"
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 5s
|
||||
networks:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
|
||||
# Traefik Reverse Proxy
|
||||
traefik:
|
||||
image: traefik:v3.0
|
||||
container_name: hq-traefik
|
||||
command:
|
||||
- "--api.dashboard=true"
|
||||
- "--providers.docker=true"
|
||||
- "--providers.docker.exposedbydefault=false"
|
||||
- "--entrypoints.web.address=:80"
|
||||
- "--entrypoints.websecure.address=:443"
|
||||
- "--ping=true"
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
- "8080:8080" # Dashboard
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
- ./traefik:/etc/traefik:ro
|
||||
healthcheck:
|
||||
test: ["CMD", "traefik", "healthcheck"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 5s
|
||||
networks:
|
||||
- frontend
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.traefik.rule=Host(`traefik.hq.local`)"
|
||||
- "traefik.http.routers.traefik.service=api@internal"
|
||||
- "traefik.http.routers.traefik.entrypoints=web"
|
||||
|
||||
# Authentik - Authentication Server
|
||||
authentik-server:
|
||||
image: ghcr.io/goauthentik/server:2024.2
|
||||
container_name: hq-authentik
|
||||
command: server
|
||||
environment:
|
||||
AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY:-change-me-in-production}
|
||||
AUTHENTIK_REDIS__HOST: redis
|
||||
AUTHENTIK_POSTGRESQL__HOST: postgres
|
||||
AUTHENTIK_POSTGRESQL__NAME: ${POSTGRES_DB:-headquarter}
|
||||
AUTHENTIK_POSTGRESQL__USER: ${POSTGRES_USER:-headquarter}
|
||||
AUTHENTIK_POSTGRESQL__PASSWORD: ${POSTGRES_PASSWORD:-headquarter}
|
||||
volumes:
|
||||
- ./authentik/media:/media
|
||||
- ./authentik/custom-templates:/templates
|
||||
ports:
|
||||
- "9000:9000"
|
||||
- "9443:9443"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend
|
||||
- frontend
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.authentik.rule=Host(`auth.hq.local`)"
|
||||
- "traefik.http.routers.authentik.entrypoints=web"
|
||||
- "traefik.http.services.authentik.loadbalancer.server.port=9000"
|
||||
|
||||
# API Service
|
||||
api:
|
||||
build:
|
||||
context: ./apps/api
|
||||
dockerfile: Dockerfile
|
||||
container_name: hq-api
|
||||
environment:
|
||||
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-headquarter}:${POSTGRES_PASSWORD:-headquarter}@postgres:5432/${POSTGRES_DB:-headquarter}
|
||||
REDIS_URL: redis://redis:6379/0
|
||||
AUTHENTIK_URL: http://authentik-server:9000
|
||||
AUTHENTIK_CLIENT_ID: ${AUTHENTIK_CLIENT_ID}
|
||||
AUTHENTIK_CLIENT_SECRET: ${AUTHENTIK_CLIENT_SECRET}
|
||||
JWT_SECRET: ${JWT_SECRET:-change-me-in-production}
|
||||
REPO_BASE_PATH: /data/repos
|
||||
volumes:
|
||||
- repo_data:/data/repos
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
authentik-server:
|
||||
condition: service_started
|
||||
networks:
|
||||
- backend
|
||||
- frontend
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.api.rule=Host(`api.hq.local`)"
|
||||
- "traefik.http.routers.api.entrypoints=web"
|
||||
- "traefik.http.services.api.loadbalancer.server.port=8000"
|
||||
|
||||
# Web Frontend
|
||||
web:
|
||||
build:
|
||||
context: ./apps/web
|
||||
dockerfile: Dockerfile
|
||||
container_name: hq-web
|
||||
environment:
|
||||
VITE_API_URL: http://api.hq.local
|
||||
VITE_AUTH_URL: http://auth.hq.local
|
||||
depends_on:
|
||||
- api
|
||||
networks:
|
||||
- frontend
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.web.rule=Host(`hq.local`) || Host(`www.hq.local`)"
|
||||
- "traefik.http.routers.web.entrypoints=web"
|
||||
- "traefik.http.services.web.loadbalancer.server.port=80"
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
redis_data:
|
||||
repo_data:
|
||||
|
||||
networks:
|
||||
frontend:
|
||||
driver: bridge
|
||||
backend:
|
||||
driver: bridge
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add docker-compose.yml
|
||||
git commit -m "feat: add Docker Compose with all services"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Create Environment Configuration Template
|
||||
|
||||
**Files:**
|
||||
- Create: `.env.example`
|
||||
|
||||
- [ ] **Step 1: Write environment template**
|
||||
|
||||
```bash
|
||||
# Database Configuration
|
||||
POSTGRES_USER=headquarter
|
||||
POSTGRES_PASSWORD=change-me-in-production
|
||||
POSTGRES_DB=headquarter
|
||||
|
||||
# Redis Configuration
|
||||
REDIS_URL=redis://redis:6379/0
|
||||
|
||||
# Authentik Configuration
|
||||
AUTHENTIK_SECRET_KEY=change-me-in-production
|
||||
AUTHENTIK_CLIENT_ID=your-authentik-client-id
|
||||
AUTHENTIK_CLIENT_SECRET=your-authentik-client-secret
|
||||
AUTHENTIK_URL=http://auth.hq.local
|
||||
|
||||
# JWT Configuration
|
||||
JWT_SECRET=change-me-in-production
|
||||
JWT_ALGORITHM=HS256
|
||||
JWT_EXPIRATION_HOURS=24
|
||||
|
||||
# Application Configuration
|
||||
APP_ENV=development
|
||||
DEBUG=true
|
||||
LOG_LEVEL=info
|
||||
REPO_BASE_PATH=/data/repos
|
||||
|
||||
# Frontend Configuration
|
||||
VITE_API_URL=http://api.hq.local
|
||||
VITE_AUTH_URL=http://auth.hq.local
|
||||
|
||||
# Docker Configuration
|
||||
COMPOSE_PROJECT_NAME=headquarter
|
||||
DOCKER_NETWORK=headquarter_default
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add .env.example
|
||||
git commit -m "docs: add environment configuration template"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Create Makefile
|
||||
|
||||
**Files:**
|
||||
- Create: `Makefile`
|
||||
|
||||
- [ ] **Step 1: Write Makefile**
|
||||
|
||||
```makefile
|
||||
.PHONY: help up down logs migrate test lint clean build
|
||||
|
||||
# Default target
|
||||
help:
|
||||
@echo "Headquarter Development Commands"
|
||||
@echo "================================"
|
||||
@echo "make up - Start all services"
|
||||
@echo "make down - Stop all services"
|
||||
@echo "make logs - View service logs"
|
||||
@echo "make migrate - Run database migrations"
|
||||
@echo "make test - Run test suites"
|
||||
@echo "make lint - Run linting"
|
||||
@echo "make build - Build all Docker images"
|
||||
@echo "make clean - Remove containers and volumes"
|
||||
@echo "make shell - Open shell in API container"
|
||||
|
||||
# Start services
|
||||
up:
|
||||
docker compose up -d
|
||||
@echo "Services starting..."
|
||||
@echo "API: http://api.hq.local"
|
||||
@echo "Web: http://hq.local"
|
||||
@echo "Auth: http://auth.hq.local"
|
||||
@echo "Traefik: http://traefik.hq.local:8080"
|
||||
|
||||
# Stop services
|
||||
down:
|
||||
docker compose down
|
||||
|
||||
# View logs
|
||||
logs:
|
||||
docker compose logs -f
|
||||
|
||||
# View specific service logs
|
||||
logs-api:
|
||||
docker compose logs -f api
|
||||
|
||||
logs-web:
|
||||
docker compose logs -f web
|
||||
|
||||
logs-db:
|
||||
docker compose logs -f postgres
|
||||
|
||||
# Run database migrations
|
||||
migrate:
|
||||
docker compose exec api alembic upgrade head
|
||||
|
||||
# Create new migration
|
||||
migration:
|
||||
docker compose exec api alembic revision --autogenerate -m "$(message)"
|
||||
|
||||
# Run tests
|
||||
test:
|
||||
docker compose exec api pytest -v
|
||||
|
||||
# Run linting
|
||||
lint:
|
||||
docker compose exec api ruff check .
|
||||
docker compose exec api mypy .
|
||||
cd apps/web && npm run lint
|
||||
|
||||
# Type checking
|
||||
typecheck:
|
||||
docker compose exec api mypy .
|
||||
cd apps/web && npm run typecheck
|
||||
|
||||
# Build all images
|
||||
build:
|
||||
docker compose build
|
||||
|
||||
# Build specific service
|
||||
build-api:
|
||||
docker compose build api
|
||||
|
||||
build-web:
|
||||
docker compose build web
|
||||
|
||||
# Clean up
|
||||
clean:
|
||||
docker compose down -v --remove-orphans
|
||||
docker system prune -f
|
||||
|
||||
# Open shell in API container
|
||||
shell:
|
||||
docker compose exec api /bin/sh
|
||||
|
||||
# Database shell
|
||||
db-shell:
|
||||
docker compose exec postgres psql -U $(POSTGRES_USER) -d $(POSTGRES_DB)
|
||||
|
||||
# Health check
|
||||
health:
|
||||
@echo "Checking service health..."
|
||||
@docker compose ps
|
||||
@docker compose exec api wget -qO- http://localhost:8000/health || echo "API health check failed"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add Makefile
|
||||
git commit -m "feat: add Makefile with common development commands"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Validate Docker Configuration
|
||||
|
||||
**Files:**
|
||||
- Test: `docker-compose.yml`
|
||||
|
||||
- [ ] **Step 1: Validate Docker Compose syntax**
|
||||
|
||||
```bash
|
||||
docker compose config
|
||||
```
|
||||
|
||||
Expected: Valid YAML output with all services configured
|
||||
|
||||
- [ ] **Step 2: Test build**
|
||||
|
||||
```bash
|
||||
make build
|
||||
```
|
||||
|
||||
Expected: Both API and web images build successfully (may warn about missing source files - that's OK)
|
||||
|
||||
- [ ] **Step 3: Test start/stop**
|
||||
|
||||
```bash
|
||||
make up
|
||||
sleep 10
|
||||
make down
|
||||
```
|
||||
|
||||
Expected: Services start (Postgres and Redis should be healthy), then stop cleanly
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "test: validate Docker infrastructure"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
Run these checks to verify everything works:
|
||||
|
||||
1. **Syntax validation:**
|
||||
```bash
|
||||
docker compose config > /dev/null && echo "Valid"
|
||||
```
|
||||
|
||||
2. **Health checks:**
|
||||
```bash
|
||||
make up
|
||||
docker compose ps
|
||||
```
|
||||
All services should show "healthy" or "running"
|
||||
|
||||
3. **Makefile commands:**
|
||||
```bash
|
||||
make help # Shows usage
|
||||
make build # Builds images
|
||||
make up # Starts services
|
||||
make logs # Shows logs
|
||||
make down # Stops services
|
||||
```
|
||||
|
||||
## Quality Gates
|
||||
|
||||
- [ ] `docker compose config` validates without errors
|
||||
- [ ] All services have health checks defined
|
||||
- [ ] API Dockerfile uses multi-stage build with non-root user
|
||||
- [ ] Web Dockerfile uses multi-stage build with non-root user
|
||||
- [ ] Makefile includes all required commands (up, down, logs, migrate, test, lint)
|
||||
- [ ] .env.example documents all required variables
|
||||
- [ ] Persistent volume for `/data/repos`
|
||||
@@ -1,205 +0,0 @@
|
||||
# Tool Manifest Specification
|
||||
|
||||
> Canonical schema reference for Headquarter's manifest-driven tool registry.
|
||||
> Version: 1.0.0 — aligned with FN-003.
|
||||
|
||||
## Overview
|
||||
|
||||
Headquarter is a manifest-driven platform: every containerized tool (OpenCode, code-server, and future tools) is declared by a YAML manifest. The orchestration backend reads these manifests to generate Docker Compose services, Traefik routing labels, volume mounts, and resource constraints.
|
||||
|
||||
**Design goal:** Adding a new standard container tool requires only a YAML manifest—no backend code changes.
|
||||
|
||||
## Manifest File Format
|
||||
|
||||
Manifests are YAML files with a single top-level mapping. They are validated on load by Pydantic v2 models.
|
||||
|
||||
### Built-in location
|
||||
|
||||
Built-in manifests live in `apps/api/app/tools/manifests/*.yml` and are loaded automatically on API startup.
|
||||
|
||||
### Minimal valid manifest
|
||||
|
||||
```yaml
|
||||
id: my-tool
|
||||
name: My Tool
|
||||
image: my-org/my-tool:latest
|
||||
ports:
|
||||
- container_port: 8080
|
||||
primary: true
|
||||
traefik:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
## Field Reference
|
||||
|
||||
### `ToolManifest` (top level)
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `id` | `string` | yes | — | Lowercase slug with hyphens only (`^[a-z0-9\-]+$`). Used as the registry key. |
|
||||
| `name` | `string` | yes | — | Human-readable tool name. |
|
||||
| `description` | `string` | no | `""` | Short description of the tool. |
|
||||
| `version` | `string` | no | `"1.0.0"` | Manifest version (semver-ish). |
|
||||
| `image` | `string` | yes | — | Docker image reference. |
|
||||
| `runtime_command` | `string[] \| null` | no | `null` | Override the container default command. |
|
||||
| `runtime_entrypoint` | `string[] \| null` | no | `null` | Override the container entrypoint. |
|
||||
| `runtime_user` | `string \| null` | no | `null` | User to run as inside the container. |
|
||||
| `runtime_working_dir` | `string \| null` | no | `null` | Working directory inside the container. |
|
||||
| `ports` | `PortConfig[]` | no | `[]` | Exposed ports. |
|
||||
| `workspace_mounts` | `MountConfig[]` | no | `[]` | Workspace volume mounts (project-scoped). |
|
||||
| `config_mounts` | `MountConfig[]` | no | `[]` | Config volume mounts (user or tool-scoped). |
|
||||
| `env` | `dict<string, string>` | no | `{}` | Static environment variables. |
|
||||
| `secrets` | `SecretRef[]` | no | `[]` | Secrets injected as environment variables. |
|
||||
| `health_check` | `HealthCheckConfig \| null` | no | `null` | Health check definition. |
|
||||
| `resource_limits` | `ResourceLimits \| null` | no | `null` | CPU and memory constraints. |
|
||||
| `executable` | `ExecutableConfig \| null` | no | `null` | Node.js runtime metadata for executable environments. |
|
||||
| `traefik` | `TraefikConfig \| null` | no | `null` | Traefik routing configuration. |
|
||||
|
||||
### `PortConfig`
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `container_port` | `int` | yes | — | Port inside the container. Range: 1–65535. |
|
||||
| `protocol` | `"tcp" \| "udp"` | no | `"tcp"` | Transport protocol. |
|
||||
| `name` | `string \| null` | no | `null` | Logical name, e.g. `"http"`, `"websocket"`. |
|
||||
| `primary` | `bool` | no | `false` | The port used for default routing and health checks. |
|
||||
|
||||
### `MountConfig`
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `type` | `"volume" \| "bind"` | no | `"volume"` | Mount type. |
|
||||
| `source_pattern` | `string` | yes | — | Template pattern resolved at spawn time, e.g. `"{project_repo}"`. |
|
||||
| `target` | `string` | yes | — | Absolute path inside the container. Must start with `/`. |
|
||||
| `read_only` | `bool` | no | `false` | Mount read-only. |
|
||||
|
||||
### `SecretRef`
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `name` | `string` | yes | — | Secret identifier in the secret store. |
|
||||
| `env_var` | `string` | yes | — | Name of the environment variable injected into the container. |
|
||||
| `required` | `bool` | no | `true` | Whether the tool fails to start if the secret is missing. |
|
||||
|
||||
### `HealthCheckConfig`
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `type` | `"http" \| "tcp" \| "command"` | no | `"http"` | Health check mechanism. |
|
||||
| `path` | `string \| null` | no | `null` | HTTP path. Required when `type == "http"`. |
|
||||
| `command` | `string[] \| null` | no | `null` | Command to execute. Required when `type == "command"`. |
|
||||
| `port` | `int \| null` | no | `null` | Override port; defaults to the primary port if unset. |
|
||||
| `interval_seconds` | `int` | no | `10` | Check interval. ≥ 1. |
|
||||
| `timeout_seconds` | `int` | no | `5` | Check timeout. ≥ 1. |
|
||||
| `retries` | `int` | no | `3` | Retries before marking unhealthy. ≥ 1. |
|
||||
| `start_period_seconds` | `int` | no | `5` | Grace period before checks count. ≥ 0. |
|
||||
|
||||
### `ResourceLimits`
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `cpus` | `float \| null` | no | `null` | CPU limit. If set, ≥ 0.01. |
|
||||
| `memory_mb` | `int \| null` | no | `null` | Memory limit in MiB. If set, ≥ 16. |
|
||||
| `memory_swap_mb` | `int \| null` | no | `null` | Swap limit in MiB. `-1` disables swap limit. |
|
||||
|
||||
### `ExecutableConfig`
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `node_version` | `string \| null` | no | `null` | Expected Node.js version, e.g. `"22"`, `"lts"`. |
|
||||
| `npm_version` | `string \| null` | no | `null` | Expected npm version. |
|
||||
| `package_manager` | `"npm" \| "pnpm" \| "yarn" \| "bun"` | no | `"npm"` | Preferred package manager. |
|
||||
| `bootstrap_commands` | `string[]` | no | `[]` | One-time setup commands run on first start. |
|
||||
| `install_commands` | `string[]` | no | `[]` | Commands run before the main command. |
|
||||
|
||||
### `TraefikConfig`
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `enabled` | `bool` | no | `true` | Whether Traefik routing is generated for this tool. |
|
||||
| `subdomain_prefix` | `string \| null` | no | `null` | Subdomain prefix. Defaults to the tool `id`. |
|
||||
| `port` | `int \| null` | no | `null` | Container port to route traffic to. |
|
||||
| `middlewares` | `string[]` | no | `[]` | Traefik middleware names to apply. |
|
||||
| `strip_prefix` | `bool` | no | `false` | Strip path prefix before forwarding. |
|
||||
| `entrypoint` | `string \| null` | no | `null` | Override the environment default Traefik entrypoint. |
|
||||
| `cert_resolver` | `string \| null` | no | `null` | Override the environment default cert resolver. |
|
||||
|
||||
## Validation Rules
|
||||
|
||||
1. `id` must match `^[a-z0-9\-]+$` (lowercase, digits, hyphens only).
|
||||
2. `MountConfig.target` must be an absolute path (`starts with "/"`).
|
||||
3. When `health_check.type == "http"`, `path` must be set and non-empty.
|
||||
4. When `health_check.type == "command"`, `command` must be set and non-empty.
|
||||
5. `container_port` must be between 1 and 65535.
|
||||
6. `cpus`, if set, must be ≥ 0.01.
|
||||
7. `memory_mb`, if set, must be ≥ 16.
|
||||
8. If `traefik.enabled` is `true`, at least one port must have `primary: true`.
|
||||
|
||||
## Example: OpenCode Manifest
|
||||
|
||||
```yaml
|
||||
id: opencode
|
||||
name: OpenCode
|
||||
description: AI-powered terminal-based development environment with web interface.
|
||||
version: "1.0.0"
|
||||
image: ghcr.io/opencode-ai/opencode:latest
|
||||
runtime_working_dir: /workspace
|
||||
ports:
|
||||
- container_port: 3000
|
||||
protocol: tcp
|
||||
name: http
|
||||
primary: true
|
||||
workspace_mounts:
|
||||
- type: volume
|
||||
source_pattern: "{project_repo}"
|
||||
target: /workspace
|
||||
read_only: false
|
||||
config_mounts:
|
||||
- type: volume
|
||||
source_pattern: "{user_config}/opencode"
|
||||
target: /root/.config/opencode
|
||||
read_only: false
|
||||
env:
|
||||
TERM: xterm-256color
|
||||
FORCE_COLOR: "1"
|
||||
health_check:
|
||||
type: http
|
||||
path: /
|
||||
port: 3000
|
||||
interval_seconds: 10
|
||||
timeout_seconds: 5
|
||||
retries: 3
|
||||
start_period_seconds: 15
|
||||
resource_limits:
|
||||
cpus: 2.0
|
||||
memory_mb: 4096
|
||||
memory_swap_mb: -1
|
||||
traefik:
|
||||
enabled: true
|
||||
subdomain_prefix: opencode
|
||||
port: 3000
|
||||
middlewares: []
|
||||
strip_prefix: false
|
||||
```
|
||||
|
||||
## Extension Guide: Adding a New Tool
|
||||
|
||||
To add a new standard container tool:
|
||||
|
||||
1. Create a new YAML file in `apps/api/app/tools/manifests/{tool-id}.yml`.
|
||||
2. Populate all required fields (`id`, `name`, `image`, `ports`).
|
||||
3. Set `traefik.enabled: true` and mark one port as `primary: true` if the tool needs HTTP routing.
|
||||
4. Declare `workspace_mounts` and `config_mounts` as needed.
|
||||
5. Restart the API (or call `registry.load_builtin_manifests()`).
|
||||
|
||||
No backend code changes are required for standard containers that expose an HTTP port and need volume mounts.
|
||||
|
||||
## Registry API
|
||||
|
||||
The in-memory registry exposes FastAPI routes under `/api/v1/tools`:
|
||||
|
||||
- `GET /api/v1/tools` — list all registered manifests.
|
||||
- `GET /api/v1/tools/{id}` — retrieve a single manifest.
|
||||
- `POST /api/v1/tools` — register a new manifest (returns 409 if `id` already exists).
|
||||
|
||||
Built-in manifests are loaded automatically on application startup via the FastAPI lifespan context manager.
|
||||
Reference in New Issue
Block a user