Fusion
8dd350286e
- Add Docker Compose and Dockerfile support for tool definitions - Implement readiness probes with configurable command, timeout, interval - Create ConfigFolder model for reusable file collections with project overrides - Add rich tool config fields: port_override, start_command, working_directory, env vars, volumes - Build unified Tool Workshop UI at /tool-workshop replacing /tool-configs and /tool-types - Update instance creation to support dockerfile builds, config folder mounting, readiness probes - Add 3 database migrations for tool_types, tool_configs, and new config_folders table - Create docker_build.py and readiness_probe.py services - Add config_folders API with CRUD and project override endpoints Quality gates: frontend build passes, Python syntax valid, all phases complete Addresses tool-workshop OpenSpec change
380 lines
21 KiB
Markdown
380 lines
21 KiB
Markdown
## Context
|
|
|
|
The tool system currently supports:
|
|
- ToolTypes with compose templates and basic metadata
|
|
- ToolConfigs as simple key-value pairs (env vars or files)
|
|
- Instance creation via compose rendering
|
|
- Basic flat-list UI at `/tool-configs`
|
|
|
|
Users need a much richer system for defining, configuring, and running development tools.
|
|
|
|
## Goals / Non-Goals
|
|
|
|
**Goals:**
|
|
- Support both Docker Compose and Dockerfile for tool definitions
|
|
- Add readiness probes with configurable commands and timeouts
|
|
- Create reusable config file collections ("folders") mountable as volumes
|
|
- Add rich tool config fields (port, start_command, working_directory, volumes, env vars)
|
|
- Build a unified "Tool Workshop" UI for all tool management
|
|
- Support per-project overrides on config folders
|
|
- Maintain backward compatibility with existing built-in tool types
|
|
|
|
**Non-Goals:**
|
|
- Docker image registry management (assume local builds or public images)
|
|
- Real-time collaborative tool editing
|
|
- Tool marketplace/sharing between users
|
|
- Advanced orchestration (Kubernetes, Swarm)
|
|
- Config folder versioning/Git integration
|
|
|
|
## Architecture
|
|
|
|
### Data Model
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ TOOL WORKSHOP │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ ToolType │────▶│ ToolConfig │◀────│ ConfigFolder │ │
|
|
│ │ (Blueprint) │ │ (Settings) │ │ (Files) │ │
|
|
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
|
│ │ │ │ │
|
|
│ │ │ │ │
|
|
│ ▼ ▼ ▼ │
|
|
│ ┌──────────────────────────────────────────────────────┐ │
|
|
│ │ ToolInstance │ │
|
|
│ │ (Runtime + Volumes) │ │
|
|
│ └──────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### ToolType Model
|
|
|
|
```python
|
|
class ToolType:
|
|
# Existing fields
|
|
name: str # unique identifier
|
|
display_name: str
|
|
description: str | None
|
|
category: str
|
|
interfaces: list[str] # ["web", "terminal"]
|
|
default_port: int
|
|
required_variables: list[str]
|
|
is_builtin: bool
|
|
|
|
# New fields
|
|
definition_type: str # "compose" | "dockerfile"
|
|
compose_template: str | None # YAML template (if definition_type == "compose")
|
|
dockerfile_template: str | None # Dockerfile content (if definition_type == "dockerfile")
|
|
build_context: dict | None # {"files": {"path": "content"}} for dockerfile builds
|
|
readiness_probe: dict | None # {"command": "...", "timeout": 30, "interval": 2}
|
|
```
|
|
|
|
**Decision**: Store both compose and dockerfile, use `definition_type` to determine which to use. This allows easy switching and migration.
|
|
|
|
### ToolConfig Model
|
|
|
|
```python
|
|
class ToolConfig:
|
|
# Existing fields
|
|
user_id: UUID
|
|
tool_type_id: UUID
|
|
project_id: UUID | None # null = global config
|
|
key: str
|
|
value: str
|
|
config_type: str # "env" | "file"
|
|
file_path: str | None
|
|
|
|
# New fields
|
|
port_override: int | None # Override tool type default port
|
|
start_command: str | None # Override container start command
|
|
working_directory: str | None # Working directory inside container
|
|
environment_variables: dict | None # JSON {"KEY": "value", ...}
|
|
volumes: list[dict] | None # JSON [{"source": "...", "target": "...", "type": "..."}]
|
|
```
|
|
|
|
**Decision**: Store env vars and volumes as JSONB for flexibility. Port as integer with validation.
|
|
|
|
### ConfigFolder Model (NEW)
|
|
|
|
```python
|
|
class ConfigFolder:
|
|
id: UUID
|
|
user_id: UUID
|
|
name: str # e.g., "my-dotfiles", "vscode-settings"
|
|
description: str | None
|
|
mount_path: str # Default mount path in container (e.g., "/home/user/.config")
|
|
files: dict # JSON {"relative/path": "content", ...}
|
|
project_overrides: dict | None # JSON {project_id: {"mount_path": "...", "files": {...}}}
|
|
is_active: bool # Quick toggle
|
|
created_at, updated_at
|
|
```
|
|
|
|
**Decision**: Files stored as JSONB with relative paths as keys. This is simple and sufficient for config files (not binary assets).
|
|
|
|
### Volume Mount Resolution
|
|
|
|
When creating an instance, volumes are resolved in this priority order:
|
|
|
|
```
|
|
1. ToolConfig.volumes (explicit per-config mounts)
|
|
2. ConfigFolder mounts (user's active config folders)
|
|
3. ToolType default volumes (from compose/dockerfile)
|
|
```
|
|
|
|
Config folder files are written to the instance directory under `volumes/<folder_name>/` and mounted from there.
|
|
|
|
### Readiness Probe System
|
|
|
|
```python
|
|
class ReadinessProbe:
|
|
command: str # e.g., "curl -f http://localhost:8080/health"
|
|
timeout: int # seconds (default: 30)
|
|
interval: int # seconds between checks (default: 2)
|
|
retries: int # max attempts (default: timeout/interval)
|
|
```
|
|
|
|
**Execution Flow**:
|
|
1. Start container
|
|
2. Wait for container to be running
|
|
3. Execute probe command inside container via `docker exec`
|
|
4. If success → mark instance as "running"
|
|
5. If timeout → mark instance as "failed" with probe output in logs
|
|
|
|
**Decision**: Probes run inside the container using `docker exec`. This works for both network-based probes (curl) and command-based probes (binary version checks).
|
|
|
|
### Instance Creation Flow
|
|
|
|
```
|
|
1. Generate instance ID and directory
|
|
2. Resolve ToolConfig (global + project-specific)
|
|
3. Write config files:
|
|
a. .env file (from env-type ToolConfigs)
|
|
b. Config files (from file-type ToolConfigs)
|
|
c. Config folder files (to volumes/<folder>/)
|
|
4. IF ToolType.definition_type == "dockerfile":
|
|
a. Write Dockerfile + build context files
|
|
b. Build image: docker build -t <instance_tag> .
|
|
c. Generate compose from template using built image
|
|
5. IF ToolType.definition_type == "compose":
|
|
a. Render compose template with variables
|
|
6. Write docker-compose.yml
|
|
7. docker compose up -d
|
|
8. Connect to backend network
|
|
9. IF readiness_probe defined:
|
|
a. Execute probe with timeout
|
|
b. Update status based on result
|
|
10. IF web interface:
|
|
a. Create Cloudflare tunnel
|
|
b. Update URL
|
|
```
|
|
|
|
## UI Design
|
|
|
|
### Tool Workshop Page (`/tool-workshop`)
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Tool Workshop [+ New] │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────────────────────────────┐ │
|
|
│ │ │ │ │ │
|
|
│ │ MY TOOLS │ │ [Tool Type Builder] │ │
|
|
│ │ │ │ │ │
|
|
│ │ ▼ Code Editor│ │ Name: [____________] │ │
|
|
│ │ □ VS Code │ │ Type: (•) Compose ( ) Dockerfile │ │
|
|
│ │ □ Cursor │ │ │ │
|
|
│ │ │ │ [Compose Template / Dockerfile] │ │
|
|
│ │ ▼ AI Tools │ │ ┌────────────────────────────────┐ │ │
|
|
│ │ □ OpenCode │ │ │ version: '3.8' │ │ │
|
|
│ │ □ Continue │ │ │ services: │ │ │
|
|
│ │ │ │ │ app: │ │ │
|
|
│ │ CONFIGS │ │ │ image: ... │ │ │
|
|
│ │ │ │ │ ports: │ │ │
|
|
│ │ ▼ Global │ │ │ - "{{PORT}}:8080" │ │ │
|
|
│ │ □ dotfiles │ │ │ volumes: │ │ │
|
|
│ │ □ api-keys │ │ │ - ... │ │ │
|
|
│ │ │ │ └────────────────────────────────┘ │ │
|
|
│ │ ▼ Project X │ │ │ │
|
|
│ │ □ overrides│ │ Readiness Probe: │ │
|
|
│ │ │ │ Command: [curl -f localhost:8080] │ │
|
|
│ │ │ │ Timeout: [30] seconds │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │ [Save Tool Type] │ │
|
|
│ │ │ │ │ │
|
|
│ └──────────────┘ └──────────────────────────────────────┘ │
|
|
│ │
|
|
│ Tabs: [Tool Types] [Configs] [Config Folders] │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
**Navigation Structure**:
|
|
- Left sidebar: Hierarchical tree
|
|
- Tool Types (expandable, shows instances count)
|
|
- Config Folders (grouped by global/project)
|
|
- Right panel: Context-aware editor based on selection
|
|
- Tab bar: Switch between Tool Types / Configs / Config Folders views
|
|
|
|
### Config Editor
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Edit Config: OpenCode API Keys │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ Basic Settings │ Advanced Settings │
|
|
│ ────────────────────────────┼──────────────────────────────── │
|
|
│ Key: [OPENAI_API_KEY] │ Port Override: [_____] │
|
|
│ Value: [sk-... ] │ Start Command: [_____] │
|
|
│ Type: (•) Env ( ) File │ Working Dir: [/workspace] │
|
|
│ File Path: [__________] │ │
|
|
│ │ Environment Variables: │
|
|
│ │ ┌──────────────────────────┐ │
|
|
│ │ │ KEY │ VALUE │ │
|
|
│ │ │ OPENAI_KEY │ sk-... │ │
|
|
│ │ │ MODEL │ gpt-4 │ │
|
|
│ │ └──────────────────────────┘ │
|
|
│ │ │
|
|
│ │ Volume Mounts: │
|
|
│ │ ┌──────────────────────────┐ │
|
|
│ │ │ SOURCE │ TARGET │ │
|
|
│ │ │ dotfiles │ ~/.config │ │
|
|
│ │ │ vscode-set │ ~/.vscode │ │
|
|
│ │ └──────────────────────────┘ │
|
|
│ │ │
|
|
│ [Delete] [Cancel] [Save] │ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Config Folder Manager
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Config Folder: my-dotfiles │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ Name: [my-dotfiles] │
|
|
│ Description: [My personal dotfiles] │
|
|
│ Default Mount Path: [/home/user] │
|
|
│ │
|
|
│ Files: │
|
|
│ ┌──────────────────────────────────────────────────────┐ │
|
|
│ │ Path │ Size │ Actions │ │
|
|
│ │ .zshrc │ 2.1KB │ [Edit] [Delete] │ │
|
|
│ │ .gitconfig │ 412B │ [Edit] [Delete] │ │
|
|
│ │ .config/starship.toml │ 1.8KB │ [Edit] [Delete] │ │
|
|
│ └──────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ [+ Add File] │
|
|
│ │
|
|
│ Project Overrides: │
|
|
│ ┌──────────────────────────────────────────────────────┐ │
|
|
│ │ Project │ Mount Path │ Files Override │ │
|
|
│ │ Project Alpha │ /home/dev │ [3 files] │ │
|
|
│ │ Project Beta │ /workspace │ [1 file] │ │
|
|
│ └──────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ [Add Override] [Delete Folder] [Save] │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Decisions
|
|
|
|
1. **Dockerfile vs Compose**: Support both. `definition_type` field determines which path to use. Compose is the default for backward compatibility.
|
|
|
|
2. **Config Folder Storage**: Store files as JSONB keyed by relative path. This avoids file system complexity and works well for text-based config files. Limit: 10MB per folder.
|
|
|
|
3. **Readiness Probe Execution**: Use `docker exec` to run commands inside the container. This is the most flexible approach (works for HTTP checks, binary checks, file checks).
|
|
|
|
4. **Volume Resolution Order**: Config-level volumes override config-folder volumes, which override tool-type defaults. Last-write-wins for conflicts.
|
|
|
|
5. **UI Organization**: Single page with three tabs (Tool Types, Configs, Config Folders) and a left sidebar for navigation. This consolidates the current `/tool-configs` and `/tool-types` pages.
|
|
|
|
6. **Project Overrides**: ConfigFolders support per-project overrides for mount_path and files. This allows project-specific customizations while keeping the base collection reusable.
|
|
|
|
## Risks / Trade-offs
|
|
|
|
- **Dockerfile build times**: Building images on-demand is slow. Mitigation: Document that users should use pre-built images in compose for faster startup; dockerfile is for custom tools.
|
|
- **Config folder size limits**: JSONB has practical limits. Mitigation: 10MB limit per folder, enforced in API.
|
|
- **Readiness probe complexity**: Commands might hang or fail in unexpected ways. Mitigation: Strict timeout, clear error messages, probe logs stored on instance.
|
|
- **Migration complexity**: Existing tool types need `definition_type` set to "compose". Mitigation: Database default, seed function update.
|
|
- **UI complexity**: Three tabs with different editors could feel overwhelming. Mitigation: Progressive disclosure (hide advanced fields, collapsible sections).
|
|
|
|
## API Endpoints
|
|
|
|
### Tool Types
|
|
- `GET /tool-types` - List all (existing)
|
|
- `POST /tool-types` - Create with new fields
|
|
- `PUT /tool-types/{id}` - Update with new fields
|
|
- `GET /tool-types/{id}/validate` - Validate compose/dockerfile syntax
|
|
|
|
### Tool Configs
|
|
- `GET /tool-configs` - List with new fields
|
|
- `POST /tool-configs` - Create with new fields
|
|
- `PUT /tool-configs/{id}` - Update with new fields
|
|
- `GET /tool-configs/defaults/{tool_type_id}` - Get suggested defaults
|
|
|
|
### Config Folders (NEW)
|
|
- `GET /config-folders` - List user's folders
|
|
- `POST /config-folders` - Create folder
|
|
- `PUT /config-folders/{id}` - Update folder (files, mount_path)
|
|
- `DELETE /config-folders/{id}` - Delete folder
|
|
- `POST /config-folders/{id}/overrides` - Add project override
|
|
- `PUT /config-folders/{id}/overrides/{project_id}` - Update override
|
|
- `DELETE /config-folders/{id}/overrides/{project_id}` - Remove override
|
|
|
|
## Database Schema
|
|
|
|
### Migration: tool_types
|
|
```sql
|
|
ALTER TABLE tool_types
|
|
ADD COLUMN definition_type VARCHAR(20) NOT NULL DEFAULT 'compose',
|
|
ADD COLUMN dockerfile_template TEXT,
|
|
ADD COLUMN build_context JSONB DEFAULT '{}',
|
|
ADD COLUMN readiness_probe JSONB;
|
|
|
|
-- Ensure consistency
|
|
ALTER TABLE tool_types
|
|
ADD CONSTRAINT chk_definition_type
|
|
CHECK (definition_type IN ('compose', 'dockerfile'));
|
|
```
|
|
|
|
### Migration: tool_configs
|
|
```sql
|
|
ALTER TABLE tool_configs
|
|
ADD COLUMN port_override INTEGER,
|
|
ADD COLUMN start_command TEXT,
|
|
ADD COLUMN working_directory TEXT,
|
|
ADD COLUMN environment_variables JSONB DEFAULT '{}',
|
|
ADD COLUMN volumes JSONB DEFAULT '[]';
|
|
|
|
ALTER TABLE tool_configs
|
|
ADD CONSTRAINT chk_port_range
|
|
CHECK (port_override IS NULL OR (port_override >= 1 AND port_override <= 65535));
|
|
```
|
|
|
|
### New Table: config_folders
|
|
```sql
|
|
CREATE TABLE config_folders (
|
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
|
name VARCHAR(255) NOT NULL,
|
|
description TEXT,
|
|
mount_path VARCHAR(1024) NOT NULL,
|
|
files JSONB NOT NULL DEFAULT '{}',
|
|
project_overrides JSONB DEFAULT '{}',
|
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
|
UNIQUE(user_id, name)
|
|
);
|
|
|
|
CREATE INDEX idx_config_folders_user ON config_folders(user_id);
|
|
```
|