Commit Graph

704 Commits

Author SHA1 Message Date
alex e672bdde54 Merge remote-tracking branch 'origin/dev' into dev 2026-05-28 21:19:41 +02:00
alex c7c4cb45a7 fix(terminal): verify container exists before creating terminal session
The instance status may say 'running' but the actual Docker container
may have been removed (e.g. docker prune, host restart). The old code
created a terminal session which immediately died because docker exec
failed with 'No such container'.

- Add get_container_status check in WebSocket handler before session creation
- Return 4004 with clear message if container is missing
- This prevents spawning zombie terminal sessions
2026-05-28 21:18:47 +02:00
Alex Blank 6e4275a510 fix: resolve Alembic multiple heads
The terminal_sessions migration and drop_tool_configs migration both pointed
to add_tool_definition_manifests as their down_revision, creating two heads.
Update drop migration to depend on terminal_sessions instead, restoring a
single linear chain.
2026-05-28 20:40:40 +02:00
Alex Blank 3ef60be623 Merge branch 'dev' of ssh://git.commumedia.org:2222/alex/headquarter into dev 2026-05-28 20:36:39 +02:00
alex a3d01dd0a5 fix(terminal): loading state, focus handling, debug logging
- Initialize loading=true in useTerminalSessions to prevent auto-create
  from firing before initial load completes
- Remove hasAutoCreated ref from TerminalPage (no longer needed)
- Add focus() to TerminalRef, call on tab switch
- Add term.focus() after term.open() in TerminalComponent
- Add console logging for WebSocket send/receive to debug no-i/o
- Revert backend _read_loop retry logic to original break-on-error
2026-05-28 20:14:54 +02:00
Alex Blank 9bd5fc5c68 refactor: remove Tool Configs and Config Folders
These features are fully superseded by Config Profiles which provide:
- Env vars, file mounts, port overrides, start commands, working dirs
- Git mounts, profile composition, cycle detection
- Default selection, project/tool-type scoping

Changes:
- Delete backend models: ToolConfig, ConfigFolder
- Delete backend APIs: tool_configs.py, config_folders.py
- Delete frontend API clients: tool_configs.ts, config_folders.ts
- Remove Tool Config fetching from start_instance, use ConfigProfile only
- Simplify merge_with_config to accept only profile (no tool_configs)
- Remove configs/folders tabs from Tool Workshop page
- Delete associated integration and unit tests
- Add Alembic migration to drop tool_configs and config_folders tables

Quality gates: backend tests 59 passed, frontend typecheck clean
2026-05-28 20:08:48 +02:00
alex b6e71e32f5 fix(terminal): prevent double session creation, restore sessions on reload
Three fixes for multi-session terminal bugs:

1. Race-condition double creation: The auto-create effect fired twice because
   loadSessions returned 0 while an earlier createSession was still in flight.
   Added hasAutoCreated guard ref to ensure only one auto-create happens.

2. Page reload spawns new sessions: After server restart, list_terminal_sessions
   filtered out DB-only sessions (no in-memory counterpart), so the frontend
   thought no sessions existed and auto-created new ones. Reverted the filter
   so DB rows are always returned. The WebSocket handler now restores the
   in-memory session from the DB row on demand when connecting.

3. No input after connection: The backend _read_loop would break on any send
   error, causing asyncio.wait to cancel the _write_loop. Made _read_loop
   retry up to 3 times before giving up, preventing transient send errors
   from killing input handling.

Quality gates: pytest (15/15 passed), tsc clean
2026-05-28 19:45:59 +02:00
alex 9ccaae04db Merge remote-tracking branch 'origin/dev' into dev
# Conflicts:
#	apps/api/.pi-lens/cache/review-graph.json
#	apps/web/.pi-lens/cache/review-graph.json
2026-05-28 19:14:56 +02:00
alex 9f90624aa6 fix(terminal): prevent xterm.js crash, websocket disconnect cascade, stale sessions
Three related bugs fixed:

1. Frontend xterm.js crash: TerminalPage rendered ALL sessions with display:none
   for inactive ones. xterm.js crashes when initialized in a hidden container
   (Viewport can't read dimensions). Fix: only render the active session's
   TerminalComponent using conditional rendering.

2. Backend websocket disconnect cascade: When client disconnected (due to #1),
   the server tried to send 'connected' status on dead socket, caught the
   WebSocketDisconnect in a generic except block, then tried to close() again
   causing RuntimeError. Fix: catch WebSocketDisconnect specifically and suppress
   close() errors.

3. Stale DB sessions: After server restart, DB still had old terminal session
   rows but no in-memory sessions. list_terminal_sessions returned these ghosts,
   causing the frontend to render dead tabs. Fix: skip DB-only sessions that
   have no live in-memory counterpart.

Quality gates: pytest (15/15 passed), tsc clean, vitest (7/7 passed)
2026-05-28 19:10:22 +02:00
Alex Blank 62c1fb3836 Merge branch 'feat/tool-definition-manifest' into dev
Conflicts resolved:
- models/__init__.py: kept both TerminalSessionModel (from dev) and
  ToolDefinitionManifest (from feature branch)
- alembic migration: kept full migration (already applied to DB)
- openspec/config.yaml: kept full config with SDD settings
2026-05-28 15:49:56 +02:00
alex 569c20cf63 fix(terminal): simplify REST endpoints to use instance_id only
The frontend router navigates to /instances/:instanceId/terminal without
project_id or repo_id. The backend terminal REST endpoints were requiring
these path params, causing 404s.

- Simplify _get_terminal_instance to validate by instance_id only
- Update all REST routes from /projects/{pid}/repositories/{rid}/instances/{iid}/terminal/*
  to /instances/{instance_id}/terminal/*
- Update frontend API client to match new paths
- Update useTerminalSessions hook to take instanceId only
- Update TerminalPage to use simplified hook
- Update tests to match new paths

Fixes: 404 on GET /projects/repositories/instances/{id}/terminal/sessions
2026-05-28 15:40:02 +02:00
Alex Blank f658b71079 feat: tool definition manifest system
Complete implementation of declarative manifest-based tool definitions.

PR 1 — Backend:
- Add tool_definition_manifests table with base image versioning
- Add manifest_compiler: resolve base, deep merge, compile Dockerfile,
  entrypoint, Compose, deterministic image tags
- Add permission_fixer: post-start chown/chmod for mount permissions
- Add CRUD API for tool definitions + compile preview endpoint
- Integrate manifest flow into start_instance alongside legacy path

PR 2 — Frontend:
- Add ManifestEditor component with base selector, package editors,
  script editors, mount designer, runtime config, live preview
- Integrate into Tool Workshop page as 'Manifest (Declarative)' type

PR 3 — Validation & Docs:
- 8 legacy fallback unit tests proving dockerfile/compose types
  continue to work unchanged
- Tool Workshop user guide

Quality gates: 152 passed (6 pre-existing unrelated failures)
2026-05-28 15:39:29 +02:00
Alex Blank 3e99e7f197 feat: legacy fallback tests and docs (PR 3)
- Add test_tool_instances_legacy.py with 8 unit tests:
  - dockerfile definition type builds from template
  - dockerfile build failure raises HTTP 500
  - compose definition type renders template
  - manifest compiler is NOT called for legacy types
  - start_instance legacy/compose/dockerfile types all skip manifest flow
  - start_instance manifest type correctly invokes compiler
- Mark T3.2 and T3.3 tasks complete in OpenSpec
- Add openspec/docs/tool-workshop-guide.md with user guide covering
  definition types, manifest creation workflow, base definitions,
  migration path, and permissions
2026-05-28 14:54:32 +02:00
alex c2c983a01e fix(alembic): bridge ghost migration 2026_05_28_add_tool_definition_manifests
The production database was stamped with a migration that no longer exists
in the codebase (created on another branch, applied, then removed). This
adds a no-op bridge migration so Alembic can reconcile the DB state.

- Create bridge migration 2026_05_28_add_tool_definition_manifests (no-op)
- Re-chain terminal_sessions migration to depend on the bridge
- Fixes startup failure: Can't locate revision identified by ...
2026-05-28 14:45:15 +02:00
Alex Blank e46b4f9249 feat: Tool Workshop manifest editor (PR 2)
- Add tool_definitions API client with types for manifests
- Add ManifestEditor component: base image selector, package editors
  (apt/npm/pip/node), script editors (build/startup), mount schema
  designer, runtime config, and live preview panel
- Integrate ManifestEditor into Tool Workshop as 'Manifest (Declarative)'
  definition type alongside Compose and Dockerfile
- Update ToolType API types to include manifest_id and 'manifest'
  definition_type
- Frontend builds clean, TypeScript typecheck passes
2026-05-28 14:35:04 +02:00
alex 8eb851793d feat: mobile auto-hide header and tabs for multi-session terminal
- Add useAutoHide hook to TerminalPage for mobile header/tab strip
- Header and tabs auto-hide after 3s, tap to reveal
- Add CSS transitions for smooth show/hide on mobile
- Fullscreen mobile mode hides header and tabs completely
2026-05-28 14:10:06 +02:00
alex 8e5e815ac9 Merge remote-tracking branch 'origin/dev' into dev
# Conflicts:
#	.gitignore
2026-05-28 14:01:19 +02:00
alex 143a254b0c chore: add pi cache dirs to .gitignore and mobile terminal styles 2026-05-28 13:49:46 +02:00
Alex Blank 5deee8c65c feat: tool definition manifest system (PR 1)
- Add ToolDefinitionManifest model with base image versioning
- Add manifest compiler: Dockerfile + Compose generation from JSON manifests
- Add permission fixer: post-start chown/chmod for mount policies
- Add tool definition CRUD API with live compile preview endpoint
- Integrate manifest-based startup flow in start_instance
- Add Alembic migration with data conversion for pi-agent
- Add 48 unit tests for manifest compiler, permission fixer, docker service
- Keep backward compatibility with legacy dockerfile_template/compose_template

Migration: applied successfully. Pi-agent converted to manifest.
Quality gates: pytest (146 passed, 4 pre-existing unrelated failures)
2026-05-28 13:37:34 +02:00
alex 62d1bdc462 feat: multi-session terminal frontend UI + tests (PR 3)
- Add TerminalSessionTabs component with status dots, rename, close, max-5 limit
- Add 7 component tests for tab rendering, selection, close, rename
- TerminalComponent: sessionId prop, forwardRef with fit() method
- TerminalPage: multi-session orchestration, tab switching, auto-create default
- Fullscreen mode: Alt+Shift+F toggle, auto-hide tabs, Esc exit
- Keyboard shortcuts: Alt+Shift+N/W/ArrowLeft/ArrowRight/R
- Add CSS for tabs, fullscreen, mobile responsive
- Update useTerminalSessions hook for session CRUD
- terminal_manager.py: lookup by internal session_id fallback

Quality gates: tsc --noEmit clean, vitest (7/7 new tests passed), pytest (182 passed)
2026-05-28 13:35:45 +02:00
alex 0b35ae3bf0 feat: multi-session terminal backend API + frontend client (PR 2)
- Add WebSocket route /ws/tool-instances/{instance_id}/terminal/{session_id}
- Preserve /terminal as default-session alias for backward compatibility
- Extract shared _handle_terminal_websocket handler for both routes
- Add REST endpoints: GET list, POST create, DELETE close, POST reset, POST rename
- Preserve legacy POST .../terminal/reset as default session alias
- Add frontend API client (apps/web/src/api/terminal.ts)
- Add useTerminalSessions React hook for session CRUD + state management
- Add integration tests for auth requirements on all new endpoints

Quality gates: pytest (8 new passed, 182 total passed, 51 pre-existing failures)
2026-05-28 12:08:37 +02:00
alex b55300ff6f feat: multi-session terminal backend core (PR 1)
- Add TerminalSessionModel DB table with instance_id FK, name, status,
  created_at, last_activity_at, closed_at columns
- Add Alembic migration for terminal_sessions table
- Refactor TerminalManager to use composite key (instance_id, session_id)
  supporting up to 5 concurrent sessions per instance
- Add create_session, get_session, get_sessions_for_instance, close_session
- Preserve get_or_create_session for backward compatibility (default session)
- Fix attach_websocket to only close sockets within same session
- Add name (auto-generated 'Session N') and status tracking to TerminalSession
- Add 7 unit tests for multi-session logic

Quality gates: pytest (7 new passed, 174 total passed, 51 pre-existing failures)
2026-05-28 11:38:22 +02:00
Alex Blank 314ba3aee4 Merge branch 'fix/container-name-case-sensitivity' into dev 2026-05-28 10:58:18 +02:00
Alex Blank 18e4a89573 Merge branch 'fix/container-name-case-sensitivity' 2026-05-28 10:56:43 +02:00
Alex Blank 29943ac239 fix: lowercase container name filter for case-sensitive docker ps
- get_container_id() and get_container_name() now lowercase the
  instance name before passing to docker ps --filter, because
  Docker container names are lowercase internally and the filter
  is case-sensitive. This caused container_id to never be captured
  when instance.name contained uppercase chars (e.g. 'Headquarter'),
  breaking terminal WebSocket connections.

- Also guard proc.stdout being None in start_cloudflared_tunnel().

- Add unit tests for get_container_id and get_container_name.

Quality gates: pytest (14 passed), python clean
2026-05-28 10:56:33 +02:00
alex 22474cdba5 style: fix all ruff and eslint errors across codebase
Backend (ruff):
- Fix 106 errors: move imports to top of file (E402)
- Remove unused imports (F401)
- Add missing imports for undefined names (F821)
- Remove unused variables (F841)
- Fix test_models.py broken RefreshToken test
- Fix test_projects_api.py missing TestClient import

Frontend (eslint):
- Remove unused imports/variables across 10 files
- Fix explicit any types in client.ts and sessions.ts
- Clean up empty block statements in terminal.tsx

Quality gates: ruff (pass), eslint (pass), tsc --noEmit (pass),
pytest (98 passed, 4 pre-existing failures)
2026-05-28 10:15:59 +02:00
alex 0c839e8c6f fix: terminal 4004 infinite reconnect loop for pi-agent tool type
- Add stdin_open: true and tty: true to dockerfile-based compose generation.
  Without these, bash (PID 1) exits immediately, causing a container restart
  loop that makes the instance invisible to docker ps and triggers 4004.
- Treat WebSocket close codes 4001/4003/4004 as permanent errors in the
  frontend. Stop retrying and show the server reason to the user.
- Prevent visibilitychange handler from resetting retry attempts after a
  permanent error has occurred.
- Use docker ps -a in get_container_id/get_container_name to find
  stopped/exited containers for diagnostics.

Quality gates: tsc --noEmit (pass), pytest (98 passed, 4 pre-existing failures)
2026-05-28 09:33:35 +02:00
Alex Blank c63cf7db50 fix: handle existing git mount dirs and invalid compose ports
- Fix git mount clone to check correct path (repo-clone subdir)
- Pull updates instead of re-cloning when git mount dir exists
- Add compose file sanitization to remove invalid port 0 mappings
- Fixes startup failures for existing instances with old compose files
2026-05-27 22:34:24 +02:00
Alex Blank d9d2b91384 fix: skip port mapping for terminal-only tools in dockerfile compose 2026-05-27 22:29:37 +02:00
Alex Blank d6ea5fb1fd chore: clean up debugging logs and console prints
Frontend:
- Remove 18 console.log/warn/error statements from terminal.tsx
- Remove console.warn from icon.tsx

Backend:
- Downgrade routine logger.info to logger.debug in tool_instances.py, terminal.py,
  terminal_session.py, terminal_manager.py, auth.py, docker_build.py, clone.py,
  config_profiles.py, user_config.py
- Keep important lifecycle events as logger.info:
  * Instance creation, start, running state
  * Docker build success/failure
  * Terminal session creation and reset
  * Auth success and user creation
  * Readiness probe success
  * Tunnel creation/stop
2026-05-27 22:16:59 +02:00
Alex Blank 1883825b18 fix: run docker build in thread pool to avoid blocking API 2026-05-27 22:08:01 +02:00
Alex Blank bc71fd6fac fix: lowercase docker image tag to avoid invalid reference format 2026-05-27 22:02:18 +02:00
Alex Blank 28aa9ccf5a feat: add pi-agent tool type migration
- Adds pi-agent to tool_types table with terminal interface
- Includes Dockerfile template for pi.dev coding agent
- Idempotent: checks for existing entry before insert
2026-05-27 16:30:27 +02:00
Alex Blank 44dd80cb58 feat: add tool-images directory with Dockerfiles for all base tool types
- Create tool-images/ directory with organized Dockerfile templates
- Add base.dockerfile with common dev tools (git, nvim, ranger, tmux, node)
- Add code-server.dockerfile with VS Code in browser
- Add jupyter.dockerfile with Jupyter Lab
- Add opencode.dockerfile with OpenCode agent
- Add pi-agent.dockerfile with Pi Coding Agent (pi.dev)
- All images include: git, neovim, ranger, tmux, htop, tree, jq, Node.js 20
- Add README with usage instructions
2026-05-27 15:51:43 +02:00
Alex Blank 23485833d8 docs: clarify git mount target path hint 2026-05-27 15:34:50 +02:00
Alex Blank e23dcdf4e1 fix: remove broken ~ expansion, require working_directory for relative paths 2026-05-27 15:33:06 +02:00
Alex Blank f05ac55875 fix: expand ~ in git mount target paths to avoid /tmp 2026-05-27 15:27:44 +02:00
Alex Blank bcefeb4163 fix: git mount add button not adding mounts 2026-05-27 15:17:34 +02:00
Alex Blank 33d08faf70 feat: allow relative target paths for git mounts
- Remove absolute path requirement from target_path validation
- Resolve relative paths against working_directory at instance startup
- Fall back to /home/user if no working_directory is configured
- Update frontend to allow relative target paths
- Update spec to document relative path support
- Update tests to allow relative paths and test path traversal rejection
2026-05-27 15:01:16 +02:00
Alex Blank 8a58c61278 fix: align git mount implementation with spec
- _checkout_branch now returns bool and falls back gracefully on failure
- Glob warning message includes matched file count
- Fix database model comment to reference remote_url
- Update tests for new branch checkout behavior

All 51 tests pass
2026-05-27 14:38:11 +02:00
Alex Blank 8231e750d9 docs: update spec to use remote_url instead of repo_id for git mounts
- Change git mount schema from repo_id to remote_url
- Update validation rules to check URL format instead of repo existence
- Update cloning scenarios to clone directly from URL
- Update UI scenarios to show URL input instead of repo selector
- Remove references to internal/existing repositories
2026-05-27 14:27:39 +02:00
Alex Blank 6ce645d210 fix: correct API endpoints for repository listing
- Change frontend to call /repositories instead of /projects/repositories
- Change parse-url endpoint to /repositories/parse-url
- Fixes 422 error from route mismatch
2026-05-27 14:18:13 +02:00
Alex Blank 89ca9f10c7 feat: simplify git mounts to use direct URLs instead of repo references
- Change git mount schema from repo_id to remote_url
- Remove database lookups for git mount resolution
- Clone directly from URL at instance startup
- Simplify frontend UI to text input for Git URL
- Fix route ordering in git_repositories.py to prevent 422 errors
- Update all tests to use remote_url field

Breaking change: Git mounts now use remote_url instead of repo_id
2026-05-27 11:53:25 +02:00
Alex Blank baabd1fa62 feat: allow creating external repositories directly from git mount editor
- Add createExternalRepository API function
- Update GitMountEditor with "+ Add new repository..." option
- Show form to enter repo name and remote URL
- Auto-create external repo and refresh list on success
- Update config-profiles page to pass onCreateRepository handler
2026-05-27 11:17:27 +02:00
Alex Blank f14fc37e75 docs: update API docs for external repositories and git mounts
- Add GET /repositories endpoint documentation
- Add POST /repositories endpoint for external repos
- Update config-profiles.md with git mount details
- Update repositories.md with external repo support
2026-05-27 11:04:48 +02:00
Alex Blank e07938098a feat: add external repository support for config profile git mounts
- Add POST /repositories endpoint for external repos (no project_id)
- Update GitRepositoryResponse to allow nullable project_id
- Update list_repositories to support listing all user repos
- Add _pull_repository_updates for auto-pull on container creation
- Update git mount validation to allow external repos
- Frontend: Update listRepositories to support optional projectId
- Spec updates: external repos, auto-clone, per-instance isolation
2026-05-27 11:03:12 +02:00
Alex Blank 943b9db5c7 fix: remove merge migration that references non-existent file
The server is missing 2026_05_27_make_project_id_nullable.py but the
merge migration referenced it. Removing the merge migration leaves a
clean single head chain.
2026-05-27 10:48:25 +02:00
Alex Blank a4604d6a9a fix: add merge migration to resolve multiple heads from renamed migration
The server has both the old (2026_05_27_make_project_id_nullable) and
new (2026_05_27_external_repos) migration files, creating two heads.
This merge migration resolves them into a single head.
2026-05-27 10:42:44 +02:00
Alex Blank 18204628cc fix: shorten alembic revision name and expand version_num column
- Rename 2026_05_27_make_project_id_nullable to 2026_05_27_external_repos
  (33 chars exceeded VARCHAR(32) limit in alembic_version table)
- Add alembic_version column expansion to VARCHAR(64) in migration
- Ensure future migrations won't hit the 32 char limit
2026-05-27 10:36:47 +02:00
Alex Blank 93b415c53e feat: support external repositories for git mounts
- Make project_id nullable in git_repositories table (migration)
- Allow external repos not tied to any project
- Update validation to allow user-owned external repos in git mounts
- Add /projects/repositories endpoint to list all user repos
- Update frontend to fetch all user repos for git mount selector
- TypeScript and build pass
2026-05-27 10:28:31 +02:00