Commit Graph

305 Commits

Author SHA1 Message Date
alex 9ccaae04db Merge remote-tracking branch 'origin/dev' into dev
# Conflicts:
#	apps/api/.pi-lens/cache/review-graph.json
#	apps/web/.pi-lens/cache/review-graph.json
2026-05-28 19:14:56 +02:00
alex 9f90624aa6 fix(terminal): prevent xterm.js crash, websocket disconnect cascade, stale sessions
Three related bugs fixed:

1. Frontend xterm.js crash: TerminalPage rendered ALL sessions with display:none
   for inactive ones. xterm.js crashes when initialized in a hidden container
   (Viewport can't read dimensions). Fix: only render the active session's
   TerminalComponent using conditional rendering.

2. Backend websocket disconnect cascade: When client disconnected (due to #1),
   the server tried to send 'connected' status on dead socket, caught the
   WebSocketDisconnect in a generic except block, then tried to close() again
   causing RuntimeError. Fix: catch WebSocketDisconnect specifically and suppress
   close() errors.

3. Stale DB sessions: After server restart, DB still had old terminal session
   rows but no in-memory sessions. list_terminal_sessions returned these ghosts,
   causing the frontend to render dead tabs. Fix: skip DB-only sessions that
   have no live in-memory counterpart.

Quality gates: pytest (15/15 passed), tsc clean, vitest (7/7 passed)
2026-05-28 19:10:22 +02:00
Alex Blank 62c1fb3836 Merge branch 'feat/tool-definition-manifest' into dev
Conflicts resolved:
- models/__init__.py: kept both TerminalSessionModel (from dev) and
  ToolDefinitionManifest (from feature branch)
- alembic migration: kept full migration (already applied to DB)
- openspec/config.yaml: kept full config with SDD settings
2026-05-28 15:49:56 +02:00
alex 569c20cf63 fix(terminal): simplify REST endpoints to use instance_id only
The frontend router navigates to /instances/:instanceId/terminal without
project_id or repo_id. The backend terminal REST endpoints were requiring
these path params, causing 404s.

- Simplify _get_terminal_instance to validate by instance_id only
- Update all REST routes from /projects/{pid}/repositories/{rid}/instances/{iid}/terminal/*
  to /instances/{instance_id}/terminal/*
- Update frontend API client to match new paths
- Update useTerminalSessions hook to take instanceId only
- Update TerminalPage to use simplified hook
- Update tests to match new paths

Fixes: 404 on GET /projects/repositories/instances/{id}/terminal/sessions
2026-05-28 15:40:02 +02:00
Alex Blank 3e99e7f197 feat: legacy fallback tests and docs (PR 3)
- Add test_tool_instances_legacy.py with 8 unit tests:
  - dockerfile definition type builds from template
  - dockerfile build failure raises HTTP 500
  - compose definition type renders template
  - manifest compiler is NOT called for legacy types
  - start_instance legacy/compose/dockerfile types all skip manifest flow
  - start_instance manifest type correctly invokes compiler
- Mark T3.2 and T3.3 tasks complete in OpenSpec
- Add openspec/docs/tool-workshop-guide.md with user guide covering
  definition types, manifest creation workflow, base definitions,
  migration path, and permissions
2026-05-28 14:54:32 +02:00
alex c2c983a01e fix(alembic): bridge ghost migration 2026_05_28_add_tool_definition_manifests
The production database was stamped with a migration that no longer exists
in the codebase (created on another branch, applied, then removed). This
adds a no-op bridge migration so Alembic can reconcile the DB state.

- Create bridge migration 2026_05_28_add_tool_definition_manifests (no-op)
- Re-chain terminal_sessions migration to depend on the bridge
- Fixes startup failure: Can't locate revision identified by ...
2026-05-28 14:45:15 +02:00
Alex Blank e46b4f9249 feat: Tool Workshop manifest editor (PR 2)
- Add tool_definitions API client with types for manifests
- Add ManifestEditor component: base image selector, package editors
  (apt/npm/pip/node), script editors (build/startup), mount schema
  designer, runtime config, and live preview panel
- Integrate ManifestEditor into Tool Workshop as 'Manifest (Declarative)'
  definition type alongside Compose and Dockerfile
- Update ToolType API types to include manifest_id and 'manifest'
  definition_type
- Frontend builds clean, TypeScript typecheck passes
2026-05-28 14:35:04 +02:00
alex 8e5e815ac9 Merge remote-tracking branch 'origin/dev' into dev
# Conflicts:
#	.gitignore
2026-05-28 14:01:19 +02:00
Alex Blank 5deee8c65c feat: tool definition manifest system (PR 1)
- Add ToolDefinitionManifest model with base image versioning
- Add manifest compiler: Dockerfile + Compose generation from JSON manifests
- Add permission fixer: post-start chown/chmod for mount policies
- Add tool definition CRUD API with live compile preview endpoint
- Integrate manifest-based startup flow in start_instance
- Add Alembic migration with data conversion for pi-agent
- Add 48 unit tests for manifest compiler, permission fixer, docker service
- Keep backward compatibility with legacy dockerfile_template/compose_template

Migration: applied successfully. Pi-agent converted to manifest.
Quality gates: pytest (146 passed, 4 pre-existing unrelated failures)
2026-05-28 13:37:34 +02:00
alex 62d1bdc462 feat: multi-session terminal frontend UI + tests (PR 3)
- Add TerminalSessionTabs component with status dots, rename, close, max-5 limit
- Add 7 component tests for tab rendering, selection, close, rename
- TerminalComponent: sessionId prop, forwardRef with fit() method
- TerminalPage: multi-session orchestration, tab switching, auto-create default
- Fullscreen mode: Alt+Shift+F toggle, auto-hide tabs, Esc exit
- Keyboard shortcuts: Alt+Shift+N/W/ArrowLeft/ArrowRight/R
- Add CSS for tabs, fullscreen, mobile responsive
- Update useTerminalSessions hook for session CRUD
- terminal_manager.py: lookup by internal session_id fallback

Quality gates: tsc --noEmit clean, vitest (7/7 new tests passed), pytest (182 passed)
2026-05-28 13:35:45 +02:00
alex 0b35ae3bf0 feat: multi-session terminal backend API + frontend client (PR 2)
- Add WebSocket route /ws/tool-instances/{instance_id}/terminal/{session_id}
- Preserve /terminal as default-session alias for backward compatibility
- Extract shared _handle_terminal_websocket handler for both routes
- Add REST endpoints: GET list, POST create, DELETE close, POST reset, POST rename
- Preserve legacy POST .../terminal/reset as default session alias
- Add frontend API client (apps/web/src/api/terminal.ts)
- Add useTerminalSessions React hook for session CRUD + state management
- Add integration tests for auth requirements on all new endpoints

Quality gates: pytest (8 new passed, 182 total passed, 51 pre-existing failures)
2026-05-28 12:08:37 +02:00
alex b55300ff6f feat: multi-session terminal backend core (PR 1)
- Add TerminalSessionModel DB table with instance_id FK, name, status,
  created_at, last_activity_at, closed_at columns
- Add Alembic migration for terminal_sessions table
- Refactor TerminalManager to use composite key (instance_id, session_id)
  supporting up to 5 concurrent sessions per instance
- Add create_session, get_session, get_sessions_for_instance, close_session
- Preserve get_or_create_session for backward compatibility (default session)
- Fix attach_websocket to only close sockets within same session
- Add name (auto-generated 'Session N') and status tracking to TerminalSession
- Add 7 unit tests for multi-session logic

Quality gates: pytest (7 new passed, 174 total passed, 51 pre-existing failures)
2026-05-28 11:38:22 +02:00
Alex Blank 314ba3aee4 Merge branch 'fix/container-name-case-sensitivity' into dev 2026-05-28 10:58:18 +02:00
Alex Blank 29943ac239 fix: lowercase container name filter for case-sensitive docker ps
- get_container_id() and get_container_name() now lowercase the
  instance name before passing to docker ps --filter, because
  Docker container names are lowercase internally and the filter
  is case-sensitive. This caused container_id to never be captured
  when instance.name contained uppercase chars (e.g. 'Headquarter'),
  breaking terminal WebSocket connections.

- Also guard proc.stdout being None in start_cloudflared_tunnel().

- Add unit tests for get_container_id and get_container_name.

Quality gates: pytest (14 passed), python clean
2026-05-28 10:56:33 +02:00
alex 22474cdba5 style: fix all ruff and eslint errors across codebase
Backend (ruff):
- Fix 106 errors: move imports to top of file (E402)
- Remove unused imports (F401)
- Add missing imports for undefined names (F821)
- Remove unused variables (F841)
- Fix test_models.py broken RefreshToken test
- Fix test_projects_api.py missing TestClient import

Frontend (eslint):
- Remove unused imports/variables across 10 files
- Fix explicit any types in client.ts and sessions.ts
- Clean up empty block statements in terminal.tsx

Quality gates: ruff (pass), eslint (pass), tsc --noEmit (pass),
pytest (98 passed, 4 pre-existing failures)
2026-05-28 10:15:59 +02:00
alex 0c839e8c6f fix: terminal 4004 infinite reconnect loop for pi-agent tool type
- Add stdin_open: true and tty: true to dockerfile-based compose generation.
  Without these, bash (PID 1) exits immediately, causing a container restart
  loop that makes the instance invisible to docker ps and triggers 4004.
- Treat WebSocket close codes 4001/4003/4004 as permanent errors in the
  frontend. Stop retrying and show the server reason to the user.
- Prevent visibilitychange handler from resetting retry attempts after a
  permanent error has occurred.
- Use docker ps -a in get_container_id/get_container_name to find
  stopped/exited containers for diagnostics.

Quality gates: tsc --noEmit (pass), pytest (98 passed, 4 pre-existing failures)
2026-05-28 09:33:35 +02:00
Alex Blank c63cf7db50 fix: handle existing git mount dirs and invalid compose ports
- Fix git mount clone to check correct path (repo-clone subdir)
- Pull updates instead of re-cloning when git mount dir exists
- Add compose file sanitization to remove invalid port 0 mappings
- Fixes startup failures for existing instances with old compose files
2026-05-27 22:34:24 +02:00
Alex Blank d9d2b91384 fix: skip port mapping for terminal-only tools in dockerfile compose 2026-05-27 22:29:37 +02:00
Alex Blank d6ea5fb1fd chore: clean up debugging logs and console prints
Frontend:
- Remove 18 console.log/warn/error statements from terminal.tsx
- Remove console.warn from icon.tsx

Backend:
- Downgrade routine logger.info to logger.debug in tool_instances.py, terminal.py,
  terminal_session.py, terminal_manager.py, auth.py, docker_build.py, clone.py,
  config_profiles.py, user_config.py
- Keep important lifecycle events as logger.info:
  * Instance creation, start, running state
  * Docker build success/failure
  * Terminal session creation and reset
  * Auth success and user creation
  * Readiness probe success
  * Tunnel creation/stop
2026-05-27 22:16:59 +02:00
Alex Blank 1883825b18 fix: run docker build in thread pool to avoid blocking API 2026-05-27 22:08:01 +02:00
Alex Blank bc71fd6fac fix: lowercase docker image tag to avoid invalid reference format 2026-05-27 22:02:18 +02:00
Alex Blank 28aa9ccf5a feat: add pi-agent tool type migration
- Adds pi-agent to tool_types table with terminal interface
- Includes Dockerfile template for pi.dev coding agent
- Idempotent: checks for existing entry before insert
2026-05-27 16:30:27 +02:00
Alex Blank e23dcdf4e1 fix: remove broken ~ expansion, require working_directory for relative paths 2026-05-27 15:33:06 +02:00
Alex Blank f05ac55875 fix: expand ~ in git mount target paths to avoid /tmp 2026-05-27 15:27:44 +02:00
Alex Blank 33d08faf70 feat: allow relative target paths for git mounts
- Remove absolute path requirement from target_path validation
- Resolve relative paths against working_directory at instance startup
- Fall back to /home/user if no working_directory is configured
- Update frontend to allow relative target paths
- Update spec to document relative path support
- Update tests to allow relative paths and test path traversal rejection
2026-05-27 15:01:16 +02:00
Alex Blank 8a58c61278 fix: align git mount implementation with spec
- _checkout_branch now returns bool and falls back gracefully on failure
- Glob warning message includes matched file count
- Fix database model comment to reference remote_url
- Update tests for new branch checkout behavior

All 51 tests pass
2026-05-27 14:38:11 +02:00
Alex Blank 89ca9f10c7 feat: simplify git mounts to use direct URLs instead of repo references
- Change git mount schema from repo_id to remote_url
- Remove database lookups for git mount resolution
- Clone directly from URL at instance startup
- Simplify frontend UI to text input for Git URL
- Fix route ordering in git_repositories.py to prevent 422 errors
- Update all tests to use remote_url field

Breaking change: Git mounts now use remote_url instead of repo_id
2026-05-27 11:53:25 +02:00
Alex Blank e07938098a feat: add external repository support for config profile git mounts
- Add POST /repositories endpoint for external repos (no project_id)
- Update GitRepositoryResponse to allow nullable project_id
- Update list_repositories to support listing all user repos
- Add _pull_repository_updates for auto-pull on container creation
- Update git mount validation to allow external repos
- Frontend: Update listRepositories to support optional projectId
- Spec updates: external repos, auto-clone, per-instance isolation
2026-05-27 11:03:12 +02:00
Alex Blank 943b9db5c7 fix: remove merge migration that references non-existent file
The server is missing 2026_05_27_make_project_id_nullable.py but the
merge migration referenced it. Removing the merge migration leaves a
clean single head chain.
2026-05-27 10:48:25 +02:00
Alex Blank a4604d6a9a fix: add merge migration to resolve multiple heads from renamed migration
The server has both the old (2026_05_27_make_project_id_nullable) and
new (2026_05_27_external_repos) migration files, creating two heads.
This merge migration resolves them into a single head.
2026-05-27 10:42:44 +02:00
Alex Blank 18204628cc fix: shorten alembic revision name and expand version_num column
- Rename 2026_05_27_make_project_id_nullable to 2026_05_27_external_repos
  (33 chars exceeded VARCHAR(32) limit in alembic_version table)
- Add alembic_version column expansion to VARCHAR(64) in migration
- Ensure future migrations won't hit the 32 char limit
2026-05-27 10:36:47 +02:00
Alex Blank 93b415c53e feat: support external repositories for git mounts
- Make project_id nullable in git_repositories table (migration)
- Allow external repos not tied to any project
- Update validation to allow user-owned external repos in git mounts
- Add /projects/repositories endpoint to list all user repos
- Update frontend to fetch all user repos for git mount selector
- TypeScript and build pass
2026-05-27 10:28:31 +02:00
Alex Blank e7adfb462b fix: use correct revision ID for alembic down_revision 2026-05-27 10:21:20 +02:00
Alex Blank ed1d6528c6 fix: resolve alembic multiple heads by correcting migration dependency chain 2026-05-27 10:16:06 +02:00
Alex Blank 0ec20b9c23 test: add comprehensive tests for config profile git mounts
- Add git mount merge function tests
- Add profile resolution tests with git mounts
- Add integration tests for CRUD with git mounts
- Add glob expansion tests (patterns, limits, repo boundary)
- Add branch checkout tests (success and failure)
- Add error handling tests for missing repos/invalid UUIDs

All 52 tests pass.
2026-05-26 22:49:12 +02:00
Alex Blank 4c11163bff feat: implement config profile git mounts
- Add git_mounts column to config_profiles table (JSONB)
- Create GitMount Pydantic models with validation
- Add repo validation in create/update endpoints
- Update profile resolver to merge git mounts from includes
- Implement auto-clone, branch checkout, and glob expansion
- Add parallel processing for git mount resolution
- Create GitMountEditor frontend component
- Update TypeScript types and API clients
- Add CSS styles for git mount UI
- Frontend type check and build pass

Implements tasks 1.1-7.9 of config-profile-git-mounts spec
2026-05-26 22:27:04 +02:00
Alex Blank 4c216dd1ca refactor: extract shared Pydantic validators
- Create shared_validators.py with validate_mount_path, validate_files, validate_env_vars, validate_volumes
- Refactor config_folders.py to use shared validators
- Refactor tool_configs.py to use shared validators
- Refactor config_profiles.py to use shared env_vars validator
- Reduce ~80 lines of duplicate validation code
2026-05-25 14:05:12 +02:00
Alex Blank a37a3122f9 refactor: extract shared validation and reduce duplication
- Extract tool_types validation to shared module (validate_compose_yaml, check_port_exposed, validate_required_variables)
- Extract _get_user and _get_owned_project to auth/dependencies.py
- Create useAsyncData hook and apply to 6 pages
- Create extractErrorMessage utility
- TypeScript and build pass
2026-05-25 14:01:32 +02:00
OpenCode Agent 01a0ef46c9 feat: terminal startup command and container tools
- Add startup_command field to ToolType model and API
- Execute startup command before interactive shell in terminal sessions
- Add tmux and ranger to OpenCode container spec
- Update Tool Workshop UI with startup_command input for terminal types
- Add backend tests for startup_command CRUD operations
- Sync specs: tool-terminal, tool-types-definition, opencode-web-server
- New spec: tool-terminal-startup-command

Quality gates: Frontend typecheck/lint passed. Backend tests blocked by environment (Python/Docker not available).

OpenSpec: terminal-startup-and-container-tools
2026-05-24 22:21:55 +00:00
alex 1fc244e818 fix: show time alongside date in session created_at 2026-05-24 21:40:26 +00:00
Fusion 51b5d723ac fix: terminal clear on reset and data loss for text starting with {
Frontend:
- Clear xterm.js screen when receiving 'connected' status after reset
- Send resize message after clearing to ensure proper dimensions
- Fixes terminal artifacts after reset

Backend:
- Fix data loss bug: text starting with '{' but not valid JSON was silently dropped
- Now writes such text to session as regular input
- Fixes missing characters when user types '{'
2026-05-24 21:15:04 +02:00
Fusion 33b482ce91 fix: terminal reconnect typing and reset functionality
Frontend:
- Fix term.onData to use wsRef.current instead of captured ws variable
- Fix fitTerminal to use wsRef.current for resize messages
- Fix sendData callback to use wsRef.current
- This fixes 'cannot type' after WebSocket reconnect

Backend:
- Add SessionRef class for mutable session reference
- Update _read_loop and _write_loop to use SessionRef
- Reset now updates session_ref.session instead of returning
- This keeps the WebSocket alive after reset instead of closing it
2026-05-24 20:57:33 +02:00
Fusion 0cb2eefd29 fix: send SIGWINCH to docker exec process for container terminal resize
Instead of sending stty commands through the user's terminal session
(which causes 'inappropriate ioctl' errors), send SIGWINCH signal to
the docker exec process on the host. Docker exec should forward this
to the container process, causing the shell to re-read its terminal size.

This avoids:
- Visible stty commands in the terminal
- ioctl errors from stty
- Interference with user's shell session
2026-05-24 20:47:14 +02:00
Fusion 5f5dc9c851 fix: revert docker exec -i change and use stty with line hiding
Reverted docker exec back to -it (required for interactive bash).
Instead, sends stty command with \r to hide it from the terminal display:
- \r moves cursor to start of line (overwrites prompt)
- stty command executes silently (no output on success)
- \r moves cursor back to start, hiding echoed command

This sends stty on EVERY resize so the container shell always matches
frontend dimensions.
2026-05-24 20:40:02 +02:00
Fusion 4864d269e8 Merge branch 'dev' of ssh://git.commumedia.org:2222/alex/headquarter into dev 2026-05-24 20:35:16 +02:00
Fusion 29e4bed9e6 fix: use docker exec -i instead of -it so host PTY resize propagates to container
Removes -t flag from docker exec so it uses our PTY slave directly instead
of creating its own PTY inside the container. This allows TIOCSWINSZ on the
host PTY master to propagate naturally to the container shell via SIGWINCH.

Also removes all stty command injection logic since resize now works natively.
2026-05-24 20:35:07 +02:00
alex b04a458975 Merge branch 'dev' of ssh://git.commumedia.org:2222/alex/headquarter into dev 2026-05-24 18:29:10 +00:00
alex 2299bd51ba fix: handle already-serialized mount dicts in profile update
When FastAPI parses the request body and model_dump() is called,
nested MountItem models are already serialized to plain dicts.
The update handler was unconditionally calling model_dump() again,
causing AttributeError on dict objects.

Fix: Check if mount items are already dicts before calling model_dump().

Fixes: 422 error when updating profiles with mounts.
2026-05-24 18:29:03 +00:00
Fusion c82e628e6f fix: send stty resize on every resize with hidden command output
Backend:
- Remove _stty_sent guard to send stty on EVERY resize
- Use stty -echo to hide command, then delete the command line with ANSI escapes
- Change log level from info to debug

Frontend:
- Add window resize listener as fallback to ResizeObserver
- 250ms debounce to avoid excessive refits
- Proper cleanup on unmount
2026-05-24 20:24:41 +02:00
Fusion 4b09f611d6 Merge branch 'dev' of ssh://git.commumedia.org:2222/alex/headquarter into dev 2026-05-24 20:13:17 +02:00