The workspace mount was failing because /data/working-copies/ was not
bind-mounted into the API container. The workspace management code writes
to /data/working-copies/ inside the API container, but tool instances
mount from the host filesystem. Without a shared bind mount, the host
saw an empty directory.
- docker-compose.yml: add /data/working-copies bind mount, replace repo_data
- docker-compose.traefik.yml: same changes
- Remove repo_data named volume declaration from both files
- Log REPO_PATH, SSH_PATH, EXTRA_VOLUMES, manifest mounts, and resolved
volumes in compile_compose() to trace why mounts may be missing
- Log repo_path and generated compose content in _prepare_manifest_instance()
to verify the full compose YAML at start time
- Convert WorkspaceHasInstancesError to store plain dicts instead of
SQLAlchemy ORM objects, preventing lazy-load failures outside async
session context (MissingGreenlet)
- Update both delete endpoints (top-level and nested) to use exc.instances
directly since they're already plain dicts
- Add no-cache headers for index.html in nginx.conf so browsers always
fetch new hashed JS/CSS bundles on deploy
1. Remove Docker build from create_instance for manifest types — the build
was blocking the HTTP request for several minutes, causing frontend
timeouts and retries. Image is now built lazily on start (via the
existing _prepare_manifest_instance path in start_instance).
2. Increase MAX_CONNECTIONS_PER_USER from 5 to 20 for SSE endpoint —
aggressive reconnect loops from the frontend were exhausting the limit
and causing 429 errors unrelated to tool starting.
Quality gates: ruff clean, tsc --noEmit clean, pytest workspaces (9 passed)
Bug 1 — in-container repo mounting:
- docker-compose.yml: added /data/working-copies:/data/working-copies mount
to API container so workspace dirs are visible on host filesystem
- Dockerfile: create /data/working-copies dir in image
Bug 2 — /home/user not writable:
- workspace_manager.py: chmod 777 workspace dirs + 666 files after clone
and after sync, so any container user can write
- manifest_compiler.py: explicit mkdir + chown + chmod 755 for home dir
in generated Dockerfile
Bug 3 — terminal text shifts left on typing:
- terminal.tsx: removed manual term.refresh() after fit (caused reflow)
- Track lastSentCols/lastSentRows and only send resize when dimensions
actually changed, preventing resize feedback loops
Bug 4 — ESC key captured by terminal:
- terminal.tsx: attachCustomKeyEventHandler allows ESC to propagate to
browser when not in alternate buffer (vim/tmux), so modals/navigation
work; ESC still sent to PTY when in vim/tmux alternate screen
Quality gates: ruff clean, tsc --noEmit clean, pytest workspaces (9 passed)
- New StartToolFAB component: fixed floating button (bottom-right) opens
a modal with workspace selector + ToolStarter
- Added to AppShell: available on every page except mobile terminal
- Dashboard (home): removed old CreateSessionForm and 'Quick create' section,
replaced with FAB + 'Workspaces quick access' prompt
- Sessions page: removed inline workspace selector + ToolStarter, now
shows prompt to use the FAB
- Styles: .start-tool-fab with hover scale, shadow, mobile offset above tab bar
Quality gates: tsc --noEmit clean, pytest workspaces API (9 passed, 1 skipped)
- Delete inline hardcoded StartToolModal from workspace-detail.tsx
- Import shared StartToolModal that fetches real tool types from API
- Pass workspace object to ToolsTab so shared modal gets proper context
- onStart handler passes optional configProfileId through to useWorkspaceInstances.create()
Quality gates: tsc --noEmit clean
- Fetch real tool types from API instead of hardcoded string names
- Use actual tool type UUID (id) as select value
- Remove fake 'terminal' option — terminal is a feature, not a tool type
- Show display_name in dropdown, handle loading/error states
Quality gates: tsc --noEmit clean
- GitService.clone() now accepts ssh_key and sets up GIT_SSH_COMMAND env
- WorkspaceManager.create() loads repo SSH key from DB and decrypts it
- Both workspace create endpoints pass session for SSH key lookup
Quality gates: ruff clean, pytest workspaces API (9 passed, 1 skipped)
Backend (git_repositories.py):
- get_repository_branches: check for .git dir OR HEAD file (handles bare repos)
- When local repo is missing, git ls-remote fallback now uses SSH key auth
via _prepare_ssh_env() for repos with ssh_key_id
- Cleans up temp SSH key file after ls-remote
- Logs ls-remote stderr/exit code for debugging
- Returns server's detail message instead of raw axios 404 text
Frontend (use-git-repo.ts):
- extractError() helper pulls server detail/message from axios responses
- User sees 'repository not found on disk — re-clone or re-create'
instead of generic 'Request failed with status code 404'
Quality gates: ruff clean, tsc --noEmit clean, 11 passed + 1 pre-existing failure
Backend (git_repositories.py):
- get_repository_branches now checks for .git subdirectory (not just dir existence)
- If local repo is corrupt/missing but has remote_url, falls back to git ls-remote
to list branches from the remote
- Returns 404 with actionable message instead of 400 with raw git stderr
- Pre-existing test failure in test_git_repository_clone_preflight.py unchanged
Frontend (workspace-create-form.tsx):
- When branch API fails, auto-switches to manual text input (no dropdown selection needed)
- Shows hint text: 'Couldn't load branches — type one manually'
- useGitRepo hook auto-fetches branches when projectId/repoId change
Quality gates: ruff clean, tsc --noEmit clean, 93 passed + 1 pre-existing failure
- Rewrite WorkspaceCreateForm as unified component used in both pages
- Standalone mode (WorkspacesPage): shows project/repo/branch selectors
- Contextual mode (ProjectsPage): accepts defaultProjectId/defaultRepoId,
skips project/repo selectors, shows only name + branch dropdown
- Branch dropdown fetched from repo via listRepositoryBranches API
- Auto-selects first/only option for project, repo, and branch
- '+ Create new branch...' option reveals text input for custom branch
- Falls back to free-text branch input if branch API fails
- Removes duplicated inline creation logic from WorkspacesPage
- TypeScript + eslint clean
- Fetch branches from selected repo via listRepositoryBranches API
- Branch dropdown with default branch pre-selected
- '+ Create new branch...' option reveals text input for custom branch
- Auto-select first option when only one available:
- Project: auto-selects when only 1 project
- Repo: auto-selects when only 1 repo
- Branch: auto-selects when only 1 branch, otherwise defaults to remote default
- Falls back to free-text branch input if branch API fails
- TypeScript + eslint clean
- Replace awkward first-workspace-guessing logic with inline project/repo selector
- New WorkspaceCreateInline component with cascading dropdowns:
- Select project → loads repositories for that project
- Select repository → enter workspace name + branch
- Submit creates workspace via top-level POST /workspaces/
- Add createWorkspaceTopLevel() API client for flat endpoint
- Works even with zero existing workspaces (shows create button in empty state)
- Add CSS grid layout for inline create form
- TypeScript + eslint clean
- Add all_workspaces_router with GET /workspaces/ (no project/repo required)
- Include project_id in workspace responses
- Frontend: useWorkspaces() calls listAllWorkspaces when no args
- Frontend: WorkspacesPage uses top-level list, derives project/repo from workspace for mutations
- Fixes 422 from invalid UUID path params
FastAPI auto-redirects /workspaces to /workspaces/ with 307.
Behind Traefik (HTTP internal), the 307 becomes http://,
triggering Mixed Content in the browser. Adding trailing
slashes avoids the redirect entirely.
Fixes NameError: ToolInstance not defined at runtime because
type annotations are evaluated at class definition time.
Deferring annotation evaluation with __future__ annotations
keeps TYPE_CHECKING imports from causing runtime crashes.
Also includes ruff formatting cleanup on workspace-related files.
- Add workspace_id to CreateInstanceRequest (optional, replaces clone_mode)
- create_instance: resolve workspace, validate repo ownership, use workspace.path
- create_instance: store workspace_id on ToolInstance record
- start_instance: use workspace.path when workspace_id is set (manifest + legacy flows)
- Skip SSH key mount for clone mode when workspace is used
- Backward compatible: clone_mode still works when workspace_id is absent
The API container used a named Docker volume (repo_data:/data/repos) for
storing repositories. When creating tool instances with direct mount mode,
the API told Docker to bind-mount /data/repos/<repo>:/workspace into the
tool container. But the Docker daemon resolves bind-mount paths on the HOST
filesystem, not inside the API container. Since the host had no /data/repos
(the repos only existed inside the named volume), tool containers mounted
empty directories.
Changed both compose files to use a host bind mount (/data/repos:/data/repos)
instead of a named volume. This ensures:
- The API container and tool containers both see the same /data/repos path
- Bind mounts from /data/repos into tool containers work correctly
For existing installations: repos previously stored in the repo_data named
volume should be copied to /data/repos on the host before restarting the
stack.
Quality gates: compose file syntax valid
- apps/api/src/services/tunnel.py: add 2-second sleep after discovering the
tunnel URL to allow Cloudflare DNS edge propagation before returning
- apps/web/src/hooks/use-instance-actions.ts: show alert() with the backend
error message when recreate tunnel fails, instead of silently swallowing
errors
Quality gates: ruff clean, tsc clean
Docker container names are case-sensitive for 'docker inspect' but case-
insensitive for Docker DNS. Compose templates may render container names
with mixed case (e.g. code-server-Headquarter-abc123), causing exact-name
docker inspect to fail while DNS resolution in tunnels works fine.
- apps/api/src/services/docker.py: get_container_id now tries exact match
first, then falls back to case-insensitive exact match via 'docker ps'
- apps/api/src/api/tool_instances.py: recreate_tunnel_endpoint uses
get_container_id instead of its own docker inspect call
Quality gates: ruff clean
docker ps --filter name= uses substring matching, so searching for
code-server-headquarter-abc123 also matches tunnel-code-server-headquarter-abc123.
This caused start_instance to store the tunnel container's ID instead of the
tool container's ID, breaking tunnel connectivity and all container operations.
Switched both helpers to docker inspect, which does exact name matching.
Quality gates: ruff clean
Adds INFO-level logging to trace exactly what happens during tunnel
recreation: container lookup, network membership, target IP/URL,
tunnel creation result, health check, and direct curl probe from API.
This will help diagnose why recreated tunnels return 502 while
original tunnels work.
Quality gates: ruff clean
Old instances may have auto-generated Docker Compose container names
that don't match instance.name.lower(), causing DNS resolution failures
for the tunnel. Also, old instances may not be on the backend network.
- apps/api/src/services/docker.py: add get_container_ip_on_network() and
is_container_on_network() helpers
- apps/api/src/services/tunnel.py: start_tunnel() and recreate_tunnel() now
accept an optional target_url parameter to override the default name-based URL
- apps/api/src/api/tool_instances.py: recreate_tunnel_endpoint now:
1. Looks up the tool container (by stored container_id or name)
2. Ensures it's connected to the backend network
3. Gets the container's IP on that network
4. Passes the IP as the explicit tunnel target
This guarantees the tunnel can reach the tool container regardless of
naming or network state.
Quality gates: ruff clean
The previous endpoint blocked recreation if the tunnel was 'healthy'
or returned an 'error_response', making the Recreate Tunnel button
ineffective in many cases.
- apps/api/src/api/tool_instances.py: removed the health-check guards
from recreate_tunnel_endpoint. It now unconditionally stops the old
tunnel and creates a new one, then commits the new URL to the DB.
- Frontend useInstanceActions already calls onRefresh() after success,
so the UI updates with the new tunnel URL automatically.
Quality gates: ruff clean
The post-creation 'docker network connect' was failing silently for
unknown reasons (race condition, container state, or Docker internals).
Instead of fighting with this, we now inject the backend network directly
into the compose file before 'docker compose up'. Docker Compose then
attaches the container to the network atomically during creation.
- apps/api/src/api/tool_instances.py: new _ensure_backend_network_in_compose()
adds 'networks: [backend_name]' to the service and declares the network
as external at the top level
- apps/api/src/api/tool_instances.py: call _ensure_backend_network_in_compose()
in both start_instance and restart_instance, right after
_ensure_container_name_in_compose()
- apps/api/src/api/tool_instances.py: removed connect_container_to_network
import and call entirely
- apps/api/src/api/tool_instances.py: import get_backend_network_name from
docker module for use in the new helper
Quality gates: ruff clean
Docker Compose prefixes network names with the project directory name
(e.g. 'headquarter_backend' instead of 'backend'). The previous code
hardcoded 'backend', causing 'network not found' errors.
- apps/api/src/services/docker.py: add get_backend_network_name() that
inspects the API container (hq-api) to find the actual network name
- apps/api/src/services/docker.py: connect_container_to_network() now
auto-detects the network name when not explicitly provided
- apps/api/src/services/tunnel.py: import and use get_backend_network_name()
- apps/api/src/api/tool_instances.py: remove explicit 'backend' arg from
connect_container_to_network() call
Quality gates: ruff clean
The host-network tunnel approach had issues because localhost inside
the tunnel container wasn't reaching the host-published ports correctly.
This reverts to running cloudflared as a Docker container on the
'backend' network, where Docker DNS resolves container names reliably.
The tunnel connects to http://{container_name}:{container_port}.
- apps/api/src/services/tunnel.py: use --network backend instead of host
- apps/api/src/api/tool_instances.py: pass container_port (default_port)
instead of published_port (host port) to tunnel functions
Quality gates: ruff clean
- Remove --rm from docker run so failed containers persist for inspection
- Add --no-autoupdate flag to prevent cloudflared from exiting on auto-update
- Capture both stdout and stderr from docker logs
- Check container exit code during wait loop; fail fast with logs if container exits early
- Include exit code in timeout error message for easier debugging
Replace the subprocess-based tunnel implementation with Docker containers
running on the host network. This eliminates all container name resolution
bugs that caused tunnel 502 errors.
New design:
- Each tunnel is a docker run --network host cloudflare/cloudflared container
- cloudflared connects to localhost:{published_port} (Docker port forwarding)
- No dependency on container names, backend network DNS, or binding diagnostics
- Tunnels named predictably: tunnel-{instance_name}
- Start/stop/recreate use container names instead of PIDs
Files changed:
- NEW: apps/api/src/services/tunnel.py — clean tunnel module (start/stop/recreate/health)
- apps/api/src/services/docker.py — removed 250 lines of old tunnel code
- apps/api/src/api/tool_instances.py — use new tunnel module, store container_name
- apps/api/src/services/health_monitor.py — updated import
- apps/web/src/components/session-card.tsx — Recreate Tunnel button always visible
Quality gates: ruff clean, 13 tests passed (health_monitor + notifications)
Show the Recreate Tunnel button on all active web-enabled session cards
(instead of only when tunnel_status is unreachable) so users can manually
trigger tunnel recreation at any time. Also adds it to the mobile action sheet.
Quality gates: eslint clean, tsc clean
- Dual-mode touch scroll:
- Normal mode: scroll .xterm-viewport directly when scrollHeight > clientHeight
- Alternate screen (tmux/vim): send SGR 1006 mouse-wheel protocol data
using cursor position so tmux knows which pane to scroll
- Add touch-action: none to .terminal-container to prevent browser gestures
- Lock both html and body overflow when terminal page is open on mobile
- Remove synthetic WheelEvent approach (xterm.js SmoothScrollableElement
doesn't reliably handle synthetic events)
- Attach capture-phase touch listeners to .terminal-container (parent of xterm)
- On vertical swipe: e.preventDefault() blocks page scroll, then directly
adjust .xterm-viewport.scrollTop by the swipe delta
- This bypasses term.scrollLines() API and directly manipulates the DOM
element that xterm.js watches via its internal scroll handler
- Remove all CSS touch-action overrides — container handles it in JS
- Add full mobile viewport CSS: overflow-y scroll, -webkit-overflow-scrolling
touch, overscroll-behavior-y contain, translate3d hardware accel,
scroll-behavior smooth, touch-action pan-y
- After term.open(), find .xterm-viewport and add passive touch listeners
that call stopPropagation() (not preventDefault) — this lets the browser
handle native touch scrolling while preventing xterm.js internal handlers
from interfering
- Based on xterm.js known issue #5489 and SCROLLING_FIX.md approach
- xterm.js has zero touch event handlers (verified: only 1 'touch' ref in
entire library), so it wasn't intercepting anything
- Our touch-action: none + preventDefault() combo was blocking the browser
from scrolling the .xterm-viewport natively
- Removed all custom touch event handlers from terminal.tsx
- Removed touch-action: none from .terminal-container
- Added touch-action: pan-y to .xterm-viewport so browser allows vertical pan
- Body scroll lock (terminal-page-open) prevents page from scrolling
- Attach touch listeners to document with capture:true instead of container
- Check if touch target is inside terminal container before handling
- This runs before xterm.js internal handlers, giving us full control
- Add touch-action: none to terminal container to prevent browser gestures
- Lower threshold to 3px, 20px per line for responsive scrolling
Alex Blank
alex