Compare commits

..

9 Commits

Author SHA1 Message Date
alex fc72c5f6e9 fix: resolve mapper configuration for ToolDefinitionManifest and Workspace
Move ToolDefinitionManifest import out of TYPE_CHECKING in tool_type.py
so SQLAlchemy can resolve the string-annotated relationship during mapper
configuration.

Add Workspace to models/__init__.py (before ToolInstance) so the
ToolInstance-Workspace relationship can be resolved.

Quality gates: py_compile passed, ruff passed, all mappers configure OK.
2026-06-03 14:24:22 +02:00
Alex Blank 51a399c775 feat: open all sessions in new tabs; sidebar terminal links
Sidebar SessionItem was only opening web tool URLs in new tabs.
Terminal sessions linked to the project page in the same tab.
SessionCard 'Open' buttons for terminal sessions navigated in-place.

Changes:
- app-shell.tsx: SessionItem now builds terminal URLs
  (/instances/:id/terminal) and always uses target=_blank
- session-card.tsx: compute openHref for both web and terminal sessions,
  render <a> links with target=_blank instead of callback buttons
- use-instance-actions.ts: handleOpen now uses window.open(..., '_blank')
  for terminal sessions and project fallback

All session opening (sidebar, cards, callbacks) now consistently opens
in a new tab.

Quality gates: tsc clean
2026-06-02 15:54:03 +02:00
Alex Blank fc75eeb76d fix: add SSH key selection to workspace tool starter + docker compose policy
tool-starter.tsx was hardcoding ssh_key_ids=[] and only showing a read-only
SSH key status. Users couldn't select keys when starting tools from workspaces.

Changes:
- tool-starter.tsx: add checkboxes for SSH key selection with repo key
  pre-selected, pass selected keys to createInstance/startInstance
- AGENTS.md: add explicit rule forbidding docker compose commands without
  user approval and proper isolation

The web container must be rebuilt to pick up the frontend changes:
  docker compose up -d --build web

Quality gates: tsc clean, pytest (19 passed, 1 skipped)
2026-06-02 15:40:20 +02:00
Alex Blank 37134b8c18 fix: terminal EOF detection and dead session cleanup
When a tool container stops, the docker exec PTY reaches EOF. Previously,
the event-driven reader silently returned on EOF, leaving websockets
attached to a dead session. Input writes then failed silently.

Changes:
- _on_fd_readable: detect EOF (empty read) and call _handle_eof()
- _handle_eof: stop reading, mark process dead, close all websockets
  with code 4001 to force frontend reconnection
- write_input: detect write errors and trigger EOF cleanup

Quality gates: pytest (19 passed, 1 skipped)
2026-06-02 15:23:16 +02:00
Alex Blank c6b804bf0a feat: fix SSH key mounting with multi-key support and unique filenames
SSH key mounting was broken because:
1. Each selected key was mounted to a separate source dir but all targeted
   the same ~/.ssh path in the container, causing Docker Compose's
   last-mount-wins behavior
2. All keys were named id_ed25519, so they'd overwrite each other

Changes:
- ssh_keys.py: add key_filename param to prepare_ssh_key_files for unique
  key names; add write_ssh_config for combined multi-key config
- tool_instances.py: collect all selected keys into a single ~/.ssh mount
  with sanitized unique filenames (id_ed25519_<name>); generate combined
  SSH config with all IdentityFile entries
- tests: add os.makedirs mock for SSH permission tests

Quality gates: pytest (19 passed, 1 skipped)
2026-06-02 15:03:26 +02:00
Alex Blank c754984df8 feat: high-performance web terminal with asyncio-native I/O
Complete rewrite of the terminal pipeline for VS Code Server-level
responsiveness. Key improvements:

Backend:
- Replace blocking select.select(0.1) with asyncio.add_reader() for
  event-driven PTY reading (eliminates ~110ms polling latency)
- Add output batching (2ms window) to reduce WebSocket frame overhead
- Add flow control: client acks processed bytes, server pauses PTY reads
  at 64KB threshold, resumes at 32KB
- Add 5s ack timeout fallback to prevent stuck sessions

Frontend:
- Switch WebSocket to binary mode (binaryType = 'arraybuffer')
- Eliminate Blob -> arrayBuffer async conversion overhead
- Add flow control ack messages (every 4096 bytes or 100ms)
- Add xterm-addon-webgl with graceful DOM fallback
- Add performance tuning (scrollback=10000, fastScrollSensitivity)

SDD artifacts:
- openspec/explorations/terminal-responsiveness.md
- openspec/proposals/terminal-responsiveness.md
- openspec/specs/terminal-responsiveness.md
- openspec/designs/terminal-responsiveness.md
- openspec/tasks/terminal-responsiveness.md

Quality gates: pytest (19 passed, 1 skipped), tsc --noEmit clean
2026-06-02 14:40:32 +02:00
Alex Blank 906aab3b73 fix: workspace creation with stale directories and missing bind mount
- workspace_manager.py: remove stale workspace directories before cloning
  to prevent 'already exists' errors from previous failed attempts
- workspaces.py: add ValueError -> 400 handling, keep 409 for duplicates
- test_tool_instances_legacy.py: fix broken patches for new helpers
  (get_container_name removed, _ensure_backend_network_in_compose added,
  workspace_id/ssh_key_ids mock attributes added)
- docker-compose.traefik.yml: add /data/working-copies bind mount

Quality gates: pytest (19 passed, 1 skipped)
2026-06-02 13:41:07 +02:00
Alex Blank e1aaf9f6fc Merge branch 'fix/workspace-mount-bind-mount' into dev
Resolve duplicate working-copies mount declaration.
2026-06-02 13:13:15 +02:00
Alex Blank 6170306d9e fix: add working-copies bind mount and replace repo_data volume
The workspace mount was failing because /data/working-copies/ was not
bind-mounted into the API container. The workspace management code writes
to /data/working-copies/ inside the API container, but tool instances
mount from the host filesystem. Without a shared bind mount, the host
saw an empty directory.

- docker-compose.yml: add /data/working-copies bind mount, replace repo_data
- docker-compose.traefik.yml: same changes
- Remove repo_data named volume declaration from both files
2026-06-02 12:59:40 +02:00
26 changed files with 1400 additions and 243 deletions
+1 -1
View File
@@ -1,3 +1,3 @@
{
"fingerprint": "fdea8a74bb4c7449c01c4bd61646c895b10ede78"
"fingerprint": "c36b11ec5edebc02aa51b1113a7a11dc2559e812"
}
+1 -2
View File
@@ -2,7 +2,7 @@
<!-- Auto-generated by gentle-pi extensions/skill-registry.ts. Run /skill-registry:refresh to regenerate. -->
Last updated: 2026-05-28
Last updated: 2026-06-02
## Sources scanned
@@ -21,7 +21,6 @@ Last updated: 2026-05-28
| Skill | Trigger / description | Scope | Path |
| --- | --- | --- | --- |
| `auto-commit` | Use when you are making multiple edits or completing significant work in a git repository to automatically create commits | user | `/home/alex/.config/opencode/skills/auto-commit/SKILL.md` |
| `openspec` | Use OpenSpec as the source of truth for planning, implementation, verification, and archive discipline. | user | `/home/alex/.config/opencode/skills/openspec/SKILL.md` |
| `openspec-apply-change` | Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks. | project | `/home/alex/projects/headquarter/.opencode/skills/openspec-apply-change/SKILL.md` |
| `openspec-archive-change` | Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete. | project | `/home/alex/projects/headquarter/.opencode/skills/openspec-archive-change/SKILL.md` |
| `openspec-explore` | Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change. | project | `/home/alex/projects/headquarter/.opencode/skills/openspec-explore/SKILL.md` |
+1
View File
@@ -75,6 +75,7 @@ Do not:
* Introduce new dependencies without clear justification.
* Treat existing code as more authoritative than OpenSpec for intended behavior.
* Decide product behavior silently when the spec is unclear.
* Run `docker compose` commands (build, up, down, etc.) without explicit user approval and proper isolation (e.g., feature branches, separate worktrees, or staged rollouts). Docker Compose operations are deployment-level changes that can affect running services, shared volumes, and network state. Always ask first.
If scope must change, propose an OpenSpec update first.
+6 -25
View File
@@ -222,8 +222,7 @@ async def _handle_terminal_websocket(
# Use mutable session reference so loops can survive reset
session_ref = SessionRef(session, slot_session_id)
# Start I/O loops and heartbeat
read_task = asyncio.create_task(_read_loop(session_ref, websocket))
# Start write loop and heartbeat (read is now event-driven in TerminalSession)
write_task = asyncio.create_task(
_write_loop(session_ref, websocket, instance_id)
)
@@ -232,7 +231,7 @@ async def _handle_terminal_websocket(
# Wait for either task to complete (indicating disconnect or error)
done, pending = await asyncio.wait(
[read_task, write_task, heartbeat_task],
[write_task, heartbeat_task],
return_when=asyncio.FIRST_COMPLETED,
)
@@ -267,28 +266,6 @@ async def _handle_terminal_websocket(
)
async def _read_loop(session_ref: SessionRef, websocket) -> None:
"""Read output from the container and send to WebSocket."""
try:
while True:
session = session_ref.session
if not session.is_alive() or session._closed:
await asyncio.sleep(0.1)
continue
data = await session.read_output()
if data:
try:
await websocket.send_bytes(data)
except WebSocketDisconnect:
break
except Exception:
break
else:
await asyncio.sleep(0.01)
except Exception:
pass
async def _write_loop(session_ref: SessionRef, websocket, instance_id: str) -> None:
"""Read input from WebSocket and send to container."""
try:
@@ -319,6 +296,10 @@ async def _write_loop(session_ref: SessionRef, websocket, instance_id: str) -> N
rows,
)
await session.resize(cols, rows)
elif msg_type == "ack":
char_count = ctrl.get("chars", 0)
if char_count > 0:
session.acknowledge_data(char_count)
elif msg_type == "reset":
# Reset terminal session (scoped to current slot)
logger.debug(
+78 -29
View File
@@ -1611,38 +1611,14 @@ async def start_instance(
# Mount selected SSH keys into container home dir
if instance.ssh_key_ids:
from src.services.ssh_keys import write_ssh_config, _sanitize_filename
# Collect all valid keys first
ssh_keys_to_mount = []
for key_id in instance.ssh_key_ids:
ssh_key = await session.get(SSHKey, uuid.UUID(key_id))
if ssh_key and ssh_key.user_id == user_id:
try:
ssh_dir = prepare_ssh_key_files(
instance_dir,
ssh_key,
subdir=f"mounts/ssh/{key_id}/.ssh",
uid=container_uid,
gid=container_gid,
)
ssh_target = os.path.join(home_dir, ".ssh")
extra_volumes.append(
{
"source": ssh_dir,
"target": ssh_target,
"type": "bind",
}
)
logger.debug(
"Mounted SSH key %s for instance %s to %s",
ssh_key.name,
instance.id,
ssh_target,
)
except Exception as exc:
logger.error(
"Failed to prepare SSH key %s for instance %s: %s",
key_id,
instance.id,
exc,
)
ssh_keys_to_mount.append(ssh_key)
else:
logger.warning(
"SSH key %s not found or not authorized for user %s",
@@ -1650,6 +1626,79 @@ async def start_instance(
user_id,
)
if ssh_keys_to_mount:
# Use a single shared .ssh directory so all keys are visible
ssh_dir = os.path.join(instance_dir, "mounts", "ssh", ".ssh")
os.makedirs(ssh_dir, exist_ok=True)
key_filenames = []
for ssh_key in ssh_keys_to_mount:
# Use sanitized key name as filename prefix to avoid collisions
key_name = _sanitize_filename(ssh_key.name)
# If multiple keys have the same name, append a short hash
base_filename = f"id_ed25519_{key_name}"
filename = base_filename
counter = 1
while filename in key_filenames:
filename = f"{base_filename}_{counter}"
counter += 1
key_filenames.append(filename)
try:
prepare_ssh_key_files(
instance_dir,
ssh_key,
subdir="mounts/ssh/.ssh",
uid=container_uid,
gid=container_gid,
key_filename=filename,
write_config=False,
)
logger.debug(
"Prepared SSH key %s as %s for instance %s",
ssh_key.name,
filename,
instance.id,
)
except Exception as exc:
logger.error(
"Failed to prepare SSH key %s for instance %s: %s",
ssh_key.id,
instance.id,
exc,
)
# Write combined SSH config with all keys
try:
write_ssh_config(
ssh_dir,
key_filenames,
uid=container_uid,
gid=container_gid,
)
except Exception as exc:
logger.error(
"Failed to write SSH config for instance %s: %s",
instance.id,
exc,
)
# Mount the single .ssh directory into container home
ssh_target = os.path.join(home_dir, ".ssh")
extra_volumes.append(
{
"source": ssh_dir,
"target": ssh_target,
"type": "bind",
}
)
logger.debug(
"Mounted %d SSH key(s) for instance %s to %s",
len(ssh_keys_to_mount),
instance.id,
ssh_target,
)
# ── MANIFEST-BASED FLOW ──────────────────────────────────────
resolved_manifest = None
+6
View File
@@ -232,6 +232,12 @@ async def create_workspace(
workspace = await manager.create(repo, user_id, name, branch, session=session)
session.add(workspace)
await session.commit()
except HTTPException:
raise
except ValueError as exc:
await session.rollback()
logger.error("Failed to create workspace: %s", exc)
raise HTTPException(status_code=400, detail=str(exc)) from exc
except Exception as exc:
await session.rollback()
logger.error("Failed to create workspace: %s", exc)
+2
View File
@@ -12,6 +12,7 @@ from src.models.tool_instance import ToolInstance
from src.models.tool_type import ToolType
from src.models.user import User
from src.models.user_config import UserConfig
from src.models.workspace import Workspace
__all__ = [
"Base",
@@ -29,4 +30,5 @@ __all__ = [
"ToolType",
"User",
"UserConfig",
"Workspace",
]
+2 -1
View File
@@ -7,8 +7,9 @@ from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.models.base import Base, TimestampMixin, UUIDPrimaryKeyMixin
from src.models.tool_definition_manifest import ToolDefinitionManifest
if TYPE_CHECKING:
from src.models.tool_definition_manifest import ToolDefinitionManifest
from src.models.user import User
+99 -26
View File
@@ -2,6 +2,7 @@
import logging
import os
import re
from pathlib import Path
from cryptography.fernet import Fernet
@@ -22,12 +23,28 @@ def _get_fernet() -> Fernet:
return Fernet(key)
def _sanitize_filename(name: str) -> str:
"""Sanitize a string for use as a filename.
Replaces non-alphanumeric characters with underscores and strips
leading/trailing underscores.
"""
sanitized = re.sub(r"[^a-zA-Z0-9_-]", "_", name)
sanitized = sanitized.strip("_")
# Ensure it's not empty
if not sanitized:
sanitized = "key"
return sanitized
def prepare_ssh_key_files(
instance_dir: str,
ssh_key,
subdir: str = ".ssh",
uid: int | None = None,
gid: int | None = None,
key_filename: str = "id_ed25519",
write_config: bool = True,
) -> str:
"""Decrypt and write SSH key files to instance directory for container mounting.
@@ -37,6 +54,12 @@ def prepare_ssh_key_files(
subdir: Subdirectory within instance_dir to write to (default: ".ssh")
uid: Optional UID to own the files (for bind-mount into non-root container)
gid: Optional GID to own the files
key_filename: Base filename for the key pair (default: "id_ed25519").
The private key will be named "{key_filename}" and the public key
"{key_filename}.pub".
write_config: Whether to write an SSH config file (default: True).
Set to False when combining multiple keys into one directory,
then call write_ssh_config() separately.
Returns:
Path to the .ssh directory
@@ -49,51 +72,101 @@ def prepare_ssh_key_files(
private_key = fernet.decrypt(ssh_key.private_key_encrypted.encode()).decode()
# Write private key with restricted permissions
private_key_path = ssh_dir / "id_ed25519"
private_key_path = ssh_dir / key_filename
private_key_path.write_text(private_key)
os.chmod(private_key_path, 0o600)
# Write public key
public_key_path = ssh_dir / "id_ed25519.pub"
public_key_path = ssh_dir / f"{key_filename}.pub"
public_key_path.write_text(ssh_key.public_key)
os.chmod(public_key_path, 0o644)
# Write SSH config
config_path = ssh_dir / "config"
config_content = """Host *
# Write SSH config (only if requested)
if write_config:
config_path = ssh_dir / "config"
config_content = f"""Host *
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
IdentityFile ~/.ssh/id_ed25519
IdentityFile ~/.ssh/{key_filename}
IdentitiesOnly yes
"""
config_path.write_text(config_content)
os.chmod(config_path, 0o644)
# Set ownership to target container user if requested
if uid is not None or gid is not None:
effective_uid = uid if uid is not None else -1
effective_gid = gid if gid is not None else -1
try:
os.chown(ssh_dir, effective_uid, effective_gid)
os.chown(private_key_path, effective_uid, effective_gid)
os.chown(public_key_path, effective_uid, effective_gid)
os.chown(config_path, effective_uid, effective_gid)
logger.debug(
"Set SSH key ownership to uid=%s gid=%s for %s",
effective_uid,
effective_gid,
ssh_dir,
)
except PermissionError as exc:
logger.warning(
"Cannot chown SSH keys to uid=%s gid=%s (running as uid=%s): %s",
effective_uid,
effective_gid,
os.getuid(),
exc,
)
else:
# Still chown the key files even if we didn't write config
if uid is not None or gid is not None:
effective_uid = uid if uid is not None else -1
effective_gid = gid if gid is not None else -1
try:
os.chown(private_key_path, effective_uid, effective_gid)
os.chown(public_key_path, effective_uid, effective_gid)
except PermissionError:
pass
return str(ssh_dir)
def write_ssh_config(
ssh_dir: str,
key_filenames: list[str],
uid: int | None = None,
gid: int | None = None,
) -> None:
"""Write an SSH config file that includes multiple IdentityFile entries.
Args:
ssh_dir: Path to the .ssh directory
key_filenames: List of key filenames (without .pub extension)
uid: Optional UID to own the config file
gid: Optional GID to own the config file
"""
ssh_dir_path = Path(ssh_dir)
ssh_dir_path.mkdir(parents=True, exist_ok=True)
config_path = ssh_dir_path / "config"
lines = ["Host *"]
lines.append(" StrictHostKeyChecking no")
lines.append(" UserKnownHostsFile /dev/null")
lines.append(" IdentitiesOnly yes")
for filename in key_filenames:
lines.append(f" IdentityFile ~/.ssh/{filename}")
lines.append("")
config_content = "\n".join(lines)
config_path.write_text(config_content)
os.chmod(config_path, 0o644)
# Set ownership to target container user if requested
if uid is not None or gid is not None:
effective_uid = uid if uid is not None else -1
effective_gid = gid if gid is not None else -1
try:
os.chown(ssh_dir, effective_uid, effective_gid)
os.chown(private_key_path, effective_uid, effective_gid)
os.chown(public_key_path, effective_uid, effective_gid)
os.chown(config_path, effective_uid, effective_gid)
logger.debug(
"Set SSH key ownership to uid=%s gid=%s for %s",
effective_uid,
effective_gid,
ssh_dir,
)
except PermissionError as exc:
logger.warning(
"Cannot chown SSH keys to uid=%s gid=%s (running as uid=%s): %s",
effective_uid,
effective_gid,
os.getuid(),
exc,
)
return str(ssh_dir)
except PermissionError:
pass
def cleanup_ssh_key_files(instance_dir: str) -> None:
+259 -70
View File
@@ -1,10 +1,13 @@
"""Terminal session management for tool instances."""
"""High-performance terminal session with asyncio-native I/O.
Replaces blocking select.select() with event-driven asyncio.add_reader()
for sub-frame latency. Includes output batching and flow control.
"""
import asyncio
import logging
import os
import pty
import select
import signal
import struct
import fcntl
@@ -17,18 +20,31 @@ logger = logging.getLogger(__name__)
class TerminalSession:
"""Manages a single terminal session connected to a docker container.
"""Manages a single terminal session with event-driven PTY I/O.
Supports persistent sessions that survive WebSocket disconnections.
Multiple WebSocket connections can attach/detach from the same session.
Uses asyncio.add_reader() instead of polling for near-zero read latency.
Output is batched (2ms window) and sent as binary WebSocket frames.
Flow control prevents memory bloat on fast output.
"""
# Circular buffer size (10KB)
# Circular buffer for replay (10KB)
BUFFER_SIZE = 10 * 1024
# Idle timeout in seconds (30 minutes)
IDLE_TIMEOUT = 30 * 60
# Output batching window in seconds
BATCH_WINDOW_S = 0.002 # 2ms
# Flow control: pause PTY reads when unacknowledged bytes exceed this
FLOW_CONTROL_PAUSE = 64 * 1024
# Flow control: resume PTY reads when unacknowledged bytes drop below this
FLOW_CONTROL_RESUME = 32 * 1024
# Max WebSocket frame size
MAX_FRAME_SIZE = 64 * 1024
# Session number counter per instance_id for auto-naming
_instance_counters: dict[str, int] = {}
@@ -47,7 +63,6 @@ class TerminalSession:
self.process: asyncio.subprocess.Process | None = None
self._closed = False
self._master_fd: int | None = None
self._slave_fd: int | None = None
# Circular buffer for output replay
self._output_buffer: deque[bytes] = deque(maxlen=self.BUFFER_SIZE)
@@ -67,6 +82,20 @@ class TerminalSession:
self.name = name or self._generate_name(str(instance_id))
self.status: str = "active"
# Output batching
self._batch_buffer = bytearray()
self._batch_timer: asyncio.TimerHandle | None = None
self._batch_lock = asyncio.Lock()
# Flow control
self._unacknowledged_bytes = 0
self._paused = False
self._read_handler_set = False
self._flow_control_lock = asyncio.Lock()
# Ack timeout fallback
self._ack_timeout_handle: asyncio.TimerHandle | None = None
@classmethod
def _generate_name(cls, instance_id: str) -> str:
"""Generate an auto-incremented session name for the instance."""
@@ -77,91 +106,225 @@ class TerminalSession:
async def start(self, startup_command: str | None = None) -> None:
"""Start the docker exec process with a shell using a PTY."""
# Create a pseudo-terminal on the host
self._master_fd, self._slave_fd = pty.openpty()
self._master_fd, slave_fd = pty.openpty()
# Set the terminal size initially
self._set_terminal_size(self._cols, self._rows)
logger.debug(
f"Starting terminal session {self.session_id} for container {self.container_id} with initial size {self._cols}x{self._rows}"
"Starting terminal session %s for container %s with initial size %sx%s",
self.session_id,
self.container_id,
self._cols,
self._rows,
)
# Build the shell command
if startup_command:
shell_cmd = f'bash -c "{startup_command}" || true; exec bash -il'
cmd = startup_command or self.startup_command
if cmd:
shell_cmd = f'bash -c "{cmd}" || true; exec bash -il'
logger.debug(
f"Using startup command for session {self.session_id}: {startup_command}"
"Using startup command for session %s: %s",
self.session_id,
cmd,
)
else:
shell_cmd = "bash -il"
# Start docker exec with the slave fd as stdin/stdout/stderr
# Using -it because the slave fd IS a TTY
self.process = await asyncio.create_subprocess_exec(
"docker",
"exec",
"-it",
"-e",
"TERM=xterm",
"TERM=xterm-256color",
self.container_id,
"bash",
"-c",
shell_cmd,
stdin=self._slave_fd,
stdout=self._slave_fd,
stderr=self._slave_fd,
stdin=slave_fd,
stdout=slave_fd,
stderr=slave_fd,
)
# Close slave fd in parent process
os.close(self._slave_fd)
self._slave_fd = None
os.close(slave_fd)
self.last_activity = time.time()
def _set_terminal_size(self, cols: int, rows: int) -> None:
"""Set the terminal size using TIOCSWINSZ."""
if self._master_fd is None:
logger.warning("Cannot resize: master_fd is None (session not started)")
return
# TIOCSWINSZ = 0x5414 on Linux
TIOCSWINSZ = 0x5414
size = struct.pack("HHHH", rows, cols, 0, 0)
try:
fcntl.ioctl(self._master_fd, TIOCSWINSZ, size)
logger.debug(f"Resized PTY to {cols}x{rows} (fd={self._master_fd})")
except (OSError, IOError) as e:
logger.error(f"Failed to resize PTY: {e}")
# Start event-driven reading
self._start_reading()
async def read_output(self) -> bytes:
"""Read output from the PTY master and store in buffer."""
if self._master_fd is None or self._closed:
return b""
def _start_reading(self) -> None:
"""Register PTY master fd with asyncio event loop for event-driven reads."""
if self._read_handler_set or self._master_fd is None or self._closed:
return
try:
# Use select to check if data is available
readable, _, _ = select.select([self._master_fd], [], [], 0.1)
if readable:
data = os.read(self._master_fd, 4096)
if data:
self._add_to_buffer(data)
self.last_activity = time.time()
return data
return b""
except (OSError, IOError, ValueError):
return b""
loop = asyncio.get_event_loop()
loop.add_reader(self._master_fd, self._on_fd_readable)
self._read_handler_set = True
logger.debug("Started event-driven reading for session %s", self.session_id)
except Exception as exc:
logger.error(
"Failed to start reading for session %s: %s", self.session_id, exc
)
def _stop_reading(self) -> None:
"""Unregister PTY master fd from asyncio event loop."""
if not self._read_handler_set or self._master_fd is None:
return
try:
loop = asyncio.get_event_loop()
loop.remove_reader(self._master_fd)
self._read_handler_set = False
except Exception:
pass
def _on_fd_readable(self) -> None:
"""Callback when PTY master fd has data available (called by event loop)."""
if self._master_fd is None or self._closed:
return
try:
data = os.read(self._master_fd, 4096)
except (OSError, IOError) as exc:
logger.debug("PTY read error for session %s: %s", self.session_id, exc)
self._handle_eof()
return
if not data:
# EOF: docker exec process exited
logger.debug("PTY EOF for session %s", self.session_id)
self._handle_eof()
return
self._add_to_buffer(data)
self.last_activity = time.time()
# Queue for batching + flow control
self._queue_output(data)
def _add_to_buffer(self, data: bytes) -> None:
"""Add data to circular buffer, maintaining size limit."""
self._output_buffer.append(data)
self._buffer_size += len(data)
# Trim if exceeds max size
while self._buffer_size > self.BUFFER_SIZE and self._output_buffer:
removed = self._output_buffer.popleft()
self._buffer_size -= len(removed)
def _queue_output(self, data: bytes) -> None:
"""Add output to batch buffer and schedule flush."""
self._batch_buffer.extend(data)
self._unacknowledged_bytes += len(data)
# Check flow control
if self._unacknowledged_bytes > self.FLOW_CONTROL_PAUSE and not self._paused:
self._pause_output()
# Schedule batch flush if not already scheduled
if self._batch_timer is None:
loop = asyncio.get_event_loop()
self._batch_timer = loop.call_later(
self.BATCH_WINDOW_S,
self._flush_batch_sync,
)
def _flush_batch_sync(self) -> None:
"""Synchronous entry point for batch flush (called from event loop)."""
self._batch_timer = None
if not self._batch_buffer or not self._websockets:
self._batch_buffer.clear()
return
payload = bytes(self._batch_buffer)
self._batch_buffer.clear()
# Send to all websockets (asyncio.create_task for async send)
dead_sockets = set()
for ws in list(self._websockets):
try:
asyncio.create_task(self._send_bytes(ws, payload))
except Exception:
dead_sockets.add(ws)
if dead_sockets:
self._websockets -= dead_sockets
async def _send_bytes(self, ws: Any, payload: bytes) -> None:
"""Send bytes to a single websocket, catching errors."""
try:
await ws.send_bytes(payload)
except Exception:
self._websockets.discard(ws)
def acknowledge_data(self, char_count: int) -> None:
"""Client acknowledges processing char_count bytes.
Called from the WebSocket handler when the client sends an 'ack' message.
"""
self._unacknowledged_bytes = max(0, self._unacknowledged_bytes - char_count)
if self._paused and self._unacknowledged_bytes < self.FLOW_CONTROL_RESUME:
self._resume_output()
# Reset ack timeout
if self._ack_timeout_handle:
self._ack_timeout_handle.cancel()
loop = asyncio.get_event_loop()
self._ack_timeout_handle = loop.call_later(5.0, self._ack_timeout_fallback)
def _ack_timeout_fallback(self) -> None:
"""If no ack received for 5s, assume client is dead and resume."""
logger.warning(
"Flow control ack timeout for session %s, resuming output",
self.session_id,
)
self._unacknowledged_bytes = 0
if self._paused:
self._resume_output()
def _pause_output(self) -> None:
"""Pause reading from PTY due to flow control."""
self._paused = True
self._stop_reading()
logger.debug(
"Paused output for session %s (%d unacked)",
self.session_id,
self._unacknowledged_bytes,
)
def _resume_output(self) -> None:
"""Resume reading from PTY."""
self._paused = False
self._start_reading()
logger.debug("Resumed output for session %s", self.session_id)
def get_buffer(self) -> bytes:
"""Get buffered output for replay."""
return b"".join(self._output_buffer)
def _handle_eof(self) -> None:
"""Handle PTY EOF: process died, close websockets to force reconnect."""
self._stop_reading()
# Mark process as done so is_alive() returns False
if self.process is not None and self.process.returncode is None:
# Force returncode to a non-None value since the process is dead
# but asyncio.subprocess may not have set it yet
try:
self.process._transport.close() # type: ignore[attr-defined]
except Exception:
pass
# Close all websockets to force frontend reconnection
dead_sockets = set(self._websockets)
self._websockets.clear()
for ws in dead_sockets:
try:
asyncio.create_task(
ws.close(code=4001, reason="Session process exited")
)
except Exception:
pass
logger.info("Session %s EOF handled, websockets closed", self.session_id)
async def write_input(self, data: bytes) -> None:
"""Write input to the PTY master."""
if self._master_fd is None or self._closed:
@@ -169,8 +332,22 @@ class TerminalSession:
try:
os.write(self._master_fd, data)
self.last_activity = time.time()
except (OSError, IOError):
pass
except (OSError, IOError) as exc:
logger.debug("PTY write error for session %s: %s", self.session_id, exc)
self._handle_eof()
def _set_terminal_size(self, cols: int, rows: int) -> None:
"""Set the terminal size using TIOCSWINSZ."""
if self._master_fd is None:
logger.warning("Cannot resize: master_fd is None (session not started)")
return
TIOCSWINSZ = 0x5414
size = struct.pack("HHHH", rows, cols, 0, 0)
try:
fcntl.ioctl(self._master_fd, TIOCSWINSZ, size)
logger.debug("Resized PTY to %sx%s (fd=%s)", cols, rows, self._master_fd)
except (OSError, IOError) as e:
logger.error("Failed to resize PTY: %s", e)
async def resize(self, cols: int, rows: int) -> None:
"""Resize the terminal."""
@@ -178,32 +355,24 @@ class TerminalSession:
logger.warning("Cannot resize: session is closed")
return
# Only resize if dimensions actually changed
if cols == self._cols and rows == self._rows:
return
self._cols = cols
self._rows = rows
logger.debug(f"resize() called for session {self.session_id}: {cols}x{rows}")
logger.debug(
"resize() called for session %s: %sx%s", self.session_id, cols, rows
)
self._set_terminal_size(cols, rows)
# Docker exec -it creates its own PTY inside the container,
# so host PTY resize doesn't propagate to the container shell.
# Send SIGWINCH to the docker exec process on the host.
# Docker exec forwards signals to the container process, which should
# cause the container's shell to re-read its terminal size.
# Send SIGWINCH to docker exec process
if self.process and self.process.pid:
try:
os.kill(self.process.pid, signal.SIGWINCH)
logger.debug(
f"Sent SIGWINCH to docker exec process {self.process.pid} for session {self.session_id}"
)
except ProcessLookupError:
logger.warning(
f"docker exec process {self.process.pid} not found for session {self.session_id}"
)
logger.warning("docker exec process %s not found", self.process.pid)
except Exception as e:
logger.warning(f"Failed to send SIGWINCH: {e}")
logger.warning("Failed to send SIGWINCH: %s", e)
async def reset(self) -> None:
"""Reset the session by killing the process and clearing state."""
@@ -213,9 +382,13 @@ class TerminalSession:
self._output_buffer.clear()
self._buffer_size = 0
self._websockets.clear()
self._batch_buffer.clear()
self._batch_timer = None
self._unacknowledged_bytes = 0
self._paused = False
self._read_handler_set = False
self.process = None
self._master_fd = None
self._slave_fd = None
self.status = "active"
async def close(self) -> None:
@@ -225,11 +398,21 @@ class TerminalSession:
self._closed = True
self.status = "closed"
self._stop_reading()
if self._batch_timer:
self._batch_timer.cancel()
self._batch_timer = None
if self._ack_timeout_handle:
self._ack_timeout_handle.cancel()
self._ack_timeout_handle = None
if self._master_fd is not None:
try:
os.close(self._master_fd)
except OSError:
pass # noqa: S110
pass
self._master_fd = None
if self.process is not None:
@@ -265,14 +448,20 @@ class TerminalSession:
return len(self._websockets) > 0
async def send_to_all(self, data: bytes) -> None:
"""Send data to all attached WebSockets."""
"""Send data to all attached WebSockets (used for control messages)."""
dead_sockets = set()
for ws in self._websockets:
try:
await ws.send_bytes(data)
except Exception:
dead_sockets.add(ws)
# Clean up dead sockets
for ws in dead_sockets:
self._websockets.discard(ws)
async def read_output(self) -> bytes:
"""Legacy method: read output synchronously.
With event-driven I/O, output is automatically sent to websockets.
This method returns any buffered data for callers that poll.
"""
return b""
@@ -88,6 +88,11 @@ class WorkspaceManager:
if not repo.remote_url:
raise ValueError("Repository has no remote URL")
# Remove stale directory from previous failed/aborted clone
if os.path.exists(path):
logger.warning("Removing stale workspace directory: %s", path)
shutil.rmtree(path, ignore_errors=True)
# Load SSH key if repo has one
ssh_key = None
if getattr(repo, "ssh_key_id", None) and session is not None:
@@ -145,10 +145,12 @@ class TestCreateInstanceDockerfileLegacy:
data = MagicMock()
data.tool_type_id = str(fake_tool_type_id)
data.display_name = None
data.workspace_id = None
data.clone_mode = "mount"
data.branch = None
data.new_branch = None
data.config_profile_id = None
data.ssh_key_ids = []
result = await create_instance(
project_id=fake_project_id,
@@ -225,10 +227,12 @@ class TestCreateInstanceDockerfileLegacy:
data = MagicMock()
data.tool_type_id = str(fake_tool_type_id)
data.display_name = None
data.workspace_id = None
data.clone_mode = "mount"
data.branch = None
data.new_branch = None
data.config_profile_id = None
data.ssh_key_ids = []
with pytest.raises(HTTPException) as exc_info:
await create_instance(
@@ -305,10 +309,12 @@ class TestCreateInstanceComposeLegacy:
data = MagicMock()
data.tool_type_id = str(fake_tool_type_id)
data.display_name = None
data.workspace_id = None
data.clone_mode = "mount"
data.branch = None
data.new_branch = None
data.config_profile_id = None
data.ssh_key_ids = []
result = await create_instance(
project_id=fake_project_id,
@@ -389,10 +395,12 @@ class TestCreateInstanceManifestNotCalledForLegacy:
data = MagicMock()
data.tool_type_id = str(fake_tool_type_id)
data.display_name = None
data.workspace_id = None
data.clone_mode = "mount"
data.branch = None
data.new_branch = None
data.config_profile_id = None
data.ssh_key_ids = []
await create_instance(
project_id=fake_project_id,
@@ -411,8 +419,10 @@ class TestStartInstanceLegacyFallback:
@patch("src.api.tool_instances.wait_for_container_running")
@patch("src.api.tool_instances.execute_compose_command")
@patch("src.api.tool_instances.get_container_id")
@patch("src.api.tool_instances.get_container_name")
@patch("src.api.tool_instances.connect_container_to_network")
@patch("src.api.tool_instances._ensure_backend_network_in_compose")
@patch("src.api.tool_instances._ensure_container_name_in_compose")
@patch("src.api.tool_instances._ensure_web_bind_address")
@patch("src.api.tool_instances._sanitize_compose_file")
@patch("src.api.tool_instances._prepare_manifest_instance")
@patch("src.api.tool_instances._get_user")
@@ -423,8 +433,10 @@ class TestStartInstanceLegacyFallback:
mock_get_user,
mock_prepare_manifest,
mock_sanitize,
mock_ensure_web_bind,
mock_ensure_container_name,
mock_backend_network,
mock_connect_network,
mock_get_container_name,
mock_get_container_id,
mock_execute_compose,
mock_wait_container,
@@ -440,7 +452,6 @@ class TestStartInstanceLegacyFallback:
mock_get_project.return_value = AsyncMock()
mock_execute_compose.return_value = (0, "started", "")
mock_get_container_id.return_value = "abc123"
mock_get_container_name.return_value = "test-container"
mock_connect_network.return_value = True
mock_wait_container.return_value = {
"success": True,
@@ -509,8 +520,10 @@ class TestStartInstanceLegacyFallback:
@patch("src.api.tool_instances.wait_for_container_running")
@patch("src.api.tool_instances.execute_compose_command")
@patch("src.api.tool_instances.get_container_id")
@patch("src.api.tool_instances.get_container_name")
@patch("src.api.tool_instances.connect_container_to_network")
@patch("src.api.tool_instances._ensure_backend_network_in_compose")
@patch("src.api.tool_instances._ensure_container_name_in_compose")
@patch("src.api.tool_instances._ensure_web_bind_address")
@patch("src.api.tool_instances._sanitize_compose_file")
@patch("src.api.tool_instances._prepare_manifest_instance")
@patch("src.api.tool_instances._get_user")
@@ -521,8 +534,10 @@ class TestStartInstanceLegacyFallback:
mock_get_user,
mock_prepare_manifest,
mock_sanitize,
mock_ensure_web_bind,
mock_ensure_container_name,
mock_backend_network,
mock_connect_network,
mock_get_container_name,
mock_get_container_id,
mock_execute_compose,
mock_wait_container,
@@ -538,7 +553,6 @@ class TestStartInstanceLegacyFallback:
mock_get_project.return_value = AsyncMock()
mock_execute_compose.return_value = (0, "started", "")
mock_get_container_id.return_value = "abc123"
mock_get_container_name.return_value = "test-container"
mock_connect_network.return_value = True
mock_wait_container.return_value = {
"success": True,
@@ -606,8 +620,10 @@ class TestStartInstanceLegacyFallback:
@patch("src.api.tool_instances.wait_for_container_running")
@patch("src.api.tool_instances.execute_compose_command")
@patch("src.api.tool_instances.get_container_id")
@patch("src.api.tool_instances.get_container_name")
@patch("src.api.tool_instances.connect_container_to_network")
@patch("src.api.tool_instances._ensure_backend_network_in_compose")
@patch("src.api.tool_instances._ensure_container_name_in_compose")
@patch("src.api.tool_instances._ensure_web_bind_address")
@patch("src.api.tool_instances._sanitize_compose_file")
@patch("src.api.tool_instances._prepare_manifest_instance")
@patch("src.api.tool_instances._get_user")
@@ -618,8 +634,10 @@ class TestStartInstanceLegacyFallback:
mock_get_user,
mock_prepare_manifest,
mock_sanitize,
mock_ensure_web_bind,
mock_ensure_container_name,
mock_backend_network,
mock_connect_network,
mock_get_container_name,
mock_get_container_id,
mock_execute_compose,
mock_wait_container,
@@ -635,7 +653,6 @@ class TestStartInstanceLegacyFallback:
mock_get_project.return_value = AsyncMock()
mock_execute_compose.return_value = (0, "started", "")
mock_get_container_id.return_value = "abc123"
mock_get_container_name.return_value = "test-container"
mock_connect_network.return_value = True
mock_wait_container.return_value = {
"success": True,
@@ -705,12 +722,15 @@ class TestStartInstanceSshPermissions:
"""SSH key mounts trigger permission fixes after container starts."""
@patch("src.api.tool_instances.write_compose_file")
@patch("src.api.tool_instances.prepare_ssh_key_files")
@patch("src.api.tool_instances.apply_ssh_permissions")
@patch("src.api.tool_instances.wait_for_container_running")
@patch("src.api.tool_instances.execute_compose_command")
@patch("src.api.tool_instances.get_container_id")
@patch("src.api.tool_instances.get_container_name")
@patch("src.api.tool_instances.connect_container_to_network")
@patch("src.api.tool_instances._ensure_backend_network_in_compose")
@patch("src.api.tool_instances._ensure_container_name_in_compose")
@patch("src.api.tool_instances._ensure_web_bind_address")
@patch("src.api.tool_instances._sanitize_compose_file")
@patch("src.api.tool_instances._get_user")
@patch("src.api.tool_instances._get_owned_project")
@@ -719,12 +739,15 @@ class TestStartInstanceSshPermissions:
mock_get_project,
mock_get_user,
mock_sanitize,
mock_ensure_web_bind,
mock_ensure_container_name,
mock_backend_network,
mock_connect_network,
mock_get_container_name,
mock_get_container_id,
mock_execute_compose,
mock_wait_container,
mock_apply_ssh,
mock_prepare_ssh,
mock_write_compose,
mock_session,
fake_user_id,
@@ -743,7 +766,6 @@ class TestStartInstanceSshPermissions:
mock_get_project.return_value = AsyncMock()
mock_execute_compose.return_value = (0, "started", "")
mock_get_container_id.return_value = "abc123"
mock_get_container_name.return_value = "test-container"
mock_connect_network.return_value = True
mock_wait_container.return_value = {
"success": True,
@@ -815,33 +837,37 @@ class TestStartInstanceSshPermissions:
mock_session.get.side_effect = _get
with patch("os.path.exists", return_value=True):
with patch(
"src.api.tool_instances._prepare_manifest_instance"
) as mock_prepare:
mock_prepare.return_value = (
"headquarter/test:latest",
"services:\n app:\n image: test",
{"name": "test-manifest", "user": {"name": "user"}},
"/home/user",
)
result = await start_instance(
project_id=fake_project_id,
repo_id=fake_repo_id,
instance_id=fake_instance_id,
data=None,
user_id=fake_user_id,
session=mock_session,
)
with patch("os.makedirs"):
with patch(
"src.api.tool_instances._prepare_manifest_instance"
) as mock_prepare:
mock_prepare.return_value = (
"headquarter/test:latest",
"services:\n app:\n image: test",
{"name": "test-manifest", "user": {"name": "user"}},
"/home/user",
)
result = await start_instance(
project_id=fake_project_id,
repo_id=fake_repo_id,
instance_id=fake_instance_id,
data=None,
user_id=fake_user_id,
session=mock_session,
)
assert result["status"] == "running"
mock_apply_ssh.assert_called_once_with("abc123", "/home/user/.ssh", "user")
@patch("src.api.tool_instances.prepare_ssh_key_files")
@patch("src.api.tool_instances.apply_ssh_permissions")
@patch("src.api.tool_instances.wait_for_container_running")
@patch("src.api.tool_instances.execute_compose_command")
@patch("src.api.tool_instances.get_container_id")
@patch("src.api.tool_instances.get_container_name")
@patch("src.api.tool_instances.connect_container_to_network")
@patch("src.api.tool_instances._ensure_backend_network_in_compose")
@patch("src.api.tool_instances._ensure_container_name_in_compose")
@patch("src.api.tool_instances._ensure_web_bind_address")
@patch("src.api.tool_instances._sanitize_compose_file")
@patch("src.api.tool_instances._get_user")
@patch("src.api.tool_instances._get_owned_project")
@@ -850,12 +876,15 @@ class TestStartInstanceSshPermissions:
mock_get_project,
mock_get_user,
mock_sanitize,
mock_ensure_web_bind,
mock_ensure_container_name,
mock_backend_network,
mock_connect_network,
mock_get_container_name,
mock_get_container_id,
mock_execute_compose,
mock_wait_container,
mock_apply_ssh,
mock_prepare_ssh,
mock_session,
fake_user_id,
fake_project_id,
@@ -870,7 +899,6 @@ class TestStartInstanceSshPermissions:
mock_get_project.return_value = AsyncMock()
mock_execute_compose.return_value = (0, "started", "")
mock_get_container_id.return_value = "abc123"
mock_get_container_name.return_value = "test-container"
mock_connect_network.return_value = True
mock_wait_container.return_value = {
"success": True,
@@ -933,14 +961,16 @@ class TestStartInstanceSshPermissions:
mock_session.get.side_effect = _get
with patch("os.path.exists", return_value=True):
result = await start_instance(
project_id=fake_project_id,
repo_id=fake_repo_id,
instance_id=fake_instance_id,
data=None,
user_id=fake_user_id,
session=mock_session,
)
with patch("os.makedirs"):
with patch("src.api.tool_instances._modify_compose_file"):
result = await start_instance(
project_id=fake_project_id,
repo_id=fake_repo_id,
instance_id=fake_instance_id,
data=None,
user_id=fake_user_id,
session=mock_session,
)
assert result["status"] == "running"
mock_apply_ssh.assert_called_once_with("abc123", "/root/.ssh", "root")
@@ -952,8 +982,10 @@ class TestStartInstanceManifestBranch:
@patch("src.api.tool_instances.wait_for_container_running")
@patch("src.api.tool_instances.execute_compose_command")
@patch("src.api.tool_instances.get_container_id")
@patch("src.api.tool_instances.get_container_name")
@patch("src.api.tool_instances.connect_container_to_network")
@patch("src.api.tool_instances._ensure_backend_network_in_compose")
@patch("src.api.tool_instances._ensure_container_name_in_compose")
@patch("src.api.tool_instances._ensure_web_bind_address")
@patch("src.api.tool_instances._sanitize_compose_file")
@patch("src.api.tool_instances._prepare_manifest_instance")
@patch("src.api.tool_instances.write_compose_file")
@@ -966,8 +998,10 @@ class TestStartInstanceManifestBranch:
mock_write_compose,
mock_prepare_manifest,
mock_sanitize,
mock_ensure_web_bind,
mock_ensure_container_name,
mock_backend_network,
mock_connect_network,
mock_get_container_name,
mock_get_container_id,
mock_execute_compose,
mock_wait_container,
@@ -987,7 +1021,6 @@ class TestStartInstanceManifestBranch:
mock_get_project.return_value = AsyncMock()
mock_execute_compose.return_value = (0, "started", "")
mock_get_container_id.return_value = "abc123"
mock_get_container_name.return_value = "test-container"
mock_connect_network.return_value = True
mock_wait_container.return_value = {
"success": True,
+12 -12
View File
@@ -16,11 +16,11 @@
"react-dom": "^18.2.0",
"react-router-dom": "^6.20.0",
"react-simple-code-editor": "^0.14.1",
"sonner": "^1.7.4",
"tailwindcss": "^3.3.0",
"xterm": "^5.3.0",
"xterm-addon-fit": "^0.8.0",
"xterm-addon-web-links": "^0.9.0"
"xterm-addon-web-links": "^0.9.0",
"xterm-addon-webgl": "^0.16.0"
},
"devDependencies": {
"@testing-library/jest-dom": "^6.9.1",
@@ -5469,16 +5469,6 @@
"node": ">=8"
}
},
"node_modules/sonner": {
"version": "1.7.4",
"resolved": "https://registry.npmjs.org/sonner/-/sonner-1.7.4.tgz",
"integrity": "sha512-DIS8z4PfJRbIyfVFDVnK9rO3eYDtse4Omcm6bt0oEr5/jtLgysmjuBl1frJ9E/EQZrFmKx2A8m/s5s9CRXIzhw==",
"license": "MIT",
"peerDependencies": {
"react": "^18.0.0 || ^19.0.0 || ^19.0.0-rc",
"react-dom": "^18.0.0 || ^19.0.0 || ^19.0.0-rc"
}
},
"node_modules/source-map-js": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
@@ -6324,6 +6314,16 @@
"xterm": "^5.0.0"
}
},
"node_modules/xterm-addon-webgl": {
"version": "0.16.0",
"resolved": "https://registry.npmjs.org/xterm-addon-webgl/-/xterm-addon-webgl-0.16.0.tgz",
"integrity": "sha512-E8cq1AiqNOv0M/FghPT+zPAEnvIQRDbAbkb04rRYSxUym69elPWVJ4sv22FCLBqM/3LcrmBLl/pELnBebVFKgA==",
"deprecated": "This package is now deprecated. Move to @xterm/addon-webgl instead.",
"license": "MIT",
"peerDependencies": {
"xterm": "^5.0.0"
}
},
"node_modules/yallist": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-3.1.1.tgz",
+2 -1
View File
@@ -22,7 +22,8 @@
"tailwindcss": "^3.3.0",
"xterm": "^5.3.0",
"xterm-addon-fit": "^0.8.0",
"xterm-addon-web-links": "^0.9.0"
"xterm-addon-web-links": "^0.9.0",
"xterm-addon-webgl": "^0.16.0"
},
"devDependencies": {
"@testing-library/jest-dom": "^6.9.1",
+15 -3
View File
@@ -35,11 +35,23 @@ const NAV_ITEMS: {
const SessionItem = ({ session }: { session: Session }) => {
const isRunning = session.status === "running";
// Determine the link target:
// - Web tools open their tunnel URL
// - Terminal tools open the terminal page
// - Everything else falls back to the project page
const hasTerminal = session.tool_type_interfaces.includes("terminal");
const hasWeb = session.tool_type_interfaces.includes("web");
const href = session.url && hasWeb
? session.url
: hasTerminal
? `/instances/${session.id}/terminal`
: `/projects/${session.project_id}`;
return (
<a
href={session.url ?? `/projects/${session.project_id}`}
target={session.url ? "_blank" : undefined}
rel={session.url ? "noopener noreferrer" : undefined}
href={href}
target="_blank"
rel="noopener noreferrer"
className="nav-item session-item"
title={`${session.display_name} (${session.status})`}
>
+9 -4
View File
@@ -58,6 +58,11 @@ export function SessionCard({
const isTerminalOnly =
session.tool_type_interfaces?.includes("terminal") &&
!session.tool_type_interfaces?.includes("web");
const openHref = session.url
? session.url
: isTerminalOnly
? `/instances/${session.id}/terminal`
: undefined;
const hasTunnelError =
!isTerminalOnly && tunnelHealth?.tunnel_status === "unreachable";
const hasAppError =
@@ -150,9 +155,9 @@ export function SessionCard({
<div className="session-card-actions mobile">
{isActive && (
<>
{session.url ? (
{openHref ? (
<a
href={session.url}
href={openHref}
target="_blank"
rel="noopener noreferrer"
className="secondary-button mobile-primary"
@@ -207,9 +212,9 @@ export function SessionCard({
<div className="session-card-actions">
{isActive && (
<>
{session.url ? (
{openHref ? (
<a
href={session.url}
href={openHref}
target="_blank"
rel="noopener noreferrer"
className="secondary-button small"
+68 -6
View File
@@ -8,6 +8,7 @@ import React, {
import { Terminal } from "xterm";
import { FitAddon } from "xterm-addon-fit";
import { WebLinksAddon } from "xterm-addon-web-links";
import { WebglAddon } from "xterm-addon-webgl";
import "xterm/css/xterm.css";
import {
@@ -107,6 +108,7 @@ export const TerminalComponent = React.forwardRef<TerminalRef, TerminalProps>(
// WebSocket connection established
const ws = new WebSocket(wsUrl);
ws.binaryType = "arraybuffer";
wsRef.current = ws;
ws.onopen = () => {
@@ -137,14 +139,35 @@ export const TerminalComponent = React.forwardRef<TerminalRef, TerminalProps>(
}, 30000);
};
// Flow control: accumulate processed bytes and send ack
let ackAccumulator = 0;
const ACK_THRESHOLD = 4096;
let ackTimeout: ReturnType<typeof setTimeout> | null = null;
const flushAck = () => {
if (ackAccumulator > 0 && ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify({ type: "ack", chars: ackAccumulator }));
ackAccumulator = 0;
}
};
ws.onmessage = (event) => {
if (!termRef.current) return;
if (event.data instanceof Blob) {
event.data.arrayBuffer().then((buffer) => {
const data = new Uint8Array(buffer);
termRef.current?.write(data);
});
if (event.data instanceof ArrayBuffer) {
const data = new Uint8Array(event.data);
termRef.current.write(data);
// Flow control: accumulate processed bytes
ackAccumulator += data.length;
if (ackAccumulator >= ACK_THRESHOLD) {
flushAck();
} else if (!ackTimeout) {
ackTimeout = setTimeout(() => {
ackTimeout = null;
flushAck();
}, 100);
}
} else if (typeof event.data === "string") {
try {
const msg = JSON.parse(event.data);
@@ -251,6 +274,11 @@ export const TerminalComponent = React.forwardRef<TerminalRef, TerminalProps>(
lineHeight: 1.2,
letterSpacing: 0,
allowTransparency: false,
scrollback: 10000,
ignoreBracketedPasteMode: false,
fastScrollSensitivity: 5,
scrollSensitivity: 1,
smoothScrollDuration: 0,
theme: {
background: "#1e1e1e",
foreground: "#d4d4d4",
@@ -282,6 +310,26 @@ export const TerminalComponent = React.forwardRef<TerminalRef, TerminalProps>(
term.loadAddon(fitAddon);
term.loadAddon(new WebLinksAddon());
// Load WebGL renderer for GPU acceleration, fall back to DOM
let webglAddon: WebglAddon | null = null;
try {
webglAddon = new WebglAddon();
term.loadAddon(webglAddon);
webglAddon.onContextLoss(() => {
console.warn("WebGL context lost, falling back to DOM renderer");
try {
webglAddon?.dispose();
} catch {
// ignore
}
webglAddon = null;
// Trigger a refit since cell dimensions may differ
requestAnimationFrame(() => fitTerminal());
});
} catch (e) {
console.warn("WebGL renderer failed to load, using DOM renderer", e);
}
const container = terminalRef.current;
// Define fitTerminal before connectWebSocket so it's available in onmessage
@@ -537,7 +585,21 @@ export const TerminalComponent = React.forwardRef<TerminalRef, TerminalProps>(
window.clearInterval(heartbeatCheckRef.current);
heartbeatCheckRef.current = null;
}
term.dispose();
// Dispose WebGL addon BEFORE the terminal to avoid race with
// RenderService.setRenderer accessing a disposed renderer
if (webglAddon) {
try {
webglAddon.dispose();
} catch {
// Ignore disposal errors from partially torn-down terminal
}
webglAddon = null;
}
try {
term.dispose();
} catch {
// Ignore disposal errors from partially torn-down terminal
}
};
}, [instanceId, connectWebSocket]);
+55 -16
View File
@@ -31,6 +31,7 @@ export function ToolStarter({
const [sshKeys, setSshKeys] = useState<SSHKey[]>([]);
const [sshKeysLoading, setSshKeysLoading] = useState(true);
const [selectedSshKeyIds, setSelectedSshKeyIds] = useState<string[]>([]);
const [starting, setStarting] = useState(false);
const [error, setError] = useState<string | null>(null);
@@ -89,14 +90,18 @@ export function ToolStarter({
try {
const data = await listSSHKeys();
setSshKeys(data);
} catch {
// ignore
// Auto-select the repository's SSH key if available
if (workspace.repo_ssh_key_id) {
setSelectedSshKeyIds([workspace.repo_ssh_key_id]);
}
} catch (err) {
console.error("Failed to load SSH keys:", err);
} finally {
setSshKeysLoading(false);
}
};
void load();
}, []);
}, [workspace.repo_ssh_key_id]);
const repoHasSshKey = !!workspace.repo_ssh_key_id;
const repoSshKey = sshKeys.find((k) => k.id === workspace.repo_ssh_key_id);
@@ -119,7 +124,7 @@ export function ToolStarter({
undefined,
undefined,
selectedProfileId || undefined,
[],
selectedSshKeyIds.length > 0 ? selectedSshKeyIds : undefined,
workspace.id,
);
await startInstance(
@@ -127,6 +132,7 @@ export function ToolStarter({
workspace.repo_id,
instance.id,
selectedProfileId || undefined,
selectedSshKeyIds.length > 0 ? selectedSshKeyIds : undefined,
);
onStarted(instance);
} catch (err) {
@@ -208,20 +214,53 @@ export function ToolStarter({
</div>
)}
{/* SSH Key Status */}
<div className="form-group ssh-key-status">
<label>SSH Key</label>
{/* SSH Key Selection */}
<div className="form-group ssh-key-selection">
<label>SSH Keys</label>
{sshKeysLoading ? (
<span className="muted">Checking...</span>
) : repoHasSshKey ? (
<span className="success-text">
<Icon name="success" size="sm" />{" "}
{repoSshKey?.name || "SSH key assigned"}
</span>
<span className="muted">Loading SSH keys...</span>
) : sshKeys.length === 0 ? (
<span className="muted">No SSH keys configured.</span>
) : (
<span className="warning-text">
<Icon name="warning" size="sm" /> No SSH key assigned to repository
</span>
<div style={{ display: "flex", flexWrap: "wrap", gap: "0.5rem" }}>
{sshKeys.map((key) => (
<label
key={key.id}
className="checkbox-label"
style={{
display: "flex",
alignItems: "center",
gap: "0.25rem",
padding: "0.375rem 0.75rem",
background: "var(--panel)",
borderRadius: "0.375rem",
border: "1px solid var(--border)",
cursor: "pointer",
}}
>
<input
type="checkbox"
checked={selectedSshKeyIds.includes(key.id)}
onChange={(e) => {
if (e.target.checked) {
setSelectedSshKeyIds((prev) => [...prev, key.id]);
} else {
setSelectedSshKeyIds((prev) =>
prev.filter((id) => id !== key.id),
);
}
}}
disabled={starting}
/>
{key.name}
</label>
))}
</div>
)}
{!sshKeysLoading && repoHasSshKey && repoSshKey && (
<div className="hint" style={{ marginTop: "0.5rem" }}>
Repository key <strong>{repoSshKey.name}</strong> is pre-selected.
</div>
)}
</div>
+2 -2
View File
@@ -40,10 +40,10 @@ export function useInstanceActions(
return;
}
if (session.tool_type_interfaces?.includes("terminal")) {
window.location.href = `/instances/${session.id}/terminal`;
window.open(`/instances/${session.id}/terminal`, "_blank", "noopener,noreferrer");
return;
}
window.location.href = `/projects/${session.project_id}`;
window.open(`/projects/${session.project_id}`, "_blank", "noopener,noreferrer");
}, []);
const handleStart = useCallback(
+6 -1
View File
@@ -13,7 +13,11 @@ services:
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-headquarter} -d ${POSTGRES_DB:-headquarter}"]
test:
[
"CMD-SHELL",
"pg_isready -U ${POSTGRES_USER:-headquarter} -d ${POSTGRES_DB:-headquarter}",
]
interval: 10s
timeout: 5s
retries: 5
@@ -93,6 +97,7 @@ services:
AUTHENTIK_TOKEN_URL: ${AUTHENTIK_TOKEN_URL:-}
volumes:
- /data/repos:/data/repos
- /data/working-copies:/data/working-copies
- /data/instances:/data/instances
- avatar_uploads:/app/uploads
- /var/run/docker.sock:/var/run/docker.sock
+1 -1
View File
@@ -62,8 +62,8 @@ services:
INSTANCE_BASE_PATH: /data/instances
volumes:
- /data/repos:/data/repos
- /data/instances:/data/instances
- /data/working-copies:/data/working-copies
- /data/instances:/data/instances
ports:
- "8000:8000"
depends_on:
+227
View File
@@ -0,0 +1,227 @@
# Design: High-Performance Web Terminal
## Component Diagram
```
┌─────────────────────────────────────────────────────────────┐
│ Browser │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Terminal │ │ WebSocket │ │ xterm.js │ │
│ │ Component │──│ Client │──│ + WebGL addon │ │
│ │ │ │ (binary) │ │ + DOM fallback │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
│ │ │ │
│ │ Flow control ack │ │
│ └──────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│ WebSocket
│ (binary frames)
┌─────────────────────────────────────────────────────────────┐
│ API Container │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Terminal │ │ WebSocket │ │ TerminalSession │ │
│ │ Manager │──│ Endpoint │──│ (new) │ │
│ │ (lifecycle) │ │ (router) │ │ - asyncio fd reader │ │
│ └─────────────┘ └─────────────┘ │ - output batcher │ │
│ │ - flow control │ │
│ └─────────────────────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ docker exec -it │ │
│ │ (subprocess) │ │
│ └────────┬────────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ Tool Container │ │
│ │ (bash shell) │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
## File Changes
### New Files
- `apps/api/src/services/terminal_session_v2.py` — New TerminalSession implementation
### Modified Files
- `apps/api/src/services/terminal_session.py` — Delete (or keep as legacy, user said no legacy needed)
- `apps/api/src/services/terminal_manager.py` — Update to use new TerminalSession
- `apps/api/src/api/terminal.py` — Update WebSocket handler for binary frames + flow control
- `apps/web/src/components/terminal.tsx` — Binary mode, WebGL, flow control ack
- `apps/web/package.json` — Add `xterm-addon-webgl`
## TerminalSession Implementation
```python
class TerminalSession:
"""High-performance terminal session with asyncio-native I/O."""
BUFFER_SIZE = 10 * 1024
IDLE_TIMEOUT = 30 * 60
BATCH_WINDOW_MS = 2
FLOW_CONTROL_THRESHOLD = 64 * 1024
FLOW_CONTROL_RESUME = 32 * 1024
def __init__(self, session_id, instance_id, container_id, ...):
self._master_fd: int | None = None
self._process: asyncio.subprocess.Process | None = None
self._output_buffer: deque[bytes] = deque(maxlen=self.BUFFER_SIZE)
self._websockets: set[WebSocket] = set()
self._batch_buffer = bytearray()
self._batch_timer: asyncio.TimerHandle | None = None
self._unacknowledged_bytes = 0
self._paused = False
self._read_handler_set = False
async def start(self):
self._master_fd, slave_fd = pty.openpty()
self._set_terminal_size(80, 24)
self._process = await asyncio.create_subprocess_exec(
"docker", "exec", "-it", "-e", "TERM=xterm-256color",
self.container_id, "bash", "-il",
stdin=slave_fd, stdout=slave_fd, stderr=slave_fd,
)
os.close(slave_fd)
def _start_reading(self):
"""Register fd with asyncio event loop for event-driven reading."""
if self._read_handler_set or self._master_fd is None:
return
loop = asyncio.get_event_loop()
loop.add_reader(self._master_fd, self._on_fd_readable)
self._read_handler_set = True
def _on_fd_readable(self):
"""Callback when PTY fd has data available."""
if self._master_fd is None or self._paused:
return
try:
data = os.read(self._master_fd, 4096)
if data:
self._add_to_buffer(data)
self._queue_output(data)
self.last_activity = time.time()
except (OSError, IOError):
pass
def _queue_output(self, data: bytes):
"""Add to batch buffer, schedule flush."""
self._batch_buffer.extend(data)
self._unacknowledged_bytes += len(data)
if self._unacknowledged_bytes > self.FLOW_CONTROL_THRESHOLD:
self._pause_output()
if self._batch_timer is None:
loop = asyncio.get_event_loop()
self._batch_timer = loop.call_later(
self.BATCH_WINDOW_MS / 1000, self._flush_batch
)
def _flush_batch(self):
"""Flush batched output to all WebSockets."""
self._batch_timer = None
if not self._batch_buffer:
return
payload = bytes(self._batch_buffer)
self._batch_buffer.clear()
dead = set()
for ws in self._websockets:
try:
asyncio.create_task(ws.send_bytes(payload))
except Exception:
dead.add(ws)
self._websockets -= dead
def acknowledge_data(self, char_count: int):
"""Client acknowledges processed bytes."""
self._unacknowledged_bytes = max(0, self._unacknowledged_bytes - char_count)
if self._paused and self._unacknowledged_bytes < self.FLOW_CONTROL_RESUME:
self._resume_output()
def _pause_output(self):
"""Pause reading from PTY."""
if self._read_handler_set and self._master_fd is not None:
loop = asyncio.get_event_loop()
loop.remove_reader(self._master_fd)
self._read_handler_set = False
self._paused = True
def _resume_output(self):
"""Resume reading from PTY."""
self._paused = False
self._start_reading()
```
## Frontend Changes
### WebSocket Binary Mode
```typescript
const ws = new WebSocket(wsUrl);
ws.binaryType = "arraybuffer"; // Receive ArrayBuffer directly
ws.onmessage = (event) => {
if (event.data instanceof ArrayBuffer) {
const data = new Uint8Array(event.data);
termRef.current?.write(data);
// Flow control: acknowledge processed bytes
ackAccumulator += data.length;
if (ackAccumulator >= 4096) {
ws.send(JSON.stringify({ type: "ack", chars: ackAccumulator }));
ackAccumulator = 0;
}
}
};
```
### WebGL Renderer
```typescript
import { WebglAddon } from "xterm-addon-webgl";
const webglAddon = new WebglAddon();
try {
term.loadAddon(webglAddon);
} catch (e) {
console.warn("WebGL failed, using DOM renderer", e);
}
```
## WebSocket Protocol
### Message Types
**Client → Server:**
- `{"type": "input", "data": "base64_encoded"}` — Keystrokes
- `{"type": "resize", "cols": 80, "rows": 24}` — Resize
- `{"type": "ack", "chars": 4096}` — Flow control acknowledgment
- `{"type": "reset"}` — Reset session
**Server → Client:**
- Binary frame: raw terminal output bytes
- `{"type": "status", "status": "connected"}` — Connection status
- `{"type": "ping"}` — Heartbeat (server → client)
## Testing
### Unit Tests
- `test_terminal_session_v2.py` — Test batching, flow control, resize, reset
- `test_terminal_manager.py` — Test session lifecycle with new session class
### Integration Tests
- `test_terminal_websocket.py` — Full WebSocket round-trip
### Performance Tests
- `benchmark_terminal_latency.py` — Measure input/output latency
- `benchmark_terminal_throughput.py` — Measure max throughput
## Rollback Plan
Since this is a full rewrite with no legacy support:
- Keep a backup branch of the old terminal code
- Feature flag in frontend: `?terminal=v2` to test before full rollout
- Monitor error rates after deployment
@@ -0,0 +1,160 @@
# SDD Exploration: Responsive Web Terminal
## Status
**Phase:** explore
**Date:** 2026-06-02
**Owner:** el Gentleman (parent session)
**Scope:** Terminal I/O latency, rendering performance, connection stability
## Goal
Achieve VS Code Server-level terminal responsiveness: near-local latency on keystrokes, smooth scrolling, no jank on output bursts, and instant resize reactions.
## Current Architecture
### Data Flow
```
Container shell → docker exec PTY → host PTY master fd → select.select(0.1s)
→ Python read loop (10ms sleep fallback) → WebSocket.send_bytes()
→ WebSocket (Blob mode) → frontend arrayBuffer decode → xterm.js.write()
```
### Key Files
| File | Responsibility |
|------|---------------|
| `apps/web/src/components/terminal.tsx` | xterm.js, WebSocket client, FitAddon |
| `apps/api/src/api/terminal.py` | WebSocket endpoint, auth, read/write/heartbeat loops |
| `apps/api/src/services/terminal_session.py` | PTY creation, docker exec subprocess, I/O |
| `apps/api/src/services/terminal_manager.py` | Session lifecycle, persistence, idle cleanup |
### Current Bottlenecks
#### 1. Blocking Read with 100ms Timeout
```python
# terminal_session.py:read_output()
readable, _, _ = select.select([self._master_fd], [], [], 0.1)
if readable:
data = os.read(self._master_fd, 4096)
```
**Problem:** `select.select` blocks up to 100ms when no data is available. With the read loop in `terminal.py` doing `asyncio.sleep(0.01)` between calls, worst-case latency from shell output to WebSocket is ~110ms.
**VS Code approach:** node-pty uses libuv's epoll/kqueue watchers — event-driven, no polling timeout.
#### 2. WebSocket Blob → arrayBuffer Conversion
```typescript
// terminal.tsx
ws.onmessage = (event) => {
if (event.data instanceof Blob) {
event.data.arrayBuffer().then((buffer) => {
const data = new Uint8Array(buffer);
termRef.current?.write(data);
});
}
}
```
**Problem:** Blob → arrayBuffer is async and adds microtask latency. Also forces GC pressure from transient Blobs.
**VS Code approach:** Uses `ws.binaryType = "arraybuffer"` — receives ArrayBuffer directly, zero-copy into Uint8Array.
#### 3. No asyncio-Native PTY Reading
The PTY master fd is read with synchronous `os.read()` inside an async coroutine. This blocks the event loop thread for the duration of the read.
**VS Code approach:** node-pty's C++ binding hooks into libuv's event loop natively — true async I/O.
#### 4. Docker Exec Subprocess Overhead
```python
# terminal_session.py:start()
self.process = await asyncio.create_subprocess_exec(
"docker", "exec", "-it", "-e", "TERM=xterm",
self.container_id, "bash", "-c", shell_cmd,
stdin=self._slave_fd, stdout=self._slave_fd, stderr=self._slave_fd,
)
```
**Problem:** Spawns a new `docker exec` process on the host. Adds process startup latency and an extra process hop.
**Alternative:** Docker Engine API's `attach` endpoint with `logs=0&stream=1&stdin=1&stdout=1&stderr=1` — streams directly to the API container via Unix socket. No host subprocess.
#### 5. No Flow Control / Backpressure
If a command dumps output faster than the WebSocket can send (e.g., `cat /dev/urandom | base64`), data piles up in:
- The PTY kernel buffer (limited, ~4KB)
- Python's deque circular buffer (10KB)
- WebSocket's internal buffer (unbounded in some implementations)
- xterm.js parser queue
**VS Code approach:** Implements explicit flow control — pauses the PTY when the client buffer exceeds a threshold, resumes when drained.
#### 6. xterm.js Renderer
Current: DOM renderer (default).
**VS Code approach:** Canvas renderer with WebGL addon for GPU-accelerated rendering.
## Measurement Baseline
Before optimization, we need metrics:
| Metric | How to Measure | Target |
|--------|---------------|--------|
| Input latency | Time from keypress to character appearing | < 16ms (1 frame) |
| Output throughput | Bytes/sec for `cat /dev/zero` | > 1 MB/s |
| Resize latency | Time from resize message to shell reacting | < 50ms |
| Reconnection time | Time from disconnect to full replay | < 200ms |
| Frame drops | Dropped frames during `yes` command | 0 |
## Improvement Directions
### Direction A: Low-Latency Read Loop (Quick Win)
Replace `select.select` + `os.read` with `asyncio` native approach:
- Use `loop.add_reader()` to register a callback when fd is readable
- Or use `asyncio.to_thread()` with blocking `os.read` and immediate wake
- Eliminate the 100ms timeout and 10ms sleep
### Direction B: WebSocket Binary Mode (Quick Win)
Set `ws.binaryType = "arraybuffer"` on frontend, send binary frames directly.
Eliminates Blob → arrayBuffer conversion.
### Direction C: Docker Engine API Attach (Medium)
Replace `docker exec` subprocess with direct container attach via Docker SDK or HTTP API:
```python
from docker import DockerClient
client = DockerClient()
container = client.containers.get(container_id)
socket = container.attach_socket(params={...})
# socket is a raw TCP/Unix socket — read with asyncio
```
**Pros:** No subprocess overhead, direct stream to container
**Cons:** Requires Docker SDK or raw HTTP over Unix socket; needs `docker` group permissions
### Direction D: Flow Control (Medium)
Add backpressure mechanism:
1. Measure WebSocket send buffer depth on backend
2. Pause reading from PTY when buffer exceeds threshold (e.g., 64KB)
3. Resume when buffer drains below threshold
4. Frontend: measure xterm.js parser queue depth, pause via control message
### Direction E: WebGL Renderer (Quick Win)
Add xterm-addon-webgl:
```typescript
import { WebglAddon } from 'xterm-addon-webgl';
term.loadAddon(new WebglAddon());
```
**Pros:** GPU-accelerated, much faster for large output bursts
**Cons:** Falls back to canvas/DOM if WebGL unavailable; slightly higher init time
### Direction F: Output Batching (Quick Win)
Batch small writes before sending over WebSocket:
- Collect output for 1-2ms
- Send as single binary frame
- Reduces WebSocket frame overhead for high-frequency small writes (e.g., progress bars)
## Recommended Next Steps
1. **Measure baseline** with synthetic benchmarks
2. **Implement Directions A + B + F** (low-risk, high-impact)
3. **Evaluate Direction C** (Docker API attach) vs keeping docker exec
4. **Add Direction D** (flow control) if throughput tests show issues
5. **Add Direction E** (WebGL) as frontend enhancement
## Risks
- Docker API attach may not support PTY mode as cleanly as `docker exec -it`
- WebGL addon may have compatibility issues on older GPUs
- Flow control adds complexity; premature optimization risk
- Changes to core I/O loop could introduce stability regressions
@@ -0,0 +1,71 @@
# Proposal: High-Performance Web Terminal
## Status
**Phase:** proposal → spec → design → tasks → apply
**Date:** 2026-06-02
**Owner:** el Gentleman
**Scope:** Terminal I/O latency, rendering performance, connection stability
## Problem
The current web terminal has noticeable latency on keystrokes, choppy scrolling, and poor performance during output bursts. Users report it feels "slow" compared to VS Code Server's terminal, which feels almost local.
## Goals
| Metric | Current | Target | How Measured |
|--------|---------|--------|--------------|
| Input latency (keypress → char visible) | ~110ms | < 16ms (1 frame) | `term.write()` timestamp diff |
| Output throughput (`cat /dev/zero`) | ~200KB/s | > 1 MB/s | Bytes/sec over 5s |
| Resize latency | ~200ms | < 50ms | Time from resize msg to shell SIGWINCH |
| Reconnection + replay | ~2s | < 300ms | Time from WS open to first rendered char |
| Frame drops during `yes` | Many | 0 | `requestAnimationFrame` counter |
## Non-Goals
- Changing the terminal UI/UX (chrome, controls, tabs)
- Adding new terminal features (search, multi-cursor, etc.)
- Changing authentication or session persistence model
- Supporting non-Docker container runtimes
## Constraints
- Must work with existing tool instance lifecycle (docker containers)
- Must preserve WebSocket-based architecture
- Must preserve session persistence across reconnections
- Must work in both development and production compose setups
## Solution Overview
Replace the blocking `select.select()` PTY read loop with asyncio-native event-driven I/O. Replace `docker exec` subprocess with Docker Engine API attach. Switch WebSocket to binary mode. Add output batching. Add WebGL renderer.
## Key Decisions
1. **Keep `docker exec` for now** — Docker SDK attach doesn't support PTY mode as cleanly. We can optimize the subprocess approach with proper fd handling.
2. **Use `asyncio.add_reader()`** — Native asyncio event-driven fd reading eliminates polling latency.
3. **Binary WebSocket frames**`ws.binaryType = "arraybuffer"` eliminates Blob conversion overhead.
4. **WebGL renderer with DOM fallback** — GPU acceleration where available, graceful fallback.
5. **Output batching with 2ms window** — Collect small writes before sending to reduce frame overhead.
6. **Flow control v2** — Client acknowledges processed bytes; server pauses reads when buffer is full.
## Risks
- **Event loop blocking**: `asyncio.add_reader()` on a PTY fd may not work on all platforms (should work on Linux)
- **WebGL compatibility**: Some GPUs/drivers may fail WebGL context creation
- **Docker exec subprocess**: Still adds overhead; may revisit Docker API attach in future
- **Full rewrite**: Large change surface; thorough testing required
## Acceptance Criteria
- [ ] Input latency < 16ms measured with synthetic benchmark
- [ ] Output throughput > 1 MB/s measured with `cat /dev/zero`
- [ ] Resize latency < 50ms
- [ ] Reconnection + replay < 300ms
- [ ] No frame drops during `yes` command
- [ ] All existing terminal tests pass
- [ ] WebGL renderer loads successfully on modern browsers
- [ ] Graceful fallback to DOM renderer if WebGL fails
- [ ] Flow control prevents memory bloat on `cat /dev/urandom | base64`
## Related
- `openspec/explorations/terminal-responsiveness.md` — Detailed bottleneck analysis
+176
View File
@@ -0,0 +1,176 @@
# Spec: High-Performance Web Terminal
## Overview
Complete rewrite of the terminal I/O pipeline for sub-frame latency and smooth rendering.
## Architecture
### Data Flow (New)
```
Container shell → docker exec PTY → host PTY master fd
→ asyncio.add_reader() callback (event-driven, zero polling)
→ output batcher (2ms window) → WebSocket.send_bytes()
→ WebSocket binary frame → frontend ArrayBuffer
→ xterm.js WebGL renderer → screen
```
### Components
#### 1. TerminalSession (backend)
**Responsibilities:**
- Create PTY via `pty.openpty()`
- Spawn `docker exec -it` with slave fd attached
- Read from PTY master fd using `asyncio.add_reader()`
- Batch output (2ms window) before sending to WebSocket
- Handle flow control (pause/resume reads based on client ack)
- Resize via `TIOCSWINSZ` + `SIGWINCH`
**Interface:**
```python
class TerminalSession:
async def start(self) -> None
async def read_loop(self, websocket) -> None # event-driven
async def write_input(self, data: bytes) -> None
async def resize(self, cols: int, rows: int) -> None
async def reset(self) -> None
async def close(self) -> None
# Flow control
def acknowledge_data(self, char_count: int) -> None
def pause_output(self) -> None
def resume_output(self) -> None
```
#### 2. TerminalManager (backend)
Unchanged responsibilities (session lifecycle, persistence, idle cleanup).
#### 3. WebSocket Handler (backend)
**Messages:**
| Direction | Type | Payload | Description |
|-----------|------|---------|-------------|
| C → S | `input` | `{"data": "base64"}` | Keystrokes / input |
| C → S | `resize` | `{"cols": 80, "rows": 24}` | Terminal resize |
| C → S | `ack` | `{"chars": 1024}` | Flow control ack |
| C → S | `reset` | `{}` | Reset session |
| S → C | `binary` | raw bytes | Terminal output |
| S → C | `status` | `{"status": "connected"}` | Connection status |
| S → C | `ping` | `{}` | Heartbeat |
**Key changes:**
- Output is sent as **binary WebSocket frames**, not Blob
- Flow control: server tracks unacknowledged bytes, pauses PTY reads at 64KB threshold
#### 4. TerminalComponent (frontend)
**Key changes:**
- `ws.binaryType = "arraybuffer"` before connection
- Binary frames written directly to xterm.js as `Uint8Array`
- Flow control: send `ack` messages every 4096 processed bytes
- WebGL renderer with DOM fallback
- Batch resize messages (debounce 50ms)
**xterm.js config:**
```typescript
const term = new Terminal({
cursorBlink: true,
fontSize: currentFontSize,
fontFamily: 'Menlo, Monaco, "Courier New", monospace',
lineHeight: 1.2,
letterSpacing: 0,
allowTransparency: false,
scrollback: 10000,
// Performance options
ignoreBracketedPasteMode: false,
fastScrollSensitivity: 5,
scrollSensitivity: 1,
});
```
### Flow Control Protocol
**Server-side buffer tracking:**
```python
self._unacknowledged_bytes = 0
self._flow_control_threshold = 64 * 1024 # 64KB
self._paused = False
def on_output(self, data: bytes) -> None:
self._unacknowledged_bytes += len(data)
if self._unacknowledged_bytes > self._flow_control_threshold:
self.pause_output()
def acknowledge_data(self, char_count: int) -> None:
self._unacknowledged_bytes = max(0, self._unacknowledged_bytes - char_count)
if self._paused and self._unacknowledged_bytes < self._flow_control_threshold / 2:
self.resume_output()
```
**Client-side ack strategy:**
- After every `term.write(data)`, accumulate processed bytes
- Send `ack` message every 4096 bytes or 100ms
### Output Batching
**Server-side batcher:**
```python
self._batch_buffer = bytearray()
self._batch_timer: asyncio.TimerHandle | None = None
self._batch_window_ms = 2
def queue_output(self, data: bytes) -> None:
self._batch_buffer.extend(data)
if self._batch_timer is None:
self._batch_timer = asyncio.get_event_loop().call_later(
self._batch_window_ms / 1000, self._flush_batch
)
async def _flush_batch(self) -> None:
self._batch_timer = None
if self._batch_buffer and websocket.open:
await websocket.send_bytes(bytes(self._batch_buffer))
self._batch_buffer.clear()
```
## Docker Exec Subprocess
**Command:**
```bash
docker exec -it -e TERM=xterm-256color <container_id> bash -il
```
**Why keep docker exec:**
- Docker SDK `attach()` doesn't support PTY allocation cleanly
- `docker exec -it` is the standard way to get an interactive TTY
- Subprocess overhead is acceptable compared to PTY latency improvements
**Optimization:** Pre-warm the connection by reusing the same `docker exec` process for the session lifetime.
## Error Handling
| Scenario | Behavior |
|----------|----------|
| WebGL init fails | Fall back to DOM renderer, log warning |
| Flow control ack lost | Server resumes after timeout (5s) |
| PTY fd closed | Close WebSocket with code 4004 |
| Docker exec exits | Close WebSocket with code 4004, allow reconnect |
| Binary frame too large | Split into multiple frames (max 64KB) |
## Testing Strategy
1. **Unit tests:** Mock PTY fd, verify batching, flow control, resize
2. **Integration tests:** Full WebSocket round-trip with test container
3. **Performance tests:**
- `time cat /dev/zero | head -c 10M` — measure throughput
- Rapid keypress script — measure input latency
- Resize storm — measure resize latency
4. **Browser tests:** WebGL fallback on devices without GPU
## Migration
Full rewrite — no migration needed. Old terminal code can be deleted.
+60
View File
@@ -0,0 +1,60 @@
# Tasks: High-Performance Web Terminal
## Task 1: Rewrite TerminalSession with asyncio-native I/O
**Status:** pending
**Files:** `apps/api/src/services/terminal_session.py` (full rewrite)
**Description:**
- Replace `select.select()` with `asyncio.add_reader()` for event-driven PTY reading
- Add output batching (2ms window)
- Add flow control (pause/resume based on client ack)
- Keep docker exec subprocess (optimized)
- Remove circular buffer (not needed with event-driven architecture)
## Task 2: Update TerminalManager for new session class
**Status:** pending
**Files:** `apps/api/src/services/terminal_manager.py`
**Description:**
- Update imports to use rewritten TerminalSession
- Verify session lifecycle methods still work
- Update DB persistence calls
## Task 3: Update WebSocket endpoint for binary frames + flow control
**Status:** pending
**Files:** `apps/api/src/api/terminal.py`
**Description:**
- Accept binary output frames from TerminalSession
- Handle `ack` flow control messages from client
- Send `ping` heartbeat
- Maintain existing auth and session management
## Task 4: Update frontend for binary WebSocket + WebGL
**Status:** pending
**Files:** `apps/web/src/components/terminal.tsx`, `apps/web/package.json`
**Description:**
- Set `ws.binaryType = "arraybuffer"`
- Send flow control `ack` messages
- Add xterm-addon-webgl with DOM fallback
- Optimize resize handling
## Task 5: Add performance benchmarks
**Status:** pending
**Files:** `apps/api/tests/benchmark_terminal.py`
**Description:**
- Input latency benchmark
- Output throughput benchmark
- Resize latency benchmark
- Reconnection time benchmark
## Task 6: Update/fix unit tests
**Status:** pending
**Files:** `apps/api/tests/unit/test_tool_instances_legacy.py`, new tests
**Description:**
- Fix any tests broken by terminal changes
- Add tests for new TerminalSession features
## Task 7: Run full test suite
**Status:** pending
**Description:**
- Run all API tests
- Verify no regressions
- Report quality gate results