diff --git a/.pi-map.index.md b/.pi-map.index.md index 0cda6a1..4c4bbf1 100644 --- a/.pi-map.index.md +++ b/.pi-map.index.md @@ -16,7 +16,7 @@ dir: . Trust boundary: index routes, map orients, source decides. ## role -Infrastructure and deployment configuration for a self-hosted project management platform with containerized services, SSO integration, and reverse proxy support. +A self-hosted project management and git repository platform with OAuth2 authentication, providing containerized deployment via Docker Compose. ## parent - ## children diff --git a/.pi-map.md b/.pi-map.md index 5d027e0..7bd5a54 100644 --- a/.pi-map.md +++ b/.pi-map.md @@ -18,7 +18,7 @@ index: ./.pi-map.index.md Trust boundary: index routes, map orients, source decides. ## role -Infrastructure and deployment configuration package for a self-hosted project management platform with containerized services, SSO integration, and reverse proxy support. +A self-hosted web-based project management and Git repository platform with OAuth2 authentication, containerized via Docker Compose. ## files - .env.example | Provides a template of environment variables for configuring a Headquarter application with PostgreSQL, Redis, Authentik SSO, and Docker/Traefik deployment - .gitignore | Specifies files and directories for Git to ignore across a multi-language project with Python, Node, and custom tooling | dep: Git @@ -31,7 +31,7 @@ Infrastructure and deployment configuration package for a self-hosted project ma - progress.md | Tracks completed and remaining tasks for a backend-frontend code refactoring project organized in 7 phases - swap-pane | Empty file with no functionality ## arch -Docker Compose-based microservices architecture with environment-driven configuration, separating PostgreSQL database, Redis cache, API backend, and web frontend behind Traefik reverse proxy with TLS termination. +Multi-tier containerized architecture with separate frontend/API services, PostgreSQL/Redis data layer, Traefik reverse proxy integration, and environment-driven configuration. ## tags docker, redis, git, application, postgresql, compose, traefik, project ## symbols diff --git a/apps/.pi-map.index.md b/apps/.pi-map.index.md index fe888c5..ccf1904 100644 --- a/apps/.pi-map.index.md +++ b/apps/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps ## role -Contains the main application entry points and executable modules for the project. +Contains the main executable applications and entry points for the project ## parent index: ./.pi-map.index.md map: ./.pi-map.md diff --git a/apps/.pi-map.md b/apps/.pi-map.md index 903c2f4..35568b3 100644 --- a/apps/.pi-map.md +++ b/apps/.pi-map.md @@ -4,10 +4,10 @@ dir: apps index: apps/.pi-map.index.md ## role -Contains the main application entry points and executable modules for the project. +Contains the main executable applications and entry points for the project ## files ## arch -Typically follows a modular architecture where each subdirectory represents a separate deployable application or service, often with shared libraries extracted to common packages. +Modular application architecture with separate deployable units, likely following a microservices or service-oriented pattern where each subdirectory represents an independent application with its own configuration, dependencies, and lifecycle ## tags - ## symbols diff --git a/apps/api/.pi-map.index.md b/apps/api/.pi-map.index.md index d4ab4dd..ed559d6 100644 --- a/apps/api/.pi-map.index.md +++ b/apps/api/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps/api ## role -FastAPI-based backend API that manages projects, git repositories, and development tools through Docker-orchestrated instances with PostgreSQL persistence. +FastAPI backend API that manages projects, git repositories, and development tools through Docker-based instances. ## parent index: apps/.pi-map.index.md map: apps/.pi-map.md diff --git a/apps/api/.pi-map.md b/apps/api/.pi-map.md index 40953fb..529882d 100644 --- a/apps/api/.pi-map.md +++ b/apps/api/.pi-map.md @@ -4,7 +4,7 @@ dir: apps/api index: apps/api/.pi-map.index.md ## role -FastAPI-based backend API that manages projects, git repositories, and development tools through Docker-orchestrated instances with PostgreSQL persistence. +FastAPI backend API that manages projects, git repositories, and development tools through Docker-based instances. ## files - .dockerignore | Specifies files and directories to exclude from Docker build context to reduce image size and avoid copying unnecessary files into containers. | dep: Docker - Dockerfile | Multi-stage Docker build for a Python application with Docker socket access, Cloudflare tunneling, and database dependency waiting | dep: python:3.11-slim, gcc, libpq-dev, docker-ce-cli, docker-compose-plugin, cloudflared, uvicorn, pyproject.toml dependencies @@ -14,7 +14,7 @@ FastAPI-based backend API that manages projects, git repositories, and developme - uv.lock | Lock file for the uv Python package manager that pins exact dependency versions and their artifact hashes for reproducible installations | dep: uv, Python 3.11+, aiosqlite, alembic, annotated-doc, annotated-types, anyio, ast-serialize, asyncpg, and many other PyPI packages - wait-for-db.sh | Wait for a PostgreSQL database to become available before executing a command, with configurable retry logic. | dep: nc (netcat), sh (POSIX shell), sleep ## arch -Async Python/FastAPI service using multi-stage Docker containers with Alembic migrations, uv dependency management, and Docker socket access for container orchestration. +Async Python/FastAPI service with PostgreSQL via Alembic migrations, multi-stage Docker deployment with Cloudflare tunneling, and uv-based dependency management. ## tags docker, alembic, python, database, fastapi, postgresql, asyncpg, uvicorn ## symbols diff --git a/apps/api/src/.pi-map.index.md b/apps/api/src/.pi-map.index.md index 231b3d0..d2ab1b4 100644 --- a/apps/api/src/.pi-map.index.md +++ b/apps/api/src/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps/api/src ## role -Core initialization and infrastructure package for the Headquarter API, providing centralized configuration, database connectivity, structured logging, and application bootstrap. +Core application package for the Headquarter API, providing configuration, database connectivity, logging infrastructure, and FastAPI application initialization. ## parent index: apps/api/.pi-map.index.md map: apps/api/.pi-map.md diff --git a/apps/api/src/.pi-map.md b/apps/api/src/.pi-map.md index ea4ad17..23f0609 100644 --- a/apps/api/src/.pi-map.md +++ b/apps/api/src/.pi-map.md @@ -4,7 +4,7 @@ dir: apps/api/src index: apps/api/src/.pi-map.index.md ## role -Core initialization and infrastructure package for the Headquarter API, providing centralized configuration, database connectivity, structured logging, and application bootstrap. +Core application package for the Headquarter API, providing configuration, database connectivity, logging infrastructure, and FastAPI application initialization. ## files - __init__.py | Marks the directory as a Python package for the Headquarter API. - config.py | Defines application configuration settings with environment-based overrides using Pydantic, including database URLs, service domains, OAuth/Authentik integration, JWT/session settings, and computed properties for environment-specific behavior. | exp: class:Settings, func:build_database_url(user: str, password: str, host: str, port: int, database: str) → str | dep: pydantic, pydantic_settings @@ -12,7 +12,7 @@ Core initialization and infrastructure package for the Headquarter API, providin - logging_config.py | Configures structured JSON logging with correlation ID injection, custom formatters, and HTTP request/exception middleware for a FastAPI application. | exp: class:CorrelationIdFilter, method:filter(self, record: logging.LogRecord) → bool, call:get_correlation_id, class:JSONFormatter, method:format(self, record: logging.LogRecord) → str, call:self.formatTime, call:record.getMessage, call:getattr, call:self.formatException, call:json.dumps, method:formatTime(self, record: logging.LogRecord, datefmt) → str, call:time.strftime, call:time.gmtime, class:RequestLoggingMiddleware, method:dispatch(self, request: Request, call_next: Callable) → Response, call:time.time, call:logger.info, call:call_next, call:int, call:logger.error, call:type, call:traceback.format_exc, class:ExceptionLoggingMiddleware, method:dispatch(self, request: Request, call_next: Callable) → Response, call:call_next, call:logger.critical, call:traceback.format_exc, func:configure_logging(level) → None, call:JSONFormatter, call:logging.StreamHandler, call:console_handler.setFormatter, call:console_handler.addFilter, call:CorrelationIdFilter, call:root_logger.setLevel, call:logging.getLogger("uvicorn").setLevel, call:logging.getLogger("uvicorn.access").setLevel, call:logging.getLogger("sqlalchemy.engine").setLevel, call:logger.info, call:logging.getLevelName | dep: json, logging, sys, time, traceback, collections.abc, fastapi, starlette.middleware.base, src.services.shared.correlation - main.py | Initializes and configures a FastAPI application for the "Headquarter API" with database setup, middleware, routing, and background services. | exp: func:_sanitize_validation_errors(errors), call:error.get, call:str, call:ctx.items, call:isinstance, call:type, call:sanitized.append, func:validation_exception_handler(request: Request, exc: RequestValidationError), call:exc.errors, call:logger.warning, call:_sanitize_validation_errors, call:JSONResponse, func:on_startup(), call:logger.info, call:init_database, call:logger.error, call:sys.exit, call:_health_monitor.start, call:seed_builtin_tool_types, func:on_shutdown(), call:logger.info, call:_health_monitor.stop | dep: logging, os, fastapi, fastapi.exceptions, fastapi.middleware.cors, fastapi.responses, fastapi.staticfiles, src.api.config, src.api.project, src.api.system, src.api.tool, src.api.user, src.api.workspace, src.config, src.models, src.database, src.logging_config, src.seeds.builtin_tool_types, src.services.instance, src.services.shared, sys, src.api.* ## arch -Layered infrastructure pattern with environment-aware Pydantic settings, async SQLAlchemy with retry resilience, structured JSON logging with correlation ID tracking, and FastAPI factory composition with middleware pipeline. +Layered architecture with environment-based Pydantic configuration, async SQLAlchemy with retry patterns, structured JSON logging with correlation ID tracking, and FastAPI middleware/routing composition. ## tags src, database, logging, call:logger.info, api, middleware, fastapi, filter ## symbols diff --git a/apps/api/src/services/.pi-map.index.md b/apps/api/src/services/.pi-map.index.md index 9a3633f..92d7ae8 100644 --- a/apps/api/src/services/.pi-map.index.md +++ b/apps/api/src/services/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps/api/src/services ## role -This directory serves as the services layer for the API application, intended to contain business logic and service implementations. +Service layer package for business logic implementation in the API application ## parent index: apps/api/src/.pi-map.index.md map: apps/api/src/.pi-map.md diff --git a/apps/api/src/services/.pi-map.md b/apps/api/src/services/.pi-map.md index 43c3ccb..68dc150 100644 --- a/apps/api/src/services/.pi-map.md +++ b/apps/api/src/services/.pi-map.md @@ -4,11 +4,11 @@ dir: apps/api/src/services index: apps/api/src/services/.pi-map.index.md ## role -This directory serves as the services layer for the API application, intended to contain business logic and service implementations. +Service layer package for business logic implementation in the API application ## files - __init__.py | Empty file with no functionality ## arch -The package follows a standard Python package structure with an empty `__init__.py` marker file, indicating it is a namespace package ready to house service modules, but currently contains no implemented services. +Standard Python package structure with placeholder for service-oriented architecture; currently uninitialized with no implemented services ## tags init, empty, functionality ## symbols diff --git a/apps/api/src/services/shared/.pi-map.index.md b/apps/api/src/services/shared/.pi-map.index.md index ea63f7e..c6ecad9 100644 --- a/apps/api/src/services/shared/.pi-map.index.md +++ b/apps/api/src/services/shared/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps/api/src/services/shared ## role -Provides reusable, cross-cutting infrastructure services for security, I/O, container operations, and workspace management used throughout the API layer. +Provides common infrastructure and utility services used across the API backend, including cross-cutting concerns like request tracing, file operations, notifications, Docker container management, and workspace lifecycle handling. ## parent index: apps/api/src/services/.pi-map.index.md map: apps/api/src/services/.pi-map.md diff --git a/apps/api/src/services/shared/.pi-map.md b/apps/api/src/services/shared/.pi-map.md index aae8550..e383c0a 100644 --- a/apps/api/src/services/shared/.pi-map.md +++ b/apps/api/src/services/shared/.pi-map.md @@ -4,7 +4,7 @@ dir: apps/api/src/services/shared index: apps/api/src/services/shared/.pi-map.index.md ## role -Provides reusable, cross-cutting infrastructure services for security, I/O, container operations, and workspace management used throughout the API layer. +Provides common infrastructure and utility services used across the API backend, including cross-cutting concerns like request tracing, file operations, notifications, Docker container management, and workspace lifecycle handling. ## files - __init__.py | Re-exports shared service classes and functions from a services package to provide a unified public API | dep: src.services.shared.correlation, src.services.shared.file_service, src.services.shared.notification_service, src.services.shared.permission_fixer, src.services.shared.readiness_probe, src.services.shared.ssh_keys, src.services.shared.tunnel, src.services.shared.workspace_manager, correlation, file_service, notification_service, permission_fixer, readiness_probe, ssh_keys, tunnel, workspace_manager - correlation.py | Provides async correlation ID tracking via context variables and FastAPI middleware for request tracing. | exp: class:CorrelationIdMiddleware, method:dispatch(self, request: Request, call_next), call:request.headers.get, call:str, call:uuid.uuid4, call:CORRELATION_ID.set, call:call_next, call:CORRELATION_ID.reset, func:get_correlation_id() → str, call:CORRELATION_ID.get, call:str, call:uuid.uuid4 | dep: contextvars, uuid, fastapi, starlette.middleware.base, fastapi.Request, starlette.middleware.base.BaseHTTPMiddleware @@ -14,11 +14,11 @@ Provides reusable, cross-cutting infrastructure services for security, I/O, cont - readiness_probe.py | Executes a retryable readiness probe command inside a Docker container with configurable timeout and interval | exp: func:execute_probe(container_id: str, command: str, timeout, interval) → tuple[bool, list[str]], call:asyncio.get_event_loop().time, call:logs.append, call:logger.debug, call:subprocess.run, call:result.stdout.strip, call:result.stderr.strip, call:asyncio.sleep | dep: asyncio, logging, subprocess - ssh_keys.py | Decrypts and writes SSH key files to instance directories for container mounting, with optional ownership configuration and SSH config generation. | exp: func:_get_fernet() → Fernet, call:Settings, call:hashlib.sha256(settings.session_secret.encode()).digest, call:settings.session_secret.encode, call:base64.urlsafe_b64encode, call:Fernet, func:_sanitize_filename(name: str) → str, call:re.sub, call:sanitized.strip, func:prepare_ssh_key_files(instance_dir: str, ssh_key, subdir, uid, gid, key_filename, write_config) → str, call:Path, call:ssh_dir.mkdir, call:_get_fernet, call:fernet.decrypt(ssh_key.private_key_encrypted.encode()).decode, call:ssh_key.private_key_encrypted.encode, call:private_key_path.write_text, call:os.chmod, call:public_key_path.write_text, call:config_path.write_text, call:os.chown, call:logger.debug, call:logger.warning, call:os.getuid, call:str, func:write_ssh_config(ssh_dir: str, key_filenames: list[str], uid, gid) → None, call:Path, call:ssh_dir_path.mkdir, call:lines.append, call:"\n".join, call:config_path.write_text, call:os.chmod, call:os.chown, func:cleanup_ssh_key_files(instance_dir: str) → None, call:Path, call:ssh_dir.exists, call:ssh_dir.iterdir, call:file_path.unlink, call:ssh_dir.rmdir | dep: logging, os, re, pathlib, cryptography.fernet, src.config, base64, hashlib - tunnel.py | Re-exports Docker tunnel functions from a nested module for backward compatibility. | dep: src.services.docker.tunnel -- workspace_manager.py | Manages Git workspace lifecycle operations including creation, deletion, sync, and migration of legacy tool instances into workspace-bound repositories. | exp: class:SyncResult, class:WorkspaceHasInstancesError, method:__init__(self, instances: list[dict]) → None, call:super().__init__, call:len, class:WorkspaceManager, method:_workspace_path(self, repo_id: uuid.UUID, name: str) → str, call:os.path.join, call:str, method:create(self, repo: GitRepository, user_id: uuid.UUID, name: str, branch, session) → Workspace, call:self._workspace_path, call:os.path.dirname, call:os.makedirs, call:contextlib.suppress, call:os.chmod, call:logger.info, call:os.path.exists, call:logger.warning, call:shutil.rmtree, call:getattr, call:session.execute, call:select(SSHKey).where, call:result.scalar_one_or_none, call:_get_fernet, call:fernet.decrypt( ssh_key_obj.private_key_encrypted.encode() ).decode, call:ssh_key_obj.private_key_encrypted.encode, call:GitService.clone, call:self._make_world_writable, call:Workspace, call:datetime.now, raise:ValueError, method:delete(self, workspace: Workspace, force, session) → None, call:self._get_instances, call:self._stop_and_delete_instance, call:os.path.exists, call:shutil.rmtree, call:logger.info, call:session.delete, raise:ValueError, raise:WorkspaceHasInstancesError, method:sync(self, workspace: Workspace, session) → SyncResult, call:logger.info, call:session.get, call:getattr, call:session.execute, call:select(SSHKey).where, call:result.scalar_one_or_none, call:_get_fernet, call:fernet.decrypt( ssh_key_obj.private_key_encrypted.encode() ).decode, call:ssh_key_obj.private_key_encrypted.encode, call:GitService.fetch, call:GitService.branch_exists_remotely, call:SyncResult, call:GitService.pull, call:self._make_world_writable, call:datetime.now, method:_make_world_writable(self, path: str) → None, call:contextlib.suppress, call:os.chmod, call:os.walk, call:os.path.join, call:os.stat, method:_get_instances(self, workspace: Workspace, session: AsyncSession) → list[ToolInstance], call:session.execute, call:select(ToolInstance).where, call:list, call:result.scalars().all, method:_stop_and_delete_instance(self, instance: ToolInstance, session: AsyncSession) → None, call:delete_tool_instance, call:logger.info, call:logger.error, method:ensure_instance_workspace(self, instance: ToolInstance, session: AsyncSession) → Workspace, call:session.get, call:self._workspace_name_exists, call:self._migrate_clone_into_workspace, call:self.create, call:session.add, call:session.commit, call:session.refresh, call:logger.info, raise:RuntimeError, method:_workspace_name_exists(self, session: AsyncSession, repo_id: uuid.UUID, name: str) → bool, call:session.execute, call:select(Workspace).where, call:result.scalar_one_or_none, method:_migrate_clone_into_workspace(self, instance: ToolInstance, repo: "GitRepository", session: AsyncSession, name: str) → Workspace, call:os.path.dirname, call:os.path.join, call:os.path.exists, call:self._workspace_path, call:os.makedirs, call:contextlib.suppress, call:os.chmod, call:shutil.rmtree, call:shutil.move, call:self._make_world_writable, call:Workspace, call:datetime.now, call:session.add, call:session.flush, raise:RuntimeError | dep: contextlib, logging, os, shutil, stat, uuid, dataclasses, datetime, typing, sqlalchemy, src.models, src.services.git.git_service, src.services.shared.ssh_keys, sqlalchemy.ext.asyncio, src.services.tool.instance_service +- workspace_manager.py | Manages workspace lifecycle operations including creation, deletion, synchronization, and migration of legacy tool instances into workspaces with git repository cloning. | exp: class:SyncResult, class:WorkspaceHasInstancesError, method:__init__(self, instances: list[dict]) → None, call:super().__init__, call:len, class:WorkspaceManager, method:_workspace_path(self, workspace_id: uuid.UUID, repo: "GitRepository") → str, call:os.path.join, call:str, call:self._repo_directory_name, method:create(self, repo: GitRepository, user_id: uuid.UUID, name: str, branch, session) → Workspace, call:self._repo_directory_name, call:uuid.uuid4, call:os.path.join, call:str, call:logger.info, call:os.path.exists, call:logger.warning, call:shutil.rmtree, call:os.makedirs, call:contextlib.suppress, call:os.chmod, call:getattr, call:session.execute, call:select(SSHKey).where, call:result.scalar_one_or_none, call:_get_fernet, call:fernet.decrypt( ssh_key_obj.private_key_encrypted.encode() ).decode, call:ssh_key_obj.private_key_encrypted.encode, call:GitService.clone, call:os.path.isdir, call:os.listdir, call:len, call:self._make_world_writable, call:Workspace, call:datetime.now, raise:ValueError, raise:RuntimeError, method:delete(self, workspace: Workspace, force, session) → None, call:self._get_instances, call:self._stop_and_delete_instance, call:os.path.exists, call:shutil.rmtree, call:logger.info, call:session.delete, raise:ValueError, raise:WorkspaceHasInstancesError, method:sync(self, workspace: Workspace, session) → SyncResult, call:logger.info, call:session.get, call:getattr, call:session.execute, call:select(SSHKey).where, call:result.scalar_one_or_none, call:_get_fernet, call:fernet.decrypt( ssh_key_obj.private_key_encrypted.encode() ).decode, call:ssh_key_obj.private_key_encrypted.encode, call:GitService.fetch, call:GitService.branch_exists_remotely, call:SyncResult, call:GitService.pull, call:self._make_world_writable, call:datetime.now, method:_make_world_writable(self, path: str) → None, call:contextlib.suppress, call:os.chmod, call:os.walk, call:os.path.join, call:os.stat, method:_get_instances(self, workspace: Workspace, session: AsyncSession) → list[ToolInstance], call:session.execute, call:select(ToolInstance).where, call:list, call:result.scalars().all, method:_stop_and_delete_instance(self, instance: ToolInstance, session: AsyncSession) → None, call:delete_tool_instance, call:logger.info, call:logger.error, method:ensure_instance_workspace(self, instance: ToolInstance, session: AsyncSession) → Workspace, call:session.get, call:self._workspace_name_exists, call:self._migrate_clone_into_workspace, call:self.create, call:session.add, call:session.commit, call:session.refresh, call:logger.info, raise:RuntimeError, method:_workspace_name_exists(self, session: AsyncSession, repo_id: uuid.UUID, name: str) → bool, call:session.execute, call:select(Workspace).where, call:result.scalar_one_or_none, method:_migrate_clone_into_workspace(self, instance: ToolInstance, repo: "GitRepository", session: AsyncSession, name: str) → Workspace, call:os.path.dirname, call:os.path.join, call:os.path.exists, call:uuid.uuid4, call:str, call:self._repo_directory_name, call:os.makedirs, call:contextlib.suppress, call:os.chmod, call:shutil.rmtree, call:shutil.move, call:self._make_world_writable, call:Workspace, call:datetime.now, call:session.add, call:session.flush, raise:RuntimeError | dep: contextlib, logging, os, shutil, stat, uuid, dataclasses, datetime, typing, sqlalchemy, src.models, src.services.git.git_service, src.services.shared.ssh_keys, src.utils.git_url_parser, sqlalchemy.ext.asyncio, src.services.tool.instance_service ## arch -Modular utility services pattern with async singletons, context variables for request tracing, defensive security (path traversal/ownership isolation), Docker exec abstraction, and Git-backed workspace lifecycle management. +Modular service-oriented architecture with singleton async services, context variable-based state propagation for request correlation, defensive path traversal protections, and Docker-centric container orchestration patterns with privilege escalation for filesystem operations. ## tags -error, workspace, call:self., get, key, src, call:session.execute, call:ssh +error, call:self., get, workspace, src, key, call:str, call:session.execute ## symbols - CorrelationIdMiddleware - FileEntry diff --git a/apps/api/src/services/shared/workspace_manager.py b/apps/api/src/services/shared/workspace_manager.py index 4d0fc4b..2b80628 100644 --- a/apps/api/src/services/shared/workspace_manager.py +++ b/apps/api/src/services/shared/workspace_manager.py @@ -17,6 +17,7 @@ from sqlalchemy import select from src.models import Workspace from src.services.git.git_service import GitService from src.services.shared.ssh_keys import _get_fernet +from src.utils.git_url_parser import extract_base_repo_url if TYPE_CHECKING: from sqlalchemy.ext.asyncio import AsyncSession @@ -47,9 +48,32 @@ class WorkspaceManager: BASE_PATH = "/data/working-copies" - def _workspace_path(self, repo_id: uuid.UUID, name: str) -> str: - """Return the filesystem path for a workspace.""" - return os.path.join(self.BASE_PATH, str(repo_id), name) + @staticmethod + def _repo_directory_name(repo: "GitRepository") -> str: + """Return the directory name git would create for a standard clone. + + Prefers the name parsed from the remote URL and falls back to the + user-provided repository name when no remote URL is available. + """ + if repo.remote_url: + base_url = extract_base_repo_url(repo.remote_url) or repo.remote_url + name = base_url.rstrip("/").split("/")[-1] + if name.endswith(".git"): + name = name[:-4] + if name: + return name + return repo.name + + def _workspace_path(self, workspace_id: uuid.UUID, repo: "GitRepository") -> str: + """Return the filesystem path for a workspace. + + Layout: /data/working-copies/{workspace_id}/{repo_name}/ + The workspace_id prevents collisions between workspaces, and the + repo_name matches the directory git clone naturally creates. + """ + return os.path.join( + self.BASE_PATH, str(workspace_id), self._repo_directory_name(repo) + ) async def create( self, @@ -74,24 +98,31 @@ class WorkspaceManager: Raises: RuntimeError: If git clone fails. """ - path = self._workspace_path(repo.id, name) - parent = os.path.dirname(path) - os.makedirs(parent, exist_ok=True) - # Ensure container users (various UIDs) can write to workspace dirs - with contextlib.suppress(OSError): - os.chmod(parent, 0o777) - - logger.info( - "Creating workspace: name=%s, repo=%s, branch=%s", name, repo.id, branch - ) - if not repo.remote_url: raise ValueError("Repository has no remote URL") + repo_dir_name = self._repo_directory_name(repo) + workspace_id = uuid.uuid4() + parent_path = os.path.join(self.BASE_PATH, str(workspace_id)) + clone_target = parent_path # git clone creates {repo_dir_name}/ inside this + + logger.info( + "Creating workspace: id=%s name=%s repo=%s branch=%s", + workspace_id, + name, + repo.id, + branch, + ) + # Remove stale directory from previous failed/aborted clone - if os.path.exists(path): - logger.warning("Removing stale workspace directory: %s", path) - shutil.rmtree(path, ignore_errors=True) + if os.path.exists(parent_path): + logger.warning("Removing stale workspace directory: %s", parent_path) + shutil.rmtree(parent_path, ignore_errors=True) + + os.makedirs(parent_path, exist_ok=True) + # Ensure container users (various UIDs) can write to workspace dirs + with contextlib.suppress(OSError): + os.chmod(parent_path, 0o777) # Load SSH key if repo has one ssh_key = None @@ -108,19 +139,38 @@ class WorkspaceManager: ssh_key_obj.private_key_encrypted.encode() ).decode() - await GitService.clone(repo.remote_url, branch, path, ssh_key=ssh_key) - self._make_world_writable(path) + await GitService.clone(repo.remote_url, branch, clone_target, ssh_key=ssh_key) + + expected_path = os.path.join(parent_path, repo_dir_name) + if not os.path.isdir(expected_path): + # git clone can create a different directory name than expected for + # some URL shapes; fall back to the single directory git created. + entries = [ + entry + for entry in os.listdir(parent_path) + if os.path.isdir(os.path.join(parent_path, entry)) + ] + if len(entries) == 1: + expected_path = os.path.join(parent_path, entries[0]) + else: + raise RuntimeError( + f"Expected git clone to create {repo_dir_name}/ under " + f"{parent_path}, but found: {entries}" + ) + + self._make_world_writable(expected_path) workspace = Workspace( + id=workspace_id, name=name, repo_id=repo.id, user_id=user_id, branch=branch, - path=path, + path=expected_path, status="ready", last_sync_at=datetime.now(), ) - logger.info("Workspace created: %s", workspace.id) + logger.info("Workspace created: %s at %s", workspace.id, workspace.path) return workspace async def delete( @@ -375,24 +425,30 @@ class WorkspaceManager: if not os.path.exists(clone_path): raise RuntimeError(f"Clone path not found: {clone_path}") - path = self._workspace_path(repo.id, name) - parent = os.path.dirname(path) - os.makedirs(parent, exist_ok=True) + workspace_id = uuid.uuid4() + parent_path = os.path.join(self.BASE_PATH, str(workspace_id)) + repo_dir_name = self._repo_directory_name(repo) + target_path = os.path.join(parent_path, repo_dir_name) + + os.makedirs(parent_path, exist_ok=True) with contextlib.suppress(OSError): - os.chmod(parent, 0o777) + os.chmod(parent_path, 0o777) - if os.path.exists(path): - shutil.rmtree(path, ignore_errors=True) + if os.path.exists(target_path): + shutil.rmtree(target_path, ignore_errors=True) - shutil.move(clone_path, path) - self._make_world_writable(path) + # Move the existing clone into the repo-named subdirectory so the + # workspace path matches the natural git clone layout. + shutil.move(clone_path, target_path) + self._make_world_writable(target_path) workspace = Workspace( + id=workspace_id, name=name, repo_id=repo.id, user_id=instance.owner_id, branch=instance.branch or "main", - path=path, + path=target_path, status="ready", last_sync_at=datetime.now(), ) diff --git a/apps/api/src/services/tool/.pi-map.index.md b/apps/api/src/services/tool/.pi-map.index.md index 8a10977..d4f861b 100644 --- a/apps/api/src/services/tool/.pi-map.index.md +++ b/apps/api/src/services/tool/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps/api/src/services/tool ## role -Provides containerized execution environment for tools with git repository access, configuration management, and secure remote connectivity. +Provides infrastructure orchestration for isolated tool instances, managing their complete lifecycle from provisioning to teardown. ## parent index: apps/api/src/services/.pi-map.index.md map: apps/api/src/services/.pi-map.md diff --git a/apps/api/src/services/tool/.pi-map.md b/apps/api/src/services/tool/.pi-map.md index 9b55631..c4275cc 100644 --- a/apps/api/src/services/tool/.pi-map.md +++ b/apps/api/src/services/tool/.pi-map.md @@ -4,11 +4,11 @@ dir: apps/api/src/services/tool index: apps/api/src/services/tool/.pi-map.index.md ## role -Provides containerized execution environment for tools with git repository access, configuration management, and secure remote connectivity. +Provides infrastructure orchestration for isolated tool instances, managing their complete lifecycle from provisioning to teardown. ## files -- instance_service.py | Manages Docker-based tool instances including git repository mounting, config profile resolution, container lifecycle operations, and SSH tunnel management. | exp: func:_get_repository_mount_name(repo: GitRepository) → str, call:extract_base_repo_url, call:base_url.rstrip("/").split, call:name.endswith, func:_chown_path(path: str, uid: int, gid: int) → None, call:os.path.isdir, call:os.walk, call:os.path.join, call:contextlib.suppress, call:os.chown, call:logger.warning, func:_chown_staged_mounts(extra_volumes: list[dict], instance_dir: str, uid: int, gid: int) → None, call:vol.get, call:source.startswith, call:_chown_path, func:resolve_git_mounts(session: AsyncSession, resolved: ResolvedProfile, instance_dir, working_directory, home_dir) → list[dict], call:tasks.append, call:resolve_single_git_mount, call:asyncio.gather, call:isinstance, call:logger.warning, call:volume_mounts.extend, func:normalize_git_mount(entry: dict) → dict, call:dict, call:entry.get, call:entry.pop, func:clone_git_repo(remote_url: str, branch: str | None, clone_parent: str) → str, call:hashlib.md5(remote_url.encode()).hexdigest, call:remote_url.encode, call:remote_url.split("/")[-1].replace, call:os.path.join, call:os.path.exists, call:os.makedirs, call:clone_repository, call:logger.debug, call:logger.warning, call:pull_repository_updates, call:checkout_branch, func:resolve_git_mount_mappings(repo_path: str, mappings: list[dict], working_directory: str | None, home_dir) → list[dict], call:mapping.get, call:logger.warning, call:expand_container_path, call:target_path.startswith, call:os.path.join, call:expand_glob_source, call:os.path.exists, call:len, call:os.path.relpath, call:volume_mounts.append, call:logger.debug, func:resolve_single_git_mount(session: AsyncSession, git_mount: dict, instance_dir, working_directory, home_dir) → list[dict], call:normalize_git_mount, call:git_mount.get, call:logger.warning, call:asyncio.to_thread, call:resolve_git_mount_mappings, func:checkout_branch(repo_path: str, branch: str) → bool, call:subprocess.run, call:logger.warning, call:result.stderr.strip, func:pull_repository_updates(repo_path: str, remote_url: str) → None, call:subprocess.run, raise:RuntimeError, func:expand_glob_source(source_path: str, repo_path: str) → list[str], call:any, call:os.path.exists, call:glob_module.glob, call:len, call:os.path.abspath, call:abs_path.startswith, call:results.append, call:logger.warning, func:validate_config_profile(session: AsyncSession, profile_id: str | None, user_id: uuid.UUID, project_id: uuid.UUID, tool_type_id: uuid.UUID) → uuid.UUID | None, call:uuid.UUID, call:session.get, raise:HTTPException, func:sanitize_compose_file(compose_path: str) → None, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].items, call:isinstance, call:port_mapping.split, call:len, call:valid_ports.append, call:compose_file.write_text, call:yaml.dump, func:modify_compose_file(compose_path: str, port_override, start_command, working_directory, extra_volumes, home_dir) → None, call:Path, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].items, call:enumerate, call:isinstance, call:port_mapping.split, call:expand_container_path, call:vol.get, call:service_config["volumes"].append, call:service_config.get, call:sort_volumes_by_specificity, call:compose_file.write_text, call:yaml.dump, func:ensure_container_name_in_compose(compose_path: str, container_name: str) → None, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].items, call:container_name.lower, call:compose_file.write_text, call:yaml.dump, call:logger.info, func:ensure_web_bind_address(compose_path: str, tool_type_name: str, default_port: int) → None, call:KNOWN_BIND_FIXES.get, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].values, call:service_config.get, call:compose_file.write_text, call:yaml.dump, call:logger.warning, call:logger.info, func:ensure_backend_network_in_compose(compose_path: str) → None, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:get_backend_network_name, call:compose_data["services"].values, call:svc_config.get, call:compose_file.write_text, call:yaml.dump, call:logger.info, func:prepare_manifest_instance(session: AsyncSession, instance: ToolInstance, instance_dir: str, repo_path: str, env_vars: dict, extra_volumes: list, working_directory: str | None) → tuple[str, str, dict, str], call:session.get, call:dict, call:resolve_base, call:deep_merge, call:logger.warning, call:merge_with_config, call:manifest.pop, call:env_vars.update, call:extra_volumes.extend, call:compute_image_tag, call:subprocess.run, call:check.stdout.strip, call:compile_dockerfile, call:compile_entrypoint, call:logger.debug, call:len, call:asyncio.to_thread, call:logger.info, call:resolve_profile, call:gm.get, call:_get_repository_mount_name, call:os.path.basename, call:os.path.normpath, call:instance.name.lower, call:compile_compose, call:manifest.get, call:datetime.now, call:get_manifest_home_dir, raise:RuntimeError, func:create_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, data: "CreateInstanceRequest") → ToolInstance, call:session.get, call:uuid.UUID, call:validate_config_profile, call:uuid.uuid4, call:select(ToolInstance) .where(ToolInstance.workspace_id == workspace_id) .where(ToolInstance.tool_type_id == tool_type_id) .where, call:select(ToolInstance) .where(ToolInstance.repository_id == repo_id) .where(ToolInstance.tool_type_id == tool_type_id) .where, call:session.execute, call:len, call:result.scalars().all, call:ensure_instance_directory, call:os.path.join, call:find_free_port, call:f"headquarter/{instance_name}:latest".lower, call:asyncio.to_thread, call:logger.error, call:logger.info, call:_get_repository_mount_name, call:instance_name.lower, call:write_compose_file, call:dict, call:resolve_base, call:deep_merge, call:compute_image_tag, call:compile_compose, call:str, call:render_compose_template, call:ToolInstance, call:session.add, call:session.commit, call:session.refresh, call:publish_lifecycle_event, raise:ValueError, raise:RuntimeError, func:start_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID, data: "StartInstanceRequest | None") → dict, call:session.get, call:validate_config_profile, call:session.commit, call:os.path.exists, call:logger.info, call:dict, call:resolve_base, call:deep_merge, call:get_manifest_home_dir, call:manifest.get, call:user_cfg.get, call:logger.debug, call:os.path.dirname, call:resolve_profile, call:apply_resolved_profile, call:env_vars.update, call:config_files.update, call:extra_volumes.extend, call:resolve_git_mounts, call:profile_hints.get, call:len, call:logger.error, call:write_env_file, call:write_config_files, call:uuid.UUID, call:ssh_keys_to_mount.append, call:logger.warning, call:os.path.join, call:os.makedirs, call:_sanitize_filename, call:key_filenames.append, call:prepare_ssh_key_files, call:write_ssh_config, call:extra_volumes.append, call:_chown_staged_mounts, call:prepare_manifest_instance, call:write_compose_file, call:logger.exception, call:modify_compose_file, call:sanitize_compose_file, call:ensure_web_bind_address, call:ensure_container_name_in_compose, call:ensure_backend_network_in_compose, call:execute_compose_command, call:instance.name.lower, call:get_container_id, call:publish_lifecycle_event, call:wait_for_container_running, call:get_container_logs, call:resolved_manifest.get, call:apply_mount_permissions, call:home_dir.startswith, call:apply_ssh_permissions, call:probe_config.get, call:execute_probe, call:datetime.now().isoformat, call:"\n".join, call:get_container_status, call:start_tunnel, call:str, call:traceback.format_exc, raise:ValueError, raise:RuntimeError, func:restart_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID) → dict, call:session.get, call:stop_tunnel, call:logger.debug, call:logger.warning, call:os.path.exists, call:os.path.dirname, call:resolve_profile, call:apply_resolved_profile, call:write_env_file, call:logger.error, call:sanitize_compose_file, call:ensure_web_bind_address, call:ensure_container_name_in_compose, call:ensure_backend_network_in_compose, call:execute_compose_command, call:datetime.now, call:session.commit, call:start_tunnel, call:publish_lifecycle_event, raise:ValueError, raise:RuntimeError, func:delete_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID, force) → None, call:session.get, call:os.path.dirname, call:os.path.join, call:os.path.exists, call:check_dirty_state, call:stop_tunnel, call:logger.debug, call:logger.warning, call:execute_compose_command, call:shutil.rmtree, call:publish_lifecycle_event, call:session.delete, call:session.commit, raise:ValueError, raise:RuntimeError, func:recreate_instance_tunnel(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID) → dict, call:session.get, call:instance.name.lower, call:logger.info, call:get_container_id, call:logger.error, call:get_backend_network_name, call:is_container_on_network, call:connect_container_to_network, call:get_container_ip_on_network, call:logger.warning, call:recreate_tunnel, call:check_tunnel_health, call:health.get, call:subprocess.run, call:probe.stdout.strip, call:session.commit, call:logger.exception, raise:ValueError, raise:RuntimeError, func:stop_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID) → dict, call:session.get, call:stop_tunnel, call:logger.warning, call:os.path.exists, call:execute_compose_command, call:datetime.now, call:session.commit, call:publish_lifecycle_event, raise:ValueError, func:rename_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID, display_name: str) → ToolInstance, call:session.get, call:display_name.strip, call:session.commit, call:session.refresh, raise:ValueError | dep: asyncio, contextlib, logging, os, subprocess, uuid, datetime, fastapi, sqlalchemy, sqlalchemy.ext.asyncio, src.models, src.schemas.tool, src.services.git.clone, src.services.config.config_profile_resolver, src.services.docker, src.services.shared.tunnel, src.services.build.docker_build, src.services.build.manifest_compiler, src.services.shared.permission_fixer, src.services.shared.readiness_probe, src.services.shared.ssh_keys, src.services.instance.event_bus, src.services.instance.lifecycle_hooks, src.utils.git_url_parser, hashlib, yaml, pathlib, traceback, shutil +- instance_service.py | Manages the lifecycle of tool instances including git repository mounting, Docker container orchestration, config profile resolution, SSH tunnel management, and permission handling for a development environment platform. | exp: func:_get_repository_mount_name(repo: GitRepository, workspace) → str, call:os.path.basename, call:os.path.normpath, call:extract_base_repo_url, call:base_url.rstrip("/").split, call:name.endswith, func:_chown_path(path: str, uid: int, gid: int) → None, call:os.path.isdir, call:os.walk, call:os.path.join, call:contextlib.suppress, call:os.chown, call:logger.warning, func:_chown_staged_mounts(extra_volumes: list[dict], instance_dir: str, uid: int, gid: int) → None, call:vol.get, call:source.startswith, call:_chown_path, func:resolve_git_mounts(session: AsyncSession, resolved: ResolvedProfile, instance_dir, working_directory, home_dir) → list[dict], call:tasks.append, call:resolve_single_git_mount, call:asyncio.gather, call:isinstance, call:logger.warning, call:volume_mounts.extend, func:normalize_git_mount(entry: dict) → dict, call:dict, call:entry.get, call:entry.pop, func:clone_git_repo(remote_url: str, branch: str | None, clone_parent: str) → str, call:hashlib.md5(remote_url.encode()).hexdigest, call:remote_url.encode, call:remote_url.split("/")[-1].replace, call:os.path.join, call:os.path.exists, call:os.makedirs, call:clone_repository, call:logger.debug, call:logger.warning, call:pull_repository_updates, call:checkout_branch, func:resolve_git_mount_mappings(repo_path: str, mappings: list[dict], working_directory: str | None, home_dir) → list[dict], call:mapping.get, call:logger.warning, call:expand_container_path, call:target_path.startswith, call:os.path.join, call:expand_glob_source, call:os.path.exists, call:len, call:os.path.relpath, call:volume_mounts.append, call:logger.debug, func:resolve_single_git_mount(session: AsyncSession, git_mount: dict, instance_dir, working_directory, home_dir) → list[dict], call:normalize_git_mount, call:git_mount.get, call:logger.warning, call:asyncio.to_thread, call:resolve_git_mount_mappings, func:checkout_branch(repo_path: str, branch: str) → bool, call:subprocess.run, call:logger.warning, call:result.stderr.strip, func:pull_repository_updates(repo_path: str, remote_url: str) → None, call:subprocess.run, raise:RuntimeError, func:expand_glob_source(source_path: str, repo_path: str) → list[str], call:any, call:os.path.exists, call:glob_module.glob, call:len, call:os.path.abspath, call:abs_path.startswith, call:results.append, call:logger.warning, func:validate_config_profile(session: AsyncSession, profile_id: str | None, user_id: uuid.UUID, project_id: uuid.UUID, tool_type_id: uuid.UUID) → uuid.UUID | None, call:uuid.UUID, call:session.get, raise:HTTPException, func:sanitize_compose_file(compose_path: str) → None, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].items, call:isinstance, call:port_mapping.split, call:len, call:valid_ports.append, call:compose_file.write_text, call:yaml.dump, func:modify_compose_file(compose_path: str, port_override, start_command, working_directory, extra_volumes, home_dir) → None, call:Path, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].items, call:enumerate, call:isinstance, call:port_mapping.split, call:expand_container_path, call:vol.get, call:service_config["volumes"].append, call:service_config.get, call:sort_volumes_by_specificity, call:compose_file.write_text, call:yaml.dump, func:ensure_container_name_in_compose(compose_path: str, container_name: str) → None, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].items, call:container_name.lower, call:compose_file.write_text, call:yaml.dump, call:logger.info, func:ensure_web_bind_address(compose_path: str, tool_type_name: str, default_port: int) → None, call:KNOWN_BIND_FIXES.get, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:compose_data["services"].values, call:service_config.get, call:compose_file.write_text, call:yaml.dump, call:logger.warning, call:logger.info, func:ensure_backend_network_in_compose(compose_path: str) → None, call:Path, call:compose_file.exists, call:compose_file.read_text, call:yaml.safe_load, call:get_backend_network_name, call:compose_data["services"].values, call:svc_config.get, call:compose_file.write_text, call:yaml.dump, call:logger.info, func:prepare_manifest_instance(session: AsyncSession, instance: ToolInstance, instance_dir: str, repo_path: str, env_vars: dict, extra_volumes: list, working_directory: str | None) → tuple[str, str, dict, str], call:session.get, call:dict, call:resolve_base, call:deep_merge, call:logger.warning, call:merge_with_config, call:manifest.pop, call:env_vars.update, call:extra_volumes.extend, call:compute_image_tag, call:subprocess.run, call:check.stdout.strip, call:compile_dockerfile, call:compile_entrypoint, call:logger.debug, call:len, call:asyncio.to_thread, call:logger.info, call:resolve_profile, call:gm.get, call:_get_repository_mount_name, call:os.path.basename, call:os.path.normpath, call:instance.name.lower, call:compile_compose, call:manifest.get, call:datetime.now, call:get_manifest_home_dir, raise:RuntimeError, func:create_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, data: "CreateInstanceRequest") → ToolInstance, call:session.get, call:uuid.UUID, call:validate_config_profile, call:uuid.uuid4, call:select(ToolInstance) .where(ToolInstance.workspace_id == workspace_id) .where(ToolInstance.tool_type_id == tool_type_id) .where, call:select(ToolInstance) .where(ToolInstance.repository_id == repo_id) .where(ToolInstance.tool_type_id == tool_type_id) .where, call:session.execute, call:len, call:result.scalars().all, call:ensure_instance_directory, call:os.path.join, call:find_free_port, call:f"headquarter/{instance_name}:latest".lower, call:asyncio.to_thread, call:logger.error, call:logger.info, call:_get_repository_mount_name, call:instance_name.lower, call:write_compose_file, call:dict, call:resolve_base, call:deep_merge, call:compute_image_tag, call:compile_compose, call:str, call:render_compose_template, call:ToolInstance, call:session.add, call:session.commit, call:session.refresh, call:publish_lifecycle_event, raise:ValueError, raise:RuntimeError, func:start_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID, data: "StartInstanceRequest | None") → dict, call:session.get, call:validate_config_profile, call:session.commit, call:os.path.exists, call:logger.info, call:dict, call:resolve_base, call:deep_merge, call:get_manifest_home_dir, call:manifest.get, call:user_cfg.get, call:logger.debug, call:os.path.dirname, call:resolve_profile, call:apply_resolved_profile, call:env_vars.update, call:config_files.update, call:extra_volumes.extend, call:resolve_git_mounts, call:profile_hints.get, call:len, call:logger.error, call:write_env_file, call:write_config_files, call:uuid.UUID, call:ssh_keys_to_mount.append, call:logger.warning, call:os.path.join, call:os.makedirs, call:_sanitize_filename, call:key_filenames.append, call:prepare_ssh_key_files, call:write_ssh_config, call:extra_volumes.append, call:_chown_staged_mounts, call:prepare_manifest_instance, call:write_compose_file, call:logger.exception, call:modify_compose_file, call:sanitize_compose_file, call:ensure_web_bind_address, call:ensure_container_name_in_compose, call:ensure_backend_network_in_compose, call:execute_compose_command, call:instance.name.lower, call:get_container_id, call:publish_lifecycle_event, call:wait_for_container_running, call:get_container_logs, call:resolved_manifest.get, call:apply_mount_permissions, call:home_dir.startswith, call:apply_ssh_permissions, call:probe_config.get, call:execute_probe, call:datetime.now().isoformat, call:"\n".join, call:get_container_status, call:start_tunnel, call:str, call:traceback.format_exc, raise:ValueError, raise:RuntimeError, func:restart_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID) → dict, call:session.get, call:stop_tunnel, call:logger.debug, call:logger.warning, call:os.path.exists, call:os.path.dirname, call:resolve_profile, call:apply_resolved_profile, call:write_env_file, call:logger.error, call:sanitize_compose_file, call:ensure_web_bind_address, call:ensure_container_name_in_compose, call:ensure_backend_network_in_compose, call:execute_compose_command, call:datetime.now, call:session.commit, call:start_tunnel, call:publish_lifecycle_event, raise:ValueError, raise:RuntimeError, func:delete_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID, force) → None, call:session.get, call:os.path.dirname, call:os.path.join, call:os.path.exists, call:check_dirty_state, call:stop_tunnel, call:logger.debug, call:logger.warning, call:execute_compose_command, call:shutil.rmtree, call:publish_lifecycle_event, call:session.delete, call:session.commit, raise:ValueError, raise:RuntimeError, func:recreate_instance_tunnel(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID) → dict, call:session.get, call:instance.name.lower, call:logger.info, call:get_container_id, call:logger.error, call:get_backend_network_name, call:is_container_on_network, call:connect_container_to_network, call:get_container_ip_on_network, call:logger.warning, call:recreate_tunnel, call:check_tunnel_health, call:health.get, call:subprocess.run, call:probe.stdout.strip, call:session.commit, call:logger.exception, raise:ValueError, raise:RuntimeError, func:stop_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID) → dict, call:session.get, call:stop_tunnel, call:logger.warning, call:os.path.exists, call:execute_compose_command, call:datetime.now, call:session.commit, call:publish_lifecycle_event, raise:ValueError, func:rename_tool_instance(session: AsyncSession, user_id: uuid.UUID, project_id: uuid.UUID, repo_id: uuid.UUID, instance_id: uuid.UUID, display_name: str) → ToolInstance, call:session.get, call:display_name.strip, call:session.commit, call:session.refresh, raise:ValueError | dep: asyncio, contextlib, logging, os, subprocess, uuid, datetime, fastapi, sqlalchemy, sqlalchemy.ext.asyncio, src.models, src.schemas.tool, src.services.git.clone, src.services.config.config_profile_resolver, src.services.docker, src.services.shared.tunnel, src.services.build.docker_build, src.services.build.manifest_compiler, src.services.shared.permission_fixer, src.services.shared.readiness_probe, src.services.shared.ssh_keys, src.services.instance.event_bus, src.services.instance.lifecycle_hooks, src.utils.git_url_parser, hashlib, yaml, pathlib, traceback, shutil, glob ## arch -Service-oriented architecture with Docker container orchestration, profile-based configuration resolution, and SSH tunneling for remote instance access. +Service-oriented architecture with a single facade service that coordinates multiple infrastructure concerns (container runtime, version control, networking, configuration, and authorization) through imperative orchestration. ## tags compose, call:compose, call:logger.warning, error, container, instance, git, mount ## symbols diff --git a/apps/api/src/services/tool/instance_service.py b/apps/api/src/services/tool/instance_service.py index 1ddc442..1771893 100644 --- a/apps/api/src/services/tool/instance_service.py +++ b/apps/api/src/services/tool/instance_service.py @@ -20,6 +20,7 @@ from src.models import ( SSHKey, ToolInstance, ToolType, + Workspace, ) from src.schemas.tool import CreateInstanceRequest, StartInstanceRequest from src.services.git.clone import check_dirty_state, clone_repository @@ -79,13 +80,21 @@ logger = logging.getLogger(__name__) _event_bus = InstanceEventBus() -def _get_repository_mount_name(repo: GitRepository) -> str: - """Return the directory name a standard git clone would create. +def _get_repository_mount_name( + repo: GitRepository, + workspace: "Workspace | None" = None, +) -> str: + """Return the directory name the repository should appear under in the container. - Prefers the repository name parsed from the remote URL so the container - mount matches what users expect from ``git clone``. Falls back to the - user-provided repository name when no remote URL is available. + When a workspace exists, the on-disk layout is + ``/data/working-copies/{workspace_id}/{repo_name}/``, so the repo-named + directory is already available as the basename of ``workspace.path``. For + legacy repo-only instances we fall back to parsing the remote URL like a + standard ``git clone`` would, then to the user-provided repository name. """ + if workspace is not None and workspace.path: + return os.path.basename(os.path.normpath(workspace.path)) + if repo.remote_url: base_url = extract_base_repo_url(repo.remote_url) or repo.remote_url name = base_url.rstrip("/").split("/")[-1] @@ -907,10 +916,14 @@ async def prepare_manifest_instance( # Use the repository name for the workspace/repo mount target, not the # directory name of a workspace/clone path (which may be "main" or similar). - # Prefer the name parsed from the remote URL so it matches a standard clone. + # Prefer the actual on-disk workspace directory name when a workspace is + # mounted, otherwise fall back to parsing the remote URL like git clone. repo = await session.get(GitRepository, instance.repository_id) + workspace: Workspace | None = None + if instance.workspace_id: + workspace = await session.get(Workspace, instance.workspace_id) repo_name = ( - _get_repository_mount_name(repo) + _get_repository_mount_name(repo, workspace) if repo else os.path.basename(os.path.normpath(repo_path)) ) @@ -1075,7 +1088,7 @@ async def create_tool_instance( ) home_dir = tool_type.home_directory or "/home/user" - mount_name = _get_repository_mount_name(repo) + mount_name = _get_repository_mount_name(repo, workspace) workspace_target = f"{home_dir}/{mount_name}" compose_content = f"""version: "3.8"\nservices: @@ -1112,14 +1125,15 @@ async def create_tool_instance( # Manifest templates use WORKSPACE_PATH; REPO_PATH is retained as a # deprecated alias for backward compatibility with older templates. + mount_name = _get_repository_mount_name(repo, workspace) variables = { "IMAGE_TAG": image_tag, "INSTANCE_NAME": instance_name.lower(), "INSTANCE_DIR": instance_dir, "WORKSPACE_PATH": repo_path, "REPO_PATH": repo_path, - "REPO_NAME": _get_repository_mount_name(repo), - "WORKSPACE_NAME": _get_repository_mount_name(repo), + "REPO_NAME": mount_name, + "WORKSPACE_NAME": mount_name, "SSH_PATH": "", "TOOL_PORT": tool_port, "EXTRA_ENV": {}, @@ -1140,7 +1154,7 @@ async def create_tool_instance( "TOOL_PORT": tool_port, "USER_ID": str(user_id), "PROJECT_ID": str(project_id), - "WORKSPACE_NAME": _get_repository_mount_name(repo), + "WORKSPACE_NAME": _get_repository_mount_name(repo, workspace), "HOME_DIRECTORY": tool_type.home_directory or "/home/user", } compose_content = render_compose_template(tool_type.compose_template, variables) diff --git a/apps/api/tests/.pi-map.index.md b/apps/api/tests/.pi-map.index.md index 8144090..d9123da 100644 --- a/apps/api/tests/.pi-map.index.md +++ b/apps/api/tests/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps/api/tests ## role -Provides shared testing infrastructure and reusable fixtures for the API test suite. +Provides shared test infrastructure and fixtures for the API application's pytest suite. ## parent index: apps/api/.pi-map.index.md map: apps/api/.pi-map.md diff --git a/apps/api/tests/.pi-map.md b/apps/api/tests/.pi-map.md index cca7de6..2f0ee3d 100644 --- a/apps/api/tests/.pi-map.md +++ b/apps/api/tests/.pi-map.md @@ -4,11 +4,11 @@ dir: apps/api/tests index: apps/api/tests/.pi-map.index.md ## role -Provides shared testing infrastructure and reusable fixtures for the API test suite. +Provides shared test infrastructure and fixtures for the API application's pytest suite. ## files - conftest.py | Provides shared pytest fixtures for testing a FastAPI application with async SQLite database, authenticated clients, and test data setup. | exp: func:test_client() → Generator[TestClient, None, None], call:create_async_engine, call:engine.begin, call:conn.run_sync, call:asyncio.run, call:init_db, call:async_sessionmaker, call:patch, call:TestClient, call:app.dependency_overrides.pop, call:engine.dispose, func:init_db(), call:engine.begin, call:conn.run_sync, func:override_get_db_session() → AsyncGenerator[AsyncSession, None], call:async_sessionmaker, func:db_session(test_client) → AsyncGenerator[AsyncSession, None], call:app.dependency_overrides.get, call:override_fn, call:gen.asend, call:gen.aclose, call:create_async_engine, call:engine.begin, call:conn.run_sync, call:async_sessionmaker, call:engine.dispose, func:authenticated_client(test_client) → Generator[TestClient, None, None], call:str, call:uuid.uuid4, call:Settings, call:app.dependency_overrides.get, call:override_fn, call:gen.asend, call:User, call:uuid.UUID, call:session.add, call:session.commit, call:gen.aclose, call:asyncio.run, call:create_test_user, call:create_session_cookie, call:test_client.cookies.set, func:create_test_user(), call:app.dependency_overrides.get, call:override_fn, call:gen.asend, call:User, call:uuid.UUID, call:session.add, call:session.commit, call:gen.aclose, func:test_project_and_repo(authenticated_client) → tuple[str, str], call:uuid.uuid4, call:Settings, call:authenticated_client.cookies.get, call:decode_session_cookie, call:uuid.UUID, call:asyncio.run, call:get_user_id, call:app.dependency_overrides.get, call:override_fn, call:gen.asend, call:Project, call:session.add, call:GitRepository, call:session.commit, call:gen.aclose, call:create_project_and_repo, call:str, raise:RuntimeError, func:get_user_id(), call:Settings, call:authenticated_client.cookies.get, call:decode_session_cookie, call:uuid.UUID, func:create_project_and_repo(), call:app.dependency_overrides.get, call:override_fn, call:gen.asend, call:Project, call:session.add, call:GitRepository, call:session.commit, call:gen.aclose, func:admin_client(test_client) → Generator[TestClient, None, None], call:str, call:uuid.uuid4, call:Settings, call:app.dependency_overrides.get, call:override_fn, call:gen.asend, call:User, call:uuid.UUID, call:session.add, call:session.commit, call:gen.aclose, call:asyncio.run, call:create_admin_user, call:create_session_cookie, call:test_client.cookies.set, func:create_admin_user(), call:app.dependency_overrides.get, call:override_fn, call:gen.asend, call:User, call:uuid.UUID, call:session.add, call:session.commit, call:gen.aclose | dep: asyncio, os, typing, unittest.mock, pytest, pytest_asyncio, fastapi.testclient, sqlalchemy.ext.asyncio, src.config, src.models.base, src.main, src.auth.dependencies, uuid, src.auth.session, src.models.user.user, src.models.project.project, src.models.project.git_repository, fastapi, sqlalchemy, aiosqlite, src.models, src.auth ## arch -Pytest fixture-based architecture using dependency injection, async context managers, and factory patterns for database/clients/test data setup. +Pytest fixture-based testing architecture with async SQLite database setup, authenticated HTTP client factories, and test data seeding utilities for FastAPI integration testing. ## tags call:app.dependency, call:create, overrides.get, call:override, fn, call:gen.asend, call:gen.aclose, user ## symbols diff --git a/apps/api/tests/unit/.pi-map.index.md b/apps/api/tests/unit/.pi-map.index.md index bef08b5..a649ca9 100644 --- a/apps/api/tests/unit/.pi-map.index.md +++ b/apps/api/tests/unit/.pi-map.index.md @@ -2,7 +2,7 @@ dir: apps/api/tests/unit ## role -Comprehensive unit test suite for the API application covering database migrations, configuration, Docker operations, Git services, file management, health monitoring, manifest compilation, and notification systems. +Comprehensive unit test suite for the API backend, covering core services, utilities, configuration, and infrastructure components. ## parent index: apps/api/tests/.pi-map.index.md map: apps/api/tests/.pi-map.md diff --git a/apps/api/tests/unit/.pi-map.md b/apps/api/tests/unit/.pi-map.md index 61944a7..0493777 100644 --- a/apps/api/tests/unit/.pi-map.md +++ b/apps/api/tests/unit/.pi-map.md @@ -4,7 +4,7 @@ dir: apps/api/tests/unit index: apps/api/tests/unit/.pi-map.index.md ## role -Comprehensive unit test suite for the API application covering database migrations, configuration, Docker operations, Git services, file management, health monitoring, manifest compilation, and notification systems. +Comprehensive unit test suite for the API backend, covering core services, utilities, configuration, and infrastructure components. ## files - __init__.py | Empty file with no functionality - test_alembic_migrations.py | Unit tests that verify Alembic database migrations are importable, have correct revision identifiers, and declare expected dependencies without requiring a live database. | exp: func:test_home_directory_migration_imports_and_rewrites() → None, call:Path, call:migration_path.exists, call:importlib.util.spec_from_file_location, call:importlib.util.module_from_spec, call:spec.loader.exec_module, call:callable, func:test_merge_migration_resolves_heads() → None, call:Path, call:migration_path.exists, call:importlib.util.spec_from_file_location, call:importlib.util.module_from_spec, call:spec.loader.exec_module, call:callable, func:test_remove_pi_agent_repo_mount_migration_imports() → None, call:Path, call:migration_path.exists, call:importlib.util.spec_from_file_location, call:importlib.util.module_from_spec, call:spec.loader.exec_module, call:callable | dep: importlib.util, pathlib, pytest, importlib @@ -19,7 +19,7 @@ Comprehensive unit test suite for the API application covering database migratio - test_git_url_parser.py | Tests for git URL parsing utilities that extract base repository URLs, validate clone URLs, and parse various git URL formats across GitHub, GitLab, and Bitbucket. | exp: class:TestExtractBaseRepoUrl, method:test_github_tree_url(self), call:extract_base_repo_url, method:test_github_blob_url(self), call:extract_base_repo_url, method:test_github_pull_url(self), call:extract_base_repo_url, method:test_github_issues_url(self), call:extract_base_repo_url, method:test_github_valid_url(self), call:extract_base_repo_url, method:test_github_url_with_query_params(self), call:extract_base_repo_url, method:test_gitlab_tree_url(self), call:extract_base_repo_url, method:test_gitlab_blob_url(self), call:extract_base_repo_url, method:test_gitlab_merge_request_url(self), call:extract_base_repo_url, method:test_gitlab_valid_url(self), call:extract_base_repo_url, method:test_bitbucket_src_url(self), call:extract_base_repo_url, method:test_bitbucket_valid_url(self), call:extract_base_repo_url, method:test_ssh_url(self), call:extract_base_repo_url, method:test_ssh_url_without_git_suffix(self), call:extract_base_repo_url, method:test_invalid_url(self), call:extract_base_repo_url, method:test_empty_url(self), call:extract_base_repo_url, class:TestIsValidCloneUrl, method:test_valid_ssh_url(self), call:is_valid_clone_url, method:test_valid_https_url(self), call:is_valid_clone_url, method:test_browser_url(self), call:is_valid_clone_url, method:test_url_without_git_suffix(self), call:is_valid_clone_url, method:test_invalid_url(self), call:is_valid_clone_url, class:TestParseGitUrl, method:test_valid_git_url(self), call:parse_git_url, method:test_browser_url(self), call:parse_git_url, method:test_invalid_url(self), call:parse_git_url, method:test_empty_url(self), call:parse_git_url, method:test_ssh_url(self), call:parse_git_url | dep: src.utils.git_url_parser, pytest - test_health_monitor.py | Unit tests for HealthMonitor state-transition logic covering container crash detection, tunnel failure detection, recovery detection, write deduplication, exception resilience, and start/stop lifecycle. | exp: func:event_bus() → InstanceEventBus, call:InstanceEventBus, call:bus._reset_for_testing, func:health_monitor(event_bus: InstanceEventBus) → HealthMonitor, call:HealthMonitor, func:_create_running_instance(db_session) → ToolInstance, call:User, call:uuid.uuid4, call:db_session.add, call:db_session.commit, call:ToolInstance, func:test_detects_container_crash(db_session, event_bus: InstanceEventBus, health_monitor: HealthMonitor) → None, call:_create_running_instance, call:events_captured.append, call:event_bus.subscribe, call:patch, call:health_monitor._check_instance, call:db_session.refresh, call:len, call:db_session.execute, call:select(HealthCheck).where, call:result.scalar_one, func:capture_event(payload: InstanceEventPayload) → None, call:events_captured.append, func:test_detects_tunnel_failure(db_session, event_bus: InstanceEventBus, health_monitor: HealthMonitor) → None, call:_create_running_instance, call:events_captured.append, call:event_bus.subscribe, call:patch, call:health_monitor._check_instance, call:db_session.refresh, call:len, call:db_session.execute, call:select(HealthCheck).where, call:result.scalar_one, func:capture_event(payload: InstanceEventPayload) → None, call:events_captured.append, func:test_detects_recovery(db_session, event_bus: InstanceEventBus, health_monitor: HealthMonitor) → None, call:_create_running_instance, call:db_session.commit, call:HealthSnapshot, call:events_captured.append, call:event_bus.subscribe, call:patch, call:health_monitor._check_instance, call:db_session.refresh, call:len, call:db_session.execute, call:select(HealthCheck).where, call:result.scalar_one, func:capture_event(payload: InstanceEventPayload) → None, call:events_captured.append, func:test_skips_writes_when_no_state_change(db_session, event_bus: InstanceEventBus, health_monitor: HealthMonitor) → None, call:_create_running_instance, call:HealthSnapshot, call:patch, call:health_monitor._check_instance, call:db_session.execute, call:select(HealthCheck).where, call:len, call:result.scalars().all, func:test_docker_exception_resilience(db_session, event_bus: InstanceEventBus, health_monitor: HealthMonitor) → None, call:_create_running_instance, call:events_captured.append, call:event_bus.subscribe, call:patch, call:RuntimeError, call:health_monitor._check_instance, call:db_session.execute, call:select(HealthCheck).where, call:result.scalar_one_or_none, func:capture_event(payload: InstanceEventPayload) → None, call:events_captured.append, func:test_monitor_start_stop(health_monitor: HealthMonitor) → None, call:health_monitor.start, call:task.done, call:health_monitor.stop, call:suppress, call:task.cancelled | dep: asyncio, uuid, contextlib, unittest.mock, pytest, sqlalchemy, src.models.system.health_check, src.models.tool.tool_instance, src.models.user.user, src.services.instance.event_bus, src.services.instance.health_monitor - test_home_path_expansion.py | Unit tests for home directory path expansion (~ and $HOME) in container paths and manifest home directory resolution. | exp: class:TestExpandContainerPath, method:test_tilde_slash_expands(self) → None, call:expand_container_path, method:test_tilde_alone_expands(self) → None, call:expand_container_path, method:test_dollar_home_slash_expands(self) → None, call:expand_container_path, method:test_dollar_home_alone_expands(self) → None, call:expand_container_path, method:test_absolute_path_unchanged(self) → None, call:expand_container_path, method:test_relative_path_unchanged(self) → None, call:expand_container_path, method:test_tilde_in_middle_unchanged(self) → None, call:expand_container_path, method:test_dollar_home_in_middle_unchanged(self) → None, call:expand_container_path, method:test_root_home(self) → None, call:expand_container_path, class:TestGetManifestHomeDir, method:test_with_user_block(self) → None, call:get_manifest_home_dir, method:test_without_user_block(self) → None, call:get_manifest_home_dir, method:test_with_empty_user_name(self) → None, call:get_manifest_home_dir, method:test_with_none_user_name(self) → None, call:get_manifest_home_dir | dep: pytest, src.services.config.config_profile_resolver, src.services.build.manifest_compiler -- test_instance_service.py | Unit tests for tool instance service functions including compose file modification, repository mount name resolution, and manifest instance preparation. | exp: class:TestModifyComposeFile, method:test_extra_volumes_expand_home_dir(self, tmp_path), call:compose_path.write_text, call:modify_compose_file, call:str, call:compose_path.read_text, method:test_working_directory_expands_home_dir(self, tmp_path), call:compose_path.write_text, call:modify_compose_file, call:str, call:compose_path.read_text, class:TestGetRepositoryMountName, method:test_prefers_remote_url_name_over_user_provided_name(self), call:MagicMock, call:_get_repository_mount_name, method:test_parses_browser_url_to_repo_name(self), call:MagicMock, call:_get_repository_mount_name, method:test_falls_back_to_repo_name_when_remote_url_missing(self), call:MagicMock, call:_get_repository_mount_name, method:test_falls_back_to_repo_name_for_unparseable_url(self), call:MagicMock, call:_get_repository_mount_name, class:Result, func:test_prepare_manifest_instance_uses_repo_name_not_workspace_dir(), call:MagicMock, call:AsyncMock, call:Result, call:prepare_manifest_instance, func:session_get(model, obj_id), func:fake_run(cmd), call:Result | dep: unittest.mock, pytest, src.services.tool.instance_service, subprocess, src.services.tool +- test_instance_service.py | Unit tests for tool instance service functions including compose file modification, repository mount name resolution, and manifest instance preparation | exp: class:TestModifyComposeFile, method:test_extra_volumes_expand_home_dir(self, tmp_path), call:compose_path.write_text, call:modify_compose_file, call:str, call:compose_path.read_text, method:test_working_directory_expands_home_dir(self, tmp_path), call:compose_path.write_text, call:modify_compose_file, call:str, call:compose_path.read_text, class:TestGetRepositoryMountName, method:test_prefers_remote_url_name_over_user_provided_name(self), call:MagicMock, call:_get_repository_mount_name, method:test_parses_browser_url_to_repo_name(self), call:MagicMock, call:_get_repository_mount_name, method:test_uses_workspace_path_basename_when_workspace_provided(self), call:MagicMock, call:_get_repository_mount_name, method:test_falls_back_to_repo_name_when_remote_url_missing(self), call:MagicMock, call:_get_repository_mount_name, method:test_falls_back_to_repo_name_for_unparseable_url(self), call:MagicMock, call:_get_repository_mount_name, class:Result, func:test_prepare_manifest_instance_uses_workspace_path_basename(), call:MagicMock, call:AsyncMock, call:Result, call:prepare_manifest_instance, func:session_get(model, obj_id), func:fake_run(cmd), call:Result | dep: unittest.mock, pytest, src.services.tool.instance_service, subprocess, src.services.tool - test_lifecycle_hooks.py | Unit tests for lifecycle hook helper functions that derive notification titles and determine whether events should trigger notifications. | exp: class:TestDeriveTitle, method:test_known_event_types(self) → None, call:_derive_title, method:test_unknown_event_type(self) → None, call:_derive_title, class:TestShouldNotify, method:test_error_events_are_notified(self) → None, call:_should_notify, method:test_health_changed_running_is_notified(self) → None, call:_should_notify, method:test_created_started_stopped_restarted_deleted_filtered(self) → None, call:_should_notify, method:test_health_changed_non_running_filtered(self) → None, call:_should_notify | dep: pytest, src.services.instance.lifecycle_hooks - test_manifest_compiler.py | Unit tests for a manifest compiler that generates Dockerfiles, docker-compose files, and entrypoint scripts from manifest configurations. | exp: class:TestGetManifestHomeDir, method:test_home_directory_in_manifest_wins(self) → None, call:get_manifest_home_dir, method:test_user_name_derives_home(self) → None, call:get_manifest_home_dir, method:test_root_fallback(self) → None, call:get_manifest_home_dir, method:test_empty_home_directory_falls_back(self) → None, call:get_manifest_home_dir, class:TestCompileDockerfileHomeDirectory, method:test_env_home_and_workdir_use_home_directory(self) → None, call:compile_dockerfile, method:test_workspace_symlink_created(self) → None, call:compile_dockerfile, method:test_runtime_workspace_not_baked_into_image(self) → None, call:compile_dockerfile, method:test_runtime_working_dir_overrides_home_workdir(self) → None, call:compile_dockerfile, method:test_working_dir_expands_tilde(self) → None, call:compile_dockerfile, class:TestCompileComposeHomeDirectory, method:test_default_repo_mount_synthesized(self) → None, call:compile_compose, method:test_explicit_repo_mount_preserved(self) → None, call:compile_compose, method:test_workspace_name_substituted_in_mount_target(self) → None, call:compile_compose, method:test_working_dir_expands_home(self) → None, call:compile_compose, class:TestCompileEntrypoint, method:test_entrypoint_creates_home_and_workspace(self) → None, call:compile_entrypoint, method:test_entrypoint_removes_stale_placeholder_directory(self) → None, call:compile_entrypoint, method:test_entrypoint_uses_root_then_sudo_for_workspace_symlink(self) → None, call:compile_entrypoint, call:entrypoint.find, method:test_entrypoint_fixes_mount_owners(self) → None, call:compile_entrypoint, func:test_compile_dockerfile_creates_config_dirs_for_user() → None, call:compile_dockerfile, func:test_compile_dockerfile_no_user_does_not_create_home() → None, call:compile_dockerfile, func:test_compile_dockerfile_uses_user_npm_prefix() → None, call:compile_dockerfile, func:test_compile_dockerfile_starts_as_root_and_drops_privileges() → None, call:compile_dockerfile, call:compile_entrypoint, func:test_compile_compose_runs_as_root() → None, call:compile_compose | dep: pytest, src.services.build.manifest_compiler - test_migration_metadata.py | Tests Alembic database migration files for correct table definitions and revision chain metadata | exp: func:test_initial_migration_defines_all_core_tables() → None, call:Path(__file__).resolve, call:spec_from_file_location, call:module_from_spec, call:spec.loader.exec_module, func:test_refresh_tokens_migration_has_expected_revision_chain() → None, call:Path(__file__).resolve, call:spec_from_file_location, call:module_from_spec, call:spec.loader.exec_module | dep: pytest, importlib.util, pathlib, pathlib.Path @@ -30,7 +30,7 @@ Comprehensive unit test suite for the API application covering database migratio - test_readiness_probe.py | Unit tests for a Docker container readiness probe service that executes commands via docker exec with retry logic. | exp: class:TestExecuteProbe, class:TestIntegrationScenarios | dep: unittest.mock, src.services.shared.readiness_probe, subprocess - test_ssh_keys.py | Unit tests for SSH key preparation functionality including file creation, permissions, ownership, and error handling | exp: class:TestPrepareSshKeyFiles | dep: os, pathlib, unittest.mock, pytest, src.services.shared.ssh_keys ## arch -Standard Python unittest/pytest structure with heavy mocking of external dependencies (Docker, Git subprocess, database), organized by service/domain with tests isolating individual components without requiring live infrastructure. +Classic unit testing pattern with pytest-style test files, heavy use of mocking for external dependencies (Docker, Git, subprocess, database), isolated per-module test coverage, and no shared test infrastructure or fixtures. ## tags test, url, call:notification, git, call:, home, merge, call:db ## symbols diff --git a/apps/api/tests/unit/test_instance_service.py b/apps/api/tests/unit/test_instance_service.py index cbe4c31..746c775 100644 --- a/apps/api/tests/unit/test_instance_service.py +++ b/apps/api/tests/unit/test_instance_service.py @@ -64,6 +64,14 @@ class TestGetRepositoryMountName: repo.remote_url = "https://github.com/acme/headquarter/tree/main" assert _get_repository_mount_name(repo) == "headquarter" + def test_uses_workspace_path_basename_when_workspace_provided(self): + repo = MagicMock() + repo.name = "src" + repo.remote_url = "git@git.example.com:acme/headquarter.git" + workspace = MagicMock() + workspace.path = "/data/working-copies/uuid/headquarter" + assert _get_repository_mount_name(repo, workspace) == "headquarter" + def test_falls_back_to_repo_name_when_remote_url_missing(self): repo = MagicMock() repo.name = "my-cool-repo" @@ -78,13 +86,19 @@ class TestGetRepositoryMountName: @pytest.mark.unit -async def test_prepare_manifest_instance_uses_repo_name_not_workspace_dir(): - """WORKSPACE_NAME must be the repository name, not the workspace path basename.""" +async def test_prepare_manifest_instance_uses_workspace_path_basename(): + """WORKSPACE_NAME must be the repo-named workspace directory, not workspace.name.""" repo = MagicMock() repo.name = "src" repo.remote_url = "git@git.example.com:acme/headquarter.git" repo.path = "/data/repos/main" + workspace = MagicMock() + workspace.id = "workspace-uuid" + workspace.repo_id = "repo-uuid" + workspace.name = "main" + workspace.path = "/data/working-copies/workspace-uuid/headquarter" + tool_type = MagicMock() tool_type.name = "pi-agent" tool_type.manifest_id = "manifest-uuid" @@ -105,7 +119,7 @@ async def test_prepare_manifest_instance_uses_repo_name_not_workspace_dir(): instance.tool_type_id = "tooltype-uuid" instance.port = 0 instance.selected_config_profile_id = None - instance.workspace_id = None + instance.workspace_id = "workspace-uuid" session = AsyncMock() @@ -116,6 +130,8 @@ async def test_prepare_manifest_instance_uses_repo_name_not_workspace_dir(): return manifest_def if model.__name__ == "GitRepository": return repo + if model.__name__ == "Workspace": + return workspace return None session.get.side_effect = session_get @@ -146,7 +162,7 @@ async def test_prepare_manifest_instance_uses_repo_name_not_workspace_dir(): session=session, instance=instance, instance_dir="/tmp/instance", - repo_path="/data/repos/main", + repo_path="/data/working-copies/workspace-uuid/headquarter", env_vars={}, extra_volumes=[], working_directory=None, @@ -155,4 +171,4 @@ async def test_prepare_manifest_instance_uses_repo_name_not_workspace_dir(): instance_service.subprocess.run = original_run assert "WORKSPACE_NAME: headquarter" in compose_content - assert "/data/repos/main:/home/user/headquarter" in compose_content + assert "/data/working-copies/workspace-uuid/headquarter:/home/user/headquarter" in compose_content diff --git a/openspec/.pi-map.index.md b/openspec/.pi-map.index.md index f37aa64..7cf8d6e 100644 --- a/openspec/.pi-map.index.md +++ b/openspec/.pi-map.index.md @@ -2,7 +2,7 @@ dir: openspec ## role -Defines a living documentation methodology and configuration framework for managing software requirements, specifications, and task tracking within a Docker-based coding agent platform. +Defines the OpenSpec methodology and project configuration for managing software requirements, specifications, and task tracking as living documentation within a Docker-based coding agent management platform. ## parent index: ./.pi-map.index.md map: ./.pi-map.md diff --git a/openspec/changes/.pi-map.index.md b/openspec/changes/.pi-map.index.md index 2a52be3..5abf126 100644 --- a/openspec/changes/.pi-map.index.md +++ b/openspec/changes/.pi-map.index.md @@ -2,7 +2,7 @@ dir: openspec/changes ## role -Manages change tracking and versioning for OpenAPI specification modifications +Tracks and manages specification changes and versioning history for the OpenAPI specification evolution. ## parent index: openspec/.pi-map.index.md map: openspec/.pi-map.md diff --git a/openspec/changes/fix-pi-container-mount-permissions/.pi-map.index.md b/openspec/changes/fix-pi-container-mount-permissions/.pi-map.index.md index 89d786d..36d0748 100644 --- a/openspec/changes/fix-pi-container-mount-permissions/.pi-map.index.md +++ b/openspec/changes/fix-pi-container-mount-permissions/.pi-map.index.md @@ -2,7 +2,7 @@ dir: openspec/changes/fix-pi-container-mount-permissions ## role -This is a change request/bug fix package for resolving filesystem permission issues with pi-agent container repository mounts and npm updates. +Documents a bug fix for container filesystem permission issues affecting Pi agent repository mounts and npm operations in containerized development environments. ## parent index: openspec/changes/.pi-map.index.md map: openspec/changes/.pi-map.md diff --git a/openspec/changes/fix-pi-container-mount-permissions/.pi-map.md b/openspec/changes/fix-pi-container-mount-permissions/.pi-map.md index 5f011cb..90663da 100644 --- a/openspec/changes/fix-pi-container-mount-permissions/.pi-map.md +++ b/openspec/changes/fix-pi-container-mount-permissions/.pi-map.md @@ -4,14 +4,14 @@ dir: openspec/changes/fix-pi-container-mount-permissions index: openspec/changes/fix-pi-container-mount-permissions/.pi-map.index.md ## role -This is a change request/bug fix package for resolving filesystem permission issues with pi-agent container repository mounts and npm updates. +Documents a bug fix for container filesystem permission issues affecting Pi agent repository mounts and npm operations in containerized development environments. ## files -- change.md | Documents a bug fix for pi-agent container repository mount paths and npm update permissions involving multiple code changes across manifest compilation, instance service, and database migrations. | dep: Alembic, manifest_compiler.py, instance_service.py, pytest, ruff, mypy, npm -- tasks.md | Tracks completion status of tasks for fixing a pi container's repository mount and npm update permissions in a software project +- change.md | Documents a bug fix for pi-agent container repository mounting paths and npm update permissions in a containerized development environment. | dep: Alembic, Docker, npm, git, pytest, ruff, mypy +- tasks.md | Task tracking document for fixing Pi container repository mount paths and npm update permission issues in a containerized agent system | dep: Alembic, manifest_compiler.py, instance_service.py, pytest, ruff, mypy ## arch -Issue-tracking documentation structure using lightweight markdown-based change management with separate change specification and task tracking files. +Documentation-only package using markdown-based change tracking (change.md for specifications, tasks.md for execution tracking) without active code artifacts. ## tags -npm, tasks, container, repository, mount, update, permissions, py +npm, agent, container, repository, paths, update, containerized, alembic ## symbols - ## workflows diff --git a/openspec/changes/fix-pi-container-mount-permissions/change.md b/openspec/changes/fix-pi-container-mount-permissions/change.md index f82709b..ea2175b 100644 --- a/openspec/changes/fix-pi-container-mount-permissions/change.md +++ b/openspec/changes/fix-pi-container-mount-permissions/change.md @@ -30,14 +30,16 @@ After implementing configurable tool container home directories, new `pi-agent` - Remove any stale literal `{{WORKSPACE_NAME}}` directory left over from older images at container startup. 3. Update `instance_service.py` to pass `REPO_NAME` and `WORKSPACE_NAME` into manifest compilation. 4. Remove the explicit repo mount from the built-in `pi-agent` manifest so the repo mount is synthesized by `compile_compose` rather than depending on tool config. Add a follow-up Alembic data migration that strips the `source_type: repo` mount from the manifest. -5. Add `_get_repository_mount_name()` helper to derive the workspace directory name from the repository's remote URL (matching standard `git clone` behavior) and fall back to the user-provided repository name. -6. Update unit tests for the new behavior. +5. Add `_get_repository_mount_name()` helper. When the instance is bound to a workspace, the helper returns the basename of `workspace.path`. For legacy repo-only instances it falls back to parsing the remote URL like `git clone` would, then to the user-provided repository name. +6. Switch workspace storage layout to `/data/working-copies/{workspace_id}/{repo_name}/` so `git clone` creates the repo-named directory naturally, making `workspace.path.basename` the correct container mount name. This replaces the previous `/data/working-copies/{repo_id}/{workspace_name}/` layout. +7. Update unit tests for the new behavior. ## Affected files - `apps/api/alembic/versions/2026_06_14_182955_fix_pi_agent_home_directory_mount.py` - `apps/api/alembic/versions/2026_06_15_090500_remove_pi_agent_explicit_repo_mount.py` - `apps/api/src/services/build/manifest_compiler.py` +- `apps/api/src/services/shared/workspace_manager.py` - `apps/api/src/services/tool/instance_service.py` - `apps/api/tests/unit/test_manifest_compiler.py` - `apps/api/tests/unit/test_instance_service.py` diff --git a/openspec/changes/fix-pi-container-mount-permissions/tasks.md b/openspec/changes/fix-pi-container-mount-permissions/tasks.md index e402c60..5229b94 100644 --- a/openspec/changes/fix-pi-container-mount-permissions/tasks.md +++ b/openspec/changes/fix-pi-container-mount-permissions/tasks.md @@ -6,6 +6,7 @@ - [x] Update instance_service.py to pass REPO_NAME/WORKSPACE_NAME - [x] Remove explicit repo mount from pi-agent manifest; synthesize mount in compile_compose - [x] Add _get_repository_mount_name() helper to derive workspace name from remote URL +- [x] Switch workspace storage layout to /data/working-copies/{workspace_id}/{repo_name}/ - [x] Update unit tests - [x] Run quality gates (pytest unit, ruff, mypy) - [ ] Commit and push