fix: pi container repo mount target and npm update permissions
- Add Alembic migration to update built-in pi-agent manifest:
* repo mount target from /workspace to ~/{{WORKSPACE_NAME}}
* keep /workspace as compatibility symlink via working_dir
* update startup chown target to $HOME/$WORKSPACE_NAME
- Pass REPO_NAME and WORKSPACE_NAME to compile_compose from instance_service
- Substitute {{WORKSPACE_NAME}} in manifest mount targets and expose it as
a container env var so the entrypoint can create the /workspace symlink
- Generate entrypoint workspace symlink from runtime WORKSPACE_NAME env var
- Install npm_global packages into {home_dir}/.npm-global with PATH so the
non-root container user can update global packages
- Update manifest compiler unit tests for the new behavior
Quality gates:
- pytest tests/unit: 207 passed
- ruff: clean on changed files
- mypy: clean on changed files
- alembic heads: single head
This commit is contained in:
Developer
@@ -2,7 +2,7 @@
|
||||
dir: apps/api/src
|
||||
|
||||
## role
|
||||
Core application bootstrap and infrastructure layer for the Headquarter API, providing configuration, database connectivity, logging, and FastAPI application initialization.
|
||||
Core API application package for the "Headquarter API" providing configuration, database infrastructure, logging, and FastAPI application initialization.
|
||||
## parent
|
||||
index: apps/api/.pi-map.index.md
|
||||
map: apps/api/.pi-map.md
|
||||
|
||||
@@ -4,7 +4,7 @@ dir: apps/api/src
|
||||
index: apps/api/src/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Core API application package for the Headquarter service, providing configuration, database connectivity, structured logging, and FastAPI application initialization.
|
||||
Core API application package for the "Headquarter API" providing configuration, database infrastructure, logging, and FastAPI application initialization.
|
||||
## files
|
||||
- __init__.py | Marks the directory as a Python package for the Headquarter API.
|
||||
- config.py | Defines application configuration settings with environment-based overrides using Pydantic, including database URLs, service domains, OAuth/Authentik integration, JWT/session settings, and computed properties for environment-specific behavior. | exp: class:Settings, func:build_database_url(user: str, password: str, host: str, port: int, database: str) → str | dep: pydantic, pydantic_settings
|
||||
@@ -12,7 +12,7 @@ Core API application package for the Headquarter service, providing configuratio
|
||||
- logging_config.py | Configures structured JSON logging with correlation ID injection, custom formatters, and HTTP request/exception middleware for a FastAPI application. | exp: class:CorrelationIdFilter, method:filter(self, record: logging.LogRecord) → bool, call:get_correlation_id, class:JSONFormatter, method:format(self, record: logging.LogRecord) → str, call:self.formatTime, call:record.getMessage, call:getattr, call:self.formatException, call:json.dumps, method:formatTime(self, record: logging.LogRecord, datefmt) → str, call:time.strftime, call:time.gmtime, class:RequestLoggingMiddleware, method:dispatch(self, request: Request, call_next: Callable) → Response, call:time.time, call:logger.info, call:call_next, call:int, call:logger.error, call:type, call:traceback.format_exc, class:ExceptionLoggingMiddleware, method:dispatch(self, request: Request, call_next: Callable) → Response, call:call_next, call:logger.critical, call:traceback.format_exc, func:configure_logging(level) → None, call:JSONFormatter, call:logging.StreamHandler, call:console_handler.setFormatter, call:console_handler.addFilter, call:CorrelationIdFilter, call:root_logger.setLevel, call:logging.getLogger("uvicorn").setLevel, call:logging.getLogger("uvicorn.access").setLevel, call:logging.getLogger("sqlalchemy.engine").setLevel, call:logger.info, call:logging.getLevelName | dep: json, logging, sys, time, traceback, collections.abc, fastapi, starlette.middleware.base, src.services.shared.correlation
|
||||
- main.py | Initializes and configures a FastAPI application for the "Headquarter API" with database setup, middleware, routing, and background services. | exp: func:_sanitize_validation_errors(errors), call:error.get, call:str, call:ctx.items, call:isinstance, call:type, call:sanitized.append, func:validation_exception_handler(request: Request, exc: RequestValidationError), call:exc.errors, call:logger.warning, call:_sanitize_validation_errors, call:JSONResponse, func:on_startup(), call:logger.info, call:init_database, call:logger.error, call:sys.exit, call:_health_monitor.start, call:seed_builtin_tool_types, func:on_shutdown(), call:logger.info, call:_health_monitor.stop | dep: logging, os, fastapi, fastapi.exceptions, fastapi.middleware.cors, fastapi.responses, fastapi.staticfiles, src.api.config, src.api.project, src.api.system, src.api.tool, src.api.user, src.api.workspace, src.config, src.models, src.database, src.logging_config, src.seeds.builtin_tool_types, src.services.instance, src.services.shared, sys, src.api.*
|
||||
## arch
|
||||
Layered architecture with environment-based Pydantic configuration, async SQLAlchemy with Alembic migrations, structured JSON logging with correlation IDs, and FastAPI middleware/routing pattern with background services integration.
|
||||
Layered architecture with Pydantic-based config management, async SQLAlchemy with Alembic migrations, structured JSON logging with correlation IDs, and FastAPI middleware/routing pattern.
|
||||
## tags
|
||||
src, database, logging, call:logger.info, api, middleware, fastapi, filter
|
||||
## symbols
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
dir: apps/api/src/services
|
||||
|
||||
## role
|
||||
Marks the services directory as a Python package for organizing business logic and service-layer abstractions.
|
||||
Provides business logic and service layer abstractions for the API application.
|
||||
## parent
|
||||
index: apps/api/src/.pi-map.index.md
|
||||
map: apps/api/src/.pi-map.md
|
||||
|
||||
@@ -4,11 +4,11 @@ dir: apps/api/src/services
|
||||
index: apps/api/src/services/.pi-map.index.md
|
||||
|
||||
## role
|
||||
This directory serves as a Python package namespace for organizing service-layer modules in the API application.
|
||||
Provides business logic and service layer abstractions for the API application.
|
||||
## files
|
||||
- __init__.py | Empty file with no functionality
|
||||
## arch
|
||||
Standard Python package structure using __init__.py to define an importable module directory, following conventional layered architecture patterns.
|
||||
Minimal or placeholder package structure with no implemented services yet, following standard Python package conventions.
|
||||
## tags
|
||||
init, empty, functionality
|
||||
## symbols
|
||||
|
||||
@@ -143,13 +143,6 @@ def compile_dockerfile(manifest: dict) -> str:
|
||||
lines.append(" rm -rf /var/lib/apt/lists/*")
|
||||
lines.append("")
|
||||
|
||||
# NPM global packages
|
||||
npm_packages = manifest.get("packages", {}).get("npm_global", [])
|
||||
if npm_packages:
|
||||
pkg_list = " ".join(shlex.quote(p) for p in npm_packages)
|
||||
lines.append(f"RUN npm install -g {pkg_list}")
|
||||
lines.append("")
|
||||
|
||||
# Pip packages
|
||||
pip_packages = manifest.get("packages", {}).get("pip", [])
|
||||
if pip_packages:
|
||||
@@ -161,6 +154,7 @@ def compile_dockerfile(manifest: dict) -> str:
|
||||
user = manifest.get("user")
|
||||
home_dir = get_manifest_home_dir(manifest)
|
||||
workspace_name = manifest.get("workspace_name", "{{WORKSPACE_NAME}}")
|
||||
npm_prefix = ""
|
||||
if user:
|
||||
name = user["name"]
|
||||
uid = user["uid"]
|
||||
@@ -195,6 +189,20 @@ def compile_dockerfile(manifest: dict) -> str:
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
# NPM global packages: install into a user-writable prefix so the
|
||||
# container user can update global packages without touching
|
||||
# /usr/lib/node_modules (which is owned by root).
|
||||
npm_packages = manifest.get("packages", {}).get("npm_global", [])
|
||||
if npm_packages:
|
||||
pkg_list = " ".join(shlex.quote(p) for p in npm_packages)
|
||||
npm_prefix = f"{home_dir}/.npm-global"
|
||||
lines.append(
|
||||
f"RUN mkdir -p {npm_prefix} && "
|
||||
f"npm install -g --prefix {npm_prefix} {pkg_list}"
|
||||
)
|
||||
lines.append(f"ENV PATH={npm_prefix}/bin:$PATH")
|
||||
lines.append("")
|
||||
|
||||
# Build scripts
|
||||
build_scripts = manifest.get("scripts", {}).get("build", [])
|
||||
for script in build_scripts:
|
||||
@@ -287,8 +295,6 @@ def compile_entrypoint(manifest: dict) -> str:
|
||||
|
||||
user = manifest.get("user")
|
||||
home_dir = get_manifest_home_dir(manifest)
|
||||
workspace_name = manifest.get("workspace_name", "{{WORKSPACE_NAME}}")
|
||||
workspace_target = f"{home_dir}/{workspace_name}"
|
||||
|
||||
# Permission fixer preamble: run as root when possible, else fall back to
|
||||
# passwordless sudo configured in the Dockerfile.
|
||||
@@ -315,7 +321,8 @@ def compile_entrypoint(manifest: dict) -> str:
|
||||
lines.append(f"USER_UID='{uid}'")
|
||||
lines.append(f"USER_GID='{gid}'")
|
||||
lines.append(f"HOME_DIR='{home_dir}'")
|
||||
lines.append(f"WORKSPACE_TARGET='{workspace_target}'")
|
||||
lines.append('WORKSPACE_NAME="${WORKSPACE_NAME:-workspace}"')
|
||||
lines.append('WORKSPACE_TARGET="${HOME_DIR}/${WORKSPACE_NAME}"')
|
||||
lines.append("")
|
||||
lines.append("fix_owner() {")
|
||||
lines.append(" local path=\"$1\"")
|
||||
@@ -343,8 +350,13 @@ def compile_entrypoint(manifest: dict) -> str:
|
||||
target = mount.get("target")
|
||||
if not target:
|
||||
continue
|
||||
# Expand any ~/$HOME placeholders in the mount target.
|
||||
expanded = target.replace("~", home_dir).replace("$HOME", home_dir)
|
||||
# Expand any ~/$HOME placeholders and the runtime workspace name
|
||||
# in the mount target so ownership is fixed at container startup.
|
||||
expanded = (
|
||||
target.replace("~", home_dir)
|
||||
.replace("$HOME", home_dir)
|
||||
.replace("{{WORKSPACE_NAME}}", "${WORKSPACE_NAME}")
|
||||
)
|
||||
if expanded.startswith(home_dir) and not mount.get("readonly", False):
|
||||
lines.append(f'fix_owner "{expanded}"')
|
||||
lines.append("")
|
||||
@@ -409,6 +421,12 @@ def compile_compose(manifest: dict, variables: dict[str, Any]) -> str:
|
||||
if env:
|
||||
service["environment"] = dict(env)
|
||||
|
||||
# Expose the workspace/repo name so the entrypoint can finalize the
|
||||
# /workspace compatibility symlink at container startup.
|
||||
if "environment" not in service:
|
||||
service["environment"] = {}
|
||||
service["environment"]["WORKSPACE_NAME"] = workspace_name
|
||||
|
||||
# Merge extra env from config
|
||||
extra_env = variables.get("EXTRA_ENV", {})
|
||||
if extra_env:
|
||||
@@ -426,6 +444,7 @@ def compile_compose(manifest: dict, variables: dict[str, Any]) -> str:
|
||||
if mount.get("source_type") == "repo":
|
||||
has_explicit_repo_mount = True
|
||||
target = expand_container_path(mount["target"], home_dir)
|
||||
target = target.replace("{{WORKSPACE_NAME}}", workspace_name)
|
||||
readonly = ":ro" if mount.get("readonly", False) else ""
|
||||
volumes.append(f"{source}:{target}{readonly}")
|
||||
|
||||
|
||||
@@ -2,12 +2,14 @@
|
||||
dir: apps/api/src/services/tool
|
||||
|
||||
## role
|
||||
Orchestrates end-to-end deployment and runtime management of development tool instances via containerized environments with remote access capabilities.
|
||||
Provides containerized execution environment for tools by managing Docker instances, git repositories, and compose orchestration.
|
||||
## parent
|
||||
index: apps/api/src/services/.pi-map.index.md
|
||||
map: apps/api/src/services/.pi-map.md
|
||||
## children
|
||||
-
|
||||
- apps/api/src/services/tool/.ruff_cache
|
||||
index: apps/api/src/services/tool/.ruff_cache/.pi-map.index.md
|
||||
map: apps/api/src/services/tool/.ruff_cache/.pi-map.md
|
||||
## files
|
||||
- instance_service.py
|
||||
## links
|
||||
@@ -16,5 +18,7 @@ map: apps/api/src/services/tool/.pi-map.md
|
||||
## workflows
|
||||
- change tool behavior
|
||||
read: instance_service.py
|
||||
- explore tool subdirectories
|
||||
index: apps/api/src/services/tool/.ruff_cache/.pi-map.index.md
|
||||
## dirty
|
||||
-
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -880,12 +880,15 @@ async def prepare_manifest_instance(
|
||||
# The actual resolution happens in resolve_git_mounts; we store placeholder
|
||||
git_mount_vars[f"GIT_MOUNT_{ref}"] = ""
|
||||
|
||||
repo_name = os.path.basename(os.path.normpath(repo_path))
|
||||
variables = {
|
||||
"IMAGE_TAG": image_tag,
|
||||
"INSTANCE_NAME": instance.name.lower(),
|
||||
"INSTANCE_DIR": instance_dir,
|
||||
"WORKSPACE_PATH": repo_path,
|
||||
"REPO_PATH": repo_path,
|
||||
"REPO_NAME": repo_name,
|
||||
"WORKSPACE_NAME": repo_name,
|
||||
"SSH_PATH": ssh_path,
|
||||
"TOOL_PORT": instance.port or 0,
|
||||
"EXTRA_ENV": env_vars,
|
||||
@@ -1070,12 +1073,15 @@ async def create_tool_instance(
|
||||
|
||||
# Manifest templates use WORKSPACE_PATH; REPO_PATH is retained as a
|
||||
# deprecated alias for backward compatibility with older templates.
|
||||
repo_name = os.path.basename(os.path.normpath(repo_path))
|
||||
variables = {
|
||||
"IMAGE_TAG": image_tag,
|
||||
"INSTANCE_NAME": instance_name.lower(),
|
||||
"INSTANCE_DIR": instance_dir,
|
||||
"WORKSPACE_PATH": repo_path,
|
||||
"REPO_PATH": repo_path,
|
||||
"REPO_NAME": repo_name,
|
||||
"WORKSPACE_NAME": repo_name,
|
||||
"SSH_PATH": "",
|
||||
"TOOL_PORT": tool_port,
|
||||
"EXTRA_ENV": {},
|
||||
|
||||
Reference in New Issue
Block a user