Add generated .pi-map.md and .pi-map.index.md files across the repository
so the project navigation maps are shared and versioned. These artifacts
are maintained by project_map_init/patch/validate and must be kept in
sync with source edits.
Note: .cache/ remains ignored (added in previous commit).
Root cause: manifest-based Dockerfile created the home directory and
chowned only the home root. Files/directories copied from /etc/skel by
useradd -m (or created later by root) remained root-owned, so apps like
ranger failed when writing to ~/.config.
Changes:
- manifest_compiler.py: recursive chown of the home directory after
useradd so /etc/skel contents are owned by the container user
- Pre-create .config, .local/share, .cache and chown them to the user
so first-run apps have writable directories immediately
- Add unit test verifying the Dockerfile emits the expected user/home
setup and config directory creation
Quality gates: py_compile all backend files pass, test file compiles,
tsc --noEmit pass, npm run build pass, 82/82 web tests pass
Note: pytest not available in this shell; backend unit test was not
executed but follows existing project conventions.
SSH key mounting was broken because:
1. Each selected key was mounted to a separate source dir but all targeted
the same ~/.ssh path in the container, causing Docker Compose's
last-mount-wins behavior
2. All keys were named id_ed25519, so they'd overwrite each other
Changes:
- ssh_keys.py: add key_filename param to prepare_ssh_key_files for unique
key names; add write_ssh_config for combined multi-key config
- tool_instances.py: collect all selected keys into a single ~/.ssh mount
with sanitized unique filenames (id_ed25519_<name>); generate combined
SSH config with all IdentityFile entries
- tests: add os.makedirs mock for SSH permission tests
Quality gates: pytest (19 passed, 1 skipped)
Fixes NameError: ToolInstance not defined at runtime because
type annotations are evaluated at class definition time.
Deferring annotation evaluation with __future__ annotations
keeps TYPE_CHECKING imports from causing runtime crashes.
Also includes ruff formatting cleanup on workspace-related files.
FastAPI matches routes in declaration order. The DELETE /notifications
endpoint (bulk clear) was registered AFTER DELETE /notifications/{id},
so the path parameter route intercepted all requests to the bulk route,
causing a 422 UUID validation error instead of hitting clear_all.
Moved clear_all_notifications above dismiss_notification in the router.
Added regression test to verify route order.
Quality gates: pytest (22 passed)
Notification filtering:
- lifecycle_hooks.py: only instance.error and instance.health_changed
with status=running generate notifications. All other lifecycle events
(created, started, stopped, restarted, deleted) are filtered out.
- health_monitor.py: only error and unhealthy states generate notifications.
Running/recovered state no longer creates info notifications.
- _derive_title now maps instance.health_changed to "Container ready".
Clear-all button:
- Added dismiss_all() to NotificationService
- Added DELETE /notifications endpoint for bulk dismiss
- Frontend: clearAllNotifications API, clearAll in notification context,
"Clear all" button in notification drawer alongside "Mark all as read"
- Added CSS for .notification-clear-all with danger hover state
- Updated notification-center tests
Quality gates: pytest (21 passed), vitest (11 passed)
- apply_mount_permissions now skips mounts with readonly=true to avoid
'Read-only file system' warnings on post-start chown/chmod
- Removed the ssh_keys mount from the pi-agent manifest definition;
instance-level SSH key mounting now handles this exclusively
- Added unit test for read-only mount skipping
Quality gates: pytest (15 passed)
- start_instance now deep-merges manifest with base definition before extracting user.uid/user.gid
- The user config is typically defined in the base image (ubuntu-24.04-dev), not the extending manifest
- Add debug logging to verify resolved uid/gid/home_dir
- Add logging to prepare_ssh_key_files for chown success/failure visibility
- Log current process uid when chown fails to diagnose permission issues
Quality gates: pytest 239 passed (6 pre-existing failures), tsc --noEmit clean
- Extend prepare_ssh_key_files() with optional uid/gid parameters
- Call os.chown on created files when uid/gid are provided
- Gracefully handle PermissionError if API process is not root
- In start_instance, extract container user UID/GID from manifest
- Pass container UID/GID when preparing instance-level SSH key mounts
- Legacy clone-mode SSH keys continue to use root (0,0)
- Add unit tests for prepare_ssh_key_files ownership logic
- Keep apply_ssh_permissions() as fallback for cases where host chown fails
Quality gates: pytest 239 passed (6 pre-existing failures), tsc --noEmit clean
- Replace apply_ssh_permissions internals with _exec_and_log for full visibility
- Log every docker exec command, stdout, and stderr at DEBUG level
- After chown/chmod, run ls -la and stat to verify final state
- Log verified state at INFO level so users can see exactly what happened
- Update tests to mock subprocess.run instead of _run_in_container
Quality gates: pytest 236 passed (6 pre-existing), tsc --noEmit clean
- Add POST /config-profiles/validate-git-url endpoint:
- Parses URL using existing parse_git_url utility
- Suggests corrected URL for browser URLs
- Runs git ls-remote --heads to verify reachability
- Lists available branches from remote
- Supports SSH key for private repos
- Returns structured response: valid, suggested_url, branches,
default_branch, error, error_code
- Update frontend GitMountEditor:
- Add Check button next to URL field with loading state
- Show validation result: valid (green), suggestion (yellow),
invalid (red)
- Suggestion includes Use this button to apply corrected URL
- Branch field becomes dropdown when URL is validated,
populated with remote branches
- Mappings section disabled until URL is validated
- Shows hint: Validate the URL first
- Quality gates: pytest (218 passed, 6 pre-existing),
tsc --noEmit (clean)
- Add notifications table with Alembic migration
- Notification model with user-scoped indexing and partial index on unread
- NotificationService singleton with create/list/count/mark-read/dismiss
- FastAPI router: GET /notifications, GET /unread, PATCH /{id}/read,
POST /mark-all-read, DELETE /{id}
- Mute categories filtering from UserConfig
- 13 unit tests for NotificationService
- 10 integration tests for API endpoints
- Updated test_models.py with new table registration
Quality gates: pytest 23 new passed, ruff clean
The previous sorting fix exposed a deeper bug: ResolvedMount always
mounted its staging directory as a single bind mount. When a config
profile mount targeted /workspace/x/y and contained a single file
z.json, the staging directory (containing only z.json) replaced the
ENTIRE /workspace/x/y directory, hiding all sibling files from git
repo mounts.
- Change apply_resolved_profile to mount each file individually:
- source: staging_dir/relative_path
- target: expanded_target/relative_path
- Sibling files from other mounts are preserved.
- Empty mounts produce no volume entries.
- Keep volume sorting (parent paths before child paths) which is
still necessary for directory mounts and ensures parent dirs exist
before file mounts inside them.
- Add 4 unit tests for file-level mount behavior.
Quality gates: pytest (218 passed, 6 pre-existing), tsc --noEmit (clean)
- Add instance_events and health_checks tables with Alembic migration
- InstanceEventBus: typed pub/sub singleton with wildcard support
- HealthMonitor: async background loop polling containers every 15s
- SSE endpoint GET /events/stream with auth and connection limits
- Lifecycle hooks in tool_instances.py (create/start/stop/restart/delete)
- Structured JSON logging with correlation IDs
- 15 new unit tests (EventBus, HealthMonitor, MonitoringModels)
Quality gates: pytest 15 new passed, ruff clean
- Add expand_container_path() helper that resolves ~/ and $HOME/ prefixes
- Add get_manifest_home_dir() to compute /home/{user.name} or /root from manifest
- Set ENV HOME=... and ENV USER=... in generated Dockerfile for runtime compatibility
- Pass home_dir through instance creation and startup pipeline
- Expand mount targets in apply_resolved_profile() for regular profile mounts
- Expand mapping targets in _resolve_git_mount_mappings() for git mounts
- Expand working_directory and volume targets in _modify_compose_file()
- Update _prepare_manifest_instance to return home_dir alongside image tag
- Fetch tool_type early in start_instance to determine home_dir before profile application
Quality gates: pytest 188 passed, frontend typecheck clean
Addresses: home-path-expansion
- Add mappings array support to git_mount entries
- Clone repository once per git_mount entry, mount multiple subdirectories
- Normalize legacy source_path+target_path to mappings on read
- Update _merge_git_mounts to dedup by (remote_url, branch) and concatenate mappings
- Add _normalize_git_mount, _clone_git_repo, _resolve_git_mount_mappings helpers
- Update GitMountItem Pydantic model with GitMountMapping and model_validator
- Update frontend GitMountEditor component with mappings UI
- Auto-convert legacy git mount entries to mappings format on load
- Add 15 backend unit tests for normalization, resolution, and glob expansion
- Update existing config profile resolver tests for new merge behavior
Quality gates: pytest 167 passed, frontend typecheck clean
Addresses: config-profile-multi-repo-mounts
- Add test_tool_instances_legacy.py with 8 unit tests:
- dockerfile definition type builds from template
- dockerfile build failure raises HTTP 500
- compose definition type renders template
- manifest compiler is NOT called for legacy types
- start_instance legacy/compose/dockerfile types all skip manifest flow
- start_instance manifest type correctly invokes compiler
- Mark T3.2 and T3.3 tasks complete in OpenSpec
- Add openspec/docs/tool-workshop-guide.md with user guide covering
definition types, manifest creation workflow, base definitions,
migration path, and permissions
- Add ToolDefinitionManifest model with base image versioning
- Add manifest compiler: Dockerfile + Compose generation from JSON manifests
- Add permission fixer: post-start chown/chmod for mount policies
- Add tool definition CRUD API with live compile preview endpoint
- Integrate manifest-based startup flow in start_instance
- Add Alembic migration with data conversion for pi-agent
- Add 48 unit tests for manifest compiler, permission fixer, docker service
- Keep backward compatibility with legacy dockerfile_template/compose_template
Migration: applied successfully. Pi-agent converted to manifest.
Quality gates: pytest (146 passed, 4 pre-existing unrelated failures)
- get_container_id() and get_container_name() now lowercase the
instance name before passing to docker ps --filter, because
Docker container names are lowercase internally and the filter
is case-sensitive. This caused container_id to never be captured
when instance.name contained uppercase chars (e.g. 'Headquarter'),
breaking terminal WebSocket connections.
- Also guard proc.stdout being None in start_cloudflared_tunnel().
- Add unit tests for get_container_id and get_container_name.
Quality gates: pytest (14 passed), python clean
- _checkout_branch now returns bool and falls back gracefully on failure
- Glob warning message includes matched file count
- Fix database model comment to reference remote_url
- Update tests for new branch checkout behavior
All 51 tests pass
- Change git mount schema from repo_id to remote_url
- Remove database lookups for git mount resolution
- Clone directly from URL at instance startup
- Simplify frontend UI to text input for Git URL
- Fix route ordering in git_repositories.py to prevent 422 errors
- Update all tests to use remote_url field
Breaking change: Git mounts now use remote_url instead of repo_id
- Add ConfigProfile and ConfigProfileInclude data models with migrations
- Implement profile resolver service with ordered includes and merge rules
- Add profile CRUD API with validation, compatibility, and cycle detection
- Add instance API plumbing for profile selection on create/start/restart
- Add resolved profile preview and default resolution APIs
- Add frontend config profile API client and management UI
- Add launch/restart profile selection UI
- Add backend integration and unit tests (31 passing)
OpenSpec: add-config-profiles
Quality gates: ruff, TypeScript compile, 31 tests passing
- Create orphan branches when HEAD does not exist yet\n- Use current branch as the default branch base in the toolbar\n- Keep push disabled for remote-less repos\n\nQuality gates: vitest repositories-settings-tab (passed); api pytest blocked by missing fastapi in environment
- Create normal working clones for remote repositories
- Initialize blank repositories with a main branch
- Align pull and branch helpers with unborn HEAD handling
- Gate fetch/pull on repositories with a remote
Quality gates: vitest repositories-settings-tab (passed); api pytest blocked by missing fastapi in environment
Backend tests:
- Unit tests for docker_build service (successful/failed builds, context, paths)
- Unit tests for readiness_probe service (success, timeout, retries, edge cases)
- Integration tests for config_folders API (CRUD + project overrides)
- Integration tests for tool_types API with new fields
- Integration tests for tool_configs API with new fields
Frontend tests:
- ToolWorkshopPage component tests (all 3 tabs, create/edit/delete)
- API client tests for tool_types and config_folders
Fixes:
- Add field_validator import to tool_configs.py
- Add JSON import to tool_config model
- Update frontend test button names to match UI (Create Tool Type, Add Config, Create Folder)
Quality gates: backend unit tests passing (23/23)
- Update test_config.py: account for conftest.py DATABASE_URL override
- Update test_migration_metadata.py: correct alembic path resolution
(alembic/ is at project root, not under src/)
Task 2.5: Remove duplicate fixtures from integration tests
- test_auth_api.py, test_auth_services.py, test_models.py
- test_projects_api.py, test_seed.py, test_users_api.py
- Fix npytest typos in all test files
Task 3.2: Update SQLAlchemy configuration for SQLite
- Use generic Uuid type instead of PostgreSQL-specific UUID
- Use generic JSON type instead of PostgreSQL-specific JSONB
- Update database.py to handle SQLite connection args
Unit tests now run without PostgreSQL (5/8 passing)
Test Organization:
- Create tests/unit/, tests/integration/, tests/system/ directories
- Move existing tests into appropriate categories
- Add pytest markers (@pytest.mark.unit, @pytest.mark.integration)
Shared Fixtures:
- Create conftest.py with SQLite engine (for unit tests)
- Add PostgreSQL session fixture with transaction rollback
- Add TestClient fixture for API tests
Configuration:
- Update pyproject.toml with asyncio_mode=auto
- Add test markers and default addopts
- Add aiosqlite dependency for SQLite support
E2E Testing:
- Initialize Playwright in e2e/ directory
- Add playwright.config.ts
- Create login flow E2E test
Build:
- Add test-unit, test-integration, test-system, test-e2e to Makefile
- Update test target to run all categories
- Add testing documentation to README
Note: Some tests have import issues due to missing python-jose
package in dev environment. This needs to be addressed separately.
Developer
alex
miguel