## Context Current `dev` has all behavioral features from the overwritten `main` merge, but the code structure is pre-refactor: - Monolithic `api/tool_instances.py` (~3000 lines) - Monolithic `api/config_profiles.py` (~1000 lines) - Monolithic `services/docker.py` - No `schemas/` directory - Flat frontend component structure with inconsistent naming The `b6f89f9` merge from `main` had a clean refactoring that we need to redo, but adapted to our current reality. ## Goals / Non-Goals **Goals:** - Extract Pydantic schemas from API routers into `src/schemas/` - Split `services/docker.py` into `services/docker/` package - Extract instance lifecycle logic from `api/tool_instances.py` into `services/instance_lifecycle.py` - Extract config profile business logic from `api/config_profiles.py` into `services/config_profiles.py` - Add `get_current_user` auth dependency and migrate routers that need the full user object - Reorganize frontend components into `features/` directories - Standardize frontend API file naming to kebab-case - Standardize frontend page naming to `*Page.tsx` **Non-Goals:** - Changing any API request/response shapes - Changing any database schemas - Adding new features - Modifying frontend component behavior or styling - Converting `ConfigProfile` mounts from JSON to relation tables (out of scope — would require migration) ## Decisions ### 0. Submodule Rule: Max 5–10 Files Per Directory **Decision:** Every directory that functions as a Python module must contain at most 5–10 `.py` files. When a module grows beyond this, split it into a package with submodules. **Rationale:** Prevents monolithic directories, makes navigation predictable, and keeps cognitive load bounded. ### 1. Schema Extraction: Domain Subpackages **Decision:** Extract Pydantic models into `schemas/` subpackages by domain: - `schemas/tool/` — tool_type.py, tool_instance.py - `schemas/config/` — config_profile.py - `schemas/user/` — user.py, user_config.py - `schemas/project/` — project.py, git_repository.py, ssh_key.py - `schemas/system/` — health.py **Rationale:** Keeps schemas close to their domain. Each subpackage has ≤5 files. ### 2. API Router Subpackages **Decision:** Split `api/` into domain subpackages: - `api/tool/` — tool_instances.py, tool_types.py, tool_definitions.py, tool_types_validation.py, sessions.py - `api/config/` — config_profiles.py, user_config.py - `api/workspace/` — workspaces.py, workspace_files.py, workspace_git.py, workspace_instances.py - `api/user/` — users.py, auth.py, ssh_keys.py - `api/project/` — projects.py, git_repositories.py - `api/system/` — health.py, events.py, notifications.py, dashboard.py, terminal.py, instance_proxy.py **Rationale:** `api/` currently has ~22 files. Splitting into 6 subpackages keeps each at 2–6 files. ### 3. Service Subpackages **Decision:** Split `services/` into subpackages: - `services/docker/` — compose.py, container.py, config_staging.py, tunnel.py, __init__.py - `services/instance/` — instance_lifecycle.py, lifecycle_hooks.py, health_monitor.py, event_bus.py - `services/config/` — config_profile_resolver.py, config_profiles.py - `services/git/` — clone.py, git_operations.py, git_service.py - `services/build/` — docker_build.py, manifest_compiler.py - `services/terminal/` — terminal_manager.py, terminal_session.py - `services/shared/` — tunnel.py, notification_service.py, file_service.py, permission_fixer.py, readiness_probe.py, ssh_keys.py, workspace_manager.py, correlation.py **Rationale:** `services/` currently has ~20 files. Subpackages keep each at ≤8 files. ### 4. Model Subpackages **Decision:** Split `models/` into subpackages: - `models/tool/` — tool_type.py, tool_instance.py, tool_definition_manifest.py - `models/config/` — config_profile.py, config_include.py, config_mount.py - `models/user/` — user.py, user_config.py, ssh_key.py - `models/project/` — project.py, git_repository.py, workspace.py - `models/system/` — health_check.py, notification.py, instance_event.py, terminal_session.py - `models/base.py` stays at root **Rationale:** `models/` currently has ~15 files. Subpackages keep each at ≤4 files. ### 5. Docker Service Split: Functional Boundaries **Decision:** Split by responsibility: - `compose.py` — compose file generation, modification, port injection, network injection - `container.py` — container status, IP lookup, network connect, logs - `config_staging.py` — staging config files into instance directories - `tunnel.py` — extracting tunnel URLs from cloudflared output **Rationale:** Each module has a single reason to change. `docker.py` mixed compose logic with container runtime queries. ### 6. Instance Lifecycle: Service Receives Raw Params, Not Request Objects **Decision:** Service functions receive model instances and primitive parameters, not FastAPI request objects. **Example:** `create_instance(session, user, project, repo, tool_type, data: CreateInstanceRequest)` → service extracts fields. **Rationale:** Keeps service layer independent of HTTP framework. Easier to test. ### 7. Auth Pattern: Gradual Migration, Not Big Bang **Decision:** Add `get_current_user` alongside existing `get_current_user_id`. Migrate routers incrementally. **Rationale:** Reduces risk. Endpoints that only need the ID can keep the old pattern. ### 8. Frontend Naming: Align with `b6f89f9` Conventions **Decision:** Use kebab-case for API files, PascalCase for page files with `Page` suffix, `features/` for component directories. **Rationale:** Matches the `b6f89f9` structure that was already reviewed and accepted. ## Module Map ### Backend — Before ``` api/ (~22 .py files) tool_instances.py (~3000 lines) — HTTP + Docker + Git + Lifecycle config_profiles.py (~1000 lines) — HTTP + Validation + Defaults tool_types.py (~500 lines) — HTTP + Schemas health.py (~150 lines) — HTTP + Schemas users.py (~100 lines) — HTTP + Schemas ... services/ (~20 .py files) docker.py (~600 lines) — Compose + Container + Tunnel models/ (~15 .py files) config_profile.py tool_instance.py ... ``` ### Backend — After ``` schemas/ (5 subpackages, ≤5 files each) tool/ __init__.py tool_type.py tool_instance.py config/ __init__.py config_profile.py user/ __init__.py user.py user_config.py project/ __init__.py project.py git_repository.py ssh_key.py system/ __init__.py health.py api/ (6 subpackages, 2–6 files each) tool/ __init__.py tool_instances.py (~300 lines) — HTTP routing only tool_types.py (~250 lines) — HTTP + validation tool_definitions.py tool_types_validation.py sessions.py config/ __init__.py config_profiles.py (~200 lines) — HTTP routing only user_config.py workspace/ __init__.py workspaces.py workspace_files.py workspace_git.py workspace_instances.py user/ __init__.py users.py (~60 lines) — HTTP only auth.py ssh_keys.py project/ __init__.py projects.py git_repositories.py system/ __init__.py health.py (~80 lines) — HTTP only events.py notifications.py dashboard.py terminal.py instance_proxy.py services/ (7 subpackages, ≤8 files each) docker/ __init__.py (~40 lines) — Re-exports compose.py (~240 lines) — Compose generation container.py (~120 lines) — Container queries config_staging.py (~80 lines) — File staging tunnel.py (~150 lines) — Tunnel URL extraction instance/ __init__.py instance_lifecycle.py (~420 lines) — Create/Start/Stop/Restart/Delete lifecycle_hooks.py health_monitor.py event_bus.py config/ __init__.py config_profile_resolver.py config_profiles.py (~300 lines) — CRUD + Defaults git/ __init__.py clone.py git_operations.py git_service.py build/ __init__.py docker_build.py manifest_compiler.py terminal/ __init__.py terminal_manager.py terminal_session.py shared/ __init__.py tunnel.py notification_service.py file_service.py permission_fixer.py readiness_probe.py ssh_keys.py workspace_manager.py correlation.py models/ (5 subpackages + base.py) tool/ __init__.py tool_type.py tool_instance.py tool_definition_manifest.py config/ __init__.py config_profile.py config_include.py config_mount.py user/ __init__.py user.py user_config.py ssh_key.py project/ __init__.py project.py git_repository.py workspace.py system/ __init__.py health_check.py notification.py instance_event.py terminal_session.py base.py (stays at root) ``` ### Frontend — Before ``` src/ api/ tool_types.ts ssh_keys.ts git_repositories.ts sessions.ts components/ git-toolbar.tsx file-editor.tsx commit-dialog.tsx ... pages/ dashboard.tsx projects.tsx sessions.tsx ... ``` ### Frontend — After ``` src/ api/ tool-types.ts ssh-keys.ts git-repositories.ts sessions.ts components/ features/ git/ GitToolbar.tsx FileBrowser.tsx FileEditor.tsx CommitDialog.tsx MergeDialog.tsx WorkspaceSidebar.tsx dashboard/ DashboardSummary.tsx ActiveSessionsList.tsx ProjectsSection.tsx QuickCreateForm.tsx RecentSessionsSection.tsx project/ RepositoriesSettingsTab.tsx tool-workshop/ ToolTypesTab.tsx ProtectedRoute.tsx AppShell.tsx pages/ DashboardPage.tsx ProjectsPage.tsx SessionsPage.tsx ... ``` ## Risks / Trade-offs **[Risk] Import cycles during extraction** → **Mitigation:** Extract schemas first (no service dependencies), then services, then thin routers last. Use TYPE_CHECKING guards. **[Risk] Merge conflicts with in-flight features** → **Mitigation:** Coordinate timing. This refactor should be the only large change on `dev` while it's in progress. Freeze other backend work. **[Risk] Frontend renaming breaks imports** → **Mitigation:** Use `git mv` for renames so git tracks history. Update all imports in a single commit. **[Risk] Missing re-export in docker/__init__.py breaks consumers** → **Mitigation:** After splitting, run a full import test across all backend files. Add any missing re-exports. ## Migration Plan 1. **Phase 1: Schemas** — Extract all Pydantic models into `schemas/`. Update imports in API routers. No logic changes. 2. **Phase 2: Services** — Split `docker.py`, extract `instance_lifecycle.py`, extract `config_profiles.py`. Update imports. 3. **Phase 3: Auth** — Add `get_current_user`, migrate routers that need the full user object. 4. **Phase 4: Frontend** — Rename files, move components, update imports. 5. **Phase 5: Verification** — Run full test suite, typecheck, build.