headquarter/openspec/changes/repo-restructure/explore.md

# Repo Restructure — Exploration Report

**Project:** Headquarter (full-stack workspace platform)
**Date:** 2026-06-02
**Scope:** Comprehensive codebase audit for structural refactoring

---

## 1. Directory Structure

### Root Layout
```
/workspace
├── apps/
│   ├── web/          # React 18 + Vite frontend
│   └── api/          # Python FastAPI + SQLAlchemy backend
├── e2e/              # Playwright tests
├── docs/             # (not heavily populated)
└── openspec/         # OpenSpec changes
```

### Frontend (`apps/web/src/`)
```
src/
├── api/              # 13 API modules (~1,200 LOC total)
│   ├── client.ts
│   ├── dashboard.ts
│   ├── git_repositories.ts
│   ├── profile.ts
│   ├── projects.ts
│   ├── sessions.ts
│   ├── settings.ts
│   ├── ssh_keys.ts
│   ├── terminal.ts
│   ├── tool_configs.ts
│   ├── tool_types.ts
│   ├── config_folders.ts
│   └── config_profiles.ts
├── components/       # 16 components (~2,100 LOC total)
│   ├── app-shell.tsx
│   ├── code-editor.tsx
│   ├── commit-dialog.tsx
│   ├── commit-panel.tsx
│   ├── file-editor.tsx
│   ├── git-toolbar.tsx
│   ├── icon.tsx
│   ├── instance-list.tsx
│   ├── merge-dialog.tsx
│   ├── protected-route.tsx
│   ├── protected-route.test.tsx
│   ├── repository-create-dialog.tsx
│   ├── settings-tab-layout.tsx
│   ├── syntax-highlighter.tsx
│   ├── workspace-header.tsx
│   └── repositories-settings-tab.tsx
├── hooks/            # 1 hook
│   └── use-theme.ts
├── pages/            # 15 pages (~3,500 LOC total)
│   ├── dashboard.tsx
│   ├── git-history.tsx
│   ├── git-repositories.tsx
│   ├── placeholder.tsx
│   ├── profile.tsx
│   ├── project-settings.tsx
│   ├── projects.tsx
│   ├── repo-workspace.tsx
│   ├── settings.tsx
│   ├── ssh-keys.tsx
│   ├── terminal.tsx
│   ├── tool-configs.tsx
│   ├── tool-types.tsx
│   └── tool-workshop.tsx
├── state/            # 2 context providers
│   ├── auth.tsx
│   └── sessions.tsx
├── types/            # 2 type modules
│   └── terminal.ts
├── utils/            # 3 utilities
│   ├── icons.ts
│   ├── language.ts
│   └── terminal-protocol.ts
├── styles.css        # 1 massive stylesheet (2,844 lines)
├── router.tsx        # Route definitions
├── main.tsx          # Entry point
└── types.ts          # Shared domain types
```

### Backend (`apps/api/`)
```
apps/api/
├── src/
│   ├── main.py                 # App entry point (~287 lines)
│   ├── config.py               # Pydantic settings (~128 lines)
│   ├── database.py             # SQLAlchemy setup (~114 lines)
│   ├── logging_config.py       # Middleware & logging (~92 lines)
│   ├── auth/
│   │   ├── session.py
│   │   └── dependencies.py
│   ├── api/                    # 15 routers
│   │   ├── auth.py
│   │   ├── config_folders.py
│   │   ├── config_profiles.py
│   │   ├── dashboard.py
│   │   ├── git_repositories.py  # ~900+ lines
│   │   ├── health.py
│   │   ├── instance_proxy.py
│   │   ├── projects.py
│   │   ├── ssh_keys.py
│   │   ├── terminal.py
│   │   ├── tool_configs.py
│   │   ├── tool_instances.py    # ~1,463 lines — CRITICAL
│   │   ├── tool_types.py
│   │   ├── user_config.py
│   │   └── users.py
│   ├── models/                 # SQLAlchemy models
│   ├── services/               # Business logic
│   │   ├── docker.py           # ~457+ lines
│   │   ├── docker_build.py
│   │   ├── git_control.py
│   │   ├── git_files.py
│   │   ├── git_history.py
│   │   ├── git_url_parser.py
│   │   ├── profile_resolver.py
│   │   └── readiness_probe.py
│   ├── utils/                  # Additional utilities
│   └── scripts/
│       └── seed.py
├── alembic/versions/           # 14+ migrations
└── tests/
    ├── conftest.py
    ├── unit/
    └── integration/
```

---

## 2. File Sizes — Files Over 200 Lines

### 🔴 CRITICAL — Over 400 Lines (Must Split)

| File | Lines | Issue |
|------|-------|-------|
| `apps/web/src/styles.css` | **2,844** | Single stylesheet for entire app; mixes layout, components, pages, syntax highlighting, and themes |
| `apps/api/src/api/tool_instances.py` | **1,463** | Monolithic router: CRUD, Docker orchestration, tunneling, proxying, config resolution, readiness probes |
| `apps/api/src/api/git_repositories.py` | **~900+** | Combined file browsing, Git control (branch/commit/merge/push/pull), URL parsing, history |
| `apps/api/src/services/docker.py` | **~457+** | Docker compose, container management, tunneling, config folder staging all in one |

### 🟡 WARNING — Over 200 Lines (Should Split)

| File | Lines | Issue |
|------|-------|-------|
| `apps/web/src/pages/tool-workshop.tsx` | **~700+** | 3-tab admin page with inline forms for tool types, configs, AND folders |
| `apps/web/src/pages/sessions.tsx` | **~668** | Sessions page with create form, active/recent lists, inline confirmations |
| `apps/web/src/pages/repo-workspace.tsx` | **~394** | Page + FileBrowser component + mixed data loading |
| `apps/web/src/components/instance-list.tsx` | **~388** | Instance CRUD + health checks + create dialog |
| `apps/web/src/pages/tool-types.tsx` | **~380** | Tool types list + create/edit dialog inline |
| `apps/web/src/pages/tool-configs.tsx` | **~354** | Tool configs list + create/edit dialog inline |
| `apps/web/src/hooks/use-terminal-connection.ts` | **~439** | WS lifecycle, ping-pong, reconnection, local echo, resize debouncing |
| `apps/web/src/pages/dashboard.tsx` | **~338** | Summary cards, session lists, quick-create form, recent sessions |
| `apps/web/src/components/terminal.tsx` | **~309** | Terminal chrome + xterm lifecycle + resize observer |
| `apps/web/src/pages/git-history.tsx` | **~233** | Commit list + detail panel with inline formatting |
| `apps/web/src/api/git_repositories.ts` | **~245** | API functions + types (reasonable, but types should move) |
| `apps/web/src/pages/projects.tsx` | **~206** | List + create dialog + delete confirmation |
| `apps/api/src/main.py` | **~287** | Router registration + startup logic + seeding + error handlers |
| `apps/api/src/api/config_profiles.py` | **~877** | Config profiles CRUD + complex resolution logic |
| `apps/api/src/api/tool_types.py` | **~616** | Tool types CRUD + compose/dockerfile validation |
| `apps/api/src/api/config_folders.py` | **~372** | Config folders CRUD |

---

## 3. Frontend Module Analysis

### Components (16 files, ~2,100 LOC, avg ~131 LOC)
**Too large:**
- `git-toolbar.tsx` (~268) — mixes git ops, branch creation form, merge dialog trigger, status summary
- `file-editor.tsx` (~241) — view/edit/commit workflow
- `instance-list.tsx` (~388) — instance CRUD + health + create dialog
- `terminal.tsx` (~309) — terminal chrome + xterm lifecycle

**Well-sized:**
- `workspace-header.tsx` (~48)
- `protected-route.tsx` (~19)
- `icon.tsx` (~165)

### Pages (15 files, ~3,500 LOC, avg ~233 LOC)
**All pages are too large.** Every page mixes:
- Data fetching (useEffect + API calls)
- Local state management (useState for forms, dialogs, loading)
- UI rendering (JSX)

**Worst offenders:**
- `tool-workshop.tsx` (~700) — 3 completely different admin interfaces in one file
- `sessions.tsx` (~668) — create form + active/recent lists + confirmations
- `repo-workspace.tsx` (~394) — contains `FileBrowser` component inline
- `dashboard.tsx` (~338) — summary, active sessions, projects list, quick-create form

### Hooks (3 files)
- `use-theme.ts` (~23) — fine
- `use-terminal-connection.ts` (~439) — too large; mixes WS lifecycle, ping-pong, reconnection, echo, resize

### API Modules (13 files, ~1,200 LOC)
- Well-organized by domain
- **Inconsistency:** Some define types inline (`api/sessions.ts` defines `ToolInstance`, `Session`), others in separate `types.ts`
- `api/client.ts` — centralized Axios instance with auth interceptor. Good pattern.

### State/Context (2 files)
- `auth.tsx` (~63) — well-sized
- `sessions.tsx` (~44) — well-sized

### Styles (1 file, 2,844 lines) — CRITICAL
**`styles.css` is the biggest problem in the frontend.** It contains:
- CSS variables / design tokens
- Global resets
- Layout (shell, nav, content grid)
- Page styles (home, settings, git-history, repo-workspace)
- Component styles (cards, buttons, dialogs, forms, file-tree, editor)
- Syntax highlighting overrides
- Responsive media queries scattered throughout

### Types
- `src/types.ts` — core domain types (SessionUser, Project)
- `src/types/terminal.ts` — terminal-specific WebSocket protocol types
- **Problem:** API modules also export their own types (`ToolInstance`, `Session`, `GitRepository`, etc.) causing duplication and confusion. `Session` is defined in BOTH `api/sessions.ts` and `state/sessions.tsx`.

### Utils
- `icons.ts` (~180) — icon name mapping
- `language.ts` (~90) — file extension → language detection
- `terminal-protocol.ts` (~76) — WS message encoding/decoding + type guards

### Router
- `router.tsx` (~58) — clean and readable

### Tests
- `components/protected-route.test.tsx` (~49)
- `api/tool_types.test.ts` (~227)
- `api/config_folders.test.ts` (~131)
- `pages/dashboard.test.tsx` (~81)
- `pages/projects.test.tsx` (~174) — failing tests (React Router context issue)
- `pages/tool-workshop.test.tsx` (~527)
- `hooks/use-terminal-connection.test.ts` (~339)
- **Massive gaps:** No tests for most pages, hooks, state providers, or components

---

## 4. Backend Module Analysis

### Entry Points
- `src/main.py` (~287) — FastAPI app setup, CORS, middleware, exception handlers, startup events, seeding, router mounting
- **Problem:** Seed data (builtin tool types) is hardcoded here (~100 lines of compose templates). Should be in `seeds/` or `services/seed_data.py`.

### Routers/Endpoints (15 files)
**Organization:** One router per domain — good structure in theory, but files are too large.

**`tool_instances.py` (1,463 lines)** — The worst offender. Contains:
- Pydantic request/response models
- Helper functions: `_modify_compose_file`, `_apply_resolved_profile`, `_get_user`, `_get_owned_project`, `_sanitize_name`, `_generate_instance_name`
- Endpoints: create, list, get, start, stop, restart, delete, logs, recreate-tunnel, health-check, proxy
- Inline Docker orchestration logic (should be in services)
- Inline config resolution (should use service layer)

**`git_repositories.py` (~900+ lines)** — Contains:
- Repository CRUD
- File browsing endpoints
- Git control endpoints (branch, checkout, commit, fetch, pull, push, merge)
- URL parsing endpoint

**`config_profiles.py` (~877 lines)** — Contains:
- Config profile CRUD
- Complex profile resolution logic
- Config folder/application logic

### Models
- Located in `src/models/` — one file per entity
- Clean separation, well-sized

### Services/Business Logic
- `docker.py` (~456) — Docker compose, container ops, tunneling, config file staging. Too large.
- `docker_build.py` (~69) — Image building
- `git_control.py` (~295) — Git operations
- `git_files.py` (~439) — File tree, read, write
- `git_history.py` (~382) — Commit history, graph, diff
- `git_url_parser.py` (~228) — URL parsing and validation
- `profile_resolver.py` (~251) — Config profile resolution
- `readiness_probe.py` (~66) — Container health probes
- `terminal_manager.py` (~193) — Terminal session lifecycle
- `terminal_session.py` (~162) — Individual terminal session handling

### Database/ORM
- `database.py` (~116) — Engine, session factory, init with alembic subprocess
- `config.py` (~143) — Pydantic settings with env var resolution
- Alembic migrations in `alembic/versions/` — 14+ migration files

---

## 5. Coupling and Dependency Patterns

### Frontend High-Coupling Files

**`repo-workspace.tsx`** imports from:
- `react-router-dom` (params, search params)
- `../api/client` (direct apiClient usage)
- `../api/git_repositories`
- `../components/commit-panel`
- `../components/file-editor`
- `../components/git-toolbar`
- `../components/instance-list`
- `../components/workspace-header`
- `../api/tool_types`

**`dashboard.tsx`** imports from:
- `../api/dashboard`, `../api/sessions`, `../api/projects`, `../api/git_repositories`, `../api/tool_types`, `../api/settings`
- `../types`, `../components/icon`

**`tool-workshop.tsx`** imports from:
- `../api/tool_types`, `../api/tool_configs`, `../api/config_folders`
- Manages 3 separate entity forms with ~20 useState variables each

### Circular Dependencies
- **No obvious circular imports detected**, but `Session` type is duplicated between `api/sessions.ts` and `state/sessions.tsx`, creating conceptual circularity.

### Business Logic Mixed with UI
- **Every page component** contains API calls directly in `useEffect`
- Form validation logic is inline in page components
- `repo-workspace.tsx` defines `FileBrowser` as an inner component — cannot be tested or reused independently

### API Call Patterns
- **Mostly centralized** in `api/` modules — good
- **Exception:** `repo-workspace.tsx`, `file-editor.tsx`, `project-settings.tsx` use `apiClient` directly instead of domain API modules
- **Exception:** `app-shell.tsx` calls `getUserSessions()` directly

---

## 6. Naming Inconsistencies

### File Naming Conventions

| Location | Convention | Examples | Issues |
|----------|-----------|----------|--------|
| `pages/` | mostly kebab-case | `git-history.tsx`, `repo-workspace.tsx` | `projects.tsx`, `profile.tsx`, `settings.tsx`, `dashboard.tsx` are NOT kebab-case |
| `components/` | kebab-case | `app-shell.tsx`, `protected-route.tsx` | `repositories-settings-tab.tsx` (long but consistent) |
| `api/` | snake_case | `tool_configs.ts`, `git_repositories.ts` | Mixes with frontend convention |
| `utils/` | kebab-case | `terminal-protocol.ts` | Good |
| `hooks/` | camelCase | `useTheme.ts` would be standard, but file is `use-theme.ts` | Actually kebab-case, which is fine but inconsistent with React convention |
| Backend routers | snake_case | `tool_instances.py`, `git_repositories.py` | Consistent within backend |
| Backend services | snake_case | `docker.py`, `profile_resolver.py` | Consistent |

### Component vs File Naming
- Component `ProtectedRoute` → file `protected-route.tsx` ✅
- Component `AppShell` → file `app-shell.tsx` ✅
- Component `GitHistoryPage` → file `git-history.tsx` ❌ (should be `GitHistoryPage` in `git-history-page.tsx` OR component renamed to `GitHistory`)
- Component `RepoWorkspace` → file `repo-workspace.tsx` ❌ (same issue)
- Page components use `Page` suffix inconsistently: `ProjectsPage`, `GitHistoryPage`, but `RepoWorkspace` has no `Page` suffix

### Function/Variable Naming
- Frontend: camelCase consistently
- Backend: snake_case consistently
- **API types:** Backend uses `snake_case` fields; frontend types mirror this (`default_ssh_key_id`, `tool_type_name`). Good for API alignment.

---

## 7. Quality Signals

### TODO/FIXME Comments
- Only **2 TODOs** found:
  - `apps/api/src/utils/git_history.py:188-189`: `# TODO: extract committer separately` (appears twice)

This is surprisingly low — suggests either good maintenance or lack of inline documentation.

### Dead Code / Unused Exports
- `dashboard.tsx` exports `HomePage as DashboardPage` — dual naming is confusing
- `src/types.ts` exports `SessionPayload` which is only used in auth context
- Several CSS classes in `styles.css` may be unused (hard to verify without build analysis)

### Duplicate Logic
- **Backend auth checks:** `_get_user()` and `_get_owned_project()` are duplicated in nearly every router file (`tool_instances.py`, `git_repositories.py`, `ssh_keys.py`, etc.)
- **Frontend loading/error patterns:** Identical `status: "loading" | "ready" | "error"` state + retry button pattern copied in ~8 page components
- **Frontend form dialogs:** Create/edit/delete confirmation pattern repeated in `projects.tsx`, `tool-types.tsx`, `tool-configs.tsx`, `ssh-keys.tsx`

### Test Coverage Gaps
- **Frontend:** 7 test files, but many pages and components untested
- **Backend:** Unit tests for `git_url_parser.py`, `migration_metadata.py`, `profile_resolver.py`, `readiness_probe.py`, `docker_build.py`, `terminal_manager.py`, `terminal_session.py`; integration tests via `conftest.py`
- **E2E tests** only cover login flow (`e2e/tests/login.spec.ts`)

---

## Recommendations

### Target Directory Structure

#### Frontend (`apps/web/src/`)
```
src/
├── api/                    # Keep — centralized API layer
│   ├── client.ts
│   ├── __mocks__/          # Add: mock API responses for tests
│   └── {domain}/           # Group by domain
│       ├── index.ts        # Re-exports
│       ├── types.ts        # Domain types ONLY
│       └── api.ts          # API functions
├── components/             # Generic UI components
│   ├── ui/                 # Primitive components (Button, Card, Dialog, Input)
│   ├── layout/             # AppShell, Navigation, Header
│   └── features/           # Domain-specific components
│       ├── git/
│       ├── project/
│       ├── session/
│       └── settings/
├── hooks/                  # Custom hooks
│   ├── use-theme.ts
│   ├── use-auth.ts         # Extract from state/auth.tsx?
│   └── use-api-query.ts    # NEW: reusable data fetching
├── pages/                  # Route entry points ONLY
│   ├── dashboard/
│   │   └── page.tsx
│   ├── projects/
│   │   ├── page.tsx
│   │   ├── project-list.tsx
│   │   └── create-project-dialog.tsx
│   └── ...
├── state/                  # Keep contexts
├── styles/
│   ├── tokens.css          # CSS variables only
│   ├── global.css          # Resets + base styles
│   ├── components/         # Component styles
│   └── pages/              # Page-specific styles
├── types/                  # Centralize ALL shared types
│   └── index.ts
└── utils/
```

#### Backend (`apps/api/src/`)
```
src/
├── main.py                 # Router mounting + middleware ONLY
├── config.py
├── database.py
├── logging_config.py
├── auth/
├── api/
│   └── v1/                 # Versioned routes
│       ├── __init__.py
│       ├── auth.py
│       ├── projects/
│       │   ├── __init__.py
│       │   ├── router.py
│       │   └── dependencies.py
│       ├── repositories/
│       │   ├── __init__.py
│       │   ├── router.py     # CRUD only
│       │   ├── files.py      # File browsing
│       │   └── git.py        # Git control operations
│       ├── instances/
│       │   ├── __init__.py
│       │   ├── router.py     # CRUD + lifecycle
│       │   ├── compose.py    # Compose file generation
│       │   ├── tunnel.py     # Cloudflare tunnel ops
│       │   └── proxy.py      # HTTP proxy
│       └── ...
├── models/
├── schemas/                # NEW: Pydantic schemas separate from routers
├── services/
│   ├── docker/
│   │   ├── __init__.py
│   │   ├── compose.py      # Extract from docker.py
│   │   ├── container.py    # Container lifecycle
│   │   ├── tunnel.py       # Cloudflare tunneling
│   │   └── config.py       # Config file staging
│   └── git/
│       ├── control.py
│       ├── files.py
│       └── history.py
├── seeds/                  # NEW: Seed data
│   └── builtin_tool_types.py
└── tests/
```

---

### Files That MUST Be Split

1. **`apps/web/src/styles.css`** → Split into 5-8 files by concern
2. **`apps/api/src/api/tool_instances.py`** → Split into router + compose service + tunnel service + proxy service
3. **`apps/api/src/api/git_repositories.py`** → Split into repository CRUD router + file router + git control router
4. **`apps/api/src/services/docker.py`** → Split into compose, container, tunnel, config staging modules
5. **`apps/web/src/pages/tool-workshop.tsx`** → Split into 3 page tabs or feature components
6. **`apps/web/src/pages/repo-workspace.tsx`** → Extract `FileBrowser` to `components/features/git/file-browser.tsx`
7. **`apps/web/src/pages/sessions.tsx`** → Extract create form, active list, recent list
8. **`apps/web/src/hooks/use-terminal-connection.ts`** → Extract WS manager, echo handler, resize debouncer

---

### Naming Convention to Standardize On

| Layer | Convention | Example |
|-------|-----------|---------|
| React components (files) | PascalCase matching component | `GitHistoryPage.tsx` |
| React hooks (files) | camelCase | `useTheme.ts` |
| Utility modules | kebab-case | `terminal-protocol.ts` |
| API modules | kebab-case | `tool-configs.ts` |
| Backend routers | snake_case | `tool_instances.py` |
| Backend services | snake_case | `profile_resolver.py` |
| CSS modules | kebab-case matching component | `git-history-page.module.css` |

---

### Order of Migration (First → Last)

**Phase 1: Safe Foundations (low risk)**
1. Extract shared types to `src/types/index.ts` (remove duplication)
2. Create `src/hooks/use-api-query.ts` for reusable data fetching
3. Extract `FileBrowser` from `repo-workspace.tsx`
4. Move seed data from `main.py` to `seeds/builtin_tool_types.py`

**Phase 2: Style System (medium risk, high reward)**
5. Split `styles.css` into design tokens + component modules
6. Introduce CSS modules or Tailwind utility extraction for component styles

**Phase 3: Backend Decomposition (medium risk)**
7. Extract `_get_user` and `_get_owned_project` to `auth/dependencies.py` or `api/dependencies.py`
8. Split `tool_instances.py` into router + services
9. Split `git_repositories.py` into CRUD + files + git control routers
10. Split `services/docker.py` into focused modules

**Phase 4: Frontend Page Decomposition (higher risk — touches UX)**
11. Split `tool-workshop.tsx` into feature components
12. Split `dashboard.tsx` into summary/session/project sections
13. Split `sessions.tsx` into create-form + lists
14. Split `settings.tsx` — move `GeneralSettingsTab` to its own file

**Phase 5: Testing & Polish**
15. Add tests for extracted components
16. Add backend integration tests for refactored routers