alex/manage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
04871bd7d405beba842531cfafc59f7f3d96a24e
manage/docs/MIGRATION_PLAN.md
T
Developer d4f95b64d4 chore(observability): externalize stack from root compose files 
Manage now connects to existing Grafana/Prometheus/Alertmanager instances
and never deploys its own stack.

- docker-compose.yml / docker-compose.dev.yml: removed prometheus, loki,
  alloy, grafana, alertmanager, node-exporter services, the monitoring
  network, and observability named volumes; they now ship only backend +
  frontend. Dev frontend now joins the web network so the Vite dev proxy
  can reach the backend.
- backend: alertmanager_url default is now empty; /api/monitoring/alerts
  and /alertmanager-status return graceful "not configured" responses
  when ALERTMANAGER_URL is unset. Added not-configured tests.
- docker-compose.observability.yml: kept as the optional standalone
  example; header clarifies Manage does not deploy it.
- Removed orphaned combined monitoring/prometheus/prometheus.yml
  (standalone stack uses prometheus.standalone.yml).
- Docs (README, REQUIREMENTS decision log, monitoring-logging-design,
  observability-runbooks, context.md, MIGRATION_PLAN, frontend/README,
  CHANGELOG) updated to the connect-to-existing model.

VITE_GRAFANA_URL / VITE_PROMETHEUS_URL remain as optional frontend
deep-link overrides. .env.example still needs a manual update (safety
policy blocks assistant edits): set ALERTMANAGER_URL empty/optional and
move standalone-only vars out of the root file.
2026-06-23 21:20:07 +00:00

242 lines
10 KiB
Markdown
Raw Blame History

# Migration Plan: Streamlit → FastAPI + React
## Overview
Split the current Streamlit monolith into:
- **Backend:** FastAPI (Python) serving a REST API
- **Frontend:** React SPA (TypeScript) consuming that API
The existing `clients/`, `domain/`, `services/`, `jobs.py`, `utils.py`, and `config.py` are already UI-independent and transfer directly to the FastAPI backend with minimal changes.
---
## Current Architecture
```
src/media_library_viewer/
├── app.py # Streamlit orchestration + caching (DELETE)
├── config.py # Env/dotenv config
├── jobs.py # SSH job templates
├── utils.py # Formatting helpers
├── clients/
│ ├── jellyfin.py # Jellyfin HTTP client
│ ├── resources.py # Remote resource collector
│ └── ssh.py # SSH command execution
├── domain/
│ └── media.py # Media normalization
├── services/
│ └── media_index.py # SQLite media index
└── ui/ # Streamlit rendering (DELETE)
├── dashboard.py
├── file_browser.py
├── media.py
└── preview.py
```
---
## Target Architecture
```
repo/
├── backend/
│ ├── main.py # FastAPI app + CORS + lifespan
│ ├── config.py # pydantic-settings based config
│ ├── dependencies.py # DI for SSH/Jellyfin clients
│ ├── routers/
│ │ ├── dashboard.py # /api/dashboard/*
│ │ ├── monitoring.py # /api/monitoring/*
│ │ ├── media.py # /api/media/*
│ │ ├── files.py # /api/files/*
│ │ └── jobs.py # /api/jobs/*
│ ├── clients/ # Copied unchanged
│ │ ├── jellyfin.py
│ │ ├── resources.py
│ │ └── ssh.py
│ ├── domain/ # Copied unchanged
│ │ └── media.py
│ ├── services/ # Copied unchanged
│ │ └── media_index.py
│ ├── jobs.py # Copied unchanged
│ ├── utils.py # Copied unchanged
│ └── pyproject.toml
│
├── frontend/
│ ├── package.json
│ ├── vite.config.ts
│ ├── tsconfig.json
│ └── src/
│ ├── main.tsx
│ ├── App.tsx
│ ├── api/
│ │ └── client.ts # Typed fetch wrappers
│ ├── hooks/
│ │ ├── useDashboard.ts
│ │ ├── useMonitoring.ts
│ │ ├── useMedia.ts
│ │ └── useFiles.ts
│ ├── pages/
│ │ ├── Dashboard.tsx
│ │ ├── Monitoring.tsx
│ │ ├── Media.tsx
│ │ └── FileBrowser.tsx
│ ├── components/
│ │ ├── NowPlaying.tsx
│ │ ├── MetricCard.tsx
│ │ ├── LibraryOverview.tsx
│ │ ├── MonitoringCharts.tsx
│ │ ├── MediaTable.tsx
│ │ ├── FileListing.tsx
│ │ └── FfprobePreview.tsx
│ └── types/
│ └── index.ts
│
├── docker-compose.yml # Backend + frontend deployment (observability is external)
└── README.md
```
---
## FastAPI Endpoints
| Endpoint | Method | Source | Description |
| ------------------------------- | ------ | ---------------------------------------- | -------------------------------- |
| `/api/dashboard/counts` | GET | `jellyfin.media_counts()` | Movie/series/episode totals |
| `/api/dashboard/libraries` | GET | `jellyfin.library_item_counts()` | Per-library breakdown |
| `/api/dashboard/now-playing` | GET | `jellyfin.active_sessions()` | Active sessions + transcode info |
| `/api/monitoring/status` | GET | `resources.resource_collector_status()` | Collector running? *(legacy/removed)* |
| `/api/monitoring/metrics` | GET | `resources.read_resource_metrics()` | Last-hour JSONL samples *(legacy/removed)* |
| `/api/monitoring/disk` | GET | `resources.disk_space()` | df for media root *(removed 2026-06-17; metrics now in Prometheus/Grafana)* |
| `/api/monitoring/start` | POST | `resources.start_resource_collector()` | Start collector |
| `/api/monitoring/stop` | POST | `resources.stop_resource_collector()` | Stop collector |
| `/api/monitoring/restart` | POST | `resources.restart_resource_collector()` | Restart collector |
| `/api/media/status` | GET | `MediaIndex.status()` | Index exists/count/age |
| `/api/media/build` | POST | `build_media_index()` | Rebuild index |
| `/api/media/query` | GET | `MediaIndex.query()` | Filtered/sorted/paged query |
| `/api/files/list?path=` | GET | `ssh.list_dir()` | Directory listing |
| `/api/files/ffprobe?path=` | GET | `ssh.ffprobe_json()` | ffprobe JSON |
| `/api/files/stat?path=` | GET | `ssh.stat_path()` | stat output |
| `/api/files/resolve-path?path=` | GET | `resolve_remote_media_path()` | Jellyfin→SSH path mapping |
| `/api/jobs/templates` | GET | `JOB_TEMPLATES` | Available job list |
| `/api/jobs/run` | POST | `run_job()` | Execute a job |
---
## React Frontend
### Tech Stack
- **Vite** + React 18+ + TypeScript
- **@tanstack/react-query** — data fetching with polling
- **ag-grid-react** — media table and file browser (same grid lib)
- **recharts** — monitoring line charts
- **react-router** — page navigation
- **tailwindcss** + **shadcn/ui** — styling
### Page → Component Mapping
| Page | Components | Polling |
| ------------ | --------------------------------------------------------- | -------------------------------- |
| Dashboard | NowPlaying, MetricCard (server overview), LibraryOverview | 15s (now-playing), 30s (metrics) |
| Monitoring | MonitoringCharts, MetricCard, CollectorControls | 15s |
| Media | MediaTable (AG Grid), filter/sort/page controls | on-demand |
| File Browser | FileListing (AG Grid), FfprobePreview, JobRunner | on-demand |
### Key Interactions
- Media row click → updates client-side file browser path state (no API call)
- File browser directory click → fetches `/api/files/list?path=...`
- File browser file click → fetches `/api/files/ffprobe?path=...`
- Path input Enter → navigates directory
- Monitoring charts → recharts line chart from `/api/monitoring/metrics`
---
## What Transfers Unchanged (~1,350 lines)
| File | Lines | Notes |
| ------------------------- | ----- | ---------------------------- |
| `clients/jellyfin.py` | 167 | Remove unused methods if any |
| `clients/ssh.py` | 146 | No changes |
| `clients/resources.py` | 316 | No changes |
| `domain/media.py` | 161 | No changes |
| `services/media_index.py` | 278 | No changes |
| `jobs.py` | 59 | No changes |
| `utils.py` | 227 | No changes |
---
## What Gets Deleted
- `src/media_library_viewer/ui/` (all Streamlit rendering)
- `src/media_library_viewer/app.py` (Streamlit orchestration)
- Root `app.py` (Streamlit launcher)
- `streamlit` and `streamlit-aggrid` dependencies
---
## Migration Steps
### Step 1: Backend scaffold
- Create `backend/` with FastAPI, pydantic-settings config, DI
- Copy `clients/`, `domain/`, `services/`, `jobs.py`, `utils.py`
- Implement routers wrapping existing functions
- Test with curl
### Step 2: Frontend scaffold
- Create `frontend/` with Vite + React + TypeScript
- API client with typed wrappers
- React Query provider + hooks
- Router with 4 pages
### Step 3: Build pages (one at a time)
1. Dashboard (simplest — just fetches and displays)
2. Monitoring (charts + controls)
3. Media (AG Grid + filters)
4. File Browser (AG Grid + ffprobe + jobs)
### Step 4: Validate feature parity
- Compare behavior side-by-side with Streamlit
- Remove Streamlit code
---
## Effort Estimate
| Component | Effort |
| ------------------------------------- | ------------- |
| FastAPI backend | 1-2 days |
| React scaffold + routing + API client | 0.5 day |
| Dashboard page | 0.5 day |
| Monitoring page + charts | 1 day |
| Media page + AG Grid | 1 day |
| File browser page | 1 day |
| Polish + testing + deployment | 1-2 days |
| **Total** | **~6-9 days** |
---
## Key Decisions
- **Polling over WebSocket initially** — simpler, matches current behavior; upgrade path exists for later.
- **Auth deferred** — rely on network access control for now; add API key or JWT later.
- **CORS enabled** — for Vite dev server and production origin.
- **Path resolution stays server-side** — frontend sends Jellyfin paths, backend resolves to SSH paths.
- **Both can coexist** — Streamlit and FastAPI can run simultaneously during transition since they share the same clients/services.
---
## Optional Post-Migration Enhancements
- WebSocket push for real-time monitoring/now-playing
- JWT authentication
- Background index build with SSE progress
- Dark/light theme
- Persistent user preferences (localStorage + optional backend sync)
- Docker Compose (backend + frontend + nginx reverse proxy)
Reference in New Issue View Git Blame Copy Permalink