From cff082c7a148e022756895e8c2ee9645192bfc87 Mon Sep 17 00:00:00 2001 From: Developer Date: Thu, 9 Jul 2026 07:55:59 +0000 Subject: [PATCH] spec(service-storage-harness): add design 10 design decisions incl: lifecycle-only harness, per-concern DB files, QbittorrentSampleStore (120-sample cap), qBit cookie client, 3 widget kinds (totals=item count, active=DL/UL state, speed=reuses Change A renderer via shared LineSeriesChart extract), MediaIndex +service_id backfill, scoped replace_items (fixes latent global-clear bug), best-effort cascade-delete. 4-slice plan. 3 source findings (worker already threads service_id). --- .../changes/service-storage-harness/design.md | 863 ++++++++++++++++++ 1 file changed, 863 insertions(+) create mode 100644 openspec/changes/service-storage-harness/design.md diff --git a/openspec/changes/service-storage-harness/design.md b/openspec/changes/service-storage-harness/design.md new file mode 100644 index 0000000..1d22135 --- /dev/null +++ b/openspec/changes/service-storage-harness/design.md @@ -0,0 +1,863 @@ +# SDD Design: Service Storage Harness (with qBittorrent widgets + MediaIndex migration) + +**Change:** `service-storage-harness` +**Phase:** design +**Date:** 2026-07-09 + +> Grounded in `proposal.md` (Q1–Q5 resolved) and the already-archived +> `prometheus-direct-charting` change. No source changes in this phase. + +## 0. Source findings (read before anything else) + +The proposal was written against a stale project map. Reading actual source +surfaced three findings that shape the design. Trust source, not the map. + +### 0.1 The media worker ALREADY threads `service_id` — but it is NOT persisted in rows + +`workers/media_index_worker.py` already accepts `--service-id` via argparse +(`run_build(final_index_path, staging_index_path, service_id="")`), and +`_resolve_jellyfin(service_id)` resolves the Jellyfin client + user_id from the +settings store. The media router's `_start_worker(index, service_id)` and +`post_build_index` already pass `jellyfin_service_id` through to the worker. + +**But:** `service_id` is used ONLY to pick the Jellyfin connection. It is never +stored in the `media_items` table (which has no `service_id` column). The +migration must close this gap: `replace_items` must scope its DELETE+INSERT by +`service_id`, and `query` must filter by it. The plumbing to get `service_id` +into the worker already exists — only the storage layer is missing. + +### 0.2 `media_items` is cleared globally on every build + +`replace_items` does `DELETE FROM media_items` (no WHERE clause). This means a +build for Jellyfin instance A wipes instance B's rows. After migration this +becomes `DELETE FROM media_items WHERE service_id = ?`. + +### 0.3 The `delete_service` cascade already exists — the harness hooks into it + +`SettingsStore.delete_service` already cascade-deletes `dashboard_widgets WHERE +service_id = ?` (with a PRAGMA-guarded column check). The harness cascade-delete +hooks into the same place: after the service row is deleted, iterate registered +concerns and delete that `service_id` from each owned table. + +### 0.4 DB path is a module constant, not in config.py + +`DEFAULT_INDEX_PATH = Path(".cache/media_library_viewer/media_index.sqlite")` in +`media_index_impl.py`. The file stays at this path (per the locked topology +decision). The harness must not move it. + +--- + +## 1. Architecture overview + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ ServiceDataHarness │ +│ (services/service_data.py — LIFECYCLE ONLY) │ +│ │ +│ • register_concern(db_filename, migrations[], tables[], │ +│ service_id_column="service_id") │ +│ • run_migrations() — on startup, per concern DB │ +│ • cascade_delete(service_id) — iterate concerns, DELETE rows │ +│ • connect(db_filename) → sqlite3.Connection (per-concern) │ +└──────────────┬───────────────────────────┬───────────────────────┘ + │ │ + ┌──────────▼──────────┐ ┌─────────▼──────────────┐ + │ QbittorrentStore │ │ MediaIndex │ + │ (services/ │ │ (services/ │ + │ qbittorrent_store) │ │ media_index_impl) │ + │ │ │ │ + │ qbittorrent.db │ │ media_index.sqlite │ + │ └ qbittorrent_ │ │ └ media_items │ + │ speed_samples │ │ (+ service_id col) │ + │ (service_id, ts, │ │ └ index_metadata │ + │ dl_speed, │ │ │ + │ up_speed) │ │ bespoke: replace_items │ + │ │ │ (scoped), query │ + │ bespoke: append, │ │ (scoped), status │ + │ window, prune │ │ │ + └──────────────────────┘ └────────────────────────┘ + │ + ┌──────────▼──────────┐ + │ QbittorrentClient │ + │ (clients/ │ + │ qbittorrent) │ + │ │ + │ login → cookie │ + │ sync/maindata │ + │ (totals + active + │ + │ speeds) │ + └──────────────────────┘ +``` + +**Key constraints carried from the proposal:** + +- Harness is lifecycle-only: migrations, service_id scoping, cascade-delete. No generic value table, no generic CRUD (D2). +- Per-concern DB files: `media_index.sqlite` stays put; new `qbittorrent.db` (D4). +- MediaIndex migration is sequenced after harness + qBit are proven (D3). +- qBit speed chart reuses the `PrometheusChartWidget` recharts renderer fed from an InService data path returning `{series}` (Q1). +- Totals = count of listed items, NOT transfer bytes (Q2). +- Active = state downloading|uploading (Q3). N instances (Q4). Username/password → cookie (Q5). + +--- + +## 2. Backend design + +### 2.1 `ServiceDataHarness` (`services/service_data.py`) + +A lifecycle-only registry of storage concerns. Each concern declares its own DB +filename, ordered migrations, owned tables, and the column used for service +scoping. + +```python +@dataclass(frozen=True) +class StorageConcern: + """A per-integration storage namespace registered with the harness.""" + + concern_key: str # e.g. "qbittorrent", "media_index" + db_filename: str # e.g. "qbittorrent.db", "media_index.sqlite" + migrations: list[str] # ordered CREATE/ALTER statements (idempotent) + tables: list[str] # tables owned by this concern (for cascade) + service_id_column: str = "service_id" + + +class ServiceDataHarness: + """Lifecycle-only registry of per-concern storage. + + Owns: DB provisioning, per-concern migrations, service_id cascade-delete. + Does NOT own: data operations (each store keeps bespoke append/window/query/etc.). + """ + + def __init__(self, base_dir: Path) -> None: + self._base_dir = Path(base_dir) + self._concerns: dict[str, StorageConcern] = {} + + def register(self, concern: StorageConcern) -> None: + """Register a storage concern. Called at module import / startup.""" + self._concerns[concern.concern_key] = concern + + def db_path(self, concern_key: str) -> Path: + """Return the absolute path to a concern's DB file.""" + concern = self._concerns[concern_key] + return self._base_dir / concern.db_filename + + def connect(self, concern_key: str) -> sqlite3.Connection: + """Open a WAL-mode connection to a concern's DB.""" + path = self.db_path(concern_key) + path.parent.mkdir(parents=True, exist_ok=True) + conn = sqlite3.connect(path, timeout=30) + conn.row_factory = sqlite3.Row + conn.execute("PRAGMA journal_mode=WAL") + conn.execute("PRAGMA busy_timeout=30000") + return conn + + def run_migrations(self) -> None: + """Run pending migrations for every registered concern.""" + for concern in self._concerns.values(): + path = self.db_path(concern.concern_key) + path.parent.mkdir(parents=True, exist_ok=True) + with sqlite3.connect(path, timeout=30) as conn: + conn.execute("PRAGMA journal_mode=WAL") + conn.executescript(";".join(concern.migrations)) + + def cascade_delete(self, service_id: str) -> None: + """Delete all rows for a service_id across every concern's tables. + + Called from SettingsStore.delete_service after the service row is removed. + """ + for concern in self._concerns.values(): + col = concern.service_id_column + with sqlite3.connect(self.db_path(concern.concern_key), timeout=30) as conn: + for table in concern.tables: + cols = {row[1] for row in conn.execute(f"PRAGMA table_info({table})").fetchall()} + if col in cols: + conn.execute(f"DELETE FROM {table} WHERE {col} = ?", (service_id,)) +``` + +**Module-level singleton + registration:** + +```python +_HARNESS: ServiceDataHarness | None = None + +def get_service_data_harness() -> ServiceDataHarness: + global _HARNESS + if _HARNESS is None: + base_dir = Path(os.environ.get("BACKEND_CACHE_DIR", ".cache/media_library_viewer")) + _HARNESS = ServiceDataHarness(base_dir) + # Register built-in concerns (each store module calls register on import) + _HARNESS.register(QBITTORRENT_CONCERN) + _HARNESS.register(MEDIA_INDEX_CONCERN) + _HARNESS.run_migrations() + return _HARNESS +``` + +**Startup hook:** called from `main.py` lifespan alongside `get_settings_store().ensure_defaults()`. The harness is lazy-initialized on first access (like `SettingsStore`), so tests can override the base dir via env. + +### 2.2 `QbittorrentSampleStore` (`services/qbittorrent_store.py`) + +Speed-sample storage for the qBit speed widget. Registered as a concern with the +harness. + +```python +QBITTORRENT_CONCERN = StorageConcern( + concern_key="qbittorrent", + db_filename="qbittorrent.db", + migrations=[ + """ + CREATE TABLE IF NOT EXISTS qbittorrent_speed_samples ( + service_id TEXT NOT NULL, + ts INTEGER NOT NULL, + dl_speed INTEGER NOT NULL DEFAULT 0, + up_speed INTEGER NOT NULL DEFAULT 0 + ); + CREATE INDEX IF NOT EXISTS idx_qbit_samples_service_ts + ON qbittorrent_speed_samples(service_id, ts); + """ + ], + tables=["qbittorrent_speed_samples"], +) + +MAX_SAMPLES = 120 # ~2 min at 1s poll, ~4 min at 2s poll + + +class QbittorrentSampleStore: + """Bespoke speed-sample store for qBittorrent widgets.""" + + def __init__(self, harness: ServiceDataHarness | None = None) -> None: + self._harness = harness or get_service_data_harness() + + def append(self, service_id: str, ts: int, dl_speed: int, up_speed: int) -> None: + """Append a sample and prune old entries beyond MAX_SAMPLES.""" + with self._harness.connect("qbittorrent") as conn: + conn.execute( + "INSERT INTO qbittorrent_speed_samples (service_id, ts, dl_speed, up_speed) VALUES (?, ?, ?, ?)", + (service_id, ts, dl_speed, up_speed), + ) + # Prune: keep only the most recent MAX_SAMPLES rows for this service + conn.execute( + """DELETE FROM qbittorrent_speed_samples + WHERE service_id = ? AND ts NOT IN ( + SELECT ts FROM qbittorrent_speed_samples + WHERE service_id = ? + ORDER BY ts DESC LIMIT ? + )""", + (service_id, service_id, MAX_SAMPLES), + ) + + def window(self, service_id: str, since_ts: int | None = None) -> list[dict[str, Any]]: + """Return all samples for a service since a timestamp (or all if None).""" + with self._harness.connect("qbittorrent") as conn: + if since_ts is not None: + rows = conn.execute( + "SELECT ts, dl_speed, up_speed FROM qbittorrent_speed_samples WHERE service_id = ? AND ts >= ? ORDER BY ts ASC", + (service_id, since_ts), + ).fetchall() + else: + rows = conn.execute( + "SELECT ts, dl_speed, up_speed FROM qbittorrent_speed_samples WHERE service_id = ? ORDER BY ts ASC", + (service_id,), + ).fetchall() + return [{"ts": r[0], "dl_speed": r[1], "up_speed": r[2]} for r in rows] +``` + +### 2.3 `QbittorrentClient` (`clients/qbittorrent.py`) + +Cookie-session HTTP client modeled on `JellyfinClient`'s session pattern. + +```python +class QbittorrentClient: + """Minimal qBittorrent Web API client (read-only: sync/maindata only).""" + + def __init__(self, base_url: str, username: str, password: str, timeout: int = 10) -> None: + self.base_url = base_url.rstrip("/") + if not self.base_url.endswith("/api/v2"): + self.base_url += "/api/v2" + self._username = username + self._password = password + self._timeout = timeout + self._session = requests.Session() + self._logged_in = False + + def _login(self) -> None: + """POST username/password to /auth/login; store the SID cookie.""" + resp = self._session.post( + f"{self.base_url}/auth/login", + data={"username": self._username, "password": self._password}, + timeout=self._timeout, + headers={"Referer": self.base_url}, + ) + resp.raise_for_status() + if resp.text.strip() != "Ok.": + raise RuntimeError(f"qBittorrent login failed: {resp.text.strip()}") + self._logged_in = True + + def _get(self, path: str, **params: Any) -> dict[str, Any]: + """GET with auto re-login on 403.""" + if not self._logged_in: + self._login() + url = f"{self.base_url}{path}" + resp = self._session.get(url, params=params, timeout=self._timeout) + if resp.status_code == 403: + self._logged_in = False + self._login() + resp = self._session.get(url, params=params, timeout=self._timeout) + resp.raise_for_status() + return resp.json() + + def maindata(self) -> dict[str, Any]: + """Fetch /sync/maindata — returns server_state + torrents dict. + + server_state contains: dl_info_speed (bytes/s), up_info_speed (bytes/s), etc. + torrents is a dict of {hash: {name, state, progress, ...}}. + """ + return self._get("/sync/maindata") +``` + +**Endpoints used (all from `/sync/maindata` — single call covers all three widgets):** + +| Widget | Data extracted from `maindata()` | +|---|---| +| **totals** | `len(response["torrents"])` — count of all listed torrents | +| **active** | `filter(t for t in response["torrents"].values() if t["state"] in {"downloading","uploading"})` | +| **speed** | `response["server_state"]["dl_info_speed"]` + `["up_info_speed"]` (current instant speed, appended to store) | + +**Note:** `/api/v2/transfer/info` is NOT needed (totals = item count per Q2, not transfer bytes). + +### 2.4 Widget source adapter — `QbittorrentWidgetSource` + +Lives in `widgets/sources.py`, implements `WidgetSource.fetch(service, widget_kind, config)`. +Resolves the client inline from `ServiceRecord` (like `PrometheusWidgetSource` does — no new +dependency-injection helper needed). + +```python +class QbittorrentWidgetSource: + """Fetch qBittorrent data for totals, active, and speed widgets.""" + + async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]: + try: + if service is None: + return {"error": "qBittorrent widget is missing its service"} + base_url = str(service.config.get("base_url") or "") + username = str(service.secrets.get("username") or "") + password = str(service.secrets.get("password") or "") + timeout = int(service.config.get("timeout_seconds") or 10) + if not base_url or not username or not password: + return {"error": "qBittorrent service is missing base_url, username, or password"} + + client = QbittorrentClient(base_url, username, password, timeout) + data = await asyncio.wait_for(asyncio.to_thread(client.maindata), timeout=timeout) + server_state = data.get("server_state", {}) + torrents = data.get("torrents", {}) + + if widget_kind == "totals": + # Q2: count of listed items, broken down by state + by_state: dict[str, int] = {} + for t in torrents.values(): + state = str(t.get("state", "unknown")) + by_state[state] = by_state.get(state, 0) + 1 + return {"total": len(torrents), "by_state": by_state} + + if widget_kind == "active": + # Q3: downloading or uploading only + active = [ + {"name": t.get("name"), "state": t.get("state"), + "size": t.get("size"), "progress": t.get("progress"), + "dl_speed": t.get("dlspeed"), "up_speed": t.get("upspeed")} + for t in torrents.values() + if str(t.get("state", "")) in {"downloading", "uploading"} + ] + return {"torrents": active} + + if widget_kind == "speed": + # Q1: append sample + return window as {series} shape + # matching PrometheusChartWidget's expected format + dl = int(server_state.get("dl_info_speed", 0)) + up = int(server_state.get("up_info_speed", 0)) + ts = int(time.time()) + store = QbittorrentSampleStore() + store.append(service.id, ts, dl, up) + samples = store.window(service.id) + series = [ + {"label": "download", + "points": [{"t": s["ts"] * 1000, "v": s["dl_speed"]} for s in samples]}, + {"label": "upload", + "points": [{"t": s["ts"] * 1000, "v": s["up_speed"]} for s in samples]}, + ] + return {"series": series} + + return {"error": f"Unknown qBittorrent widget kind: {widget_kind}"} + except asyncio.TimeoutError: + return {"error": "qBittorrent data fetch timed out"} + except Exception as exc: + logger.exception("qbittorrent adapter failed") + return {"error": f"qBittorrent fetch failed: {exc}"} +``` + +**Registered in `SERVICE_ADAPTERS`:** + +```python +SERVICE_ADAPTERS: dict[str, WidgetSource] = { + "prometheus": PrometheusWidgetSource(), + "qbittorrent": QbittorrentWidgetSource(), # NEW + "alertmanager": AlertmanagerWidgetSource(), + "jellyfin": JellyfinWidgetSource(), + "ssh_tasks": SshTaskWidgetSource(), +} +``` + +### 2.5 Integration registration — `integrations/qbittorrent.py` + +Models config + secret schema on `prometheus.py`. + +```python +class QbittorrentConfig(ServiceConfigBase): + base_url: ServiceBaseUrl + timeout_seconds: int = 10 + + +# No per-widget config needed for any of the three kinds +# (all derive from the service connection). +class QbittorrentWidgetConfig(WidgetConfigBase): + pass + + +DEFINITION = ServiceDefinition( + service_type="qbittorrent", + name="qBittorrent", + description="Torrent client activity, speeds, and item counts.", + config_model=QbittorrentConfig, + secret_fields=[ + SecretField(key="username", label="Username", required=True), + SecretField(key="password", label="Password", required=True, helper="Stored encrypted"), + ], + widget_kinds=[ + widget_kind(kind="totals", name="Totals", + description="Count of all listed torrents, broken down by state.", + model_cls=QbittorrentWidgetConfig, default_config={}, + refresh_interval_ms=30_000), + widget_kind(kind="active", name="Active torrents", + description="Torrents currently downloading or uploading.", + model_cls=QbittorrentWidgetConfig, default_config={}, + refresh_interval_ms=15_000), + widget_kind(kind="speed", name="Speed chart", + description="Live download/upload speed over a short window.", + model_cls=QbittorrentWidgetConfig, default_config={}, + refresh_interval_ms=5_000), + ], +) +``` + +**Registered in `integrations/registry.py`:** + +```python +from media_library_viewer_api.integrations.qbittorrent import DEFINITION as QBITTORRENT +SERVICE_DEFINITIONS["qbittorrent"] = QBITTORRENT +``` + +### 2.6 Cascade-delete wiring + +`SettingsStore.delete_service` gains a harness call after the service row is +deleted: + +```python +def delete_service(self, service_id: str) -> None: + self.init_schema() + with self.connect() as conn: + # existing widget cascade ... + conn.execute("DELETE FROM services WHERE id = ?", (service_id,)) + # NEW: cascade-delete harness-managed data + try: + from media_library_viewer_api.services.service_data import get_service_data_harness + get_service_data_harness().cascade_delete(service_id) + except Exception: + logger.exception("harness cascade-delete failed for service %s", service_id) +``` + +The try/except guard prevents a harness failure from blocking service deletion +(data cleanup is best-effort; the service row is already gone). + +--- + +## 3. MediaIndex migration design (load-bearing) + +### 3.1 Schema migration: add `service_id` column + +The `init_schema` method's `CREATE TABLE IF NOT EXISTS` gains the new column. +For existing databases, a migration adds it: + +```sql +-- In MEDIA_INDEX_CONCERN.migrations (runs via harness on startup): +-- The init_schema already creates the table for new installs WITH service_id. +-- This migration handles existing DBs that lack the column. + +-- media_index_impl.py init_schema: add service_id to the CREATE TABLE columns. +-- Harness migration (runs on existing DBs): +ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT ''; +``` + +The `DEFAULT ''` backfills all existing rows to empty string (the legacy +sentinel — see §3.3 for how this is resolved). + +### 3.2 `replace_items` — scoped delete + insert + +```python +def replace_items(self, rows: Iterable[dict[str, Any]], service_id: str = "") -> int: + self.init_schema() + row_list = list(rows) + # ... columns list gains "service_id" ... + with self.connect() as conn: + # SCOPED: only delete this service's rows + conn.execute("DELETE FROM media_items WHERE service_id = ?", (service_id,)) + conn.executemany( + f"INSERT OR REPLACE INTO media_items ({','.join(columns)}) VALUES ({placeholders})", + [[service_id] + [row.get(column) for column in columns_without_service_id] for row in row_list], + ) + # ... metadata ... + return len(row_list) +``` + +### 3.3 `query` — scoped filter + +```python +def query(self, service_id: str = "", ...) -> tuple[list[dict[str, Any]], int]: + # ... existing where clauses ... + # Add service_id filter: if non-empty, scope; if empty (legacy), show all + # (backward-compatibility for the period before multi-instance is wired in UI) + if service_id: + where.append("service_id = ?") + params.append(service_id) + # ... rest unchanged ... +``` + +**Backfill semantics:** existing rows get `service_id = ''` (empty string). When +`service_id` is empty string in the query, the filter is skipped, so the Media +page shows all items (backward-compatible behavior). When a specific Jellyfin +service triggers a rebuild, `replace_items(rows, service_id=that_service)` scopes +the delete + insert. New builds set the real service_id; legacy rows remain +visible until a rebuild replaces them. + +### 3.4 `build_media_index` — thread service_id + +`build_media_index` already receives no `service_id` today. Add it as a +parameter and pass it through to `replace_items`: + +```python +def build_media_index( + client, user_id, libraries, index=None, page_size=500, + media_root="", fallback_prefix="", + progress_callback=None, should_cancel=None, + service_id: str = "", # NEW +) -> int: + # ... existing logic ... + # Pass service_id to replace_items: + processed_total = index.replace_items(normalized_rows, service_id=service_id) +``` + +### 3.5 Worker — already threads `service_id`, just pass it to `build_media_index` + +The worker's `run_build(final_index_path, staging_index_path, service_id="")` +already receives `service_id` from argparse. The only change: pass it to +`build_media_index(..., service_id=service_id)`. + +### 3.6 Media router — thread `service_id` into `query_media` + +`query_media` already resolves `client: JellyfinClient = Depends(get_jellyfin_client)`. +The service_id is available via request query param (the dependency layer resolves +it from `?jellyfin_service_id=...`). Add it to the query call: + +```python +@router.get("/query") +def query_media( + ..., + jellyfin_service_id: str | None = None, # already available pattern + index: MediaIndex = Depends(get_media_index), +) -> dict[str, Any]: + ... + rows, total = index.query(service_id=jellyfin_service_id or "", ...) +``` + +### 3.7 MediaIndex concern registration + +```python +MEDIA_INDEX_CONCERN = StorageConcern( + concern_key="media_index", + db_filename="media_index.sqlite", # SAME FILE, unchanged path + migrations=[ + "ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT ''", + # This ALTER is idempotent-safe: init_schema creates the table WITH + # service_id for new installs; this migration adds it to existing DBs. + # SQLite ALTER TABLE ADD COLUMN is a no-op if the column already exists + # (we guard with a PRAGMA check in run_migrations, or catch the error). + ], + tables=["media_items"], +) +``` + +**IMPORTANT — ALTER TABLE idempotency:** SQLite raises an error if the column +already exists. The `run_migrations` method should catch this per-statement or +pre-check via `PRAGMA table_info`. Design choice: wrap each migration in a +try/except for "duplicate column name" errors: + +```python +def run_migrations(self) -> None: + for concern in self._concerns.values(): + path = self.db_path(concern.concern_key) + path.parent.mkdir(parents=True, exist_ok=True) + with sqlite3.connect(path, timeout=30) as conn: + conn.execute("PRAGMA journal_mode=WAL") + for stmt in concern.migrations: + try: + conn.executescript(stmt) + except sqlite3.OperationalError as exc: + if "duplicate column name" not in str(exc).lower(): + raise +``` + +### 3.8 File location confirmation + +`DEFAULT_INDEX_PATH = Path(".cache/media_library_viewer/media_index.sqlite")` — +unchanged. The harness `base_dir` defaults to `Path(".cache/media_library_viewer")` +(the same parent). The `db_filename = "media_index.sqlite"` matches. No data move. + +--- + +## 4. Frontend design + +### 4.1 Three widget components + +All three live in `frontend/src/widgets/`, modeled on existing patterns +(`AlertmanagerAlertsWidget`, `PrometheusChartWidget`, `MetricCard`). + +**`QbittorrentTotalsWidget.tsx`** — MetricCard-style count tile: + +```tsx +// Renders {total: number, by_state: {...}} from useWidgetData. +// Uses SectionCard + numeric display (like BackupsWidget / MetricCard). +// Shows total count prominently + state breakdown badges. +``` + +**`QbittorrentActiveTorrentsWidget.tsx`** — active torrents list: + +```tsx +// Renders {torrents: [{name, state, size, progress, dl_speed, up_speed}]}. +// Uses SectionCard + a compact list/table (DataTable or manual Table rows). +// Shows name, state badge, progress bar, speeds. Max 10 rows with scroll. +``` + +**`QbittorrentSpeedWidget.tsx`** — speed chart reusing Change A's renderer: + +```tsx +// Renders {series: [{label, points:[{t,v}]}]} — IDENTICAL to PrometheusChartWidget. +// Two options: +// (a) Import PrometheusChartWidget directly and pass props (if its props accept +// the series externally rather than via useWidgetData). +// (b) Extract the recharts rendering into a shared +// component that both PrometheusChartWidget and QbittorrentSpeedWidget use. +// +// RECOMMENDED: option (b) — extract a shared LineSeriesChart component (~40 lines) +// into frontend/src/components/LineSeriesChart.tsx. Both widgets call useWidgetData +// independently (different refresh intervals) but share the renderer. +``` + +**`LineSeriesChart.tsx`** (shared renderer extraction): + +```tsx +// Extracts: mergeSeries, formatTime, CHART_COLORS, and the +// + JSX from PrometheusChartWidget. +// Props: { series: ChartSeries[], height?: number } +// PrometheusChartWidget becomes a thin wrapper: useWidgetData → +// QbittorrentSpeedWidget: same pattern, different refresh interval (5s vs 60s). +``` + +This keeps the recharts rendering in ONE place (no duplication) while letting each +widget own its polling lifecycle. + +### 4.2 Frontend registry binding (`integrations/registry.ts`) + +```typescript +qbittorrent: { + serviceType: "qbittorrent", + name: "qBittorrent", + description: "Torrent client activity, speeds, and item counts.", + widgets: [ + { kind: "totals", name: "Totals", description: "...", + refreshIntervalMs: 30_000, defaultConfig: {}, + configSchema: { type: "object", properties: {}, required: [] }, + component: QbittorrentTotalsWidget }, + { kind: "active", name: "Active torrents", description: "...", + refreshIntervalMs: 15_000, defaultConfig: {}, + configSchema: { type: "object", properties: {}, required: [] }, + component: QbittorrentActiveTorrentsWidget }, + { kind: "speed", name: "Speed chart", description: "...", + refreshIntervalMs: 5_000, defaultConfig: {}, + configSchema: { type: "object", properties: {}, required: [] }, + component: QbittorrentSpeedWidget }, + ], +}, +``` + +### 4.3 Types + +No new TypeScript types needed for widget payloads — data flows through the +existing `WidgetDataResponse` + `useWidgetData` polling. The series shape +(`{series:[{label,points:[{t,v}]}]}`) is already used by `PrometheusChartWidget`. + +--- + +## 5. Tests + +### 5.1 Backend tests + +| Test file | Coverage | +|---|---| +| `backend/tests/test_service_data.py` (NEW) | Harness: register, run_migrations (creates tables), cascade_delete (removes rows by service_id), migration idempotency (ALTER doesn't crash on re-run) | +| `backend/tests/test_qbittorrent_store.py` (NEW) | Store: append + prune (MAX_SAMPLES cap), window (returns samples in order), service_id isolation (two services don't cross-contaminate) | +| `backend/tests/test_qbittorrent_client.py` (NEW) | Client: login flow (POST /auth/login → Ok.), cookie reuse, 403 → re-login, maindata parsing, timeout handling | +| `backend/tests/test_widgets.py` (extend) | `QbittorrentWidgetSource`: totals (counts all torrents), active (filters state), speed (appends sample + returns {series}), missing-service error, timeout error | + +### 5.2 Frontend tests + +| Test file | Coverage | +|---|---| +| `QbittorrentTotalsWidget.test.tsx` (NEW) | Loading skeleton, error alert, rendered count + state badges | +| `QbittorrentActiveTorrentsWidget.test.tsx` (NEW) | Loading, error, rendered torrent rows | +| `QbittorrentSpeedWidget.test.tsx` (NEW) | Loading, error, rendered chart (series present) | +| `LineSeriesChart.test.tsx` (NEW) | Renders lines from series data, empty state | + +### 5.3 Existing MediaIndex tests must stay green + +`backend/tests/test_media_index.py` exercises `replace_items`, `query`, `status`. +After migration, these call with the new `service_id=""` default (backward-compatible). +The tests pass unchanged because empty-string service_id shows all rows. + +--- + +## 6. Slice plan (for tasks.md) + +Four slices, each ≤400 changed lines, each leaving `pytest` + `npm run build` + +`npm run lint` green. Slices 1–2 prove the harness; Slice 3 migrates MediaIndex; +Slice 4 wires cascade-delete end-to-end. + +### Slice 1: Harness + QbittorrentSampleStore + QbittorrentClient + integration (~280–350 lines) + +**Files:** + +- `backend/src/media_library_viewer_api/services/service_data.py` (NEW — harness + StorageConcern) +- `backend/src/media_library_viewer_api/services/qbittorrent_store.py` (NEW — store + concern) +- `backend/src/media_library_viewer_api/clients/qbittorrent.py` (NEW — client) +- `backend/src/media_library_viewer_api/integrations/qbittorrent.py` (NEW — definition) +- `backend/src/media_library_viewer_api/integrations/registry.py` (MODIFY — add qbittorrent) +- `backend/src/media_library_viewer_api/main.py` (MODIFY — call harness init in lifespan) +- `backend/tests/test_service_data.py` (NEW) +- `backend/tests/test_qbittorrent_store.py` (NEW) +- `backend/tests/test_qbittorrent_client.py` (NEW) + +**Exit gate:** harness creates tables + runs migrations + cascade_delete works in +tests. qBit client login/maindata tested with mocked HTTP. No frontend changes yet. + +### Slice 2: Widget adapter + frontend widgets + registry binding (~320–400 lines) + +**Files:** + +- `backend/src/media_library_viewer_api/widgets/sources.py` (MODIFY — add QbittorrentWidgetSource + SERVICE_ADAPTERS entry) +- `backend/tests/test_widgets.py` (EXTEND — qBit adapter tests) +- `frontend/src/components/LineSeriesChart.tsx` (NEW — shared renderer extracted from PrometheusChartWidget) +- `frontend/src/widgets/PrometheusChartWidget.tsx` (MODIFY — use LineSeriesChart) +- `frontend/src/widgets/QbittorrentTotalsWidget.tsx` (NEW) +- `frontend/src/widgets/QbittorrentActiveTorrentsWidget.tsx` (NEW) +- `frontend/src/widgets/QbittorrentSpeedWidget.tsx` (NEW) +- `frontend/src/widgets/index.ts` (MODIFY — barrel exports) +- `frontend/src/integrations/registry.ts` (MODIFY — add qbittorrent binding) +- `frontend/src/widgets/__tests__/QbittorrentTotalsWidget.test.tsx` (NEW) +- `frontend/src/widgets/__tests__/QbittorrentActiveTorrentsWidget.test.tsx` (NEW) +- `frontend/src/widgets/__tests__/QbittorrentSpeedWidget.test.tsx` (NEW) +- `frontend/src/components/__tests__/LineSeriesChart.test.tsx` (NEW) + +**Exit gate:** qBit widgets render loading/error/data states; speed widget shows +a recharts line chart from {series}; LineSeriesChart is shared with PrometheusChartWidget. + +### Slice 3: MediaIndex migration onto harness (~200–280 lines) + +**Files:** + +- `backend/src/media_library_viewer_api/services/service_data.py` (MODIFY — register MEDIA_INDEX_CONCERN) +- `backend/src/media_library_viewer_api/services/media_index_impl.py` (MODIFY — add service_id to schema, replace_items, query) +- `backend/src/media_library_viewer_api/services/media_index.py` (re-export unchanged) +- `backend/src/media_library_viewer_api/workers/media_index_worker.py` (MODIFY — pass service_id to build_media_index) +- `backend/src/media_library_viewer_api/routers/media.py` (MODIFY — thread jellyfin_service_id into query_media) +- `backend/tests/test_media_index.py` (EXTEND — service_id-scoped replace_items + query) + +**Exit gate:** all existing MediaIndex tests pass unchanged (empty-string default); +new tests verify scoped delete/insert/query. Media page works identically. + +### Slice 4: Cascade-delete end-to-end + integration test (~80–120 lines) + +**Files:** + +- `backend/src/media_library_viewer_api/services/settings_store.py` (MODIFY — call harness.cascade_delete in delete_service) +- `backend/tests/test_services.py` (EXTEND — verify cascade-delete removes qBit samples when service is deleted) + +**Exit gate:** deleting a qBittorrent service removes its speed samples; deleting +a Jellyfin service removes its media items. + +--- + +## 7. Key design decisions summary + +| # | Decision | Rationale | +|---|---|---| +| D1 | Harness is lifecycle-only (migrations, service_id, cascade) | n=2 justifies abstraction, but data-ops differ wildly (append/window vs replace_all/query). Generalize only the shared lifecycle. | +| D2 | Per-concern DB files via harness `base_dir` + `db_filename` | media_index.sqlite stays put (no data move); qbittorrent.db is new. Separate writers. | +| D3 | `run_migrations` catches "duplicate column name" per-statement | SQLite ALTER TABLE ADD COLUMN is not idempotent; migrations must not crash on re-run. | +| D4 | Speed widget extracts a shared `LineSeriesChart` component | Both PrometheusChartWidget and QbittorrentSpeedWidget need the same recharts renderer; extract it once (DRY) rather than duplicating or tightly coupling. | +| D5 | qBit adapter resolves client inline from ServiceRecord (like PrometheusWidgetSource) | No new dependency-injection helper; the adapter gets config+secrets from the service record. | +| D6 | MediaIndex backfill uses `service_id = ''` (empty string) sentinel | Backward-compatible: existing tests + Media page work unchanged; empty-string queries skip the filter (show all). Real service_ids overwrite on next rebuild. | +| D7 | `replace_items` deletes `WHERE service_id = ?` instead of all | Fixes the existing global-clear bug (building for one Jellyfin wipes others). | +| D8 | Cascade-delete is best-effort (try/except) in delete_service | Harness failure must not block service deletion; data cleanup is non-critical. | +| D9 | Speed sample timestamps multiplied by 1000 for frontend | PrometheusChartWidget expects `t` in milliseconds (JS epoch); SQLite stores Unix seconds. | +| D10 | MAX_SAMPLES = 120 cap with per-append prune | ~2 min history at 1s poll; bounded DB growth; single-statement prune. | + +--- + +## 8. Risks and mitigations + +| Risk | Mitigation | +|---|---| +| **MediaIndex migration breaks existing tests.** | `service_id` defaults to `""`; empty-string queries skip the filter → all rows visible. Existing tests pass unchanged. | +| **ALTER TABLE fails on fresh installs** where init_schema already created the column. | `run_migrations` catches "duplicate column name" errors per-statement. | +| **Harness lazy-init hides migration failures.** | `run_migrations` runs on first access (startup lifespan); failures raise (not swallowed) except the expected duplicate-column case. | +| **Speed widget poll interval (5s) too aggressive.** | 5s is the default; configurable via widget config if needed. MAX_SAMPLES cap bounds storage. | +| **qBit cookie expiry between polls.** | Client re-logins on 403 transparently; persistent auth failure surfaces as widget error state. | +| **LineSeriesChart extraction breaks PrometheusChartWidget.** | Slice 2 includes tests for both; extraction is mechanical (move JSX + helpers, pass series as prop). | +| **Review budget (>400 lines).** | Four slices, each ≤400 lines. Slices 1–2 are independently shippable (qBit works without MediaIndex migration). | + +--- + +## 9. Data flow diagrams + +### 9.1 Speed widget data flow (InService path) + +``` +Dashboard poll (useWidgetData, 5s) + └► GET /api/widgets/instances/{id}/data + └► QbittorrentWidgetSource.fetch(service, "speed", {}) + ├► QbittorrentClient.maindata() + │ └► /api/v2/sync/maindata → {server_state:{dl_info_speed, up_info_speed}} + ├► QbittorrentSampleStore.append(service.id, ts, dl, up) + │ └► INSERT + prune (keep 120) + ├► QbittorrentSampleStore.window(service.id) + │ └► SELECT ts, dl_speed, up_speed → [{ts, dl_speed, up_speed}] + └► return {series: [{label:"download", points:[{t,v}]}, {label:"upload", points:[{t,v}]}]} + └► Frontend: QbittorrentSpeedWidget → +``` + +### 9.2 Cascade-delete data flow + +``` +DELETE /api/services/instances/{id} + └► SettingsStore.delete_service(id) + ├► DELETE FROM dashboard_widgets WHERE service_id = ? + ├► DELETE FROM services WHERE id = ? + └► ServiceDataHarness.cascade_delete(id) + ├► DELETE FROM qbittorrent_speed_samples WHERE service_id = ? + └► DELETE FROM media_items WHERE service_id = ? +```