manage/openspec/changes/archive/2026-07-09-service-storage-harness/design.md

SDD Design: Service Storage Harness (with qBittorrent widgets + MediaIndex migration)

Change: service-storage-harness Phase: design Date: 2026-07-09

Grounded in proposal.md (Q1–Q5 resolved) and the already-archived prometheus-direct-charting change. No source changes in this phase.

0. Source findings (read before anything else)

The proposal was written against a stale project map. Reading actual source surfaced three findings that shape the design. Trust source, not the map.

0.1 The media worker ALREADY threads service_id — but it is NOT persisted in rows

workers/media_index_worker.py already accepts --service-id via argparse (run_build(final_index_path, staging_index_path, service_id="")), and _resolve_jellyfin(service_id) resolves the Jellyfin client + user_id from the settings store. The media router's _start_worker(index, service_id) and post_build_index already pass jellyfin_service_id through to the worker.

But: service_id is used ONLY to pick the Jellyfin connection. It is never stored in the media_items table (which has no service_id column). The migration must close this gap: replace_items must scope its DELETE+INSERT by service_id, and query must filter by it. The plumbing to get service_id into the worker already exists — only the storage layer is missing.

0.2 media_items is cleared globally on every build

replace_items does DELETE FROM media_items (no WHERE clause). This means a build for Jellyfin instance A wipes instance B's rows. After migration this becomes DELETE FROM media_items WHERE service_id = ?.

0.3 The delete_service cascade already exists — the harness hooks into it

SettingsStore.delete_service already cascade-deletes dashboard_widgets WHERE service_id = ? (with a PRAGMA-guarded column check). The harness cascade-delete hooks into the same place: after the service row is deleted, iterate registered concerns and delete that service_id from each owned table.

0.4 DB path is a module constant, not in config.py

DEFAULT_INDEX_PATH = Path(".cache/media_library_viewer/media_index.sqlite") in media_index_impl.py. The file stays at this path (per the locked topology decision). The harness must not move it.

---

1. Architecture overview

┌──────────────────────────────────────────────────────────────────┐
│                     ServiceDataHarness                            │
│  (services/service_data.py — LIFECYCLE ONLY)                     │
│                                                                  │
│  • register_concern(db_filename, migrations[], tables[],         │
│                     service_id_column="service_id")              │
│  • run_migrations() — on startup, per concern DB                 │
│  • cascade_delete(service_id) — iterate concerns, DELETE rows    │
│  • connect(db_filename) → sqlite3.Connection (per-concern)       │
└──────────────┬───────────────────────────┬───────────────────────┘
               │                           │
    ┌──────────▼──────────┐      ┌─────────▼──────────────┐
    │ QbittorrentStore     │      │ MediaIndex             │
    │ (services/           │      │ (services/             │
    │  qbittorrent_store)  │      │  media_index_impl)     │
    │                      │      │                        │
    │ qbittorrent.db       │      │ media_index.sqlite     │
    │  └ qbittorrent_      │      │  └ media_items         │
    │    speed_samples     │      │    (+ service_id col)  │
    │    (service_id, ts,  │      │  └ index_metadata      │
    │     dl_speed,        │      │                        │
    │     up_speed)        │      │ bespoke: replace_items │
    │                      │      │ (scoped), query        │
    │ bespoke: append,     │      │ (scoped), status       │
    │ window, prune        │      │                        │
    └──────────────────────┘      └────────────────────────┘
               │
    ┌──────────▼──────────┐
    │ QbittorrentClient    │
    │ (clients/            │
    │  qbittorrent)        │
    │                      │
    │ login → cookie       │
    │ sync/maindata        │
    │ (totals + active +   │
    │  speeds)             │
    └──────────────────────┘

Key constraints carried from the proposal:

• Harness is lifecycle-only: migrations, service_id scoping, cascade-delete. No generic value table, no generic CRUD (D2).
• Per-concern DB files: media_index.sqlite stays put; new qbittorrent.db (D4).
• MediaIndex migration is sequenced after harness + qBit are proven (D3).
• qBit speed chart reuses the PrometheusChartWidget recharts renderer fed from an InService data path returning {series} (Q1).
• Totals = count of listed items, NOT transfer bytes (Q2).
• Active = state downloading|uploading (Q3). N instances (Q4). Username/password → cookie (Q5).

---

2. Backend design

2.1 ServiceDataHarness (services/service_data.py)

A lifecycle-only registry of storage concerns. Each concern declares its own DB filename, ordered migrations, owned tables, and the column used for service scoping.

@dataclass(frozen=True)
class StorageConcern:
    """A per-integration storage namespace registered with the harness."""

    concern_key: str               # e.g. "qbittorrent", "media_index"
    db_filename: str               # e.g. "qbittorrent.db", "media_index.sqlite"
    migrations: list[str]          # ordered CREATE/ALTER statements (idempotent)
    tables: list[str]              # tables owned by this concern (for cascade)
    service_id_column: str = "service_id"


class ServiceDataHarness:
    """Lifecycle-only registry of per-concern storage.

    Owns: DB provisioning, per-concern migrations, service_id cascade-delete.
    Does NOT own: data operations (each store keeps bespoke append/window/query/etc.).
    """

    def __init__(self, base_dir: Path) -> None:
        self._base_dir = Path(base_dir)
        self._concerns: dict[str, StorageConcern] = {}

    def register(self, concern: StorageConcern) -> None:
        """Register a storage concern. Called at module import / startup."""
        self._concerns[concern.concern_key] = concern

    def db_path(self, concern_key: str) -> Path:
        """Return the absolute path to a concern's DB file."""
        concern = self._concerns[concern_key]
        return self._base_dir / concern.db_filename

    def connect(self, concern_key: str) -> sqlite3.Connection:
        """Open a WAL-mode connection to a concern's DB."""
        path = self.db_path(concern_key)
        path.parent.mkdir(parents=True, exist_ok=True)
        conn = sqlite3.connect(path, timeout=30)
        conn.row_factory = sqlite3.Row
        conn.execute("PRAGMA journal_mode=WAL")
        conn.execute("PRAGMA busy_timeout=30000")
        return conn

    def run_migrations(self) -> None:
        """Run pending migrations for every registered concern."""
        for concern in self._concerns.values():
            path = self.db_path(concern.concern_key)
            path.parent.mkdir(parents=True, exist_ok=True)
            with sqlite3.connect(path, timeout=30) as conn:
                conn.execute("PRAGMA journal_mode=WAL")
                conn.executescript(";".join(concern.migrations))

    def cascade_delete(self, service_id: str) -> None:
        """Delete all rows for a service_id across every concern's tables.

        Called from SettingsStore.delete_service after the service row is removed.
        """
        for concern in self._concerns.values():
            col = concern.service_id_column
            with sqlite3.connect(self.db_path(concern.concern_key), timeout=30) as conn:
                for table in concern.tables:
                    cols = {row[1] for row in conn.execute(f"PRAGMA table_info({table})").fetchall()}
                    if col in cols:
                        conn.execute(f"DELETE FROM {table} WHERE {col} = ?", (service_id,))

Module-level singleton + registration:

_HARNESS: ServiceDataHarness | None = None

def get_service_data_harness() -> ServiceDataHarness:
    global _HARNESS
    if _HARNESS is None:
        base_dir = Path(os.environ.get("BACKEND_CACHE_DIR", ".cache/media_library_viewer"))
        _HARNESS = ServiceDataHarness(base_dir)
        # Register built-in concerns (each store module calls register on import)
        _HARNESS.register(QBITTORRENT_CONCERN)
        _HARNESS.register(MEDIA_INDEX_CONCERN)
        _HARNESS.run_migrations()
    return _HARNESS

Startup hook: called from main.py lifespan alongside get_settings_store().ensure_defaults(). The harness is lazy-initialized on first access (like SettingsStore), so tests can override the base dir via env.

2.2 QbittorrentSampleStore (services/qbittorrent_store.py)

Speed-sample storage for the qBit speed widget. Registered as a concern with the harness.

QBITTORRENT_CONCERN = StorageConcern(
    concern_key="qbittorrent",
    db_filename="qbittorrent.db",
    migrations=[
        """
        CREATE TABLE IF NOT EXISTS qbittorrent_speed_samples (
            service_id TEXT NOT NULL,
            ts INTEGER NOT NULL,
            dl_speed INTEGER NOT NULL DEFAULT 0,
            up_speed INTEGER NOT NULL DEFAULT 0
        );
        CREATE INDEX IF NOT EXISTS idx_qbit_samples_service_ts
            ON qbittorrent_speed_samples(service_id, ts);
        """
    ],
    tables=["qbittorrent_speed_samples"],
)

MAX_SAMPLES = 120  # ~2 min at 1s poll, ~4 min at 2s poll


class QbittorrentSampleStore:
    """Bespoke speed-sample store for qBittorrent widgets."""

    def __init__(self, harness: ServiceDataHarness | None = None) -> None:
        self._harness = harness or get_service_data_harness()

    def append(self, service_id: str, ts: int, dl_speed: int, up_speed: int) -> None:
        """Append a sample and prune old entries beyond MAX_SAMPLES."""
        with self._harness.connect("qbittorrent") as conn:
            conn.execute(
                "INSERT INTO qbittorrent_speed_samples (service_id, ts, dl_speed, up_speed) VALUES (?, ?, ?, ?)",
                (service_id, ts, dl_speed, up_speed),
            )
            # Prune: keep only the most recent MAX_SAMPLES rows for this service
            conn.execute(
                """DELETE FROM qbittorrent_speed_samples
                   WHERE service_id = ? AND ts NOT IN (
                       SELECT ts FROM qbittorrent_speed_samples
                       WHERE service_id = ?
                       ORDER BY ts DESC LIMIT ?
                   )""",
                (service_id, service_id, MAX_SAMPLES),
            )

    def window(self, service_id: str, since_ts: int | None = None) -> list[dict[str, Any]]:
        """Return all samples for a service since a timestamp (or all if None)."""
        with self._harness.connect("qbittorrent") as conn:
            if since_ts is not None:
                rows = conn.execute(
                    "SELECT ts, dl_speed, up_speed FROM qbittorrent_speed_samples WHERE service_id = ? AND ts >= ? ORDER BY ts ASC",
                    (service_id, since_ts),
                ).fetchall()
            else:
                rows = conn.execute(
                    "SELECT ts, dl_speed, up_speed FROM qbittorrent_speed_samples WHERE service_id = ? ORDER BY ts ASC",
                    (service_id,),
                ).fetchall()
        return [{"ts": r[0], "dl_speed": r[1], "up_speed": r[2]} for r in rows]

2.3 QbittorrentClient (clients/qbittorrent.py)

Cookie-session HTTP client modeled on JellyfinClient's session pattern.

class QbittorrentClient:
    """Minimal qBittorrent Web API client (read-only: sync/maindata only)."""

    def __init__(self, base_url: str, username: str, password: str, timeout: int = 10) -> None:
        self.base_url = base_url.rstrip("/")
        if not self.base_url.endswith("/api/v2"):
            self.base_url += "/api/v2"
        self._username = username
        self._password = password
        self._timeout = timeout
        self._session = requests.Session()
        self._logged_in = False

    def _login(self) -> None:
        """POST username/password to /auth/login; store the SID cookie."""
        resp = self._session.post(
            f"{self.base_url}/auth/login",
            data={"username": self._username, "password": self._password},
            timeout=self._timeout,
            headers={"Referer": self.base_url},
        )
        resp.raise_for_status()
        if resp.text.strip() != "Ok.":
            raise RuntimeError(f"qBittorrent login failed: {resp.text.strip()}")
        self._logged_in = True

    def _get(self, path: str, **params: Any) -> dict[str, Any]:
        """GET with auto re-login on 403."""
        if not self._logged_in:
            self._login()
        url = f"{self.base_url}{path}"
        resp = self._session.get(url, params=params, timeout=self._timeout)
        if resp.status_code == 403:
            self._logged_in = False
            self._login()
            resp = self._session.get(url, params=params, timeout=self._timeout)
        resp.raise_for_status()
        return resp.json()

    def maindata(self) -> dict[str, Any]:
        """Fetch /sync/maindata — returns server_state + torrents dict.

        server_state contains: dl_info_speed (bytes/s), up_info_speed (bytes/s), etc.
        torrents is a dict of {hash: {name, state, progress, ...}}.
        """
        return self._get("/sync/maindata")

Endpoints used (all from /sync/maindata — single call covers all three widgets):

Widget	Data extracted from maindata()
totals	len(response["torrents"]) — count of all listed torrents
active	filter(t for t in response["torrents"].values() if t["state"] in {"downloading","uploading"})
speed	response["server_state"]["dl_info_speed"] + ["up_info_speed"] (current instant speed, appended to store)

Note: /api/v2/transfer/info is NOT needed (totals = item count per Q2, not transfer bytes).

2.4 Widget source adapter — QbittorrentWidgetSource

Lives in widgets/sources.py, implements WidgetSource.fetch(service, widget_kind, config). Resolves the client inline from ServiceRecord (like PrometheusWidgetSource does — no new dependency-injection helper needed).

class QbittorrentWidgetSource:
    """Fetch qBittorrent data for totals, active, and speed widgets."""

    async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
        try:
            if service is None:
                return {"error": "qBittorrent widget is missing its service"}
            base_url = str(service.config.get("base_url") or "")
            username = str(service.secrets.get("username") or "")
            password = str(service.secrets.get("password") or "")
            timeout = int(service.config.get("timeout_seconds") or 10)
            if not base_url or not username or not password:
                return {"error": "qBittorrent service is missing base_url, username, or password"}

            client = QbittorrentClient(base_url, username, password, timeout)
            data = await asyncio.wait_for(asyncio.to_thread(client.maindata), timeout=timeout)
            server_state = data.get("server_state", {})
            torrents = data.get("torrents", {})

            if widget_kind == "totals":
                # Q2: count of listed items, broken down by state
                by_state: dict[str, int] = {}
                for t in torrents.values():
                    state = str(t.get("state", "unknown"))
                    by_state[state] = by_state.get(state, 0) + 1
                return {"total": len(torrents), "by_state": by_state}

            if widget_kind == "active":
                # Q3: downloading or uploading only
                active = [
                    {"name": t.get("name"), "state": t.get("state"),
                     "size": t.get("size"), "progress": t.get("progress"),
                     "dl_speed": t.get("dlspeed"), "up_speed": t.get("upspeed")}
                    for t in torrents.values()
                    if str(t.get("state", "")) in {"downloading", "uploading"}
                ]
                return {"torrents": active}

            if widget_kind == "speed":
                # Q1: append sample + return window as {series} shape
                # matching PrometheusChartWidget's expected format
                dl = int(server_state.get("dl_info_speed", 0))
                up = int(server_state.get("up_info_speed", 0))
                ts = int(time.time())
                store = QbittorrentSampleStore()
                store.append(service.id, ts, dl, up)
                samples = store.window(service.id)
                series = [
                    {"label": "download",
                     "points": [{"t": s["ts"] * 1000, "v": s["dl_speed"]} for s in samples]},
                    {"label": "upload",
                     "points": [{"t": s["ts"] * 1000, "v": s["up_speed"]} for s in samples]},
                ]
                return {"series": series}

            return {"error": f"Unknown qBittorrent widget kind: {widget_kind}"}
        except asyncio.TimeoutError:
            return {"error": "qBittorrent data fetch timed out"}
        except Exception as exc:
            logger.exception("qbittorrent adapter failed")
            return {"error": f"qBittorrent fetch failed: {exc}"}

Registered in SERVICE_ADAPTERS:

SERVICE_ADAPTERS: dict[str, WidgetSource] = {
    "prometheus": PrometheusWidgetSource(),
    "qbittorrent": QbittorrentWidgetSource(),   # NEW
    "alertmanager": AlertmanagerWidgetSource(),
    "jellyfin": JellyfinWidgetSource(),
    "ssh_tasks": SshTaskWidgetSource(),
}

2.5 Integration registration — integrations/qbittorrent.py

Models config + secret schema on prometheus.py.

class QbittorrentConfig(ServiceConfigBase):
    base_url: ServiceBaseUrl
    timeout_seconds: int = 10


# No per-widget config needed for any of the three kinds
# (all derive from the service connection).
class QbittorrentWidgetConfig(WidgetConfigBase):
    pass


DEFINITION = ServiceDefinition(
    service_type="qbittorrent",
    name="qBittorrent",
    description="Torrent client activity, speeds, and item counts.",
    config_model=QbittorrentConfig,
    secret_fields=[
        SecretField(key="username", label="Username", required=True),
        SecretField(key="password", label="Password", required=True, helper="Stored encrypted"),
    ],
    widget_kinds=[
        widget_kind(kind="totals", name="Totals",
                    description="Count of all listed torrents, broken down by state.",
                    model_cls=QbittorrentWidgetConfig, default_config={},
                    refresh_interval_ms=30_000),
        widget_kind(kind="active", name="Active torrents",
                    description="Torrents currently downloading or uploading.",
                    model_cls=QbittorrentWidgetConfig, default_config={},
                    refresh_interval_ms=15_000),
        widget_kind(kind="speed", name="Speed chart",
                    description="Live download/upload speed over a short window.",
                    model_cls=QbittorrentWidgetConfig, default_config={},
                    refresh_interval_ms=5_000),
    ],
)

Registered in integrations/registry.py:

from media_library_viewer_api.integrations.qbittorrent import DEFINITION as QBITTORRENT
SERVICE_DEFINITIONS["qbittorrent"] = QBITTORRENT

2.6 Cascade-delete wiring

SettingsStore.delete_service gains a harness call after the service row is deleted:

def delete_service(self, service_id: str) -> None:
    self.init_schema()
    with self.connect() as conn:
        # existing widget cascade ...
        conn.execute("DELETE FROM services WHERE id = ?", (service_id,))
    # NEW: cascade-delete harness-managed data
    try:
        from media_library_viewer_api.services.service_data import get_service_data_harness
        get_service_data_harness().cascade_delete(service_id)
    except Exception:
        logger.exception("harness cascade-delete failed for service %s", service_id)

The try/except guard prevents a harness failure from blocking service deletion (data cleanup is best-effort; the service row is already gone).

---

3. MediaIndex migration design (load-bearing)

3.1 Schema migration: add service_id column

The init_schema method's CREATE TABLE IF NOT EXISTS gains the new column. For existing databases, a migration adds it:

-- In MEDIA_INDEX_CONCERN.migrations (runs via harness on startup):
-- The init_schema already creates the table for new installs WITH service_id.
-- This migration handles existing DBs that lack the column.

-- media_index_impl.py init_schema: add service_id to the CREATE TABLE columns.
-- Harness migration (runs on existing DBs):
ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT '';

The DEFAULT '' backfills all existing rows to empty string (the legacy sentinel — see §3.3 for how this is resolved).

3.2 replace_items — scoped delete + insert

def replace_items(self, rows: Iterable[dict[str, Any]], service_id: str = "") -> int:
    self.init_schema()
    row_list = list(rows)
    # ... columns list gains "service_id" ...
    with self.connect() as conn:
        # SCOPED: only delete this service's rows
        conn.execute("DELETE FROM media_items WHERE service_id = ?", (service_id,))
        conn.executemany(
            f"INSERT OR REPLACE INTO media_items ({','.join(columns)}) VALUES ({placeholders})",
            [[service_id] + [row.get(column) for column in columns_without_service_id] for row in row_list],
        )
        # ... metadata ...
    return len(row_list)

3.3 query — scoped filter

def query(self, service_id: str = "", ...) -> tuple[list[dict[str, Any]], int]:
    # ... existing where clauses ...
    # Add service_id filter: if non-empty, scope; if empty (legacy), show all
    # (backward-compatibility for the period before multi-instance is wired in UI)
    if service_id:
        where.append("service_id = ?")
        params.append(service_id)
    # ... rest unchanged ...

Backfill semantics: existing rows get service_id = '' (empty string). When service_id is empty string in the query, the filter is skipped, so the Media page shows all items (backward-compatible behavior). When a specific Jellyfin service triggers a rebuild, replace_items(rows, service_id=that_service) scopes the delete + insert. New builds set the real service_id; legacy rows remain visible until a rebuild replaces them.

3.4 build_media_index — thread service_id

build_media_index already receives no service_id today. Add it as a parameter and pass it through to replace_items:

def build_media_index(
    client, user_id, libraries, index=None, page_size=500,
    media_root="", fallback_prefix="",
    progress_callback=None, should_cancel=None,
    service_id: str = "",   # NEW
) -> int:
    # ... existing logic ...
    # Pass service_id to replace_items:
    processed_total = index.replace_items(normalized_rows, service_id=service_id)

3.5 Worker — already threads service_id, just pass it to build_media_index

The worker's run_build(final_index_path, staging_index_path, service_id="") already receives service_id from argparse. The only change: pass it to build_media_index(..., service_id=service_id).

3.6 Media router — thread service_id into query_media

query_media already resolves client: JellyfinClient = Depends(get_jellyfin_client). The service_id is available via request query param (the dependency layer resolves it from ?jellyfin_service_id=...). Add it to the query call:

@router.get("/query")
def query_media(
    ...,
    jellyfin_service_id: str | None = None,   # already available pattern
    index: MediaIndex = Depends(get_media_index),
) -> dict[str, Any]:
    ...
    rows, total = index.query(service_id=jellyfin_service_id or "", ...)

3.7 MediaIndex concern registration

MEDIA_INDEX_CONCERN = StorageConcern(
    concern_key="media_index",
    db_filename="media_index.sqlite",  # SAME FILE, unchanged path
    migrations=[
        "ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT ''",
        # This ALTER is idempotent-safe: init_schema creates the table WITH
        # service_id for new installs; this migration adds it to existing DBs.
        # SQLite ALTER TABLE ADD COLUMN is a no-op if the column already exists
        # (we guard with a PRAGMA check in run_migrations, or catch the error).
    ],
    tables=["media_items"],
)

IMPORTANT — ALTER TABLE idempotency: SQLite raises an error if the column already exists. The run_migrations method should catch this per-statement or pre-check via PRAGMA table_info. Design choice: wrap each migration in a try/except for "duplicate column name" errors:

def run_migrations(self) -> None:
    for concern in self._concerns.values():
        path = self.db_path(concern.concern_key)
        path.parent.mkdir(parents=True, exist_ok=True)
        with sqlite3.connect(path, timeout=30) as conn:
            conn.execute("PRAGMA journal_mode=WAL")
            for stmt in concern.migrations:
                try:
                    conn.executescript(stmt)
                except sqlite3.OperationalError as exc:
                    if "duplicate column name" not in str(exc).lower():
                        raise

3.8 File location confirmation

DEFAULT_INDEX_PATH = Path(".cache/media_library_viewer/media_index.sqlite") — unchanged. The harness base_dir defaults to Path(".cache/media_library_viewer") (the same parent). The db_filename = "media_index.sqlite" matches. No data move.

---

4. Frontend design

4.1 Three widget components

All three live in frontend/src/widgets/, modeled on existing patterns (AlertmanagerAlertsWidget, PrometheusChartWidget, MetricCard).

QbittorrentTotalsWidget.tsx — MetricCard-style count tile:

// Renders {total: number, by_state: {...}} from useWidgetData.
// Uses SectionCard + numeric display (like BackupsWidget / MetricCard).
// Shows total count prominently + state breakdown badges.

QbittorrentActiveTorrentsWidget.tsx — active torrents list:

// Renders {torrents: [{name, state, size, progress, dl_speed, up_speed}]}.
// Uses SectionCard + a compact list/table (DataTable or manual Table rows).
// Shows name, state badge, progress bar, speeds. Max 10 rows with scroll.

QbittorrentSpeedWidget.tsx — speed chart reusing Change A's renderer:

// Renders {series: [{label, points:[{t,v}]}]} — IDENTICAL to PrometheusChartWidget.
// Two options:
//   (a) Import PrometheusChartWidget directly and pass props (if its props accept
//       the series externally rather than via useWidgetData).
//   (b) Extract the recharts rendering into a shared <LineSeriesChart series={...} />
//       component that both PrometheusChartWidget and QbittorrentSpeedWidget use.
//
// RECOMMENDED: option (b) — extract a shared LineSeriesChart component (~40 lines)
// into frontend/src/components/LineSeriesChart.tsx. Both widgets call useWidgetData
// independently (different refresh intervals) but share the renderer.

LineSeriesChart.tsx (shared renderer extraction):

// Extracts: mergeSeries, formatTime, CHART_COLORS, and the <ResponsiveContainer>
// + <LineChart> JSX from PrometheusChartWidget.
// Props: { series: ChartSeries[], height?: number }
// PrometheusChartWidget becomes a thin wrapper: useWidgetData → <LineSeriesChart series={data.data.series} />
// QbittorrentSpeedWidget: same pattern, different refresh interval (5s vs 60s).

This keeps the recharts rendering in ONE place (no duplication) while letting each widget own its polling lifecycle.

4.2 Frontend registry binding (integrations/registry.ts)

qbittorrent: {
    serviceType: "qbittorrent",
    name: "qBittorrent",
    description: "Torrent client activity, speeds, and item counts.",
    widgets: [
        { kind: "totals", name: "Totals", description: "...",
          refreshIntervalMs: 30_000, defaultConfig: {},
          configSchema: { type: "object", properties: {}, required: [] },
          component: QbittorrentTotalsWidget },
        { kind: "active", name: "Active torrents", description: "...",
          refreshIntervalMs: 15_000, defaultConfig: {},
          configSchema: { type: "object", properties: {}, required: [] },
          component: QbittorrentActiveTorrentsWidget },
        { kind: "speed", name: "Speed chart", description: "...",
          refreshIntervalMs: 5_000, defaultConfig: {},
          configSchema: { type: "object", properties: {}, required: [] },
          component: QbittorrentSpeedWidget },
    ],
},

4.3 Types

No new TypeScript types needed for widget payloads — data flows through the existing WidgetDataResponse + useWidgetData polling. The series shape ({series:[{label,points:[{t,v}]}]}) is already used by PrometheusChartWidget.

---

5. Tests

5.1 Backend tests

Test file	Coverage
backend/tests/test_service_data.py (NEW)	Harness: register, run_migrations (creates tables), cascade_delete (removes rows by service_id), migration idempotency (ALTER doesn't crash on re-run)
backend/tests/test_qbittorrent_store.py (NEW)	Store: append + prune (MAX_SAMPLES cap), window (returns samples in order), service_id isolation (two services don't cross-contaminate)
backend/tests/test_qbittorrent_client.py (NEW)	Client: login flow (POST /auth/login → Ok.), cookie reuse, 403 → re-login, maindata parsing, timeout handling
backend/tests/test_widgets.py (extend)	QbittorrentWidgetSource: totals (counts all torrents), active (filters state), speed (appends sample + returns {series}), missing-service error, timeout error

5.2 Frontend tests

Test file	Coverage
QbittorrentTotalsWidget.test.tsx (NEW)	Loading skeleton, error alert, rendered count + state badges
QbittorrentActiveTorrentsWidget.test.tsx (NEW)	Loading, error, rendered torrent rows
QbittorrentSpeedWidget.test.tsx (NEW)	Loading, error, rendered chart (series present)
LineSeriesChart.test.tsx (NEW)	Renders lines from series data, empty state

5.3 Existing MediaIndex tests must stay green

backend/tests/test_media_index.py exercises replace_items, query, status. After migration, these call with the new service_id="" default (backward-compatible). The tests pass unchanged because empty-string service_id shows all rows.

---

6. Slice plan (for tasks.md)

Four slices, each ≤400 changed lines, each leaving pytest + npm run build + npm run lint green. Slices 1–2 prove the harness; Slice 3 migrates MediaIndex; Slice 4 wires cascade-delete end-to-end.

Slice 1: Harness + QbittorrentSampleStore + QbittorrentClient + integration (~280–350 lines)

Files:

• backend/src/media_library_viewer_api/services/service_data.py (NEW — harness + StorageConcern)
• backend/src/media_library_viewer_api/services/qbittorrent_store.py (NEW — store + concern)
• backend/src/media_library_viewer_api/clients/qbittorrent.py (NEW — client)
• backend/src/media_library_viewer_api/integrations/qbittorrent.py (NEW — definition)
• backend/src/media_library_viewer_api/integrations/registry.py (MODIFY — add qbittorrent)
• backend/src/media_library_viewer_api/main.py (MODIFY — call harness init in lifespan)
• backend/tests/test_service_data.py (NEW)
• backend/tests/test_qbittorrent_store.py (NEW)
• backend/tests/test_qbittorrent_client.py (NEW)

Exit gate: harness creates tables + runs migrations + cascade_delete works in tests. qBit client login/maindata tested with mocked HTTP. No frontend changes yet.

Slice 2: Widget adapter + frontend widgets + registry binding (~320–400 lines)

Files:

• backend/src/media_library_viewer_api/widgets/sources.py (MODIFY — add QbittorrentWidgetSource + SERVICE_ADAPTERS entry)
• backend/tests/test_widgets.py (EXTEND — qBit adapter tests)
• frontend/src/components/LineSeriesChart.tsx (NEW — shared renderer extracted from PrometheusChartWidget)
• frontend/src/widgets/PrometheusChartWidget.tsx (MODIFY — use LineSeriesChart)
• frontend/src/widgets/QbittorrentTotalsWidget.tsx (NEW)
• frontend/src/widgets/QbittorrentActiveTorrentsWidget.tsx (NEW)
• frontend/src/widgets/QbittorrentSpeedWidget.tsx (NEW)
• frontend/src/widgets/index.ts (MODIFY — barrel exports)
• frontend/src/integrations/registry.ts (MODIFY — add qbittorrent binding)
• frontend/src/widgets/__tests__/QbittorrentTotalsWidget.test.tsx (NEW)
• frontend/src/widgets/__tests__/QbittorrentActiveTorrentsWidget.test.tsx (NEW)
• frontend/src/widgets/__tests__/QbittorrentSpeedWidget.test.tsx (NEW)
• frontend/src/components/__tests__/LineSeriesChart.test.tsx (NEW)

Exit gate: qBit widgets render loading/error/data states; speed widget shows a recharts line chart from {series}; LineSeriesChart is shared with PrometheusChartWidget.

Slice 3: MediaIndex migration onto harness (~200–280 lines)

Files:

• backend/src/media_library_viewer_api/services/service_data.py (MODIFY — register MEDIA_INDEX_CONCERN)
• backend/src/media_library_viewer_api/services/media_index_impl.py (MODIFY — add service_id to schema, replace_items, query)
• backend/src/media_library_viewer_api/services/media_index.py (re-export unchanged)
• backend/src/media_library_viewer_api/workers/media_index_worker.py (MODIFY — pass service_id to build_media_index)
• backend/src/media_library_viewer_api/routers/media.py (MODIFY — thread jellyfin_service_id into query_media)
• backend/tests/test_media_index.py (EXTEND — service_id-scoped replace_items + query)

Exit gate: all existing MediaIndex tests pass unchanged (empty-string default); new tests verify scoped delete/insert/query. Media page works identically.

Slice 4: Cascade-delete end-to-end + integration test (~80–120 lines)

Files:

• backend/src/media_library_viewer_api/services/settings_store.py (MODIFY — call harness.cascade_delete in delete_service)
• backend/tests/test_services.py (EXTEND — verify cascade-delete removes qBit samples when service is deleted)

Exit gate: deleting a qBittorrent service removes its speed samples; deleting a Jellyfin service removes its media items.

---

7. Key design decisions summary

#	Decision	Rationale
D1	Harness is lifecycle-only (migrations, service_id, cascade)	n=2 justifies abstraction, but data-ops differ wildly (append/window vs replace_all/query). Generalize only the shared lifecycle.
D2	Per-concern DB files via harness base_dir + db_filename	media_index.sqlite stays put (no data move); qbittorrent.db is new. Separate writers.
D3	run_migrations catches "duplicate column name" per-statement	SQLite ALTER TABLE ADD COLUMN is not idempotent; migrations must not crash on re-run.
D4	Speed widget extracts a shared LineSeriesChart component	Both PrometheusChartWidget and QbittorrentSpeedWidget need the same recharts renderer; extract it once (DRY) rather than duplicating or tightly coupling.
D5	qBit adapter resolves client inline from ServiceRecord (like PrometheusWidgetSource)	No new dependency-injection helper; the adapter gets config+secrets from the service record.
D6	MediaIndex backfill uses service_id = '' (empty string) sentinel	Backward-compatible: existing tests + Media page work unchanged; empty-string queries skip the filter (show all). Real service_ids overwrite on next rebuild.
D7	replace_items deletes WHERE service_id = ? instead of all	Fixes the existing global-clear bug (building for one Jellyfin wipes others).
D8	Cascade-delete is best-effort (try/except) in delete_service	Harness failure must not block service deletion; data cleanup is non-critical.
D9	Speed sample timestamps multiplied by 1000 for frontend	PrometheusChartWidget expects t in milliseconds (JS epoch); SQLite stores Unix seconds.
D10	MAX_SAMPLES = 120 cap with per-append prune	~2 min history at 1s poll; bounded DB growth; single-statement prune.

---

8. Risks and mitigations

Risk	Mitigation
MediaIndex migration breaks existing tests.	service_id defaults to ""; empty-string queries skip the filter → all rows visible. Existing tests pass unchanged.
ALTER TABLE fails on fresh installs where init_schema already created the column.	run_migrations catches "duplicate column name" errors per-statement.
Harness lazy-init hides migration failures.	run_migrations runs on first access (startup lifespan); failures raise (not swallowed) except the expected duplicate-column case.
Speed widget poll interval (5s) too aggressive.	5s is the default; configurable via widget config if needed. MAX_SAMPLES cap bounds storage.
qBit cookie expiry between polls.	Client re-logins on 403 transparently; persistent auth failure surfaces as widget error state.
LineSeriesChart extraction breaks PrometheusChartWidget.	Slice 2 includes tests for both; extraction is mechanical (move JSX + helpers, pass series as prop).
Review budget (>400 lines).	Four slices, each ≤400 lines. Slices 1–2 are independently shippable (qBit works without MediaIndex migration).

---

9. Data flow diagrams

9.1 Speed widget data flow (InService path)

Dashboard poll (useWidgetData, 5s)
  └► GET /api/widgets/instances/{id}/data
       └► QbittorrentWidgetSource.fetch(service, "speed", {})
            ├► QbittorrentClient.maindata()
            │     └► /api/v2/sync/maindata → {server_state:{dl_info_speed, up_info_speed}}
            ├► QbittorrentSampleStore.append(service.id, ts, dl, up)
            │     └► INSERT + prune (keep 120)
            ├► QbittorrentSampleStore.window(service.id)
            │     └► SELECT ts, dl_speed, up_speed → [{ts, dl_speed, up_speed}]
            └► return {series: [{label:"download", points:[{t,v}]}, {label:"upload", points:[{t,v}]}]}
                 └► Frontend: QbittorrentSpeedWidget → <LineSeriesChart series={...} />

9.2 Cascade-delete data flow

DELETE /api/services/instances/{id}
  └► SettingsStore.delete_service(id)
       ├► DELETE FROM dashboard_widgets WHERE service_id = ?
       ├► DELETE FROM services WHERE id = ?
       └► ServiceDataHarness.cascade_delete(id)
            ├► DELETE FROM qbittorrent_speed_samples WHERE service_id = ?
            └► DELETE FROM media_items WHERE service_id = ?