chore(service-storage-harness): archive verified+synced change
Move to openspec/changes/archive/2026-07-09-service-storage-harness/ (history preserved via rename detection). 9 artifacts: proposal/spec/design/ tasks/apply-progress/verify-report/sync-report/archive-report + delta spec. Canonical openspec/specs/service-storage/ remains. Native status engine discrepancy (ambiguous change selection) disregarded per parent verification.
This commit is contained in:
Developer
@@ -0,0 +1,55 @@
|
||||
# Apply Progress: Service Storage Harness
|
||||
|
||||
**Change:** `service-storage-harness`
|
||||
**Phase:** apply-progress
|
||||
**Date:** 2026-07-09
|
||||
**Status:** complete — all 35 tasks done, all gates green, verified (see `verify-report.md`)
|
||||
|
||||
## Slices delivered
|
||||
|
||||
Four slices, each its own commit, each leaving `pytest` / `npm run build` / `npm run lint` / `ruff` green.
|
||||
|
||||
### Slice 1 — Harness + qBit store + client + integration (commit `e7bd0af`, amended)
|
||||
|
||||
- `services/service_data.py` — `ServiceDataHarness`: lifecycle-only (concern registration via dataclass, idempotent `run_migrations` catching "duplicate column name" per-statement, `cascade_delete(service_id)` iterating owned tables). No generic data ops (SS-101..104).
|
||||
- `services/qbittorrent_store.py` — `QbittorrentSampleStore`: `qbittorrent_speed_samples(service_id, ts, dl_speed, up_speed)` + index in dedicated `qbittorrent.db`; `append/window/prune`, MAX_SAMPLES=120; registered as a harness concern (SS-105..107).
|
||||
- `clients/qbittorrent.py` — `QbittorrentClient`: cookie login via `/api/v2/auth/login`, 403 re-login+retry, `maindata()` via `/api/v2/sync/maindata` (SS-108..110).
|
||||
- `integrations/qbittorrent.py` + registry entry — config (base_url, timeout_seconds) + secret (username, password) schema; 3 widget kinds declared.
|
||||
- Initialized in `main.py` lifespan.
|
||||
- Tests: `test_service_data.py` (harness lifecycle), `test_qbittorrent_store.py`, `test_qbittorrent_client.py`.
|
||||
|
||||
### Slice 2 — qBit widgets + LineSeriesChart extract (commit `8b0e7ea`, amended)
|
||||
|
||||
- `QbittorrentWidgetSource` in `widgets/sources.py` — 3 branches: `totals` (item count from maindata), `active` (filter state ∈ {downloading, uploading}), `speed` (append sample + return `{series}` from `.window()`, ts×1000 for JS ms). Errors → `{error}`. Registered in `SERVICE_ADAPTERS` (SS-111..115).
|
||||
- `frontend/src/components/LineSeriesChart.tsx` — shared recharts renderer extracted from `PrometheusChartWidget` (~40 lines, props `{series, height?}`); `PrometheusChartWidget` becomes a thin wrapper. **Extraction non-regressive: 4 PrometheusChartWidget tests stay green** (SS-118).
|
||||
- `QbittorrentTotalsWidget` (count), `QbittorrentActiveTorrentsWidget` (list), `QbittorrentSpeedWidget` (uses LineSeriesChart). Registry binding + barrel + tests (SS-116..117).
|
||||
|
||||
### Slice 3 — MediaIndex migration (commit `c87f398`) — LOAD-BEARING
|
||||
|
||||
- Idempotent harness migration: `ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT ''`. `media_index.db` file location UNCHANGED. Existing rows backfill to `service_id=''` via DEFAULT (SS-119, SS-120, SS-124).
|
||||
- **Scoped `replace_items`** — `DELETE FROM media_items WHERE service_id = ?` replaces the prior global `DELETE FROM media_items`. **FIXES the latent global-clear bug** where rebuilding for one Jellyfin wiped another's rows. `service_id` stamped into inserted rows. Regression test `test_replace_scoped_by_service_id_preserves_other_services` proves svc-A survives svc-B's rebuild (SS-121).
|
||||
- `query(service_id="")` shows all rows (backward-compat); `query(service_id="X")` scopes. **Existing MediaIndex + API tests pass unchanged** (62 passed) (SS-122).
|
||||
- Worker threads the real `service_id` (already plumbed via `--service-id`) into `replace_items` so new rows are stamped (SS-123).
|
||||
|
||||
### Slice 4 — Cascade-delete wiring (commit `75c949a`)
|
||||
|
||||
- `settings_store.delete_service` calls `ServiceDataHarness.cascade_delete(service_id)` after existing cleanup, best-effort try/except (failure logs, doesn't crash the delete) (SS-125).
|
||||
- Integration test proves end-to-end cascade across BOTH concerns (qBit samples + media items) with multi-instance preservation (SS-126).
|
||||
|
||||
## Deviations from tasks.md
|
||||
|
||||
- **Slice 2 over the 400-line review budget** (verify-report flagged +644 source lines). The slice is additive (3 new widgets + extraction + tests), no scope creep, but the per-slice budget from `openspec/config.yaml` was exceeded. Retrospectively this could have been split (extraction in one slice, qBit widgets in another). No code defect; recorded here as a process note for future slicing. The verify agent flagged it WARNING, not blocking.
|
||||
|
||||
## Final gate results
|
||||
|
||||
| Gate | Result |
|
||||
|---|---|
|
||||
| `backend && PYTHONPATH=src python3 -m pytest -q` | **322 passed**, 2 warnings (pre-existing pythonjsonlogger DeprecationWarning) |
|
||||
| `backend && PYTHONPATH=src python3 -m ruff check src tests` | **All checks passed** |
|
||||
| `frontend && npm run build` | **exit 0** (pre-existing chunk-size warning) |
|
||||
| `frontend && npm run lint` | **0 errors**, 1 pre-existing warning (`WidgetConfigDialog.tsx`, untouched) |
|
||||
| `frontend && npx vitest run PrometheusChartWidget.test.tsx` | **4 passed** (extraction non-regression confirmed) |
|
||||
|
||||
## Verification
|
||||
|
||||
See `verify-report.md` — adversarial fresh-context review: **28/28 PASS**. No blocking code findings. Archive blocker is doc-only (this file + the ticked tasks.md clear it).
|
||||
@@ -0,0 +1,220 @@
|
||||
# Archive Report — `service-storage-harness`
|
||||
|
||||
> Phase: **archive** · Change: `service-storage-harness` · Repo: `/home/user/manage`
|
||||
> Mode: file-backed (`artifactStore: openspec`). No source-code edits; only OpenSpec artifacts
|
||||
> were touched. **Not committed** — the parent/orchestrator owns the archive commit. No push, no `gh`.
|
||||
|
||||
**Status: ARCHIVED.** All eight lifecycle phases are complete (proposal → spec → design → tasks →
|
||||
apply → verify → sync → **archive**). Every archive precondition is verified PASS (see §2). The
|
||||
canonical `openspec/specs/service-storage/spec.md` (created by `sdd-sync`) remains in place as the
|
||||
durable end-state spec and is **not** moved (archive never moves canonical specs). The change folder
|
||||
was moved to `openspec/changes/archive/2026-07-09-service-storage-harness/` via `git mv` to preserve
|
||||
history (the renames were then unstaged so the parent commits from a clean index with explicit paths).
|
||||
|
||||
---
|
||||
|
||||
## 0. Archive disposition
|
||||
|
||||
- **Disposition: `archived`.** The folder move was performed inline as instructed (unlike a
|
||||
`documented-pending-manual` outcome): the parent explicitly requested the `git mv` and owns the
|
||||
commit, so the move is executed here and left as working-tree changes for the parent's
|
||||
explicit-path commit.
|
||||
- **Archive convention:** OpenSpec SDD archive contract for `openspec` mode — completed file-backed
|
||||
sync → write the in-folder archive report → move the change folder to
|
||||
`openspec/changes/archive/YYYY-MM-DD-{change}/`. No standalone manifest/index exists under
|
||||
`openspec/` (only `config.yaml`, `changes/`, `specs/`), so the folder move **is** the archive
|
||||
mechanism. No `rules.archive` override exists in `openspec/config.yaml`.
|
||||
- **Target archived path:** `openspec/changes/archive/2026-07-09-service-storage-harness/`
|
||||
- **Archive date:** `2026-07-09` (ISO).
|
||||
- **Canonical spec left in place (not moved):** `openspec/specs/service-storage/spec.md` —
|
||||
28 requirements (SS-101 … SS-128). Verified present and byte-identical before/after the move
|
||||
(sha256 `6629e307…` unchanged). The other canonical domains (`web-ui`, `prometheus-charting`)
|
||||
were also left untouched.
|
||||
- **Audit-trail integrity:** the change folder was moved as a whole, including the legacy flat
|
||||
`spec.md` and the per-domain delta `specs/service-storage/spec.md`, which travel with the
|
||||
record. Nothing was silently deleted or rewritten. The flat `spec.md` is retained as the
|
||||
authoritative planning artifact the work was built against.
|
||||
|
||||
## 1. Native `sdd-status` read & discrepancy statement
|
||||
|
||||
The native `gentle-pi.sdd-status` engine supplied by the parent reports **non-actionable state for
|
||||
this archive** because it was resolved without a change context: `changeName: null`,
|
||||
`artifacts: all missing`, `applyState: blocked`, `dependencies.archive: blocked`,
|
||||
`blockedReasons: ["Change selection is ambiguous: mobile-responsive-parity, service-storage-harness,
|
||||
services-as-hub-ia."]`, `isNonAuthoritative: false`. This is a **parent-resolution artifact**: the
|
||||
engine auto-detected three active changes and could not pick one. The ambiguity does **not** reflect
|
||||
the state of `service-storage-harness`, which this archive task was **explicitly assigned**.
|
||||
|
||||
**Discrepancy with the parent's authoritative confirmed state — RESOLVED in favor of the parent.**
|
||||
The parent physically verified (and this executor re-confirmed directly against the filesystem in
|
||||
§2) that all 35 tasks are ticked, `apply-progress.md` exists and records the four landed slices, the
|
||||
verify report is clearly passing (28/28 PASS, all four gates green), and `sync-report.md` records a
|
||||
completed sync. Per the archive contract's non-authoritative-store carve-out guidance and the
|
||||
parent's explicit instruction ("DISREGARD; PROCEED"), the stale `archive: blocked` / "ambiguous"
|
||||
labels are **disregarded** and the archive **proceeds**.
|
||||
|
||||
Direct filesystem re-validation (§2) is the source of truth for this report.
|
||||
|
||||
## 2. Archive preconditions (validated directly against the filesystem)
|
||||
|
||||
| Precondition | Evidence | Result |
|
||||
|---|---|---|
|
||||
| Verify report present | `verify-report.md` | ✓ verdict **PASS** |
|
||||
| Verify clearly passing — no unresolved `FAIL`/`BLOCKED`/`CRITICAL` | verify-report: **28/28 PASS** (SS-101…SS-128); the sole CRITICAL was an archive-only checkbox/apply-progress gap (now reconciled) | ✓ |
|
||||
| Sync report present & successful | `sync-report.md` → **Status: SYNCED** | ✓ |
|
||||
| Canonical spec exists (sync target) | `openspec/specs/service-storage/spec.md` (28 requirements) | ✓ |
|
||||
| Change-side domain delta exists | `specs/service-storage/spec.md` | ✓ |
|
||||
| Delta op-class = pure `## ADDED` (non-destructive) | ADDED=1, MODIFIED=0, REMOVED=0, RENAMED=0 (new domain) | ✓ |
|
||||
| Requirement-ID parity (flat ↔ delta ↔ canonical) | 28 == 28 == 28, identical IDs SS-101…SS-128 | ✓ |
|
||||
| proposal / design / tasks artifacts present | all populated | ✓ |
|
||||
| **Final Task Completion Gate — zero unchecked `- [ ]`** | `grep -nE '^\s*- \[ \]' tasks.md` → **NONE**; `grep -cE '^\s*- \[x\]'` → **35** | ✓ |
|
||||
| `apply-progress.md` present & records the work | present; status "complete — all 35 tasks done", 4 slices + final gates documented | ✓ |
|
||||
| No active same-domain (`service-storage`) collision | only this change carries a `service-storage` delta; `web-ui`/`prometheus-charting` untouched | ✓ |
|
||||
| Canonical untouched by the move | sha256 of `openspec/specs/service-storage/spec.md` identical before/after the `git mv` | ✓ |
|
||||
|
||||
**Stale-checkbox reconciliation note.** At verify time, 30 implementation/verification checkboxes
|
||||
(Slices §1.1–1.9, §2.1–2.9, §3.1–3.9, §4.1–4.3 and Integration §5.1–5.5) were unchecked and
|
||||
`apply-progress.md` did not exist. That condition was reconciled **before** archive: the boxes are
|
||||
now all ticked (re-confirmed directly: `grep -nE '^\s*- \[ \]' tasks.md` → none; 35 `[x]`) and
|
||||
`apply-progress.md` was authored documenting the four landed slices. `apply-progress.md` plus the
|
||||
verify report prove every previously-unchecked task complete. No archive-time mechanical checkbox
|
||||
repair was needed — the gate now passes on the persisted `tasks.md` as-is. No partial-archive
|
||||
approval applies.
|
||||
|
||||
## 3. Artifacts read (archive preflight)
|
||||
|
||||
- `openspec/changes/service-storage-harness/proposal.md`
|
||||
- `openspec/changes/service-storage-harness/spec.md` (flat, authoritative planning artifact — 28 requirements)
|
||||
- `openspec/changes/service-storage-harness/specs/service-storage/spec.md` (change-side domain delta)
|
||||
- `openspec/changes/service-storage-harness/design.md`
|
||||
- `openspec/changes/service-storage-harness/tasks.md`
|
||||
- `openspec/changes/service-storage-harness/apply-progress.md`
|
||||
- `openspec/changes/service-storage-harness/verify-report.md`
|
||||
- `openspec/changes/service-storage-harness/sync-report.md`
|
||||
- `openspec/specs/service-storage/spec.md` (canonical, sync target — verified present and byte-identical after the move)
|
||||
- `openspec/config.yaml` (rules: proposal/tasks; no `rules.archive` override)
|
||||
|
||||
> The legacy flat `spec.md` is **not** the only spec artifact: a per-domain delta
|
||||
> (`specs/service-storage/spec.md`) and a canonical spec both exist, so the "legacy flat spec as the
|
||||
> *only* artifact" archive-block condition does not apply. The flat spec travels with the archived
|
||||
> folder as part of the audit trail.
|
||||
|
||||
## 4. Domains synced & requirement delta
|
||||
|
||||
| Domain | Change-side delta | Canonical | Action |
|
||||
|---|---|---|---|
|
||||
| `service-storage` | `specs/service-storage/spec.md` | `openspec/specs/service-storage/spec.md` | **NEW domain** — pure ADDED (28 requirements) |
|
||||
|
||||
- **ADDED (28)** — all to the new `service-storage` domain (canonical did not exist pre-change). IDs
|
||||
and text preserved verbatim from the verified flat `spec.md`. Grouped logically:
|
||||
- *ServiceDataHarness (lifecycle layer)* — SS-101, SS-102, SS-103, SS-104
|
||||
- *QbittorrentSampleStore* — SS-105, SS-106, SS-107
|
||||
- *QbittorrentClient* — SS-108, SS-109, SS-110
|
||||
- *qBittorrent widget source adapter* — SS-111, SS-112, SS-113, SS-114, SS-115
|
||||
- *qBittorrent frontend widgets* — SS-116, SS-117, SS-118
|
||||
- *MediaIndex migration onto harness* — SS-119, SS-120, SS-121, SS-122, SS-123, SS-124
|
||||
- *Cascade-delete wiring* — SS-125, SS-126
|
||||
- *Test and build greenness* — SS-127, SS-128
|
||||
- **MODIFIED (0)** · **REMOVED (0)** · **RENAMED (0)** — new domain; nothing destructive.
|
||||
|
||||
> No destructive-merge guard or parent approval was triggered (zero REMOVED / zero MODIFIED). The
|
||||
> new `service-storage` domain is distinct from the existing `web-ui` and `prometheus-charting`
|
||||
> canonical domains, neither of which was touched.
|
||||
|
||||
## 5. Final lifecycle status (all 8 phases done)
|
||||
|
||||
| Phase | Status | Evidence |
|
||||
|---|---|---|
|
||||
| Proposal | ✅ done | `proposal.md` |
|
||||
| Spec | ✅ done | flat `spec.md` (28) + domain delta `specs/service-storage/spec.md` (28 ADDED) |
|
||||
| Design | ✅ done | `design.md` |
|
||||
| Tasks | ✅ done | `tasks.md` — **35/35** checked, zero `- [ ]` |
|
||||
| Apply | ✅ done | 4 slices delivered (`e7bd0af`, `1fb12b8`, `c87f398`, `75c949a`) |
|
||||
| Verify | ✅ PASS | `verify-report.md` — 28/28 PASS; gates green |
|
||||
| Sync | ✅ done | `sync-report.md` — SYNCED; canonical `service-storage` domain created |
|
||||
| Archive | ✅ done | this report + folder move performed |
|
||||
|
||||
## 6. Gate results (per verify-report; head `c9404f0`)
|
||||
|
||||
| Gate | Command | Result |
|
||||
|---|---|---|
|
||||
| Backend tests | `cd backend && PYTHONPATH=src python3 -m pytest -q` | **PASS** — 322 passed (2 pre-existing warnings) |
|
||||
| Backend lint | `cd backend && PYTHONPATH=src python3 -m ruff check src tests` | **PASS** — All checks passed |
|
||||
| Frontend build | `cd frontend && npm run build` | **PASS** — exit 0 (pre-existing chunk-size warning) |
|
||||
| Frontend lint | `cd frontend && npm run lint` | **PASS** — 0 errors (1 pre-existing warning) |
|
||||
| Extraction non-regression | `npx vitest run …/PrometheusChartWidget.test.tsx` | **PASS** — 4 passed (SS-118) |
|
||||
| New FE widget/chart tests | `npx vitest run …/widgets/__tests__ …/LineSeriesChart.test.tsx` | **PASS** — 8 files, 31 tests passed |
|
||||
|
||||
## 7. Carry-over follow-ups (non-blocking; recorded per verify/apply-progress)
|
||||
|
||||
1. **[WARNING] Slice-2 review-budget variance** — slice 2 (`1fb12b8`) lands **~644 non-test source
|
||||
lines** (qBit widget adapter + `LineSeriesChart` extraction + 3 FE widgets), above the 400-line
|
||||
per-slice budget (`openspec/config.yaml`) and above the slice-2 "~350–400" forecast. No
|
||||
`size:exception` was recorded. This is a forecast-vs-actual variance on an **additive** slice —
|
||||
the boundary is exactly the qBit-widget feature (no unrelated files, no scope creep). Non-blocking;
|
||||
recorded for the record (could have split extraction into its own slice). The verify agent rated
|
||||
it WARNING, not blocking.
|
||||
2. **[INFO] Slice-2 commit-hash drift** — the brief/apply-progress cited `8b0e7ea` (an earlier amend
|
||||
state); the actual landed commit is `1fb12b8`. Content matches spec/design/tasks. Informational.
|
||||
3. **[INFO] Stale generated `.pi-map.md`** — generated project-map artifacts (`.pi-map.md` /
|
||||
`.pi-map.index.md`) predate the new modules (`service_data.py`, `qbittorrent_store.py`,
|
||||
`clients/qbittorrent.py`, `Qbittorrent*.tsx`, `LineSeriesChart.tsx`). These are **generated
|
||||
artifacts, not deliverable source**, and are out of scope for this change — regenerate via
|
||||
`project_map_patch` / `project_map_validate` in a separate housekeeping pass; the project-map
|
||||
protocol already flags these `dirty`.
|
||||
4. **[INFO] `LineSeriesChart.test.tsx` smoke-only** — asserts the component mounts
|
||||
(`container.firstChild` non-null) but does not assert the recharts `<Line>` SVG series rendered.
|
||||
Acceptable as a crash-guard; non-blocking coverage note.
|
||||
5. **[INFO] qBit store prune edge case** — `QbittorrentSampleStore.append` prunes via
|
||||
`ts NOT IN (SELECT ts … LIMIT 120)`; in the degenerate case of two samples sharing an identical
|
||||
`ts` the keep-set could exceed `MAX_SAMPLES=120`. At a 5 s poll the probability is effectively nil
|
||||
and the per-service cap holds in all realistic operation. Non-blocking.
|
||||
6. **[INFO] Orthogonal dirty working-tree items** — uncommitted cosmetic reformat of
|
||||
`frontend/src/pages/service-tabs/MediaTab.tsx` and untracked `.pi-tmp/*` (predating/orthogonal to
|
||||
this change) were **not touched** by this archive.
|
||||
|
||||
## 8. Residual risks & destructive-merge statement
|
||||
|
||||
- **Destructive sync / merge:** **not applicable.** Zero REMOVED and zero MODIFIED requirements (new
|
||||
`service-storage` domain; pure ADDED). No destructive-merge guard or parent approval was triggered.
|
||||
- **Backend / data-contract impact:** none beyond the planned change. The `ServiceDataHarness` is
|
||||
lifecycle-only (provisioning, idempotent migrations, `service_id` cascade-delete) with **no**
|
||||
generic value table or CRUD. The load-bearing MediaIndex migration (SS-119..SS-124) is correct:
|
||||
`replace_items` is scoped `WHERE service_id = ?` (fixing the latent global-clear bug), with a real
|
||||
regression test; the `media_index.db` file location is unchanged; existing callers are
|
||||
backward-compatible via the `service_id=""` default. Archive touched only OpenSpec docs + the folder
|
||||
move.
|
||||
- **No critical verification issues** remain (CRITICAL issues are non-overridable; the one verify
|
||||
CRITICAL was the reconcilable checkbox/apply-progress gap, now resolved).
|
||||
- **No browser/visual smoke** was performed (out of scope); the qBit speed recharts line and the
|
||||
Prometheus/gauge/mean charts are only structurally tested.
|
||||
- **Memory observation IDs:** none — `artifactStore: openspec`; traceability lives in the filesystem
|
||||
archive + canonical spec.
|
||||
|
||||
## 9. Move performed
|
||||
|
||||
```
|
||||
git mv openspec/changes/service-storage-harness openspec/changes/archive/2026-07-09-service-storage-harness
|
||||
```
|
||||
|
||||
- **All 9 artifacts confirmed present at the archived path:** `proposal.md`, `spec.md` (flat),
|
||||
`specs/service-storage/spec.md` (delta), `design.md`, `tasks.md`, `apply-progress.md`,
|
||||
`verify-report.md`, `sync-report.md`, `archive-report.md` (this file).
|
||||
- Of the 9, **8 were git-tracked** (renamed by `git mv`, preserving history via git rename
|
||||
detection) and **1** (`archive-report.md`) is newly written and traveled with the directory move.
|
||||
- **Canonical `openspec/specs/service-storage/spec.md` remains in place** — verified byte-identical
|
||||
(sha256 `6629e307…`) before and after the move. `openspec/specs/web-ui/spec.md` and
|
||||
`openspec/specs/prometheus-charting/spec.md` also untouched.
|
||||
- **Not committed / not pushed** — the parent owns the commit with explicit paths.
|
||||
|
||||
---
|
||||
|
||||
### Appendix — Files written/moved by this archive (OpenSpec only; no source code)
|
||||
|
||||
- **Written:** `openspec/changes/service-storage-harness/archive-report.md` (this file) — at the
|
||||
active path before the move; travels with the move into the archive.
|
||||
- **Moved (via `git mv`):** the entire
|
||||
`openspec/changes/service-storage-harness/` directory →
|
||||
`openspec/changes/archive/2026-07-09-service-storage-harness/`.
|
||||
- **Left in place (durable canonical):** `openspec/specs/service-storage/spec.md`.
|
||||
- **Not committed / not pushed** — the parent owns the commit with explicit paths.
|
||||
@@ -0,0 +1,863 @@
|
||||
# SDD Design: Service Storage Harness (with qBittorrent widgets + MediaIndex migration)
|
||||
|
||||
**Change:** `service-storage-harness`
|
||||
**Phase:** design
|
||||
**Date:** 2026-07-09
|
||||
|
||||
> Grounded in `proposal.md` (Q1–Q5 resolved) and the already-archived
|
||||
> `prometheus-direct-charting` change. No source changes in this phase.
|
||||
|
||||
## 0. Source findings (read before anything else)
|
||||
|
||||
The proposal was written against a stale project map. Reading actual source
|
||||
surfaced three findings that shape the design. Trust source, not the map.
|
||||
|
||||
### 0.1 The media worker ALREADY threads `service_id` — but it is NOT persisted in rows
|
||||
|
||||
`workers/media_index_worker.py` already accepts `--service-id` via argparse
|
||||
(`run_build(final_index_path, staging_index_path, service_id="")`), and
|
||||
`_resolve_jellyfin(service_id)` resolves the Jellyfin client + user_id from the
|
||||
settings store. The media router's `_start_worker(index, service_id)` and
|
||||
`post_build_index` already pass `jellyfin_service_id` through to the worker.
|
||||
|
||||
**But:** `service_id` is used ONLY to pick the Jellyfin connection. It is never
|
||||
stored in the `media_items` table (which has no `service_id` column). The
|
||||
migration must close this gap: `replace_items` must scope its DELETE+INSERT by
|
||||
`service_id`, and `query` must filter by it. The plumbing to get `service_id`
|
||||
into the worker already exists — only the storage layer is missing.
|
||||
|
||||
### 0.2 `media_items` is cleared globally on every build
|
||||
|
||||
`replace_items` does `DELETE FROM media_items` (no WHERE clause). This means a
|
||||
build for Jellyfin instance A wipes instance B's rows. After migration this
|
||||
becomes `DELETE FROM media_items WHERE service_id = ?`.
|
||||
|
||||
### 0.3 The `delete_service` cascade already exists — the harness hooks into it
|
||||
|
||||
`SettingsStore.delete_service` already cascade-deletes `dashboard_widgets WHERE
|
||||
service_id = ?` (with a PRAGMA-guarded column check). The harness cascade-delete
|
||||
hooks into the same place: after the service row is deleted, iterate registered
|
||||
concerns and delete that `service_id` from each owned table.
|
||||
|
||||
### 0.4 DB path is a module constant, not in config.py
|
||||
|
||||
`DEFAULT_INDEX_PATH = Path(".cache/media_library_viewer/media_index.sqlite")` in
|
||||
`media_index_impl.py`. The file stays at this path (per the locked topology
|
||||
decision). The harness must not move it.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture overview
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ ServiceDataHarness │
|
||||
│ (services/service_data.py — LIFECYCLE ONLY) │
|
||||
│ │
|
||||
│ • register_concern(db_filename, migrations[], tables[], │
|
||||
│ service_id_column="service_id") │
|
||||
│ • run_migrations() — on startup, per concern DB │
|
||||
│ • cascade_delete(service_id) — iterate concerns, DELETE rows │
|
||||
│ • connect(db_filename) → sqlite3.Connection (per-concern) │
|
||||
└──────────────┬───────────────────────────┬───────────────────────┘
|
||||
│ │
|
||||
┌──────────▼──────────┐ ┌─────────▼──────────────┐
|
||||
│ QbittorrentStore │ │ MediaIndex │
|
||||
│ (services/ │ │ (services/ │
|
||||
│ qbittorrent_store) │ │ media_index_impl) │
|
||||
│ │ │ │
|
||||
│ qbittorrent.db │ │ media_index.sqlite │
|
||||
│ └ qbittorrent_ │ │ └ media_items │
|
||||
│ speed_samples │ │ (+ service_id col) │
|
||||
│ (service_id, ts, │ │ └ index_metadata │
|
||||
│ dl_speed, │ │ │
|
||||
│ up_speed) │ │ bespoke: replace_items │
|
||||
│ │ │ (scoped), query │
|
||||
│ bespoke: append, │ │ (scoped), status │
|
||||
│ window, prune │ │ │
|
||||
└──────────────────────┘ └────────────────────────┘
|
||||
│
|
||||
┌──────────▼──────────┐
|
||||
│ QbittorrentClient │
|
||||
│ (clients/ │
|
||||
│ qbittorrent) │
|
||||
│ │
|
||||
│ login → cookie │
|
||||
│ sync/maindata │
|
||||
│ (totals + active + │
|
||||
│ speeds) │
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
**Key constraints carried from the proposal:**
|
||||
|
||||
- Harness is lifecycle-only: migrations, service_id scoping, cascade-delete. No generic value table, no generic CRUD (D2).
|
||||
- Per-concern DB files: `media_index.sqlite` stays put; new `qbittorrent.db` (D4).
|
||||
- MediaIndex migration is sequenced after harness + qBit are proven (D3).
|
||||
- qBit speed chart reuses the `PrometheusChartWidget` recharts renderer fed from an InService data path returning `{series}` (Q1).
|
||||
- Totals = count of listed items, NOT transfer bytes (Q2).
|
||||
- Active = state downloading|uploading (Q3). N instances (Q4). Username/password → cookie (Q5).
|
||||
|
||||
---
|
||||
|
||||
## 2. Backend design
|
||||
|
||||
### 2.1 `ServiceDataHarness` (`services/service_data.py`)
|
||||
|
||||
A lifecycle-only registry of storage concerns. Each concern declares its own DB
|
||||
filename, ordered migrations, owned tables, and the column used for service
|
||||
scoping.
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class StorageConcern:
|
||||
"""A per-integration storage namespace registered with the harness."""
|
||||
|
||||
concern_key: str # e.g. "qbittorrent", "media_index"
|
||||
db_filename: str # e.g. "qbittorrent.db", "media_index.sqlite"
|
||||
migrations: list[str] # ordered CREATE/ALTER statements (idempotent)
|
||||
tables: list[str] # tables owned by this concern (for cascade)
|
||||
service_id_column: str = "service_id"
|
||||
|
||||
|
||||
class ServiceDataHarness:
|
||||
"""Lifecycle-only registry of per-concern storage.
|
||||
|
||||
Owns: DB provisioning, per-concern migrations, service_id cascade-delete.
|
||||
Does NOT own: data operations (each store keeps bespoke append/window/query/etc.).
|
||||
"""
|
||||
|
||||
def __init__(self, base_dir: Path) -> None:
|
||||
self._base_dir = Path(base_dir)
|
||||
self._concerns: dict[str, StorageConcern] = {}
|
||||
|
||||
def register(self, concern: StorageConcern) -> None:
|
||||
"""Register a storage concern. Called at module import / startup."""
|
||||
self._concerns[concern.concern_key] = concern
|
||||
|
||||
def db_path(self, concern_key: str) -> Path:
|
||||
"""Return the absolute path to a concern's DB file."""
|
||||
concern = self._concerns[concern_key]
|
||||
return self._base_dir / concern.db_filename
|
||||
|
||||
def connect(self, concern_key: str) -> sqlite3.Connection:
|
||||
"""Open a WAL-mode connection to a concern's DB."""
|
||||
path = self.db_path(concern_key)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
conn = sqlite3.connect(path, timeout=30)
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
conn.execute("PRAGMA busy_timeout=30000")
|
||||
return conn
|
||||
|
||||
def run_migrations(self) -> None:
|
||||
"""Run pending migrations for every registered concern."""
|
||||
for concern in self._concerns.values():
|
||||
path = self.db_path(concern.concern_key)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with sqlite3.connect(path, timeout=30) as conn:
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
conn.executescript(";".join(concern.migrations))
|
||||
|
||||
def cascade_delete(self, service_id: str) -> None:
|
||||
"""Delete all rows for a service_id across every concern's tables.
|
||||
|
||||
Called from SettingsStore.delete_service after the service row is removed.
|
||||
"""
|
||||
for concern in self._concerns.values():
|
||||
col = concern.service_id_column
|
||||
with sqlite3.connect(self.db_path(concern.concern_key), timeout=30) as conn:
|
||||
for table in concern.tables:
|
||||
cols = {row[1] for row in conn.execute(f"PRAGMA table_info({table})").fetchall()}
|
||||
if col in cols:
|
||||
conn.execute(f"DELETE FROM {table} WHERE {col} = ?", (service_id,))
|
||||
```
|
||||
|
||||
**Module-level singleton + registration:**
|
||||
|
||||
```python
|
||||
_HARNESS: ServiceDataHarness | None = None
|
||||
|
||||
def get_service_data_harness() -> ServiceDataHarness:
|
||||
global _HARNESS
|
||||
if _HARNESS is None:
|
||||
base_dir = Path(os.environ.get("BACKEND_CACHE_DIR", ".cache/media_library_viewer"))
|
||||
_HARNESS = ServiceDataHarness(base_dir)
|
||||
# Register built-in concerns (each store module calls register on import)
|
||||
_HARNESS.register(QBITTORRENT_CONCERN)
|
||||
_HARNESS.register(MEDIA_INDEX_CONCERN)
|
||||
_HARNESS.run_migrations()
|
||||
return _HARNESS
|
||||
```
|
||||
|
||||
**Startup hook:** called from `main.py` lifespan alongside `get_settings_store().ensure_defaults()`. The harness is lazy-initialized on first access (like `SettingsStore`), so tests can override the base dir via env.
|
||||
|
||||
### 2.2 `QbittorrentSampleStore` (`services/qbittorrent_store.py`)
|
||||
|
||||
Speed-sample storage for the qBit speed widget. Registered as a concern with the
|
||||
harness.
|
||||
|
||||
```python
|
||||
QBITTORRENT_CONCERN = StorageConcern(
|
||||
concern_key="qbittorrent",
|
||||
db_filename="qbittorrent.db",
|
||||
migrations=[
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS qbittorrent_speed_samples (
|
||||
service_id TEXT NOT NULL,
|
||||
ts INTEGER NOT NULL,
|
||||
dl_speed INTEGER NOT NULL DEFAULT 0,
|
||||
up_speed INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_qbit_samples_service_ts
|
||||
ON qbittorrent_speed_samples(service_id, ts);
|
||||
"""
|
||||
],
|
||||
tables=["qbittorrent_speed_samples"],
|
||||
)
|
||||
|
||||
MAX_SAMPLES = 120 # ~2 min at 1s poll, ~4 min at 2s poll
|
||||
|
||||
|
||||
class QbittorrentSampleStore:
|
||||
"""Bespoke speed-sample store for qBittorrent widgets."""
|
||||
|
||||
def __init__(self, harness: ServiceDataHarness | None = None) -> None:
|
||||
self._harness = harness or get_service_data_harness()
|
||||
|
||||
def append(self, service_id: str, ts: int, dl_speed: int, up_speed: int) -> None:
|
||||
"""Append a sample and prune old entries beyond MAX_SAMPLES."""
|
||||
with self._harness.connect("qbittorrent") as conn:
|
||||
conn.execute(
|
||||
"INSERT INTO qbittorrent_speed_samples (service_id, ts, dl_speed, up_speed) VALUES (?, ?, ?, ?)",
|
||||
(service_id, ts, dl_speed, up_speed),
|
||||
)
|
||||
# Prune: keep only the most recent MAX_SAMPLES rows for this service
|
||||
conn.execute(
|
||||
"""DELETE FROM qbittorrent_speed_samples
|
||||
WHERE service_id = ? AND ts NOT IN (
|
||||
SELECT ts FROM qbittorrent_speed_samples
|
||||
WHERE service_id = ?
|
||||
ORDER BY ts DESC LIMIT ?
|
||||
)""",
|
||||
(service_id, service_id, MAX_SAMPLES),
|
||||
)
|
||||
|
||||
def window(self, service_id: str, since_ts: int | None = None) -> list[dict[str, Any]]:
|
||||
"""Return all samples for a service since a timestamp (or all if None)."""
|
||||
with self._harness.connect("qbittorrent") as conn:
|
||||
if since_ts is not None:
|
||||
rows = conn.execute(
|
||||
"SELECT ts, dl_speed, up_speed FROM qbittorrent_speed_samples WHERE service_id = ? AND ts >= ? ORDER BY ts ASC",
|
||||
(service_id, since_ts),
|
||||
).fetchall()
|
||||
else:
|
||||
rows = conn.execute(
|
||||
"SELECT ts, dl_speed, up_speed FROM qbittorrent_speed_samples WHERE service_id = ? ORDER BY ts ASC",
|
||||
(service_id,),
|
||||
).fetchall()
|
||||
return [{"ts": r[0], "dl_speed": r[1], "up_speed": r[2]} for r in rows]
|
||||
```
|
||||
|
||||
### 2.3 `QbittorrentClient` (`clients/qbittorrent.py`)
|
||||
|
||||
Cookie-session HTTP client modeled on `JellyfinClient`'s session pattern.
|
||||
|
||||
```python
|
||||
class QbittorrentClient:
|
||||
"""Minimal qBittorrent Web API client (read-only: sync/maindata only)."""
|
||||
|
||||
def __init__(self, base_url: str, username: str, password: str, timeout: int = 10) -> None:
|
||||
self.base_url = base_url.rstrip("/")
|
||||
if not self.base_url.endswith("/api/v2"):
|
||||
self.base_url += "/api/v2"
|
||||
self._username = username
|
||||
self._password = password
|
||||
self._timeout = timeout
|
||||
self._session = requests.Session()
|
||||
self._logged_in = False
|
||||
|
||||
def _login(self) -> None:
|
||||
"""POST username/password to /auth/login; store the SID cookie."""
|
||||
resp = self._session.post(
|
||||
f"{self.base_url}/auth/login",
|
||||
data={"username": self._username, "password": self._password},
|
||||
timeout=self._timeout,
|
||||
headers={"Referer": self.base_url},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
if resp.text.strip() != "Ok.":
|
||||
raise RuntimeError(f"qBittorrent login failed: {resp.text.strip()}")
|
||||
self._logged_in = True
|
||||
|
||||
def _get(self, path: str, **params: Any) -> dict[str, Any]:
|
||||
"""GET with auto re-login on 403."""
|
||||
if not self._logged_in:
|
||||
self._login()
|
||||
url = f"{self.base_url}{path}"
|
||||
resp = self._session.get(url, params=params, timeout=self._timeout)
|
||||
if resp.status_code == 403:
|
||||
self._logged_in = False
|
||||
self._login()
|
||||
resp = self._session.get(url, params=params, timeout=self._timeout)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
def maindata(self) -> dict[str, Any]:
|
||||
"""Fetch /sync/maindata — returns server_state + torrents dict.
|
||||
|
||||
server_state contains: dl_info_speed (bytes/s), up_info_speed (bytes/s), etc.
|
||||
torrents is a dict of {hash: {name, state, progress, ...}}.
|
||||
"""
|
||||
return self._get("/sync/maindata")
|
||||
```
|
||||
|
||||
**Endpoints used (all from `/sync/maindata` — single call covers all three widgets):**
|
||||
|
||||
| Widget | Data extracted from `maindata()` |
|
||||
|---|---|
|
||||
| **totals** | `len(response["torrents"])` — count of all listed torrents |
|
||||
| **active** | `filter(t for t in response["torrents"].values() if t["state"] in {"downloading","uploading"})` |
|
||||
| **speed** | `response["server_state"]["dl_info_speed"]` + `["up_info_speed"]` (current instant speed, appended to store) |
|
||||
|
||||
**Note:** `/api/v2/transfer/info` is NOT needed (totals = item count per Q2, not transfer bytes).
|
||||
|
||||
### 2.4 Widget source adapter — `QbittorrentWidgetSource`
|
||||
|
||||
Lives in `widgets/sources.py`, implements `WidgetSource.fetch(service, widget_kind, config)`.
|
||||
Resolves the client inline from `ServiceRecord` (like `PrometheusWidgetSource` does — no new
|
||||
dependency-injection helper needed).
|
||||
|
||||
```python
|
||||
class QbittorrentWidgetSource:
|
||||
"""Fetch qBittorrent data for totals, active, and speed widgets."""
|
||||
|
||||
async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
|
||||
try:
|
||||
if service is None:
|
||||
return {"error": "qBittorrent widget is missing its service"}
|
||||
base_url = str(service.config.get("base_url") or "")
|
||||
username = str(service.secrets.get("username") or "")
|
||||
password = str(service.secrets.get("password") or "")
|
||||
timeout = int(service.config.get("timeout_seconds") or 10)
|
||||
if not base_url or not username or not password:
|
||||
return {"error": "qBittorrent service is missing base_url, username, or password"}
|
||||
|
||||
client = QbittorrentClient(base_url, username, password, timeout)
|
||||
data = await asyncio.wait_for(asyncio.to_thread(client.maindata), timeout=timeout)
|
||||
server_state = data.get("server_state", {})
|
||||
torrents = data.get("torrents", {})
|
||||
|
||||
if widget_kind == "totals":
|
||||
# Q2: count of listed items, broken down by state
|
||||
by_state: dict[str, int] = {}
|
||||
for t in torrents.values():
|
||||
state = str(t.get("state", "unknown"))
|
||||
by_state[state] = by_state.get(state, 0) + 1
|
||||
return {"total": len(torrents), "by_state": by_state}
|
||||
|
||||
if widget_kind == "active":
|
||||
# Q3: downloading or uploading only
|
||||
active = [
|
||||
{"name": t.get("name"), "state": t.get("state"),
|
||||
"size": t.get("size"), "progress": t.get("progress"),
|
||||
"dl_speed": t.get("dlspeed"), "up_speed": t.get("upspeed")}
|
||||
for t in torrents.values()
|
||||
if str(t.get("state", "")) in {"downloading", "uploading"}
|
||||
]
|
||||
return {"torrents": active}
|
||||
|
||||
if widget_kind == "speed":
|
||||
# Q1: append sample + return window as {series} shape
|
||||
# matching PrometheusChartWidget's expected format
|
||||
dl = int(server_state.get("dl_info_speed", 0))
|
||||
up = int(server_state.get("up_info_speed", 0))
|
||||
ts = int(time.time())
|
||||
store = QbittorrentSampleStore()
|
||||
store.append(service.id, ts, dl, up)
|
||||
samples = store.window(service.id)
|
||||
series = [
|
||||
{"label": "download",
|
||||
"points": [{"t": s["ts"] * 1000, "v": s["dl_speed"]} for s in samples]},
|
||||
{"label": "upload",
|
||||
"points": [{"t": s["ts"] * 1000, "v": s["up_speed"]} for s in samples]},
|
||||
]
|
||||
return {"series": series}
|
||||
|
||||
return {"error": f"Unknown qBittorrent widget kind: {widget_kind}"}
|
||||
except asyncio.TimeoutError:
|
||||
return {"error": "qBittorrent data fetch timed out"}
|
||||
except Exception as exc:
|
||||
logger.exception("qbittorrent adapter failed")
|
||||
return {"error": f"qBittorrent fetch failed: {exc}"}
|
||||
```
|
||||
|
||||
**Registered in `SERVICE_ADAPTERS`:**
|
||||
|
||||
```python
|
||||
SERVICE_ADAPTERS: dict[str, WidgetSource] = {
|
||||
"prometheus": PrometheusWidgetSource(),
|
||||
"qbittorrent": QbittorrentWidgetSource(), # NEW
|
||||
"alertmanager": AlertmanagerWidgetSource(),
|
||||
"jellyfin": JellyfinWidgetSource(),
|
||||
"ssh_tasks": SshTaskWidgetSource(),
|
||||
}
|
||||
```
|
||||
|
||||
### 2.5 Integration registration — `integrations/qbittorrent.py`
|
||||
|
||||
Models config + secret schema on `prometheus.py`.
|
||||
|
||||
```python
|
||||
class QbittorrentConfig(ServiceConfigBase):
|
||||
base_url: ServiceBaseUrl
|
||||
timeout_seconds: int = 10
|
||||
|
||||
|
||||
# No per-widget config needed for any of the three kinds
|
||||
# (all derive from the service connection).
|
||||
class QbittorrentWidgetConfig(WidgetConfigBase):
|
||||
pass
|
||||
|
||||
|
||||
DEFINITION = ServiceDefinition(
|
||||
service_type="qbittorrent",
|
||||
name="qBittorrent",
|
||||
description="Torrent client activity, speeds, and item counts.",
|
||||
config_model=QbittorrentConfig,
|
||||
secret_fields=[
|
||||
SecretField(key="username", label="Username", required=True),
|
||||
SecretField(key="password", label="Password", required=True, helper="Stored encrypted"),
|
||||
],
|
||||
widget_kinds=[
|
||||
widget_kind(kind="totals", name="Totals",
|
||||
description="Count of all listed torrents, broken down by state.",
|
||||
model_cls=QbittorrentWidgetConfig, default_config={},
|
||||
refresh_interval_ms=30_000),
|
||||
widget_kind(kind="active", name="Active torrents",
|
||||
description="Torrents currently downloading or uploading.",
|
||||
model_cls=QbittorrentWidgetConfig, default_config={},
|
||||
refresh_interval_ms=15_000),
|
||||
widget_kind(kind="speed", name="Speed chart",
|
||||
description="Live download/upload speed over a short window.",
|
||||
model_cls=QbittorrentWidgetConfig, default_config={},
|
||||
refresh_interval_ms=5_000),
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
**Registered in `integrations/registry.py`:**
|
||||
|
||||
```python
|
||||
from media_library_viewer_api.integrations.qbittorrent import DEFINITION as QBITTORRENT
|
||||
SERVICE_DEFINITIONS["qbittorrent"] = QBITTORRENT
|
||||
```
|
||||
|
||||
### 2.6 Cascade-delete wiring
|
||||
|
||||
`SettingsStore.delete_service` gains a harness call after the service row is
|
||||
deleted:
|
||||
|
||||
```python
|
||||
def delete_service(self, service_id: str) -> None:
|
||||
self.init_schema()
|
||||
with self.connect() as conn:
|
||||
# existing widget cascade ...
|
||||
conn.execute("DELETE FROM services WHERE id = ?", (service_id,))
|
||||
# NEW: cascade-delete harness-managed data
|
||||
try:
|
||||
from media_library_viewer_api.services.service_data import get_service_data_harness
|
||||
get_service_data_harness().cascade_delete(service_id)
|
||||
except Exception:
|
||||
logger.exception("harness cascade-delete failed for service %s", service_id)
|
||||
```
|
||||
|
||||
The try/except guard prevents a harness failure from blocking service deletion
|
||||
(data cleanup is best-effort; the service row is already gone).
|
||||
|
||||
---
|
||||
|
||||
## 3. MediaIndex migration design (load-bearing)
|
||||
|
||||
### 3.1 Schema migration: add `service_id` column
|
||||
|
||||
The `init_schema` method's `CREATE TABLE IF NOT EXISTS` gains the new column.
|
||||
For existing databases, a migration adds it:
|
||||
|
||||
```sql
|
||||
-- In MEDIA_INDEX_CONCERN.migrations (runs via harness on startup):
|
||||
-- The init_schema already creates the table for new installs WITH service_id.
|
||||
-- This migration handles existing DBs that lack the column.
|
||||
|
||||
-- media_index_impl.py init_schema: add service_id to the CREATE TABLE columns.
|
||||
-- Harness migration (runs on existing DBs):
|
||||
ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT '';
|
||||
```
|
||||
|
||||
The `DEFAULT ''` backfills all existing rows to empty string (the legacy
|
||||
sentinel — see §3.3 for how this is resolved).
|
||||
|
||||
### 3.2 `replace_items` — scoped delete + insert
|
||||
|
||||
```python
|
||||
def replace_items(self, rows: Iterable[dict[str, Any]], service_id: str = "") -> int:
|
||||
self.init_schema()
|
||||
row_list = list(rows)
|
||||
# ... columns list gains "service_id" ...
|
||||
with self.connect() as conn:
|
||||
# SCOPED: only delete this service's rows
|
||||
conn.execute("DELETE FROM media_items WHERE service_id = ?", (service_id,))
|
||||
conn.executemany(
|
||||
f"INSERT OR REPLACE INTO media_items ({','.join(columns)}) VALUES ({placeholders})",
|
||||
[[service_id] + [row.get(column) for column in columns_without_service_id] for row in row_list],
|
||||
)
|
||||
# ... metadata ...
|
||||
return len(row_list)
|
||||
```
|
||||
|
||||
### 3.3 `query` — scoped filter
|
||||
|
||||
```python
|
||||
def query(self, service_id: str = "", ...) -> tuple[list[dict[str, Any]], int]:
|
||||
# ... existing where clauses ...
|
||||
# Add service_id filter: if non-empty, scope; if empty (legacy), show all
|
||||
# (backward-compatibility for the period before multi-instance is wired in UI)
|
||||
if service_id:
|
||||
where.append("service_id = ?")
|
||||
params.append(service_id)
|
||||
# ... rest unchanged ...
|
||||
```
|
||||
|
||||
**Backfill semantics:** existing rows get `service_id = ''` (empty string). When
|
||||
`service_id` is empty string in the query, the filter is skipped, so the Media
|
||||
page shows all items (backward-compatible behavior). When a specific Jellyfin
|
||||
service triggers a rebuild, `replace_items(rows, service_id=that_service)` scopes
|
||||
the delete + insert. New builds set the real service_id; legacy rows remain
|
||||
visible until a rebuild replaces them.
|
||||
|
||||
### 3.4 `build_media_index` — thread service_id
|
||||
|
||||
`build_media_index` already receives no `service_id` today. Add it as a
|
||||
parameter and pass it through to `replace_items`:
|
||||
|
||||
```python
|
||||
def build_media_index(
|
||||
client, user_id, libraries, index=None, page_size=500,
|
||||
media_root="", fallback_prefix="",
|
||||
progress_callback=None, should_cancel=None,
|
||||
service_id: str = "", # NEW
|
||||
) -> int:
|
||||
# ... existing logic ...
|
||||
# Pass service_id to replace_items:
|
||||
processed_total = index.replace_items(normalized_rows, service_id=service_id)
|
||||
```
|
||||
|
||||
### 3.5 Worker — already threads `service_id`, just pass it to `build_media_index`
|
||||
|
||||
The worker's `run_build(final_index_path, staging_index_path, service_id="")`
|
||||
already receives `service_id` from argparse. The only change: pass it to
|
||||
`build_media_index(..., service_id=service_id)`.
|
||||
|
||||
### 3.6 Media router — thread `service_id` into `query_media`
|
||||
|
||||
`query_media` already resolves `client: JellyfinClient = Depends(get_jellyfin_client)`.
|
||||
The service_id is available via request query param (the dependency layer resolves
|
||||
it from `?jellyfin_service_id=...`). Add it to the query call:
|
||||
|
||||
```python
|
||||
@router.get("/query")
|
||||
def query_media(
|
||||
...,
|
||||
jellyfin_service_id: str | None = None, # already available pattern
|
||||
index: MediaIndex = Depends(get_media_index),
|
||||
) -> dict[str, Any]:
|
||||
...
|
||||
rows, total = index.query(service_id=jellyfin_service_id or "", ...)
|
||||
```
|
||||
|
||||
### 3.7 MediaIndex concern registration
|
||||
|
||||
```python
|
||||
MEDIA_INDEX_CONCERN = StorageConcern(
|
||||
concern_key="media_index",
|
||||
db_filename="media_index.sqlite", # SAME FILE, unchanged path
|
||||
migrations=[
|
||||
"ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT ''",
|
||||
# This ALTER is idempotent-safe: init_schema creates the table WITH
|
||||
# service_id for new installs; this migration adds it to existing DBs.
|
||||
# SQLite ALTER TABLE ADD COLUMN is a no-op if the column already exists
|
||||
# (we guard with a PRAGMA check in run_migrations, or catch the error).
|
||||
],
|
||||
tables=["media_items"],
|
||||
)
|
||||
```
|
||||
|
||||
**IMPORTANT — ALTER TABLE idempotency:** SQLite raises an error if the column
|
||||
already exists. The `run_migrations` method should catch this per-statement or
|
||||
pre-check via `PRAGMA table_info`. Design choice: wrap each migration in a
|
||||
try/except for "duplicate column name" errors:
|
||||
|
||||
```python
|
||||
def run_migrations(self) -> None:
|
||||
for concern in self._concerns.values():
|
||||
path = self.db_path(concern.concern_key)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with sqlite3.connect(path, timeout=30) as conn:
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
for stmt in concern.migrations:
|
||||
try:
|
||||
conn.executescript(stmt)
|
||||
except sqlite3.OperationalError as exc:
|
||||
if "duplicate column name" not in str(exc).lower():
|
||||
raise
|
||||
```
|
||||
|
||||
### 3.8 File location confirmation
|
||||
|
||||
`DEFAULT_INDEX_PATH = Path(".cache/media_library_viewer/media_index.sqlite")` —
|
||||
unchanged. The harness `base_dir` defaults to `Path(".cache/media_library_viewer")`
|
||||
(the same parent). The `db_filename = "media_index.sqlite"` matches. No data move.
|
||||
|
||||
---
|
||||
|
||||
## 4. Frontend design
|
||||
|
||||
### 4.1 Three widget components
|
||||
|
||||
All three live in `frontend/src/widgets/`, modeled on existing patterns
|
||||
(`AlertmanagerAlertsWidget`, `PrometheusChartWidget`, `MetricCard`).
|
||||
|
||||
**`QbittorrentTotalsWidget.tsx`** — MetricCard-style count tile:
|
||||
|
||||
```tsx
|
||||
// Renders {total: number, by_state: {...}} from useWidgetData.
|
||||
// Uses SectionCard + numeric display (like BackupsWidget / MetricCard).
|
||||
// Shows total count prominently + state breakdown badges.
|
||||
```
|
||||
|
||||
**`QbittorrentActiveTorrentsWidget.tsx`** — active torrents list:
|
||||
|
||||
```tsx
|
||||
// Renders {torrents: [{name, state, size, progress, dl_speed, up_speed}]}.
|
||||
// Uses SectionCard + a compact list/table (DataTable or manual Table rows).
|
||||
// Shows name, state badge, progress bar, speeds. Max 10 rows with scroll.
|
||||
```
|
||||
|
||||
**`QbittorrentSpeedWidget.tsx`** — speed chart reusing Change A's renderer:
|
||||
|
||||
```tsx
|
||||
// Renders {series: [{label, points:[{t,v}]}]} — IDENTICAL to PrometheusChartWidget.
|
||||
// Two options:
|
||||
// (a) Import PrometheusChartWidget directly and pass props (if its props accept
|
||||
// the series externally rather than via useWidgetData).
|
||||
// (b) Extract the recharts rendering into a shared <LineSeriesChart series={...} />
|
||||
// component that both PrometheusChartWidget and QbittorrentSpeedWidget use.
|
||||
//
|
||||
// RECOMMENDED: option (b) — extract a shared LineSeriesChart component (~40 lines)
|
||||
// into frontend/src/components/LineSeriesChart.tsx. Both widgets call useWidgetData
|
||||
// independently (different refresh intervals) but share the renderer.
|
||||
```
|
||||
|
||||
**`LineSeriesChart.tsx`** (shared renderer extraction):
|
||||
|
||||
```tsx
|
||||
// Extracts: mergeSeries, formatTime, CHART_COLORS, and the <ResponsiveContainer>
|
||||
// + <LineChart> JSX from PrometheusChartWidget.
|
||||
// Props: { series: ChartSeries[], height?: number }
|
||||
// PrometheusChartWidget becomes a thin wrapper: useWidgetData → <LineSeriesChart series={data.data.series} />
|
||||
// QbittorrentSpeedWidget: same pattern, different refresh interval (5s vs 60s).
|
||||
```
|
||||
|
||||
This keeps the recharts rendering in ONE place (no duplication) while letting each
|
||||
widget own its polling lifecycle.
|
||||
|
||||
### 4.2 Frontend registry binding (`integrations/registry.ts`)
|
||||
|
||||
```typescript
|
||||
qbittorrent: {
|
||||
serviceType: "qbittorrent",
|
||||
name: "qBittorrent",
|
||||
description: "Torrent client activity, speeds, and item counts.",
|
||||
widgets: [
|
||||
{ kind: "totals", name: "Totals", description: "...",
|
||||
refreshIntervalMs: 30_000, defaultConfig: {},
|
||||
configSchema: { type: "object", properties: {}, required: [] },
|
||||
component: QbittorrentTotalsWidget },
|
||||
{ kind: "active", name: "Active torrents", description: "...",
|
||||
refreshIntervalMs: 15_000, defaultConfig: {},
|
||||
configSchema: { type: "object", properties: {}, required: [] },
|
||||
component: QbittorrentActiveTorrentsWidget },
|
||||
{ kind: "speed", name: "Speed chart", description: "...",
|
||||
refreshIntervalMs: 5_000, defaultConfig: {},
|
||||
configSchema: { type: "object", properties: {}, required: [] },
|
||||
component: QbittorrentSpeedWidget },
|
||||
],
|
||||
},
|
||||
```
|
||||
|
||||
### 4.3 Types
|
||||
|
||||
No new TypeScript types needed for widget payloads — data flows through the
|
||||
existing `WidgetDataResponse` + `useWidgetData` polling. The series shape
|
||||
(`{series:[{label,points:[{t,v}]}]}`) is already used by `PrometheusChartWidget`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Tests
|
||||
|
||||
### 5.1 Backend tests
|
||||
|
||||
| Test file | Coverage |
|
||||
|---|---|
|
||||
| `backend/tests/test_service_data.py` (NEW) | Harness: register, run_migrations (creates tables), cascade_delete (removes rows by service_id), migration idempotency (ALTER doesn't crash on re-run) |
|
||||
| `backend/tests/test_qbittorrent_store.py` (NEW) | Store: append + prune (MAX_SAMPLES cap), window (returns samples in order), service_id isolation (two services don't cross-contaminate) |
|
||||
| `backend/tests/test_qbittorrent_client.py` (NEW) | Client: login flow (POST /auth/login → Ok.), cookie reuse, 403 → re-login, maindata parsing, timeout handling |
|
||||
| `backend/tests/test_widgets.py` (extend) | `QbittorrentWidgetSource`: totals (counts all torrents), active (filters state), speed (appends sample + returns {series}), missing-service error, timeout error |
|
||||
|
||||
### 5.2 Frontend tests
|
||||
|
||||
| Test file | Coverage |
|
||||
|---|---|
|
||||
| `QbittorrentTotalsWidget.test.tsx` (NEW) | Loading skeleton, error alert, rendered count + state badges |
|
||||
| `QbittorrentActiveTorrentsWidget.test.tsx` (NEW) | Loading, error, rendered torrent rows |
|
||||
| `QbittorrentSpeedWidget.test.tsx` (NEW) | Loading, error, rendered chart (series present) |
|
||||
| `LineSeriesChart.test.tsx` (NEW) | Renders lines from series data, empty state |
|
||||
|
||||
### 5.3 Existing MediaIndex tests must stay green
|
||||
|
||||
`backend/tests/test_media_index.py` exercises `replace_items`, `query`, `status`.
|
||||
After migration, these call with the new `service_id=""` default (backward-compatible).
|
||||
The tests pass unchanged because empty-string service_id shows all rows.
|
||||
|
||||
---
|
||||
|
||||
## 6. Slice plan (for tasks.md)
|
||||
|
||||
Four slices, each ≤400 changed lines, each leaving `pytest` + `npm run build` +
|
||||
`npm run lint` green. Slices 1–2 prove the harness; Slice 3 migrates MediaIndex;
|
||||
Slice 4 wires cascade-delete end-to-end.
|
||||
|
||||
### Slice 1: Harness + QbittorrentSampleStore + QbittorrentClient + integration (~280–350 lines)
|
||||
|
||||
**Files:**
|
||||
|
||||
- `backend/src/media_library_viewer_api/services/service_data.py` (NEW — harness + StorageConcern)
|
||||
- `backend/src/media_library_viewer_api/services/qbittorrent_store.py` (NEW — store + concern)
|
||||
- `backend/src/media_library_viewer_api/clients/qbittorrent.py` (NEW — client)
|
||||
- `backend/src/media_library_viewer_api/integrations/qbittorrent.py` (NEW — definition)
|
||||
- `backend/src/media_library_viewer_api/integrations/registry.py` (MODIFY — add qbittorrent)
|
||||
- `backend/src/media_library_viewer_api/main.py` (MODIFY — call harness init in lifespan)
|
||||
- `backend/tests/test_service_data.py` (NEW)
|
||||
- `backend/tests/test_qbittorrent_store.py` (NEW)
|
||||
- `backend/tests/test_qbittorrent_client.py` (NEW)
|
||||
|
||||
**Exit gate:** harness creates tables + runs migrations + cascade_delete works in
|
||||
tests. qBit client login/maindata tested with mocked HTTP. No frontend changes yet.
|
||||
|
||||
### Slice 2: Widget adapter + frontend widgets + registry binding (~320–400 lines)
|
||||
|
||||
**Files:**
|
||||
|
||||
- `backend/src/media_library_viewer_api/widgets/sources.py` (MODIFY — add QbittorrentWidgetSource + SERVICE_ADAPTERS entry)
|
||||
- `backend/tests/test_widgets.py` (EXTEND — qBit adapter tests)
|
||||
- `frontend/src/components/LineSeriesChart.tsx` (NEW — shared renderer extracted from PrometheusChartWidget)
|
||||
- `frontend/src/widgets/PrometheusChartWidget.tsx` (MODIFY — use LineSeriesChart)
|
||||
- `frontend/src/widgets/QbittorrentTotalsWidget.tsx` (NEW)
|
||||
- `frontend/src/widgets/QbittorrentActiveTorrentsWidget.tsx` (NEW)
|
||||
- `frontend/src/widgets/QbittorrentSpeedWidget.tsx` (NEW)
|
||||
- `frontend/src/widgets/index.ts` (MODIFY — barrel exports)
|
||||
- `frontend/src/integrations/registry.ts` (MODIFY — add qbittorrent binding)
|
||||
- `frontend/src/widgets/__tests__/QbittorrentTotalsWidget.test.tsx` (NEW)
|
||||
- `frontend/src/widgets/__tests__/QbittorrentActiveTorrentsWidget.test.tsx` (NEW)
|
||||
- `frontend/src/widgets/__tests__/QbittorrentSpeedWidget.test.tsx` (NEW)
|
||||
- `frontend/src/components/__tests__/LineSeriesChart.test.tsx` (NEW)
|
||||
|
||||
**Exit gate:** qBit widgets render loading/error/data states; speed widget shows
|
||||
a recharts line chart from {series}; LineSeriesChart is shared with PrometheusChartWidget.
|
||||
|
||||
### Slice 3: MediaIndex migration onto harness (~200–280 lines)
|
||||
|
||||
**Files:**
|
||||
|
||||
- `backend/src/media_library_viewer_api/services/service_data.py` (MODIFY — register MEDIA_INDEX_CONCERN)
|
||||
- `backend/src/media_library_viewer_api/services/media_index_impl.py` (MODIFY — add service_id to schema, replace_items, query)
|
||||
- `backend/src/media_library_viewer_api/services/media_index.py` (re-export unchanged)
|
||||
- `backend/src/media_library_viewer_api/workers/media_index_worker.py` (MODIFY — pass service_id to build_media_index)
|
||||
- `backend/src/media_library_viewer_api/routers/media.py` (MODIFY — thread jellyfin_service_id into query_media)
|
||||
- `backend/tests/test_media_index.py` (EXTEND — service_id-scoped replace_items + query)
|
||||
|
||||
**Exit gate:** all existing MediaIndex tests pass unchanged (empty-string default);
|
||||
new tests verify scoped delete/insert/query. Media page works identically.
|
||||
|
||||
### Slice 4: Cascade-delete end-to-end + integration test (~80–120 lines)
|
||||
|
||||
**Files:**
|
||||
|
||||
- `backend/src/media_library_viewer_api/services/settings_store.py` (MODIFY — call harness.cascade_delete in delete_service)
|
||||
- `backend/tests/test_services.py` (EXTEND — verify cascade-delete removes qBit samples when service is deleted)
|
||||
|
||||
**Exit gate:** deleting a qBittorrent service removes its speed samples; deleting
|
||||
a Jellyfin service removes its media items.
|
||||
|
||||
---
|
||||
|
||||
## 7. Key design decisions summary
|
||||
|
||||
| # | Decision | Rationale |
|
||||
|---|---|---|
|
||||
| D1 | Harness is lifecycle-only (migrations, service_id, cascade) | n=2 justifies abstraction, but data-ops differ wildly (append/window vs replace_all/query). Generalize only the shared lifecycle. |
|
||||
| D2 | Per-concern DB files via harness `base_dir` + `db_filename` | media_index.sqlite stays put (no data move); qbittorrent.db is new. Separate writers. |
|
||||
| D3 | `run_migrations` catches "duplicate column name" per-statement | SQLite ALTER TABLE ADD COLUMN is not idempotent; migrations must not crash on re-run. |
|
||||
| D4 | Speed widget extracts a shared `LineSeriesChart` component | Both PrometheusChartWidget and QbittorrentSpeedWidget need the same recharts renderer; extract it once (DRY) rather than duplicating or tightly coupling. |
|
||||
| D5 | qBit adapter resolves client inline from ServiceRecord (like PrometheusWidgetSource) | No new dependency-injection helper; the adapter gets config+secrets from the service record. |
|
||||
| D6 | MediaIndex backfill uses `service_id = ''` (empty string) sentinel | Backward-compatible: existing tests + Media page work unchanged; empty-string queries skip the filter (show all). Real service_ids overwrite on next rebuild. |
|
||||
| D7 | `replace_items` deletes `WHERE service_id = ?` instead of all | Fixes the existing global-clear bug (building for one Jellyfin wipes others). |
|
||||
| D8 | Cascade-delete is best-effort (try/except) in delete_service | Harness failure must not block service deletion; data cleanup is non-critical. |
|
||||
| D9 | Speed sample timestamps multiplied by 1000 for frontend | PrometheusChartWidget expects `t` in milliseconds (JS epoch); SQLite stores Unix seconds. |
|
||||
| D10 | MAX_SAMPLES = 120 cap with per-append prune | ~2 min history at 1s poll; bounded DB growth; single-statement prune. |
|
||||
|
||||
---
|
||||
|
||||
## 8. Risks and mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| **MediaIndex migration breaks existing tests.** | `service_id` defaults to `""`; empty-string queries skip the filter → all rows visible. Existing tests pass unchanged. |
|
||||
| **ALTER TABLE fails on fresh installs** where init_schema already created the column. | `run_migrations` catches "duplicate column name" errors per-statement. |
|
||||
| **Harness lazy-init hides migration failures.** | `run_migrations` runs on first access (startup lifespan); failures raise (not swallowed) except the expected duplicate-column case. |
|
||||
| **Speed widget poll interval (5s) too aggressive.** | 5s is the default; configurable via widget config if needed. MAX_SAMPLES cap bounds storage. |
|
||||
| **qBit cookie expiry between polls.** | Client re-logins on 403 transparently; persistent auth failure surfaces as widget error state. |
|
||||
| **LineSeriesChart extraction breaks PrometheusChartWidget.** | Slice 2 includes tests for both; extraction is mechanical (move JSX + helpers, pass series as prop). |
|
||||
| **Review budget (>400 lines).** | Four slices, each ≤400 lines. Slices 1–2 are independently shippable (qBit works without MediaIndex migration). |
|
||||
|
||||
---
|
||||
|
||||
## 9. Data flow diagrams
|
||||
|
||||
### 9.1 Speed widget data flow (InService path)
|
||||
|
||||
```
|
||||
Dashboard poll (useWidgetData, 5s)
|
||||
└► GET /api/widgets/instances/{id}/data
|
||||
└► QbittorrentWidgetSource.fetch(service, "speed", {})
|
||||
├► QbittorrentClient.maindata()
|
||||
│ └► /api/v2/sync/maindata → {server_state:{dl_info_speed, up_info_speed}}
|
||||
├► QbittorrentSampleStore.append(service.id, ts, dl, up)
|
||||
│ └► INSERT + prune (keep 120)
|
||||
├► QbittorrentSampleStore.window(service.id)
|
||||
│ └► SELECT ts, dl_speed, up_speed → [{ts, dl_speed, up_speed}]
|
||||
└► return {series: [{label:"download", points:[{t,v}]}, {label:"upload", points:[{t,v}]}]}
|
||||
└► Frontend: QbittorrentSpeedWidget → <LineSeriesChart series={...} />
|
||||
```
|
||||
|
||||
### 9.2 Cascade-delete data flow
|
||||
|
||||
```
|
||||
DELETE /api/services/instances/{id}
|
||||
└► SettingsStore.delete_service(id)
|
||||
├► DELETE FROM dashboard_widgets WHERE service_id = ?
|
||||
├► DELETE FROM services WHERE id = ?
|
||||
└► ServiceDataHarness.cascade_delete(id)
|
||||
├► DELETE FROM qbittorrent_speed_samples WHERE service_id = ?
|
||||
└► DELETE FROM media_items WHERE service_id = ?
|
||||
```
|
||||
@@ -0,0 +1,148 @@
|
||||
# SDD Proposal: Service Storage Harness (with qBittorrent widgets + MediaIndex migration)
|
||||
|
||||
**Change:** `service-storage-harness`
|
||||
**Phase:** proposal
|
||||
**Date:** 2026-07-08
|
||||
|
||||
## 1. Problem / Why Now
|
||||
|
||||
Two things are happening at once, and this change addresses both:
|
||||
|
||||
1. **Feature request — qBittorrent widgets.** The operator wants at-a-glance qBittorrent visibility on the Manage dashboard: active downloads/uploads, total bytes transferred, and a download/upload speed indicator. qBittorrent is currently a blind spot — it is not modeled in the service registry and exposes no widgets.
|
||||
|
||||
2. **Platform gap — services have no owned persistence.** Manage already has a service registry (machines, Jellyfin, Grafana, Prometheus, Alertmanager, Jellyseerr, ssh_tasks) and a configurable widget system. But when a service needs to *remember operational data over time*, there is no shared answer. `MediaIndex` is a one-off: it owns its own `media_index.db`, runs its own schema, and is built by a dedicated subprocess worker. It is a "special snowflake." Rather than add a *second* snowflake for qBittorrent speed history, we extract the shared lifecycle into a small **`ServiceDataHarness`** and prove it with qBittorrent, then migrate `MediaIndex` onto it so the pattern has two evidence-based consumers.
|
||||
|
||||
The two are bundled because the second service (qBittorrent) is the one that justifies generalizing from the first (`MediaIndex`) — n=2 is what makes the abstraction worth its cost.
|
||||
|
||||
## 2. Target Users and Situations
|
||||
|
||||
- **Primary users:** Homelab operators running qBittorrent alongside Manage, who want download/upload activity visible without opening the qBittorrent UI.
|
||||
- **Workflow moments:**
|
||||
- Glance at the dashboard: "is a download running, how fast, how much has been transferred?"
|
||||
- Decide whether qBittorrent is healthy without leaving Manage.
|
||||
- Browse the Jellyfin media catalog (unchanged UX) — which now runs on the same storage harness, validating the abstraction.
|
||||
- **Urgency:** Medium. The feature is valuable but not breaking; the platform refactor is opportunistic (do it now while only two consumers exist, before a third snowflake appears).
|
||||
|
||||
## 3. Product Outcome
|
||||
|
||||
After this change, an authenticated user can:
|
||||
|
||||
- Register one or more **qBittorrent service instances** in the existing Services UI (URL + username + password, stored as encrypted secrets — same posture as Grafana/Prometheus).
|
||||
- Place three discrete **qBittorrent widget kinds** on the dashboard:
|
||||
1. **Totals tile** — count of currently-listed torrents (total items in the qBittorrent list, which may exceed the active count because rate/connection limits leave some torrents non-transferring). NOT cumulative bytes transferred.
|
||||
2. **Active torrents list** — torrents whose state is `downloading` or `uploading`.
|
||||
3. **Speed chart** — live download/upload speed over a short rolling window, rendered with the **`PrometheusChartWidget` recharts renderer established by the `prometheus-direct-charting` change** (fed from the `QbittorrentSampleStore` via an InService-style data path returning the same `{series}` shape). No hand-rolled SVG sparkline; the thin-dashboard rule was already repealed for recharts charting by the foundational change.
|
||||
- Continue using the Jellyfin **Media page** exactly as before; its underlying storage moves onto the harness transparently and additionally becomes **multi-instance capable** (scoped per Jellyfin service).
|
||||
|
||||
## 4. Scope Boundaries and Non-Goals
|
||||
|
||||
### In scope
|
||||
|
||||
- **`ServiceDataHarness`** — a general lifecycle layer: per-concern DB files, per-integration schema migrations, `service_id` scoping of all tables, and cascade-delete when a service instance is removed.
|
||||
- **qBittorrent integration** — new `integrations/qbittorrent.py` (config + secret schema + widget kinds), new `clients/qbittorrent.py` (Web API client, cookie login), a `QbittorrentSampleStore` (speed samples), widget source adapter(s) in `widgets/sources.py`, registry entry.
|
||||
- **qBittorrent frontend** — three widget components under `frontend/src/widgets/`, binding in `integrations/registry.ts`, types, API client functions.
|
||||
- **MediaIndex migration** — `MediaIndex` registered with the harness, `media_items` scoped by `service_id`, subprocess worker updated. The `media_index.db` file location is unchanged (only gains a column + harness registration) to minimize churn on a load-bearing feature.
|
||||
- **Cascade-delete wiring** — removing a service instance cleans up its owned data in every harness-managed table.
|
||||
|
||||
### Non-goals (explicitly out of scope)
|
||||
|
||||
- **Torrent management UI** — no add/pause/delete/recheck/priority UI. Read-only visibility only.
|
||||
- **A generic time-series database.** The harness owns *lifecycle*, not a generic `(service_id, key, ts, value)` table. Each integration owns its own schema and operations.
|
||||
- **A generic CRUD/ORM layer.** Stores keep bespoke operations (`append/window` vs `replace_all/query`); only the lifecycle is shared.
|
||||
- **Per-user or per-tenant storage partitioning.** Storage is scoped by `service_id` only.
|
||||
- **Re-indexing/migrating existing media data.** The `media_index.db` file stays in place; the migration is a schema column add + harness registration, not a data move.
|
||||
- **WebSocket / push updates.** Polling via existing `useWidgetData(widgetId, refreshIntervalMs)` is sufficient.
|
||||
- **Transfer-byte totals.** The totals widget counts torrent *items*, not cumulative bytes uploaded/downloaded (per the §8 Q2 resolution). Byte totals are out of scope.
|
||||
- **qBittorrent Prometheus exporter.** Not built; the speed chart is sourced from the local `QbittorrentSampleStore`, not Prometheus.
|
||||
|
||||
## 5. High-Level Approach
|
||||
|
||||
### 5.1 Architectural decisions (locked during grilling)
|
||||
|
||||
| Decision | Outcome |
|
||||
|---|---|
|
||||
| **Storage tech** | SQLite. Low volume; proven in stack. |
|
||||
| **Abstraction level** | `ServiceDataHarness` owns the *lifecycle* (DB filename, migrations, `service_id` scoping, cascade-delete). Each integration owns its *operations* in a bespoke Store. General where shared; bespoke where not. |
|
||||
| **MediaIndex migration** | Included in this change, but **sequenced**: harness + qBit proven first; MediaIndex folded in after. Inside one change, never two simultaneous risks. |
|
||||
| **DB topology** | Per-concern DB files. `media_index.db` keeps its file (gains `service_id` + registration); new `qbittorrent.db`. |
|
||||
|
||||
### 5.2 Backend
|
||||
|
||||
1. **`ServiceDataHarness`** (new, `services/service_data.py` or similar):
|
||||
- A registry of *concerns*: each integration declares a DB filename, an ordered list of migration SQL statements, the tables it owns, and a `service_id` column convention.
|
||||
- On startup: runs pending migrations per concern DB.
|
||||
- On service deletion: cascades — for each concern table, `DELETE FROM <table> WHERE service_id = ?`.
|
||||
- Provides connection management per concern DB (separate connections → bulk media writes don't share a writer with chatty qBit appends).
|
||||
|
||||
2. **qBittorrent client** (`clients/qbittorrent.py`):
|
||||
- Login via `/api/v2/auth/login` → cookie session; reuse cookie across calls; re-login on 403.
|
||||
- Endpoints used: `/api/v2/transfer/info` (global totals), `/api/v2/sync/maindata` (active torrents + current speeds), `/api/v2/torrents/info` (filtered lists if needed).
|
||||
- Timeout + reachability handling consistent with `JellyfinClient`/`GrafanaWidgetSource`.
|
||||
|
||||
3. **`QbittorrentSampleStore`** (new):
|
||||
- Table `qbittorrent_speed_samples(service_id TEXT, ts INTEGER, dl_speed INTEGER, up_speed INTEGER)` in `qbittorrent.db`.
|
||||
- Operations: `append(service_id, ts, dl, up)`, `window(service_id, since_ts)`, `prune(service_id, older_than_ts)`.
|
||||
- Pruning runs on each append (cap retention to the configured window, e.g. 120 samples).
|
||||
|
||||
4. **Widget source adapter** — `QbittorrentWidgetSource` in `widgets/sources.py` implementing the existing `WidgetSource.fetch(service, widget_kind, config)` contract. For the speed kind, `fetch` appends a sample to the store and returns the current window for rendering.
|
||||
|
||||
5. **Integration registration** — add `qbittorrent` to `integrations/registry.py` with config schema (base URL, timeout) and secret schema (username, password); declare widget kinds.
|
||||
|
||||
6. **MediaIndex migration** — register `MediaIndex` as a concern; add `service_id` column (backfill existing rows with the default/local Jellyfin service id); update `replace_items`/`query`/worker to scope by `service_id`; route through the harness connection.
|
||||
|
||||
### 5.3 Frontend
|
||||
|
||||
1. **Three widget components** under `frontend/src/widgets/`:
|
||||
- `QbittorrentTotalsWidget.tsx` — numeric tiles for all-time bytes (reuses `MetricCard`/`SectionCard`).
|
||||
- `QbittorrentActiveTorrentsWidget.tsx` — list/table of active torrents (reuses the already-migrated `DataTable` + `@tanstack/react-table`; **no new DataGrid migration risk**).
|
||||
- `QbittorrentSpeedWidget.tsx` — visual form decided by §8 Q1 (numeric tiles + delta, or sparkline if an exception is granted, or a PrometheusMetricWidget binding if the exporter path is chosen).
|
||||
2. **Registry binding** — add a `qbittorrent` entry to `SERVICE_REGISTRY` in `frontend/src/integrations/registry.ts` mapping the three widget kinds to components + config schemas.
|
||||
3. **Types + API client** — `frontend/src/types/index.ts` and `frontend/src/api/client.ts` gain qBittorrent-aware widget kinds only (data flows through the existing `useWidgetData` polling; no new endpoints beyond widget CRUD).
|
||||
|
||||
### 5.4 Type contracts
|
||||
|
||||
- Backend Pydantic models for qBittorrent config/secret schema in `integrations/qbittorrent.py`.
|
||||
- Frontend TypeScript interfaces for qBittorrent widget payloads.
|
||||
- Harness has no data-shape types of its own (it is lifecycle-only) — keeping the seam clean.
|
||||
|
||||
## 6. Success Criteria / Acceptance Criteria
|
||||
|
||||
1. A user can register a qBittorrent service instance (URL + username + password) and the three widget kinds appear as bindable on the dashboard.
|
||||
2. Totals tile shows all-time bytes from `transferInfo`; active list shows downloading/uploading torrents; speed indicator reflects current rates.
|
||||
3. `ServiceDataHarness` runs migrations on startup and cascades deletes across qBittorrent samples *and* media items when a service is removed.
|
||||
4. `MediaIndex` continues to power the Media page identically (all existing Media tests green) and is now scoped by `service_id`.
|
||||
5. Removing a Jellyfin service removes only that service's media rows; removing a qBittorrent service removes only that service's speed samples.
|
||||
6. A misconfigured/unreachable qBittorrent instance degrades gracefully per-widget (error state), the rest of the dashboard renders.
|
||||
7. Backend tests (`pytest`) and frontend `npm run build` + `npm run lint` stay green.
|
||||
8. No secrets land in widget `config_json`; qBittorrent password is encrypted via the existing Fernet path.
|
||||
9. No new charting dependency is added unless §8 Q1 grants an explicit exception.
|
||||
|
||||
## 7. Risks and Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| **Compound risk: new abstraction × load-bearing refactor.** A wrong harness shape breaks the Media page. | Sequence inside the change: harness + qBit land and pass tests first; MediaIndex migrates only after the harness shape is settled in code. |
|
||||
| **Harness over-generalization.** Building a meta-framework from n=2. | Keep the harness lifecycle-only; do NOT add a generic value table or generic CRUD. Each store keeps bespoke operations. Extract further only when a third shape appears. |
|
||||
| **Thin-dashboard rule collision (§8 Q1).** In-app speed graph violates the stated "no in-app charting" rule. | Resolve in the question round before spec; default to the rule-compliant numeric-tile + Grafana deep-link option unless an explicit exception is granted. |
|
||||
| **qBittorrent auth lifecycle.** Cookie expiry / 403 handling. | Re-login transparently on 403; short request timeout; surface persistent auth failure as widget error state. |
|
||||
| **MediaIndex backfill correctness.** Adding `service_id` to existing rows. | Backfill all existing rows to the default/local Jellyfin service id; migration is additive; existing Media tests must pass unchanged. |
|
||||
| **Review budget (>400 changed lines).** Bundled scope is large. | Slice into chained PRs (Slice 1: harness + qBit; Slice 2: MediaIndex migration). Each slice leaves `npm run build` + `npm run lint` + `pytest` green. |
|
||||
| **DataGrid migration (config rule callout).** | Not a risk here: the active-torrents list reuses the already-migrated `DataTable` (`@tanstack/react-table`). No new DataGrid migration is introduced. The key technical risks are the harness abstraction and the MediaIndex refactor, above. |
|
||||
|
||||
## 8. Resolved Questions (question round complete)
|
||||
|
||||
All five product/semantic questions resolved during grilling + the Change A (`prometheus-direct-charting`) lifecycle:
|
||||
|
||||
- **Q1 — Speed visualization.** RESOLVED: the thin-dashboard rule was repealed by Change A; in-app recharts charting is now sanctioned. The qBit speed widget reuses the `PrometheusChartWidget` recharts renderer, fed from `QbittorrentSampleStore.window()` via an InService data path returning the same `{series}` shape. No SVG sparkline, no Grafana dependency.
|
||||
- **Q2 — "Totals" semantics.** RESOLVED: count of currently-listed torrent items (total in the qBittorrent list), which may exceed the active count due to rate/connection limits. NOT transfer bytes, NOT per-session counters.
|
||||
- **Q3 — "Active" definition.** RESOLVED: torrents in state `downloading` or `uploading` only.
|
||||
- **Q4 — Instances.** RESOLVED: support N qBittorrent instances, each a service row, independently scoped by `service_id`.
|
||||
- **Q5 — Auth model.** RESOLVED: username/password login → cookie session, encrypted via Fernet. No reverse-proxy no-auth flag in this pass.
|
||||
|
||||
## 9. Future Phases
|
||||
|
||||
1. **Third service consumer** — when a service with a genuinely new storage shape arrives, reconsider promoting the harness toward a broader abstraction (evidence-based, n=3).
|
||||
2. **Torrent management** — add/pause/delete actions (would require write endpoints + confirmation UX).
|
||||
3. **Storage admin UI** — surface harness-managed table sizes and a "clear cached data" action per service.
|
||||
4. **Reverse-proxy / no-auth qBittorrent flag** — for setups behind Authentik where native login is bypassed.
|
||||
5. **Transfer-byte totals** — if desired later, add a separate widget using `transferInfo.globalUploaded/Downloaded`.
|
||||
@@ -0,0 +1,155 @@
|
||||
# SDD Spec: Service Storage Harness (qBittorrent widgets + MediaIndex migration)
|
||||
|
||||
**Change:** `service-storage-harness`
|
||||
**Phase:** spec
|
||||
**Date:** 2026-07-09
|
||||
|
||||
This spec defines the acceptance requirements for the change, derived from the reconciled `proposal.md` and `design.md`. Requirements are testable.
|
||||
|
||||
## Requirement categories
|
||||
|
||||
1. ServiceDataHarness (lifecycle layer)
|
||||
2. QbittorrentSampleStore
|
||||
3. QbittorrentClient
|
||||
4. qBittorrent widget source adapter
|
||||
5. qBittorrent frontend widgets
|
||||
6. MediaIndex migration onto harness
|
||||
7. Cascade-delete wiring
|
||||
8. Test + build greenness
|
||||
|
||||
---
|
||||
|
||||
## 1. ServiceDataHarness (lifecycle layer)
|
||||
|
||||
### SS-101 — Harness is lifecycle-only
|
||||
|
||||
A `ServiceDataHarness` class exists in `backend/src/media_library_viewer_api/services/service_data.py` that owns ONLY lifecycle concerns: per-concern DB provisioning, per-integration migrations, `service_id` cascade-delete. It MUST NOT provide generic data operations (no generic value table, no generic CRUD).
|
||||
|
||||
### SS-102 — Concern registration
|
||||
|
||||
An integration/concern registers via a dataclass carrying: DB filename, ordered migration SQL list, owned-tables list, and `service_id` column name. The harness stores registered concerns.
|
||||
|
||||
### SS-103 — Idempotent migrations
|
||||
|
||||
`run_migrations` runs each concern's migration statements and MUST be idempotent — specifically, re-running `ALTER TABLE ... ADD COLUMN` on an already-migrated DB MUST NOT raise (the harness catches "duplicate column name" per-statement).
|
||||
|
||||
### SS-104 — Cascade-delete across concerns
|
||||
|
||||
`cascade_delete(service_id)` iterates every registered concern and, for each owned table, executes `DELETE FROM <table> WHERE <service_id_column> = ?`. It MUST cover every registered concern (qBittorrent samples + media items after this change).
|
||||
|
||||
## 2. QbittorrentSampleStore
|
||||
|
||||
### SS-105 — Schema
|
||||
|
||||
`QbittorrentSampleStore` owns a `qbittorrent_speed_samples` table with columns `(service_id, ts, dl_speed, up_speed)` and an index on `(service_id, ts)`, in a dedicated `qbittorrent.db` file (per-concern topology).
|
||||
|
||||
### SS-106 — append/window/prune operations
|
||||
|
||||
The store exposes `append(service_id, ts, dl_speed, up_speed)`, `window(service_id, since_ts)` returning ordered rows, and prunes per-append to `MAX_SAMPLES = 120`.
|
||||
|
||||
### SS-107 — Registered as a harness concern
|
||||
|
||||
The qBittorrent sample store is registered with the harness so its table participates in cascade-delete (SS-104).
|
||||
|
||||
## 3. QbittorrentClient
|
||||
|
||||
### SS-108 — Cookie login
|
||||
|
||||
`QbittorrentClient` authenticates via `POST /api/v2/auth/login` with username/password, stores the resulting cookie, and reuses it for subsequent requests. Credentials are resolved from the `ServiceRecord` secrets (Fernet-encrypted at rest).
|
||||
|
||||
### SS-109 — 403 re-login
|
||||
|
||||
On HTTP 403 the client MUST transparently re-login once and retry the request.
|
||||
|
||||
### SS-110 — maindata fetch
|
||||
|
||||
The client exposes `maindata()` calling `/api/v2/sync/maindata` and returning its dict. Errors (timeout, connection, non-2xx) propagate as exceptions for the adapter to catch.
|
||||
|
||||
## 4. qBittorrent widget source adapter
|
||||
|
||||
### SS-111 — Three widget kinds dispatched
|
||||
|
||||
`QbittorrentWidgetSource.fetch(service, widget_kind, config)` dispatches on `widget_kind` ∈ {`totals`, `active`, `speed`}, resolving the client inline from `ServiceRecord` (same pattern as `PrometheusWidgetSource`).
|
||||
|
||||
### SS-112 — totals = item count
|
||||
|
||||
`totals` returns the count of currently-listed torrents from `maindata()` as `{total: int}`. It is NOT cumulative transfer bytes.
|
||||
|
||||
### SS-113 — active = downloading/uploading filter
|
||||
|
||||
`active` returns the subset of torrents whose `state` is `downloading` or `uploading` as `{torrents: [...]}`.
|
||||
|
||||
### SS-114 — speed appends sample + returns series
|
||||
|
||||
`speed` reads the current dl/up speeds from `maindata()`, appends a sample via `QbittorrentSampleStore.append`, and returns `{series: [{label: "download", points: [...]}, {label: "upload", points: [...]}]}` from `.window()` — the exact shape `LineSeriesChart` consumes (timestamps in JS milliseconds).
|
||||
|
||||
### SS-115 — Errors degrade gracefully
|
||||
|
||||
Adapter errors (auth failure, timeout, connection) return `{error: str}` and MUST NOT raise.
|
||||
|
||||
## 5. qBittorrent frontend widgets
|
||||
|
||||
### SS-116 — Three widget components
|
||||
|
||||
`QbittorrentTotalsWidget`, `QbittorrentActiveTorrentsWidget`, `QbittorrentSpeedWidget` exist under `frontend/src/widgets/` and are bound to the `qbittorrent` service binding in `integrations/registry.ts` with their respective kinds.
|
||||
|
||||
### SS-117 — Speed widget reuses shared renderer
|
||||
|
||||
`QbittorrentSpeedWidget` renders via the shared `LineSeriesChart` component (extracted from `PrometheusChartWidget`). No new charting code or charting dependency.
|
||||
|
||||
### SS-118 — LineSeriesChart extraction is non-regressive
|
||||
|
||||
The extraction of the recharts body into `frontend/src/components/LineSeriesChart.tsx` MUST leave `PrometheusChartWidget`'s behavior and tests green; `PrometheusChartWidget` becomes a thin wrapper.
|
||||
|
||||
## 6. MediaIndex migration onto harness
|
||||
|
||||
### SS-119 — service_id column added
|
||||
|
||||
`media_items` gains a `service_id TEXT NOT NULL DEFAULT ''` column via an idempotent harness migration. The `media_index.db` file location is UNCHANGED.
|
||||
|
||||
### SS-120 — Existing rows backfill
|
||||
|
||||
All pre-existing `media_items` rows receive `service_id = ''` (via the column DEFAULT), preserving their visibility.
|
||||
|
||||
### SS-121 — replace_items is scoped (bug fix)
|
||||
|
||||
`replace_items(service_id=...)` deletes only `WHERE service_id = ?` (not a global `DELETE FROM media_items`). This FIXES the latent global-clear bug where rebuilding for one Jellyfin instance wiped another's rows. A regression test MUST prove service A's rows survive service B's rebuild.
|
||||
|
||||
### SS-122 — query is backward-compatible
|
||||
|
||||
`query(service_id="")` returns all rows (no filter); `query(service_id="X")` scopes to service X. Existing tests that pass no `service_id` MUST continue to pass unchanged.
|
||||
|
||||
### SS-123 — Worker threads service_id into rows
|
||||
|
||||
The media index worker (which already receives `--service-id`) passes it into `replace_items` so newly-built rows are stamped with the real service_id.
|
||||
|
||||
### SS-124 — Registered as a harness concern
|
||||
|
||||
MediaIndex registers `media_items` as a harness-owned table so it participates in cascade-delete.
|
||||
|
||||
## 7. Cascade-delete wiring
|
||||
|
||||
### SS-125 — delete_service triggers harness cascade
|
||||
|
||||
`settings_store.delete_service` calls `ServiceDataHarness.cascade_delete(service_id)` after its existing cleanup, in a best-effort try/except (a cascade failure MUST NOT crash the service deletion; it logs and continues).
|
||||
|
||||
### SS-126 — End-to-end cascade across both concerns
|
||||
|
||||
Deleting a service removes both its qBittorrent samples AND its media rows. An integration test MUST prove this across both concerns, and MUST prove rows of OTHER services are preserved.
|
||||
|
||||
## 8. Test + build greenness
|
||||
|
||||
### SS-127 — Backend tests + lint green
|
||||
|
||||
`PYTHONPATH=src python3 -m pytest -q` and `PYTHONPATH=src python3 -m ruff check src tests` from `backend/` MUST pass, including new tests for: harness lifecycle, qBit store, qBit client (login/403/maindata), qBit widget adapter (3 branches), MediaIndex scoped replace_items regression, and cascade-delete integration.
|
||||
|
||||
### SS-128 — Frontend build + lint green
|
||||
|
||||
`npm run build` and `npm run lint` from `frontend/` MUST pass (0 errors). New widget tests cover loading/error/rendered states; PrometheusChartWidget tests stay green (SS-118).
|
||||
|
||||
---
|
||||
|
||||
## Notes for downstream phases
|
||||
|
||||
- **Verify** should confirm SS-101..SS-128 against source; the load-bearing items are SS-121 (scoped replace_items bug fix), SS-122/SS-123 (MediaIndex backward-compat), and SS-118 (non-regressive extraction).
|
||||
- **Sync + archive** follow the Change A pattern (delta into a new canonical domain `service-storage`, then archive to `openspec/changes/archive/2026-07-09-service-storage-harness/`).
|
||||
+139
@@ -0,0 +1,139 @@
|
||||
# Service Storage — Delta (`service-storage-harness`)
|
||||
|
||||
> Change: `service-storage-harness` · Domain: `service-storage` · Phase: **spec** (reconciled during `sdd-sync`).
|
||||
> Distilled verbatim from the verified flat `spec.md` (28 requirements, SS-101 … SS-128) of change
|
||||
> `service-storage-harness`, cross-referenced against `design.md` and `verify-report.md`. Captures
|
||||
> the **durable, post-change end-state contracts** for the lifecycle-only `ServiceDataHarness`, the
|
||||
> qBittorrent store/client/widget stack built on it, the MediaIndex migration onto the harness, and
|
||||
> the cross-concern cascade-delete wiring.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
> The canonical `openspec/specs/service-storage/spec.md` did not exist before this change. All
|
||||
> requirements below are therefore **ADDED** to a new `service-storage` domain; `sdd-sync` copies
|
||||
> them into the canonical spec (native helper rule: when the canonical spec does not exist, the
|
||||
> change spec becomes the new canonical spec).
|
||||
>
|
||||
> Requirement IDs (SS-101 … SS-128) and body text are preserved **exactly** from the verified flat
|
||||
> `spec.md`. Requirements are grouped logically and listed in the following group order:
|
||||
>
|
||||
> - **ServiceDataHarness (lifecycle layer)** — SS-101 … SS-104
|
||||
> - **QbittorrentSampleStore** — SS-105 … SS-107
|
||||
> - **QbittorrentClient** — SS-108 … SS-110
|
||||
> - **qBittorrent widget source adapter** — SS-111 … SS-115
|
||||
> - **qBittorrent frontend widgets** — SS-116 … SS-118
|
||||
> - **MediaIndex migration onto harness** — SS-119 … SS-124
|
||||
> - **Cascade-delete wiring** — SS-125 … SS-126
|
||||
> - **Test and build greenness** — SS-127 … SS-128
|
||||
|
||||
### Requirement: SS-101 — Harness is lifecycle-only
|
||||
|
||||
A `ServiceDataHarness` class exists in `backend/src/media_library_viewer_api/services/service_data.py` that owns ONLY lifecycle concerns: per-concern DB provisioning, per-integration migrations, `service_id` cascade-delete. It MUST NOT provide generic data operations (no generic value table, no generic CRUD).
|
||||
|
||||
### Requirement: SS-102 — Concern registration
|
||||
|
||||
An integration/concern registers via a dataclass carrying: DB filename, ordered migration SQL list, owned-tables list, and `service_id` column name. The harness stores registered concerns.
|
||||
|
||||
### Requirement: SS-103 — Idempotent migrations
|
||||
|
||||
`run_migrations` runs each concern's migration statements and MUST be idempotent — specifically, re-running `ALTER TABLE ... ADD COLUMN` on an already-migrated DB MUST NOT raise (the harness catches "duplicate column name" per-statement).
|
||||
|
||||
### Requirement: SS-104 — Cascade-delete across concerns
|
||||
|
||||
`cascade_delete(service_id)` iterates every registered concern and, for each owned table, executes `DELETE FROM <table> WHERE <service_id_column> = ?`. It MUST cover every registered concern (qBittorrent samples + media items after this change).
|
||||
|
||||
### Requirement: SS-105 — Schema
|
||||
|
||||
`QbittorrentSampleStore` owns a `qbittorrent_speed_samples` table with columns `(service_id, ts, dl_speed, up_speed)` and an index on `(service_id, ts)`, in a dedicated `qbittorrent.db` file (per-concern topology).
|
||||
|
||||
### Requirement: SS-106 — append/window/prune operations
|
||||
|
||||
The store exposes `append(service_id, ts, dl_speed, up_speed)`, `window(service_id, since_ts)` returning ordered rows, and prunes per-append to `MAX_SAMPLES = 120`.
|
||||
|
||||
### Requirement: SS-107 — Registered as a harness concern
|
||||
|
||||
The qBittorrent sample store is registered with the harness so its table participates in cascade-delete (SS-104).
|
||||
|
||||
### Requirement: SS-108 — Cookie login
|
||||
|
||||
`QbittorrentClient` authenticates via `POST /api/v2/auth/login` with username/password, stores the resulting cookie, and reuses it for subsequent requests. Credentials are resolved from the `ServiceRecord` secrets (Fernet-encrypted at rest).
|
||||
|
||||
### Requirement: SS-109 — 403 re-login
|
||||
|
||||
On HTTP 403 the client MUST transparently re-login once and retry the request.
|
||||
|
||||
### Requirement: SS-110 — maindata fetch
|
||||
|
||||
The client exposes `maindata()` calling `/api/v2/sync/maindata` and returning its dict. Errors (timeout, connection, non-2xx) propagate as exceptions for the adapter to catch.
|
||||
|
||||
### Requirement: SS-111 — Three widget kinds dispatched
|
||||
|
||||
`QbittorrentWidgetSource.fetch(service, widget_kind, config)` dispatches on `widget_kind` ∈ {`totals`, `active`, `speed`}, resolving the client inline from `ServiceRecord` (same pattern as `PrometheusWidgetSource`).
|
||||
|
||||
### Requirement: SS-112 — totals = item count
|
||||
|
||||
`totals` returns the count of currently-listed torrents from `maindata()` as `{total: int}`. It is NOT cumulative transfer bytes.
|
||||
|
||||
### Requirement: SS-113 — active = downloading/uploading filter
|
||||
|
||||
`active` returns the subset of torrents whose `state` is `downloading` or `uploading` as `{torrents: [...]}`.
|
||||
|
||||
### Requirement: SS-114 — speed appends sample + returns series
|
||||
|
||||
`speed` reads the current dl/up speeds from `maindata()`, appends a sample via `QbittorrentSampleStore.append`, and returns `{series: [{label: "download", points: [...]}, {label: "upload", points: [...]}]}` from `.window()` — the exact shape `LineSeriesChart` consumes (timestamps in JS milliseconds).
|
||||
|
||||
### Requirement: SS-115 — Errors degrade gracefully
|
||||
|
||||
Adapter errors (auth failure, timeout, connection) return `{error: str}` and MUST NOT raise.
|
||||
|
||||
### Requirement: SS-116 — Three widget components
|
||||
|
||||
`QbittorrentTotalsWidget`, `QbittorrentActiveTorrentsWidget`, `QbittorrentSpeedWidget` exist under `frontend/src/widgets/` and are bound to the `qbittorrent` service binding in `integrations/registry.ts` with their respective kinds.
|
||||
|
||||
### Requirement: SS-117 — Speed widget reuses shared renderer
|
||||
|
||||
`QbittorrentSpeedWidget` renders via the shared `LineSeriesChart` component (extracted from `PrometheusChartWidget`). No new charting code or charting dependency.
|
||||
|
||||
### Requirement: SS-118 — LineSeriesChart extraction is non-regressive
|
||||
|
||||
The extraction of the recharts body into `frontend/src/components/LineSeriesChart.tsx` MUST leave `PrometheusChartWidget`'s behavior and tests green; `PrometheusChartWidget` becomes a thin wrapper.
|
||||
|
||||
### Requirement: SS-119 — service_id column added
|
||||
|
||||
`media_items` gains a `service_id TEXT NOT NULL DEFAULT ''` column via an idempotent harness migration. The `media_index.db` file location is UNCHANGED.
|
||||
|
||||
### Requirement: SS-120 — Existing rows backfill
|
||||
|
||||
All pre-existing `media_items` rows receive `service_id = ''` (via the column DEFAULT), preserving their visibility.
|
||||
|
||||
### Requirement: SS-121 — replace_items is scoped (bug fix)
|
||||
|
||||
`replace_items(service_id=...)` deletes only `WHERE service_id = ?` (not a global `DELETE FROM media_items`). This FIXES the latent global-clear bug where rebuilding for one Jellyfin instance wiped another's rows. A regression test MUST prove service A's rows survive service B's rebuild.
|
||||
|
||||
### Requirement: SS-122 — query is backward-compatible
|
||||
|
||||
`query(service_id="")` returns all rows (no filter); `query(service_id="X")` scopes to service X. Existing tests that pass no `service_id` MUST continue to pass unchanged.
|
||||
|
||||
### Requirement: SS-123 — Worker threads service_id into rows
|
||||
|
||||
The media index worker (which already receives `--service-id`) passes it into `replace_items` so newly-built rows are stamped with the real service_id.
|
||||
|
||||
### Requirement: SS-124 — Registered as a harness concern
|
||||
|
||||
MediaIndex registers `media_items` as a harness-owned table so it participates in cascade-delete.
|
||||
|
||||
### Requirement: SS-125 — delete_service triggers harness cascade
|
||||
|
||||
`settings_store.delete_service` calls `ServiceDataHarness.cascade_delete(service_id)` after its existing cleanup, in a best-effort try/except (a cascade failure MUST NOT crash the service deletion; it logs and continues).
|
||||
|
||||
### Requirement: SS-126 — End-to-end cascade across both concerns
|
||||
|
||||
Deleting a service removes both its qBittorrent samples AND its media rows. An integration test MUST prove this across both concerns, and MUST prove rows of OTHER services are preserved.
|
||||
|
||||
### Requirement: SS-127 — Backend tests + lint green
|
||||
|
||||
`PYTHONPATH=src python3 -m pytest -q` and `PYTHONPATH=src python3 -m ruff check src tests` from `backend/` MUST pass, including new tests for: harness lifecycle, qBit store, qBit client (login/403/maindata), qBit widget adapter (3 branches), MediaIndex scoped replace_items regression, and cascade-delete integration.
|
||||
|
||||
### Requirement: SS-128 — Frontend build + lint green
|
||||
|
||||
`npm run build` and `npm run lint` from `frontend/` MUST pass (0 errors). New widget tests cover loading/error/rendered states; PrometheusChartWidget tests stay green (SS-118).
|
||||
@@ -0,0 +1,166 @@
|
||||
# Sync Report — `service-storage-harness`
|
||||
|
||||
> Phase: **sync** · Change: `service-storage-harness` · Repo: `/home/user/manage`
|
||||
> Mode: file-backed (`artifactStore: openspec`). No source-code edits; only OpenSpec artifacts were
|
||||
> written. Not committed (parent owns the commit). The change folder was **not** moved (that is
|
||||
> `sdd-archive`'s job).
|
||||
|
||||
**Status: SYNCED.** A new canonical domain `openspec/specs/service-storage/spec.md` was created
|
||||
from the verified change, and the change-side domain delta spec that unblocks the native status
|
||||
engine's `sync`/`archive` gates is also in place.
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
The `service-storage-harness` change shipped a **complete but flat** `openspec/changes/service-storage-harness/spec.md`
|
||||
(28 requirements, SS-101 … SS-128) with **no** per-domain delta spec under
|
||||
`openspec/changes/service-storage-harness/specs/<domain>/`. `sdd-sync` requires a domain delta
|
||||
spec; the flat spec alone does not satisfy the canonical-merge contract.
|
||||
|
||||
Verify already returned **PASS** (verdict in `verify-report.md`; all four gates green — backend
|
||||
`pytest` 322 passed, `ruff` clean, frontend `npm run build` exit 0, `npm run lint` 0 errors).
|
||||
Functional coverage was **28/28 fully PASS**. The verify report's single CRITICAL was an **archive**
|
||||
blocker (30 unchecked task checkboxes + missing `apply-progress.md`); `apply-progress.md` now exists
|
||||
and reconciles the 35 tasks (the task tracker condition does **not** block `sdd-sync` of the green
|
||||
code).
|
||||
|
||||
This sync **reconciles** the flat-spec-vs-domain-spec gap:
|
||||
|
||||
1. Authored the missing **change-side domain delta spec** —
|
||||
`openspec/changes/service-storage-harness/specs/service-storage/spec.md` — using a clean
|
||||
`## ADDED Requirements` structure that preserves the exact requirement IDs (SS-101 … SS-128) and
|
||||
text from the verified flat `spec.md`. This is what flips the native status engine's `specs`
|
||||
artifact from partial → done.
|
||||
2. **Synced** the end-state into the **canonical store** —
|
||||
`openspec/specs/service-storage/spec.md` — the actual sync target. Because the canonical
|
||||
`service-storage` domain did not previously exist, the native helper rule applies: *when the
|
||||
canonical spec does not exist, the change spec becomes the new canonical spec.* The two files
|
||||
therefore carry identical requirement bodies (delta under `## ADDED Requirements`; canonical
|
||||
under `## Requirements`).
|
||||
|
||||
Domain name **`service-storage`** was chosen (per the dispatch brief) because it covers the full
|
||||
new model: the lifecycle-only `ServiceDataHarness` layer, the qBittorrent store + client + widget
|
||||
stack, and the MediaIndex migration that established the per-service storage pattern. It is distinct
|
||||
from the existing canonical domains `web-ui` (MUI→shadcn migration) and `prometheus-charting`
|
||||
(direct Prometheus metric visualization), neither of which was **touched**.
|
||||
|
||||
## 2. Structured status & actionContext findings
|
||||
|
||||
The native `gentle-pi.sdd-status` passed by the parent reports `changeName: null` with
|
||||
`blockedReasons: ["Change selection is ambiguous: mobile-responsive-parity, service-storage-harness,
|
||||
services-as-hub-ia."]` because the engine auto-detected three active changes. This sync task was
|
||||
**explicitly assigned** `service-storage-harness`; the ambiguity is a parent-resolution artifact
|
||||
and does not block this phase (`isNonAuthoritative: false`).
|
||||
|
||||
- `artifactStore: openspec`; change root `openspec/changes/service-storage-harness/`.
|
||||
- Artifacts present: `proposal.md`, `spec.md`, `design.md`, `tasks.md`, `verify-report.md`,
|
||||
`apply-progress.md`.
|
||||
- `verify: PASS` (verify-report verdict; gates green at `c9404f0`).
|
||||
- `actionContext`: `mode: repo-local`, `workspaceRoot: /home/user/manage`,
|
||||
`allowedEditRoots: ["/home/user/manage"]`, `warnings: []`. All three files written are inside the
|
||||
authoritative workspace / allowed edit roots. ✓
|
||||
- `relationships.sameDomainActiveChanges: []`, `collisions: []` — **no active same-domain
|
||||
collisions**, so no archive/sync ordering decision was required.
|
||||
- The new `service-storage` domain is distinct from the existing `web-ui` and `prometheus-charting`
|
||||
canonical domains; both were left untouched.
|
||||
|
||||
**Post-sync structural change:** `openspec/changes/service-storage-harness/specs/service-storage/spec.md`
|
||||
now exists (`hasDomainSpecs` → true), resolving the missing-domain-spec condition that gated sync.
|
||||
The flat `spec.md` is intentionally **left in place** as the authoritative planning artifact the
|
||||
work was built against (the archive convention keeps flat specs too); it no longer triggers the
|
||||
"flat spec without domain specs" condition now that a domain delta sits alongside it.
|
||||
|
||||
## 3. Domains synced & canonical files updated
|
||||
|
||||
| Domain | Change-side delta (source) | Canonical (sync target) | Action |
|
||||
|---|---|---|---|
|
||||
| `service-storage` | `openspec/changes/service-storage-harness/specs/service-storage/spec.md` | `openspec/specs/service-storage/spec.md` | **NEW domain** — `## ADDED Requirements` copied into canonical as a new spec |
|
||||
|
||||
- **Canonical file created:** `openspec/specs/service-storage/spec.md` (28 requirements).
|
||||
- **Change-side delta created:** `openspec/changes/service-storage-harness/specs/service-storage/spec.md`
|
||||
(28 requirements, all `## ADDED Requirements`).
|
||||
|
||||
## 4. Requirement delta (ADDED / MODIFIED / REMOVED)
|
||||
|
||||
- **ADDED (28)** — all to the new `service-storage` domain (canonical did not exist pre-change).
|
||||
IDs and text preserved verbatim from the verified flat `spec.md`. Grouped logically:
|
||||
- *ServiceDataHarness (lifecycle layer)* — SS-101, SS-102, SS-103, SS-104
|
||||
- *QbittorrentSampleStore* — SS-105, SS-106, SS-107
|
||||
- *QbittorrentClient* — SS-108, SS-109, SS-110
|
||||
- *qBittorrent widget source adapter* — SS-111, SS-112, SS-113, SS-114, SS-115
|
||||
- *qBittorrent frontend widgets* — SS-116, SS-117, SS-118
|
||||
- *MediaIndex migration onto harness* — SS-119, SS-120, SS-121, SS-122, SS-123, SS-124
|
||||
- *Cascade-delete wiring* — SS-125, SS-126
|
||||
- *Test and build greenness* — SS-127, SS-128
|
||||
- **MODIFIED (0)** — none (new domain; no pre-existing canonical requirements to replace).
|
||||
- **REMOVED (0)** — none.
|
||||
- **RENAMED (0)** — none (RENAMED is intentionally unsupported by the native delta helper; not used).
|
||||
|
||||
## 5. Guardrails, approvals & destructive-sync assessment
|
||||
|
||||
- **Same-domain collisions:** none (`sameDomainActiveChanges: []`, `collisions: []`). The new
|
||||
`service-storage` domain does not overlap the existing `web-ui` or `prometheus-charting`
|
||||
canonical domains. No ordering decision was needed.
|
||||
- **Destructive sync:** **not applicable.** There are zero REMOVED requirements and zero large
|
||||
MODIFIED blocks (new domain; everything is ADDED). No destructive-sync parent approval was
|
||||
required beyond the explicit reconciliation instruction in the task.
|
||||
- **Legacy flat spec:** detected pre-sync; resolved by adding the domain delta spec alongside it
|
||||
(the block condition is specifically "flat spec *without* domain specs"). The flat spec was left
|
||||
in place as a planning artifact.
|
||||
- **`web-ui` / `prometheus-charting` canonical isolation:** the existing
|
||||
`openspec/specs/web-ui/spec.md` (MUI→shadcn rework) and `openspec/specs/prometheus-charting/spec.md`
|
||||
(direct Prometheus charting) were **not modified** — verified untouched by `git status`. The three
|
||||
domains are independent.
|
||||
|
||||
## 6. Validation / checks performed (file-backed, read-only)
|
||||
|
||||
Run from `/home/user/manage` (no source edits, no test re-runs — those are owned by verify and were
|
||||
already green at `c9404f0`):
|
||||
|
||||
| Check | Command | Result |
|
||||
|---|---|---|
|
||||
| Canonical store populated | `ls openspec/specs/service-storage/spec.md` | present ✓ |
|
||||
| Change-side domain spec present | `ls openspec/changes/service-storage-harness/specs/service-storage/spec.md` | present ✓ |
|
||||
| Requirement-ID parity (flat ↔ delta ↔ canonical) | `grep -oE 'SS-[0-9]+'` all three files, `sort -u` | **28 == 28 == 28**, identical IDs SS-101…SS-128 ✓ |
|
||||
| Body-text parity (delta ↔ canonical) | `diff` of the `^### Requirement:` region of both files | **identical** ✓ |
|
||||
| Delta is pure ADDED | count `## ADDED/MODIFIED/REMOVED/RENAMED Requirements` | ADDED=1, MODIFIED=0, REMOVED=0, RENAMED=0 ✓ (no destructive sync) |
|
||||
| Other canonicals untouched | `git status --porcelain openspec/specs/web-ui openspec/specs/prometheus-charting` | empty (not modified) ✓ |
|
||||
| No edits outside openspec | `git status --porcelain` (filtered) | only `openspec/specs/service-storage/`, `openspec/changes/service-storage-harness/specs/`, and this report added ✓ |
|
||||
| Markdown validity | write-time lint | all three files "Markdown clean" ✓ |
|
||||
|
||||
## 7. Carry-over items for the archive summary
|
||||
|
||||
These verify-phase findings are non-blocking for sync and should land in the archive summary:
|
||||
|
||||
1. **[CRITICAL-process, archive-only] Unchecked task checkboxes.** At verify time, 30
|
||||
implementation/verification task checkboxes (§1.1–5.5) were unchecked and `apply-progress.md`
|
||||
was missing. `apply-progress.md` now exists (created after the verify pass, reconciling all 35
|
||||
tasks). `sdd-archive` should re-scan the native status engine to confirm `tasks: done` /
|
||||
`applyProgress: present` before moving the change to archive, and tick any remaining unchecked
|
||||
boxes if needed.
|
||||
2. **[WARNING] Slice 2 over the 400-line review budget** (~644 source insertions vs the 400-line
|
||||
budget / ~350–400 forecast). Additive feature slice (3 widgets + `LineSeriesChart` extraction +
|
||||
tests); boundary is exactly the qBit-widget feature, no scope creep. No `size:exception`
|
||||
recorded; non-blocking — record the actual in the archive summary.
|
||||
3. **[INFO] `LineSeriesChart.test.tsx` is smoke-only** (asserts mount, not rendered `<Line>` series).
|
||||
Non-blocking coverage note.
|
||||
4. **[INFO] Stale generated `.pi-map.md`** files predate the new modules (`service_data.py`,
|
||||
`qbittorrent_store.py`, `clients/qbittorrent.py`, `Qbittorrent*.tsx`, `LineSeriesChart.tsx`);
|
||||
not deliverable source. Regenerate via `project_map_patch` / `project_map_validate`.
|
||||
|
||||
## 8. Next recommended phase
|
||||
|
||||
→ **`sdd-archive`** (clean). Confirm the native status re-scan reports `specs: done` / `sync: ready`
|
||||
/ `archive: ready`, then move the change to
|
||||
`openspec/changes/archive/2026-07-09-service-storage-harness`, carrying over the items in §7 into
|
||||
the archive summary. Do **not** commit or push — the parent owns the commit with explicit paths.
|
||||
|
||||
---
|
||||
|
||||
### Appendix — Files written by this sync (OpenSpec only; no source code)
|
||||
|
||||
- `openspec/changes/service-storage-harness/specs/service-storage/spec.md` — **change-side
|
||||
domain delta (`## ADDED Requirements`), 28 requirements SS-101…SS-128.**
|
||||
- `openspec/specs/service-storage/spec.md` — **canonical spec (new domain), 28 requirements.**
|
||||
- `openspec/changes/service-storage-harness/sync-report.md` — this report.
|
||||
@@ -0,0 +1,293 @@
|
||||
# SDD Tasks: Service Storage Harness (qBittorrent widgets + MediaIndex migration)
|
||||
|
||||
**Change:** `service-storage-harness`
|
||||
**Phase:** tasks
|
||||
**Date:** 2026-07-09
|
||||
|
||||
## Review Workload Forecast
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Estimated changed lines | ~1,180–1,380 (sum of four implementation slices) |
|
||||
| 400-line budget risk | High |
|
||||
| Chained PRs recommended | Yes |
|
||||
| Suggested split | PR 1: Harness + QbittorrentSampleStore + client + integration → PR 2: Widget adapter + LineSeriesChart extract + 3 FE widgets → PR 3: MediaIndex migration → PR 4: Cascade-delete wiring |
|
||||
| Delivery strategy | auto-chain |
|
||||
| Chain strategy | stacked-to-main |
|
||||
|
||||
```text
|
||||
Decision needed before apply: No
|
||||
Chained PRs recommended: Yes
|
||||
Chain strategy: stacked-to-main
|
||||
400-line budget risk: High
|
||||
```
|
||||
|
||||
> **Slice ordering rationale:** Slices 1–2 prove the harness and deliver all qBittorrent functionality independently of the MediaIndex migration. Slice 3 migrates the load-bearing MediaIndex feature. Slice 4 wires cascade-delete end-to-end. This ordering ensures the two biggest risks (new abstraction × load-bearing refactor) never fire in the same slice — the harness shape is settled in code before MediaIndex touches it.
|
||||
|
||||
---
|
||||
|
||||
## Slice 1: ServiceDataHarness + QbittorrentSampleStore + QbittorrentClient + integration registration
|
||||
|
||||
**Goal:** Build the lifecycle-only storage harness, prove it with the qBittorrent speed-sample store, add the qBittorrent HTTP client and integration registration. No widgets or frontend yet — this slice is backend-only and proves the harness contract with a real consumer.
|
||||
|
||||
- [x] **1.1 Create `ServiceDataHarness` lifecycle module**
|
||||
- Files: `backend/src/media_library_viewer_api/services/service_data.py` (NEW)
|
||||
- Lines: ~100
|
||||
- Dependencies: none
|
||||
- Details: `StorageConcern` dataclass (`concern_key`, `db_filename`, `migrations: list[str]`, `tables: list[str]`, `service_id_column="service_id"`). `ServiceDataHarness` class with `register(concern)`, `db_path(concern_key)`, `connect(concern_key)` (WAL + busy_timeout 30s), `run_migrations()` (per-concern, per-statement try/except for "duplicate column name"), `cascade_delete(service_id)` (iterate concerns, PRAGMA-check column exists, DELETE WHERE col = ?). Module singleton `get_service_data_harness()` that lazy-inits with `base_dir = BACKEND_CACHE_DIR or ".cache/media_library_viewer"` and calls `run_migrations()`. NO generic value table, NO generic CRUD.
|
||||
|
||||
- [x] **1.2 Create `QbittorrentSampleStore` with concern registration**
|
||||
- Files: `backend/src/media_library_viewer_api/services/qbittorrent_store.py` (NEW)
|
||||
- Lines: ~80
|
||||
- Dependencies: 1.1
|
||||
- Details: Define `QBITTORRENT_CONCERN` (`db_filename="qbittorrent.db"`, migration creates `qbittorrent_speed_samples(service_id, ts, dl_speed, up_speed)` + index on `(service_id, ts)`, `tables=["qbittorrent_speed_samples"]`). `MAX_SAMPLES = 120`. `QbittorrentSampleStore` class: `append(service_id, ts, dl_speed, up_speed)` (INSERT + prune keeping most recent MAX_SAMPLES for this service_id), `window(service_id, since_ts=None)` (SELECT ordered ASC). Register `QBITTORRENT_CONCERN` with harness on import.
|
||||
|
||||
- [x] **1.3 Create `QbittorrentClient` HTTP client**
|
||||
- Files: `backend/src/media_library_viewer_api/clients/qbittorrent.py` (NEW)
|
||||
- Lines: ~90
|
||||
- Dependencies: none
|
||||
- Details: `QbittorrentClient(base_url, username, password, timeout=10)` modeled on `JellyfinClient`'s session pattern. `_login()` POSTs to `/auth/login` with `Referer` header, expects `"Ok."` response, stores SID cookie. `_get(path, **params)` auto-logins on first call, re-logins on 403. `maindata()` calls `/sync/maindata` returning `{server_state: {...}, torrents: {hash: {...}}}`. Base URL normalization: append `/api/v2` if not present. Timeout + `requests.RequestException` handling consistent with existing clients.
|
||||
|
||||
- [x] **1.4 Create qBittorrent integration definition**
|
||||
- Files: `backend/src/media_library_viewer_api/integrations/qbittorrent.py` (NEW)
|
||||
- Lines: ~55
|
||||
- Dependencies: 1.1 (for `ServiceConfigBase`, `WidgetConfigBase`, `SecretField`, `ServiceDefinition`, `widget_kind` from `base.py`)
|
||||
- Details: `QbittorrentConfig(ServiceConfigBase)` with `base_url: ServiceBaseUrl`, `timeout_seconds: int = 10`. `QbittorrentWidgetConfig(WidgetConfigBase)` empty (all three widget kinds derive from service connection). `DEFINITION = ServiceDefinition(service_type="qbittorrent", ...)` with `secret_fields=[username (required), password (required)]` and three `widget_kinds`: `totals` (30s refresh), `active` (15s), `speed` (5s). Model on `integrations/prometheus.py`.
|
||||
|
||||
- [x] **1.5 Register qBittorrent in backend integration registry**
|
||||
- Files: `backend/src/media_library_viewer_api/integrations/registry.py` (MODIFY)
|
||||
- Lines: ~5
|
||||
- Dependencies: 1.4
|
||||
- Details: Import `DEFINITION as QBITTORRENT` from `integrations/qbittorrent.py`; add `SERVICE_DEFINITIONS["qbittorrent"] = QBITTORRENT`. Verify `list_service_types()` now includes `qbittorrent`.
|
||||
|
||||
- [x] **1.6 Initialize harness in application lifespan**
|
||||
- Files: `backend/src/media_library_viewer_api/main.py` (MODIFY, lifespan function ~line 41)
|
||||
- Lines: ~5
|
||||
- Dependencies: 1.1, 1.2
|
||||
- Details: In `lifespan()` after `get_settings_store().ensure_defaults()`, add a try/except block calling `get_service_data_harness()` (triggers lazy init + migration run). Log on failure (same pattern as the settings seed block at lines 51–54). Import is local inside lifespan to avoid circular import risk.
|
||||
|
||||
- [x] **1.7 Create backend tests for harness + store**
|
||||
- Files: `backend/tests/test_service_data.py` (NEW)
|
||||
- Lines: ~110
|
||||
- Dependencies: 1.1, 1.2
|
||||
- Details: Test `ServiceDataHarness`: register concern → `run_migrations()` creates table; `cascade_delete(service_id)` removes rows by service_id but leaves other services' rows; migration idempotency (re-run doesn't crash). Test `QbittorrentSampleStore`: `append` + prune (append 130 samples, assert only 120 remain), `window` returns ordered samples, two services don't cross-contaminate. Use `tmp_path` for base_dir isolation.
|
||||
|
||||
- [x] **1.8 Create backend tests for qBittorrent client**
|
||||
- Files: `backend/tests/test_qbittorrent_client.py` (NEW)
|
||||
- Lines: ~85
|
||||
- Dependencies: 1.3
|
||||
- Details: Test login flow (mock `requests.Session.post` → `"Ok."` + cookie), cookie reuse on subsequent calls, 403 → re-login flow (first GET returns 403, second returns 200 after re-login), `maindata()` parsing, base URL normalization (appends `/api/v2`), login failure (response ≠ "Ok." raises RuntimeError), timeout handling. Use `unittest.mock.patch` on the session.
|
||||
|
||||
- [x] **1.9 Verify Slice 1 (backend only)**
|
||||
- Run: `cd backend && PYTHONPATH=src ruff check src tests && PYTHONPATH=src python -m pytest tests/test_service_data.py tests/test_qbittorrent_client.py -v`
|
||||
- Verify: all new tests pass; existing tests unaffected.
|
||||
|
||||
**Slice 1 total:** ~310–390 changed lines. Backend-only; no frontend changes.
|
||||
|
||||
---
|
||||
|
||||
## Slice 2: Widget source adapter + LineSeriesChart extraction + frontend widgets + registry binding
|
||||
|
||||
**Goal:** Wire qBittorrent data into the widget system, extract a shared chart renderer, and build the three frontend widgets. This slice touches `PrometheusChartWidget` (extraction) — its existing tests MUST stay green.
|
||||
|
||||
- [x] **2.1 Add `QbittorrentWidgetSource` adapter to `widgets/sources.py`**
|
||||
- Files: `backend/src/media_library_viewer_api/widgets/sources.py` (MODIFY)
|
||||
- Lines: ~75
|
||||
- Dependencies: Slice 1 (1.3, 1.2)
|
||||
- Details: Add `QbittorrentWidgetSource` implementing `WidgetSource.fetch(service, widget_kind, config)`. Resolves client inline from `ServiceRecord` config+secrets (like `PrometheusWidgetSource`). Three branches:
|
||||
- `totals`: `len(torrents)` + `by_state` dict breakdown. Returns `{total, by_state}`.
|
||||
- `active`: filter `state in {"downloading","uploading"}`. Returns `{torrents: [{name, state, size, progress, dl_speed, up_speed}]}`.
|
||||
- `speed`: append sample to `QbittorrentSampleStore(service.id, ts, dl, up)`, read `window(service.id)`, return `{series: [{label:"download", points:[{t: ts*1000, v: dl}]}, {label:"upload", ...}]}` (timestamps ×1000 for JS epoch).
|
||||
All branches wrap in try/except → `{"error": ...}`. Add `"qbittorrent": QbittorrentWidgetSource()` to `SERVICE_ADAPTERS`.
|
||||
|
||||
- [x] **2.2 Add backend tests for qBittorrent widget adapter**
|
||||
- Files: `backend/tests/test_widgets.py` (EXTEND)
|
||||
- Lines: ~90
|
||||
- Dependencies: 2.1
|
||||
- Details: Test `QbittorrentWidgetSource` with mocked `QbittorrentClient.maindata()`: totals counts all torrents + breaks down by state; active filters to DL/UL only (assert queued/stalled excluded); speed appends sample and returns `{series}` shape with two labeled series; missing service → `{"error": ...}`; timeout → `{"error": ...}`. Use `tmp_path` harness override for the store.
|
||||
|
||||
- [x] **2.3 Extract shared `LineSeriesChart` component from `PrometheusChartWidget`**
|
||||
- Files: `frontend/src/components/LineSeriesChart.tsx` (NEW), `frontend/src/widgets/PrometheusChartWidget.tsx` (MODIFY)
|
||||
- Lines: ~75 (new file) + ~20 (PrometheusChartWidget thin wrapper)
|
||||
- Dependencies: none (refactor of existing code)
|
||||
- Details: **RISK ITEM (a).** Extract `mergeSeries`, `formatTime`, `CHART_COLORS`, `ChartSeries`/`SeriesPoint` types, and the `<ResponsiveContainer>+<LineChart>+<XAxis>+<YAxis>+<CartesianGrid>+<Tooltip>+<Line>` JSX from `PrometheusChartWidget.tsx` into `components/LineSeriesChart.tsx`. Props: `{ series: ChartSeries[], height?: number }`. `PrometheusChartWidget` becomes: `useWidgetData` → `<LineSeriesChart series={data.data.series} />`. Export `ChartSeries` type from `LineSeriesChart` for reuse. **PrometheusChartWidget's existing tests MUST stay green — the extraction is mechanical (move + import), not a rewrite.**
|
||||
|
||||
- [x] **2.4 Create `QbittorrentTotalsWidget` component**
|
||||
- Files: `frontend/src/widgets/QbittorrentTotalsWidget.tsx` (NEW)
|
||||
- Lines: ~60
|
||||
- Dependencies: 2.3
|
||||
- Details: Renders `{total, by_state}` from `useWidgetData`. Uses `SectionCard` + prominent total count + state breakdown badges (e.g. `downloading: 3`, `uploading: 1`, `paused: 5`). Loading skeleton + error alert + empty state. Model on `BackupsWidget`/`MetricCard` patterns.
|
||||
|
||||
- [x] **2.5 Create `QbittorrentActiveTorrentsWidget` component**
|
||||
- Files: `frontend/src/widgets/QbittorrentActiveTorrentsWidget.tsx` (NEW)
|
||||
- Lines: ~80
|
||||
- Dependencies: 2.3
|
||||
- Details: Renders `{torrents: [...]}` from `useWidgetData`. Uses `SectionCard` + compact table/list (manual `Table` rows, NOT DataTable — keep it simple for ≤10 active torrents). Columns: name, state badge, progress bar, dl/up speed. Scrollable container for overflow. Loading + error + empty ("No active torrents") states. Reuses `humanSize`-style formatting for speeds (bytes/s → MB/s).
|
||||
|
||||
- [x] **2.6 Create `QbittorrentSpeedWidget` component**
|
||||
- Files: `frontend/src/widgets/QbittorrentSpeedWidget.tsx` (NEW)
|
||||
- Lines: ~40
|
||||
- Dependencies: 2.3
|
||||
- Details: Renders `{series}` from `useWidgetData` — IDENTICAL pattern to `PrometheusChartWidget`. Thin wrapper: `useWidgetData` → `<LineSeriesChart series={data.data.series} height={220} />`. Loading skeleton + error alert + empty state ("No speed data yet"). Refresh interval from registry binding (5s).
|
||||
|
||||
- [x] **2.7 Add qBittorrent binding to frontend registry**
|
||||
- Files: `frontend/src/integrations/registry.ts` (MODIFY)
|
||||
- Lines: ~40
|
||||
- Dependencies: 2.4, 2.5, 2.6
|
||||
- Details: Import the three new widget components. Add `qbittorrent` entry to `SERVICE_REGISTRY` with three widget kinds: `totals` (30s, `QbittorrentTotalsWidget`), `active` (15s, `QbittorrentActiveTorrentsWidget`), `speed` (5s, `QbittorrentSpeedWidget`). All with empty config schemas (`{type:"object", properties:{}, required:[]}`). Update barrel `widgets/index.ts` with the three new exports.
|
||||
|
||||
- [x] **2.8 Create frontend tests for new widgets + LineSeriesChart**
|
||||
- Files: `frontend/src/widgets/__tests__/QbittorrentTotalsWidget.test.tsx` (NEW), `frontend/src/widgets/__tests__/QbittorrentActiveTorrentsWidget.test.tsx` (NEW), `frontend/src/widgets/__tests__/QbittorrentSpeedWidget.test.tsx` (NEW), `frontend/src/components/__tests__/LineSeriesChart.test.tsx` (NEW)
|
||||
- Lines: ~110 (4 files × ~25-30 lines)
|
||||
- Dependencies: 2.3, 2.4, 2.5, 2.6
|
||||
- Details: Each widget test: loading skeleton (`data-slot="skeleton"`), error alert (destructive variant), rendered data case. `LineSeriesChart.test.tsx`: renders lines from series data, empty state (no series). Model on existing `PrometheusChartWidget.test.tsx` mock pattern (`vi.mocked(useWidgetData).mockReturnValue(...)`). **Verify `PrometheusChartWidget.test.tsx` still passes after extraction.**
|
||||
|
||||
- [x] **2.9 Verify Slice 2 (full stack)**
|
||||
- Run: `cd backend && PYTHONPATH=src ruff check src tests && PYTHONPATH=src python -m pytest tests/test_widgets.py -v`
|
||||
- Run: `cd frontend && npm run lint && npm run build && npx vitest run src/widgets/__tests__/Qbittorrent src/widgets/__tests__/PrometheusChart src/components/__tests__/LineSeriesChart`
|
||||
- Verify: PrometheusChartWidget tests GREEN (extraction not a regression); qBit widget tests GREEN; build + lint GREEN.
|
||||
|
||||
**Slice 2 total:** ~350–400 changed lines.
|
||||
|
||||
---
|
||||
|
||||
## Slice 3: MediaIndex migration onto harness (load-bearing refactor)
|
||||
|
||||
**Goal:** Register MediaIndex as a harness concern, add `service_id` scoping to the schema, fix the latent global-clear bug in `replace_items`, and thread `service_id` through query/build/worker. All existing MediaIndex tests MUST pass unchanged.
|
||||
|
||||
- [x] **3.1 Define `MEDIA_INDEX_CONCERN` and register with harness**
|
||||
- Files: `backend/src/media_library_viewer_api/services/media_index_impl.py` (MODIFY — add concern constant near top), `backend/src/media_library_viewer_api/services/service_data.py` (MODIFY — register concern in `get_service_data_harness`)
|
||||
- Lines: ~25
|
||||
- Dependencies: Slice 1 (1.1)
|
||||
- Details: `MEDIA_INDEX_CONCERN = StorageConcern(concern_key="media_index", db_filename="media_index.sqlite", migrations=["ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT ''"], tables=["media_items"])`. Register in harness singleton alongside `QBITTORRENT_CONCERN`. The `db_filename` MUST match the existing `DEFAULT_INDEX_PATH` leaf name (`media_index.sqlite`). The harness `base_dir` (`.cache/media_library_viewer`) is the same parent — no file move.
|
||||
|
||||
- [x] **3.2 Add `service_id` column to `init_schema` CREATE TABLE**
|
||||
- Files: `backend/src/media_library_viewer_api/services/media_index_impl.py` (MODIFY, `init_schema` ~line 107)
|
||||
- Lines: ~3
|
||||
- Dependencies: 3.1
|
||||
- Details: Add `service_id TEXT NOT NULL DEFAULT ''` as the last column in the `CREATE TABLE IF NOT EXISTS media_items` statement. New installs get the column from `init_schema`; existing DBs get it from the harness ALTER migration. Both paths converge on the same schema.
|
||||
|
||||
- [x] **3.3 Scope `replace_items` by `service_id` (FIXES latent global-clear bug)**
|
||||
- Files: `backend/src/media_library_viewer_api/services/media_index_impl.py` (MODIFY, `replace_items` ~line 166)
|
||||
- Lines: ~20
|
||||
- Dependencies: 3.2
|
||||
- Details: **RISK ITEM (b).** Add `service_id: str = ""` parameter. Add `"service_id"` to the columns list. Change `DELETE FROM media_items` → `DELETE FROM media_items WHERE service_id = ?` with `(service_id,)`. Change the `executemany` rows to prepend `service_id` to each row's values. This FIXES the latent bug where building for one Jellyfin wipes another's rows. Existing tests call `replace_items(rows)` with no `service_id` → defaults to `""` → deletes `WHERE service_id = ''` → backward-compatible (existing test data has `service_id=''` from backfill).
|
||||
|
||||
- [x] **3.4 Scope `query` by `service_id`**
|
||||
- Files: `backend/src/media_library_viewer_api/services/media_index_impl.py` (MODIFY, `query` ~line 279)
|
||||
- Lines: ~10
|
||||
- Dependencies: 3.2
|
||||
- Details: Add `service_id: str = ""` parameter. After existing WHERE clause building, add: `if service_id: where.append("service_id = ?"); params.append(service_id)`. When `service_id=""` (empty), skip the filter → shows all rows (backward-compatible). This means existing tests (which pass no `service_id`) see all rows unchanged.
|
||||
|
||||
- [x] **3.5 Thread `service_id` through `build_media_index` → `replace_items`**
|
||||
- Files: `backend/src/media_library_viewer_api/services/media_index_impl.py` (MODIFY, `build_media_index` ~line 331)
|
||||
- Lines: ~5
|
||||
- Dependencies: 3.3
|
||||
- Details: Add `service_id: str = ""` parameter to `build_media_index(...)`. Change the `count = index.replace_items(normalized_rows)` call (~line 458) to `count = index.replace_items(normalized_rows, service_id=service_id)`.
|
||||
|
||||
- [x] **3.6 Pass `service_id` from worker to `build_media_index`**
|
||||
- Files: `backend/src/media_library_viewer_api/workers/media_index_worker.py` (MODIFY, `run_build` ~line 144, `build_media_index` call ~line 168)
|
||||
- Lines: ~3
|
||||
- Dependencies: 3.5
|
||||
- Details: The worker's `run_build(final_index_path, staging_index_path, service_id="")` ALREADY receives `service_id` from argparse (`--service-id`). Change the `build_media_index(...)` call (~line 168) to pass `service_id=service_id`.
|
||||
|
||||
- [x] **3.7 Add `jellyfin_service_id` to `query_media` and pass to `query`**
|
||||
- Files: `backend/src/media_library_viewer_api/routers/media.py` (MODIFY, `query_media` ~line 276)
|
||||
- Lines: ~5
|
||||
- Dependencies: 3.4
|
||||
- Details: `query_media` does NOT currently have a `jellyfin_service_id` param (unlike `post_build_index` which does). Add `jellyfin_service_id: str | None = None` to the signature (after `offset`). Pass `service_id=jellyfin_service_id or ""` to the `index.query(...)` call (~line 310). Existing callers that don't send the param → `None` → `""` → all rows (backward-compatible).
|
||||
|
||||
- [x] **3.8 Add backend tests for scoped MediaIndex operations**
|
||||
- Files: `backend/tests/test_media_index.py` (EXTEND)
|
||||
- Lines: ~45
|
||||
- Dependencies: 3.3, 3.4
|
||||
- Details: Add tests for the NEW scoped behavior: `replace_items(rows, service_id="svc-a")` then `replace_items(other_rows, service_id="svc-b")` — assert svc-a rows survive svc-b's replace (proves the bug fix). `query(service_id="svc-a")` returns only svc-a rows. `query(service_id="")` returns all rows. `query()` (no arg) returns all rows. **Existing MediaIndex tests MUST pass unchanged** — verify they don't break by running the full file.
|
||||
|
||||
- [x] **3.9 Verify Slice 3 (backend, load-bearing)**
|
||||
- Run: `cd backend && PYTHONPATH=src ruff check src tests && PYTHONPATH=src python -m pytest tests/test_media_index.py -v`
|
||||
- Run: `cd backend && PYTHONPATH=src python -m pytest -q` (full suite — no regressions)
|
||||
- Verify: ALL existing MediaIndex tests pass unchanged; new scoped tests pass; full suite green.
|
||||
|
||||
**Slice 3 total:** ~200–260 changed lines. Backend-only; Media page UX unchanged.
|
||||
|
||||
---
|
||||
|
||||
## Slice 4: Cascade-delete wiring end-to-end + integration test
|
||||
|
||||
**Goal:** Hook `ServiceDataHarness.cascade_delete` into `SettingsStore.delete_service` so removing a service wipes its owned data across all harness-managed tables. End-to-end integration test.
|
||||
|
||||
- [x] **4.1 Wire harness cascade-delete into `delete_service`**
|
||||
- Files: `backend/src/media_library_viewer_api/services/settings_store.py` (MODIFY, `delete_service` ~line 1742)
|
||||
- Lines: ~10
|
||||
- Dependencies: Slice 1 (1.1), Slice 3 (3.1)
|
||||
- Details: After the existing `DELETE FROM services WHERE id = ?` (line 1754), add a try/except block: `from media_library_viewer_api.services.service_data import get_service_data_harness; get_service_data_harness().cascade_delete(service_id)`. Log on failure (`logger.exception(...)`) — cascade is best-effort (service row is already deleted; data cleanup must not block deletion). Local import to avoid circular dependency.
|
||||
|
||||
- [x] **4.2 Add integration test for cascade-delete**
|
||||
- Files: `backend/tests/test_services.py` (EXTEND)
|
||||
- Lines: ~45
|
||||
- Dependencies: 4.1
|
||||
- Details: Test end-to-end cascade: (1) create a qBittorrent service, append speed samples for it, delete the service → assert samples gone from `qbittorrent.db`. (2) Create two Jellyfin services, build media for both with distinct `service_id`s, delete one → assert only that service's media rows removed, other survives. Use `tmp_path` for both settings DB and harness base_dir. Verify `delete_service` doesn't raise even if harness fails (best-effort guard).
|
||||
|
||||
- [x] **4.3 Verify Slice 4 (full suite)**
|
||||
- Run: `cd backend && PYTHONPATH=src ruff check src tests && PYTHONPATH=src python -m pytest -q`
|
||||
- Run: `cd frontend && npm run lint && npm run build`
|
||||
- Verify: full backend suite green (293+ tests + new); frontend unaffected (no FE changes in this slice).
|
||||
|
||||
**Slice 4 total:** ~80–110 changed lines. Backend-only.
|
||||
|
||||
---
|
||||
|
||||
## Integration and acceptance verification
|
||||
|
||||
- [x] **5.1 Full backend test run**
|
||||
- Run: `cd backend && PYTHONPATH=src python -m pytest -q`
|
||||
- Verify: all existing tests pass + all new tests (harness, store, client, widget adapter, scoped MediaIndex, cascade-delete) pass.
|
||||
|
||||
- [x] **5.2 Full frontend build + lint**
|
||||
- Run: `cd frontend && npm run lint && npm run build`
|
||||
- Verify: no TypeScript errors; no new lint failures; PrometheusChartWidget tests still green after LineSeriesChart extraction.
|
||||
|
||||
- [x] **5.3 Verify qBittorrent end-to-end (manual or integration)**
|
||||
- Verify: a qBittorrent service can be registered; three widget kinds are bindable; totals shows item count; active shows DL/UL torrents; speed shows a live recharts line chart.
|
||||
|
||||
- [x] **5.4 Verify MediaIndex backward-compat**
|
||||
- Verify: existing Media page works identically (no `service_id` → sees all rows); building for a specific Jellyfin service scopes correctly.
|
||||
|
||||
- [x] **5.5 Verify cascade-delete**
|
||||
- Verify: deleting a service removes its qBit samples + media items; deleting one of multiple Jellyfin services removes only its rows.
|
||||
|
||||
---
|
||||
|
||||
## Total estimate
|
||||
|
||||
| Slice | Changed lines | Risk |
|
||||
|-------|---------------|------|
|
||||
| Slice 1: Harness + store + client + integration | ~310–390 | Low (new code, no existing behavior touched) |
|
||||
| Slice 2: Widget adapter + LineSeriesChart + FE widgets | ~350–400 | Medium (touches PrometheusChartWidget) |
|
||||
| Slice 3: MediaIndex migration | ~200–260 | **High** (load-bearing; must fix replace_items bug + keep tests green) |
|
||||
| Slice 4: Cascade-delete wiring | ~80–110 | Low (small, well-isolated) |
|
||||
| Integration tests | ~45 | Low |
|
||||
| **Total** | **~985–1,205** | |
|
||||
|
||||
---
|
||||
|
||||
## Guard lines
|
||||
|
||||
```text
|
||||
Decision needed before apply: No
|
||||
Chained PRs recommended: Yes
|
||||
Chain strategy: stacked-to-main
|
||||
400-line budget risk: High
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risk callouts (for reviewer + apply agents)
|
||||
|
||||
| # | Risk | Slice | Mitigation |
|
||||
|---|------|-------|------------|
|
||||
| **R1** | `LineSeriesChart` extraction breaks `PrometheusChartWidget` tests | S2 (2.3, 2.8) | Extraction is mechanical (move JSX + helpers, pass series as prop). S2 exit gate explicitly verifies `PrometheusChartWidget.test.tsx` still passes. |
|
||||
| **R2** | MediaIndex migration breaks existing tests or introduces a regression | S3 (3.3, 3.8) | `service_id` defaults to `""`; empty-string queries skip the filter → all rows visible. `replace_items` bug fix changes `DELETE FROM media_items` → `DELETE WHERE service_id = ?` with `""` default → existing tests (which use `""`) are unaffected. S3 exit gate runs full backend suite. |
|
||||
| **R3** | ALTER TABLE fails on fresh installs where `init_schema` already created the column | S3 (3.1) | `run_migrations` catches "duplicate column name" per-statement (designed in Slice 1). |
|
||||
| **R4** | Harness lazy-init hides migration failures on startup | S1 (1.6) | `run_migrations` runs on first access in lifespan; failures raise (only "duplicate column name" is swallowed). |
|
||||
| **R5** | Circular import between `settings_store` and `service_data` | S4 (4.1) | Cascade-delete uses local import inside `delete_service` method, not module-level. |
|
||||
@@ -0,0 +1,353 @@
|
||||
# Verify Report — service-storage-harness
|
||||
|
||||
> Phase: **verify** · Change: `service-storage-harness` · Repo: `/home/user/manage`
|
||||
> FRESH-CONTEXT adversarial read-only verification of the change against
|
||||
> `proposal.md`, `spec.md`, `design.md`, and `tasks.md`. **No source edits.**
|
||||
> This verify report is the only file written.
|
||||
|
||||
**Head commit verified:** `c9404f0` (`spec(service-storage-harness): add spec`).
|
||||
Four implementation slices are committed underneath it:
|
||||
|
||||
- `e7bd0af` slice 1 — harness + qbit store + client + integration
|
||||
- `1fb12b8` slice 2 — qbit widgets + LineSeriesChart extract *(brief cited `8b0e7ea`,
|
||||
an earlier amend state; the actual landed commit is `1fb12b8`. Content matches
|
||||
the spec/design/tasks; informational, not a defect.)*
|
||||
- `c87f398` slice 3 — migrate MediaIndex onto harness (scoped replace_items, +service_id)
|
||||
- `75c949a` slice 4 — cascade-delete wiring + integration test
|
||||
|
||||
---
|
||||
|
||||
## 0. Executive summary / verdict
|
||||
|
||||
**VERDICT: PASS — implementation complete and green; archive BLOCKED on a
|
||||
task-hygiene / missing-`apply-progress` issue (reconcilable without code).**
|
||||
|
||||
Every functional requirement **SS-101 … SS-128** was checked against source and
|
||||
**passes**. The `ServiceDataHarness` is genuinely lifecycle-only (provisioning,
|
||||
idempotent migrations, `service_id` cascade-delete) with **no** generic value
|
||||
table or CRUD. The qBittorrent stack (store + cookie-reusing client + 3-branch
|
||||
adapter + 3 frontend widgets) is wired consistently in **both** registries with
|
||||
matching kinds/names/refresh intervals. The **load-bearing** MediaIndex migration
|
||||
(SS-119..SS-124) is correct: `replace_items` is scoped `WHERE service_id = ?`
|
||||
(fixing the latent global-clear bug), with a real regression test proving
|
||||
service A's rows survive service B's rebuild; existing MediaIndex tests pass
|
||||
unchanged via the `service_id=""` default. The `LineSeriesChart` extraction is
|
||||
non-regressive (PrometheusChartWidget tests stay green; it becomes a thin
|
||||
wrapper). Cascade-delete is wired into `delete_service` (best-effort try/except)
|
||||
and initialized in `lifespan`, with an end-to-end integration test covering
|
||||
**both** concerns with multi-instance preservation. All four gates are green:
|
||||
backend `pytest` (**322 passed**), `ruff` (**clean**), frontend
|
||||
`npm run build` (**exit 0**), `npm run lint` (**0 errors**).
|
||||
|
||||
Findings:
|
||||
|
||||
- **[CRITICAL — archive blocker, NOT a code defect]** **All 30
|
||||
implementation/verification task checkboxes** remain unchecked in `tasks.md`
|
||||
(§1.1–1.9, §2.1–2.9, §3.1–3.9, §4.1–4.3, §5.1–5.5), and **`apply-progress.md`
|
||||
does not exist** to reconcile them. The underlying work *is* done and verified
|
||||
complete against source; the blocker is that the task tracker was never updated
|
||||
and no apply-progress artifact was produced. Reconciliation = tick the boxes +
|
||||
write `apply-progress.md` (no code change). See §4.
|
||||
- **[WARNING — review workload]** Slice 2 (`1fb12b8`) lands **~644 non-test
|
||||
source lines** (widget adapter + `LineSeriesChart` extraction + 3 FE widgets),
|
||||
above the 400-line budget and above the slice-2 forecast of "~350–400". The
|
||||
boundary is exactly the qBit-widget feature (no unrelated files, no scope
|
||||
creep), and the forecast itself rated slice 2 "Medium"; but no `size:exception`
|
||||
was recorded. Recommend recording the slice-2 actual in the archive summary.
|
||||
**Non-blocking.** See §6.
|
||||
- **[INFO]** `LineSeriesChart.test.tsx` is smoke-only (asserts `container.firstChild`
|
||||
is not null; does not assert recharts `<Line>` SVGs rendered). Same weak-render
|
||||
pattern noted in the Change A report. The PrometheusChartWidget rendered case
|
||||
is title-only. Non-blocking coverage note.
|
||||
- **[INFO]** `QbittorrentSampleStore.append` prunes via
|
||||
`ts NOT IN (SELECT ts … LIMIT 120)`; in the degenerate case of two samples
|
||||
sharing an identical `ts`, the keep-set could exceed `MAX_SAMPLES`. At a 5 s
|
||||
poll the probability is effectively nil; the per-service cap invariant holds in
|
||||
all realistic operation. Non-blocking.
|
||||
- **[INFO]** Stale generated `.pi-map.md` files predate these new files
|
||||
(`service_data.py`, `qbittorrent_store.py`, `clients/qbittorrent.py`,
|
||||
`Qbittorrent*.tsx`, `LineSeriesChart.tsx`, etc.). Not deliverable source;
|
||||
regenerate via `project_map_patch`/`project_map_validate`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Structured status & actionContext findings
|
||||
|
||||
The native `gentle-pi.sdd-status` reports `changeName: null` /
|
||||
`blockedReasons: ["Change selection is ambiguous: mobile-responsive-parity,
|
||||
service-storage-harness, services-as-hub-ia"]` because the engine auto-detected
|
||||
three active changes. This verify task was **explicitly assigned**
|
||||
`service-storage-harness`; the ambiguity is a parent-resolution artifact and does
|
||||
not block this phase.
|
||||
|
||||
- `artifactStore: openspec`; change root
|
||||
`openspec/changes/service-storage-harness/`.
|
||||
- Artifacts present: `proposal.md`, `design.md`, `spec.md`, `tasks.md`.
|
||||
- **`apply-progress.md`: MISSING** (confirmed: directory contains only the four
|
||||
planning docs). This is the root cause of the §4 archive blocker.
|
||||
- `actionContext`: `mode: repo-local`, `workspaceRoot: /home/user/manage`,
|
||||
`allowedEditRoots: ["/home/user/manage"]`, `warnings: []`. Implementation
|
||||
ownership and all target files are provably inside the authoritative workspace. ✓
|
||||
|
||||
## 2. Gate results (actual output, run at `c9404f0`)
|
||||
|
||||
| Gate | Command | Result | Evidence |
|
||||
|------|---------|--------|----------|
|
||||
| Backend tests | `cd backend && PYTHONPATH=src python3 -m pytest -q` | **PASS** | **322 passed, 2 warnings** in 39.36s. Includes new `test_service_data.py` (9), `test_qbittorrent_client.py` (12), qBit adapter tests (6) in `test_widgets.py`, scoped MediaIndex regression (`test_replace_scoped_by_service_id_preserves_other_services`), and cascade integration (`test_cascade_delete_removes_harness_data_across_concerns`). |
|
||||
| Backend lint | `cd backend && PYTHONPATH=src python3 -m ruff check src tests` | **PASS** | `All checks passed!` |
|
||||
| Frontend build | `cd frontend && npm run build` (`tsc -b` + `vite build`) | **PASS** exit 0 | `✓ built in 1.22s`; 2543 modules transformed. Non-fatal `>500 kB` chunk-size warning (pre-existing, orthogonal). |
|
||||
| Frontend lint | `cd frontend && npm run lint` (`eslint .`) | **PASS** exit 0 | `0 errors, 1 warning`. The warning is `react-hooks/exhaustive-deps` in `WidgetConfigDialog.tsx:370` — **pre-existing, untouched by this change** (no slice modified that file). |
|
||||
| Extraction non-regression | `cd frontend && npx vitest run src/widgets/__tests__/PrometheusChartWidget.test.tsx` | **PASS** | **4 passed** (SS-118: LineSeriesChart extraction did not regress PrometheusChartWidget). |
|
||||
| New FE widget + chart tests | `npx vitest run src/widgets/__tests__/ src/components/__tests__/LineSeriesChart.test.tsx` | **PASS** | **8 files, 31 tests passed** (3 qBit widgets + PrometheusChart/Gauge/Mean + LineSeriesChart). |
|
||||
|
||||
## 3. Spec coverage (SS-101 … SS-128)
|
||||
|
||||
| SS | Requirement | Verdict | Evidence |
|
||||
|----|-------------|---------|----------|
|
||||
| SS-101 | Harness is lifecycle-only | **PASS** | `service_data.py::ServiceDataHarness` exposes only `register`/`db_path`/`connect`/`run_migrations`/`cascade_delete`. No generic value table, no generic CRUD — each integration keeps bespoke stores (`append/window`, `replace_items/query`). Module docstring states the contract explicitly. |
|
||||
| SS-102 | Concern registration dataclass | **PASS** | `StorageConcern` frozen dataclass carries `concern_key`, `db_filename`, `migrations: list[str]`, `tables: list[str]`, `service_id_column="service_id"`. `register()` stores by `concern_key`. |
|
||||
| SS-103 | Idempotent migrations | **PASS** | `run_migrations` splits each migration on `;` and executes per-statement; catches `sqlite3.OperationalError` for "duplicate column name" **and** "no such table", logging at debug and continuing. Test `test_cascade_delete_*` + harness tests re-init without raising. |
|
||||
| SS-104 | Cascade-delete across concerns | **PASS** | `cascade_delete(service_id)` iterates **every** registered concern, PRAGMA-checks the column exists, `DELETE FROM <table> WHERE <col> = ?`. Both `QBITTORRENT_CONCERN` and `MEDIA_INDEX_CONCERN` are registered in `get_service_data_harness()`, so it covers `qbittorrent_speed_samples` + `media_items`. Skips concern DBs that don't exist (no crash). |
|
||||
| SS-105 | Sample store schema | **PASS** | `qbittorrent_store.py` migration creates `qbittorrent_speed_samples(service_id, ts, dl_speed, up_speed)` + index `idx_qbit_samples_service_ts ON (service_id, ts)`, in `qbittorrent.db` (per-concern topology via `db_filename`). |
|
||||
| SS-106 | append/window/prune | **PASS** | `append(service_id, ts, dl, up)` INSERTs then prunes to `MAX_SAMPLES=120` (DELETE `ts NOT IN (top-120 for this service_id)`). `window(service_id, since_ts=None)` SELECTs ordered ASC. Test appends 130 → asserts 120 remain; two services don't cross-contaminate. |
|
||||
| SS-107 | Store registered as concern | **PASS** | `QBITTORRENT_CONCERN` registered in `get_service_data_harness()` alongside the media concern. |
|
||||
| SS-108 | Cookie login | **PASS** | `QbittorrentClient._login()` POSTs `/auth/login` (with `Referer` header), expects `"Ok."`, stores SID cookie in the shared `requests.Session`. Credentials resolved from `ServiceRecord.secrets` (Fernet-encrypted at rest) in the adapter. Test `test_login_posts_credentials`. |
|
||||
| SS-109 | 403 re-login | **PASS** | `_get` retries once after 403: sets `_logged_in=False`, re-`_login()`, re-GETs. Test `test_403_triggers_re_login` asserts `session.get` called twice + `session.post` once. |
|
||||
| SS-110 | maindata fetch | **PASS** | `maindata()` → `_get("/sync/maindata")` returns the dict (`server_state` + `torrents`). Timeout/connection/non-2xx propagate as exceptions (`raise_for_status`); the adapter catches them (SS-115). Test `test_maindata_returns_full_payload`. |
|
||||
| SS-111 | Three widget kinds dispatched | **PASS** | `QbittorrentWidgetSource.fetch` resolves `QbittorrentClient` inline from `ServiceRecord` config+secrets, dispatches on `widget_kind ∈ {totals, active, speed}`. Registered in `SERVICE_ADAPTERS["qbittorrent"]`. |
|
||||
| SS-112 | totals = item count | **PASS** | `totals` → `{"total": len(torrents), "by_state": {...}}`. Count of currently-listed torrents, **not** cumulative bytes. Test `test_qbittorrent_totals_counts_all_torrents`. |
|
||||
| SS-113 | active = DL/UL filter | **PASS** | `active` keeps only `state in {"downloading","uploading"}` → `{"torrents": [...]}`. Test `test_qbittorrent_active_filters_dl_ul_only` asserts queued/stalled excluded. |
|
||||
| SS-114 | speed appends + returns series | **PASS** | `speed` reads `dl_info_speed`/`up_info_speed`, `store.append(service.id, ts, dl, up)`, returns `{"series": [{label:"download", points:[...]}, {label:"upload", points:[...]}]}` from `.window()` with `t = ts * 1000` (JS ms). Test `test_qbittorrent_speed_appends_and_returns_series` asserts labels + last download value. |
|
||||
| SS-115 | Errors degrade gracefully | **PASS** | `fetch` body wrapped in `try/except`: `asyncio.TimeoutError → {"error": "...timed out"}`, generic `Exception → {"error": "...failed: ..."}`. Missing service / missing creds return `{"error":...}` before any network call. Tests cover missing-service, missing-credentials, timeout. Never raises. |
|
||||
| SS-116 | Three FE widget components + binding | **PASS** | `QbittorrentTotalsWidget`, `QbittorrentActiveTorrentsWidget`, `QbittorrentSpeedWidget` exist under `frontend/src/widgets/`; `registry.ts` binds them to `qbittorrent` with kinds `totals`/`active`/`speed`. |
|
||||
| SS-117 | Speed widget reuses shared renderer | **PASS** | `QbittorrentSpeedWidget` → `<LineSeriesChart series={series} height={220} />`. No new charting code, no new charting dependency. |
|
||||
| SS-118 | LineSeriesChart extraction non-regressive | **PASS** | `LineSeriesChart.tsx` owns `mergeSeries`/`formatTime`/`CHART_COLORS`/`ChartSeries`/recharts JSX; `PrometheusChartWidget.tsx` is a thin wrapper (`useWidgetData` → `<LineSeriesChart series={series} />`). `PrometheusChartWidget.test.tsx` → **4 passed**. |
|
||||
| SS-119 | service_id column added (idempotent) | **PASS** | `MEDIA_INDEX_CONCERN.migrations = ["ALTER TABLE media_items ADD COLUMN service_id TEXT NOT NULL DEFAULT ''"]`; `init_schema` CREATE TABLE also includes it. `db_filename="media_index.sqlite"` == `DEFAULT_INDEX_PATH.name`; harness `base_dir` == `.cache/media_library_viewer` → **file location unchanged**. |
|
||||
| SS-120 | Existing rows backfilled | **PASS** | Column `DEFAULT ''` backfills all pre-existing rows to `service_id=""` (preserving visibility). |
|
||||
| SS-121 | replace_items scoped (bug fix) | **PASS** | `replace_items(rows, service_id="")` does `DELETE FROM media_items WHERE service_id = ?` (NOT global). **Regression test `test_replace_scoped_by_service_id_preserves_other_services`** inserts svc-a (2 rows), replaces svc-b (1 row), asserts `item_count==3` (svc-a survives), and scoped queries return only each service's rows. Latent global-clear bug confirmed fixed. |
|
||||
| SS-122 | query backward-compatible | **PASS** | `query(..., service_id="")`: non-empty appends `WHERE service_id = ?`; empty (default) skips the filter → all rows. Existing `test_media_index.py` / `test_api.py` callers pass no `service_id` and run **unchanged** (322 passed). |
|
||||
| SS-123 | Worker threads service_id | **PASS** | `run_build(..., service_id="")` receives `--service-id` from argparse and passes `service_id=service_id` into `build_media_index(...)`, which calls `index.replace_items(normalized_rows, service_id=service_id)` (line 488). Newly-built rows stamped with the real service_id. |
|
||||
| SS-124 | MediaIndex registered as concern | **PASS** | `MEDIA_INDEX_CONCERN` (tables=`["media_items"]`) registered in `get_service_data_harness()`, so `media_items` participates in cascade-delete. |
|
||||
| SS-125 | delete_service triggers cascade | **PASS** | `settings_store.delete_service` (after `DELETE FROM services WHERE id=?`) does `try: get_service_data_harness().cascade_delete(service_id); except Exception: logger.exception(...)` — **best-effort, local import** to avoid circular dependency. A cascade failure does not crash deletion. |
|
||||
| SS-126 | End-to-end cascade across both concerns | **PASS** | `test_cascade_delete_removes_harness_data_across_concerns`: qBit A+B samples appended, delete A → A gone, B survives; media items A+B built with distinct service_ids, delete A → A rows gone (`total_a_after==0`), B survives (`total_b_after==1`). Both concerns + multi-instance preservation proven. |
|
||||
| SS-127 | Backend tests + lint green | **PASS** | `pytest` → **322 passed**; `ruff check src tests` → `All checks passed!`. New tests present for harness lifecycle, qBit store, qBit client (login/403/cookie-reuse/maindata), qBit adapter (3 branches + errors), scoped MediaIndex regression, cascade-delete integration. |
|
||||
| SS-128 | Frontend build + lint green | **PASS** | `npm run build` exit 0; `npm run lint` 0 errors (1 pre-existing warning). New widget tests cover **loading/error/rendered** states (Totals includes an explicit `isLoading:true` skeleton case — improving on Change A's SC-125 gap). PrometheusChartWidget tests green (SS-118). |
|
||||
|
||||
**Functional spec coverage: 28/28 fully PASS.**
|
||||
|
||||
---
|
||||
|
||||
## 4. Task completion status — ⚠ archive blocker (reconcilable)
|
||||
|
||||
`tasks.md` checkbox state (via `grep -nE '^\s*- \[' tasks.md`):
|
||||
|
||||
- **Slice 1 (§1.1–1.9): all 9 `[ ]` — UNCHECKED**
|
||||
- **Slice 2 (§2.1–2.9): all 9 `[ ]` — UNCHECKED**
|
||||
- **Slice 3 (§3.1–3.9): all 9 `[ ]` — UNCHECKED**
|
||||
- **Slice 4 (§4.1–4.3): all 3 `[ ]` — UNCHECKED**
|
||||
- **Integration (§5.1–5.5): all 5 `[ ]` — UNCHECKED**
|
||||
|
||||
Total: **0 checked, 30 unchecked**. **`apply-progress.md` does not exist.**
|
||||
|
||||
**This is a CRITICAL completeness issue for the archive gate per the verify
|
||||
contract.** However — as with Change A — **all 30 items are verifiably DONE
|
||||
against source**. Selected reconciliation (full mapping available on request):
|
||||
|
||||
| Unchecked task | Actual state (verified) |
|
||||
|----------------|-------------------------|
|
||||
| 1.1 `ServiceDataHarness` module | **done** (`service_data.py` exists; lifecycle-only) |
|
||||
| 1.2 `QbittorrentSampleStore` | **done** (`qbittorrent_store.py`; `QBITTORRENT_CONCERN` registered) |
|
||||
| 1.3 `QbittorrentClient` | **done** (`clients/qbittorrent.py`; login/403/maindata) |
|
||||
| 1.4/1.5 integration def + registry | **done** (`integrations/qbittorrent.py` + `registry.py` entry) |
|
||||
| 1.6 harness init in lifespan | **done** (`main.py:56-60` try/except) |
|
||||
| 1.7/1.8 backend tests | **done** (`test_service_data.py`=9, `test_qbittorrent_client.py`=12) |
|
||||
| 2.1 adapter | **done** (`QbittorrentWidgetSource` + `SERVICE_ADAPTERS`) |
|
||||
| 2.3 LineSeriesChart extract | **done** (`components/LineSeriesChart.tsx`; PrometheusChartWidget thin wrapper) |
|
||||
| 2.4–2.6/2.7 3 FE widgets + binding | **done** (`QbittorrentTotals/Active/SpeedWidget` + registry.ts) |
|
||||
| 2.8 FE tests | **done** (3 widget tests + LineSeriesChart test) |
|
||||
| 3.1–3.7 MediaIndex migration | **done** (concern + column + scoped replace_items + scoped query + worker/router threading) |
|
||||
| 3.8 scoped MediaIndex tests | **done** (`test_replace_scoped_by_service_id_preserves_other_services`) |
|
||||
| 4.1/4.2 cascade wiring + integration test | **done** (`delete_service` cascade + `test_cascade_delete_removes_harness_data_across_concerns`) |
|
||||
| 5.1–5.5 full-suite gates | **done** (322 passed / ruff clean / build+lint exit 0 / extraction non-regression) |
|
||||
|
||||
The unchecked boxes are **stale** (work performed, tracker not updated), and **no
|
||||
`apply-progress.md` exists** to serve as the stale-checkbox reconciliation record
|
||||
the contract permits. Resolution is a **documentation-only** step: tick
|
||||
§1.1–1.9, §2.1–2.9, §3.1–3.9, §4.1–4.3, §5.1–5.5, and author `apply-progress.md`
|
||||
documenting the four landed slices. **No code change is required.**
|
||||
|
||||
> Per the verify contract, an unchecked implementation-task line is an archive
|
||||
> blocker until reconciled. Because the implementation is verified complete, this
|
||||
> blocks **archive** but does **not** block `sdd-sync` of the green code.
|
||||
|
||||
## 5. TDD compliance & assertion-quality assessment
|
||||
|
||||
Strict-TDD was **not** declared active for this change in `openspec/config.yaml` /
|
||||
parent prompt / (absent) `apply-progress.md`, so the formal TDD-cycle-evidence
|
||||
check is **not applicable**. Assertion quality was audited adversarially.
|
||||
|
||||
**Backend assertions — GENUINELY BEHAVIORAL (good).** Spot-checked:
|
||||
|
||||
- `test_media_index.py::test_replace_scoped_by_service_id_preserves_other_services`:
|
||||
inserts two services' rows, replaces the second, asserts the **count** (3, not
|
||||
1 — proves the global-clear bug is fixed) and the **id sets** per scoped query
|
||||
(`{"a1","a2"}`, `{"b1"}`). No tautology; directly locks in SS-121.
|
||||
- `test_qbittorrent_client.py`: `test_cookie_reuse_does_not_re_login` asserts
|
||||
`session.post.assert_not_called()` after login (proves no re-auth per request);
|
||||
`test_403_triggers_re_login` asserts `get` call_count==2 + `post` called once.
|
||||
- `test_widgets.py` qBit adapters: totals asserts `total` + `by_state` mapping;
|
||||
active asserts DL/UL kept and queued/stalled excluded; speed asserts series
|
||||
labels (`["download","upload"]`) + last download value.
|
||||
- `test_services.py::test_cascade_delete_removes_harness_data_across_concerns`:
|
||||
asserts qBit A window `== []` after delete while B `len==1`; media A
|
||||
`total_a_after==0` while B `total_b_after==1`. Real cross-concern, multi-instance.
|
||||
|
||||
**Frontend assertions — adequate, two minor notes.**
|
||||
|
||||
- qBit widget tests render the **actual value text** (e.g. `"4"`, state badges
|
||||
`"downloading: 1"`) and the error alert — meaningful. The Totals test includes
|
||||
an explicit **loading** skeleton case (`isLoading:true` → `[data-slot="skeleton"]`),
|
||||
which closes the gap Change A's SC-125 left open. Good.
|
||||
- `LineSeriesChart.test.tsx` is **smoke-only** (asserts `container.firstChild`
|
||||
non-null; does not assert the recharts `<Line>` series rendered). Acceptable as
|
||||
a "renders without crashing" guard but does not prove lines drew. Non-blocking.
|
||||
|
||||
No ghost loops, no type-only assertions, no implementation-detail CSS assertions,
|
||||
no tautologies found.
|
||||
|
||||
## 6. Review-workload / PR-boundary findings
|
||||
|
||||
Per-slice changed lines (numstat; "source" excludes tests/docs):
|
||||
|
||||
| Commit | Slice | Source Δ | Total Δ | Over 400? | Verdict |
|
||||
|--------|-------|----------|---------|-----------|---------|
|
||||
| `e7bd0af` | 1 harness+store+client+integration | +400 / -0 | +685 / -1 | at budget (source) | **OK** |
|
||||
| `1fb12b8` | 2 adapter+LineSeriesChart+3 FE widgets | **+644 / -81** | +831 / -81 | **over** | **WARNING** — additive feature code+tests; boundary is exactly qBit widgets (see below) |
|
||||
| `c87f398` | 3 MediaIndex migration | +46 / -7 | +93 / -7 | under | **OK** |
|
||||
| `75c949a` | 4 cascade-delete wiring | +13 / -3 | +99 / -22 | under | **OK** |
|
||||
|
||||
The `tasks.md` Review Workload Forecast (`stacked-to-main`, 4 slices,
|
||||
~985–1,205 total) was followed: S1→S2→S3→S4, each independently green. **Slice 2's
|
||||
+644 source insertions exceed the 400-line budget** and the slice's own
|
||||
"~350–400" forecast (which under-counted the 3 components + LineSeriesChart
|
||||
extraction + their tests). No `size:exception` was recorded. This is a
|
||||
**forecast-vs-actual variance on an additive slice**, not scope creep — the
|
||||
boundary is exactly the qBit-widget feature and no unrelated files were touched.
|
||||
Recommend recording the slice-2 actual in the archive summary. **Non-blocking.**
|
||||
|
||||
Scope was honored: no backend API/type-contract widening beyond the planned
|
||||
`jellyfin_service_id` query param (defaulted, backward-compatible); MediaIndex
|
||||
file location unchanged; no coupling to the unrelated `mobile-responsive-parity`
|
||||
or `services-as-hub-ia` changes (this change builds/tests green independently).
|
||||
|
||||
## 7. Adversarial checks
|
||||
|
||||
- **Harness initialized at startup?** **Yes.** `main.py::lifespan` calls
|
||||
`get_service_data_harness()` (which lazy-registers both concerns and runs
|
||||
migrations) inside a try/except after `ensure_defaults()` (lines 56–60). So
|
||||
`cascade_delete` is operational at runtime, not just in tests.
|
||||
- **qBit client cookie reuse (no re-auth per request)?** **Correct.** `_get`
|
||||
only calls `_login()` when `not self._logged_in`; the SID cookie persists in
|
||||
the `requests.Session`. `test_cookie_reuse_does_not_re_login` locks this in.
|
||||
- **Speed timestamp ×1000 vs LineSeriesChart expectation?** **Consistent.**
|
||||
Adapter: `{"t": s["ts"] * 1000, ...}` (store keeps unix seconds; ×1000 → JS ms).
|
||||
`LineSeriesChart.formatTime(ms)` = `new Date(ms).toLocaleTimeString()`. Same
|
||||
`{t:ms, v}` contract Prometheus's `normalize_prometheus_matrix` produces, so
|
||||
both chart consumers are interchangeable.
|
||||
- **Dead code / unregistered concerns?** **None found.** Both
|
||||
`QBITTORRENT_CONCERN` and `MEDIA_INDEX_CONCERN` register in
|
||||
`get_service_data_harness()`. `SERVICE_ADAPTERS` has `qbittorrent`;
|
||||
`SERVICE_DEFINITIONS` has `qbittorrent`; frontend `SERVICE_REGISTRY` has
|
||||
`qbittorrent`. `ruff` (catches unused imports) is clean; `tsc`/`eslint` clean.
|
||||
(`authentik` is backend-only by design — pre-existing, unrelated.)
|
||||
- **qBit widget kinds declared consistently in BOTH registries?** **Yes.** Backend
|
||||
`integrations/qbittorrent.py` declares `totals`(30s)/`active`(15s)/`speed`(5s);
|
||||
frontend `registry.ts` declares the same three kinds with **identical** names,
|
||||
descriptions, and refresh intervals. Empty config schemas match (`{type:"object",
|
||||
properties:{}, required:[]}` ↔ empty `model_cls`).
|
||||
- **MediaIndex backward-compat real?** **Yes.** Existing `test_media_index.py`
|
||||
calls `replace_items(rows)` / `.query(...)` with no `service_id`; they default
|
||||
to `""` → delete `WHERE service_id=''` / no filter → all rows. Full suite (322)
|
||||
green, including `test_api.py::TestMediaIndexApi`.
|
||||
|
||||
## 8. Residual risks / non-blocking findings
|
||||
|
||||
1. **[CRITICAL-process] 30 unchecked tasks + missing `apply-progress.md`** (§4) — archive blocker; reconciliation is doc-only.
|
||||
2. **[WARNING] Slice 2 over budget** (§6) — +644 source lines vs 400-line budget; additive feature slice, no scope creep; no `size:exception` recorded.
|
||||
3. **[INFO] `LineSeriesChart.test.tsx` smoke-only** (§5) — asserts mount, not rendered lines.
|
||||
4. **[INFO] Prune `NOT IN (… LIMIT 120)` edge case** — duplicate `ts` values could in theory keep >120 rows; negligible at 5 s poll.
|
||||
5. **[INFO] Stale generated `.pi-map.md`** files predate the new modules; not deliverable source. Regenerate.
|
||||
6. **[INFO] Slice-2 commit hash drift** — brief cited `8b0e7ea`; actual is `1fb12b8`. Content matches; informational.
|
||||
7. **[INFO] Uncommitted `MediaTab.tsx`** cosmetic reformat + untracked `.pi-tmp/*` (orthogonal to this change). **No files are staged** (`git diff --cached` empty).
|
||||
8. **[INFO] Chunk-size build warning** (~1.1 MB JS) — non-fatal, pre-existing, orthogonal.
|
||||
9. **No browser/visual smoke** performed (out of scope); the qBit speed recharts line is only structurally tested.
|
||||
|
||||
## 9. Exact blockers
|
||||
|
||||
- **BLOCKER (archive only, doc-reconcilable):** 30 unchecked
|
||||
implementation/verification tasks (§1.1–5.5) and absent `apply-progress.md`.
|
||||
Implementation is verified complete; resolution = tick boxes + write
|
||||
`apply-progress.md`.
|
||||
|
||||
No code-level blockers. All functional requirements SS-101…SS-128 pass. All four
|
||||
gates green. **Code is ready for `sdd-sync`; archive requires the
|
||||
checkbox/apply-progress reconciliation.**
|
||||
|
||||
## 10. Recommended next phase
|
||||
|
||||
→ **`sdd-sync`** (code PASS). Concurrently/after: author `apply-progress.md`
|
||||
documenting the four landed slices, and tick §1.1–1.9, §2.1–2.9, §3.1–3.9,
|
||||
§4.1–4.3, §5.1–5.5 in `tasks.md` to clear the archive blocker. Optionally
|
||||
strengthen `LineSeriesChart.test.tsx` to assert rendered `<Line>` elements, and
|
||||
regenerate the stale `.pi-map.md`.
|
||||
|
||||
---
|
||||
|
||||
### Appendix A — Verification commands run (at `c9404f0`)
|
||||
|
||||
```
|
||||
cd backend && PYTHONPATH=src python3 -m pytest -q → 322 passed (2 warnings)
|
||||
cd backend && PYTHONPATH=src python3 -m ruff check src tests → All checks passed!
|
||||
cd backend && PYTHONPATH=src python3 -m pytest tests/test_service_data.py tests/test_qbittorrent_client.py -q
|
||||
→ 40 passed
|
||||
cd frontend && npm run build → exit 0 (✓ built; >500kB warning pre-existing)
|
||||
cd frontend && npm run lint → exit 0 (0 errors, 1 pre-existing warning)
|
||||
cd frontend && npx vitest run src/widgets/__tests__/PrometheusChartWidget.test.tsx
|
||||
→ 4 passed (SS-118 non-regression)
|
||||
cd frontend && npx vitest run src/widgets/__tests__/ src/components/__tests__/LineSeriesChart.test.tsx
|
||||
→ 8 files, 31 tests passed
|
||||
grep -nE '^\s*- \[' openspec/changes/service-storage-harness/tasks.md → 30 unchecked (all tasks)
|
||||
ls openspec/changes/service-storage-harness/apply-progress.md → ENOENT (missing)
|
||||
git diff --cached --name-only → empty (no staged files)
|
||||
```
|
||||
|
||||
### Appendix B — Files substantively changed
|
||||
|
||||
**Backend (new)**
|
||||
|
||||
- `services/service_data.py` — `StorageConcern`, `ServiceDataHarness` (lifecycle), `get_service_data_harness()`.
|
||||
- `services/qbittorrent_store.py` — `QBITTORRENT_CONCERN`, `QbittorrentSampleStore` (append/window/prune).
|
||||
- `clients/qbittorrent.py` — `QbittorrentClient` (cookie login, 403 re-login, maindata).
|
||||
- `integrations/qbittorrent.py` — `QbittorrentConfig`/`QbittorrentWidgetConfig`/`DEFINITION` (3 widget kinds).
|
||||
|
||||
**Backend (modified)**
|
||||
|
||||
- `integrations/registry.py` — `qbittorrent` registered in `SERVICE_DEFINITIONS`.
|
||||
- `widgets/sources.py` — `QbittorrentWidgetSource` (3 branches) + `SERVICE_ADAPTERS["qbittorrent"]`.
|
||||
- `services/media_index_impl.py` — `MEDIA_INDEX_CONCERN`, `+service_id` column, scoped `replace_items`/`query`, `build_media_index(service_id=...)`.
|
||||
- `workers/media_index_worker.py` — threads `--service-id` into `build_media_index`.
|
||||
- `routers/media.py` — `query_media(jellyfin_service_id=...)` → `index.query(service_id=...)`.
|
||||
- `services/settings_store.py` — `delete_service` calls harness `cascade_delete` (best-effort).
|
||||
- `main.py` — `lifespan` initializes `get_service_data_harness()`.
|
||||
|
||||
**Backend tests (new/extended)**
|
||||
|
||||
- `tests/test_service_data.py` (new, 9), `tests/test_qbittorrent_client.py` (new, 12).
|
||||
- `tests/test_media_index.py` (+scoped regression), `tests/test_widgets.py` (+6 qBit adapter tests), `tests/test_services.py` (+cascade integration).
|
||||
|
||||
**Frontend (new/modified)**
|
||||
|
||||
- `components/LineSeriesChart.tsx` (new, extracted), `widgets/PrometheusChartWidget.tsx` (thin wrapper).
|
||||
- `widgets/QbittorrentTotalsWidget.tsx`, `QbittorrentActiveTorrentsWidget.tsx`, `QbittorrentSpeedWidget.tsx` (new).
|
||||
- `integrations/registry.ts` — `qbittorrent` binding (3 widget kinds).
|
||||
- Tests: `QbittorrentTotals/Active/SpeedWidget.test.tsx` (new) + `LineSeriesChart.test.tsx` (new).
|
||||
Reference in New Issue
Block a user