alex/manage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add typed qBittorrent scheduled polling

Browse Source
This commit is contained in:
Developer
2026-07-14 15:22:34 +00:00
parent eac9b5d33d
commit a9488af0b4
22 changed files with 2036 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/plans/2026-07-14-scheduled-actions-test-plan.md
+156
View File
@@ -0,0 +1,156 @@
# Scheduled Actions Test Plan / QA Checklist
## Test objectives
Verify that qBittorrent speed collection is backend-owned, configurable per service, resilient to transient failures, observable, and safe when the frontend is not open.
## Backend unit tests
### Configuration
- [ ] Existing qBittorrent configs receive polling defaults without changing secrets.
- [ ] `polling_enabled` accepts booleans and defaults to enabled.
- [ ] Poll intervals below 5 seconds or above 300 seconds are rejected.
- [ ] Retention below 60 seconds or above 86,400 seconds is rejected.
- [ ] Sample caps below the minimum or above 1,200 are rejected.
- [ ] Credential-looking fields remain rejected from service config.
### Sample storage
- [ ] Samples remain isolated by `service_id`.
- [ ] Timestamp retention removes rows older than the configured window.
- [ ] Row-cap retention removes the oldest rows when the cap is exceeded.
- [ ] Both retention rules are applied together.
- [ ] Samples are ordered oldest to newest for chart responses.
- [ ] Service cascade deletion removes samples.
- [ ] Empty and missing databases initialize safely.
### Scheduler core
- [ ] Only registered action keys can execute.
- [ ] Disabled services do not run.
- [ ] Enabled services run immediately after startup with deterministic staggering.
- [ ] A normal run waits the configured interval after completion.
- [ ] A slow run cannot overlap itself.
- [ ] A slow run does not create queued backlog entries.
- [ ] Stop interrupts the wait and joins the worker within the configured timeout.
- [ ] A configuration change is applied on the next reconciliation cycle.
- [ ] Disabling a service allows an in-flight run to finish, then prevents new runs.
- [ ] Deleting a service removes its schedule state and history.
### Backoff and manual runs
- [ ] A failed run is persisted with safe error text and increments failure state.
- [ ] Retry delay increases exponentially and respects the configured cap.
- [ ] Backoff does not create duplicate queued runs.
- [ ] A successful scheduled run clears failures and backoff.
- [ ] A successful manual run clears failures and backoff.
- [ ] A failed manual run follows normal failure persistence and retry behavior.
- [ ] Manual runs do not alter the configured interval.
### qBittorrent action and widget separation
- [ ] The scheduled action calls qBittorrent and appends exactly one sample per successful poll.
- [ ] The qBittorrent client cache is reused correctly per service.
- [ ] Timeouts become failed runs without blocking the scheduler indefinitely.
- [ ] The `speed` widget adapter reads samples but does not call qBittorrent.
- [ ] Multiple widgets or browser tabs do not multiply samples.
- [ ] A failed latest poll still returns last-known samples with stale status.
## Backend API tests
- [ ] Scheduler status requires normal API authentication.
- [ ] Status returns effective configuration, last attempt, last success, failure count, backoff, and stale state.
- [ ] Missing service returns 404 or the project’s established service error shape.
- [ ] Disabled service status is explicit and does not run an action.
- [ ] Run history is paginated and supports status/trigger filters.
- [ ] Run history is bounded by 30 days and 1,000 records per service/action.
- [ ] Manual-run endpoint returns a typed result and records the attempt.
- [ ] Samples endpoint accepts a display window and is read-only.
- [ ] Widget-data requests never cause a qBittorrent external call.
- [ ] Responses never expose usernames, passwords, API keys, headers, or raw payloads.
## Observability tests
- [ ] Run counters increment for success and failure.
- [ ] Duration metrics record completed attempts.
- [ ] Last-success gauges update only after successful sampling.
- [ ] Failure/stale gauges reset after recovery.
- [ ] Metric labels are bounded and contain no secrets or raw URLs.
- [ ] Structured logs include action/service/status context and sanitize errors.
## Frontend unit/component tests
- [ ] qBittorrent schedule fields render only for qBittorrent services.
- [ ] Invalid interval, retention, and cap values show validation feedback.
- [ ] Save preserves existing encrypted-secret behavior.
- [ ] Disabled polling clearly shows paused state.
- [ ] Status card renders healthy, running, backoff, stale, disabled, and never-run states.
- [ ] `Run now` shows pending state and disables duplicate clicks.
- [ ] Successful manual run refreshes status/history and clears backoff display.
- [ ] Failed manual run renders a safe error.
- [ ] Chart window selector requests samples without changing sampler settings.
- [ ] Stale warning appears while last-known speed data remains visible.
- [ ] Run history renders pagination, trigger, status, duration, timestamp, and error details.
- [ ] Empty history and no-data states are readable on mobile.
## Integration / lifespan tests
- [ ] Starting the FastAPI lifespan starts the scheduler exactly once.
- [ ] Repeated `start()` calls do not create duplicate workers.
- [ ] Lifespan shutdown stops the scheduler and does not leak a thread.
- [ ] A test app can override the scheduler/action registry cleanly.
- [ ] Existing mail queue and backup poller lifecycle behavior remains unchanged.
## Manual QA scenarios
### Headless collection
1. Configure an enabled qBittorrent service.
2. Start the backend without opening the frontend.
3. Wait for at least two intervals.
4. Query scheduler status and samples directly.
5. Confirm samples and successful run records exist.
### Duplicate prevention
1. Open the speed widget in multiple browser tabs.
2. Compare sample count growth to scheduler run count.
3. Confirm browser refreshes do not add samples or external qBittorrent calls.
### Outage and recovery
1. Make qBittorrent unreachable.
2. Confirm failures and increasing backoff appear in status/history.
3. Confirm last-known samples remain visible with a stale warning.
4. Restore qBittorrent.
5. Confirm the next successful scheduled or manual run clears backoff and stale state.
### Configuration reload
1. Change the interval and retention in the qBittorrent service editor.
2. Confirm the existing worker remains alive.
3. Confirm the new effective values appear after reconciliation.
4. Confirm pruning follows the new retention/cap.
### Service lifecycle
1. Disable a service and confirm no new runs are created while history remains.
2. Re-enable it and confirm polling resumes.
3. Delete it and confirm service-owned samples and run records are removed.
### Deployment constraint
1. Run the documented single-worker deployment.
2. Confirm one scheduler worker is active.
3. Verify the deployment documentation warns against multiple scheduler-capable replicas.
## Release gate
- [ ] Backend test suite passes.
- [ ] Frontend tests pass.
- [ ] Frontend lint passes with no new violations.
- [ ] Frontend build passes.
- [ ] No secret appears in logs, API responses, metrics, or scheduler run records.
- [ ] Project maps are patched for new files and validated.
- [ ] Requirements and runbook documentation match the shipped behavior.
docs/superpowers/plans/2026-07-14-scheduled-actions.md
+124
View File
@@ -0,0 +1,124 @@
# Typed Scheduler and qBittorrent Polling Implementation Plan
> This plan implements the approved design in `docs/superpowers/specs/2026-07-14-scheduled-actions-design.md`.
**Goal:** Move qBittorrent speed collection into a backend-owned typed scheduler while adding per-service controls, run history, stale-data status, and a manual run action.
**Scope:** qBittorrent speed polling only. The scheduler registry is extensible, but arbitrary widgets, SSH tasks, and other integrations remain out of scope.
## Delivery slices
### Slice 1 — Contracts and persistence
- [ ] Add validated qBittorrent config fields to `backend/src/media_library_viewer_api/integrations/qbittorrent.py`:
- `polling_enabled` default `true`;
- `poll_interval_seconds` default `15`, range `5..300`;
- `sample_retention_seconds` default `1800`, range `60..86400`;
- `sample_max_rows` default `1200`, range `60..1200`.
- [ ] Add migration/default normalization tests for existing qBittorrent records.
- [ ] Extend `QbittorrentSampleStore` to prune by retention timestamp and capped row count.
- [ ] Add a generic scheduler storage concern with run records, indexes, pruning, and cascade deletion by service ID.
- [ ] Add typed backend models for scheduler status, run records, samples, and manual-run responses.
**Acceptance:** Existing service records validate without edits; qBittorrent samples remain isolated by service; scheduler records cannot contain secrets; deletion removes service-owned samples and runs.
### Slice 2 — Typed scheduler core
- [ ] Create a scheduler action protocol and registry.
- [ ] Implement a single-worker, lifespan-managed scheduler coordinator with responsive stop behavior.
- [ ] Add qBittorrent speed sampling as the first registered action.
- [ ] Extract external polling from `QbittorrentWidgetSource` into a reusable sampler/action helper.
- [ ] Implement immediate startup execution with deterministic staggering.
- [ ] Implement fixed-delay, no-overlap execution and bounded exponential backoff.
- [ ] Reconcile enabled services/config changes on each cycle.
- [ ] Add safe structured logs and Prometheus metrics.
- [ ] Start/stop the scheduler in `main.py` alongside the existing mail queue and backup poller.
**Acceptance:** With no frontend open, enabled qBittorrent services append samples; one slow service cannot create overlapping runs or a backlog; shutdown joins the worker; a successful manual or scheduled run resets backoff.
### Slice 3 — Read-only APIs
- [ ] Add `backend/src/media_library_viewer_api/routers/scheduler.py`.
- [ ] Add status, paginated runs, manual-run, and read-only samples endpoints.
- [ ] Keep service configuration writes on the existing service-instance API.
- [ ] Change the qBittorrent speed widget adapter to read samples only.
- [ ] Add stale-data calculation and safe error truncation.
- [ ] Add API tests for disabled/missing services, stale data, pagination, manual runs, backoff, and authentication.
**Acceptance:** Opening or refreshing a speed widget never contacts qBittorrent and never appends a sample; API responses expose timestamps and status but no credentials.
### Slice 4 — Frontend controls and history
- [ ] Add scheduler TypeScript types, API functions, and React Query hooks.
- [ ] Add qBittorrent schedule controls to the existing schema-driven service editor.
- [ ] Add status/backoff/stale-data presentation and a `Run now` action.
- [ ] Add user-selectable chart windows.
- [ ] Add a paginated run-history table with safe error details.
- [ ] Keep UI refreshes separate from sampler cadence.
- [ ] Add frontend tests for validation, disabled state, stale warning, manual-run reset, chart-window selection, and run-history rendering.
**Acceptance:** Operators can configure, inspect, and manually trigger qBittorrent polling from the service surface without opening the dashboard; the chart remains useful during outages and identifies stale data.
### Slice 5 — Documentation and operational verification
- [ ] Update `docs/REQUIREMENTS.md` with scheduler requirements and the one-worker constraint.
- [ ] Update deployment/runbook documentation with scheduler startup, shutdown, and replica guidance.
- [ ] Add migration/recovery notes for sample and run-history retention.
- [ ] Run backend tests, frontend tests, lint, and build.
- [ ] Verify a headless collection scenario against a mocked qBittorrent service.
- [ ] Verify project-map artifacts after files are added.
## Suggested file map
### Backend
| File | Change |
| --- | --- |
| `backend/src/media_library_viewer_api/integrations/qbittorrent.py` | Schedule config schema and defaults |
| `backend/src/media_library_viewer_api/services/qbittorrent_store.py` | Duration/cap pruning and sample queries |
| `backend/src/media_library_viewer_api/services/service_data.py` | Register scheduler run concern |
| `backend/src/media_library_viewer_api/services/scheduler.py` | Worker lifecycle, reconciliation, timing, backoff |
| `backend/src/media_library_viewer_api/services/scheduler_actions.py` | Typed registry and qBittorrent action |
| `backend/src/media_library_viewer_api/services/scheduler_store.py` | Run-record persistence and pruning |
| `backend/src/media_library_viewer_api/models/scheduler.py` | Response/request models |
| `backend/src/media_library_viewer_api/routers/scheduler.py` | Status, history, samples, manual-run API |
| `backend/src/media_library_viewer_api/widgets/sources.py` | Make qBittorrent speed reads side-effect free |
| `backend/src/media_library_viewer_api/main.py` | Start/stop scheduler |
| `backend/tests/test_scheduler.py` | Scheduler lifecycle/timing/backoff tests |
| `backend/tests/test_scheduler_api.py` | Endpoint and auth tests |
| `backend/tests/test_service_data.py` | Migration/cascade coverage |
| `backend/tests/test_widgets.py` | Read-only qBittorrent widget coverage |
### Frontend
| File | Change |
| --- | --- |
| `frontend/src/types/scheduler.ts` | Scheduler status/run/sample types |
| `frontend/src/api/scheduler.ts` | Typed endpoint wrappers |
| `frontend/src/hooks/useScheduler.ts` | Queries and manual-run mutation |
| `frontend/src/pages/ServicesPage.tsx` | qBittorrent schedule controls/status surface |
| `frontend/src/pages/ServicePage.tsx` or qBittorrent service tab | Status, chart window, history surface |
| `frontend/src/widgets/QbittorrentSpeedWidget.tsx` | Read-only sample window and stale warning |
| `frontend/src/integrations/registry.ts` | Schedule metadata/config exposure if needed |
| `frontend/src/types/index.ts` | Shared exports |
## Risks and mitigations
- **Duplicate polling:** widget adapter becomes read-only; only scheduler action calls qBittorrent.
- **Multiple backend workers:** document and log the one-worker constraint; do not silently duplicate work.
- **Unbounded storage:** prune by both duration and row cap; test pruning under rapid polling.
- **Credential leakage:** reuse existing secret resolution and sanitize run errors/log fields.
- **Scheduler shutdown races:** use a stop event, per-action lock, and bounded joins; test lifespan shutdown.
- **Config changes during a run:** let the current run finish, then reconcile on the next cycle.
- **Stale but useful data:** return samples plus explicit stale status rather than blanking the chart.
## Verification commands
```bash
cd backend && PYTHONPATH=src pytest
cd frontend && npm test
cd frontend && npm run lint
cd frontend && npm run build
```
Do not begin implementation until the final module names, retry cap/jitter, and chart-window response shape are confirmed during the implementation pass.