Developer
157 lines
7.0 KiB
Markdown
157 lines
7.0 KiB
Markdown
# Scheduled Actions Test Plan / QA Checklist
|
||
|
||
## Test objectives
|
||
|
||
Verify that qBittorrent speed collection is backend-owned, configurable per service, resilient to transient failures, observable, and safe when the frontend is not open.
|
||
|
||
## Backend unit tests
|
||
|
||
### Configuration
|
||
|
||
- [ ] Existing qBittorrent configs receive polling defaults without changing secrets.
|
||
- [ ] `polling_enabled` accepts booleans and defaults to enabled.
|
||
- [ ] Poll intervals below 5 seconds or above 300 seconds are rejected.
|
||
- [ ] Retention below 60 seconds or above 86,400 seconds is rejected.
|
||
- [ ] Sample caps below the minimum or above 1,200 are rejected.
|
||
- [ ] Credential-looking fields remain rejected from service config.
|
||
|
||
### Sample storage
|
||
|
||
- [ ] Samples remain isolated by `service_id`.
|
||
- [ ] Timestamp retention removes rows older than the configured window.
|
||
- [ ] Row-cap retention removes the oldest rows when the cap is exceeded.
|
||
- [ ] Both retention rules are applied together.
|
||
- [ ] Samples are ordered oldest to newest for chart responses.
|
||
- [ ] Service cascade deletion removes samples.
|
||
- [ ] Empty and missing databases initialize safely.
|
||
|
||
### Scheduler core
|
||
|
||
- [ ] Only registered action keys can execute.
|
||
- [ ] Disabled services do not run.
|
||
- [ ] Enabled services run immediately after startup with deterministic staggering.
|
||
- [ ] A normal run waits the configured interval after completion.
|
||
- [ ] A slow run cannot overlap itself.
|
||
- [ ] A slow run does not create queued backlog entries.
|
||
- [ ] Stop interrupts the wait and joins the worker within the configured timeout.
|
||
- [ ] A configuration change is applied on the next reconciliation cycle.
|
||
- [ ] Disabling a service allows an in-flight run to finish, then prevents new runs.
|
||
- [ ] Deleting a service removes its schedule state and history.
|
||
|
||
### Backoff and manual runs
|
||
|
||
- [ ] A failed run is persisted with safe error text and increments failure state.
|
||
- [ ] Retry delay increases exponentially and respects the configured cap.
|
||
- [ ] Backoff does not create duplicate queued runs.
|
||
- [ ] A successful scheduled run clears failures and backoff.
|
||
- [ ] A successful manual run clears failures and backoff.
|
||
- [ ] A failed manual run follows normal failure persistence and retry behavior.
|
||
- [ ] Manual runs do not alter the configured interval.
|
||
|
||
### qBittorrent action and widget separation
|
||
|
||
- [ ] The scheduled action calls qBittorrent and appends exactly one sample per successful poll.
|
||
- [ ] The qBittorrent client cache is reused correctly per service.
|
||
- [ ] Timeouts become failed runs without blocking the scheduler indefinitely.
|
||
- [ ] The `speed` widget adapter reads samples but does not call qBittorrent.
|
||
- [ ] Multiple widgets or browser tabs do not multiply samples.
|
||
- [ ] A failed latest poll still returns last-known samples with stale status.
|
||
|
||
## Backend API tests
|
||
|
||
- [ ] Scheduler status requires normal API authentication.
|
||
- [ ] Status returns effective configuration, last attempt, last success, failure count, backoff, and stale state.
|
||
- [ ] Missing service returns 404 or the project’s established service error shape.
|
||
- [ ] Disabled service status is explicit and does not run an action.
|
||
- [ ] Run history is paginated and supports status/trigger filters.
|
||
- [ ] Run history is bounded by 30 days and 1,000 records per service/action.
|
||
- [ ] Manual-run endpoint returns a typed result and records the attempt.
|
||
- [ ] Samples endpoint accepts a display window and is read-only.
|
||
- [ ] Widget-data requests never cause a qBittorrent external call.
|
||
- [ ] Responses never expose usernames, passwords, API keys, headers, or raw payloads.
|
||
|
||
## Observability tests
|
||
|
||
- [ ] Run counters increment for success and failure.
|
||
- [ ] Duration metrics record completed attempts.
|
||
- [ ] Last-success gauges update only after successful sampling.
|
||
- [ ] Failure/stale gauges reset after recovery.
|
||
- [ ] Metric labels are bounded and contain no secrets or raw URLs.
|
||
- [ ] Structured logs include action/service/status context and sanitize errors.
|
||
|
||
## Frontend unit/component tests
|
||
|
||
- [ ] qBittorrent schedule fields render only for qBittorrent services.
|
||
- [ ] Invalid interval, retention, and cap values show validation feedback.
|
||
- [ ] Save preserves existing encrypted-secret behavior.
|
||
- [ ] Disabled polling clearly shows paused state.
|
||
- [ ] Status card renders healthy, running, backoff, stale, disabled, and never-run states.
|
||
- [ ] `Run now` shows pending state and disables duplicate clicks.
|
||
- [ ] Successful manual run refreshes status/history and clears backoff display.
|
||
- [ ] Failed manual run renders a safe error.
|
||
- [ ] Chart window selector requests samples without changing sampler settings.
|
||
- [ ] Stale warning appears while last-known speed data remains visible.
|
||
- [ ] Run history renders pagination, trigger, status, duration, timestamp, and error details.
|
||
- [ ] Empty history and no-data states are readable on mobile.
|
||
|
||
## Integration / lifespan tests
|
||
|
||
- [ ] Starting the FastAPI lifespan starts the scheduler exactly once.
|
||
- [ ] Repeated `start()` calls do not create duplicate workers.
|
||
- [ ] Lifespan shutdown stops the scheduler and does not leak a thread.
|
||
- [ ] A test app can override the scheduler/action registry cleanly.
|
||
- [ ] Existing mail queue and backup poller lifecycle behavior remains unchanged.
|
||
|
||
## Manual QA scenarios
|
||
|
||
### Headless collection
|
||
|
||
1. Configure an enabled qBittorrent service.
|
||
2. Start the backend without opening the frontend.
|
||
3. Wait for at least two intervals.
|
||
4. Query scheduler status and samples directly.
|
||
5. Confirm samples and successful run records exist.
|
||
|
||
### Duplicate prevention
|
||
|
||
1. Open the speed widget in multiple browser tabs.
|
||
2. Compare sample count growth to scheduler run count.
|
||
3. Confirm browser refreshes do not add samples or external qBittorrent calls.
|
||
|
||
### Outage and recovery
|
||
|
||
1. Make qBittorrent unreachable.
|
||
2. Confirm failures and increasing backoff appear in status/history.
|
||
3. Confirm last-known samples remain visible with a stale warning.
|
||
4. Restore qBittorrent.
|
||
5. Confirm the next successful scheduled or manual run clears backoff and stale state.
|
||
|
||
### Configuration reload
|
||
|
||
1. Change the interval and retention in the qBittorrent service editor.
|
||
2. Confirm the existing worker remains alive.
|
||
3. Confirm the new effective values appear after reconciliation.
|
||
4. Confirm pruning follows the new retention/cap.
|
||
|
||
### Service lifecycle
|
||
|
||
1. Disable a service and confirm no new runs are created while history remains.
|
||
2. Re-enable it and confirm polling resumes.
|
||
3. Delete it and confirm service-owned samples and run records are removed.
|
||
|
||
### Deployment constraint
|
||
|
||
1. Run the documented single-worker deployment.
|
||
2. Confirm one scheduler worker is active.
|
||
3. Verify the deployment documentation warns against multiple scheduler-capable replicas.
|
||
|
||
## Release gate
|
||
|
||
- [ ] Backend test suite passes.
|
||
- [ ] Frontend tests pass.
|
||
- [ ] Frontend lint passes with no new violations.
|
||
- [ ] Frontend build passes.
|
||
- [ ] No secret appears in logs, API responses, metrics, or scheduler run records.
|
||
- [ ] Project maps are patched for new files and validated.
|
||
- [ ] Requirements and runbook documentation match the shipped behavior.
|