Commit Graph

49 Commits

Author SHA1 Message Date
Developer 37533dd219 refactor: unify SSH machines as services 2026-07-14 20:58:46 +00:00
Developer a9488af0b4 feat: add typed qBittorrent scheduled polling 2026-07-14 15:22:34 +00:00
Developer e25240c2f3 feat(jellyseer): move jellyseerr_api_key to an encrypted secret (slice 2/3)
The Jellyseerr API key was stored as plaintext in the Jellyfin service config.
It is now a SecretField on the Jellyfin service, so it is encrypted at rest and
rendered as a masked secret input (the generic config editor stops exposing
it, and the secret editor picks it up automatically).

Migration (idempotent, runs in ensure_defaults):
- _migrate_jellyseerr_api_key_to_secret: for every Jellyfin service with a
  plaintext jellyseerr_api_key still in config, encrypt it ONCE into the
  secrets blob (direct UPDATE so existing encrypted secrets are preserved, not
  re-encrypted) and remove it from config.
- _migrate_jellyseerr_into_jellyfin: standalone-jellyseerr absorption now
  stores the key as a secret, and decrypts the Jellyfin api_key before handing
  it to upsert_service (fixes a pre-existing double-encrypt on that rare path).

The stats provider already reads jellyseerr_api_key from secrets-or-config, so
it works before, during, and after the migration.

Tests: absorbed-key lands in secrets (and existing api_key isn't corrupted);
new plaintext-config -> secret migration + idempotency. 401/401 backend pass.
2026-07-12 13:40:56 +00:00
Developer 3bc7ce5269 feat(per-instance-hook-scoping): scope observability + backup hooks by instance 2026-07-09 23:54:31 +00:00
Developer ec7ebd7013 chore: regenerate stale .pi-map.md / .pi-map.index.md project maps
Project map had drifted: claimed recharts unused (was used), ObservabilityPage.tsx
exists (refactored to service-tabs/), missed JellyfinNowPlayingWidget +
authentik/backups service types. Regenerate to reflect post-change reality:
new Qbittorrent/LineSeriesChart/Prometheus{Gauge,Mean}Widget files,
service_data.py/qbittorrent_store.py, canonical prometheus-charting +
service-storage specs, archived changes.
2026-07-09 14:05:56 +00:00
Developer 75c949ad25 feat(service-storage-harness): slice 4 — cascade-delete wiring + integration test
Wire ServiceDataHarness.cascade_delete into SettingsStore.delete_service
(best-effort try/except, logs on failure). Fix migration runner to also
catch 'no such table' on fresh DBs (ALTER TABLE before init_schema).
Integration test proves end-to-end cascade across both concerns (qBit
samples + media items) with multi-instance preservation.

Backend: 322 pytest pass, ruff clean.
2026-07-09 09:11:42 +00:00
Developer c87f398e37 feat(service-storage-harness): slice 3 — migrate MediaIndex onto harness (scoped replace_items, +service_id)
Register MediaIndex as a harness concern with ALTER TABLE migration to add
service_id column (idempotent). Scope replace_items by service_id (FIXES latent
global-clear bug where building for one Jellyfin wiped another's rows). Scope
query by service_id (empty-string = all rows, backward-compat). Thread
service_id through build_media_index + worker + query_media router. New
regression test proves scoped replace preserves other services' rows.

Backend: 321 pytest pass, ruff clean. Frontend: build green.
2026-07-09 09:00:31 +00:00
Developer e7bd0afdd1 feat(service-storage-harness): slice 1 — harness + qbit store + client + integration
ServiceDataHarness (services/service_data.py): lifecycle-only registry of
per-concern storage — DB provisioning, idempotent migrations (ALTER TABLE
duplicate-column-name caught per-statement), cascade_delete(service_id).
QbittorrentSampleStore: append/window/prune (MAX_SAMPLES=120) in
qbittorrent.db. QbittorrentClient: cookie-login Web API client (403 re-login,
/sync/maindata). Integration registered with 3 widget kinds (totals/active/
speed). Harness initialized in main.py lifespan.

Backend: 314 pytest pass (21 new), ruff clean.
2026-07-09 08:24:08 +00:00
Developer 67ca0fc3bc feat(prometheus-direct-charting): slice 3 — remove grafana + config rewrite + changelog
Remove the entire Grafana surface: integrations/grafana.py, GrafanaWidgetSource
(+ _fetch_chart, now redundant since prometheus chart exists), GrafanaLinkWidget,
LinksTab, get_grafana_status endpoint, useGrafanaStatus hook, GrafanaStatus type,
fetchGrafanaStatus client fn, registry/nav/tab entries (FE+BE). Rewrite
config.yaml thin-dashboard rule to match reality (recharts is sanctioned for
Prometheus-backed series). CHANGELOG migration note added.

Backend: 293 pytest pass, ruff clean. Frontend: build+lint green (0 errors).
SC-115/116 grep-clean (only prometheus_range.py migration comments + Dashboard.test
shortcut fixture remain — both spec-allowed).
2026-07-08 22:35:06 +00:00
Developer 691d78ff06 Dedup _resolve_service_record into shared service_resolution module
Extract the duplicated _resolve_service_record helper (identical in
routers/monitoring.py and routers/authentik_users.py) into a shared
services/service_resolution.py module. Both routers now import
resolve_service_record from the shared module.

The authentik router previously hardcoded service_type='authentik' in
its local copy; the shared helper takes service_type as a param (same
as monitoring's did).

Tests updated: test_api.py patches now target the correct module paths
(resolve_service_record on the monitoring module where it's imported,
build_service_record on the service_resolution module).

283 backend tests pass; ruff clean.
2026-07-06 12:25:17 +00:00
Developer 1e636fdbe2 Follow-ups: reference reorder, detach service_id, named-dashboard widgets
Three reusable-widget follow-up fixes:

1. Reference sort_order independently reorderable. Reordering a
   referenced widget now updates the widget_references.sort_order (per-
   dashboard), not the shared widget instance sort_order. New backend
   update_widget_reference method + PUT /api/widgets/references/{id}
   endpoint. Frontend moveInstance checks _ref_id to choose the right
   mutation (updateRef for references, saveWidget for owned).

2. Detach preserves service_id. detach_widget_reference now copies the
   original widget's service_id into the clone, so service-bound widgets
   (Grafana chart, Jellyfin activity) continue to render after detach.

3. Named dashboards support widget references. NamedDashboardPage
   fetches useWidgetReferences('named:<slug>') and renders them via
   WidgetInstanceCard alongside pinned links. 'Edit widgets' button
   opens WidgetConfigDialog with dashboardScope='named:<slug>'.

Also: removed useMemo on combinedWidgets in WidgetConfigDialog to fix
a react-hooks/preserve-manual-memoization lint error (the React Compiler
ESLint plugin couldn't verify the spread+sort memoization).

283 backend tests pass (+1 update_reference test); 128 frontend tests
pass; ruff clean; 0 lint errors.
2026-07-06 12:11:47 +00:00
Developer c36262d7b6 Reusable widgets: reference widgets across dashboards + detach to clone
Widgets configured on one dashboard (e.g., a Grafana service's Overview)
can now be live-referenced on other dashboards. Editing the widget config
updates it everywhere it's referenced. References can be detached into
independent clones.

Backend: new widget_references table (dashboard_scope, widget_id,
sort_order) with ON DELETE CASCADE. CRUD methods + 4 endpoints:
GET/POST /api/widgets/references, DELETE /api/widgets/references/{id},
POST /api/widgets/references/{id}/detach (clones the widget into a
standalone instance, then removes the reference).

Frontend: WidgetConfigDialog gains a dashboardScope prop. When set
(the main Dashboard passes 'main'), the dialog shows:
- Owned + referenced widgets in a combined list, with a link badge on
  references.
- 'Add existing widget' picker: searchable list of ALL widget instances
  not already on this dashboard. Click to create a reference.
- Detach button on references: clones the widget (service_id=NULL) and
  removes the reference.
- Delete on a reference removes the REFERENCE (not the original widget).

Dashboard renders referenced widgets alongside owned widgets.

Detaching a service-bound widget clones it with service_id=NULL — the
clone may need re-binding to a service to render correctly. Named
dashboards don't pass dashboardScope yet (pinned-links-only); when they
gain widget support, the backend already handles any scope string.

282 backend tests pass (+2 reference lifecycle); 127 frontend tests
pass; ruff/eslint/tsc/vite all green.
2026-07-06 11:34:48 +00:00
Developer 8d2e4c9bfd Service IA refinement: nav naming, instance tabs, config to Settings, configurable Overview
Four coupled changes to the services-as-hub IA:

1. Nav entries use service TYPE names (Jellyfin, SSH Tasks, Alertmanager,
   Grafana, Prometheus, Backups, Authentik) instead of conceptual names
   (Media, Files, Actions, Alerts, Users). ssh_tasks collapses to one
   entry ('SSH Tasks') instead of two. The content tabs inside each
   service page surface the concepts (Files, Actions).

2. Service page gains a two-level tab structure when multiple enabled
   instances of the same type exist: instance tabs on top ([Main Jellyfin]
   [Backup Jellyfin]), content tabs below ([Overview] [Media] [Requests]
   [Widgets]). Clicking an instance tab navigates to the sibling's route.
   Single instance: no instance tabs. Replaces the dropdown switcher.

3. Config tab (connection fields, secrets, enable/disable, delete) moves
   from the service page to Settings > Services tab. The service page
   becomes a PURE operational view (Overview + content tabs + Widgets) --
   no save/delete/config state. Settings gains a 4th tab 'Services' with
   ServiceConfigEditor per instance (schema-driven config fields, secrets
   with leave-blank-to-keep semantics, ConfirmDialog on delete).

4. Overview tab is now a configurable widget grid per service instance.
   Each instance manages its own set of widgets on its Overview. Backend
   widget list endpoints gain ?service_id= and ?scope= (dashboard|service)
   filter params; the main Dashboard uses scope=dashboard to exclude
   service-scoped widgets. The OverviewTab reuses WidgetInstanceCard +
   WidgetConfigDialog. Empty state CTA for instances with no widgets.

All service-tab stubs are replaced; stubs.tsx deleted.

272 backend tests pass (+1 widget filter); 121 frontend tests pass (+3
instance-tabs + OverviewTab); lint/build green both sides.
2026-06-26 22:25:46 +00:00
Developer 01527ae4f0 Rebase services-as-hub-ia onto mobile-responsive-parity
Combine both branches into a single coherent branch:
- Full mobile responsive parity (useIsMobile, MobileCardRow, SheetForm,
  .mobile-touch-target, mobile cards, SheetForm forms, 44px targets,
  dirty-state confirm, TablePagination, refetchIntervalInBackground).
- Full services-as-hub IA (data-driven nav, service-page tab skeleton,
  new service types, Authentik directory + messaging, named dashboards,
  legacy routes 404, Observability split, Jellyseerr absorbed).

Enhancement: service tabs now use mobile-parity primitives:
- MediaTab: MobileCardRow below md (title/size/HDR/library/year) +
  TablePagination; DataTable at md+ (desktop branch preserved).
- FilesTab: MobileCardRow below md (name/type/size/modified) +
  handleRowClick; DataTable at md+.
- ServicePage: SheetForm branch below md (open-on-mount, sticky header
  + save bar, cancel navigates back to /services, dirty-state guard).
- Dashboard: single-column + section anchors below md (from mobile-parity)
  + empty-state CTA (from services-hub).
- App.tsx: useIsMobile() replaces inline matchMedia (from mobile-parity)
  + data-driven useNavItems (from services-hub).
- Backup tables (BackupAlerts/Jobs/Runs) already have MobileCardRow from
  mobile-parity; JobsTab inherits mobile behavior through its sub-components.

Conflict resolutions:
- Backend: entirely from services-hub (mobile didn't touch it).
- Deleted pages (Media/FileBrowser/Actions/Users/UsersPage/Applications/
  ObservabilityPage/BackupsPage + hooks/useUsers + tests): kept deleted
  (services-hub deleted them; content moved into service tabs).
- New service-tabs/*: from services-hub, enhanced with mobile patterns.
- App.tsx: services-hub's data-driven nav + mobile-parity's useIsMobile.
- Dashboard.tsx: merged (services-hub CTA + mobile-parity sections/anchors).
- ServicePage.tsx: services-hub's tab skeleton + mobile-parity's SheetForm.
- Primitives (useIsMobile/mobile-card/sheet-form/etc.): from mobile-parity.

117 frontend tests pass (mobile-parity's 122 - 5 deleted page tests +
services-hub's new tab/dashboard tests); 271 backend tests pass; lint/
build green both sides.
2026-06-26 21:08:51 +00:00
Developer 7107815a5c refactor(settings): drop jellyfin machine service + dead machine path fields
Slice 1 of jellyfin-service-registry. Jellyfin is configured exclusively
via the service registry now; the machine-level media_root/path_prefix
fields were dead duplicates of the global config.

- services/settings_store.py: DEFAULT_SERVICES no longer includes
  "jellyfin" (now ["monitoring", "files"]). Removed machine-level
  media_root/path_prefix from _default_local_machine, _row_to_machine,
  _normalize_machine_payload, _seed_local_machine, get_machine_config,
  and upsert_machine. _default_local_machine no longer reads global
  config, so the get_settings import is dropped.
- routers/settings.py: removed media_root/path_prefix from
  MonitoringMachineInput (dead API input; store already ignored them).

The global config remote_media_root/path_prefix properties + path_utils.py
are unchanged (files.py and media_index still use them for Jellyfin->SSH
path resolution). ruff clean; 239 backend tests pass.
2026-06-24 14:38:47 +00:00
Developer 38b2de54ff chore: archive observability-service-registry, track pi-map artifacts
- Archive the completed observability-service-registry SDD change into
  openspec/changes/archive/ (delivered across 5 slices; only
  jellyfin-service-registry remains active).
- Stop ignoring .pi-map.md / .pi-map.index.md so the navigation maps are
  versioned alongside the code, and add the regenerated map pairs repo-wide.
2026-06-24 13:28:23 +00:00
Developer b200025daa refactor(observability): drop file-SD writer for http_sd_configs
Slice 4 of observability-service-registry. Removes the shared-file
Prometheus bridge; external Prometheus now consumes node-exporter targets
via http_sd_configs against GET /api/monitoring/prometheus-targets.

- services/targets.py: removed write_prometheus_targets() (the file
  writer) and its json/Path/get_settings imports; updated module docstring.
  build_node_exporter_targets() is unchanged and still powers the HTTP
  endpoint.
- main.py: removed the startup write_prometheus_targets call.
- routers/settings.py: removed the _write_prometheus_targets helper and
  its three post machine create/update/delete call sites + the now-unused
  targets import.
- config.py: removed the prometheus_file_sd_dir field.
- docker-compose.yml / docker-compose.dev.yml: removed the
  PROMETHEUS_FILE_SD_DIR backend env var.
- tests: removed TestWritePrometheusTargets + the write_prometheus_targets
  import in test_targets.py; rewrote the two TestSettingsMachines tests to
  assert machines appear/disappear from /api/monitoring/prometheus-targets
  (the surviving HTTP path) instead of the removed file-writer side effect.

ruff clean; 239 backend tests pass.
2026-06-24 08:37:20 +00:00
Developer 50eb76a10d feat(tasks): unify saved tasks on ssh_tasks services
- Add shared task_runner.run_saved_task helper used by routers/tasks.py and
  widgets/sources.py SshTaskWidgetSource.
- Saved tasks now target ssh_tasks service instances via default_service_id;
  the legacy default_machine_id and saved_task_runs are removed.
- Actions page lists ssh_tasks services for default and run-time selection.
- Update types, API client, hooks, tests, docs, and changelog.

Backend tests: 222 passed. Frontend lint/build/test: clean (71 passed).
2026-06-23 16:46:46 +00:00
Developer cfb9977532 chore: remove dead machine-level Jellyfin/Jellyseerr fields
Follow-up #1 to the service-registry change. Jellyfin/Jellyseerr now resolve
from the service registry, so the machine-level app fields are dead config.

- dependencies.py: drop dead _jellyseerr_client_for; simplify _resolve_machine
  to SSH-only.
- settings_store.py + routers/settings.py: remove jellyfin_*/jellyseerr_* from
  machine default config, get_machine_config, normalization, row mappers, and
  MachineInput.
- frontend types + Settings.tsx: drop the fields and the Jellyfin/Jellyseerr
  form sections + service options.
- Update frontend test fixtures.

Existing DB rows may still carry these keys in config_json; they are inert and
drop on the next machine save. Verification: backend ruff clean, pytest 222;
frontend lint 0 errors, build success, 70 tests.
2026-06-23 12:54:27 +00:00
Developer f6a86310cc style(widgets): apply formatter to widget rebind files 2026-06-22 18:22:18 +00:00
Developer 10fd4ead4a feat(widgets): rebind widgets to the service registry
PR 2 of 4 for the runtime service registry change.

- dashboard_widgets gains service_id + widget_kind columns (legacy
  addon_id/widget_type kept but unused).
- Source adapters take (service: ServiceRecord | None, widget_kind, config).
  SERVICE_ADAPTERS keyed by service_type; BUILTIN_ADAPTERS for backups/static.
- Backups and static stay as service-less built-ins (service_id nullable),
  exposed via GET /api/widgets/builtin.
- SSH task adapter resolves the task + instance, runs over SSH, and appends a
  service_task_runs history row on success/failure/timeout/error.
- Retire widgets/registry.py; widget metadata now comes from the integrations
  registry + widgets/builtin. Remove /api/widgets/types and /api/widgets/sources.
- Stop default widget seeding (fresh install = empty dashboard).
- Rewrite widget tests around the service-bound + built-in model (26 tests).

Backend-only breaking change; frontend is reconciled in Slice 3. Build/lint
stay green; pytest 222 passed.
2026-06-22 16:42:56 +00:00
Developer fd534a816b style(services): apply formatter to service registry files 2026-06-22 14:00:07 +00:00
Developer 8cdeadd6dd feat(services): backend service registry foundation (encryption, definitions, CRUD)
PR 1 of 4 for the runtime service registry change.

- Add Fernet encryption helper (services/secrets.py) with a required
  MANAGE_ENCRYPTION_KEY; validate it on startup.
- Add closed integrations/ registry with Pydantic config + widget-config
  definitions for grafana, prometheus, jellyfin, nextcloud, and ssh_tasks.
- Add services + service_task_runs tables and SettingsStore CRUD with
  cascade-delete (defensive until widgets carry service_id).
- Add /api/services/types and /api/services/instances CRUD (encrypted secrets,
  secrets_set flags only; never plaintext).
- Declare cryptography as a direct dependency.
- Require MANAGE_ENCRYPTION_KEY in compose + .env.example + README.
- Add 25 backend tests (registry, encryption, CRUD, cascade, task-run history).

Verification: ruff clean; pytest 225 passed; frontend lint/build green.
2026-06-22 12:56:03 +00:00
Developer 200d319fb0 feat(widgets): add backend CRUD, registry, and default seeding
Introduce a closed, compile-time widget registry and backend CRUD for
dashboard widget instances.

- Add dashboard_widgets SQLite table in SettingsStore with CRUD helpers and
  default seeding (Jellyfin + Backups) on first install.
- Add Pydantic models with credential-key and secret-value rejection.
- Add widgets router: /api/widgets/sources, /types, /instances CRUD.
- Call ensure_defaults() in app lifespan so fresh installs seed defaults.
- Add backend tests covering registry, CRUD, validation, and seeding.
- Include SDD artifacts: exploration, proposal, spec, design, tasks.
2026-06-19 20:07:47 +00:00
Developer 08a3b616f6 refactor(monitoring): decommission legacy SSH-scraping poller (slice 1)
The 2026-06-16/17 observability update externalised metrics to
Prometheus + node_exporter + Grafana, but the legacy Manage-side
SSH-scraping monitor was never removed. It duplicated the new stack,
ran SSH df on every machine every 300s, and fed nothing (its UI was
deleted in e2ad731). This slice decommissions the duplication.

Removed (backend):
- services/monitoring_poller.py (MonitoringPoller) — entire file
- services/monitoring_actions.py (disk_space, run_machine_operation,
  poll_machine_snapshot, build_machine_client) — entire file;
  run_machine_operation had only 2 callers (the poller + /disk), both gone
- tests/test_monitoring_actions.py
- endpoints: POST /api/monitoring/poller, GET /machines/{id}/actions,
  GET /disk (and the now-dead _resolve_machine helper)
- lifespan wiring (main.py), dependency wrapper (dependencies.py),
  poller.start()/kick() from machine save (routers/settings.py)
- SettingsStore: monitoring_machine_actions table CREATE + 2 indexes +
  record/list/prune_machine_actions methods; DROP TABLE IF EXISTS on
  startup cleans existing DBs (user-approved)
- config knobs: monitoring_poll_interval_seconds,
  monitoring_poll_initial_delay_seconds, monitoring_action_retention_days
- test_api.py: TestMonitoring._ensure_machine + test_disk

Kept (fits the new model): /machines, /prometheus-targets, /alerts,
/alertmanager-status, /alertmanager-webhook; the disk_usage JOB template
(manual on-demand, not monitoring); node_exporter_* machine fields
(they point Prometheus at the right host).

Gate: backend pytest 173 passed; ruff clean.
2026-06-17 20:48:56 +00:00
Developer e2ad731b5f feat(observability): add Prometheus/Grafana/Loki/Alertmanager/Alloy stack and remove legacy Monitoring UI 2026-06-16 13:44:35 +00:00
alex a748512d22 test: verify backup monitoring implementation 2026-05-11 21:57:31 +02:00
alex 836f733a2e feat: Add API key authentication for backup tool
- Add get_api_key() and require_api_key() to auth.py
- Add generic key-value settings storage to SettingsStore
- Add test_api_key_auth to verify the implementation
- Uses secrets.compare_digest for timing-safe comparison
- Auto-generates API key on first use and stores in settings DB
2026-05-11 21:25:46 +02:00
alex f912623677 feat: add backup alert background poller 2026-05-11 20:47:08 +02:00
alex cc8056d398 feat: add backup alert generation engine 2026-05-11 20:41:33 +02:00
alex dd375a7e16 feat: add backup monitoring CRUD operations 2026-05-11 20:38:13 +02:00
alex 83d05ffea4 feat: add backup monitoring database schema 2026-05-11 20:22:23 +02:00
alex 80d38ed86d fixes and improvements 2026-05-11 17:00:46 +02:00
alex 7e80733837 fixes and improvements 2026-05-07 23:09:08 +02:00
alex a5910b0fd6 fixes and improvements 2026-05-07 22:49:56 +02:00
alex c73cfa27c3 fixes and improvements 2026-05-07 20:36:37 +02:00
alex 84dc2bd2f6 Add SSH connection validation for settings 2026-05-07 18:31:49 +02:00
alex 2219a5730f Fix SSH key handling for monitoring 2026-05-07 17:39:59 +02:00
alex 6d8b08d30f fixes and improvements 2026-05-07 15:43:22 +02:00
alex efc9b247c7 fixes and improvements 2026-05-07 13:48:10 +02:00
alex bba23165ab fixes and improvements 2026-05-07 12:34:16 +02:00
alex 2ecca84892 fixes and improvements 2026-05-07 10:17:58 +02:00
alex 016e3255f5 Add missing frontend and backend files 2026-05-06 23:46:08 +02:00
alex 8f0a9650b0 refactor frontend and backend modules 2026-05-04 22:57:57 +02:00
alex 7950bc9f64 redesign 2026-05-04 17:24:41 +02:00
alex f1694ac99b bugfixes 2026-05-04 17:07:44 +02:00
alex 4cdb4b144c simplifications and refactorings 2026-05-04 16:59:06 +02:00
alex 4226628d5a Phase 2: Docker and OIDC auth 2026-05-04 13:50:53 +02:00
alex 51b10438a9 Restructure into backend/ and frontend/ subprojects
- backend/ uses proper Python src layout (src/media_library_viewer_api/)
  with pyproject.toml, hatchling build, and PYTHONPATH=src convention
- frontend/ is a Vite + React + TypeScript SPA
- archive/ preserves the original Streamlit prototype for reference
- Cleaned up root to only contain docs, license, and subproject dirs
- Updated README for the new dual-subproject architecture
2026-04-30 21:48:46 +02:00