Commit Graph

86 Commits

Author SHA1 Message Date
Developer 75c949ad25 feat(service-storage-harness): slice 4 — cascade-delete wiring + integration test
Wire ServiceDataHarness.cascade_delete into SettingsStore.delete_service
(best-effort try/except, logs on failure). Fix migration runner to also
catch 'no such table' on fresh DBs (ALTER TABLE before init_schema).
Integration test proves end-to-end cascade across both concerns (qBit
samples + media items) with multi-instance preservation.

Backend: 322 pytest pass, ruff clean.
2026-07-09 09:11:42 +00:00
Developer c87f398e37 feat(service-storage-harness): slice 3 — migrate MediaIndex onto harness (scoped replace_items, +service_id)
Register MediaIndex as a harness concern with ALTER TABLE migration to add
service_id column (idempotent). Scope replace_items by service_id (FIXES latent
global-clear bug where building for one Jellyfin wiped another's rows). Scope
query by service_id (empty-string = all rows, backward-compat). Thread
service_id through build_media_index + worker + query_media router. New
regression test proves scoped replace preserves other services' rows.

Backend: 321 pytest pass, ruff clean. Frontend: build green.
2026-07-09 09:00:31 +00:00
Developer 1fb12b8a0a feat(service-storage-harness): slice 2 — qbit widgets + LineSeriesChart extract 2026-07-09 08:49:20 +00:00
Developer e7bd0afdd1 feat(service-storage-harness): slice 1 — harness + qbit store + client + integration
ServiceDataHarness (services/service_data.py): lifecycle-only registry of
per-concern storage — DB provisioning, idempotent migrations (ALTER TABLE
duplicate-column-name caught per-statement), cascade_delete(service_id).
QbittorrentSampleStore: append/window/prune (MAX_SAMPLES=120) in
qbittorrent.db. QbittorrentClient: cookie-login Web API client (403 re-login,
/sync/maindata). Integration registered with 3 widget kinds (totals/active/
speed). Harness initialized in main.py lifespan.

Backend: 314 pytest pass (21 new), ruff clean.
2026-07-09 08:24:08 +00:00
Developer 67ca0fc3bc feat(prometheus-direct-charting): slice 3 — remove grafana + config rewrite + changelog
Remove the entire Grafana surface: integrations/grafana.py, GrafanaWidgetSource
(+ _fetch_chart, now redundant since prometheus chart exists), GrafanaLinkWidget,
LinksTab, get_grafana_status endpoint, useGrafanaStatus hook, GrafanaStatus type,
fetchGrafanaStatus client fn, registry/nav/tab entries (FE+BE). Rewrite
config.yaml thin-dashboard rule to match reality (recharts is sanctioned for
Prometheus-backed series). CHANGELOG migration note added.

Backend: 293 pytest pass, ruff clean. Frontend: build+lint green (0 errors).
SC-115/116 grep-clean (only prometheus_range.py migration comments + Dashboard.test
shortcut fixture remain — both spec-allowed).
2026-07-08 22:35:06 +00:00
Developer 65bae95e3c feat(prometheus-direct-charting): slice 2 — gauge + mean widgets
Add gauge widget (recharts RadialBarChart with configurable threshold
bands, scalar-only per SC-111) and mean widget (client-side average over
range-query window, scalar-only per SC-114). Extract shared _instant_query
helper from the metric path; _fetch_gauge and _fetch_mean dispatch in
PrometheusWidgetSource.fetch(). Both new widget kinds declared in
integrations/prometheus.py and frontend registry.

Backend: 305 pytest pass, ruff clean. Frontend: 136 vitest pass, build+lint green.
2026-07-08 22:10:11 +00:00
Developer 5dad98231f feat(prometheus-direct-charting): slice 1 — prom range query + chart rebrand
Add PrometheusWidgetSource._fetch_chart hitting /api/v1/query_range directly
(SC-101..104). New shared helpers in widgets/prometheus_range.py:
step_for_window (window preset -> step, ~200pts) and normalize_prometheus_matrix
(extracted label/dedup rule, retargeted at Prom matrix, robust to malformed
data). Chart widget kind moved grafana->prometheus in both registries;
GrafanaChartWidget renamed -> PrometheusChartWidget (git mv, recharts body
preserved). Grafana binding/service untouched (removed in slice 3).

Backend: 298 pytest pass, ruff clean. Frontend: 128 vitest pass, build+lint green.
2026-07-08 21:55:02 +00:00
Developer 67c51f9fc0 Fix: user_id validation causes read timeout on slow Jellyfin
The previous fix called client.users() unconditionally to validate the
user_id, adding an extra HTTP round-trip before the build. On a slow
Jellyfin connection this burned through the 10s timeout before the
actual libraries() call.

Restructured to try libraries(user_id) directly first. Only when that
fails does the worker resolve the username via the users API. So:
- Correctly configured user_id (internal hash): zero extra round-trips.
- Username like 'admin': libraries() fails → users() resolves it → retry.

283 backend tests pass; ruff clean.
2026-07-06 17:35:19 +00:00
Developer 7497469d5e Fix: user_id 'admin' rejected by Jellyfin API (400)
The Jellyfin service config's user_id field accepts either the internal
Jellyfin user ID (a long hash) or a username (e.g. 'admin'). The worker
passed the raw value directly to client.libraries(user_id), but Jellyfin's
API rejects usernames with a 400.

Now validates the configured user_id against the Jellyfin users API:
1. If it matches a user's Id (internal hash), use it directly.
2. If it matches a user's Name (username like 'admin'), resolve the Id.
3. If no match, fall back to the first user and log a warning.

283 backend tests pass; ruff clean.
2026-07-06 15:45:51 +00:00
Developer a3888026ab Fix: media worker crashes silently before writing error state
The worker called _resolve_jellyfin() and client.libraries() OUTSIDE the
try/except block. If Jellyfin was unreachable, the worker crashed with
an unhandled exception and NO error state was written to the DB — the
status stayed 'queued' forever with zero feedback.

Moved _resolve_jellyfin + client.libraries INSIDE the try block, and
created final_index + staging_index BEFORE the try so the except handler
can write the error state. Now any failure (connection refused, timeout,
missing config) is written to the DB as build_error and the frontend
shows it inline.

283 backend tests pass; ruff clean.
2026-07-06 15:36:07 +00:00
Developer 787f46700f Fix: media index build never starts (blocking Jellyfin dependency)
The build endpoint had Depends(get_jellyfin_client) and Depends(get_user_id)
which executed BEFORE the function body. If Jellyfin was unreachable, these
raised HTTPException(503), the function never ran, and the worker was never
started. The frontend mutation had no onError handler, so the failure was
completely silent — the button briefly showed 'Building...' then reverted
to 'Build index' with zero feedback.

Backend fix: removed the Jellyfin dependencies from post_build_index.
The worker subprocess resolves its own Jellyfin connection via
_resolve_jellyfin(service_id) — the endpoint just needs to start the
worker process. The libraries count starts at 0 and gets updated by
the worker once it connects.

Frontend fix: added onError to useBuildIndex that invalidates the status
query (so the UI reflects the non-building state). MediaTab now displays
the build error inline: 'Build failed: <message>' next to the button.

283 backend tests pass (updated build test for new no-dependency flow);
128 frontend tests pass; ruff/eslint clean.
2026-07-06 15:25:48 +00:00
Developer d8c0a37210 Fix: settings master/detail, widget kind filter, reorder, media worker
Four fixes:

1. Settings > Services tab: master/detail layout. Replaced the vertical
   stack of ServiceConfigEditor cards with a SelectionRailCard (list on
   left) + SectionCard (details on right) — same pattern as Machines
   and SSH Keys tabs. Click a service in the rail to edit it.

2. Service Overview widget restriction. When adding widgets on a
   service's Overview, the dialog now only shows built-in widgets +
   widgets for THAT service type (not all services). Dashboard/named
   dashboards (no serviceId) still see all.

3. Reorder buttons fixed. The swap-sort_order approach was a no-op when
   both items had sort_order=0 (the default). Now moveInstance
   renumbers ALL items by their new index position (i * 10) after the
   swap, guaranteeing values change. References use updateRef, owned
   widgets use saveWidget, both sequential.

4. Media index worker resolution. The subprocess worker called
   get_jellyfin_client/get_user_id (FastAPI request dependencies) which
   don't work outside request context. Now accepts a service_id
   parameter (passed from post_build_index) and resolves the Jellyfin
   instance directly from the settings store via _resolve_jellyfin.
   Raises RuntimeError (not HTTPException) on failure.

283 backend tests pass; 128 frontend tests pass; ruff/eslint clean.
2026-07-06 14:03:05 +00:00
Developer 691d78ff06 Dedup _resolve_service_record into shared service_resolution module
Extract the duplicated _resolve_service_record helper (identical in
routers/monitoring.py and routers/authentik_users.py) into a shared
services/service_resolution.py module. Both routers now import
resolve_service_record from the shared module.

The authentik router previously hardcoded service_type='authentik' in
its local copy; the shared helper takes service_type as a param (same
as monitoring's did).

Tests updated: test_api.py patches now target the correct module paths
(resolve_service_record on the monitoring module where it's imported,
build_service_record on the service_resolution module).

283 backend tests pass; ruff clean.
2026-07-06 12:25:17 +00:00
Developer 1e636fdbe2 Follow-ups: reference reorder, detach service_id, named-dashboard widgets
Three reusable-widget follow-up fixes:

1. Reference sort_order independently reorderable. Reordering a
   referenced widget now updates the widget_references.sort_order (per-
   dashboard), not the shared widget instance sort_order. New backend
   update_widget_reference method + PUT /api/widgets/references/{id}
   endpoint. Frontend moveInstance checks _ref_id to choose the right
   mutation (updateRef for references, saveWidget for owned).

2. Detach preserves service_id. detach_widget_reference now copies the
   original widget's service_id into the clone, so service-bound widgets
   (Grafana chart, Jellyfin activity) continue to render after detach.

3. Named dashboards support widget references. NamedDashboardPage
   fetches useWidgetReferences('named:<slug>') and renders them via
   WidgetInstanceCard alongside pinned links. 'Edit widgets' button
   opens WidgetConfigDialog with dashboardScope='named:<slug>'.

Also: removed useMemo on combinedWidgets in WidgetConfigDialog to fix
a react-hooks/preserve-manual-memoization lint error (the React Compiler
ESLint plugin couldn't verify the spread+sort memoization).

283 backend tests pass (+1 update_reference test); 128 frontend tests
pass; ruff clean; 0 lint errors.
2026-07-06 12:11:47 +00:00
Developer c36262d7b6 Reusable widgets: reference widgets across dashboards + detach to clone
Widgets configured on one dashboard (e.g., a Grafana service's Overview)
can now be live-referenced on other dashboards. Editing the widget config
updates it everywhere it's referenced. References can be detached into
independent clones.

Backend: new widget_references table (dashboard_scope, widget_id,
sort_order) with ON DELETE CASCADE. CRUD methods + 4 endpoints:
GET/POST /api/widgets/references, DELETE /api/widgets/references/{id},
POST /api/widgets/references/{id}/detach (clones the widget into a
standalone instance, then removes the reference).

Frontend: WidgetConfigDialog gains a dashboardScope prop. When set
(the main Dashboard passes 'main'), the dialog shows:
- Owned + referenced widgets in a combined list, with a link badge on
  references.
- 'Add existing widget' picker: searchable list of ALL widget instances
  not already on this dashboard. Click to create a reference.
- Detach button on references: clones the widget (service_id=NULL) and
  removes the reference.
- Delete on a reference removes the REFERENCE (not the original widget).

Dashboard renders referenced widgets alongside owned widgets.

Detaching a service-bound widget clones it with service_id=NULL — the
clone may need re-binding to a service to render correctly. Named
dashboards don't pass dashboardScope yet (pinned-links-only); when they
gain widget support, the backend already handles any scope string.

282 backend tests pass (+2 reference lifecycle); 127 frontend tests
pass; ruff/eslint/tsc/vite all green.
2026-07-06 11:34:48 +00:00
Developer 94bf830955 Fix invisible chart lines + extract Prometheus labels for multi-series
Two fixes for the Grafana chart widget:

1. Invisible lines: the CHART_COLORS used 'hsl(var(--chart-1))' but the
   CSS variable is named '--color-chart-1' and already contains a hex
   color (#4f8cff). The hsl() wrapper produced invalid CSS, making
   every stroke invisible. Fixed to var(--color-chart-1).

2. Multiple series collision: the backend labeled all Prometheus series
   with the value field name (often just 'Value'), so multiple time
   series collided on the same recharts dataKey and overwrote each
   other. Now extracts meaningful labels from the Grafana frame metadata:
   prefers displayName, then Prometheus metric labels (e.g.
   'instance=server1:9100 mode=iowait'), then falls back to the field
   name. Duplicate labels get a numeric suffix for uniqueness.

Multi-series queries now render correctly: each Prometheus time series
gets its own colored line with a unique label in the legend/tooltip.

280 backend tests pass (+1 labels test); 127 frontend tests pass; ruff/
eslint clean.
2026-07-06 10:59:16 +00:00
Developer 447775048c Replace Grafana iframe panel with server-side chart widget
The iframe-based 'panel' widget didn't work: the browser couldn't
authenticate against the OIDC-protected Grafana (Authentik), and
iframes can't carry Bearer tokens or share cross-origin session
cookies. Result: blank iframe or login redirect.

Replace it with a 'chart' widget that queries Grafana's datasource
API server-side:

Backend (GrafanaWidgetSource): POSTs to /api/ds/query with the stored
api_key (which bypasses OIDC), using the widget's configured PromQL
query, datasource_uid, time range, and resolution. Normalizes Grafana's
frame-based response into a simple {series: [{label, points: [{t, v}]}]}
shape. The api_key is never exposed to the browser.

Frontend (GrafanaChartWidget): renders the series data as a recharts
LineChart with dark-mode-aware colors (Tailwind --chart-* tokens),
responsive container, custom tooltip, and per-series lines. Loading
skeleton, error Alert, and empty state. recharts ^3.9.2 added.

The 'link' widget kind (deep-link URL) is unchanged. The 'panel' kind
and GrafanaPanelWidget are fully removed.

Backend: 279 tests pass (+1 net: -2 panel + 3 chart). Frontend: 127
tests pass (net 0: -3 panel + 3 chart). Lint/build green both sides.
2026-07-06 10:19:57 +00:00
Developer b877a32ad8 Add Jellyfin Now Playing + Grafana Panel embed widgets
Two new additive widget kinds:

Jellyfin 'now_playing': like the existing 'activity' widget but filters
to only sessions with active playback (NowPlayingItem present + not
paused). Shows who's actually watching right now. The 'activity' kind
is unchanged (shows all sessions including idle).

Grafana 'panel': embeds a single Grafana panel directly in the app via
an iframe, using Grafana's /d-solo/ endpoint (renders one panel without
dashboard chrome, kiosk=tv). Configurable dashboard_uid, panel_id, and
time range (from/to, defaults now-1h/now). Includes a fallback 'Open in
Grafana' link for when embedding is blocked by X-Frame-Options/CSP. The
'link' kind is unchanged (still builds a deep-link URL).

Backend: new widget configs + definitions on jellyfin/grafana; source
adapter logic (session filter for now_playing; d-solo embed URL for
panel); 6 new tests.

Frontend: JellyfinNowPlayingWidget + GrafanaPanelWidget components;
registry bindings; 6 new tests.

278 backend tests pass (+6); 127 frontend tests pass (+6); lint/build
green both sides.
2026-06-28 15:26:43 +00:00
Developer 8d2e4c9bfd Service IA refinement: nav naming, instance tabs, config to Settings, configurable Overview
Four coupled changes to the services-as-hub IA:

1. Nav entries use service TYPE names (Jellyfin, SSH Tasks, Alertmanager,
   Grafana, Prometheus, Backups, Authentik) instead of conceptual names
   (Media, Files, Actions, Alerts, Users). ssh_tasks collapses to one
   entry ('SSH Tasks') instead of two. The content tabs inside each
   service page surface the concepts (Files, Actions).

2. Service page gains a two-level tab structure when multiple enabled
   instances of the same type exist: instance tabs on top ([Main Jellyfin]
   [Backup Jellyfin]), content tabs below ([Overview] [Media] [Requests]
   [Widgets]). Clicking an instance tab navigates to the sibling's route.
   Single instance: no instance tabs. Replaces the dropdown switcher.

3. Config tab (connection fields, secrets, enable/disable, delete) moves
   from the service page to Settings > Services tab. The service page
   becomes a PURE operational view (Overview + content tabs + Widgets) --
   no save/delete/config state. Settings gains a 4th tab 'Services' with
   ServiceConfigEditor per instance (schema-driven config fields, secrets
   with leave-blank-to-keep semantics, ConfirmDialog on delete).

4. Overview tab is now a configurable widget grid per service instance.
   Each instance manages its own set of widgets on its Overview. Backend
   widget list endpoints gain ?service_id= and ?scope= (dashboard|service)
   filter params; the main Dashboard uses scope=dashboard to exclude
   service-scoped widgets. The OverviewTab reuses WidgetInstanceCard +
   WidgetConfigDialog. Empty state CTA for instances with no widgets.

All service-tab stubs are replaced; stubs.tsx deleted.

272 backend tests pass (+1 widget filter); 121 frontend tests pass (+3
instance-tabs + OverviewTab); lint/build green both sides.
2026-06-26 22:25:46 +00:00
Developer 01527ae4f0 Rebase services-as-hub-ia onto mobile-responsive-parity
Combine both branches into a single coherent branch:
- Full mobile responsive parity (useIsMobile, MobileCardRow, SheetForm,
  .mobile-touch-target, mobile cards, SheetForm forms, 44px targets,
  dirty-state confirm, TablePagination, refetchIntervalInBackground).
- Full services-as-hub IA (data-driven nav, service-page tab skeleton,
  new service types, Authentik directory + messaging, named dashboards,
  legacy routes 404, Observability split, Jellyseerr absorbed).

Enhancement: service tabs now use mobile-parity primitives:
- MediaTab: MobileCardRow below md (title/size/HDR/library/year) +
  TablePagination; DataTable at md+ (desktop branch preserved).
- FilesTab: MobileCardRow below md (name/type/size/modified) +
  handleRowClick; DataTable at md+.
- ServicePage: SheetForm branch below md (open-on-mount, sticky header
  + save bar, cancel navigates back to /services, dirty-state guard).
- Dashboard: single-column + section anchors below md (from mobile-parity)
  + empty-state CTA (from services-hub).
- App.tsx: useIsMobile() replaces inline matchMedia (from mobile-parity)
  + data-driven useNavItems (from services-hub).
- Backup tables (BackupAlerts/Jobs/Runs) already have MobileCardRow from
  mobile-parity; JobsTab inherits mobile behavior through its sub-components.

Conflict resolutions:
- Backend: entirely from services-hub (mobile didn't touch it).
- Deleted pages (Media/FileBrowser/Actions/Users/UsersPage/Applications/
  ObservabilityPage/BackupsPage + hooks/useUsers + tests): kept deleted
  (services-hub deleted them; content moved into service tabs).
- New service-tabs/*: from services-hub, enhanced with mobile patterns.
- App.tsx: services-hub's data-driven nav + mobile-parity's useIsMobile.
- Dashboard.tsx: merged (services-hub CTA + mobile-parity sections/anchors).
- ServicePage.tsx: services-hub's tab skeleton + mobile-parity's SheetForm.
- Primitives (useIsMobile/mobile-card/sheet-form/etc.): from mobile-parity.

117 frontend tests pass (mobile-parity's 122 - 5 deleted page tests +
services-hub's new tab/dashboard tests); 271 backend tests pass; lint/
build green both sides.
2026-06-26 21:08:51 +00:00
Developer 3d331e4c72 Make service connection config editable on service page
The service detail page showed non-secret connection config (base_url,
user_id, username, timeout_seconds) as read-only. Render schema-driven
editable inputs (reusing the create-dialog pattern) with a draftConfig
state hydrated from the instance, and unify the save button to persist
both config and secrets. Number fields render as type=number; the base_url
schema description surfaces as helper text.
2026-06-26 09:52:43 +00:00
Developer eebc86a52b Enforce http(s) schema on service base_url fields
Add a shared ServiceBaseUrl type (BeforeValidator + Field description) in
integrations/base.py and apply it to base_url across all six service configs
(grafana, prometheus, alertmanager, jellyfin, jellyseerr, nextcloud). Missing
http:// or https:// schema now fails fast with a clear 422 instead of breaking
HTTP clients silently. Tests cover reject/accept cases; REQUIREMENTS updated.
2026-06-26 09:52:20 +00:00
Developer 648320abfd chore(project-map): refresh .pi-map role/arch summaries
Regenerate project map artifacts across backend, docs, openspec, and
root to refresh role descriptions and architectural notes after recent
service-registry and observability changes.

Co-authored-by: el Gentleman <gentleman@pi.local>
2026-06-26 09:10:19 +00:00
Developer 7d252489de fix(api): return 503 instead of 500 when Jellyfin/SSH not configured
On a fresh deploy with no Jellyfin service configured yet,
get_jellyfin_client (and get_user_id / get_ssh_client) raised a plain
RuntimeError, which bubbled up as a 500 traceback on every
Jellyfin-dependent route (dashboard counts/libraries/activity, media,
users). Convert those RuntimeErrors to HTTPException(503) with a clear
detail message so FastAPI returns a clean 503 JSON response instead of
a 500, and the frontend can render a not-configured state.

- dependencies.py: get_jellyfin_client (no service / missing creds),
  get_user_id (no users discovered), and get_ssh_client (no SSH machine
  + no legacy key path) now raise HTTPException(503, detail=...).
- tests/test_api.py: added
  TestDashboard.test_jellyfin_endpoints_return_503_when_not_configured
  covering /api/dashboard/counts and /activity.

ruff clean; 240 backend tests pass.
2026-06-26 08:39:39 +00:00
Developer 7107815a5c refactor(settings): drop jellyfin machine service + dead machine path fields
Slice 1 of jellyfin-service-registry. Jellyfin is configured exclusively
via the service registry now; the machine-level media_root/path_prefix
fields were dead duplicates of the global config.

- services/settings_store.py: DEFAULT_SERVICES no longer includes
  "jellyfin" (now ["monitoring", "files"]). Removed machine-level
  media_root/path_prefix from _default_local_machine, _row_to_machine,
  _normalize_machine_payload, _seed_local_machine, get_machine_config,
  and upsert_machine. _default_local_machine no longer reads global
  config, so the get_settings import is dropped.
- routers/settings.py: removed media_root/path_prefix from
  MonitoringMachineInput (dead API input; store already ignored them).

The global config remote_media_root/path_prefix properties + path_utils.py
are unchanged (files.py and media_index still use them for Jellyfin->SSH
path resolution). ruff clean; 239 backend tests pass.
2026-06-24 14:38:47 +00:00
Developer 38b2de54ff chore: archive observability-service-registry, track pi-map artifacts
- Archive the completed observability-service-registry SDD change into
  openspec/changes/archive/ (delivered across 5 slices; only
  jellyfin-service-registry remains active).
- Stop ignoring .pi-map.md / .pi-map.index.md so the navigation maps are
  versioned alongside the code, and add the regenerated map pairs repo-wide.
2026-06-24 13:28:23 +00:00
Developer b1a66a1ab7 chore(observability): remove remaining observability env vars, docs
Slice 5 (final) of observability-service-registry. Completes the move to
service-registry-only observability config: no observability service env
vars remain.

- config.py: removed alertmanager_url + alertmanager_webhook_url fields.
- docker-compose.yml / docker-compose.dev.yml: removed ALERTMANAGER_URL,
  ALERTMANAGER_WEBHOOK_URL (backend env), and VITE_GRAFANA_URL,
  VITE_PROMETHEUS_URL (frontend build args / dev env).
- frontend/Dockerfile: removed the VITE_GRAFANA_URL / VITE_PROMETHEUS_URL
  ARG, build-stage ENV, and dev-stage ENV lines.
- docs: REQUIREMENTS decision-log entry; CHANGELOG Added/Changed/BREAKING
  for the observability service registry; backend/README monitoring
  section (Observability page, services page config, http_sd_configs,
  new health endpoints, log-only webhook).

The only observability env var remaining is PROMETHEUS_ENABLED (Manage's
own /metrics toggle). Grep-gated: no live references to the removed
vars/fields in backend src, frontend src, compose, or Dockerfile.

ruff clean; 239 backend tests pass; frontend 0 lint errors, build clean,
72 tests pass.

.env.example is assistant-edit-blocked; user follow-up noted in the SDD
tasks: drop the removed vars there too.
2026-06-24 08:49:53 +00:00
Developer b200025daa refactor(observability): drop file-SD writer for http_sd_configs
Slice 4 of observability-service-registry. Removes the shared-file
Prometheus bridge; external Prometheus now consumes node-exporter targets
via http_sd_configs against GET /api/monitoring/prometheus-targets.

- services/targets.py: removed write_prometheus_targets() (the file
  writer) and its json/Path/get_settings imports; updated module docstring.
  build_node_exporter_targets() is unchanged and still powers the HTTP
  endpoint.
- main.py: removed the startup write_prometheus_targets call.
- routers/settings.py: removed the _write_prometheus_targets helper and
  its three post machine create/update/delete call sites + the now-unused
  targets import.
- config.py: removed the prometheus_file_sd_dir field.
- docker-compose.yml / docker-compose.dev.yml: removed the
  PROMETHEUS_FILE_SD_DIR backend env var.
- tests: removed TestWritePrometheusTargets + the write_prometheus_targets
  import in test_targets.py; rewrote the two TestSettingsMachines tests to
  assert machines appear/disappear from /api/monitoring/prometheus-targets
  (the surviving HTTP path) instead of the removed file-writer side effect.

ruff clean; 239 backend tests pass.
2026-06-24 08:37:20 +00:00
Developer 0c5698c903 feat(observability): service discovery, health cards, alertmanager widget
Slice 3 of observability-service-registry (frontend). The Observability
page discovers Grafana from the service registry instead of env vars,
adds Grafana + Prometheus health cards, and ships an alertmanager
active_alerts dashboard widget.

- types: added GrafanaStatus + PrometheusStatus; added optional
  service_id/error to AlertmanagerStatus.
- api/client.ts + hooks/useObservability.ts: fetchGrafanaStatus,
  fetchPrometheusStatus, useGrafanaStatus, usePrometheusStatus.
- widgets/AlertmanagerAlertsWidget.tsx (new): presentational widget
  consuming the active_alerts summary shape (total/by_severity/alerts);
  exported from widgets/index.ts.
- integrations/registry.ts: alertmanager binding (active_alerts kind,
  30s refresh, optional severity_filter); registry.test.ts updated to
  6 service types incl alertmanager + a resolve test.
- components/ObservabilityPage.tsx: removed
  import.meta.env.VITE_GRAFANA_URL; derive GRAFANA_BASE_URL from the
  first enabled grafana service via useServiceInstances("grafana");
  added Grafana + Prometheus HealthCards (up/not-configured/unreachable)
  with QueryError retry blocks; machine dashboard shows a "No Grafana
  service configured" empty-state linking to /services when none is set.

npm run build (tsc -b + vite) clean; 0 lint errors; 72 frontend tests
pass. Reviewed fresh-context (read-only): no blockers.
2026-06-24 08:23:23 +00:00
Developer 14771ae990 feat(observability): resolve services from registry, add health endpoints
Slice 2 of observability-service-registry. The monitoring router resolves
observability components from the service registry instead of env vars.

- routers/monitoring.py: removed _alertmanager_client/_webhook_client env
  readers + the get_settings import. Added _resolve_service_record(store,
  service_type, service_id?) -> ServiceRecord|None (requested instance with
  type+enabled checks, else first enabled instance), plus _base_url/_timeout/
  _auth_headers (Bearer from api_key)/_status_response helpers.
- /alerts + /alertmanager-status now take service_id? + Depends(store),
  resolve an alertmanager service, return graceful not-configured/
  unreachable payloads including service_id/name; status down-branches now
  include peers:[] + error (fixes prior type drift).
- NEW /grafana-status (probes /api/health) and /prometheus-status (probes
  /-/healthy then /api/v1/status/buildinfo) returning
  {up,version,service_id,name,error}.
- Webhook receiver is now log-only (dropped the outbound
  ALERTMANAGER_WEBHOOK_URL forward).
- tests: rewrote TestAlertmanager + TestAlertmanagerWebhook to mock
  _resolve_service_record/requests.get (not-configured via empty registry);
  added TestGrafanaStatus/TestPrometheusStatus and a TestResolveServiceRecord
  unit class covering service_id match/type-mismatch/disabled and first-
  enabled/none-enabled paths.

Orphaned config fields alertmanager_url/alertmanager_webhook_url and the
env-var removal land in Slice 5. ruff clean; 240 backend tests pass.

Reviewed fresh-context (read-only): no blockers.
2026-06-24 07:53:25 +00:00
Developer 7d49df3e7d feat(observability): add alertmanager service type and widget
Slice 1 of observability-service-registry. Alertmanager becomes a
first-class service-registry type, mirroring grafana/prometheus.

- integrations/alertmanager.py (new): AlertmanagerConfig
  (base_url, timeout_seconds), AlertmanagerAlertsWidgetConfig (optional
  severity_filter), shared summarize_alerts() helper, and DEFINITION
  (service_type "alertmanager", secret api_key, widget "active_alerts").
- integrations/registry.py: register ALERTMANAGER (7 types now).
- widgets/sources.py: AlertmanagerWidgetSource fetches
  {base_url}/api/v1/alerts, sends optional Bearer token from the api_key
  secret, applies optional severity_filter, and summarizes via the shared
  helper; registered in SERVICE_ADAPTERS.
- routers/monitoring.py: _summary_from_alerts delegates to the shared
  summarize_alerts (behavior unchanged).
- tests: registry now 7 types; /api/services/types lists alertmanager;
  4 new adapter tests (summarize, severity filter, bearer token, missing
  service).

Backend-only slice; the frontend active_alerts widget binding lands in a
later slice. ruff clean; 228 backend tests pass.

Reviewed fresh-context (read-only): no blockers.
2026-06-23 22:25:50 +00:00
Developer d4f95b64d4 chore(observability): externalize stack from root compose files
Manage now connects to existing Grafana/Prometheus/Alertmanager instances
and never deploys its own stack.

- docker-compose.yml / docker-compose.dev.yml: removed prometheus, loki,
  alloy, grafana, alertmanager, node-exporter services, the monitoring
  network, and observability named volumes; they now ship only backend +
  frontend. Dev frontend now joins the web network so the Vite dev proxy
  can reach the backend.
- backend: alertmanager_url default is now empty; /api/monitoring/alerts
  and /alertmanager-status return graceful "not configured" responses
  when ALERTMANAGER_URL is unset. Added not-configured tests.
- docker-compose.observability.yml: kept as the optional standalone
  example; header clarifies Manage does not deploy it.
- Removed orphaned combined monitoring/prometheus/prometheus.yml
  (standalone stack uses prometheus.standalone.yml).
- Docs (README, REQUIREMENTS decision log, monitoring-logging-design,
  observability-runbooks, context.md, MIGRATION_PLAN, frontend/README,
  CHANGELOG) updated to the connect-to-existing model.

VITE_GRAFANA_URL / VITE_PROMETHEUS_URL remain as optional frontend
deep-link overrides. .env.example still needs a manual update (safety
policy blocks assistant edits): set ALERTMANAGER_URL empty/optional and
move standalone-only vars out of the root file.
2026-06-23 21:20:07 +00:00
Developer 50eb76a10d feat(tasks): unify saved tasks on ssh_tasks services
- Add shared task_runner.run_saved_task helper used by routers/tasks.py and
  widgets/sources.py SshTaskWidgetSource.
- Saved tasks now target ssh_tasks service instances via default_service_id;
  the legacy default_machine_id and saved_task_runs are removed.
- Actions page lists ssh_tasks services for default and run-time selection.
- Update types, API client, hooks, tests, docs, and changelog.

Backend tests: 222 passed. Frontend lint/build/test: clean (71 passed).
2026-06-23 16:46:46 +00:00
Developer cfb9977532 chore: remove dead machine-level Jellyfin/Jellyseerr fields
Follow-up #1 to the service-registry change. Jellyfin/Jellyseerr now resolve
from the service registry, so the machine-level app fields are dead config.

- dependencies.py: drop dead _jellyseerr_client_for; simplify _resolve_machine
  to SSH-only.
- settings_store.py + routers/settings.py: remove jellyfin_*/jellyseerr_* from
  machine default config, get_machine_config, normalization, row mappers, and
  MachineInput.
- frontend types + Settings.tsx: drop the fields and the Jellyfin/Jellyseerr
  form sections + service options.
- Update frontend test fixtures.

Existing DB rows may still carry these keys in config_json; they are inert and
drop on the next machine save. Verification: backend ruff clean, pytest 222;
frontend lint 0 errors, build success, 70 tests.
2026-06-23 12:54:27 +00:00
Developer 5eb49be697 style(dependencies): apply formatter to dependencies rewrite 2026-06-23 11:48:25 +00:00
Developer 8ff735d644 feat(services): resolve Jellyfin/Jellyseerr from the service registry (backend)
Slice 4b backend half. Jellyfin and Jellyseerr clients are now resolved from
service instances instead of machine-level app config.

- Add jellyseerr service definition (6 service types total); add user_id to
  the Jellyfin service config.
- dependencies.py: jellyfin_service_id query param + _service_record
  (decrypt-on-read); get_jellyfin_client / get_jellyseerr_client / get_user_id
  resolve against the service registry (first enabled instance as fallback).
- SSH/Files transport (get_ssh_client) unchanged; still uses machine_id.
- Update service-registry tests for 6 types.

Selection model: split params — ?jellyfin_service_id= for Jellyfin/Jellyseerr,
?machine_id= for SSH/Files. Frontend threading follows in the next PR.

Verification: backend ruff clean, pytest 222 passed; frontend green (unchanged).
2026-06-23 11:43:33 +00:00
Developer c9c72be0b6 feat(services): cleanup, services admin UI, docs
PR 4a of the runtime service registry change.

- Remove addon pages (/addons/:addonId, AddonPage, addons/*) superseded by
  service pages.
- Remove grafana_url/prometheus_url from backend config, compose, .env.example,
  and README (URLs now live on service records; VITE_ frontend deep-link vars
  retained).
- Add Services page (/services) with create/list/delete + sidebar nav, so
  services are configurable in the tool itself and service pages are reachable.
- Update docs/REQUIREMENTS.md service-registry section; add CHANGELOG.md with
  the breaking-upgrade note (MANAGE_ENCRYPTION_KEY required; grafana/prometheus
  env vars removed; default widget seeding removed).

Verification: backend ruff clean, pytest 222 passed; frontend lint 0 errors,
build success, 70 tests passed.
2026-06-23 10:57:30 +00:00
Developer f6a86310cc style(widgets): apply formatter to widget rebind files 2026-06-22 18:22:18 +00:00
Developer 10fd4ead4a feat(widgets): rebind widgets to the service registry
PR 2 of 4 for the runtime service registry change.

- dashboard_widgets gains service_id + widget_kind columns (legacy
  addon_id/widget_type kept but unused).
- Source adapters take (service: ServiceRecord | None, widget_kind, config).
  SERVICE_ADAPTERS keyed by service_type; BUILTIN_ADAPTERS for backups/static.
- Backups and static stay as service-less built-ins (service_id nullable),
  exposed via GET /api/widgets/builtin.
- SSH task adapter resolves the task + instance, runs over SSH, and appends a
  service_task_runs history row on success/failure/timeout/error.
- Retire widgets/registry.py; widget metadata now comes from the integrations
  registry + widgets/builtin. Remove /api/widgets/types and /api/widgets/sources.
- Stop default widget seeding (fresh install = empty dashboard).
- Rewrite widget tests around the service-bound + built-in model (26 tests).

Backend-only breaking change; frontend is reconciled in Slice 3. Build/lint
stay green; pytest 222 passed.
2026-06-22 16:42:56 +00:00
Developer fd534a816b style(services): apply formatter to service registry files 2026-06-22 14:00:07 +00:00
Developer 8cdeadd6dd feat(services): backend service registry foundation (encryption, definitions, CRUD)
PR 1 of 4 for the runtime service registry change.

- Add Fernet encryption helper (services/secrets.py) with a required
  MANAGE_ENCRYPTION_KEY; validate it on startup.
- Add closed integrations/ registry with Pydantic config + widget-config
  definitions for grafana, prometheus, jellyfin, nextcloud, and ssh_tasks.
- Add services + service_task_runs tables and SettingsStore CRUD with
  cascade-delete (defensive until widgets carry service_id).
- Add /api/services/types and /api/services/instances CRUD (encrypted secrets,
  secrets_set flags only; never plaintext).
- Declare cryptography as a direct dependency.
- Require MANAGE_ENCRYPTION_KEY in compose + .env.example + README.
- Add 25 backend tests (registry, encryption, CRUD, cascade, task-run history).

Verification: ruff clean; pytest 225 passed; frontend lint/build green.
2026-06-22 12:56:03 +00:00
Developer 1cd8e926de feat(widgets): add backend source adapters and per-widget data endpoint
PR 2 of 4 for configurable dashboard widgets.

- Add grafana_url and prometheus_url settings (config.py + compose/env).
- Create WidgetSource protocol and adapters for jellyfin, backups, grafana,
  prometheus, ssh_task, and static sources.
- Add GET /api/widgets/instances/{id}/data endpoint.
- Extract shared dashboard helpers into domain/dashboard.py so widgets and
  the dashboard router reuse the same logic.
- Add adapter and data-endpoint tests.
- Update apply-progress.md.

Verification: ruff clean; backend pytest 200 passed; frontend lint/build green.
2026-06-21 10:09:45 +00:00
Developer 200d319fb0 feat(widgets): add backend CRUD, registry, and default seeding
Introduce a closed, compile-time widget registry and backend CRUD for
dashboard widget instances.

- Add dashboard_widgets SQLite table in SettingsStore with CRUD helpers and
  default seeding (Jellyfin + Backups) on first install.
- Add Pydantic models with credential-key and secret-value rejection.
- Add widgets router: /api/widgets/sources, /types, /instances CRUD.
- Call ensure_defaults() in app lifespan so fresh installs seed defaults.
- Add backend tests covering registry, CRUD, validation, and seeding.
- Include SDD artifacts: exploration, proposal, spec, design, tasks.
2026-06-19 20:07:47 +00:00
Developer 08a3b616f6 refactor(monitoring): decommission legacy SSH-scraping poller (slice 1)
The 2026-06-16/17 observability update externalised metrics to
Prometheus + node_exporter + Grafana, but the legacy Manage-side
SSH-scraping monitor was never removed. It duplicated the new stack,
ran SSH df on every machine every 300s, and fed nothing (its UI was
deleted in e2ad731). This slice decommissions the duplication.

Removed (backend):
- services/monitoring_poller.py (MonitoringPoller) — entire file
- services/monitoring_actions.py (disk_space, run_machine_operation,
  poll_machine_snapshot, build_machine_client) — entire file;
  run_machine_operation had only 2 callers (the poller + /disk), both gone
- tests/test_monitoring_actions.py
- endpoints: POST /api/monitoring/poller, GET /machines/{id}/actions,
  GET /disk (and the now-dead _resolve_machine helper)
- lifespan wiring (main.py), dependency wrapper (dependencies.py),
  poller.start()/kick() from machine save (routers/settings.py)
- SettingsStore: monitoring_machine_actions table CREATE + 2 indexes +
  record/list/prune_machine_actions methods; DROP TABLE IF EXISTS on
  startup cleans existing DBs (user-approved)
- config knobs: monitoring_poll_interval_seconds,
  monitoring_poll_initial_delay_seconds, monitoring_action_retention_days
- test_api.py: TestMonitoring._ensure_machine + test_disk

Kept (fits the new model): /machines, /prometheus-targets, /alerts,
/alertmanager-status, /alertmanager-webhook; the disk_usage JOB template
(manual on-demand, not monitoring); node_exporter_* machine fields
(they point Prometheus at the right host).

Gate: backend pytest 173 passed; ruff clean.
2026-06-17 20:48:56 +00:00
Developer e2ad731b5f feat(observability): add Prometheus/Grafana/Loki/Alertmanager/Alloy stack and remove legacy Monitoring UI 2026-06-16 13:44:35 +00:00
alex a748512d22 test: verify backup monitoring implementation 2026-05-11 21:57:31 +02:00
alex 1a4f56cdd9 feat: add backup dashboard summary endpoint 2026-05-11 21:41:49 +02:00
alex 6b0f2df629 feat: add backup monitoring API endpoints 2026-05-11 21:34:52 +02:00
alex 836f733a2e feat: Add API key authentication for backup tool
- Add get_api_key() and require_api_key() to auth.py
- Add generic key-value settings storage to SettingsStore
- Add test_api_key_auth to verify the implementation
- Uses secrets.compare_digest for timing-safe comparison
- Auto-generates API key on first use and stores in settings DB
2026-05-11 21:25:46 +02:00
alex f912623677 feat: add backup alert background poller 2026-05-11 20:47:08 +02:00