diff --git a/openspec/changes/service-registry/design.md b/openspec/changes/service-registry/design.md new file mode 100644 index 0000000..ac99d3c --- /dev/null +++ b/openspec/changes/service-registry/design.md @@ -0,0 +1,420 @@ +# Design: Runtime Service Registry + +**Change:** `service-registry` +**Phase:** design +**Date:** 2026-06-19 + +## 1. Architecture overview + +``` +┌────────────────────────────────────────────────────────────────────┐ +│ Browser │ +│ /services/:type/:id ─► ServicePage ─► frontend SERVICE_REGISTRY +│ Dashboard ─► WidgetInstance ─► widget component │ +└────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌────────────────────────────────────────────────────────────────────┐ +│ FastAPI /api/services + /api/widgets │ +│ CRUD service instances · registry metadata · widget data │ +└────────────────────────────────────────────────────────────────────┘ + │ + ┌──────────────────────────┼───────────────────────────┐ + ▼ ▼ ▼ + ServiceStore (SQLite) integrations/ definitions source adapters + services table (Pydantic, closed registry) (resolve service + dashboard_widgets table grafana/prometheus/jellyfin/ → decrypt → call) + nextcloud/ssh_tasks +``` + +Two closed, compile-time registries cooperate: + +- **`integrations.registry.SERVICE_DEFINITIONS`** maps `service_type → ServiceDefinition`. + Each definition declares config schema, secret fields, and widget kinds. +- The widget types available to the dashboard are **derived** from + `SERVICE_DEFINITIONS` at startup, not hand-maintained. + +## 2. Backend data model + +### 2.1 New `services` table + +Extend `SettingsStore.init_schema()`: + +```sql +CREATE TABLE IF NOT EXISTS services ( + id TEXT PRIMARY KEY, + service_type TEXT NOT NULL, + name TEXT NOT NULL, + config_json TEXT NOT NULL DEFAULT '{}', + secrets_json TEXT NOT NULL DEFAULT '{}', -- encrypted blob (Fernet) + enabled INTEGER NOT NULL DEFAULT 1, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL +); +CREATE INDEX IF NOT EXISTS idx_services_type ON services(service_type); +``` + +- `config_json` — non-secret config validated against the service definition's + `config_schema`. +- `secrets_json` — a JSON object of `{field_name: ciphertext}` produced by the + encryption helper. Never returned to the client in plaintext; only the boolean + "is set" flags are surfaced. + +### 2.2 `dashboard_widgets` schema change + +The existing table gains two columns and loses the global meaning of `widget_type`: + +```sql +ALTER TABLE dashboard_widgets ADD COLUMN service_id TEXT; +ALTER TABLE dashboard_widgets ADD COLUMN widget_kind TEXT; +``` + +- `widget_kind` is the kind declared by the service definition (e.g. `"link"`, + `"metric"`, `"activity"`). +- `service_id` references `services.id`. +- `widget_type` is retained temporarily as `"{service_type}.{widget_kind}"` for + backwards-compatible reads during the transition, then dropped in the final slice. +- The old `addon_id` column is dropped; addon identity is now `service_type`. + +## 3. Service definitions (Pydantic, in repo) + +New package: `backend/src/media_library_viewer_api/integrations/` +(chosen to avoid collision with the existing `services/` infra package). + +### 3.1 Base classes — `integrations/base.py` + +```python +from typing import Any, ClassVar +from pydantic import BaseModel, Field + +class SecretField(BaseModel): + key: str + label: str + required: bool = False + helper: str | None = None + +class WidgetKind(BaseModel): + kind: str # e.g. "link", "metric", "activity" + name: str + description: str + config_schema: dict[str, Any] # JSON schema for widget config + default_config: dict[str, Any] = {} + refresh_interval_ms: int = 0 + +class ServiceConfigBase(BaseModel): + """Subclass per service to define non-secret config fields.""" + +class ServiceDefinition(BaseModel): + service_type: ClassVar[str] + name: ClassVar[str] + description: ClassVar[str] + config_schema: ClassVar[dict[str, Any]] + secret_fields: ClassVar[list[SecretField]] + widget_kinds: ClassVar[list[WidgetKind]] + + # Adapters are referenced by dotted path or registered separately; + # see §4. The definition itself stays a pure data/schema object. +``` + +### 3.2 Example — `integrations/grafana.py` + +```python +class GrafanaConfig(ServiceConfigBase): + base_url: str = Field(..., description="Grafana base URL, e.g. https://grafana.example.com") + +GRAFANA_DEFINITION = ServiceDefinition( + service_type="grafana", + name="Grafana", + description="Dashboards, metrics, and logs.", + config_schema=GrafanaConfig.model_json_schema(), + secret_fields=[SecretField(key="api_key", label="API key", helper="Service account token")], + widget_kinds=[ + WidgetKind( + kind="link", + name="Dashboard link", + description="Deep-link to a Grafana dashboard or panel.", + config_schema={ + "type": "object", + "properties": { + "dashboard_uid": {"type": "string"}, + "panel_id": {"type": "integer"}, + }, + "required": ["dashboard_uid"], + }, + default_config={"dashboard_uid": ""}, + refresh_interval_ms=0, + ), + ], +) +``` + +Other definition modules follow the same shape: `prometheus.py`, `jellyfin.py`, +`nextcloud.py`, `ssh_tasks.py`. + +### 3.3 Registry — `integrations/registry.py` + +```python +SERVICE_DEFINITIONS: dict[str, ServiceDefinition] = { + "grafana": GRAFANA_DEFINITION, + "prometheus": PROMETHEUS_DEFINITION, + "jellyfin": JELLYFIN_DEFINITION, + "nextcloud": NEXTCLOUD_DEFINITION, + "ssh_tasks": SSH_TASKS_DEFINITION, +} + +def list_service_types() -> list[str]: ... +def get_service_definition(service_type: str) -> ServiceDefinition | None: ... +def get_widget_kind(service_type: str, widget_kind: str) -> WidgetKind | None: ... +``` + +The closed `widgets/registry.py` from Phase 1 is **retired**; its metadata is now derived +from `SERVICE_DEFINITIONS`. + +## 4. Source adapters + +`widgets/sources.py` is refactored so each adapter resolves a **service instance** rather +than reading `get_settings()`: + +```python +class WidgetSource(Protocol): + async def fetch( + self, + service: ServiceRecord, # config + decrypted secrets + widget_kind: str, + config: dict[str, Any], + ) -> dict[str, Any]: ... +``` + +- `ServiceRecord` is a runtime object built by `ServiceStore` carrying the decrypted + secret dict in memory for the duration of the fetch. +- `SOURCE_ADAPTERS` is keyed by `service_type`. +- The data endpoint loads the widget's `service_id`, builds the `ServiceRecord`, then + calls `adapter.fetch(service, widget_kind, widget_config)`. + +## 5. Encryption — `services/secrets.py` + +```python +from cryptography.fernet import Fernet, InvalidToken + +def get_encryption_key() -> bytes: + raw = os.environ.get("MANAGE_ENCRYPTION_KEY") + if not raw: + raise RuntimeError("MANAGE_ENCRYPTION_KEY is required") + return raw.encode() + +def encrypt_secrets(values: dict[str, str]) -> dict[str, str]: ... +def decrypt_secrets(blob: dict[str, str]) -> dict[str, str]: ... +``` + +- `cryptography.fernet.Fernet` (already a transitive dependency to verify). +- Startup validation: `validate_auth_settings` is extended to require + `MANAGE_ENCRYPTION_KEY` and to reject an obviously invalid key. +- Secrets are encrypted field-by-field so the "which secrets are set" metadata is cheap + to compute without decrypting. + +## 6. REST API + +### Services + +| Method | Path | Handler | +|--------|------|---------| +| GET | `/api/services/types` | List service definitions (metadata + config schema + widget kinds). | +| GET | `/api/services` | List service instances (no plaintext secrets; only "set" flags). | +| POST | `/api/services` | Create instance (validates type, config, secret schema). | +| PUT | `/api/services/{id}` | Update instance. | +| DELETE | `/api/services/{id}` | Delete instance; **cascade-deletes** widgets referencing it in the same transaction. | + +### Widgets (unchanged paths, new semantics) + +| Method | Path | Handler | +|--------|------|---------| +| GET | `/api/widgets/instances` | List widgets; each carries `service_id`, `widget_kind`. | +| POST/PUT/DELETE | `/api/widgets/instances/{id}` | CRUD; validation uses service definition's widget schema. | +| GET | `/api/widgets/instances/{id}/data` | Resolve service → adapter → fetch. | + +`GET /api/widgets/types` and `/api/widgets/sources` are removed; widget metadata is +served via `/api/services/types` (widget kinds under each service). + +## 7. Frontend + +### 7.1 New `frontend/src/integrations/registry.ts` + +Closed frontend registry mirroring the backend: `serviceType → ServiceDefinition` +(config fields, secret fields with `secret: true`, widget kinds, default refresh +intervals, and a `component` for the service page). + +### 7.2 Service pages + +- Route: `/services/:serviceType/:serviceId` (replaces `/addons/:addonId`). +- `ServicePage` looks up the definition and renders the service-specific component, + a config editor, and the list of widget kinds that can be added to the dashboard. +- `App.tsx` removes the `/addons/:addonId` route; old addon URLs redirect to the + default service of that type (or a not-found alert). + +### 7.3 Dashboard config dialog + +- "Add widget" flow becomes: **pick service → pick widget kind → configure**. +- The widget card shows the parent service name. + +### 7.4 Types / API / hooks + +- `frontend/src/api/services.ts` + `hooks/useServices.ts` for the services API. +- `frontend/src/types/index.ts` gains `ServiceInstance`, `ServiceInstanceInput`, + `ServiceTypeInfo`, `ServiceWidgetKind`. + +## 8. Migration and breaking changes + +- **DB migration on startup:** add `services` table; add `service_id` / `widget_kind` + columns to `dashboard_widgets`; drop `addon_id`. +- **Machine app fields removed:** `jellyfin_url`, `jellyfin_user_id`, `jellyfin_api_key`, + `jellyseerr_url`, `jellyseerr_api_key` are dropped from machine records and the + `MonitoringMachine` model. Machines keep SSH + node_exporter transport fields only. +- **Env vars removed from `config.py`:** `grafana_url`, `prometheus_url`. (Grafana/Prometheus + URLs now live on service records.) `MANAGE_ENCRYPTION_KEY` is added as required. +- **Default widget seeding** is removed; a fresh install starts with no widgets. The user + adds Jellyfin/Backups widgets after configuring the corresponding services. +- **`docs/REQUIREMENTS.md` and `README.md`** updated to describe services, the + `MANAGE_ENCRYPTION_KEY` requirement, and the breaking upgrade note. + +## 9. File-level plan + +### Create (backend) + +| File | Purpose | +|------|---------| +| `integrations/__init__.py` | Package marker. | +| `integrations/base.py` | `ServiceDefinition`, `WidgetKind`, `SecretField`, `ServiceConfigBase`. | +| `integrations/registry.py` | Closed `SERVICE_DEFINITIONS` + helpers. | +| `integrations/grafana.py`, `prometheus.py`, `jellyfin.py`, `nextcloud.py`, `ssh_tasks.py` | One module per service. | +| `services/secrets.py` | Fernet encrypt/decrypt + key validation. | +| `services/service_store.py` | CRUD for `services` table; decrypt-on-read for adapters. | +| `routers/services.py` | `/api/services*` endpoints. | +| `models/services.py` | Pydantic request/response models. | + +### Modify (backend) + +| File | Change | +|-------|--------| +| `services/settings_store.py` | `services` table; widget columns; drop machine app fields. | +| `widgets/sources.py` | Adapters take a `ServiceRecord`. | +| `widgets/registry.py` | Retired (metadata served by `integrations/registry.py`). | +| `routers/widgets.py` | Validate against service widget schema; resolve service on data fetch. | +| `config.py` | Remove `grafana_url`/`prometheus_url`; document `MANAGE_ENCRYPTION_KEY` (read in `secrets.py`). | +| `main.py` | Register `services_router`; validate encryption key on startup. | +| `dependencies.py` | Jellyfin/SSH resolution now goes via services, not machine app fields. | + +### Create (frontend) + +| File | Purpose | +|------|---------| +| `integrations/registry.ts` | Closed frontend service registry. | +| `api/services.ts`, `hooks/useServices.ts` | Services API + hooks. | +| `pages/ServicePage.tsx` | Generic `/services/:type/:id` page. | +| `integrations/components/*` | Per-service page components. | + +### Modify (frontend) + +| File | Change | +|-------|--------| +| `App.tsx` | Replace `/addons/:addonId` with `/services/:serviceType/:serviceId`. | +| `components/WidgetConfigDialog.tsx` | Service → widget-kind picker. | +| `widgets/registry.ts` | Retired; widgets derived from service registry. | +| `types/index.ts` | Service types; widget gains `service_id` + `widget_kind`. | +| `pages/Settings.tsx` | Remove machine Jellyfin/Jellyseerr fields. | + +## 10. Slice boundaries (chained PRs) + +Each slice keeps `pytest`, `ruff`, `npm run lint`, and `npm run build` green. + +1. **Backend foundation** — encryption helper, `integrations/` base + 5 definitions + + registry, `services` table + store, `/api/services*` endpoints, tests. No widget + changes yet. +2. **Backend widget rebind** — add `service_id`/`widget_kind` to widgets, refactor + adapters to take a `ServiceRecord`, retire old `widgets/registry.py`, update data + endpoint. +3. **Frontend services runtime** — types, API, hooks, `integrations/registry.ts`, + service pages, route swap, remove addon pages. +4. **Frontend dashboard + settings rework** — service-based widget picker, drop machine + app fields from Settings, remove `grafana_url`/`prometheus_url` from config, + re-seed behavior, docs (`README.md`, `REQUIREMENTS.md`), changelog breaking-change + note. + +Estimated total: ~2,000–2,400 changed lines across four PRs. + +## 11. Decisions resolved + +1. **Deleting a service that still has widgets** → **cascade delete.** The store deletes + every `dashboard_widgets` row referencing the service inside the same transaction as + the service delete. Simple and safe in SQLite; no 409 pre-check. +2. **`MANAGE_ENCRYPTION_KEY` dev default** → **always required.** No fallback, even when + `AUTH_ENABLED=false`. Startup fails fast if it is missing or not a valid Fernet key. +3. **SSH task runner shape** → **multi-instance, reusable tasks, persisted run history.** + See §12 for the full model. + +## 12. SSH task runner model + +The SSH task runner is the most involved service type. Instances absorb the SSH task +execution role currently held by machines; tasks stay global and reusable; every +invocation is logged. + +### 12.1 Instances + +- `service_type = "ssh_tasks"`. +- Each instance is an SSH endpoint: `host`, `port`, `username`, `ssh_key_id`, optional + `passphrase`. Connection config lives on the service record; the SSH key itself stays + in the existing saved-key registry (referenced by `ssh_key_id`). +- Multi-instance by design ("home server", "media box", …). + +### 12.2 Tasks (global, reusable) + +- Saved tasks remain a **global** registry (`name`, `task_type` shell/python, `content`, + `enabled`). A task is **not** owned by an instance. +- Each task gains `default_service_id` (replaces the old `default_machine_id`) — the + instance it targets by default. At run time the caller may override the target + instance. +- A task can therefore run against any instance; the link is captured per-run. + +### 12.3 Run history (logs) + +A new `service_task_runs` table records every invocation: + +```sql +CREATE TABLE IF NOT EXISTS service_task_runs ( + id TEXT PRIMARY KEY, + task_id TEXT NOT NULL, + service_id TEXT NOT NULL, + status TEXT NOT NULL, -- success | failure | timeout | error + exit_status INTEGER, + duration_ms INTEGER, + stdout_tail TEXT, + stderr_tail TEXT, + error TEXT, + created_at INTEGER NOT NULL +); +CREATE INDEX IF NOT EXISTS idx_service_task_runs_service ON service_task_runs(service_id, created_at DESC); +CREATE INDEX IF NOT EXISTS idx_service_task_runs_task ON service_task_runs(task_id, created_at DESC); +``` + +- Populated by the SSH task adapter on every widget data fetch and by the Actions + runner on manual runs. +- Surfaced on the instance's service page as a log/history list, and on the task detail + as recent runs. +- Replaces the legacy `saved_task_runs` concept once the Actions page is rebuilt on + services (Slice 4 / a follow-up). + +### 12.4 SSH task widget + +Widget config for `ssh_tasks` becomes `{ task_id, service_id? }`: + +- If `service_id` is omitted, the task's `default_service_id` is used. +- The adapter loads the task, resolves the instance, runs it, appends a + `service_task_runs` row, and returns the trimmed stdout/stderr/exit status. + +### 12.5 Relationship to machines + +- The SSH task execution role moves **out of machines** into `ssh_tasks` instances. +- Machines **keep** their role for the File Browser and node_exporter monitoring + transport in this change, to avoid also reworking Files/Monitoring here. +- Practical consequence: an SSH host used for both files and tasks may be defined twice + (once as a machine, once as an ssh_tasks instance) during the transition. Unifying + machines under services is an explicit **follow-up change**, not part of this one. diff --git a/openspec/changes/service-registry/proposal.md b/openspec/changes/service-registry/proposal.md new file mode 100644 index 0000000..908f74d --- /dev/null +++ b/openspec/changes/service-registry/proposal.md @@ -0,0 +1,99 @@ +# Proposal: Runtime Service Registry + +**Change:** `service-registry` +**Phase:** proposal +**Date:** 2026-06-19 +**Status:** awaiting review (design only — no implementation yet) + +## Context and problem + +Phase 1 shipped a configurable dashboard widget system whose service URLs (Grafana, +Prometheus) and app credentials (Jellyfin, Jellyseerr) are driven by environment +variables and machine-level fields. This has three problems: + +1. **Operators cannot change services without a redeploy.** Adding a second Grafana, + pointing Prometheus at a different host, or rotating a Jellyfin API key requires + editing env vars and restarting containers. +2. **Configuration is split across three places.** Service URLs live in env vars + (`GRAFANA_URL`, `PROMETHEUS_URL`), Jellyfin/Jellyseerr live on machine records, and + widget instances live in the widget table. There is no single "what is configured" + view. +3. **The widget registry is decoupled from the services it depends on.** A Grafana + widget does not know which Grafana instance it talks to; the widget config holds a + `dashboard_uid` while the base URL is global. + +## Proposal + +Introduce a **runtime service registry** persisted in the backend SQLite database: + +- Each **service instance** (e.g. "Production Grafana", "Home Jellyfin") is a DB record + carrying its non-secret config and encrypted secret fields. +- **Service definitions** live as Python modules with Pydantic classes in the repo. Each + definition declares its config schema, its secret fields, and the **widget kinds** it + provides (with their own config schemas). +- **Service pages** at `/services/:serviceType/:serviceId` render the service-specific UI + and list the widgets that service can contribute to the dashboard. These replace the + existing addon pages. +- **Dashboard widgets** become service-bound: a widget instance references a `service_id` + and a `widget_kind` drawn from that service's definition. +- Machine records are reduced to **transport only** (SSH + node_exporter); the + machine-level Jellyfin/Jellyseerr app fields are removed. + +## Goals + +- One source of truth for every external service the app talks to. +- Add/reconfigure/rotate a service from the UI with no redeploy. +- Multiple instances per service type (two Grafanas, two Jellyfins). +- Centralized, version-controlled service definitions that are easy to extend. +- Widgets discoverable per-service and individually addable to the dashboard. +- Secrets (API keys / tokens) encrypted at rest. + +## Non-goals + +- **No general-purpose plugin/marketplace system.** Service definitions are closed, + compile-time code. Adding a brand-new service still requires a backend deploy and a + Python module. +- **No OAuth token exchange per service in this change.** Only API keys / tokens are + stored (encrypted). OAuth-proxy flows (e.g. Grafana behind Authentik) continue to be + handled externally. +- **No drag-and-drop dashboard layout, no grid, no per-user dashboards.** This change + keeps the existing single stacked-column dashboard model. +- **No in-app charting.** The thin-dashboard observability rule still holds; service + pages surface deep-links and metadata only. +- **No silent data migration.** Machine-level Jellyfin/Jellyseerr config is removed + without an automatic converter (see Decisions). + +## Decisions (from grilling) + +| Topic | Decision | +|-------|----------| +| Scope of services | All current services: Grafana, Prometheus, Jellyfin, Nextcloud, and the SSH task runner. Definitions centralized in repo. | +| Definition format | Python modules with Pydantic classes for service config and widget config, combined under each service definition. | +| Auth storage | API keys / tokens only, encrypted at rest. | +| Encryption key | Single env-provided master key (`MANAGE_ENCRYPTION_KEY`). | +| Machine app config | Services **replace** machine-level Jellyfin/Jellyseerr app config. Machines become SSH/monitoring transport only. | +| Migration | **Break backwards compatibility.** Users re-enter service config after upgrade; no automatic converter. | +| Multi-instance | Yes — multiple service records per service type. | +| Addon pages | Replaced by generic service pages at `/services/:serviceType/:serviceId`. | +| Widget binding | The service definition **owns** its widget config schemas. Widgets are instantiated from a service instance + a widget kind. | + +## Risks + +- **Breaking upgrade.** Existing deployments lose their Jellyfin config and must re-enter + it. We must document this loudly in the changelog and README. +- **Encryption key management.** Losing `MANAGE_ENCRYPTION_KEY` makes all stored secrets + unrecoverable. Key rotation requires re-encrypting every service record. +- **Large surface area.** This change touches backend models, settings store, widget + registry, adapters, frontend routing, dashboard config UI, and docs. It must be split + into reviewable PRs (see `tasks.md`). +- **SSH task runner as a service** needs care: saved tasks already have their own + registry. The service record should hold connection/auth; the task registry stays. +- **Env vars are not fully eliminated.** The encryption key and core auth/OIDC settings + still require env vars; only service URLs/credentials move to the DB. + +## Out of scope for this proposal + +- Automatic migration tooling from machine app config to service records. +- Secret rotation UI or key-rotation workflow. +- Per-user or multi-dashboard support. +- Runtime/hot-reload of service definition files (definitions are loaded at startup). diff --git a/openspec/changes/service-registry/tasks.md b/openspec/changes/service-registry/tasks.md new file mode 100644 index 0000000..6025cfc --- /dev/null +++ b/openspec/changes/service-registry/tasks.md @@ -0,0 +1,212 @@ +# Tasks: Runtime Service Registry + +**Change:** `service-registry` +**Phase:** tasks +**Date:** 2026-06-19 + +## Review workload forecast + +| Field | Value | +|-------|-------| +| Estimated changed lines | ~2,000–2,400 | +| 400-line budget risk | High | +| Chained PRs recommended | Yes (4 PRs) | +| Chain strategy | stacked-to-main | + +```text +Decision needed before apply: Yes (see design §11 open questions) +Chained PRs recommended: Yes +Chain strategy: stacked-to-main +``` + +## Slice 1: Backend service foundation (no widget changes) + +**Goal:** Persist service instances with encrypted secrets and expose CRUD + metadata. + +- [ ] **1.1 Add encryption helper** + - Files: `backend/src/media_library_viewer_api/services/secrets.py` (new) + - Lines: ~60 + - Details: Fernet-based `encrypt_secrets` / `decrypt_secrets` / `get_encryption_key`. + Raise on missing `MANAGE_ENCRYPTION_KEY`. Add `cryptography` dependency if missing. +- [ ] **1.2 Add integrations base classes** + - Files: `integrations/__init__.py`, `integrations/base.py` (new) + - Lines: ~80 + - Details: `ServiceDefinition`, `WidgetKind`, `SecretField`, `ServiceConfigBase`. +- [ ] **1.3 Add five service definitions + registry** + - Files: `integrations/grafana.py`, `prometheus.py`, `jellyfin.py`, `nextcloud.py`, + `ssh_tasks.py`, `integrations/registry.py` (new) + - Lines: ~220 + - Details: One `ServiceDefinition` per service with config schema, secret fields, and + widget kinds. `SERVICE_DEFINITIONS` + `get_service_definition` / + `get_widget_kind` helpers. +- [ ] **1.4 Add service store + `services` table** + - Files: `services/settings_store.py` (modify), `services/service_store.py` (new) + - Lines: ~120 + - Details: `services` table in `init_schema`; CRUD helpers; decrypt-on-read for + adapters; "set" flags for the API without plaintext. **Cascade delete:** removing a + service deletes its widgets in the same transaction. Also add the + `service_task_runs` table (design §12.3) now so later slices can populate it. +- [ ] **1.5 Add service Pydantic models + router** + - Files: `models/services.py` (new), `routers/services.py` (new), `main.py` (modify) + - Lines: ~110 + - Details: `GET /api/services/types`, `GET /api/services`, `POST/PUT/DELETE + /api/services/{id}`. Validate type, config, and secret schema against the definition. +- [ ] **1.6 Validate encryption key on startup** + - Files: `auth.py` or `main.py` lifespan (modify) + - Lines: ~10 + - Details: Extend startup validation to require `MANAGE_ENCRYPTION_KEY`. +- [ ] **1.7 Add backend tests** + - Files: `backend/tests/test_services.py` (new) + - Lines: ~140 + - Details: Registry contents, CRUD round-trip, secret encryption/decryption, + unknown service type → 422, missing/invalid encryption key → startup error, + cascade-delete removes a service's widgets. +- [ ] **1.8 Verify** + - Run: `cd backend && .venv/bin/ruff check . && PYTHONPATH=src .venv/bin/python -m pytest` + +**Slice 1 total:** ~720 changed lines (smallest coherent backend foundation). + +## Slice 2: Backend widget rebind to services + +**Goal:** Widgets reference a service instance + widget kind; adapters resolve services. + +- [ ] **2.1 Add widget columns + migrate table** + - Files: `services/settings_store.py` (modify) + - Lines: ~40 + - Details: Add `service_id`, `widget_kind` to `dashboard_widgets`; keep `widget_type` + as `{service_type}.{kind}` during transition; drop `addon_id`. +- [ ] **2.2 Refactor source adapters** + - Files: `widgets/sources.py` (modify) + - Lines: ~160 + - Details: Each adapter takes `(service: ServiceRecord, widget_kind, config)`. + `SOURCE_ADAPTERS` keyed by `service_type`. Jellyfin/Grafana/Prometheus/SSH adapters + resolve connection from the service record. The SSH adapter resolves the task + + instance, runs it, and **appends a `service_task_runs` row** (design §12.3). +- [ ] **2.3 Retire old widget registry** + - Files: `widgets/registry.py` (delete or hollow out), `widgets/__init__.py` + - Lines: ~-60 + - Details: Widget metadata now comes from `integrations/registry.py`. +- [ ] **2.4 Update widgets router + models** + - Files: `routers/widgets.py`, `models/widgets.py` (modify) + - Lines: ~90 + - Details: Validation uses the service definition's widget schema; data endpoint + loads service, builds `ServiceRecord`, calls adapter. +- [ ] **2.5 Update widget tests** + - Files: `backend/tests/test_widgets.py` (modify) + - Lines: ~120 + - Details: Rewrite adapter/data tests around service instances; cover + service-missing, wrong-kind, and encrypted-secret resolution. +- [ ] **2.6 Verify** + - Run: `cd backend && .venv/bin/ruff check . && PYTHONPATH=src .venv/bin/python -m pytest` + +**Slice 2 total:** ~330 changed lines. + +## Slice 3: Frontend services runtime + +**Goal:** Service types/API/hooks, frontend service registry, service pages, route swap. + +- [ ] **3.1 Add service types** + - Files: `frontend/src/types/index.ts` (modify) + - Lines: ~50 + - Details: `ServiceInstance`, `ServiceInstanceInput`, `ServiceTypeInfo`, + `ServiceWidgetKind`. Widget gains `service_id`, `widget_kind`. +- [ ] **3.2 Add services API + hooks** + - Files: `frontend/src/api/services.ts`, `frontend/src/hooks/useServices.ts` (new) + - Lines: ~110 + - Details: Fetch/create/update/delete service instances and types. +- [ ] **3.3 Add frontend service registry** + - Files: `frontend/src/integrations/registry.ts` (new) + - Lines: ~120 + - Details: Closed registry mirroring backend: config fields, secret fields + (`secret: true`), widget kinds, service page components. +- [ ] **3.4 Add service page + components** + - Files: `frontend/src/pages/ServicePage.tsx`, `frontend/src/integrations/components/*` + (new) + - Lines: ~180 + - Details: Generic page dispatches by service type; renders config editor + widget + kinds. Add per-service components (Grafana, Prometheus, Jellyfin, Nextcloud, + SSH tasks). +- [ ] **3.5 Swap routes; remove addon pages** + - Files: `frontend/src/App.tsx`, `frontend/src/pages/AddonPage.tsx`, + `frontend/src/addons/*` (modify/delete) + - Lines: ~-40 net + - Details: `/services/:serviceType/:serviceId`; redirect old `/addons/*` to the + default service of that type. +- [ ] **3.6 Add frontend registry test** + - Files: `frontend/src/integrations/registry.test.ts` (new) + - Lines: ~40 + - Details: Assert all five service types and their widget kinds. +- [ ] **3.7 Verify** + - Run: `cd frontend && npm run lint && npm run build && npm run test -- src/integrations/registry.test.ts` + +**Slice 3 total:** ~460 changed lines. + +## Slice 4: Dashboard picker, settings rework, cleanup, docs + +**Goal:** End-to-end service-based dashboard; remove legacy machine app config + env vars. + +- [ ] **4.1 Rework widget config dialog** + - Files: `frontend/src/components/WidgetConfigDialog.tsx` (modify) + - Lines: ~120 + - Details: "Add widget" = pick service → pick widget kind → configure. Widget cards + show parent service name. +- [ ] **4.2 Update widget components to service model** + - Files: `frontend/src/widgets/*` (modify) + - Lines: ~120 + - Details: Components read `widget_kind`; data shapes unchanged but sourced from the + service adapter. SSH task widget shows last run status from `service_task_runs`. +- [ ] **4.3 Remove machine Jellyfin/Jellyseerr fields** + - Files: `frontend/src/pages/Settings.tsx`, `frontend/src/types/index.ts` + (modify) + - Lines: ~-60 + - Details: Machines are SSH/monitoring transport only. +- [ ] **4.4 Remove grafana_url / prometheus_url from backend config** + - Files: `backend/src/media_library_viewer_api/config.py`, + `docker-compose.yml`, `docker-compose.dev.yml`, `.env.example` + - Lines: ~-10 + - Details: URLs now live on service records. Add `MANAGE_ENCRYPTION_KEY` to compose + - `.env.example`. +- [ ] **4.5 Stop default widget seeding** + - Files: `services/settings_store.py` (modify) + - Lines: ~-20 + - Details: Fresh installs start with no widgets; user adds them after configuring + services. +- [ ] **4.6 Docs + changelog** + - Files: `docs/REQUIREMENTS.md`, `README.md`, `docs/CHANGELOG.md` (new or modify) + - Lines: ~80 + - Details: Service registry section; `MANAGE_ENCRYPTION_KEY` requirement; breaking + upgrade note (re-enter Jellyfin config). +- [ ] **4.7 Verify full stack** + - Run: backend `ruff` + `pytest`; frontend `lint` + `build` + `test`. + +**Slice 4 total:** ~330 changed lines. + +## Integration and acceptance + +- [ ] **5.1 Backend full test run** — `PYTHONPATH=src pytest`, all green. +- [ ] **5.2 Frontend full build/lint/test** — `npm run lint && npm run build && npm run test`. +- [ ] **5.3 Manual dev-stack check** — `docker compose -f docker-compose.dev.yml up --build`: + - Create a Grafana service from the UI; verify the dashboard link widget works. + - Create a Jellyfin service; verify the activity widget resolves it. + - Delete a service with widgets → widgets are cascade-deleted and the service is gone. + - Restart the stack; secrets remain usable (key stable). + - Missing `MANAGE_ENCRYPTION_KEY` → backend refuses to start. + - SSH task runner: define two instances, run the same reusable task against each, + and see both runs in the instance's history log. + +## Guards + +```text +Decision needed before apply: No (design §11 resolved) +Chained PRs recommended: Yes +Chain strategy: stacked-to-main +400-line budget risk: High +``` + +## Explicit follow-ups (out of scope for this change) + +- Rebuild the Actions page UI on top of services (global reusable tasks + + `default_service_id`), replacing the current machine-based saved-task runner. +- Unify machines under services so an SSH host is defined once (today machines still + own File Browser + node_exporter transport; see design §12.5). +- Key rotation / re-encrypt workflow for `MANAGE_ENCRYPTION_KEY`.