manage/openspec/changes/archive/service-registry/proposal.md

Proposal: Runtime Service Registry

Change: service-registry Phase: proposal Date: 2026-06-19 Status: awaiting review (design only — no implementation yet)

Context and problem

Phase 1 shipped a configurable dashboard widget system whose service URLs (Grafana, Prometheus) and app credentials (Jellyfin, Jellyseerr) are driven by environment variables and machine-level fields. This has three problems:

1. Operators cannot change services without a redeploy. Adding a second Grafana, pointing Prometheus at a different host, or rotating a Jellyfin API key requires editing env vars and restarting containers.
2. Configuration is split across three places. Service URLs live in env vars (GRAFANA_URL, PROMETHEUS_URL), Jellyfin/Jellyseerr live on machine records, and widget instances live in the widget table. There is no single "what is configured" view.
3. The widget registry is decoupled from the services it depends on. A Grafana widget does not know which Grafana instance it talks to; the widget config holds a dashboard_uid while the base URL is global.

Proposal

Introduce a runtime service registry persisted in the backend SQLite database:

• Each service instance (e.g. "Production Grafana", "Home Jellyfin") is a DB record carrying its non-secret config and encrypted secret fields.
• Service definitions live as Python modules with Pydantic classes in the repo. Each definition declares its config schema, its secret fields, and the widget kinds it provides (with their own config schemas).
• Service pages at /services/:serviceType/:serviceId render the service-specific UI and list the widgets that service can contribute to the dashboard. These replace the existing addon pages.
• Dashboard widgets become service-bound: a widget instance references a service_id and a widget_kind drawn from that service's definition.
• Machine records are reduced to transport only (SSH + node_exporter); the machine-level Jellyfin/Jellyseerr app fields are removed.

Goals

• One source of truth for every external service the app talks to.
• Add/reconfigure/rotate a service from the UI with no redeploy.
• Multiple instances per service type (two Grafanas, two Jellyfins).
• Centralized, version-controlled service definitions that are easy to extend.
• Widgets discoverable per-service and individually addable to the dashboard.
• Secrets (API keys / tokens) encrypted at rest.

Non-goals

• No general-purpose plugin/marketplace system. Service definitions are closed, compile-time code. Adding a brand-new service still requires a backend deploy and a Python module.
• No OAuth token exchange per service in this change. Only API keys / tokens are stored (encrypted). OAuth-proxy flows (e.g. Grafana behind Authentik) continue to be handled externally.
• No drag-and-drop dashboard layout, no grid, no per-user dashboards. This change keeps the existing single stacked-column dashboard model.
• No in-app charting. The thin-dashboard observability rule still holds; service pages surface deep-links and metadata only.
• No silent data migration. Machine-level Jellyfin/Jellyseerr config is removed without an automatic converter (see Decisions).

Decisions (from grilling)

Topic	Decision
Scope of services	All current services: Grafana, Prometheus, Jellyfin, Nextcloud, and the SSH task runner. Definitions centralized in repo.
Definition format	Python modules with Pydantic classes for service config and widget config, combined under each service definition.
Auth storage	API keys / tokens only, encrypted at rest.
Encryption key	Single env-provided master key (MANAGE_ENCRYPTION_KEY).
Machine app config	Services replace machine-level Jellyfin/Jellyseerr app config. Machines become SSH/monitoring transport only.
Migration	Break backwards compatibility. Users re-enter service config after upgrade; no automatic converter.
Multi-instance	Yes — multiple service records per service type.
Addon pages	Replaced by generic service pages at /services/:serviceType/:serviceId.
Widget binding	The service definition owns its widget config schemas. Widgets are instantiated from a service instance + a widget kind.

Risks

• Breaking upgrade. Existing deployments lose their Jellyfin config and must re-enter it. We must document this loudly in the changelog and README.
• Encryption key management. Losing MANAGE_ENCRYPTION_KEY makes all stored secrets unrecoverable. Key rotation requires re-encrypting every service record.
• Large surface area. This change touches backend models, settings store, widget registry, adapters, frontend routing, dashboard config UI, and docs. It must be split into reviewable PRs (see tasks.md).
• SSH task runner as a service needs care: saved tasks already have their own registry. The service record should hold connection/auth; the task registry stays.
• Env vars are not fully eliminated. The encryption key and core auth/OIDC settings still require env vars; only service URLs/credentials move to the DB.

Out of scope for this proposal

• Automatic migration tooling from machine app config to service records.
• Secret rotation UI or key-rotation workflow.
• Per-user or multi-dashboard support.
• Runtime/hot-reload of service definition files (definitions are loaded at startup).