manage/openspec/changes/service-registry/proposal.md

# Proposal: Runtime Service Registry

**Change:** `service-registry`
**Phase:** proposal
**Date:** 2026-06-19
**Status:** awaiting review (design only — no implementation yet)

## Context and problem

Phase 1 shipped a configurable dashboard widget system whose service URLs (Grafana,
Prometheus) and app credentials (Jellyfin, Jellyseerr) are driven by environment
variables and machine-level fields. This has three problems:

1. **Operators cannot change services without a redeploy.** Adding a second Grafana,
   pointing Prometheus at a different host, or rotating a Jellyfin API key requires
   editing env vars and restarting containers.
2. **Configuration is split across three places.** Service URLs live in env vars
   (`GRAFANA_URL`, `PROMETHEUS_URL`), Jellyfin/Jellyseerr live on machine records, and
   widget instances live in the widget table. There is no single "what is configured"
   view.
3. **The widget registry is decoupled from the services it depends on.** A Grafana
   widget does not know which Grafana instance it talks to; the widget config holds a
   `dashboard_uid` while the base URL is global.

## Proposal

Introduce a **runtime service registry** persisted in the backend SQLite database:

- Each **service instance** (e.g. "Production Grafana", "Home Jellyfin") is a DB record
  carrying its non-secret config and encrypted secret fields.
- **Service definitions** live as Python modules with Pydantic classes in the repo. Each
  definition declares its config schema, its secret fields, and the **widget kinds** it
  provides (with their own config schemas).
- **Service pages** at `/services/:serviceType/:serviceId` render the service-specific UI
  and list the widgets that service can contribute to the dashboard. These replace the
  existing addon pages.
- **Dashboard widgets** become service-bound: a widget instance references a `service_id`
  and a `widget_kind` drawn from that service's definition.
- Machine records are reduced to **transport only** (SSH + node_exporter); the
  machine-level Jellyfin/Jellyseerr app fields are removed.

## Goals

- One source of truth for every external service the app talks to.
- Add/reconfigure/rotate a service from the UI with no redeploy.
- Multiple instances per service type (two Grafanas, two Jellyfins).
- Centralized, version-controlled service definitions that are easy to extend.
- Widgets discoverable per-service and individually addable to the dashboard.
- Secrets (API keys / tokens) encrypted at rest.

## Non-goals

- **No general-purpose plugin/marketplace system.** Service definitions are closed,
  compile-time code. Adding a brand-new service still requires a backend deploy and a
  Python module.
- **No OAuth token exchange per service in this change.** Only API keys / tokens are
  stored (encrypted). OAuth-proxy flows (e.g. Grafana behind Authentik) continue to be
  handled externally.
- **No drag-and-drop dashboard layout, no grid, no per-user dashboards.** This change
  keeps the existing single stacked-column dashboard model.
- **No in-app charting.** The thin-dashboard observability rule still holds; service
  pages surface deep-links and metadata only.
- **No silent data migration.** Machine-level Jellyfin/Jellyseerr config is removed
  without an automatic converter (see Decisions).

## Decisions (from grilling)

| Topic | Decision |
|-------|----------|
| Scope of services | All current services: Grafana, Prometheus, Jellyfin, Nextcloud, and the SSH task runner. Definitions centralized in repo. |
| Definition format | Python modules with Pydantic classes for service config and widget config, combined under each service definition. |
| Auth storage | API keys / tokens only, encrypted at rest. |
| Encryption key | Single env-provided master key (`MANAGE_ENCRYPTION_KEY`). |
| Machine app config | Services **replace** machine-level Jellyfin/Jellyseerr app config. Machines become SSH/monitoring transport only. |
| Migration | **Break backwards compatibility.** Users re-enter service config after upgrade; no automatic converter. |
| Multi-instance | Yes — multiple service records per service type. |
| Addon pages | Replaced by generic service pages at `/services/:serviceType/:serviceId`. |
| Widget binding | The service definition **owns** its widget config schemas. Widgets are instantiated from a service instance + a widget kind. |

## Risks

- **Breaking upgrade.** Existing deployments lose their Jellyfin config and must re-enter
  it. We must document this loudly in the changelog and README.
- **Encryption key management.** Losing `MANAGE_ENCRYPTION_KEY` makes all stored secrets
  unrecoverable. Key rotation requires re-encrypting every service record.
- **Large surface area.** This change touches backend models, settings store, widget
  registry, adapters, frontend routing, dashboard config UI, and docs. It must be split
  into reviewable PRs (see `tasks.md`).
- **SSH task runner as a service** needs care: saved tasks already have their own
  registry. The service record should hold connection/auth; the task registry stays.
- **Env vars are not fully eliminated.** The encryption key and core auth/OIDC settings
  still require env vars; only service URLs/credentials move to the DB.

## Out of scope for this proposal

- Automatic migration tooling from machine app config to service records.
- Secret rotation UI or key-rotation workflow.
- Per-user or multi-dashboard support.
- Runtime/hot-reload of service definition files (definitions are loaded at startup).