diff --git a/openspec/changes/grafana-metric-gateway/proposal.md b/openspec/changes/grafana-metric-gateway/proposal.md new file mode 100644 index 0000000..7f90e53 --- /dev/null +++ b/openspec/changes/grafana-metric-gateway/proposal.md @@ -0,0 +1,134 @@ +# SDD Proposal: Grafana Metric Gateway + +**Change:** `grafana-metric-gateway` +**Phase:** proposal +**Date:** 2026-07-09 + +## 1. Problem / Why Now + +**New network constraint surfaced after `prometheus-direct-charting` shipped (2026-07-08):** the Prometheus instance is *not directly reachable* from the Manage backend (firewalled / different network / no auth proxy). Grafana is the only reachable surface and already holds an authenticated, datasource-aware API. The direct-Prom path added by `prometheus-direct-charting` therefore cannot work in production — every chart/gauge/mean/metric widget call fails at the network layer. + +Rather than expose Prometheus directly (infra change outside Manage's control), route metric queries through Grafana's existing `/api/ds/query` datasource proxy. Grafana becomes the transport; Prometheus remains the logical source. + +This is a **partial revert of `prometheus-direct-charting`**, justified by new information that was not on the table when direct-Prom was chosen. It is not a fourth flip of preference — the network reality forces it. + +## 2. Target Users and Situations + +- **Primary users:** Operators whose Prometheus sits behind a firewall or on an unreachable network, with Grafana as the only authenticated entry point. +- **Workflow moments:** identical to today — open a dashboard, see metric charts/gauges/means. The only change is that the data flows Grafana→Prom instead of Manage→Prom directly. +- **Urgency:** High. The widgets added in Change A are currently non-functional in this network topology. + +## 3. Product Outcome + +After this change: + +- Operators configure a **`prometheus` service** whose connection fields point at a Grafana gateway (URL + API key + datasource UID), not at Prometheus directly. +- The **chart / gauge / mean / metric** widget kinds behave exactly as today (same config, same rendering, same `{series}` shape) — only the transport changes. +- Widgets are renamed to **neutral `Metric*`** names (`MetricChartWidget`, `MetricGaugeWidget`, `MetricMeanWidget`) so the component names survive future source changes and don't mislead (the widgets talk to Grafana now, not Prometheus directly). +- The `prometheus` service type remains the sole metric surface — no `grafana` service type is reintroduced. + +## 4. Scope Boundaries and Non-Goals + +### In scope + +- **`prometheus` service config schema changes:** drop the direct-Prom `base_url`; add `grafana_url` (ServiceBaseUrl), `datasource_uid` (default `"prometheus"`), keep `timeout_seconds`. +- **`prometheus` service secret schema changes:** add `grafana_api_key` (encrypted, required). +- **`MetricSource` adapter** (rename of `PrometheusWidgetSource`): the `chart` / `gauge` / `mean` / `metric` dispatch paths issue `POST {grafana_url}/api/ds/query` with the API key + datasource UID + PromQL expr, and normalize Grafana's frames response into the existing `{series}` / `{value}` shapes. +- **Frames→series normalization restored** (recovered from git history at commit `65bae95`, the pre-Change-A `GrafanaWidgetSource._fetch_chart`), refactored into the existing `widgets/prometheus_range.py` helper module alongside `normalize_prometheus_matrix`. +- **Widget rename:** `PrometheusChartWidget` → `MetricChartWidget`, `PrometheusGaugeWidget` → `MetricGaugeWidget`, `PrometheusMeanWidget` → `MetricMeanWidget` (via `git mv`, preserving history). Bindings in `registry.ts` move to the new names; the `prometheus` service's widget kinds stay `chart`/`gauge`/`mean`/`metric` (unchanged). +- **Status check update:** `get_prometheus_status` now runs a trivial query (`up`) through the Grafana gateway rather than hitting Prom `/api/v1/status/buildinfo` directly. This actually validates the full path (Grafana up + datasource reachable + Prom responding) — a stronger signal than the pre-change check. +- **`service-credential-tester` proposal update:** the Prom test routine in `openspec/changes/service-credential-tester/proposal.md` must be updated to test the Grafana gateway path (POST `/api/ds/query` with `up`), not direct Prom. Dependency, not a blocker. +- **CHANGELOG migration note:** existing `prometheus` service instances must be reconfigured (replace `base_url` with `grafana_url` + add `grafana_api_key` secret + optionally `datasource_uid`). + +### Non-goals (explicitly out of scope) + +- **Reintroducing the `grafana` service type.** Grafana is the transport, not a first-class service. No `grafana` registry entry, no `GrafanaLinkWidget`, no `LinksTab`, no `get_grafana_status`. (Those were intentionally removed; this change does not restore them.) +- **Grafana deep-link widgets** (dashboard UID + panel ID links). Out of scope; the user explicitly wanted only the query gateway. +- **A pluggable "data source middleware" framework.** Per-widget-kind direct wiring only; no adapter registry abstraction. (Same discipline as Change A.) +- **Touching the qBittorrent widgets, the storage harness, or MediaIndex.** Unaffected. +- **Renaming the `prometheus` service type itself.** It stays `prometheus` (per operator decision — "Prometheus is the concept, Grafana is how I reach it"). +- **Exposing Prometheus directly as an alternative.** Direct-Prom is removed, not kept-as-fallback. If a future topology can reach Prom directly, that's a separate change. +- **Grafana status / health as a separate surface.** Grafana reachability is folded into the Prom status check (it validates the gateway path). + +## 5. High-Level Approach + +### 5.1 Backend + +1. **`integrations/prometheus.py`** — update `PrometheusConfig`: + - Remove `base_url`. + - Add `grafana_url: ServiceBaseUrl` (reuses the existing schema-enforced http(s) validator). + - Add `datasource_uid: str = "prometheus"`. + - Keep `timeout_seconds: int = 10`. + - Secret schema: replace any existing secret with `grafana_api_key` (required, encrypted). +2. **`widgets/prometheus_range.py`** — add `normalize_grafana_frames(raw) -> list[series]` alongside the existing `normalize_prometheus_matrix`. Recovered from `65bae95`, refactored to share the label-dedup rule with the matrix normalizer. +3. **`widgets/sources.py`** — rename `PrometheusWidgetSource` → `MetricSource` (internal class; service-type key stays `"prometheus"` in `SERVICE_ADAPTERS`). Each of `_fetch_chart`, `_fetch_gauge`, `_fetch_mean`, and the instant-`metric` path switches from `/api/v1/query[_range]` to `POST {grafana_url}/api/ds/query` with body: + + ```json + {"queries": [{"datasource": {"uid": "", "type": "prometheus"}, + "expr": "", "format": "time_series", + "intervalMs": , "maxDataPoints": , "refId": "A"}], + "from": "", "to": "now"} + ``` + + Then normalize via `normalize_grafana_frames`. Window presets (1h/6h/24h/7d) and step derivation (Change A's `step_for_window`) map to Grafana's `from`/`intervalMs`. +4. **`routers/monitoring.py`** — `get_prometheus_status` runs `up` through the gateway; success → `{up: true, version: }`; failure → specific message (auth, unreachable, datasource not found). +5. **Tests** — update existing tests: the chart/gauge/mean adapter tests now assert a `POST /api/ds/query` call (mocked) instead of a direct Prom GET. Reuse the same `{series}` / `{value}` assertions. + +### 5.2 Frontend + +1. **`git mv` widget files** (history-preserving): `PrometheusChartWidget.tsx` → `MetricChartWidget.tsx`, `PrometheusGaugeWidget.tsx` → `MetricGaugeWidget.tsx`, `PrometheusMeanWidget.tsx` → `MetricMeanWidget.tsx`, plus their test files. Rename the exported components. The recharts/`LineSeriesChart`/gauge-band rendering is preserved unchanged. +2. **`integrations/registry.ts`** — the `prometheus` service binding keeps widget kinds `chart`/`gauge`/`mean`/`metric` but points at the renamed `Metric*` components. No user-visible change. +3. **Service config form (`ServiceConfigFields`)** — automatically reflects the new schema (grafana_url, datasource_uid) since it renders from the backend-provided `config_schema`. No bespoke UI work beyond relabeling the secret field. +4. **Types** — `PrometheusStatus` type stays; possibly gains a `gateway` field noting the path. Minimal change. + +### 5.3 Canonical spec impact — first non-additive sync + +The `prometheus-charting` canonical domain (`openspec/specs/prometheus-charting/spec.md`) was created by Change A with requirements stating direct Prom querying (SC-101, SC-102, SC-104, etc.). This change **MODIFIES** those requirements: the data source becomes Grafana's gateway, not direct Prom. This is the first non-additive canonical sync in the project — `sdd-sync` will need a `## MODIFIED Requirements` delta rather than `## ADDED`. Flagged for the sync phase; the requirements' *intent* (multi-series chart, gauge bands, mean over window) is unchanged, only the *mechanism* (transport) changes. + +## 6. Success Criteria / Acceptance Criteria + +1. A `prometheus` service configured with Grafana gateway fields (URL + API key + datasource UID) successfully powers `chart`/`gauge`/`mean`/`metric` widgets — no direct Prom network call is made. +2. The chart widget renders multi-series line charts with the same look as today (recharts via `LineSeriesChart`). +3. The gauge widget renders threshold bands; the mean widget renders the windowed mean — both sourced via Grafana. +4. The instant `metric` widget returns a scalar via Grafana. +5. `get_prometheus_status` validates the full gateway path; a broken Grafana or unreachable datasource produces a specific error message. +6. Widgets are renamed to `MetricChartWidget` / `MetricGaugeWidget` / `MetricMeanWidget`; no `Prometheus*Widget` names remain in frontend source; tests renamed consistently. +7. The `prometheus-charting` canonical domain's transport-related requirements are MODIFIED to reflect the Grafana gateway; durable requirements (chart/gauge/mean behavior) are preserved. +8. The `service-credential-tester` proposal's Prom test routine is updated to test the gateway path. +9. CHANGELOG documents the operator migration (reconfigure `prometheus` services with Grafana gateway fields). +10. `pytest`, `npm run build`, `npm run lint`, `npm run test` all green. + +## 7. Risks and Mitigations + +| Risk | Mitigation | +|------|------------| +| **Reintroducing complexity that Change A removed.** The frames→series normalization was deleted for a reason. | Recover the known-good implementation from `65bae95` rather than rewriting; refactor into `prometheus_range.py` so matrix + frames normalizers share the label-dedup rule. | +| **Grafana `/api/ds/query` response shape varies across Grafana versions.** | The recovered normalizer already handles the frames schema defensively (displayName → labels → field name fallback). Add a test fixture from a current Grafana version. | +| **Existing `prometheus` service instances break silently** (their `base_url` config no longer exists). | SC-9 CHANGELOG note; startup config validation rejects the old shape with a clear migration message. | +| **First non-additive canonical sync** — `sdd-sync` MODIFIED delta could mismatch. | Author the delta carefully; match requirement IDs exactly; verify with `sdd-status` before archive. | +| **Credential-tester proposal dependency.** | Update it in the same change (or sequence this change before the credential tester's apply). | +| **Operators with a directly-reachable Prom lose that option.** | Explicit non-goal; documented. If needed later, add a `direct_url` optional field as a separate change. | +| **Review budget (>400 lines).** | Likely 2 slices: (S1) backend config + adapter + frames normalizer + tests; (S2) frontend widget renames + registry + status check. | + +## 8. Resolved Questions (no question round needed) + +- **Q1 — Keep `prometheus` service type?** Yes. It remains the user-facing logical service; only its config/secret schema and adapter internals change. +- **Q2 — Restore `grafana` service type?** No. Grafana is the transport, not a service. No deep-link widgets, no status surface. +- **Q3 — Widget naming?** Neutral `Metric*` (MetricChartWidget / MetricGaugeWidget / MetricMeanWidget). Decouples widget identity from transport; survives future source changes. +- **Q4 — Direct Prom path kept as fallback?** No. Removed. If a future topology needs it, separate change. +- **Q5 — Status check approach?** Run `up` through the gateway; validates the full path in one call. + +## 9. Future Phases + +1. **Optional `direct_url` fallback** on the `prometheus` service for topologies where Prom IS directly reachable (skip the Grafana hop). +2. **Multi-datasource support** — one Grafana gateway, many datasources (Loki, InfluxDB) selectable per widget. +3. **Datasource health breakdown** — status check distinguishes "Grafana down" vs "Prom datasource down" via Grafana's `/api/datasources/health`. + +--- + +## Notes for downstream phases + +- **Dependency:** the `service-credential-tester` proposal must be updated when this change's design is firm (its Prom test routine changes from direct Prom to gateway query). +- **Canonical first:** this is the project's first non-additive canonical sync (`prometheus-charting` MODIFIED requirements). The sync phase should be deliberate. +- **Recovery source:** the frames→series normalizer is known-good at git commit `65bae95` (`backend/src/media_library_viewer_api/widgets/sources.py`, `_fetch_chart`). Restore from there, don't rewrite. +- **Honest scope note:** this partially reverts `prometheus-direct-charting` (2026-07-08). That change's direct-Prom path and Grafana removal are undone; its gauge/mean/LineSeriesChart/rendering work is preserved. The reversal is justified by the new network constraint (Q1, §1).