spec(grafana-metric-gateway): add proposal
Route metric queries through Grafana /api/ds/query instead of direct Prom
(Prom is firewalled / unreachable from Manage; Grafana is the only path).
Partial revert of prometheus-direct-charting: keep the gauge/mean/
LineSeriesChart rendering, restore the frames->series normalizer (recovered
from git 65bae95), change prometheus service config to hold Grafana gateway
fields (url + api_key + datasource_uid). Rename widgets to neutral Metric*.
First non-additive canonical sync (prometheus-charting MODIFIED). Updates the
pending service-credential-tester proposal dependency.
This commit is contained in:
Developer
@@ -0,0 +1,134 @@
|
||||
# SDD Proposal: Grafana Metric Gateway
|
||||
|
||||
**Change:** `grafana-metric-gateway`
|
||||
**Phase:** proposal
|
||||
**Date:** 2026-07-09
|
||||
|
||||
## 1. Problem / Why Now
|
||||
|
||||
**New network constraint surfaced after `prometheus-direct-charting` shipped (2026-07-08):** the Prometheus instance is *not directly reachable* from the Manage backend (firewalled / different network / no auth proxy). Grafana is the only reachable surface and already holds an authenticated, datasource-aware API. The direct-Prom path added by `prometheus-direct-charting` therefore cannot work in production — every chart/gauge/mean/metric widget call fails at the network layer.
|
||||
|
||||
Rather than expose Prometheus directly (infra change outside Manage's control), route metric queries through Grafana's existing `/api/ds/query` datasource proxy. Grafana becomes the transport; Prometheus remains the logical source.
|
||||
|
||||
This is a **partial revert of `prometheus-direct-charting`**, justified by new information that was not on the table when direct-Prom was chosen. It is not a fourth flip of preference — the network reality forces it.
|
||||
|
||||
## 2. Target Users and Situations
|
||||
|
||||
- **Primary users:** Operators whose Prometheus sits behind a firewall or on an unreachable network, with Grafana as the only authenticated entry point.
|
||||
- **Workflow moments:** identical to today — open a dashboard, see metric charts/gauges/means. The only change is that the data flows Grafana→Prom instead of Manage→Prom directly.
|
||||
- **Urgency:** High. The widgets added in Change A are currently non-functional in this network topology.
|
||||
|
||||
## 3. Product Outcome
|
||||
|
||||
After this change:
|
||||
|
||||
- Operators configure a **`prometheus` service** whose connection fields point at a Grafana gateway (URL + API key + datasource UID), not at Prometheus directly.
|
||||
- The **chart / gauge / mean / metric** widget kinds behave exactly as today (same config, same rendering, same `{series}` shape) — only the transport changes.
|
||||
- Widgets are renamed to **neutral `Metric*`** names (`MetricChartWidget`, `MetricGaugeWidget`, `MetricMeanWidget`) so the component names survive future source changes and don't mislead (the widgets talk to Grafana now, not Prometheus directly).
|
||||
- The `prometheus` service type remains the sole metric surface — no `grafana` service type is reintroduced.
|
||||
|
||||
## 4. Scope Boundaries and Non-Goals
|
||||
|
||||
### In scope
|
||||
|
||||
- **`prometheus` service config schema changes:** drop the direct-Prom `base_url`; add `grafana_url` (ServiceBaseUrl), `datasource_uid` (default `"prometheus"`), keep `timeout_seconds`.
|
||||
- **`prometheus` service secret schema changes:** add `grafana_api_key` (encrypted, required).
|
||||
- **`MetricSource` adapter** (rename of `PrometheusWidgetSource`): the `chart` / `gauge` / `mean` / `metric` dispatch paths issue `POST {grafana_url}/api/ds/query` with the API key + datasource UID + PromQL expr, and normalize Grafana's frames response into the existing `{series}` / `{value}` shapes.
|
||||
- **Frames→series normalization restored** (recovered from git history at commit `65bae95`, the pre-Change-A `GrafanaWidgetSource._fetch_chart`), refactored into the existing `widgets/prometheus_range.py` helper module alongside `normalize_prometheus_matrix`.
|
||||
- **Widget rename:** `PrometheusChartWidget` → `MetricChartWidget`, `PrometheusGaugeWidget` → `MetricGaugeWidget`, `PrometheusMeanWidget` → `MetricMeanWidget` (via `git mv`, preserving history). Bindings in `registry.ts` move to the new names; the `prometheus` service's widget kinds stay `chart`/`gauge`/`mean`/`metric` (unchanged).
|
||||
- **Status check update:** `get_prometheus_status` now runs a trivial query (`up`) through the Grafana gateway rather than hitting Prom `/api/v1/status/buildinfo` directly. This actually validates the full path (Grafana up + datasource reachable + Prom responding) — a stronger signal than the pre-change check.
|
||||
- **`service-credential-tester` proposal update:** the Prom test routine in `openspec/changes/service-credential-tester/proposal.md` must be updated to test the Grafana gateway path (POST `/api/ds/query` with `up`), not direct Prom. Dependency, not a blocker.
|
||||
- **CHANGELOG migration note:** existing `prometheus` service instances must be reconfigured (replace `base_url` with `grafana_url` + add `grafana_api_key` secret + optionally `datasource_uid`).
|
||||
|
||||
### Non-goals (explicitly out of scope)
|
||||
|
||||
- **Reintroducing the `grafana` service type.** Grafana is the transport, not a first-class service. No `grafana` registry entry, no `GrafanaLinkWidget`, no `LinksTab`, no `get_grafana_status`. (Those were intentionally removed; this change does not restore them.)
|
||||
- **Grafana deep-link widgets** (dashboard UID + panel ID links). Out of scope; the user explicitly wanted only the query gateway.
|
||||
- **A pluggable "data source middleware" framework.** Per-widget-kind direct wiring only; no adapter registry abstraction. (Same discipline as Change A.)
|
||||
- **Touching the qBittorrent widgets, the storage harness, or MediaIndex.** Unaffected.
|
||||
- **Renaming the `prometheus` service type itself.** It stays `prometheus` (per operator decision — "Prometheus is the concept, Grafana is how I reach it").
|
||||
- **Exposing Prometheus directly as an alternative.** Direct-Prom is removed, not kept-as-fallback. If a future topology can reach Prom directly, that's a separate change.
|
||||
- **Grafana status / health as a separate surface.** Grafana reachability is folded into the Prom status check (it validates the gateway path).
|
||||
|
||||
## 5. High-Level Approach
|
||||
|
||||
### 5.1 Backend
|
||||
|
||||
1. **`integrations/prometheus.py`** — update `PrometheusConfig`:
|
||||
- Remove `base_url`.
|
||||
- Add `grafana_url: ServiceBaseUrl` (reuses the existing schema-enforced http(s) validator).
|
||||
- Add `datasource_uid: str = "prometheus"`.
|
||||
- Keep `timeout_seconds: int = 10`.
|
||||
- Secret schema: replace any existing secret with `grafana_api_key` (required, encrypted).
|
||||
2. **`widgets/prometheus_range.py`** — add `normalize_grafana_frames(raw) -> list[series]` alongside the existing `normalize_prometheus_matrix`. Recovered from `65bae95`, refactored to share the label-dedup rule with the matrix normalizer.
|
||||
3. **`widgets/sources.py`** — rename `PrometheusWidgetSource` → `MetricSource` (internal class; service-type key stays `"prometheus"` in `SERVICE_ADAPTERS`). Each of `_fetch_chart`, `_fetch_gauge`, `_fetch_mean`, and the instant-`metric` path switches from `/api/v1/query[_range]` to `POST {grafana_url}/api/ds/query` with body:
|
||||
|
||||
```json
|
||||
{"queries": [{"datasource": {"uid": "<datasource_uid>", "type": "prometheus"},
|
||||
"expr": "<promql>", "format": "time_series",
|
||||
"intervalMs": <step_ms>, "maxDataPoints": <pts>, "refId": "A"}],
|
||||
"from": "<window_start>", "to": "now"}
|
||||
```
|
||||
|
||||
Then normalize via `normalize_grafana_frames`. Window presets (1h/6h/24h/7d) and step derivation (Change A's `step_for_window`) map to Grafana's `from`/`intervalMs`.
|
||||
4. **`routers/monitoring.py`** — `get_prometheus_status` runs `up` through the gateway; success → `{up: true, version: <grafana-prom-datasource-implication>}`; failure → specific message (auth, unreachable, datasource not found).
|
||||
5. **Tests** — update existing tests: the chart/gauge/mean adapter tests now assert a `POST /api/ds/query` call (mocked) instead of a direct Prom GET. Reuse the same `{series}` / `{value}` assertions.
|
||||
|
||||
### 5.2 Frontend
|
||||
|
||||
1. **`git mv` widget files** (history-preserving): `PrometheusChartWidget.tsx` → `MetricChartWidget.tsx`, `PrometheusGaugeWidget.tsx` → `MetricGaugeWidget.tsx`, `PrometheusMeanWidget.tsx` → `MetricMeanWidget.tsx`, plus their test files. Rename the exported components. The recharts/`LineSeriesChart`/gauge-band rendering is preserved unchanged.
|
||||
2. **`integrations/registry.ts`** — the `prometheus` service binding keeps widget kinds `chart`/`gauge`/`mean`/`metric` but points at the renamed `Metric*` components. No user-visible change.
|
||||
3. **Service config form (`ServiceConfigFields`)** — automatically reflects the new schema (grafana_url, datasource_uid) since it renders from the backend-provided `config_schema`. No bespoke UI work beyond relabeling the secret field.
|
||||
4. **Types** — `PrometheusStatus` type stays; possibly gains a `gateway` field noting the path. Minimal change.
|
||||
|
||||
### 5.3 Canonical spec impact — first non-additive sync
|
||||
|
||||
The `prometheus-charting` canonical domain (`openspec/specs/prometheus-charting/spec.md`) was created by Change A with requirements stating direct Prom querying (SC-101, SC-102, SC-104, etc.). This change **MODIFIES** those requirements: the data source becomes Grafana's gateway, not direct Prom. This is the first non-additive canonical sync in the project — `sdd-sync` will need a `## MODIFIED Requirements` delta rather than `## ADDED`. Flagged for the sync phase; the requirements' *intent* (multi-series chart, gauge bands, mean over window) is unchanged, only the *mechanism* (transport) changes.
|
||||
|
||||
## 6. Success Criteria / Acceptance Criteria
|
||||
|
||||
1. A `prometheus` service configured with Grafana gateway fields (URL + API key + datasource UID) successfully powers `chart`/`gauge`/`mean`/`metric` widgets — no direct Prom network call is made.
|
||||
2. The chart widget renders multi-series line charts with the same look as today (recharts via `LineSeriesChart`).
|
||||
3. The gauge widget renders threshold bands; the mean widget renders the windowed mean — both sourced via Grafana.
|
||||
4. The instant `metric` widget returns a scalar via Grafana.
|
||||
5. `get_prometheus_status` validates the full gateway path; a broken Grafana or unreachable datasource produces a specific error message.
|
||||
6. Widgets are renamed to `MetricChartWidget` / `MetricGaugeWidget` / `MetricMeanWidget`; no `Prometheus*Widget` names remain in frontend source; tests renamed consistently.
|
||||
7. The `prometheus-charting` canonical domain's transport-related requirements are MODIFIED to reflect the Grafana gateway; durable requirements (chart/gauge/mean behavior) are preserved.
|
||||
8. The `service-credential-tester` proposal's Prom test routine is updated to test the gateway path.
|
||||
9. CHANGELOG documents the operator migration (reconfigure `prometheus` services with Grafana gateway fields).
|
||||
10. `pytest`, `npm run build`, `npm run lint`, `npm run test` all green.
|
||||
|
||||
## 7. Risks and Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| **Reintroducing complexity that Change A removed.** The frames→series normalization was deleted for a reason. | Recover the known-good implementation from `65bae95` rather than rewriting; refactor into `prometheus_range.py` so matrix + frames normalizers share the label-dedup rule. |
|
||||
| **Grafana `/api/ds/query` response shape varies across Grafana versions.** | The recovered normalizer already handles the frames schema defensively (displayName → labels → field name fallback). Add a test fixture from a current Grafana version. |
|
||||
| **Existing `prometheus` service instances break silently** (their `base_url` config no longer exists). | SC-9 CHANGELOG note; startup config validation rejects the old shape with a clear migration message. |
|
||||
| **First non-additive canonical sync** — `sdd-sync` MODIFIED delta could mismatch. | Author the delta carefully; match requirement IDs exactly; verify with `sdd-status` before archive. |
|
||||
| **Credential-tester proposal dependency.** | Update it in the same change (or sequence this change before the credential tester's apply). |
|
||||
| **Operators with a directly-reachable Prom lose that option.** | Explicit non-goal; documented. If needed later, add a `direct_url` optional field as a separate change. |
|
||||
| **Review budget (>400 lines).** | Likely 2 slices: (S1) backend config + adapter + frames normalizer + tests; (S2) frontend widget renames + registry + status check. |
|
||||
|
||||
## 8. Resolved Questions (no question round needed)
|
||||
|
||||
- **Q1 — Keep `prometheus` service type?** Yes. It remains the user-facing logical service; only its config/secret schema and adapter internals change.
|
||||
- **Q2 — Restore `grafana` service type?** No. Grafana is the transport, not a service. No deep-link widgets, no status surface.
|
||||
- **Q3 — Widget naming?** Neutral `Metric*` (MetricChartWidget / MetricGaugeWidget / MetricMeanWidget). Decouples widget identity from transport; survives future source changes.
|
||||
- **Q4 — Direct Prom path kept as fallback?** No. Removed. If a future topology needs it, separate change.
|
||||
- **Q5 — Status check approach?** Run `up` through the gateway; validates the full path in one call.
|
||||
|
||||
## 9. Future Phases
|
||||
|
||||
1. **Optional `direct_url` fallback** on the `prometheus` service for topologies where Prom IS directly reachable (skip the Grafana hop).
|
||||
2. **Multi-datasource support** — one Grafana gateway, many datasources (Loki, InfluxDB) selectable per widget.
|
||||
3. **Datasource health breakdown** — status check distinguishes "Grafana down" vs "Prom datasource down" via Grafana's `/api/datasources/health`.
|
||||
|
||||
---
|
||||
|
||||
## Notes for downstream phases
|
||||
|
||||
- **Dependency:** the `service-credential-tester` proposal must be updated when this change's design is firm (its Prom test routine changes from direct Prom to gateway query).
|
||||
- **Canonical first:** this is the project's first non-additive canonical sync (`prometheus-charting` MODIFIED requirements). The sync phase should be deliberate.
|
||||
- **Recovery source:** the frames→series normalizer is known-good at git commit `65bae95` (`backend/src/media_library_viewer_api/widgets/sources.py`, `_fetch_chart`). Restore from there, don't rewrite.
|
||||
- **Honest scope note:** this partially reverts `prometheus-direct-charting` (2026-07-08). That change's direct-Prom path and Grafana removal are undone; its gauge/mean/LineSeriesChart/rendering work is preserved. The reversal is justified by the new network constraint (Q1, §1).
|
||||
Reference in New Issue
Block a user