Developer
bf8de32815
Gateway transport: prometheus service config gains grafana_url/api_key/ datasource_uid; all queries via POST /api/ds/query; frames->series restored. First non-additive canonical sync: MODIFIES 17 SC- requirements (transport changes, intent preserved), PRESERVES 11, ADDS 3 (gateway status, startup validation, sanctioned-transport statement). 3 spec assumptions settled where proposal was silent.
306 lines
18 KiB
Markdown
306 lines
18 KiB
Markdown
# SDD Spec: Grafana Metric Gateway
|
||
|
||
**Change:** `grafana-metric-gateway`
|
||
**Phase:** spec
|
||
**Date:** 2026-07-09
|
||
|
||
This spec defines the acceptance requirements for the change. Requirements use the `GM-` prefix (the `prometheus-charting` canonical domain, created by `prometheus-direct-charting`, uses `SC-`; this change supersedes several of those — see §"Canonical delta intent").
|
||
|
||
## Requirement categories
|
||
|
||
1. Gateway transport (config + query path)
|
||
2. Frames → series normalization
|
||
3. Window presets + step mapping
|
||
4. Widget kind behavior (chart / gauge / mean / metric)
|
||
5. Status check via gateway
|
||
6. Widget rename to neutral `Metric*`
|
||
7. Startup validation + migration
|
||
8. Test + build greenness
|
||
|
||
---
|
||
|
||
## 1. Gateway transport (config + query path)
|
||
|
||
### GM-101 — Prometheus service config points at a Grafana gateway
|
||
|
||
The `prometheus` service type's config model MUST declare `grafana_url: ServiceBaseUrl` (http(s)-schema-enforced), `datasource_uid: str = "prometheus"`, and `timeout_seconds: int = 10`. The former `base_url` field MUST be removed. The secret schema MUST declare `grafana_api_key` (required, encrypted via Fernet) and MUST NOT carry the old optional `api_key`.
|
||
|
||
#### Scenario: Config schema reflects gateway fields
|
||
|
||
- GIVEN the `grafana-metric-gateway` change is applied
|
||
- WHEN `integrations/prometheus.py` `PrometheusConfig` is inspected
|
||
- THEN it declares `grafana_url`, `datasource_uid`, `timeout_seconds`
|
||
- AND it does NOT declare `base_url`
|
||
- AND `DEFINITION.secret_fields` contains `grafana_api_key` (required=True)
|
||
- AND `DEFINITION.secret_fields` does NOT contain `api_key`
|
||
|
||
### GM-102 — All metric queries route through Grafana /api/ds/query
|
||
|
||
The widget source adapter (renamed `MetricSource`, service-type key stays `"prometheus"`) MUST issue `POST {grafana_url}/api/ds/query` for all four widget kinds (`chart`, `gauge`, `mean`, `metric`). The request body MUST contain a `queries` array with `{datasource: {uid: <datasource_uid>, type: "prometheus"}, expr: <promql>, format: "time_series", intervalMs: <step_ms>, maxDataPoints: <pts>, refId: "A"}` plus `from`/`to` time bounds. The request MUST carry `Authorization: Bearer {grafana_api_key}`. The adapter MUST NOT issue any direct `GET /api/v1/query` or `GET /api/v1/query_range` call to a Prometheus URL.
|
||
|
||
#### Scenario: Chart fetch uses gateway POST
|
||
|
||
- GIVEN a `prometheus` service configured with `grafana_url`, `grafana_api_key`, `datasource_uid`
|
||
- WHEN a `chart` widget data fetch is executed (mocked HTTP)
|
||
- THEN the adapter issues `POST {grafana_url}/api/ds/query`
|
||
- AND the request body contains `queries[0].datasource.uid == datasource_uid`
|
||
- AND the request body contains `queries[0].expr == <promql>`
|
||
- AND the `Authorization` header is `Bearer {grafana_api_key}`
|
||
- AND no `GET /api/v1/query_range` call is made
|
||
|
||
#### Scenario: No direct Prom HTTP call
|
||
|
||
- GIVEN the change is applied
|
||
- WHEN `widgets/sources.py` `MetricSource` is inspected
|
||
- THEN no code path constructs a URL matching `/api/v1/query` or `/api/v1/query_range` against a Prometheus `base_url`
|
||
|
||
### GM-103 — Gateway errors degrade gracefully
|
||
|
||
A Grafana timeout, connection error, HTTP 401/403 (auth), or non-2xx response MUST cause the widget data fetch to return `{ "error": str }` (not raise), so the frontend renders the standard per-widget error state and the rest of the dashboard remains functional. The error message MUST be specific enough to distinguish auth failure from unreachable-host from datasource-not-found where the Grafana response permits.
|
||
|
||
#### Scenario: Auth failure surfaces a readable error
|
||
|
||
- GIVEN a `prometheus` service with an invalid `grafana_api_key`
|
||
- WHEN a widget data fetch is executed (mocked HTTP returns 401)
|
||
- THEN the adapter returns `{ "error": <str mentioning auth or 401> }`
|
||
- AND no exception propagates
|
||
|
||
#### Scenario: Timeout surfaces a readable error
|
||
|
||
- GIVEN a `prometheus` service whose Grafana gateway does not respond within the timeout
|
||
- WHEN a widget data fetch is executed
|
||
- THEN the adapter returns `{ "error": <str mentioning timeout> }`
|
||
|
||
---
|
||
|
||
## 2. Frames → series normalization
|
||
|
||
### GM-104 — Frames normalizer restored and shared
|
||
|
||
A `normalize_grafana_frames(raw) -> list[dict]` helper MUST exist in `widgets/prometheus_range.py` alongside the existing `normalize_prometheus_matrix`. It MUST parse Grafana's `/api/v1/ds/query` response (`results.<refId>.frames[]` with `data.values` + `schema.fields`) and produce the SAME `{label, points:[{t:int, v:float|null}]}` series shape the frontend chart renderer already consumes. The label-dedup rule (`label (n)` suffix on collision) MUST be shared with `normalize_prometheus_matrix` (extracted to a common inner helper, not duplicated).
|
||
|
||
#### Scenario: Frames normalized to series shape
|
||
|
||
- GIVEN a sample Grafana `/api/ds/query` response with one frame containing timestamps `[1000, 2000]` and values `[1.5, 2.5]`
|
||
- WHEN `normalize_grafana_frames(raw)` is called
|
||
- THEN it returns `[{"label": <str>, "points": [{"t": 1000, "v": 1.5}, {"t": 2000, "v": 2.5}]}]`
|
||
|
||
#### Scenario: Label fallback chain
|
||
|
||
- GIVEN a frame whose value-field has `config.displayName` set
|
||
- WHEN normalized
|
||
- THEN the label is the displayName
|
||
- GIVEN a frame whose value-field has Prometheus `labels` but no displayName
|
||
- WHEN normalized
|
||
- THEN the label is the sorted `k=v` join (excluding `__`-prefixed keys)
|
||
- GIVEN a frame with neither displayName nor labels
|
||
- WHEN normalized
|
||
- THEN the label is `"value"`
|
||
|
||
#### Scenario: Dedup shared with matrix path
|
||
|
||
- GIVEN two frames that would produce the same label
|
||
- WHEN normalized
|
||
- THEN the second gets a `(1)` suffix
|
||
- AND the dedup logic is not copy-pasted (a shared helper or shared suffix rule is used by both `normalize_grafana_frames` and `normalize_prometheus_matrix`)
|
||
|
||
---
|
||
|
||
## 3. Window presets + step mapping
|
||
|
||
### GM-105 — Window presets map to Grafana from/to + intervalMs
|
||
|
||
The existing `WINDOW_PRESETS` (`1h`/`6h`/`24h`/`7d` → seconds) and `step_for_window` math MUST be reused to derive the gateway request's `from` (e.g. `"now-1h"`), `to` (`"now"`), and `intervalMs` (`step * 1000`). The resulting point count MUST land in the 100–300 band (same target as the pre-change direct-Prom path). Users do not configure `from`/`to`/`step`/`intervalMs` directly.
|
||
|
||
#### Scenario: 1h preset maps correctly
|
||
|
||
- GIVEN a `chart` widget with `window: "1h"`
|
||
- WHEN the gateway request body is constructed
|
||
- THEN `from` resolves to approximately `now - 3600s`
|
||
- AND `to` is `"now"`
|
||
- AND `intervalMs` is `step_for_window(3600) * 1000`
|
||
- AND the expected point count is ~200
|
||
|
||
---
|
||
|
||
## 4. Widget kind behavior (chart / gauge / mean / metric)
|
||
|
||
### GM-106 — Chart renders multi-series via gateway
|
||
|
||
The `chart` widget MUST query the gateway with a range query (PromQL expr + window preset) and return `{ "series": [...] }` via `normalize_grafana_frames`. The frontend `MetricChartWidget` MUST render all returned series via `LineSeriesChart`, each as its own line with a distinct color. There is no single-series restriction.
|
||
|
||
#### Scenario: Multi-series chart through gateway
|
||
|
||
- GIVEN a `chart` widget whose PromQL returns two series
|
||
- WHEN data is fetched via the gateway (mocked)
|
||
- THEN the adapter returns `{ "series": [<2 entries>] }`
|
||
- AND the frontend renders two distinct lines
|
||
|
||
### GM-107 — Gauge renders instant scalar via gateway
|
||
|
||
The `gauge` widget MUST query the gateway with an instant PromQL query (mapped to a Grafana instant datasource query or a range query with a single point) and return `{ "value": float, "warn_at"?, "crit_at"?, "min"?, "max"?, "unit"? }`. If the query returns multiple series, the adapter MUST return `{ "error": str }` (scalar-only). The frontend `MetricGaugeWidget` MUST render threshold bands when `warn_at`/`crit_at` are set; otherwise a single neutral gauge.
|
||
|
||
#### Scenario: Gauge scalar-only through gateway
|
||
|
||
- GIVEN a `gauge` widget whose PromQL returns one scalar
|
||
- WHEN fetched via the gateway
|
||
- THEN the adapter returns `{ "value": <float>, ... }`
|
||
- GIVEN a `gauge` widget whose PromQL returns multiple series
|
||
- WHEN fetched
|
||
- THEN the adapter returns `{ "error": <str> }`
|
||
|
||
### GM-108 — Mean computes over a gateway range window
|
||
|
||
The `mean` widget MUST query the gateway with a range query over the configured window preset, average all non-null point values of the single series client-side, and return `{ "value": float, "unit"? }`. Scalar-only: multiple series → `{ "error": str }`.
|
||
|
||
#### Scenario: Mean through gateway
|
||
|
||
- GIVEN a `mean` widget with `window: "1h"` and a PromQL returning one series
|
||
- WHEN fetched via the gateway
|
||
- THEN the adapter returns `{ "value": <mean of non-null points> }`
|
||
|
||
### GM-109 — Metric widget returns scalar via gateway
|
||
|
||
The instant `metric` widget MUST query the gateway and return `{ "result": <data> }` (same shape as today) sourced via Grafana rather than direct Prom.
|
||
|
||
#### Scenario: Metric instant query through gateway
|
||
|
||
- GIVEN a `metric` widget with a PromQL expression
|
||
- WHEN fetched
|
||
- THEN the adapter issues a gateway POST (not a direct Prom GET)
|
||
- AND returns the scalar result in the existing `{ "result": ... }` shape
|
||
|
||
---
|
||
|
||
## 5. Status check via gateway
|
||
|
||
### GM-110 — get_prometheus_status validates the full gateway path
|
||
|
||
`get_prometheus_status` MUST run a trivial query (e.g. `up`) through the Grafana gateway. Success → `{ "up": true, "version": <str>, ... }` (version derived from the successful path; if Grafana's response doesn't carry a Prom version, a sentinel like `"ok"` is acceptable). Failure → `{ "up": false, "error": <specific str> }` distinguishing auth failure, unreachable host, and datasource-not-found where the response permits.
|
||
|
||
#### Scenario: Healthy gateway
|
||
|
||
- GIVEN a `prometheus` service with valid gateway credentials
|
||
- WHEN `get_prometheus_status` is called
|
||
- THEN it issues a gateway POST with `expr: "up"`
|
||
- AND returns `{ "up": true, ... }`
|
||
|
||
#### Scenario: Auth failure in status
|
||
|
||
- GIVEN a `prometheus` service with an invalid `grafana_api_key`
|
||
- WHEN `get_prometheus_status` is called
|
||
- THEN it returns `{ "up": false, "error": <str mentioning auth> }`
|
||
|
||
---
|
||
|
||
## 6. Widget rename to neutral Metric*
|
||
|
||
### GM-111 — Widget files renamed via git mv
|
||
|
||
`PrometheusChartWidget.tsx` → `MetricChartWidget.tsx`, `PrometheusGaugeWidget.tsx` → `MetricGaugeWidget.tsx`, `PrometheusMeanWidget.tsx` → `MetricMeanWidget.tsx`, plus their test files, MUST be renamed via `git mv` (history preserved). The exported component names MUST change to `MetricChartWidget` / `MetricGaugeWidget` / `MetricMeanWidget`. The rendering code (recharts, `LineSeriesChart`, gauge bands) MUST be preserved unchanged.
|
||
|
||
#### Scenario: No Prometheus*Widget names remain
|
||
|
||
- GIVEN the change is applied
|
||
- WHEN `grep -r "PrometheusChartWidget\|PrometheusGaugeWidget\|PrometheusMeanWidget" frontend/src` is run
|
||
- THEN it returns no matches
|
||
- AND `MetricChartWidget.tsx`, `MetricGaugeWidget.tsx`, `MetricMeanWidget.tsx` exist
|
||
|
||
### GM-112 — Registry binds Metric* components
|
||
|
||
`integrations/registry.ts` MUST bind the `prometheus` service's `chart`/`gauge`/`mean` widget kinds to the renamed `Metric*` components. The widget KIND names (`chart`/`gauge`/`mean`/`metric`) MUST NOT change (they are persisted in widget instance rows).
|
||
|
||
#### Scenario: Registry uses Metric* components
|
||
|
||
- GIVEN the change is applied
|
||
- WHEN `SERVICE_REGISTRY.prometheus.widgets` is inspected
|
||
- THEN the `chart` kind's `component` is `MetricChartWidget`
|
||
- AND the kind strings are still `"chart"`, `"gauge"`, `"mean"`, `"metric"`
|
||
|
||
---
|
||
|
||
## 7. Startup validation + migration
|
||
|
||
### GM-113 — Startup rejects old prometheus config shape
|
||
|
||
When the backend starts and a persisted `prometheus` service instance has a `config` dict containing the old `base_url` key (without `grafana_url`), the startup config validation MUST log a clear migration message (naming the service) and either skip the instance gracefully or mark it as misconfigured — it MUST NOT crash startup.
|
||
|
||
#### Scenario: Old-shape config logged not crashed
|
||
|
||
- GIVEN a persisted `prometheus` service with `config: {"base_url": "http://prom:9090"}` (no `grafana_url`)
|
||
- WHEN the backend starts
|
||
- THEN startup completes (no crash)
|
||
- AND a log message names the service and instructs reconfiguration
|
||
|
||
### GM-114 — CHANGELOG documents the migration
|
||
|
||
`CHANGELOG.md` `[Unreleased]` MUST include an entry stating that `prometheus` service instances must be reconfigured: replace `base_url` with `grafana_url`, add the `grafana_api_key` secret, and optionally set `datasource_uid`. It MUST be marked **BREAKING**.
|
||
|
||
#### Scenario: CHANGELOG entry present
|
||
|
||
- GIVEN the change is applied
|
||
- WHEN `CHANGELOG.md` is inspected
|
||
- THEN an `[Unreleased]` entry mentions `grafana_url`, `grafana_api_key`, `datasource_uid`
|
||
- AND it is marked BREAKING
|
||
|
||
---
|
||
|
||
## 8. Test + build greenness
|
||
|
||
### GM-115 — Backend tests + ruff green
|
||
|
||
`PYTHONPATH=src python3 -m pytest -q` and `PYTHONPATH=src python3 -m ruff check src tests` from `backend/` MUST pass. New/updated tests MUST cover: gateway POST assertion (mocked), `normalize_grafana_frames` (label fallback chain, dedup, NaN handling), gauge/mean scalar-only through gateway, status check through gateway, and the startup old-config validation.
|
||
|
||
### GM-116 — Frontend build + lint + test green
|
||
|
||
`npm run build`, `npm run lint`, and `npx vitest run` from `frontend/` MUST pass (0 errors). The renamed `Metric*` widget tests MUST cover loading, error, and rendered states.
|
||
|
||
---
|
||
|
||
## Canonical delta intent
|
||
|
||
This is the project's **first non-additive canonical sync**. The `prometheus-charting` canonical domain (`openspec/specs/prometheus-charting/spec.md`) has 27 requirements (SC-101..SC-127). This change MODIFIES the transport-related requirements, PRESERVES the durable-behavior requirements, and ADDS new requirements. The sync phase (`sdd-sync`) MUST author a `## MODIFIED Requirements` delta (not `## ADDED` only).
|
||
|
||
### MODIFIED (supersedes a canonical SC-xxx — intent may be preserved or changed)
|
||
|
||
| GM-req | Supersedes | What changes |
|
||
|---|---|---|
|
||
| GM-101 | SC-101 (partial) | Transport: direct Prom `/api/v1/query_range` → Grafana `/api/ds/query`; config `base_url` → `grafana_url` + `datasource_uid` + `grafana_api_key` |
|
||
| GM-102 | SC-101 (partial) | No direct Prom HTTP; all queries via gateway POST |
|
||
| GM-103 | SC-103 | Error semantics preserved (graceful `{error}`), but transport-specific failures (Grafana 401/403, datasource-not-found) are new |
|
||
| GM-104 | SC-102 | Shared helper now holds BOTH `normalize_prometheus_matrix` (kept for qBit) AND `normalize_grafana_frames` (new/restored); dedup shared |
|
||
| GM-105 | SC-104 | Step math reused but mapped to `intervalMs`/`maxDataPoints` instead of Prom `step` |
|
||
| GM-106 | SC-107 | Multi-series intent preserved; transport changes |
|
||
| GM-107 | SC-109, SC-111 | Gauge scalar-only preserved; instant query now via gateway |
|
||
| GM-108 | SC-112, SC-114 | Mean over window preserved; range query now via gateway |
|
||
| GM-109 | SC-105 (partial) | `metric` kind still bound to `prometheus`; transport changes |
|
||
| GM-110 | (new behavior for status) | `get_prometheus_status` now validates gateway path (previously direct Prom health) |
|
||
| GM-111 | SC-125 | Widget names change: `Prometheus*Widget` → `Metric*Widget` |
|
||
| GM-112 | SC-105 (partial) | Registry still binds kinds to `prometheus`; component refs rename |
|
||
| GM-113 | SC-126 (partial) | No auto-migration still holds; startup validation of old shape is new |
|
||
| GM-114 | SC-122 | CHANGELOG migration note changes from "delete grafana, recreate prometheus" to "reconfigure prometheus with gateway fields" |
|
||
| — | SC-115 | **MODIFIED**: "no grafana refs in backend" is no longer fully true — `normalize_grafana_frames` is a grafana-named function. The criterion becomes "no `grafana` SERVICE TYPE refs" (transport refs allowed). |
|
||
| — | SC-116 | **MODIFIED**: same as SC-115 for frontend — grafana transport is allowed; no grafana service type / link widget. |
|
||
| — | SC-117 | **PRESERVED** — no `grafana` service type is reintroduced. Still true. |
|
||
| — | SC-118 | **PRESERVED** — `get_grafana_status` stays removed; no grafana status surface. |
|
||
| — | SC-119 | **PRESERVED** — orphaned widgets still degrade gracefully. |
|
||
| — | SC-120 | **PRESERVED** — no grafana-specific tests reintroduced. |
|
||
| — | SC-121 | **MODIFIED**: `config.yaml` now references Grafana as the sanctioned transport (not "must not reference grafana"). |
|
||
|
||
### PRESERVED (unchanged in canonical — no delta entry needed)
|
||
|
||
SC-106 (chart renderer preserved), SC-108 (chart window preset), SC-110 (gauge threshold bands), SC-113 (mean plain PromQL), SC-117 (no grafana service type), SC-118 (no grafana status), SC-119 (orphan degrade), SC-120 (no grafana tests), SC-123 (backend tests pass), SC-124 (frontend build+lint), SC-127 (independent of service-storage-harness).
|
||
|
||
### ADDS (new canonical requirements — `## ADDED Requirements` in delta)
|
||
|
||
GM-110 (gateway-path status check — new behavior), GM-113 (startup old-config validation — new), and the canonical domain gains an explicit statement that Grafana is the sanctioned transport for the `prometheus` service.
|
||
|
||
---
|
||
|
||
## Assumptions (where the proposal was silent and the spec settles)
|
||
|
||
1. **Grafana instant-query mapping.** Grafana's `/api/ds/query` is inherently a range query. For the `gauge` (instant) and `metric` (instant) paths, the spec assumes the adapter issues a gateway query with a minimal window (e.g. `from=now-1m, to=now, maxDataPoints=1`) and extracts the single latest point, rather than attempting a separate Grafana instant-query endpoint. This mirrors how the pre-Change-A Grafana path handled chart data and is the pragmatic mapping. If Grafana exposes a cleaner instant path, the design phase may refine this — but the requirement (scalar result via gateway) holds either way.
|
||
2. **Version field in status.** Grafana's `/api/ds/query` response does not carry Prometheus build-info. `get_prometheus_status` returns `version: "ok"` (or omits it) on success. A richer version (via Grafana's datasource health endpoint) is a future-phase item, not this change.
|
||
3. **`normalize_prometheus_matrix` retention.** The matrix normalizer is kept (not deleted) because `QbittorrentSampleStore.window()` returns data in a matrix-adjacent shape consumed by `QbittorrentSpeedWidget`. Removing it would break the qBit speed widget. Both normalizers coexist.
|
||
4. **Old `api_key` secret.** The old optional `api_key` secret on the `prometheus` service is removed from the definition. Persisted rows may still carry it in their encrypted secrets blob; it is inert (the adapter reads `grafana_api_key` only) and gets dropped on the next save.
|