spec(grafana-metric-gateway): sync — FIRST non-additive (MODIFIED) canonical
16 MODIFIED prometheus-charting requirements (transport: direct Prom -> Grafana gateway; intent preserved where applicable), 3 ADDED (SC-128 gateway status, SC-129 startup validation, SC-130 sanctioned transport), 11 PRESERVED, 0 REMOVED. All 16 MODIFIED headers matched canonical exactly. Post-sync: 30 requirements. web-ui + service-storage canonicals untouched.
This commit is contained in:
Developer
@@ -1,43 +1,51 @@
|
||||
# Prometheus Charting
|
||||
|
||||
> Domain: `prometheus-charting` · **Canonical specification.** Synced from change `prometheus-direct-charting`.
|
||||
> Domain: `prometheus-charting` · **Canonical specification.** Created by `prometheus-direct-charting`;
|
||||
> transport model modified by `grafana-metric-gateway` (Grafana gateway is the sanctioned
|
||||
> `prometheus` query path; chart/gauge/mean widget components renamed to neutral `Metric*`).
|
||||
>
|
||||
> This is the merged end-state of cutting Grafana out of the metric-visualization path and making
|
||||
> Prometheus the direct data source. It captures the durable, post-change contracts for in-app
|
||||
> Prometheus-backed metric rendering, not the per-slice delivery strategy (which remains on record
|
||||
> in the change's `spec.md` / `tasks.md` under `openspec/changes/prometheus-direct-charting/`).
|
||||
> This is the merged end-state of Manage's metric-visualization surface. The `prometheus` service is
|
||||
> the sole metric surface; its `chart` / `gauge` / `mean` / `metric` widget queries route through a
|
||||
> **Grafana gateway** (`POST {grafana_url}/api/ds/query`) — Prometheus remains the logical source,
|
||||
> Grafana is the transport. In-app charting via `recharts` is the sanctioned renderer; no `grafana`
|
||||
> *service type* (or its widgets/status/UI surfaces) is reintroduced; orphaned Grafana widget
|
||||
> instances degrade gracefully; and operators reconfigure `prometheus` services for the gateway (no
|
||||
> silent auto-migration). This spec is acceptance-focused and verifiable; it deliberately does not
|
||||
> prescribe implementation.
|
||||
|
||||
## Purpose
|
||||
|
||||
Define WHAT must be true of Manage's metric-visualization surface after the change: Prometheus is
|
||||
queried **directly** (no Grafana middleman) to back a multi-series line **chart**, a scalar
|
||||
**gauge**, and a windowed **mean**, alongside the existing instant-query numeric **metric** widget.
|
||||
In-app charting via `recharts` is the sanctioned renderer; the `grafana` service type, its widgets,
|
||||
status endpoint, and UI surfaces are fully removed; orphaned Grafana widget instances degrade
|
||||
gracefully; and operators migrate by recreating services/widgets (no silent data migration). This
|
||||
spec is acceptance-focused and verifiable; it deliberately does not prescribe implementation.
|
||||
Define WHAT must be true of Manage's metric-visualization surface after the `grafana-metric-gateway`
|
||||
change: metric queries for the `prometheus` service route through a **Grafana gateway**
|
||||
(`POST {grafana_url}/api/ds/query`) to back a multi-series line **chart**, a scalar **gauge**, and a
|
||||
windowed **mean**, alongside the instant-query numeric **metric** widget — Prometheus remains the
|
||||
logical source, Grafana is the transport. In-app charting via `recharts` is the sanctioned renderer;
|
||||
no `grafana` *service type* (or its widgets/status/UI surfaces) is reintroduced; orphaned Grafana
|
||||
widget instances degrade gracefully; and operators reconfigure `prometheus` services for the gateway
|
||||
(no silent auto-migration). This spec is acceptance-focused and verifiable; it deliberately does not
|
||||
prescribe implementation.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: SC-101 — Prometheus range query returns the existing series shape
|
||||
|
||||
When a `prometheus` widget of kind `chart` is fetched, the backend MUST query `{base_url}/api/v1/query_range` with `query`, `start`, `end`, and `step` derived from the widget config, and return a payload of shape `{ "series": [{ "label": str, "points": [{ "t": int, "v": float|null }] }] }` — the exact shape the frontend chart renderer already consumes.
|
||||
The `prometheus` service type's config MUST point at a Grafana gateway — declaring `grafana_url` (http(s)-schema-enforced), `datasource_uid` (default `"prometheus"`), and `timeout_seconds` — with a required, Fernet-encrypted `grafana_api_key` secret. The former direct-Prometheus `base_url` config field and the old optional `api_key` secret MUST be gone. When a `prometheus` widget of kind `chart` is fetched, the backend MUST issue `POST {grafana_url}/api/ds/query` (carrying `Authorization: Bearer {grafana_api_key}` and a `queries[0]` entry keyed by `datasource_uid`), MUST NOT issue any direct `GET /api/v1/query_range` against a Prometheus URL, and MUST return a payload of shape `{ "series": [{ "label": str, "points": [{ "t": int, "v": float|null }] }] }` — the exact shape the frontend chart renderer already consumes.
|
||||
|
||||
### Requirement: SC-102 — Series label normalization is shared and Prometheus-native
|
||||
|
||||
The metric-label → readable-label normalization MUST live in a single shared helper (not duplicated in a Grafana path) and MUST produce meaningful labels for Prometheus matrix results, including deduplicating repeated labels via a `label (n)` suffix.
|
||||
The metric-label → readable-label normalization MUST live in a single shared helper module that holds **both** normalizers — the retained Prom-matrix normalizer (`normalize_prometheus_matrix`) and the (re-)added Grafana-frames normalizer (`normalize_grafana_frames`) — and MUST deduplicate repeated labels via a `label (n)` suffix using **one shared dedup rule** (not duplicated across the two normalizers). Both normalizers produce the same `{label, points}` series shape, and the frames normalizer MUST apply the label fallback chain (displayName → sorted Prometheus labels excluding `__`-prefixed keys → `"value"`).
|
||||
|
||||
### Requirement: SC-103 — Range query errors degrade gracefully
|
||||
|
||||
A Prometheus timeout, connection error, or non-2xx response MUST cause the widget data fetch to return `{ "error": str }` (not raise), so the frontend renders the standard per-widget error state and the rest of the dashboard remains functional.
|
||||
A Grafana gateway timeout, connection error, HTTP 401/403 (auth), datasource-not-found, or non-2xx response MUST cause the widget data fetch to return `{ "error": str }` (not raise), so the frontend renders the standard per-widget error state and the rest of the dashboard remains functional. The error message MUST be specific enough to distinguish auth failure from unreachable-host from datasource-not-found where the Grafana response permits.
|
||||
|
||||
### Requirement: SC-104 — Step is derived from the window preset
|
||||
|
||||
Given a window preset (1h / 6h / 24h / 7d), the backend MUST derive a `step` that yields a reasonable number of points (target ~100–300 points). Users do not configure `step` directly.
|
||||
Given a window preset (1h / 6h / 24h / 7d), the backend MUST reuse the existing `WINDOW_PRESETS` and `step_for_window` math to derive the gateway request's `intervalMs` (`step * 1000`), `maxDataPoints`, and `from`/`to` time bounds, landing the resulting point count in the same ~100–300 band as the pre-change direct-Prom path. Users do not configure `from`/`to`/`step`/`intervalMs` directly.
|
||||
|
||||
### Requirement: SC-105 — Chart widget moves from grafana to prometheus
|
||||
|
||||
A widget kind named `chart` MUST be bound to the `prometheus` service type in both the backend registry and the frontend `SERVICE_REGISTRY`. The `grafana` service type MUST NOT offer a `chart` kind.
|
||||
Widget kinds `chart`, `gauge`, `mean`, and `metric` MUST remain bound to the `prometheus` service type — the kind strings are persisted in widget instance rows and MUST NOT change. The `grafana` service type MUST NOT be reintroduced and MUST NOT offer any widget kind. The backend adapter (internal class `MetricSource`, registry key still `"prometheus"`) and the frontend `SERVICE_REGISTRY` bind these kinds to the neutral `Metric*` components for `chart`/`gauge`/`mean` (and `PrometheusMetricWidget` for `metric`); all four kinds source their data via the Grafana gateway.
|
||||
|
||||
### Requirement: SC-106 — Chart renderer is reused unchanged
|
||||
|
||||
@@ -45,7 +53,7 @@ The recharts rendering (line chart, multi-series, axes, tooltip, `mergeSeries`,
|
||||
|
||||
### Requirement: SC-107 — Chart supports multiple series
|
||||
|
||||
The `chart` widget MUST render all series returned by the range query, each as its own line with a distinct color. There is no single-series restriction on `chart`.
|
||||
The `chart` widget MUST render all series returned by the gateway range query, each as its own line with a distinct color (via `normalize_grafana_frames`). There is no single-series restriction on `chart`.
|
||||
|
||||
### Requirement: SC-108 — Chart window is a preset
|
||||
|
||||
@@ -53,7 +61,7 @@ The `chart` widget config MUST expose the time window as a preset selector (`1h`
|
||||
|
||||
### Requirement: SC-109 — Gauge renders an instant scalar
|
||||
|
||||
A widget kind named `gauge` MUST be bound to the `prometheus` service. Its data fetch MUST run an instant PromQL query (`/api/v1/query`) and return the scalar result for rendering as a gauge.
|
||||
A widget kind named `gauge` MUST be bound to the `prometheus` service. Its data fetch MUST run an instant PromQL query through the Grafana gateway (a gateway query with a minimal window / single data point) and return the scalar result for rendering as a gauge.
|
||||
|
||||
### Requirement: SC-110 — Gauge supports configurable threshold bands
|
||||
|
||||
@@ -61,11 +69,11 @@ The `gauge` widget config MUST accept optional threshold values (e.g. `warn_at`,
|
||||
|
||||
### Requirement: SC-111 — Gauge is scalar-only
|
||||
|
||||
The `gauge` widget MUST render exactly one scalar value. If the instant query returns multiple series, the adapter MUST return `{ "error": str }` (not silently pick one), directing the user to refine the PromQL.
|
||||
The `gauge` widget MUST render exactly one scalar value. If the gateway query returns multiple series, the adapter MUST return `{ "error": str }` (not silently pick one), directing the user to refine the PromQL.
|
||||
|
||||
### Requirement: SC-112 — Mean computes client-side over a window
|
||||
|
||||
A widget kind named `mean` MUST be bound to the `prometheus` service. Its data fetch MUST run a range query over the configured window preset and return the arithmetic mean of all non-null point values as a single scalar.
|
||||
A widget kind named `mean` MUST be bound to the `prometheus` service. Its data fetch MUST run a range query through the Grafana gateway over the configured window preset and return the arithmetic mean of all non-null point values of the single series as a single scalar.
|
||||
|
||||
### Requirement: SC-113 — Mean uses plain PromQL + window preset
|
||||
|
||||
@@ -73,15 +81,15 @@ The `mean` widget config MUST accept a plain PromQL expression (no requirement t
|
||||
|
||||
### Requirement: SC-114 — Mean is scalar-only
|
||||
|
||||
The `mean` widget MUST render exactly one scalar value. If the range query returns multiple series, the adapter MUST return `{ "error": str }` (not silently aggregate across series).
|
||||
The `mean` widget MUST render exactly one scalar value. If the gateway range query returns multiple series, the adapter MUST return `{ "error": str }` (not silently aggregate across series).
|
||||
|
||||
### Requirement: SC-115 — No grafana references in backend source
|
||||
|
||||
After the change, `grep -ri grafana backend/src --include='*.py'` MUST return no matches (excluding comments/changelog that are explicitly about the removal, if any are retained — but ideally zero).
|
||||
Grafana is the sanctioned **transport** for the `prometheus` service, so backend source legitimately contains Grafana transport references (e.g. `grafana_url`, `grafana_api_key`, `normalize_grafana_frames`, `POST …/api/ds/query`). After the change, those references are transport-only: there MUST be no `grafana` **service type**, no `grafana` service definition, and no `grafana` entry in the backend `SERVICE_DEFINITIONS` / `SERVICE_ADAPTERS` registries. The `integrations/grafana.py` service-type module stays deleted.
|
||||
|
||||
### Requirement: SC-116 — No grafana references in frontend source
|
||||
|
||||
After the change, `grep -ri grafana frontend/src` MUST return no matches, **excluding** (a) test fixtures where "Grafana" appears as a user-authored dashboard *shortcut label* unrelated to the grafana service type (e.g. `Dashboard.test.tsx`), and (b) `LinksTab.tsx` / `service-tabs/index.ts` lines that are themselves being deleted as part of SC-118. (Source finding: `ObservabilityPage.tsx` was refactored into `service-tabs/`.)
|
||||
Grafana transport is allowed in the frontend — the `prometheus` service config form renders Grafana gateway fields (`grafana_url`, `datasource_uid`) from the backend-provided schema. No frontend code MUST reintroduce a `grafana` **service type**, a Grafana deep-link / dashboard widget, `LinksTab`, or any Grafana-status UI surface. (Test fixtures where "Grafana" appears as a user-authored dashboard *shortcut label* unrelated to a service type remain excluded.)
|
||||
|
||||
### Requirement: SC-117 — Grafana service type is gone from registries
|
||||
|
||||
@@ -101,11 +109,11 @@ All Grafana-specific tests (backend and frontend) MUST be deleted; no test refer
|
||||
|
||||
### Requirement: SC-121 — config.yaml matches implementation
|
||||
|
||||
`openspec/config.yaml` MUST NOT contain the stale claims "Do NOT re-implement charting in-app" or "No recharts/d3 is in use." It MUST reflect that in-app charting via `recharts` is the sanctioned approach for Prometheus-backed series, and MUST NOT reference Grafana as a chart path.
|
||||
`openspec/config.yaml` MUST reflect that metric queries for the `prometheus` service are routed through a Grafana gateway (`POST /api/ds/query`) and that in-app charting via `recharts` is the sanctioned renderer. It MUST NOT carry the stale claims "Do NOT re-implement charting in-app", "No recharts/d3 is in use", or "Prometheus is queried directly (no Grafana middleman)".
|
||||
|
||||
### Requirement: SC-122 — CHANGELOG documents the migration
|
||||
|
||||
`CHANGELOG.md` MUST include an entry instructing operators to delete existing Grafana service instances and recreate them as Prometheus services, noting that configured `grafana/chart` widgets must be recreated as `prometheus/chart` widgets.
|
||||
`CHANGELOG.md` `[Unreleased]` MUST include a **BREAKING** entry stating that existing `prometheus` service instances must be reconfigured: replace the old `base_url` with `grafana_url`, add the `grafana_api_key` secret, and optionally set `datasource_uid`, because metric queries now route through Grafana's `POST /api/ds/query`.
|
||||
|
||||
### Requirement: SC-123 — Backend tests pass
|
||||
|
||||
@@ -117,12 +125,24 @@ All Grafana-specific tests (backend and frontend) MUST be deleted; no test refer
|
||||
|
||||
### Requirement: SC-125 — New widget kinds have tests
|
||||
|
||||
`PrometheusChartWidget`, `PrometheusGaugeWidget`, and `PrometheusMeanWidget` MUST each have a frontend test covering at least: loading state, error state, and a rendered data case.
|
||||
The renamed widgets `MetricChartWidget`, `MetricGaugeWidget`, and `MetricMeanWidget` MUST each have a frontend test covering at least: loading state, error state, and a rendered data case. (The `metric`-kind widget `PrometheusMetricWidget` is not renamed but remains tested.)
|
||||
|
||||
### Requirement: SC-126 — No silent data migration
|
||||
|
||||
The change MUST NOT attempt to auto-migrate existing `grafana` service rows into `prometheus` rows (URLs differ; true migration is impossible). Migration is operator-driven per the CHANGELOG note.
|
||||
The change MUST NOT auto-migrate persisted `prometheus` service instances from the old `base_url` config shape to the new `grafana_url` gateway shape (URLs and credentials differ; a safe automated rewrite is not possible). Migration is operator-driven per the CHANGELOG note; old-shape instances are flagged on startup (see SC-129) and degrade at runtime to a gateway error until reconfigured.
|
||||
|
||||
### Requirement: SC-127 — Non-blocking on the service-storage-harness change
|
||||
|
||||
This change MUST NOT depend on the `service-storage-harness` change. It is independently buildable, testable, and deployable. (The reverse dependency holds: the qBit speed widget depends on this change's chart capability.)
|
||||
|
||||
### Requirement: SC-128 — get_prometheus_status validates the full gateway path
|
||||
|
||||
`get_prometheus_status` MUST validate the full metric path by running a trivial query (e.g. `up`) through the Grafana gateway (`POST {grafana_url}/api/ds/query`). Success MUST return `{ "up": true, "version": <str>, … }` — a sentinel such as `"ok"` is acceptable when Grafana's `/api/ds/query` response carries no Prometheus build-info. Failure MUST return `{ "up": false, "error": <specific str> }` distinguishing auth failure (HTTP 401/403), unreachable host, and datasource/gateway error where the response permits.
|
||||
|
||||
### Requirement: SC-129 — Startup logs old-shape prometheus config without crashing
|
||||
|
||||
When the backend starts and a persisted `prometheus` service instance carries the old `base_url` config key without `grafana_url`, startup MUST log a clear migration message naming the service and instructing reconfiguration, and MUST complete without crashing (it MUST NOT delete, disable, or otherwise mutate the instance). The instance's widget fetches degrade to a gateway error until the operator reconfigures it.
|
||||
|
||||
### Requirement: SC-130 — Grafana is the sanctioned transport for the prometheus service
|
||||
|
||||
The `prometheus` service type is the sole metric surface in Manage. Its metric queries (`chart`, `gauge`, `mean`, `metric`) are sanctioned to route through a Grafana gateway — `POST {grafana_url}/api/ds/query` with a `grafana_api_key` and a Prometheus `datasource_uid`. Grafana is the transport, not a first-class service type; a direct-Prometheus HTTP path is not supported.
|
||||
|
||||
Reference in New Issue
Block a user