manage/openspec/changes/grafana-metric-gateway/specs/prometheus-charting/spec.md

Prometheus Charting — Delta (grafana-metric-gateway)

Change: grafana-metric-gateway · Domain: prometheus-charting · Phase: spec (authored during sdd-sync).

This is the project's first non-additive canonical sync. The prometheus-charting canonical domain (SC-101 … SC-127, created by prometheus-direct-charting) described the direct Prometheus transport. This change switches the prometheus service to a Grafana gateway transport (POST {grafana_url}/api/ds/query), renames the chart/gauge/mean widgets to neutral Metric* names, and validates the gateway path. Durable behavior requirements (multi-series chart, gauge threshold bands, windowed mean) are preserved; only the transport and a few naming/status statements are modified. The mapping of every change requirement (GM-101 … GM-116) to a canonical SC-xxx is recorded in the flat spec.md §"Canonical delta intent".

This delta therefore contains both a ## MODIFIED Requirements block (full replacement bodies for each superseded SC-xxx — matched by exact requirement ID) and an ## ADDED Requirements block (genuinely new requirements assigned fresh canonical IDs SC-128 … SC-130). PRESERVED requirements (SC-106, SC-108, SC-110, SC-113, SC-117 … SC-120, SC-123, SC-124, SC-127) appear in neither section — they are unchanged in the canonical spec.

MODIFIED Requirements

Each block below replaces the full canonical requirement of the same ID (matched by the exact ### Requirement: SC-xxx — <name> header). The requirement intent is preserved where the change only swaps the mechanism (direct Prom → Grafana gateway); the body text is updated to the gateway transport. No requirement IDs are renamed (RENAMED is unsupported by the native delta helper).

Requirement: SC-101 — Prometheus range query returns the existing series shape

The prometheus service type's config MUST point at a Grafana gateway — declaring grafana_url (http(s)-schema-enforced), datasource_uid (default "prometheus"), and timeout_seconds — with a required, Fernet-encrypted grafana_api_key secret. The former direct-Prometheus base_url config field and the old optional api_key secret MUST be gone. When a prometheus widget of kind chart is fetched, the backend MUST issue POST {grafana_url}/api/ds/query (carrying Authorization: Bearer {grafana_api_key} and a queries[0] entry keyed by datasource_uid), MUST NOT issue any direct GET /api/v1/query_range against a Prometheus URL, and MUST return a payload of shape { "series": [{ "label": str, "points": [{ "t": int, "v": float|null }] }] } — the exact shape the frontend chart renderer already consumes.

Requirement: SC-102 — Series label normalization is shared and Prometheus-native

The metric-label → readable-label normalization MUST live in a single shared helper module that holds both normalizers — the retained Prom-matrix normalizer (normalize_prometheus_matrix) and the (re-)added Grafana-frames normalizer (normalize_grafana_frames) — and MUST deduplicate repeated labels via a label (n) suffix using one shared dedup rule (not duplicated across the two normalizers). Both normalizers produce the same {label, points} series shape, and the frames normalizer MUST apply the label fallback chain (displayName → sorted Prometheus labels excluding __-prefixed keys → "value").

Requirement: SC-103 — Range query errors degrade gracefully

A Grafana gateway timeout, connection error, HTTP 401/403 (auth), datasource-not-found, or non-2xx response MUST cause the widget data fetch to return { "error": str } (not raise), so the frontend renders the standard per-widget error state and the rest of the dashboard remains functional. The error message MUST be specific enough to distinguish auth failure from unreachable-host from datasource-not-found where the Grafana response permits.

Requirement: SC-104 — Step is derived from the window preset

Given a window preset (1h / 6h / 24h / 7d), the backend MUST reuse the existing WINDOW_PRESETS and step_for_window math to derive the gateway request's intervalMs (step * 1000), maxDataPoints, and from/to time bounds, landing the resulting point count in the same ~100–300 band as the pre-change direct-Prom path. Users do not configure from/to/step/intervalMs directly.

Requirement: SC-105 — Chart widget moves from grafana to prometheus

Widget kinds chart, gauge, mean, and metric MUST remain bound to the prometheus service type — the kind strings are persisted in widget instance rows and MUST NOT change. The grafana service type MUST NOT be reintroduced and MUST NOT offer any widget kind. The backend adapter (internal class MetricSource, registry key still "prometheus") and the frontend SERVICE_REGISTRY bind these kinds to the neutral Metric* components for chart/gauge/mean (and PrometheusMetricWidget for metric); all four kinds source their data via the Grafana gateway.

Requirement: SC-107 — Chart supports multiple series

The chart widget MUST render all series returned by the gateway range query, each as its own line with a distinct color (via normalize_grafana_frames). There is no single-series restriction on chart.

Requirement: SC-109 — Gauge renders an instant scalar

A widget kind named gauge MUST be bound to the prometheus service. Its data fetch MUST run an instant PromQL query through the Grafana gateway (a gateway query with a minimal window / single data point) and return the scalar result for rendering as a gauge.

Requirement: SC-111 — Gauge is scalar-only

The gauge widget MUST render exactly one scalar value. If the gateway query returns multiple series, the adapter MUST return { "error": str } (not silently pick one), directing the user to refine the PromQL.

Requirement: SC-112 — Mean computes client-side over a window

A widget kind named mean MUST be bound to the prometheus service. Its data fetch MUST run a range query through the Grafana gateway over the configured window preset and return the arithmetic mean of all non-null point values of the single series as a single scalar.

Requirement: SC-114 — Mean is scalar-only

The mean widget MUST render exactly one scalar value. If the gateway range query returns multiple series, the adapter MUST return { "error": str } (not silently aggregate across series).

Requirement: SC-115 — No grafana references in backend source

Grafana is the sanctioned transport for the prometheus service, so backend source legitimately contains Grafana transport references (e.g. grafana_url, grafana_api_key, normalize_grafana_frames, POST …/api/ds/query). After the change, those references are transport-only: there MUST be no grafana service type, no grafana service definition, and no grafana entry in the backend SERVICE_DEFINITIONS / SERVICE_ADAPTERS registries. The integrations/grafana.py service-type module stays deleted.

Requirement: SC-116 — No grafana references in frontend source

Grafana transport is allowed in the frontend — the prometheus service config form renders Grafana gateway fields (grafana_url, datasource_uid) from the backend-provided schema. No frontend code MUST reintroduce a grafana service type, a Grafana deep-link / dashboard widget, LinksTab, or any Grafana-status UI surface. (Test fixtures where "Grafana" appears as a user-authored dashboard shortcut label unrelated to a service type remain excluded.)

Requirement: SC-121 — config.yaml matches implementation

openspec/config.yaml MUST reflect that metric queries for the prometheus service are routed through a Grafana gateway (POST /api/ds/query) and that in-app charting via recharts is the sanctioned renderer. It MUST NOT carry the stale claims "Do NOT re-implement charting in-app", "No recharts/d3 is in use", or "Prometheus is queried directly (no Grafana middleman)".

Requirement: SC-122 — CHANGELOG documents the migration

CHANGELOG.md [Unreleased] MUST include a BREAKING entry stating that existing prometheus service instances must be reconfigured: replace the old base_url with grafana_url, add the grafana_api_key secret, and optionally set datasource_uid, because metric queries now route through Grafana's POST /api/ds/query.

Requirement: SC-125 — New widget kinds have tests

The renamed widgets MetricChartWidget, MetricGaugeWidget, and MetricMeanWidget MUST each have a frontend test covering at least: loading state, error state, and a rendered data case. (The metric-kind widget PrometheusMetricWidget is not renamed but remains tested.)

Requirement: SC-126 — No silent data migration

The change MUST NOT auto-migrate persisted prometheus service instances from the old base_url config shape to the new grafana_url gateway shape (URLs and credentials differ; a safe automated rewrite is not possible). Migration is operator-driven per the CHANGELOG note; old-shape instances are flagged on startup (see SC-129) and degrade at runtime to a gateway error until reconfigured.

ADDED Requirements

Genuinely new requirements with fresh canonical IDs continuing the SC- sequence. The GM- change prefixes are intentionally not reused in the canonical store.

Requirement: SC-128 — get_prometheus_status validates the full gateway path

get_prometheus_status MUST validate the full metric path by running a trivial query (e.g. up) through the Grafana gateway (POST {grafana_url}/api/ds/query). Success MUST return { "up": true, "version": <str>, … } — a sentinel such as "ok" is acceptable when Grafana's /api/ds/query response carries no Prometheus build-info. Failure MUST return { "up": false, "error": <specific str> } distinguishing auth failure (HTTP 401/403), unreachable host, and datasource/gateway error where the response permits.

Requirement: SC-129 — Startup logs old-shape prometheus config without crashing

When the backend starts and a persisted prometheus service instance carries the old base_url config key without grafana_url, startup MUST log a clear migration message naming the service and instructing reconfiguration, and MUST complete without crashing (it MUST NOT delete, disable, or otherwise mutate the instance). The instance's widget fetches degrade to a gateway error until the operator reconfigures it.

Requirement: SC-130 — Grafana is the sanctioned transport for the prometheus service

The prometheus service type is the sole metric surface in Manage. Its metric queries (chart, gauge, mean, metric) are sanctioned to route through a Grafana gateway — POST {grafana_url}/api/ds/query with a grafana_api_key and a Prometheus datasource_uid. Grafana is the transport, not a first-class service type; a direct-Prometheus HTTP path is not supported.