manage/openspec/specs/prometheus-charting/spec.md

Prometheus Charting

Domain: prometheus-charting · Canonical specification. Created by prometheus-direct-charting; transport model modified by grafana-metric-gateway (Grafana gateway is the sanctioned prometheus query path; chart/gauge/mean widget components renamed to neutral Metric*).

This is the merged end-state of Manage's metric-visualization surface. The prometheus service is the sole metric surface; its chart / gauge / mean / metric widget queries route through a Grafana gateway (POST {grafana_url}/api/ds/query) — Prometheus remains the logical source, Grafana is the transport. In-app charting via recharts is the sanctioned renderer; no grafana service type (or its widgets/status/UI surfaces) is reintroduced; orphaned Grafana widget instances degrade gracefully; and operators reconfigure prometheus services for the gateway (no silent auto-migration). This spec is acceptance-focused and verifiable; it deliberately does not prescribe implementation.

Purpose

Define WHAT must be true of Manage's metric-visualization surface after the grafana-metric-gateway change: metric queries for the prometheus service route through a Grafana gateway (POST {grafana_url}/api/ds/query) to back a multi-series line chart, a scalar gauge, and a windowed mean, alongside the instant-query numeric metric widget — Prometheus remains the logical source, Grafana is the transport. In-app charting via recharts is the sanctioned renderer; no grafana service type (or its widgets/status/UI surfaces) is reintroduced; orphaned Grafana widget instances degrade gracefully; and operators reconfigure prometheus services for the gateway (no silent auto-migration). This spec is acceptance-focused and verifiable; it deliberately does not prescribe implementation.

Requirements

Requirement: SC-101 — Prometheus range query returns the existing series shape

The prometheus service type's config MUST point at a Grafana gateway — declaring grafana_url (http(s)-schema-enforced), datasource_uid (default "prometheus"), and timeout_seconds — with a required, Fernet-encrypted grafana_api_key secret. The former direct-Prometheus base_url config field and the old optional api_key secret MUST be gone. When a prometheus widget of kind chart is fetched, the backend MUST issue POST {grafana_url}/api/ds/query (carrying Authorization: Bearer {grafana_api_key} and a queries[0] entry keyed by datasource_uid), MUST NOT issue any direct GET /api/v1/query_range against a Prometheus URL, and MUST return a payload of shape { "series": [{ "label": str, "points": [{ "t": int, "v": float|null }] }] } — the exact shape the frontend chart renderer already consumes.

Requirement: SC-102 — Series label normalization is shared and Prometheus-native

The metric-label → readable-label normalization MUST live in a single shared helper module that holds both normalizers — the retained Prom-matrix normalizer (normalize_prometheus_matrix) and the (re-)added Grafana-frames normalizer (normalize_grafana_frames) — and MUST deduplicate repeated labels via a label (n) suffix using one shared dedup rule (not duplicated across the two normalizers). Both normalizers produce the same {label, points} series shape, and the frames normalizer MUST apply the label fallback chain (displayName → sorted Prometheus labels excluding __-prefixed keys → "value").

Requirement: SC-103 — Range query errors degrade gracefully

A Grafana gateway timeout, connection error, HTTP 401/403 (auth), datasource-not-found, or non-2xx response MUST cause the widget data fetch to return { "error": str } (not raise), so the frontend renders the standard per-widget error state and the rest of the dashboard remains functional. The error message MUST be specific enough to distinguish auth failure from unreachable-host from datasource-not-found where the Grafana response permits.

Requirement: SC-104 — Step is derived from the window preset

Given a window preset (5m / 15m / 30m / 1h / 3h / 6h / 12h / 24h / 2d / 7d / 14d / 30d), the backend MUST reuse the existing WINDOW_PRESETS and step_for_window math to derive the gateway request's intervalMs (step * 1000), maxDataPoints, and from/to time bounds. Windows of 30 minutes or more must land in the ~100–300 point band; 5m and 15m may return 20 and 60 points respectively because Prometheus resolution is never set below 15 seconds. Users do not configure from/to/step/intervalMs directly.

Requirement: SC-105 — Chart widget moves from grafana to prometheus

Widget kinds chart, gauge, mean, and metric MUST remain bound to the prometheus service type — the kind strings are persisted in widget instance rows and MUST NOT change. The grafana service type MUST NOT be reintroduced and MUST NOT offer any widget kind. The backend adapter (internal class MetricSource, registry key still "prometheus") and the frontend SERVICE_REGISTRY bind these kinds to the neutral Metric* components for chart/gauge/mean (and PrometheusMetricWidget for metric); all four kinds source their data via the Grafana gateway.

Requirement: SC-106 — Chart renderer is reused unchanged

The recharts rendering (line chart, multi-series, axes, tooltip, mergeSeries, color tokens) MUST be preserved in the rebranded PrometheusChartWidget. The rename is structural; the rendering code is not rewritten.

Requirement: SC-107 — Chart supports multiple series

The chart widget MUST render all series returned by the gateway range query, each as its own line with a distinct color (via normalize_grafana_frames). There is no single-series restriction on chart.

Requirement: SC-108 — Chart window is a preset

The chart widget config MUST expose the time window as a preset selector (5m, 15m, 30m, 1h, 3h, 6h, 12h, 24h, 2d, 7d, 14d, 30d), not raw from/to/step fields. The preset is stored in widget config and resolved to start/end server-side. The shared chart renderer also offers an All values display option that removes the client-side cutoff from the values returned by that configured query.

Requirement: SC-109 — Gauge renders an instant scalar

A widget kind named gauge MUST be bound to the prometheus service. Its data fetch MUST run an instant PromQL query through the Grafana gateway (a gateway query with a minimal window / single data point) and return the scalar result for rendering as a gauge.

Requirement: SC-110 — Gauge supports configurable threshold bands

The gauge widget config MUST accept optional threshold values (e.g. warn_at, crit_at) and the renderer MUST display green / amber / red bands accordingly. When thresholds are omitted, the gauge renders with a single neutral color and no bands.

Requirement: SC-111 — Gauge is scalar-only

The gauge widget MUST render exactly one scalar value. If the gateway query returns multiple series, the adapter MUST return { "error": str } (not silently pick one), directing the user to refine the PromQL.

Requirement: SC-112 — Mean computes client-side over a window

A widget kind named mean MUST be bound to the prometheus service. Its data fetch MUST run a range query through the Grafana gateway over the configured window preset and return the arithmetic mean of all non-null point values of the single series as a single scalar.

Requirement: SC-113 — Mean uses plain PromQL + window preset

The mean widget config MUST accept a plain PromQL expression (no requirement to wrap in avg_over_time) plus a window preset. Users do not write range-vector functions.

Requirement: SC-114 — Mean is scalar-only

The mean widget MUST render exactly one scalar value. If the gateway range query returns multiple series, the adapter MUST return { "error": str } (not silently aggregate across series).

Requirement: SC-115 — No grafana references in backend source

Grafana is the sanctioned transport for the prometheus service, so backend source legitimately contains Grafana transport references (e.g. grafana_url, grafana_api_key, normalize_grafana_frames, POST …/api/ds/query). After the change, those references are transport-only: there MUST be no grafana service type, no grafana service definition, and no grafana entry in the backend SERVICE_DEFINITIONS / SERVICE_ADAPTERS registries. The integrations/grafana.py service-type module stays deleted.

Requirement: SC-116 — No grafana references in frontend source

Grafana transport is allowed in the frontend — the prometheus service config form renders Grafana gateway fields (grafana_url, datasource_uid) from the backend-provided schema. No frontend code MUST reintroduce a grafana service type, a Grafana deep-link / dashboard widget, LinksTab, or any Grafana-status UI surface. (Test fixtures where "Grafana" appears as a user-authored dashboard shortcut label unrelated to a service type remain excluded.)

Requirement: SC-117 — Grafana service type is gone from registries

Neither the backend SERVICE_DEFINITIONS / SERVICE_ADAPTERS nor the frontend SERVICE_REGISTRY / BUILTIN_WIDGETS MUST contain a grafana entry. The integrations/grafana.py file MUST be deleted.

Requirement: SC-118 — Grafana status checks and UI sections are removed

The get_grafana_status endpoint and its frontend hook (useGrafanaStatus) MUST be removed. The UI surface previously in ObservabilityPage.tsx has been refactored into a per-service-type service-tabs/ architecture; the Grafana removal targets are therefore service-tabs/LinksTab.tsx + its test, the grafana case in service-tabs/index.ts, the grafana entry in integrations/navEntries.ts, the grafana member of Dashboard.tsx's OBSERVABILITY_TYPES set, and any Grafana empty-state copy in ServicesPage.tsx. The literal ObservabilityPage.tsx no longer exists; SC-118's intent (no Grafana UI surface) is what is verified.

Requirement: SC-119 — Grafana widget instances degrade gracefully

An existing persisted widget row referencing a grafana service MUST NOT crash the dashboard. It resolves to the existing "unknown widget" error state and surfaces a clear message; the operator can then delete it.

Requirement: SC-120 — Grafana tests are removed

All Grafana-specific tests (backend and frontend) MUST be deleted; no test references grafana.

Requirement: SC-121 — config.yaml matches implementation

openspec/config.yaml MUST reflect that metric queries for the prometheus service are routed through a Grafana gateway (POST /api/ds/query) and that in-app charting via recharts is the sanctioned renderer. It MUST NOT carry the stale claims "Do NOT re-implement charting in-app", "No recharts/d3 is in use", or "Prometheus is queried directly (no Grafana middleman)".

Requirement: SC-122 — CHANGELOG documents the migration

CHANGELOG.md [Unreleased] MUST include a BREAKING entry stating that existing prometheus service instances must be reconfigured: replace the old base_url with grafana_url, add the grafana_api_key secret, and optionally set datasource_uid, because metric queries now route through Grafana's POST /api/ds/query.

Requirement: SC-123 — Backend tests pass

pytest run from backend/ MUST pass, including new tests covering: Prom range query → {series} normalization, gauge scalar-only enforcement, mean client-side aggregation, and the shared label helper.

Requirement: SC-124 — Frontend typechecks, builds, and lints

npm run build (which runs tsc -b + vite build) and npm run lint from frontend/ MUST pass.

Requirement: SC-125 — New widget kinds have tests

The renamed widgets MetricChartWidget, MetricGaugeWidget, and MetricMeanWidget MUST each have a frontend test covering at least: loading state, error state, and a rendered data case. (The metric-kind widget PrometheusMetricWidget is not renamed but remains tested.)

Requirement: SC-126 — No silent data migration

The change MUST NOT auto-migrate persisted prometheus service instances from the old base_url config shape to the new grafana_url gateway shape (URLs and credentials differ; a safe automated rewrite is not possible). Migration is operator-driven per the CHANGELOG note; old-shape instances are flagged on startup (see SC-129) and degrade at runtime to a gateway error until reconfigured.

Requirement: SC-127 — Non-blocking on the service-storage-harness change

This change MUST NOT depend on the service-storage-harness change. It is independently buildable, testable, and deployable. (The reverse dependency holds: the qBit speed widget depends on this change's chart capability.)

Requirement: SC-128 — get_prometheus_status validates the full gateway path

get_prometheus_status MUST validate the full metric path by running a trivial query (e.g. up) through the Grafana gateway (POST {grafana_url}/api/ds/query). Success MUST return { "up": true, "version": <str>, … } — a sentinel such as "ok" is acceptable when Grafana's /api/ds/query response carries no Prometheus build-info. Failure MUST return { "up": false, "error": <specific str> } distinguishing auth failure (HTTP 401/403), unreachable host, and datasource/gateway error where the response permits.

Requirement: SC-129 — Startup logs old-shape prometheus config without crashing

When the backend starts and a persisted prometheus service instance carries the old base_url config key without grafana_url, startup MUST log a clear migration message naming the service and instructing reconfiguration, and MUST complete without crashing (it MUST NOT delete, disable, or otherwise mutate the instance). The instance's widget fetches degrade to a gateway error until the operator reconfigures it.

Requirement: SC-130 — Grafana is the sanctioned transport for the prometheus service

The prometheus service type is the sole metric surface in Manage. Its metric queries (chart, gauge, mean, metric) are sanctioned to route through a Grafana gateway — POST {grafana_url}/api/ds/query with a grafana_api_key and a Prometheus datasource_uid. Grafana is the transport, not a first-class service type; a direct-Prometheus HTTP path is not supported.