manage/openspec/changes/prometheus-direct-charting/tasks.md

SDD Tasks: Prometheus Direct Charting (drop Grafana middleman)

Change: prometheus-direct-charting Phase: tasks Date: 2026-07-08

Review Workload Forecast

Field	Value
Estimated changed lines	~850–1,110 (sum of three implementation slices)
400-line budget risk	Medium
Chained PRs recommended	Yes
Suggested split	PR 1: Prom range path + chart rebrand → PR 2: gauge + mean widgets → PR 3: Grafana removal + config rewrite
Delivery strategy	auto-chain
Chain strategy	stacked-to-main

Decision needed before apply: No
Chained PRs recommended: Yes
Chain strategy: stacked-to-main
400-line budget risk: Medium

Each slice individually lands under the 400-line review budget. Slices are ordered S1 → S2 → S3; S1 and S2 are independently shippable, S3 must follow S1 (it removes the grafana chart binding S1 replaces). Per openspec/config.yaml rules, each slice leaves npm run build (tsc -b + vite build), npm run lint, and backend pytest green.

---

Slice ordering rationale (critical)

Slice 1 ADDS the prometheus chart capability without removing anything from grafana. After S1:

• prometheus binding has metric + chart; grafana binding has only link (chart kind moved away).
• GrafanaWidgetSource still exists (serves the link kind; its _fetch_chart is now dead but harmless).
• GrafanaLinkWidget and the grafana service type are untouched.
• A prometheus/chart widget renders from a direct /api/v1/query_range call.

This ordering ensures the chart capability is proven against Prometheus before the grafana surface is removed in S3, so the two risks (new query path + grafana removal) never compound in a single slice.

---

Slice 1: Prometheus range query path + shared helper + chart rebrand

Goal: Make a prometheus/chart widget render multi-series line charts from a direct /api/v1/query_range call, reusing the existing recharts renderer. Extract the series-normalization and step-derivation helpers into a testable module. Move the chart kind from grafana to prometheus in both registries. Do NOT yet remove the grafana service type, GrafanaWidgetSource, or GrafanaLinkWidget.

Satisfies: SC-101, SC-102, SC-103, SC-104, SC-105, SC-106, SC-107, SC-108.

• 1.1 Create shared Prometheus range helpers module

  • Files: backend/src/media_library_viewer_api/widgets/prometheus_range.py (new)
  • Lines: ~60
  • Dependencies: none
  • Details: Implement WINDOW_PRESETS = {"1h": 3600, "6h": 21600, "24h": 86400, "7d": 604800}, step_for_window(window_seconds, target_points=200) -> int returning max(15, round(window_seconds / target_points)), and normalize_prometheus_matrix(result: list[dict]) -> list[dict] that converts a Prometheus /api/v1/query_range data.result matrix into {label, points:[{t:int, v:float|None}]} series. Label rule: drop __name__ from metric labels; join remaining as k=v; fall back to "value"; dedup collisions with (n) suffix. Null handling: "NaN", "+Inf", "-Inf", None → v: None.

• 1.2 Add backend unit tests for range helpers

  • Files: backend/tests/test_prometheus_range.py (new)
  • Lines: ~70
  • Dependencies: 1.1
  • Details: Test step_for_window for all four presets asserts result yields 100–300 points. Test normalize_prometheus_matrix: feed a two-entry sample matrix (one with __name__, one colliding label) → assert {series} shape, label dedup (1) suffix, null handling for "NaN" string.

• 1.3 Add _fetch_chart + shared range-query plumbing to PrometheusWidgetSource

  • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
  • Lines: ~55
  • Dependencies: 1.1
  • Details: Import WINDOW_PRESETS, step_for_window, normalize_prometheus_matrix from prometheus_range. Add a private _range_query(base_url, timeout, promql, window) -> dict returning {"matrix": result} or {"error": ...} (shared by chart now and mean in S2). Add _fetch_chart(self, base_url, timeout, config) calling _range_query and returning {"series": normalize_prometheus_matrix(matrix)}. Dispatch widget_kind == "chart" in .fetch(). Existing metric path stays byte-for-byte unchanged. Errors (timeout, RequestException) → {"error": str}, never raise.

• 1.4 Declare chart widget kind in Prometheus integration

  • Files: backend/src/media_library_viewer_api/integrations/prometheus.py (modify)
  • Lines: ~15
  • Dependencies: 1.3
  • Details: Add PrometheusChartWidgetConfig(WidgetConfigBase) with promql: str and window: str = "1h". Add a widget_kind(...) entry for chart (refresh 60s, default config {"promql": "", "window": "1h"}). Leave metric kind untouched.

• 1.5 Rename GrafanaChartWidget → PrometheusChartWidget

  • Files: frontend/src/widgets/GrafanaChartWidget.tsx → frontend/src/widgets/PrometheusChartWidget.tsx (git mv)
  • Lines: ~5 changed (rename export, fix empty-state copy)
  • Dependencies: none
  • Details: git mv to preserve history. Rename exported function GrafanaChartWidget → PrometheusChartWidget. The recharts body (LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, ResponsiveContainer, mergeSeries, CHART_COLORS, formatTime) is preserved unchanged (SC-106). Fix empty-state copy: "Check your query and datasource_uid" → "Check your PromQL query and window."

• 1.6 Rename chart widget test

  • Files: frontend/src/widgets/__tests__/GrafanaChartWidget.test.tsx → frontend/src/widgets/__tests__/PrometheusChartWidget.test.tsx (git mv)
  • Lines: ~10 changed (import path, component name, error-message assertion)
  • Dependencies: 1.5
  • Details: git mv. Update import to PrometheusChartWidget. Update error-state assertion: the old Grafana-specific error string ("Grafana api_key is required") → a Prom error string (e.g. "promql is required"). Keep loading + rendered-data test cases.

• 1.7 Rebind chart from grafana to prometheus in frontend registry

  • Files: frontend/src/integrations/registry.ts (modify)
  • Lines: ~30
  • Dependencies: 1.5
  • Details: Import PrometheusChartWidget. Add a chart entry to the prometheus binding's widgets array (kind chart, refresh 60s, configSchema with promql + window). Remove the chart entry from the grafana binding's widgets array (leave link intact). Do NOT delete the grafana binding itself.

• 1.8 Update frontend widgets barrel export

  • Files: frontend/src/widgets/index.ts (modify)
  • Lines: ~2
  • Dependencies: 1.5
  • Details: Rename the GrafanaChartWidget export to PrometheusChartWidget. Leave GrafanaLinkWidget export intact.

• 1.9 Update registry tests for chart rebind

  • Files: frontend/src/integrations/registry.test.ts (modify)
  • Lines: ~15
  • Dependencies: 1.7
  • Details: Assert prometheus binding has metric + chart kinds. Assert grafana binding has only link (no chart).

• 1.10 Verify Slice 1 (build + lint + test)

  • Run: cd backend && PYTHONPATH=src pytest tests/test_prometheus_range.py tests/test_widgets.py && cd ../frontend && npm run build && npm run lint
  • Verify: helpers tests pass; existing widget tests pass (grafana link adapter still wired); frontend typechecks and lints; prometheus/chart widget resolves to PrometheusChartWidget.

Slice 1 total: ~260–330 changed lines.

---

Slice 2: Gauge + mean widgets

Goal: Add gauge and mean widget kinds to the prometheus service. Gauge renders an instant scalar with configurable threshold bands. Mean renders a single value computed client-side over a range-query window. Both are scalar-only.

Satisfies: SC-109, SC-110, SC-111, SC-112, SC-113, SC-114.

• 2.1 Extract shared _instant_query helper + add _fetch_gauge to PrometheusWidgetSource

  • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
  • Lines: ~40
  • Dependencies: Slice 1 (1.3)
  • Details: Extract the instant-query HTTP call from the existing metric path into a private _instant_query(base_url, timeout, promql) -> dict returning {"result": [...]} or {"error": ...}. Refactor the metric path to use it (behavior unchanged). Add _fetch_gauge(self, base_url, timeout, config) using _instant_query: assert len(result) == 1 (scalar-only, SC-111); parse float(result[0]["value"][1]); return {"value": float, "warn_at": config.get("warn_at"), "crit_at": config.get("crit_at"), "min": config.get("min"), "max": config.get("max"), "unit": config.get("unit")}. Dispatch widget_kind == "gauge" in .fetch().

• 2.2 Add _fetch_mean to PrometheusWidgetSource

  • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
  • Lines: ~30
  • Dependencies: 2.1, Slice 1 (1.3 for _range_query)
  • Details: Add _fetch_mean(self, base_url, timeout, config) using the shared _range_query from S1. Assert len(result) == 1 (scalar-only, SC-114). Collect non-null numeric values from the single series; compute arithmetic mean; return {"value": mean, "unit": config.get("unit")}. If no numeric samples → {"error": "..."}. Dispatch widget_kind == "mean" in .fetch().

• 2.3 Declare gauge + mean widget kinds in Prometheus integration

  • Files: backend/src/media_library_viewer_api/integrations/prometheus.py (modify)
  • Lines: ~25
  • Dependencies: 2.1, 2.2
  • Details: Add PrometheusGaugeWidgetConfig (promql: str, warn_at: float|None, crit_at: float|None, min: float|None, max: float|None, unit: str|None) and PrometheusMeanWidgetConfig (promql: str, window: str = "1h", unit: str|None). Add widget_kind(...) entries: gauge (refresh 30s), mean (refresh 60s).

• 2.4 Add backend tests for gauge + mean adapters

  • Files: backend/tests/test_widgets.py (modify) or backend/tests/test_prometheus_range.py (modify)
  • Lines: ~60
  • Dependencies: 2.1, 2.2
  • Details: Mock requests.get for gauge: instant query returning 1 series → assert {value, ...} shape; returning 2 series → assert {"error": ...} (SC-111). Mock for mean: range query returning 1 series with known values [1.0, 2.0, 3.0] → assert mean 2.0; returning 2 series → assert {"error": ...} (SC-114). Test timeout/RequestException → {"error": ...}.

• 2.5 Create PrometheusGaugeWidget component

  • Files: frontend/src/widgets/PrometheusGaugeWidget.tsx (new)
  • Lines: ~90
  • Dependencies: Slice 1 (1.5 for widget pattern)
  • Details: Render via recharts RadialBarChart (no new dep; SC-110). Threshold bands: three stacked RadialBar track cells (green 0→warn, amber warn→crit, red crit→max) + a value cell. When warn_at/crit_at absent → single neutral-color track. min/max default to 0/max(value, 1). Reuse SectionCard + Alert/Skeleton for loading/error states. Consume data?.data?.value, warn_at, etc. off useWidgetData.

• 2.6 Create PrometheusMeanWidget component

  • Files: frontend/src/widgets/PrometheusMeanWidget.tsx (new)
  • Lines: ~50
  • Dependencies: Slice 1
  • Details: Single-value display reusing the MetricCard pattern (big number + optional unit suffix + subtext "mean over last {window}"). Loading/error/empty via Skeleton/Alert. No charting library — it's a number (SC-112).

• 2.7 Create gauge + mean frontend tests

  • Files: frontend/src/widgets/__tests__/PrometheusGaugeWidget.test.tsx (new), frontend/src/widgets/__tests__/PrometheusMeanWidget.test.tsx (new)
  • Lines: ~60
  • Dependencies: 2.5, 2.6
  • Details: Each covers loading, error, and rendered-data case (SC-125). Gauge test: render with bands (warn_at/crit_at set) and without (single color). Mean test: render with value + unit.

• 2.8 Add gauge + mean bindings to frontend registry

  • Files: frontend/src/integrations/registry.ts (modify)
  • Lines: ~35
  • Dependencies: 2.5, 2.6
  • Details: Import PrometheusGaugeWidget + PrometheusMeanWidget. Add gauge (refresh 30s, configSchema with promql, warn_at, crit_at, min, max, unit) and mean (refresh 60s, configSchema with promql, window, unit) entries to the prometheus binding's widgets array alongside metric and chart.

• 2.9 Update widgets barrel + registry tests

  • Files: frontend/src/widgets/index.ts (modify), frontend/src/integrations/registry.test.ts (modify)
  • Lines: ~10
  • Dependencies: 2.5, 2.6, 2.8
  • Details: Export PrometheusGaugeWidget + PrometheusMeanWidget. Assert prometheus binding has metric, chart, gauge, mean (four kinds).

• 2.10 Verify Slice 2 (build + lint + test)

  • Run: cd backend && PYTHONPATH=src pytest tests/test_prometheus_range.py tests/test_widgets.py && cd ../frontend && npm run build && npm run lint
  • Verify: gauge/mean adapter tests pass; frontend typechecks and lints; all four prometheus widget kinds resolve.

Slice 2 total: ~310–400 changed lines.

---

Slice 3: Grafana removal + config rewrite + changelog

Goal: Remove the entire Grafana surface (service type, link widget, chart source, status checks, nav entries, UI tabs). Rewrite config.yaml to match reality. Add CHANGELOG migration note. Leave the app grep-clean of grafana service references.

Satisfies: SC-115, SC-116, SC-117, SC-118, SC-119, SC-120, SC-121, SC-122, SC-126.

Spec-text drift note: SC-118 literally names ObservabilityPage.tsx, which was refactored into service-tabs/. The intent (no Grafana UI surface) is verified by the removals below. Dashboard.test.tsx contains "Grafana" as a user-authored shortcut label (unrelated to the grafana service) — SC-116 grep should not flag it; it is left intact.

• 3.1 Delete backend Grafana integration module

  • Files: backend/src/media_library_viewer_api/integrations/grafana.py (delete)
  • Lines: ~−20 (deletion)
  • Dependencies: Slice 1 (chart kind already moved to prometheus)
  • Details: Delete the file. It contains GrafanaConfig + GrafanaLinkWidgetConfig.

• 3.2 Remove Grafana from backend integration registry

  • Files: backend/src/media_library_viewer_api/integrations/registry.py (modify)
  • Lines: ~−3
  • Dependencies: 3.1
  • Details: Drop from ...grafana import DEFINITION as GRAFANA and the GRAFANA.service_type: GRAFANA entry from SERVICE_DEFINITIONS.

• 3.3 Remove GrafanaWidgetSource + adapter registration

  • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
  • Lines: ~−100 (deletion of GrafanaWidgetSource class + _fetch_chart)
  • Dependencies: Slice 1 (normalization logic already extracted to prometheus_range.py)
  • Details: Delete the GrafanaWidgetSource class entirely (including _fetch_chart — its logic was extracted to normalize_prometheus_matrix in S1). Remove "grafana": GrafanaWidgetSource() from SERVICE_ADAPTERS.

• 3.4 Remove get_grafana_status endpoint

  • Files: backend/src/media_library_viewer_api/routers/monitoring.py (modify)
  • Lines: ~−25
  • Dependencies: none
  • Details: Delete the @router.get("/grafana-status") endpoint and its helper. Leave get_prometheus_status / get_alertmanager_status intact.

• 3.5 Remove Grafana backend tests

  • Files: backend/tests/test_widgets.py (modify), backend/tests/test_api.py (modify), backend/tests/test_services.py (modify)
  • Lines: ~−40
  • Dependencies: 3.3
  • Details: Delete grafana adapter tests (test_grafana_adapter_*), grafana service fixtures, and the TestGrafanaStatus test class. SC-120.

• 3.6 Delete frontend GrafanaLinkWidget + barrel export

  • Files: frontend/src/widgets/GrafanaLinkWidget.tsx (delete), frontend/src/widgets/index.ts (modify)
  • Lines: ~−35
  • Dependencies: none
  • Details: Delete the file. Remove the GrafanaLinkWidget export from widgets/index.ts.

• 3.7 Remove grafana binding from frontend registry

  • Files: frontend/src/integrations/registry.ts (modify)
  • Lines: ~−30
  • Dependencies: 3.6
  • Details: Delete the entire grafana key from SERVICE_REGISTRY. Drop the GrafanaLinkWidget import. SC-117.

• 3.8 Remove Grafana nav entry

  • Files: frontend/src/integrations/navEntries.ts (modify)
  • Lines: ~−5
  • Dependencies: none
  • Details: Drop the grafana entry from SERVICE_TYPE_NAV_ENTRIES and any now-unused icon import (e.g. Link2).

• 3.9 Remove Grafana status hook + API client function + type

  • Files: frontend/src/hooks/useObservability.ts (modify), frontend/src/api/client.ts (modify), frontend/src/types/index.ts (modify)
  • Lines: ~−15
  • Dependencies: 3.4
  • Details: Drop useGrafanaStatus + its fetchGrafanaStatus import from useObservability.ts. Drop fetchGrafanaStatus from client.ts. Drop GrafanaStatus interface from types/index.ts.

• 3.10 Delete Grafana service tab + remove from tab index

  • Files: frontend/src/pages/service-tabs/LinksTab.tsx (delete), frontend/src/pages/service-tabs/__tests__/LinksTab.test.tsx (delete), frontend/src/pages/service-tabs/index.ts (modify)
  • Lines: ~−80
  • Dependencies: none
  • Details: Delete LinksTab.tsx (grafana-specific per its docstring) and its test. Remove the LinksTab import and case "grafana": from service-tabs/index.ts.

• 3.11 Remove Grafana from Dashboard + ServicesPage

  • Files: frontend/src/pages/Dashboard.tsx (modify), frontend/src/pages/ServicesPage.tsx (modify)
  • Lines: ~−5
  • Dependencies: none
  • Details: Drop "grafana" from OBSERVABILITY_TYPES set in Dashboard.tsx. Update ServicesPage.tsx empty-state copy: "Add a Grafana, Prometheus, …" → "Add a Prometheus, …".

• 3.12 Rewrite openspec/config.yaml thin-dashboard rule

  • Files: openspec/config.yaml (modify)
  • Lines: ~8
  • Dependencies: none
  • Details: Remove "Do NOT re-implement charting in-app" and "No recharts/d3 is in use" claims. Replace with accurate wording: in-app charting via recharts is the sanctioned approach for Prometheus-backed series; Grafana is no longer referenced. SC-121.

• 3.13 Add CHANGELOG migration note

  • Files: CHANGELOG.md (modify)
  • Lines: ~8
  • Dependencies: none
  • Details: Add entry under an appropriate version heading: instruct operators to delete existing Grafana service instances and recreate them as Prometheus services; note that grafana/chart widgets must be recreated as prometheus/chart widgets. SC-122, SC-126.

• 3.14 Verify Slice 3 (grep-clean + build + lint + test)

  • Run: grep -ri grafana backend/src --include='*.py' → expect zero matches (SC-115)
  • Run: grep -ri grafana frontend/src → expect zero matches except Dashboard.test.tsx shortcut fixture (SC-116)
  • Run: cd backend && PYTHONPATH=src pytest && cd ../frontend && npm run build && npm run lint
  • Verify: no grafana service references remain; all tests pass; frontend builds and lints.

Slice 3 total: ~250–350 changed lines (mostly deletions).

---

Integration and acceptance verification

• 4.1 Full backend test run

  • Run: cd backend && PYTHONPATH=src pytest
  • Verify: all tests pass; no grafana test references remain; prometheus range/gauge/mean tests pass.

• 4.2 Full frontend build + lint

  • Run: cd frontend && npm run build && npm run lint
  • Verify: TypeScript compiles; no lint failures; no grafana imports unresolved.

• 4.3 Grep-clean verification

  • Run: grep -ri grafana backend/src --include='*.py' → zero matches
  • Run: grep -ri grafana frontend/src → zero matches excluding Dashboard.test.tsx shortcut fixture
  • Verify: SC-115, SC-116 satisfied.

• 4.4 config.yaml accuracy check

  • Verify: openspec/config.yaml does not contain "Do NOT re-implement charting" or "No recharts/d3"; reflects recharts as sanctioned renderer (SC-121).

• 4.5 CHANGELOG check

  • Verify: CHANGELOG.md documents the grafana→prometheus migration (SC-122).

---

Total estimate

Slice	Changed lines	Satisfies
Slice 1: Prom range path + chart rebrand + helper	~260–330	SC-101…108
Slice 2: Gauge + mean widgets	~310–400	SC-109…114
Slice 3: Grafana removal + config + changelog	~250–350	SC-115…122, 126
Integration verification	~0	SC-123…125, 127
Total	~820–1,080	SC-101…127

Each slice is under the 400-line review budget. Use three chained PRs (S1 → S2 → S3), each independently buildable and green.

---

Risk flags for the apply phase

1. git mv for chart widget rename — use git mv (not delete+create) to preserve file history (design §9).
2. recharts RadialBarChart gauge — try recharts first; if rendering proves fiddly, the proposal sanctions a ~50-line SVG fallback (no new dep). Decision at apply time.
3. Dashboard.test.tsx "Grafana" literal — this is a user-authored shortcut label in a test fixture, not a grafana service reference. SC-116 grep should not flag it. Flagged for reviewer awareness.
4. SC-118 textual drift — ObservabilityPage.tsx no longer exists (refactored to service-tabs/). The intent is satisfied by removing get_grafana_status + useGrafanaStatus + LinksTab. Verify against intent, not literal filename.
5. _range_query / _instant_query refactoring timing — _range_query is created in S1 (1.3) for _fetch_chart; _instant_query is extracted in S2 (2.1) when _fetch_gauge needs it. Both shared helpers must be in place before S3 (which deletes GrafanaWidgetSource and its private chart logic).
6. Stale project map — the pi-map references ObservabilityPage.tsx and 7 service types (actual: 8, including authentik + backups). Trust source, not the map. Run project_map_patch after source edits and project_map_validate before final handoff.