alex/manage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
044d386ac7ac85bcb3592259eaca20b11fdde93e
manage/openspec/changes/archive/2026-07-08-prometheus-direct-charting/tasks.md
T
Developer 7efc06a629 chore(prometheus-direct-charting): archive verified+synced change 
Move to openspec/changes/archive/2026-07-08-prometheus-direct-charting/
(git mv, history preserved). 9 artifacts: proposal/spec/design/tasks/
apply-progress/verify-report/sync-report/archive-report + delta spec.
Canonical openspec/specs/prometheus-charting/ remains.
2026-07-08 23:01:35 +00:00

22 KiB
Raw Blame History

SDD Tasks: Prometheus Direct Charting (drop Grafana middleman)

Change: prometheus-direct-charting Phase: tasks Date: 2026-07-08

Review Workload Forecast

Field Value
Estimated changed lines ~850–1,110 (sum of three implementation slices)
400-line budget risk Medium
Chained PRs recommended Yes
Suggested split PR 1: Prom range path + chart rebrand → PR 2: gauge + mean widgets → PR 3: Grafana removal + config rewrite
Delivery strategy auto-chain
Chain strategy stacked-to-main 
Decision needed before apply: No
Chained PRs recommended: Yes
Chain strategy: stacked-to-main
400-line budget risk: Medium

Each slice individually lands under the 400-line review budget. Slices are ordered S1 → S2 → S3; S1 and S2 are independently shippable, S3 must follow S1 (it removes the grafana chart binding S1 replaces). Per openspec/config.yaml rules, each slice leaves npm run build (tsc -b + vite build), npm run lint, and backend pytest green.

Slice ordering rationale (critical)

Slice 1 ADDS the prometheus chart capability without removing anything from grafana. After S1:

  • prometheus binding has metric + chart; grafana binding has only link (chart kind moved away).
  • GrafanaWidgetSource still exists (serves the link kind; its _fetch_chart is now dead but harmless).
  • GrafanaLinkWidget and the grafana service type are untouched.
  • A prometheus/chart widget renders from a direct /api/v1/query_range call.

This ordering ensures the chart capability is proven against Prometheus before the grafana surface is removed in S3, so the two risks (new query path + grafana removal) never compound in a single slice.

Slice 1: Prometheus range query path + shared helper + chart rebrand

Goal: Make a prometheus/chart widget render multi-series line charts from a direct /api/v1/query_range call, reusing the existing recharts renderer. Extract the series-normalization and step-derivation helpers into a testable module. Move the chart kind from grafana to prometheus in both registries. Do NOT yet remove the grafana service type, GrafanaWidgetSource, or GrafanaLinkWidget.

Satisfies: SC-101, SC-102, SC-103, SC-104, SC-105, SC-106, SC-107, SC-108.

  • 1.1 Create shared Prometheus range helpers module

    • Files: backend/src/media_library_viewer_api/widgets/prometheus_range.py (new)
    • Lines: ~60
    • Dependencies: none
    • Details: Implement WINDOW_PRESETS = {"1h": 3600, "6h": 21600, "24h": 86400, "7d": 604800}, step_for_window(window_seconds, target_points=200) -> int returning max(15, round(window_seconds / target_points)), and normalize_prometheus_matrix(result: list[dict]) -> list[dict] that converts a Prometheus /api/v1/query_range data.result matrix into {label, points:[{t:int, v:float|None}]} series. Label rule: drop __name__ from metric labels; join remaining as k=v; fall back to "value"; dedup collisions with (n) suffix. Null handling: "NaN", "+Inf", "-Inf", None → v: None.

  • 1.2 Add backend unit tests for range helpers

    • Files: backend/tests/test_prometheus_range.py (new)
    • Lines: ~70
    • Dependencies: 1.1
    • Details: Test step_for_window for all four presets asserts result yields 100–300 points. Test normalize_prometheus_matrix: feed a two-entry sample matrix (one with __name__, one colliding label) → assert {series} shape, label dedup (1) suffix, null handling for "NaN" string.

  • 1.3 Add _fetch_chart + shared range-query plumbing to PrometheusWidgetSource

    • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
    • Lines: ~55
    • Dependencies: 1.1
    • Details: Import WINDOW_PRESETS, step_for_window, normalize_prometheus_matrix from prometheus_range. Add a private _range_query(base_url, timeout, promql, window) -> dict returning {"matrix": result} or {"error": ...} (shared by chart now and mean in S2). Add _fetch_chart(self, base_url, timeout, config) calling _range_query and returning {"series": normalize_prometheus_matrix(matrix)}. Dispatch widget_kind == "chart" in .fetch(). Existing metric path stays byte-for-byte unchanged. Errors (timeout, RequestException) → {"error": str}, never raise.

  • 1.4 Declare chart widget kind in Prometheus integration

    • Files: backend/src/media_library_viewer_api/integrations/prometheus.py (modify)
    • Lines: ~15
    • Dependencies: 1.3
    • Details: Add PrometheusChartWidgetConfig(WidgetConfigBase) with promql: str and window: str = "1h". Add a widget_kind(...) entry for chart (refresh 60s, default config {"promql": "", "window": "1h"}). Leave metric kind untouched.

  • 1.5 Rename GrafanaChartWidget → PrometheusChartWidget

    • Files: frontend/src/widgets/GrafanaChartWidget.tsx → frontend/src/widgets/PrometheusChartWidget.tsx (git mv)
    • Lines: ~5 changed (rename export, fix empty-state copy)
    • Dependencies: none
    • Details: git mv to preserve history. Rename exported function GrafanaChartWidget → PrometheusChartWidget. The recharts body (LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, ResponsiveContainer, mergeSeries, CHART_COLORS, formatTime) is preserved unchanged (SC-106). Fix empty-state copy: "Check your query and datasource_uid" → "Check your PromQL query and window."

  • 1.6 Rename chart widget test

    • Files: frontend/src/widgets/__tests__/GrafanaChartWidget.test.tsx → frontend/src/widgets/__tests__/PrometheusChartWidget.test.tsx (git mv)
    • Lines: ~10 changed (import path, component name, error-message assertion)
    • Dependencies: 1.5
    • Details: git mv. Update import to PrometheusChartWidget. Update error-state assertion: the old Grafana-specific error string ("Grafana api_key is required") → a Prom error string (e.g. "promql is required"). Keep loading + rendered-data test cases.

  • 1.7 Rebind chart from grafana to prometheus in frontend registry

    • Files: frontend/src/integrations/registry.ts (modify)
    • Lines: ~30
    • Dependencies: 1.5
    • Details: Import PrometheusChartWidget. Add a chart entry to the prometheus binding's widgets array (kind chart, refresh 60s, configSchema with promql + window). Remove the chart entry from the grafana binding's widgets array (leave link intact). Do NOT delete the grafana binding itself.

  • 1.8 Update frontend widgets barrel export

    • Files: frontend/src/widgets/index.ts (modify)
    • Lines: ~2
    • Dependencies: 1.5
    • Details: Rename the GrafanaChartWidget export to PrometheusChartWidget. Leave GrafanaLinkWidget export intact.

  • 1.9 Update registry tests for chart rebind

    • Files: frontend/src/integrations/registry.test.ts (modify)
    • Lines: ~15
    • Dependencies: 1.7
    • Details: Assert prometheus binding has metric + chart kinds. Assert grafana binding has only link (no chart).

  • 1.10 Verify Slice 1 (build + lint + test)

    • Run: cd backend && PYTHONPATH=src pytest tests/test_prometheus_range.py tests/test_widgets.py && cd ../frontend && npm run build && npm run lint
    • Verify: helpers tests pass; existing widget tests pass (grafana link adapter still wired); frontend typechecks and lints; prometheus/chart widget resolves to PrometheusChartWidget.

Slice 1 total: ~260–330 changed lines.

Slice 2: Gauge + mean widgets

Goal: Add gauge and mean widget kinds to the prometheus service. Gauge renders an instant scalar with configurable threshold bands. Mean renders a single value computed client-side over a range-query window. Both are scalar-only.

Satisfies: SC-109, SC-110, SC-111, SC-112, SC-113, SC-114.

  • 2.1 Extract shared _instant_query helper + add _fetch_gauge to PrometheusWidgetSource

    • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
    • Lines: ~40
    • Dependencies: Slice 1 (1.3)
    • Details: Extract the instant-query HTTP call from the existing metric path into a private _instant_query(base_url, timeout, promql) -> dict returning {"result": [...]} or {"error": ...}. Refactor the metric path to use it (behavior unchanged). Add _fetch_gauge(self, base_url, timeout, config) using _instant_query: assert len(result) == 1 (scalar-only, SC-111); parse float(result[0]["value"][1]); return {"value": float, "warn_at": config.get("warn_at"), "crit_at": config.get("crit_at"), "min": config.get("min"), "max": config.get("max"), "unit": config.get("unit")}. Dispatch widget_kind == "gauge" in .fetch().

  • 2.2 Add _fetch_mean to PrometheusWidgetSource

    • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
    • Lines: ~30
    • Dependencies: 2.1, Slice 1 (1.3 for _range_query)
    • Details: Add _fetch_mean(self, base_url, timeout, config) using the shared _range_query from S1. Assert len(result) == 1 (scalar-only, SC-114). Collect non-null numeric values from the single series; compute arithmetic mean; return {"value": mean, "unit": config.get("unit")}. If no numeric samples → {"error": "..."}. Dispatch widget_kind == "mean" in .fetch().

  • 2.3 Declare gauge + mean widget kinds in Prometheus integration

    • Files: backend/src/media_library_viewer_api/integrations/prometheus.py (modify)
    • Lines: ~25
    • Dependencies: 2.1, 2.2
    • Details: Add PrometheusGaugeWidgetConfig (promql: str, warn_at: float|None, crit_at: float|None, min: float|None, max: float|None, unit: str|None) and PrometheusMeanWidgetConfig (promql: str, window: str = "1h", unit: str|None). Add widget_kind(...) entries: gauge (refresh 30s), mean (refresh 60s).

  • 2.4 Add backend tests for gauge + mean adapters

    • Files: backend/tests/test_widgets.py (modify) or backend/tests/test_prometheus_range.py (modify)
    • Lines: ~60
    • Dependencies: 2.1, 2.2
    • Details: Mock requests.get for gauge: instant query returning 1 series → assert {value, ...} shape; returning 2 series → assert {"error": ...} (SC-111). Mock for mean: range query returning 1 series with known values [1.0, 2.0, 3.0] → assert mean 2.0; returning 2 series → assert {"error": ...} (SC-114). Test timeout/RequestException → {"error": ...}.

  • 2.5 Create PrometheusGaugeWidget component

    • Files: frontend/src/widgets/PrometheusGaugeWidget.tsx (new)
    • Lines: ~90
    • Dependencies: Slice 1 (1.5 for widget pattern)
    • Details: Render via recharts RadialBarChart (no new dep; SC-110). Threshold bands: three stacked RadialBar track cells (green 0→warn, amber warn→crit, red crit→max) + a value cell. When warn_at/crit_at absent → single neutral-color track. min/max default to 0/max(value, 1). Reuse SectionCard + Alert/Skeleton for loading/error states. Consume data?.data?.value, warn_at, etc. off useWidgetData.

  • 2.6 Create PrometheusMeanWidget component

    • Files: frontend/src/widgets/PrometheusMeanWidget.tsx (new)
    • Lines: ~50
    • Dependencies: Slice 1
    • Details: Single-value display reusing the MetricCard pattern (big number + optional unit suffix + subtext "mean over last {window}"). Loading/error/empty via Skeleton/Alert. No charting library — it's a number (SC-112).

  • 2.7 Create gauge + mean frontend tests

    • Files: frontend/src/widgets/__tests__/PrometheusGaugeWidget.test.tsx (new), frontend/src/widgets/__tests__/PrometheusMeanWidget.test.tsx (new)
    • Lines: ~60
    • Dependencies: 2.5, 2.6
    • Details: Each covers loading, error, and rendered-data case (SC-125). Gauge test: render with bands (warn_at/crit_at set) and without (single color). Mean test: render with value + unit.

  • 2.8 Add gauge + mean bindings to frontend registry

    • Files: frontend/src/integrations/registry.ts (modify)
    • Lines: ~35
    • Dependencies: 2.5, 2.6
    • Details: Import PrometheusGaugeWidget + PrometheusMeanWidget. Add gauge (refresh 30s, configSchema with promql, warn_at, crit_at, min, max, unit) and mean (refresh 60s, configSchema with promql, window, unit) entries to the prometheus binding's widgets array alongside metric and chart.

  • 2.9 Update widgets barrel + registry tests

    • Files: frontend/src/widgets/index.ts (modify), frontend/src/integrations/registry.test.ts (modify)
    • Lines: ~10
    • Dependencies: 2.5, 2.6, 2.8
    • Details: Export PrometheusGaugeWidget + PrometheusMeanWidget. Assert prometheus binding has metric, chart, gauge, mean (four kinds).

  • 2.10 Verify Slice 2 (build + lint + test)

    • Run: cd backend && PYTHONPATH=src pytest tests/test_prometheus_range.py tests/test_widgets.py && cd ../frontend && npm run build && npm run lint
    • Verify: gauge/mean adapter tests pass; frontend typechecks and lints; all four prometheus widget kinds resolve.

Slice 2 total: ~310–400 changed lines.

Slice 3: Grafana removal + config rewrite + changelog

Goal: Remove the entire Grafana surface (service type, link widget, chart source, status checks, nav entries, UI tabs). Rewrite config.yaml to match reality. Add CHANGELOG migration note. Leave the app grep-clean of grafana service references.

Satisfies: SC-115, SC-116, SC-117, SC-118, SC-119, SC-120, SC-121, SC-122, SC-126.

Spec-text drift note: SC-118 literally names ObservabilityPage.tsx, which was refactored into service-tabs/. The intent (no Grafana UI surface) is verified by the removals below. Dashboard.test.tsx contains "Grafana" as a user-authored shortcut label (unrelated to the grafana service) — SC-116 grep should not flag it; it is left intact.

  • 3.1 Delete backend Grafana integration module

    • Files: backend/src/media_library_viewer_api/integrations/grafana.py (delete)
    • Lines: ~−20 (deletion)
    • Dependencies: Slice 1 (chart kind already moved to prometheus)
    • Details: Delete the file. It contains GrafanaConfig + GrafanaLinkWidgetConfig.

  • 3.2 Remove Grafana from backend integration registry

    • Files: backend/src/media_library_viewer_api/integrations/registry.py (modify)
    • Lines: ~−3
    • Dependencies: 3.1
    • Details: Drop from ...grafana import DEFINITION as GRAFANA and the GRAFANA.service_type: GRAFANA entry from SERVICE_DEFINITIONS.

  • 3.3 Remove GrafanaWidgetSource + adapter registration

    • Files: backend/src/media_library_viewer_api/widgets/sources.py (modify)
    • Lines: ~−100 (deletion of GrafanaWidgetSource class + _fetch_chart)
    • Dependencies: Slice 1 (normalization logic already extracted to prometheus_range.py)
    • Details: Delete the GrafanaWidgetSource class entirely (including _fetch_chart — its logic was extracted to normalize_prometheus_matrix in S1). Remove "grafana": GrafanaWidgetSource() from SERVICE_ADAPTERS.

  • 3.4 Remove get_grafana_status endpoint

    • Files: backend/src/media_library_viewer_api/routers/monitoring.py (modify)
    • Lines: ~−25
    • Dependencies: none
    • Details: Delete the @router.get("/grafana-status") endpoint and its helper. Leave get_prometheus_status / get_alertmanager_status intact.

  • 3.5 Remove Grafana backend tests

    • Files: backend/tests/test_widgets.py (modify), backend/tests/test_api.py (modify), backend/tests/test_services.py (modify)
    • Lines: ~−40
    • Dependencies: 3.3
    • Details: Delete grafana adapter tests (test_grafana_adapter_*), grafana service fixtures, and the TestGrafanaStatus test class. SC-120.

  • 3.6 Delete frontend GrafanaLinkWidget + barrel export

    • Files: frontend/src/widgets/GrafanaLinkWidget.tsx (delete), frontend/src/widgets/index.ts (modify)
    • Lines: ~−35
    • Dependencies: none
    • Details: Delete the file. Remove the GrafanaLinkWidget export from widgets/index.ts.

  • 3.7 Remove grafana binding from frontend registry

    • Files: frontend/src/integrations/registry.ts (modify)
    • Lines: ~−30
    • Dependencies: 3.6
    • Details: Delete the entire grafana key from SERVICE_REGISTRY. Drop the GrafanaLinkWidget import. SC-117.

  • 3.8 Remove Grafana nav entry

    • Files: frontend/src/integrations/navEntries.ts (modify)
    • Lines: ~−5
    • Dependencies: none
    • Details: Drop the grafana entry from SERVICE_TYPE_NAV_ENTRIES and any now-unused icon import (e.g. Link2).

  • 3.9 Remove Grafana status hook + API client function + type

    • Files: frontend/src/hooks/useObservability.ts (modify), frontend/src/api/client.ts (modify), frontend/src/types/index.ts (modify)
    • Lines: ~−15
    • Dependencies: 3.4
    • Details: Drop useGrafanaStatus + its fetchGrafanaStatus import from useObservability.ts. Drop fetchGrafanaStatus from client.ts. Drop GrafanaStatus interface from types/index.ts.

  • 3.10 Delete Grafana service tab + remove from tab index

    • Files: frontend/src/pages/service-tabs/LinksTab.tsx (delete), frontend/src/pages/service-tabs/__tests__/LinksTab.test.tsx (delete), frontend/src/pages/service-tabs/index.ts (modify)
    • Lines: ~−80
    • Dependencies: none
    • Details: Delete LinksTab.tsx (grafana-specific per its docstring) and its test. Remove the LinksTab import and case "grafana": from service-tabs/index.ts.

  • 3.11 Remove Grafana from Dashboard + ServicesPage

    • Files: frontend/src/pages/Dashboard.tsx (modify), frontend/src/pages/ServicesPage.tsx (modify)
    • Lines: ~−5
    • Dependencies: none
    • Details: Drop "grafana" from OBSERVABILITY_TYPES set in Dashboard.tsx. Update ServicesPage.tsx empty-state copy: "Add a Grafana, Prometheus, …" → "Add a Prometheus, …".

  • 3.12 Rewrite openspec/config.yaml thin-dashboard rule

    • Files: openspec/config.yaml (modify)
    • Lines: ~8
    • Dependencies: none
    • Details: Remove "Do NOT re-implement charting in-app" and "No recharts/d3 is in use" claims. Replace with accurate wording: in-app charting via recharts is the sanctioned approach for Prometheus-backed series; Grafana is no longer referenced. SC-121.

  • 3.13 Add CHANGELOG migration note

    • Files: CHANGELOG.md (modify)
    • Lines: ~8
    • Dependencies: none
    • Details: Add entry under an appropriate version heading: instruct operators to delete existing Grafana service instances and recreate them as Prometheus services; note that grafana/chart widgets must be recreated as prometheus/chart widgets. SC-122, SC-126.

  • 3.14 Verify Slice 3 (grep-clean + build + lint + test)

    • Run: grep -ri grafana backend/src --include='*.py' → expect zero matches (SC-115)
    • Run: grep -ri grafana frontend/src → expect zero matches except Dashboard.test.tsx shortcut fixture (SC-116)
    • Run: cd backend && PYTHONPATH=src pytest && cd ../frontend && npm run build && npm run lint
    • Verify: no grafana service references remain; all tests pass; frontend builds and lints.

Slice 3 total: ~250–350 changed lines (mostly deletions).

Integration and acceptance verification

  • 4.1 Full backend test run

    • Run: cd backend && PYTHONPATH=src pytest
    • Verify: all tests pass; no grafana test references remain; prometheus range/gauge/mean tests pass.

  • 4.2 Full frontend build + lint

    • Run: cd frontend && npm run build && npm run lint
    • Verify: TypeScript compiles; no lint failures; no grafana imports unresolved.

  • 4.3 Grep-clean verification

    • Run: grep -ri grafana backend/src --include='*.py' → zero matches
    • Run: grep -ri grafana frontend/src → zero matches excluding Dashboard.test.tsx shortcut fixture
    • Verify: SC-115, SC-116 satisfied.

  • 4.4 config.yaml accuracy check

    • Verify: openspec/config.yaml does not contain "Do NOT re-implement charting" or "No recharts/d3"; reflects recharts as sanctioned renderer (SC-121).

  • 4.5 CHANGELOG check

    • Verify: CHANGELOG.md documents the grafana→prometheus migration (SC-122).

Total estimate

Slice Changed lines Satisfies
Slice 1: Prom range path + chart rebrand + helper ~260–330 SC-101…108
Slice 2: Gauge + mean widgets ~310–400 SC-109…114
Slice 3: Grafana removal + config + changelog ~250–350 SC-115…122, 126
Integration verification ~0 SC-123…125, 127
Total ~820–1,080 SC-101…127

Each slice is under the 400-line review budget. Use three chained PRs (S1 → S2 → S3), each independently buildable and green.

Risk flags for the apply phase

  1. git mv for chart widget rename — use git mv (not delete+create) to preserve file history (design §9).
  2. recharts RadialBarChart gauge — try recharts first; if rendering proves fiddly, the proposal sanctions a ~50-line SVG fallback (no new dep). Decision at apply time.
  3. Dashboard.test.tsx "Grafana" literal — this is a user-authored shortcut label in a test fixture, not a grafana service reference. SC-116 grep should not flag it. Flagged for reviewer awareness.
  4. SC-118 textual drift — ObservabilityPage.tsx no longer exists (refactored to service-tabs/). The intent is satisfied by removing get_grafana_status + useGrafanaStatus + LinksTab. Verify against intent, not literal filename.
  5. _range_query / _instant_query refactoring timing — _range_query is created in S1 (1.3) for _fetch_chart; _instant_query is extracted in S2 (2.1) when _fetch_gauge needs it. Both shared helpers must be in place before S3 (which deletes GrafanaWidgetSource and its private chart logic).
  6. Stale project map — the pi-map references ObservabilityPage.tsx and 7 service types (actual: 8, including authentik + backups). Trust source, not the map. Run project_map_patch after source edits and project_map_validate before final handoff.
Reference in New Issue View Git Blame Copy Permalink