78e273efe8
5 locked design decisions: step derivation formula (max(15, round(s/200))), shared normalize_prometheus_matrix helper in widgets/prometheus_range.py, recharts RadialBarChart gauge w/ threshold bands, mean via client-side avg over query_range, 3-slice plan each <=400 lines. Source-verified: ObservabilityPage refactored into service-tabs/ (map stale).
463 lines
29 KiB
Markdown
463 lines
29 KiB
Markdown
# SDD Design: Prometheus Direct Charting (drop Grafana middleman)
|
||
|
||
**Change:** `prometheus-direct-charting`
|
||
**Phase:** design
|
||
**Date:** 2026-07-08
|
||
|
||
## 0. Source findings (read before anything else)
|
||
|
||
The proposal and spec were written against a **stale project map**. Reading actual source surfaced deviations the design must account for. Trust source, not the map.
|
||
|
||
| Spec claim | Actual source reality | Design impact |
|
||
|---|---|---|
|
||
| SC-118: "remove the Grafana branch from `ObservabilityPage`" | **`ObservabilityPage.tsx` no longer exists.** It was refactored into a per-service-type `service-tabs/` architecture (confirmed by `LinksTab.tsx` docstring: "Lifts the Grafana deep-link content from the old cross-service ObservabilityPage into an instance-scoped tab"). The stale reference survives only in `.pi-map.md` files. | Removal targets are `service-tabs/LinksTab.tsx` + its test, the `grafana` case in `service-tabs/index.ts`, the `navEntries.ts` grafana entry, `Dashboard.tsx`'s `OBSERVABILITY_TYPES` set, `ServicesPage.tsx` empty-state text, and the `useGrafanaStatus` hook. See §5. |
|
||
| "No recharts/d3 in use" (config rule to repeal) | `recharts ^3.9.2` is **declared, installed, and imported** by `GrafanaChartWidget.tsx`. Confirmed. | Repeal is a doc fix matching reality; recharts is the sanctioned renderer. |
|
||
| Proposal §5.1: "extend `PrometheusWidgetSource` ... to handle `chart`" | `PrometheusWidgetSource.fetch` currently only does instant `/api/v1/query` and returns `{"result": data}`. The frontend `PrometheusMetricWidget` consumes `data.result`. | The chart/mean paths return a **different** shape (`{series}` / `{value}`); dispatch inside `.fetch()` by `widget_kind`. See §2.2. |
|
||
| Map said registry has 7 service types | Source registry has **8**: `alertmanager, authentik, backups, grafana, jellyfin, nextcloud, prometheus, ssh_tasks`. (`authentik` and `backups` were added after the map froze.) | No design impact beyond acknowledging grafana is one of eight, not one of seven. |
|
||
| Frontend `SERVICE_REGISTRY` (registry.ts) | Has a `grafana` binding with `link` + `chart` kinds, and a `prometheus` binding with only `metric`. The `chart` kind must move grafana→prometheus. | Confirmed; §3.2 details the rebinding. |
|
||
|
||
No proposal/spec scope change is required — the *intent* (remove Grafana, direct Prom charting) still holds. Only the **removal targets** differ from what SC-118 literally names. This is flagged explicitly so the tasks phase and reviewer aren't surprised.
|
||
|
||
---
|
||
|
||
## 1. Architecture overview
|
||
|
||
This change cuts Grafana out of the chart data path and makes Prometheus the direct source. The existing recharts renderer is reused unchanged for `chart`; two new widget kinds (`gauge`, `mean`) are added to the `prometheus` service. Grafana is removed entirely.
|
||
|
||
```
|
||
BEFORE AFTER
|
||
─────── ─────
|
||
WidgetData fetch WidgetData fetch
|
||
└► GrafanaWidgetSource └► PrometheusWidgetSource
|
||
└► POST {grafana}/api/ds/query ├► kind=metric → /api/v1/query {result} [unchanged]
|
||
(datasource.type=prometheus) ├► kind=chart → /api/v1/query_range {series} [NEW]
|
||
└► normalize frames → {series} ├► kind=gauge → /api/v1/query {value,...} [NEW]
|
||
└► kind=mean → /api/v1/query_range {value} [NEW]
|
||
|
||
Frontend GrafanaChartWidget (recharts) Frontend PrometheusChartWidget (recharts) [renamed, reused]
|
||
+ PrometheusGaugeWidget (recharts RadialBarChart) [NEW]
|
||
+ PrometheusMeanWidget (MetricCard-style) [NEW]
|
||
|
||
grafana service type / LinkWidget / [REMOVED entirely]
|
||
LinksTab / navEntry / status endpoint
|
||
```
|
||
|
||
**Key constraints carried from the spec:**
|
||
|
||
- Reuse the recharts chart renderer unchanged (SC-106) — the rename is structural.
|
||
- Shared series normalization (SC-102) — one helper, no duplication.
|
||
- Step derived from window presets (SC-104), users never set `step`.
|
||
- `gauge`/`mean` are scalar-only (SC-111, SC-114); `chart` stays multi-series (SC-107).
|
||
- Window presets: `1h`, `6h`, `24h`, `7d` (SC-108, SC-112).
|
||
- All adapter errors return `{"error": str}`, never raise (SC-103).
|
||
- No Grafana references remain (SC-115, SC-116, SC-117, SC-120).
|
||
|
||
---
|
||
|
||
## 2. Backend design
|
||
|
||
### 2.1 New module: `widgets/prometheus_range.py` (shared helpers)
|
||
|
||
Holds the two pieces the spec's downstream-notes asked for: the step-derivation function (SC-104) and the shared normalization helper (SC-102). Putting them in their own module (rather than inside `sources.py`) makes them unit-testable in isolation and reusable by the future `service-storage-harness` change's in-service data path, without `sources.py` growing unbounded.
|
||
|
||
**Window presets and step derivation (SC-104):**
|
||
|
||
```python
|
||
WINDOW_PRESETS: dict[str, int] = {
|
||
"1h": 3_600,
|
||
"6h": 21_600,
|
||
"24h": 86_400,
|
||
"7d": 604_800,
|
||
}
|
||
|
||
# Target ~200 points per window. Step is clamped to >= 15s so Prometheus
|
||
# doesn't reject sub-15s resolutions on high-cardinality queries.
|
||
def step_for_window(window_seconds: int, target_points: int = 200) -> int:
|
||
return max(15, round(window_seconds / target_points))
|
||
```
|
||
|
||
Resulting table (verified, all within the 100–300 target band):
|
||
|
||
| Preset | Window (s) | Derived step (s) | Points |
|
||
|--------|-----------|------------------|--------|
|
||
| `1h` | 3,600 | `max(15, round(3600/200))` = 18 | 200 |
|
||
| `6h` | 21,600 | 108 | 200 |
|
||
| `24h` | 86,400 | 432 | 200 |
|
||
| `7d` | 604,800 | 3,024 | 200 |
|
||
|
||
(At apply time the implementer may round steps to "nicer" values like 15/60/300/1800 for cache-friendliness; the spec only requires the 100–300 band, which the formula satisfies. The formula is the source of truth; the table is illustrative.)
|
||
|
||
**Shared normalization (SC-102) — `normalize_prometheus_matrix`:**
|
||
|
||
```python
|
||
def normalize_prometheus_matrix(
|
||
result: list[dict[str, Any]],
|
||
) -> list[dict[str, Any]]:
|
||
"""Turn a Prometheus /api/v1/query_range `data.result` matrix into
|
||
`{label, points:[{t:int, v:float|None}]}` series — the exact shape the
|
||
frontend chart renderer consumes.
|
||
|
||
Label derivation reuses the rule from the removed Grafana path:
|
||
1. Drop __name__ from metric labels.
|
||
2. If labels remain, join as `k=v k=v`.
|
||
3. Else fall back to "value".
|
||
4. Dedup collisions with a ` (n)` suffix.
|
||
"""
|
||
series: list[dict[str, Any]] = []
|
||
seen: dict[str, int] = {}
|
||
for entry in result:
|
||
metric = entry.get("metric") or {}
|
||
values = entry.get("values") or []
|
||
parts = [f"{k}={v}" for k, v in sorted(metric.items()) if not k.startswith("__")]
|
||
label = " ".join(parts) if parts else "value"
|
||
if label in seen:
|
||
seen[label] += 1
|
||
label = f"{label} ({seen[label]})"
|
||
else:
|
||
seen[label] = 0
|
||
points = []
|
||
for ts, raw in values:
|
||
v = float(raw) if raw not in (None, "NaN", "+Inf", "-Inf") else None
|
||
points.append({"t": int(ts), "v": v})
|
||
series.append({"label": label, "points": points})
|
||
return series
|
||
```
|
||
|
||
This is a **direct extraction** of the label/dedup logic currently inside `GrafanaWidgetSource._fetch_chart`, retargeted at the Prometheus matrix shape (`{metric, values:[[ts,"str"],...]}`) instead of Grafana frames. The dedup rule is identical so users moving a `grafana/chart` widget to `prometheus/chart` see the same labels.
|
||
|
||
### 2.2 `PrometheusWidgetSource` — extend `.fetch()` by `widget_kind`
|
||
|
||
Today `.fetch()` does only the instant-query → `{"result": ...}` path. Extend it to dispatch by `widget_kind` while preserving the existing `metric` behavior byte-for-byte:
|
||
|
||
```python
|
||
class PrometheusWidgetSource:
|
||
async def fetch(self, service, widget_kind, config) -> dict[str, Any]:
|
||
if service is None:
|
||
return {"error": "Prometheus widget is missing its service"}
|
||
base_url = str(service.config.get("base_url") or "").rstrip("/")
|
||
timeout = int(service.config.get("timeout_seconds") or 10)
|
||
|
||
if widget_kind == "metric":
|
||
return await self._fetch_instant(base_url, timeout, config) # unchanged
|
||
if widget_kind == "chart":
|
||
return await self._fetch_chart(base_url, timeout, config) # NEW
|
||
if widget_kind == "gauge":
|
||
return await self._fetch_gauge(base_url, timeout, config) # NEW
|
||
if widget_kind == "mean":
|
||
return await self._fetch_mean(base_url, timeout, config) # NEW
|
||
return {"error": f"Unknown widget kind: {widget_kind}"}
|
||
```
|
||
|
||
**`_fetch_chart` (SC-101, SC-103, SC-104):**
|
||
|
||
```python
|
||
async def _fetch_chart(self, base_url, timeout, config) -> dict[str, Any]:
|
||
promql = config.get("promql")
|
||
if not promql:
|
||
return {"error": "promql is required"}
|
||
window = WINDOW_PRESETS.get(config.get("window", "1h"), WINDOW_PRESETS["1h"])
|
||
step = step_for_window(window)
|
||
end = int(time.time())
|
||
start = end - window
|
||
try:
|
||
resp = await asyncio.wait_for(asyncio.to_thread(
|
||
requests.get, f"{base_url}/api/v1/query_range",
|
||
params={"query": promql, "start": start, "end": end, "step": step},
|
||
timeout=timeout,
|
||
), timeout=timeout)
|
||
resp.raise_for_status()
|
||
payload = resp.json()
|
||
except asyncio.TimeoutError:
|
||
return {"error": "Prometheus query timed out"}
|
||
except requests.RequestException as exc:
|
||
return {"error": f"Prometheus query failed: {exc}"}
|
||
result = payload.get("data", {}).get("result", [])
|
||
return {"series": normalize_prometheus_matrix(result)}
|
||
```
|
||
|
||
**`_fetch_gauge` (SC-109, SC-110, SC-111) — instant query, scalar-only:**
|
||
|
||
```python
|
||
async def _fetch_gauge(self, base_url, timeout, config) -> dict[str, Any]:
|
||
promql = config.get("promql")
|
||
if not promql:
|
||
return {"error": "promql is required"}
|
||
raw = await self._instant_query(base_url, timeout, promql) # shared helper
|
||
if "error" in raw:
|
||
return raw
|
||
result = raw["result"]
|
||
if len(result) != 1:
|
||
return {"error": "Gauge requires a single-series query; refine your PromQL"}
|
||
# vector entry: {metric, value:[ts, "str"]}
|
||
try:
|
||
value = float(result[0]["value"][1])
|
||
except (KeyError, IndexError, ValueError, TypeError):
|
||
return {"error": "Gauge query returned no scalar value"}
|
||
return {
|
||
"value": value,
|
||
"warn_at": config.get("warn_at"),
|
||
"crit_at": config.get("crit_at"),
|
||
"min": config.get("min"),
|
||
"max": config.get("max"),
|
||
"unit": config.get("unit"),
|
||
}
|
||
```
|
||
|
||
`_instant_query` is extracted from the current `metric` path so `metric`/`gauge` share it; it returns `{"result": [...]}` or `{"error": ...}`.
|
||
|
||
**`_fetch_mean` (SC-112, SC-113, SC-114) — range query, client-side mean, scalar-only:**
|
||
|
||
```python
|
||
async def _fetch_mean(self, base_url, timeout, config) -> dict[str, Any]:
|
||
promql = config.get("promql")
|
||
if not promql:
|
||
return {"error": "promql is required"}
|
||
window = WINDOW_PRESETS.get(config.get("window", "1h"), WINDOW_PRESETS["1h"])
|
||
step = step_for_window(window)
|
||
end = int(time.time()); start = end - window
|
||
# ... run query_range, handle errors identically to _fetch_chart ...
|
||
result = payload.get("data", {}).get("result", [])
|
||
if len(result) != 1:
|
||
return {"error": "Mean requires a single-series query; refine your PromQL"}
|
||
points = result[0].get("values") or []
|
||
nums = [float(v) for _, v in points if v not in (None, "NaN", "+Inf", "-Inf")]
|
||
if not nums:
|
||
return {"error": "Mean query returned no numeric samples in the window"}
|
||
mean = sum(nums) / len(nums)
|
||
return {"value": mean, "unit": config.get("unit")}
|
||
```
|
||
|
||
The range-query HTTP call + error handling is shared between `_fetch_chart` and `_fetch_mean` via a private `_range_query(base_url, timeout, promql, window) -> dict` returning `{"matrix": result}` or `{"error": ...}`. This keeps the chart and mean paths DRY without inventing a generic adapter registry (non-goal enforced).
|
||
|
||
### 2.3 `integrations/prometheus.py` — declare new widget kinds
|
||
|
||
Add two Pydantic widget-config models and two `widget_kind(...)` entries; leave `PrometheusMetricWidgetConfig` and the existing `metric` kind untouched (non-goal: `prometheus_metric` stays as-is):
|
||
|
||
```python
|
||
class PrometheusChartWidgetConfig(WidgetConfigBase):
|
||
promql: str
|
||
window: str = "1h" # one of 1h/6h/24h/7d
|
||
|
||
class PrometheusGaugeWidgetConfig(WidgetConfigBase):
|
||
promql: str
|
||
warn_at: float | None = None
|
||
crit_at: float | None = None
|
||
min: float | None = None
|
||
max: float | None = None
|
||
unit: str | None = None
|
||
|
||
class PrometheusMeanWidgetConfig(WidgetConfigBase):
|
||
promql: str
|
||
window: str = "1h"
|
||
unit: str | None = None
|
||
```
|
||
|
||
`DEFINITION.widget_kinds` gains `chart`, `gauge`, `mean` (refresh 60s for chart/mean, 30s for gauge). Validation that `window ∈ {1h,6h,24h,7d}` can be a `field_validator` on the two models that use it, or simply enforced by `WINDOW_PRESETS.get(..., default)` server-side — prefer the latter (lenient) so a future preset addition doesn't require a model change.
|
||
|
||
### 2.4 `integrations/grafana.py` — delete; registry drops the entry
|
||
|
||
- Delete the file.
|
||
- In `integrations/registry.py`, drop the `from ...grafana import DEFINITION as GRAFANA` import and the `GRAFANA.service_type: GRAFANA,` line from `SERVICE_DEFINITIONS`. No other registry change.
|
||
|
||
### 2.5 `widgets/sources.py` — drop Grafana; wire nothing new
|
||
|
||
- Delete `GrafanaWidgetSource` and its `_fetch_chart`.
|
||
- Remove `"grafana": GrafanaWidgetSource(),` from `SERVICE_ADAPTERS`. (The label/dedup logic has already been *extracted* into `prometheus_range.normalize_prometheus_matrix` in §2.1; it is not lost when the Grafana class is deleted.)
|
||
- `PrometheusWidgetSource` gains the three new methods from §2.2.
|
||
|
||
### 2.6 `routers/monitoring.py` — remove `get_grafana_status`
|
||
|
||
Delete the `@router.get("/grafana-status")` endpoint (lines ~175–195). No other monitoring change; `get_prometheus_status` / `get_alertmanager_status` stay.
|
||
|
||
### 2.7 No SettingsStore or DB schema change
|
||
|
||
Widget *instances* are stored generically (`service_id`, `widget_kind`, `config_json`). A `prometheus` `chart` instance is just a row with `service_id=<prom instance>` and `widget_kind="chart"`. Existing `grafana/chart` rows become orphans resolved by the existing "unknown widget" path (SC-119) — no migration code, no schema change (SC-126).
|
||
|
||
---
|
||
|
||
## 3. Frontend design
|
||
|
||
### 3.1 Rename + reuse: `GrafanaChartWidget` → `PrometheusChartWidget`
|
||
|
||
- Rename `frontend/src/widgets/GrafanaChartWidget.tsx` → `PrometheusChartWidget.tsx`; rename the exported function/component.
|
||
- The recharts body (`LineChart`, `Line`, `XAxis`, `YAxis`, `CartesianGrid`, `Tooltip`, `ResponsiveContainer`, `mergeSeries`, `CHART_COLORS`, `formatTime`) is **preserved unchanged** (SC-106).
|
||
- One tiny text fix: the empty-state Alert copy "Check your query and datasource_uid" becomes "Check your PromQL query and window." (`datasource_uid` no longer exists.)
|
||
- Rename test file `__tests__/GrafanaChartWidget.test.tsx` → `PrometheusChartWidget.test.tsx`; update import + the one error-message assertion (`"Grafana api_key is required"` → a Prom error string).
|
||
|
||
### 3.2 `integrations/registry.ts` — rebind + add
|
||
|
||
- Delete the entire `grafana` key from `SERVICE_REGISTRY`.
|
||
- In the `prometheus` binding's `widgets` array, add `chart`, `gauge`, `mean` alongside the existing `metric`. Each has a `configSchema` mirroring §2.3 (`promql`, `window` for chart/mean; `promql`, `warn_at`, `crit_at`, `min`, `max`, `unit` for gauge). `chart`/`mean` refresh 60s; `gauge` 30s.
|
||
- Import the renamed `PrometheusChartWidget` and the two new components.
|
||
|
||
### 3.3 New: `PrometheusGaugeWidget.tsx` (SC-109, SC-110, SC-111)
|
||
|
||
**Renderer choice: recharts `RadialBarChart`.** Justification: recharts is already a dependency (no new dep), `RadialBarChart` renders a single-value gauge with domain bands natively, and it shares tooltip/styling conventions with the chart widget — keeping the two visualizations consistent. The alternative (a ~50-line bespoke SVG gauge) was rejected because it would introduce a second rendering dialect for no benefit; the proposal's "fall back to SVG if recharts proves heavy" fallback remains documented but is not the default.
|
||
|
||
Threshold-band rendering: render three stacked `RadialBar` cells (green `0→warn`, amber `warn→crit`, red `crit→max`) as the track, and a fourth cell (the actual value) as the needle/bar. When `warn_at`/`crit_at` are absent, render a single neutral-color track. `min`/`max` default to `0`/`max(value, 1)` when omitted so the gauge has a sane domain. The component reuses `SectionCard` + `Alert`/`Skeleton` for loading/error states, matching every other widget.
|
||
|
||
Config fields surfaced to the user (matching §2.3): `promql`, `warn_at`, `crit_at`, `min`, `max`, `unit`.
|
||
|
||
### 3.4 New: `PrometheusMeanWidget.tsx` (SC-112, SC-113, SC-114)
|
||
|
||
A single-value display reusing the existing `MetricCard` pattern (already used by `BackupDashboardWidget` / `PrometheusMetricWidget`-adjacent tiles): big number, optional `unit` suffix, optional subtext showing the window ("mean over last 1h"). Loading/error/empty states via `Skeleton`/`Alert` as usual. No charting library involvement — it's a number, deliberately.
|
||
|
||
### 3.5 Grafana removal on the frontend
|
||
|
||
Per the source findings (§0), the removal targets are **not** an `ObservabilityPage` section. They are:
|
||
|
||
| File | Change |
|
||
|------|--------|
|
||
| `widgets/GrafanaLinkWidget.tsx` | **delete** |
|
||
| `widgets/GrafanaChartWidget.tsx` | **rename** to PrometheusChartWidget (§3.1) — not a pure delete |
|
||
| `widgets/index.ts` | drop `GrafanaLinkWidget` export; rename the chart export |
|
||
| `integrations/registry.ts` | drop `grafana` binding (§3.2) |
|
||
| `integrations/navEntries.ts` | drop the `grafana` entry from `SERVICE_TYPE_NAV_ENTRIES`; drop the now-unused `Link2` icon import |
|
||
| `hooks/useObservability.ts` | drop `useGrafanaStatus` + its `fetchGrafanaStatus` import |
|
||
| `api/client.ts` | drop `fetchGrafanaStatus` |
|
||
| `types/index.ts` | drop `GrafanaStatus` interface |
|
||
| `pages/service-tabs/LinksTab.tsx` | **delete** (it is the grafana-specific tab; its `GrafanaLinkCard` + machine deep-links are Grafana-only) |
|
||
| `pages/service-tabs/__tests__/LinksTab.test.tsx` | **delete** |
|
||
| `pages/service-tabs/index.ts` | drop the `LinksTab` import and the `case "grafana":` from `serviceContentTabs` |
|
||
| `pages/Dashboard.tsx` | drop `"grafana"` from `OBSERVABILITY_TYPES` set (line ~68) |
|
||
| `pages/ServicesPage.tsx` | update empty-state copy "Add a Grafana, Prometheus, …" → "Add a Prometheus, …" |
|
||
| `pages/__tests__/Dashboard.test.tsx` | the one "Grafana" label literal there is for a *shortcut* (a website link), unrelated to the Grafana service — **leave it** (grep-clean criterion SC-116 still passes; it's not a grafana service reference, just user-typed shortcut text in a test fixture). Flag for reviewer. |
|
||
|
||
**Spec-text note (not a design change):** SC-118 literally names "the ObservabilityPage Grafana section," which no longer exists. The *intent* of SC-118 ("Grafana status checks are removed") is satisfied by dropping `get_grafana_status` (§2.6) + `useGrafanaStatus`. The tasks phase should note this textual drift so the reviewer doesn't treat it as a missed requirement. If the parent prefers, SC-118 can be reworded in `spec.md` to name `LinksTab`/`useGrafanaStatus` instead; this design does not require that edit to proceed.
|
||
|
||
### 3.6 Types
|
||
|
||
`types/index.ts`: remove `GrafanaStatus`. No new widget-payload types — `chart` uses `{series}` (existing), `gauge` uses `{value, warn_at, crit_at, min, max, unit}` (all optional beyond `value`), `mean` uses `{value, unit?}`. These are read off `data?.data` untyped-as-before; no `WidgetDataResponse` generic change is needed (it's already `data: dict | None`).
|
||
|
||
---
|
||
|
||
## 4. Data flow
|
||
|
||
1. **Chart** — `useWidgetData(widgetId, 60_000)` → `GET /api/widgets/instances/{id}/data` → `PrometheusWidgetSource.fetch(kind="chart")` → `_fetch_chart` → `GET {prom}/api/v1/query_range?query=...&start=...&end=...&step=...` → `normalize_prometheus_matrix` → `{"series":[...]}` → `PrometheusChartWidget` renders recharts (unchanged).
|
||
2. **Gauge** — `useWidgetData(widgetId, 30_000)` → `fetch(kind="gauge")` → `_fetch_gauge` → instant query → scalar-only check → `{"value":..., "warn_at":..., ...}` → `PrometheusGaugeWidget` renders RadialBarChart.
|
||
3. **Mean** — `useWidgetData(widgetId, 60_000)` → `fetch(kind="mean")` → `_fetch_mean` → range query → client-side mean → `{"value":..., "unit":...}` → `PrometheusMeanWidget` renders a MetricCard.
|
||
4. **Metric (unchanged)** — existing path preserved byte-for-byte.
|
||
5. **Orphaned grafana widget** — `resolveWidget` finds no `grafana` binding → returns `undefined` → `WidgetInstanceCard` renders its existing "unknown widget" Alert (SC-119). No crash, no migration.
|
||
|
||
Errors at any step return `{"error": str}` (SC-103); the per-widget `Alert variant="destructive"` renders it and siblings keep polling.
|
||
|
||
---
|
||
|
||
## 5. Testing approach
|
||
|
||
### Backend
|
||
|
||
Extend `backend/tests/test_widgets.py` (and/or a focused `test_prometheus_range.py`):
|
||
|
||
- `normalize_prometheus_matrix`: feed a sample Prom `/api/v1/query_range` `data.result` (two entries, one with `__name__`, one colliding label) → assert `{series}` shape, label dedup, null handling for `"NaN"`.
|
||
- `step_for_window`: assert the 1h/6h/24h/7d → step mapping stays within 100–300 points.
|
||
- `PrometheusWidgetSource` chart path: mock `requests.get` → assert `query_range` URL + params (`start`/`end`/`step` present, no `from_ts`/`to_ts`) and `{"series": ...}` return.
|
||
- Gauge scalar-only: mock instant query returning 2 series → assert `{"error": ...}`.
|
||
- Mean scalar-only + client-side mean: mock range query returning 1 series with known values → assert the arithmetic mean; mock 2 series → assert error.
|
||
- Error paths: timeout / `RequestException` → `{"error": ...}` (no raise).
|
||
|
||
### Frontend
|
||
|
||
- `PrometheusChartWidget.test.tsx` (renamed from GrafanaChartWidget test): loading, error, rendered-data cases (SC-125).
|
||
- `PrometheusGaugeWidget.test.tsx`: loading, error, rendered-with-bands, rendered-without-bands.
|
||
- `PrometheusMeanWidget.test.tsx`: loading, error, rendered value.
|
||
- `registry.test.ts`: assert no `grafana` binding; assert `prometheus` binding has `metric`, `chart`, `gauge`, `mean`.
|
||
|
||
### Build/lint gates
|
||
|
||
Each slice: `PYTHONPATH=src pytest` (from `backend/`), `npm run build`, `npm run lint` must be green.
|
||
|
||
---
|
||
|
||
## 6. File-level plan
|
||
|
||
### Create
|
||
|
||
| File | Rationale |
|
||
|------|-----------|
|
||
| `backend/src/media_library_viewer_api/widgets/prometheus_range.py` | `WINDOW_PRESETS`, `step_for_window`, `normalize_prometheus_matrix` (SC-102, SC-104). |
|
||
| `backend/tests/test_prometheus_range.py` | Unit tests for the helpers. |
|
||
| `frontend/src/widgets/PrometheusGaugeWidget.tsx` | recharts RadialBarChart gauge (SC-109/110/111). |
|
||
| `frontend/src/widgets/PrometheusMeanWidget.tsx` | MetricCard-style mean (SC-112/113/114). |
|
||
| `frontend/src/widgets/__tests__/PrometheusGaugeWidget.test.tsx` | SC-125. |
|
||
| `frontend/src/widgets/__tests__/PrometheusMeanWidget.test.tsx` | SC-125. |
|
||
|
||
### Rename (git mv)
|
||
|
||
| From → To | Rationale |
|
||
|-----------|-----------|
|
||
| `widgets/GrafanaChartWidget.tsx` → `widgets/PrometheusChartWidget.tsx` | SC-105/106. Body preserved. |
|
||
| `widgets/__tests__/GrafanaChartWidget.test.tsx` → `widgets/__tests__/PrometheusChartWidget.test.tsx` | match rename. |
|
||
|
||
### Modify
|
||
|
||
| File | Rationale |
|
||
|------|-----------|
|
||
| `backend/.../widgets/sources.py` | Drop `GrafanaWidgetSource`; add chart/gauge/mean to `PrometheusWidgetSource`. |
|
||
| `backend/.../integrations/prometheus.py` | Add 3 widget-config models + 3 `widget_kind` entries. |
|
||
| `backend/.../integrations/registry.py` | Drop grafana import + entry. |
|
||
| `backend/.../routers/monitoring.py` | Drop `get_grafana_status`. |
|
||
| `backend/tests/test_widgets.py` | Drop grafana adapter tests; add prom chart/gauge/mean tests. |
|
||
| `frontend/src/integrations/registry.ts` | Drop grafana binding; add chart/gauge/mean to prometheus. |
|
||
| `frontend/src/integrations/navEntries.ts` | Drop grafana entry. |
|
||
| `frontend/src/hooks/useObservability.ts` | Drop `useGrafanaStatus`. |
|
||
| `frontend/src/api/client.ts` | Drop `fetchGrafanaStatus`. |
|
||
| `frontend/src/types/index.ts` | Drop `GrafanaStatus`. |
|
||
| `frontend/src/widgets/index.ts` | Drop GrafanaLinkWidget export; rename chart export. |
|
||
| `frontend/src/pages/service-tabs/index.ts` | Drop LinksTab import + grafana case. |
|
||
| `frontend/src/pages/Dashboard.tsx` | Drop "grafana" from `OBSERVABILITY_TYPES`. |
|
||
| `frontend/src/pages/ServicesPage.tsx` | Update empty-state copy. |
|
||
| `openspec/config.yaml` | Repeal stale thin-dashboard/no-recharts wording (SC-121). |
|
||
| `CHANGELOG.md` | Migration note (SC-122). |
|
||
|
||
### Delete
|
||
|
||
| File | Rationale |
|
||
|------|-----------|
|
||
| `backend/.../integrations/grafana.py` | SC-117. |
|
||
| `frontend/src/widgets/GrafanaLinkWidget.tsx` | SC-117. |
|
||
| `frontend/src/pages/service-tabs/LinksTab.tsx` | Grafana-only tab; no Prom equivalent needed (chart widget covers viz). |
|
||
| `frontend/src/pages/service-tabs/__tests__/LinksTab.test.tsx` | matches deletion. |
|
||
|
||
---
|
||
|
||
## 7. Slice boundaries (≤400 changed lines each)
|
||
|
||
Each slice leaves `pytest` + `npm run build` + `npm run lint` green and the app in a working state.
|
||
|
||
### Slice 1 — Prom range path + chart rebrand + shared helper (foundational)
|
||
|
||
**Files:** create `prometheus_range.py` + test; rename `GrafanaChartWidget`→`PrometheusChartWidget` (+ test); modify `sources.py` (`_fetch_chart`, drop nothing yet), `integrations/prometheus.py` (add `chart` kind only), `registry.ts` (move `chart` grafana→prometheus), `widgets/index.ts`, `registry.test.ts`.
|
||
**State after S1:** a `prometheus/chart` widget renders from direct Prom `query_range`; the `grafana/chart` binding is gone but `GrafanaWidgetSource`/`GrafanaLinkWidget`/grafana service type still exist (removal is S3). Grafana link widgets still work. ~300–380 lines.
|
||
|
||
### Slice 2 — Gauge + mean widgets (additive)
|
||
|
||
**Files:** create `PrometheusGaugeWidget.tsx` (+test), `PrometheusMeanWidget.tsx` (+test); modify `sources.py` (`_fetch_gauge`, `_fetch_mean`, shared `_instant_query`/`_range_query`), `integrations/prometheus.py` (add `gauge`/`mean` kinds), `registry.ts` (add gauge/mean bindings), `test_widgets.py` (add adapter tests).
|
||
**State after S2:** gauge + mean widgets selectable and rendering; no grafana change. ~300–380 lines.
|
||
|
||
### Slice 3 — Grafana removal + config + changelog (cleanup)
|
||
|
||
**Files:** delete `integrations/grafana.py`, `GrafanaLinkWidget.tsx`, `LinksTab.tsx` (+test); modify `sources.py` (drop `GrafanaWidgetSource` + `SERVICE_ADAPTERS` entry), `registry.py`, `monitoring.py`, `navEntries.ts`, `useObservability.ts`, `client.ts`, `types/index.ts`, `widgets/index.ts`, `service-tabs/index.ts`, `Dashboard.tsx`, `ServicesPage.tsx`, `test_widgets.py` (drop grafana tests); rewrite `config.yaml`; add `CHANGELOG.md` entry.
|
||
**State after S3:** grep-clean (SC-115/116), config accurate (SC-121), migration documented (SC-122). ~250–350 lines (mostly deletions).
|
||
|
||
**Order:** S1 → S2 → S3. S1 and S2 are independently shippable; S3 must follow S1 (it removes the grafana chart binding S1 replaces).
|
||
|
||
---
|
||
|
||
## 8. Decisions log (answers to spec downstream-notes)
|
||
|
||
1. **Step derivation (SC-104):** `step_for_window(window_seconds, target_points=200) = max(15, round(window_seconds/200))`, in `widgets/prometheus_range.py`. Table in §2.1.
|
||
2. **Shared normalization (SC-102):** `normalize_prometheus_matrix(result)` in `widgets/prometheus_range.py`; extracts the dedup rule from the to-be-deleted Grafana path; retargeted at Prom matrix shape. Signature in §2.1.
|
||
3. **Gauge renderer (SC-110):** recharts `RadialBarChart` (no new dep, consistent styling). Threshold bands via stacked track cells; neutral single-color when `warn_at`/`crit_at` absent. Config fields: `promql, warn_at, crit_at, min, max, unit`. §3.3.
|
||
4. **Mean adapter (SC-112/113):** `_fetch_mean` runs `query_range` over the window preset, averages non-null samples of the single series client-side, returns `{value, unit?}`. Scalar-only enforced via `len(result) != 1 → error`. §2.2.
|
||
5. **Slice plan:** 3 slices (S1 range+rebrand+helper, S2 gauge+mean, S3 grafana removal+config+changelog), each ≤400 lines, order S1→S2→S3. §7.
|
||
|
||
---
|
||
|
||
## 9. Open items for the tasks/apply phases
|
||
|
||
- Implementer should `git mv` the chart widget/test so history is preserved (not delete+create).
|
||
- Implementer should verify the recharts `RadialBarChart` gauge renders acceptably; if it proves fiddly, the proposal's SVG fallback (~50 lines) is sanctioned — but try recharts first.
|
||
- SC-118 text names a file that no longer exists; tasks phase should record this so verify doesn't flag it as a miss. Intent is satisfied by removing `get_grafana_status` + `useGrafanaStatus` + `LinksTab`.
|
||
- `Dashboard.test.tsx` contains the literal "Grafana" in a shortcut test fixture (not a grafana service reference) — grep for SC-116 should be scoped to service/widget references, or that line whitelisted. Flag for reviewer.
|