docs(monitoring): record legacy poller decommission (slice 3)
Update docs to reflect that Manage no longer scrapes its own system metrics (slices 1-2). AGENTS.md, REQUIREMENTS.md (decision log + observability section), monitoring-logging-design.md, MIGRATION_PLAN.md. Gate: docs only; backend pytest (173) + frontend build/lint/test (22/63) remain green from slices 1-2.
This commit is contained in:
@@ -60,10 +60,16 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
|
||||
### Metrics
|
||||
|
||||
- `backend/src/media_library_viewer_api/clients/resources.py` deploys a POSIX shell collector to `/tmp` on each remote machine.
|
||||
- The collector samples `/proc/stat`, `/proc/meminfo`, `/proc/net/dev`, and `/sys/block/*/stat` every 10s and writes JSONL to `/tmp/media_library_viewer_metrics.jsonl`.
|
||||
- `MonitoringPoller` (`monitoring_poller.py`) runs every 5 minutes, reads the remote JSONL, and stores snapshots in SQLite (`monitoring_machine_actions`).
|
||||
- Retention defaults to 30 days with periodic pruning.
|
||||
> **Historical note (2026-06-17):** The legacy Manage-side `MonitoringPoller` that
|
||||
> SSH-scraped `/proc` + `df` into a local SQLite table (`monitoring_machine_actions`)
|
||||
> has been **decommissioned**. System metrics now live entirely in the external
|
||||
> observability stack: `node_exporter` on each machine is scraped by **Prometheus**
|
||||
> and visualised in **Grafana** (see the standalone `docker-compose.observability.yml`
|
||||
> stack). Manage is a thin dashboard: it surfaces Alertmanager alerts + Prometheus
|
||||
> target health + Grafana deep-links, and does not collect or store its own metrics.
|
||||
|
||||
- `main.py` has a `log_requests` middleware that emits method, path, client IP, status code, and elapsed time.
|
||||
- Frontend uses standard `console.log` / browser dev tools; no server-side log aggregation.
|
||||
|
||||
### Alerting
|
||||
|
||||
@@ -210,7 +216,7 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
- Manage API overview (request rate, latency, errors).
|
||||
- Manage operations (SSH commands, media index builds, mail queue).
|
||||
- Backup runs and alert history.
|
||||
- Manage iframe embeds point to specific dashboard panels using Grafana's `panelId` and ` kiosk` mode.
|
||||
- Manage iframe embeds point to specific dashboard panels using Grafana's `panelId` and `kiosk` mode.
|
||||
|
||||
### Manage React UI
|
||||
|
||||
@@ -309,6 +315,7 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
- [x] Wire Grafana OAuth to Authentik.
|
||||
|
||||
**Phase 1 files**:
|
||||
|
||||
- `monitoring/prometheus/prometheus.yml`
|
||||
- `monitoring/prometheus/rules/backup_alerts.yml`
|
||||
- `monitoring/loki/loki.yml`
|
||||
@@ -336,6 +343,7 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
- [x] Remove POSIX collector fallback. The legacy collector code in `backend/src/media_library_viewer_api/clients/resources.py` has been deleted, the collector control endpoints were removed from `routers/monitoring.py`, and `disk_space` was relocated to `services/monitoring_actions.py` as a lightweight SSH/local helper. Metrics are now sourced exclusively from Prometheus/Node Exporter.
|
||||
|
||||
**Phase 2 files**:
|
||||
|
||||
- `backend/src/media_library_viewer_api/jobs.py` (Node Exporter job templates).
|
||||
- `backend/src/media_library_viewer_api/routers/settings.py` (machine input fields + target regeneration).
|
||||
- `backend/src/media_library_viewer_api/services/settings_store.py` (machine persistence fields).
|
||||
@@ -364,6 +372,7 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
- [x] Added tests for the Alertmanager endpoints and the backup success gauge.
|
||||
|
||||
**Phase 3 files**:
|
||||
|
||||
- `backend/src/media_library_viewer_api/routers/monitoring.py` (`/alerts` and `/alertmanager-status` endpoints).
|
||||
- `backend/src/media_library_viewer_api/observability.py` (`BACKUP_RUNS_LAST_SUCCESS` gauge + updated `record_backup_run`).
|
||||
- `backend/src/media_library_viewer_api/routers/backups.py` (pass `success=True` to `record_backup_run` on successful reports).
|
||||
@@ -386,6 +395,7 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
- [x] Wire the new `/observability` route into `App.tsx` and the sidebar navigation.
|
||||
|
||||
**Phase 4 files**:
|
||||
|
||||
- `frontend/src/components/ObservabilityPage.tsx` (page component).
|
||||
- `frontend/src/hooks/useObservability.ts` (React Query hooks).
|
||||
- `frontend/src/api/client.ts` (API client functions).
|
||||
@@ -406,6 +416,7 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
- [ ] Optional: add OpenTelemetry Collector as a translation layer for traces later.
|
||||
|
||||
**Phase 5 files**:
|
||||
|
||||
- `docker-compose.yml` and `docker-compose.dev.yml` (health checks, resource limits, `depends_on` conditions).
|
||||
- `monitoring/prometheus/prometheus.yml` (additional scrape jobs for observability services).
|
||||
- `monitoring/prometheus/rules/backup_alerts.yml` (renamed scope to include observability health alerts).
|
||||
|
||||
Reference in New Issue
Block a user