alex/manage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2eb649eceb98e7d00f6a1482e9779491808d9989
manage/docs/superpowers/plans/2026-05-11-backup-monitoring.md
T

2114 lines
67 KiB
Markdown
Raw Blame History

# Backup Monitoring Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build a standalone backup monitoring system that receives HTTP reports from an external backup tool, stores job/run/alert data in SQLite, and provides a React frontend with validation, alerting, and charts.
**Architecture:** FastAPI backend with dedicated SQLite tables (`backup_jobs`, `backup_runs`, `backup_alerts`), background poller for missed schedule detection, React frontend with TanStack Query and D3 charts, Bearer token auth for the backup tool API.
**Tech Stack:** FastAPI, SQLite (WAL), Pydantic, React, TypeScript, TanStack Query, D3, MUI
---
## File Structure
### Backend
| File | Responsibility |
|------|--------------|
| `backend/src/media_library_viewer_api/services/settings_store.py` | Add CRUD methods for backup tables |
| `backend/src/media_library_viewer_api/models/backups.py` | Pydantic models for request/response validation |
| `backend/src/media_library_viewer_api/services/backup_alert_engine.py` | Alert generation logic (failed, missed, anomaly) |
| `backend/src/media_library_viewer_api/services/backup_poller.py` | Background poller for missed schedule checks |
| `backend/src/media_library_viewer_api/routers/backups.py` | FastAPI router with all backup endpoints |
| `backend/src/media_library_viewer_api/routers/dashboard.py` | Add `/api/dashboard/backups` endpoint |
| `backend/src/media_library_viewer_api/main.py` | Register backup router and start poller |
| `backend/src/media_library_viewer_api/auth.py` | Add API key auth dependency |
| `backend/tests/test_backups.py` | API and integration tests |
### Frontend
| File | Responsibility |
|------|--------------|
| `frontend/src/types/backups.ts` | TypeScript interfaces for backup data |
| `frontend/src/hooks/useBackups.ts` | TanStack Query hooks for backup API |
| `frontend/src/api/backups.ts` | API client functions |
| `frontend/src/components/BackupsPage.tsx` | Main backups page with tabs |
| `frontend/src/components/BackupJobsTable.tsx` | Jobs list table |
| `frontend/src/components/BackupRunsTable.tsx` | Runs list table with filters |
| `frontend/src/components/BackupAlertsTable.tsx` | Alerts table with acknowledge |
| `frontend/src/components/BackupDashboardWidget.tsx` | Dashboard summary widget |
| `frontend/src/App.tsx` | Add `/backups` route and nav tab |
---
## Task 1: Database Schema
**Files:**
- Modify: `backend/src/media_library_viewer_api/services/settings_store.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_backup_schema_created():
store = get_settings_store()
store.init_schema()
with store.connect() as conn:
tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE 'backup_%'").fetchall()
table_names = [t[0] for t in tables]
assert "backup_jobs" in table_names
assert "backup_runs" in table_names
assert "backup_alerts" in table_names
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_backup_schema_created -v`
Expected: FAIL — tables do not exist
- [ ] **Step 3: Add schema creation to `init_schema`**
In `backend/src/media_library_viewer_api/services/settings_store.py`, add to the `init_schema` method (after existing table creation):
```python
def init_schema(self) -> None:
# ... existing code ...
with self.connect() as conn:
# ... existing tables ...
conn.execute("""
CREATE TABLE IF NOT EXISTS backup_jobs (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
source TEXT,
target TEXT,
schedule_interval_seconds INTEGER,
created_at INTEGER NOT NULL
)
""")
conn.execute("""
CREATE TABLE IF NOT EXISTS backup_runs (
id TEXT PRIMARY KEY,
job_id TEXT NOT NULL,
started_at INTEGER NOT NULL,
ended_at INTEGER,
status TEXT NOT NULL,
bytes_transferred INTEGER,
duration_ms INTEGER,
error_message TEXT,
details_json TEXT,
created_at INTEGER NOT NULL,
FOREIGN KEY (job_id) REFERENCES backup_jobs(id)
)
""")
conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_runs_job_id ON backup_runs(job_id)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_runs_status ON backup_runs(status)")
conn.execute("""
CREATE TABLE IF NOT EXISTS backup_alerts (
id TEXT PRIMARY KEY,
job_id TEXT NOT NULL,
run_id TEXT,
alert_type TEXT NOT NULL,
severity TEXT NOT NULL,
message TEXT NOT NULL,
acknowledged INTEGER NOT NULL DEFAULT 0,
resolved_at INTEGER,
created_at INTEGER NOT NULL,
FOREIGN KEY (job_id) REFERENCES backup_jobs(id),
FOREIGN KEY (run_id) REFERENCES backup_runs(id)
)
""")
conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_job_id ON backup_alerts(job_id)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_acknowledged ON backup_alerts(acknowledged)")
```
- [ ] **Step 4: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_backup_schema_created -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/src/media_library_viewer_api/services/settings_store.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring database schema"
```
---
## Task 2: Pydantic Models
**Files:**
- Create: `backend/src/media_library_viewer_api/models/backups.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_backup_report_model():
from media_library_viewer_api.models.backups import BackupReportRequest
report = BackupReportRequest(
name="test-backup",
started_at="2026-05-11T02:00:00Z",
status="success",
ended_at="2026-05-11T02:15:00Z",
duration_ms=900000,
bytes_transferred=1024,
)
assert report.name == "test-backup"
assert report.status == "success"
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_backup_report_model -v`
Expected: FAIL — module not found
- [ ] **Step 3: Create Pydantic models**
Create `backend/src/media_library_viewer_api/models/backups.py`:
```python
from datetime import datetime
from typing import Any
from pydantic import BaseModel, Field, field_validator
class BackupReportRequest(BaseModel):
name: str = Field(..., min_length=1)
source: str | None = None
target: str | None = None
schedule_interval_seconds: int | None = Field(None, gt=0)
started_at: datetime
ended_at: datetime | None = None
status: str = Field(..., pattern="^(success|failure|in_progress)$")
bytes_transferred: int | None = Field(None, ge=0)
duration_ms: int | None = None
error_message: str | None = None
details: dict[str, Any] | None = None
@field_validator("ended_at", "duration_ms")
@classmethod
def validate_success_fields(cls, v, info):
if info.data.get("status") == "success" and v is None:
raise ValueError(f"{info.field_name} is required when status is 'success'")
return v
class BackupJobResponse(BaseModel):
id: str
name: str
source: str | None
target: str | None
schedule_interval_seconds: int | None
created_at: int
class BackupRunResponse(BaseModel):
id: str
job_id: str
started_at: int
ended_at: int | None
status: str
bytes_transferred: int | None
duration_ms: int | None
error_message: str | None
details_json: dict[str, Any] | None
created_at: int
class BackupAlertResponse(BaseModel):
id: str
job_id: str
run_id: str | None
alert_type: str
severity: str
message: str
acknowledged: bool
resolved_at: int | None
created_at: int
class BackupDashboardSummary(BaseModel):
total_jobs: int
success_rate_24h: float
active_alerts: int
last_failed_at: int | None
```
- [ ] **Step 4: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_backup_report_model -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/src/media_library_viewer_api/models/backups.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring pydantic models"
```
---
## Task 3: Settings Store CRUD Methods
**Files:**
- Modify: `backend/src/media_library_viewer_api/services/settings_store.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_create_job_and_run():
store = get_settings_store()
store.init_schema()
job = store.upsert_backup_job({
"name": "test-job",
"source": "server-a:/data",
"target": "server-b:/backups",
"schedule_interval_seconds": 86400,
})
assert job["name"] == "test-job"
run = store.create_backup_run({
"job_id": job["id"],
"started_at": 1715392800,
"status": "success",
"ended_at": 1715393700,
"duration_ms": 900000,
"bytes_transferred": 1024,
})
assert run["job_id"] == job["id"]
assert run["status"] == "success"
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_create_job_and_run -v`
Expected: FAIL — methods don't exist
- [ ] **Step 3: Implement CRUD methods**
Add to `backend/src/media_library_viewer_api/services/settings_store.py`:
```python
def _row_to_job(self, row: sqlite3.Row) -> dict[str, Any]:
return {
"id": row["id"],
"name": row["name"],
"source": row["source"],
"target": row["target"],
"schedule_interval_seconds": row["schedule_interval_seconds"],
"created_at": row["created_at"],
}
def _row_to_run(self, row: sqlite3.Row) -> dict[str, Any]:
details = None
if row["details_json"]:
try:
details = json.loads(row["details_json"])
except json.JSONDecodeError:
details = {"raw": row["details_json"]}
return {
"id": row["id"],
"job_id": row["job_id"],
"started_at": row["started_at"],
"ended_at": row["ended_at"],
"status": row["status"],
"bytes_transferred": row["bytes_transferred"],
"duration_ms": row["duration_ms"],
"error_message": row["error_message"],
"details_json": details,
"created_at": row["created_at"],
}
def _row_to_alert(self, row: sqlite3.Row) -> dict[str, Any]:
return {
"id": row["id"],
"job_id": row["job_id"],
"run_id": row["run_id"],
"alert_type": row["alert_type"],
"severity": row["severity"],
"message": row["message"],
"acknowledged": bool(row["acknowledged"]),
"resolved_at": row["resolved_at"],
"created_at": row["created_at"],
}
def get_backup_job_by_name(self, name: str) -> dict[str, Any] | None:
self.init_schema()
with self.connect() as conn:
row = conn.execute("SELECT * FROM backup_jobs WHERE name = ?", (name,)).fetchone()
return self._row_to_job(row) if row else None
def upsert_backup_job(self, payload: dict[str, Any]) -> dict[str, Any]:
self.init_schema()
job_id = payload.get("id") or str(uuid.uuid4())
now = int(time.time())
with self.connect() as conn:
existing = conn.execute("SELECT created_at FROM backup_jobs WHERE id = ?", (job_id,)).fetchone()
if not existing:
existing = conn.execute("SELECT id, created_at FROM backup_jobs WHERE name = ?", (payload["name"],)).fetchone()
if existing:
job_id = existing["id"]
created_at = int(existing["created_at"]) if existing else now
conn.execute("""
INSERT INTO backup_jobs (id, name, source, target, schedule_interval_seconds, created_at)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
name = excluded.name,
source = COALESCE(excluded.source, source),
target = COALESCE(excluded.target, target),
schedule_interval_seconds = COALESCE(excluded.schedule_interval_seconds, schedule_interval_seconds),
created_at = excluded.created_at
""", (job_id, payload["name"], payload.get("source"), payload.get("target"),
payload.get("schedule_interval_seconds"), created_at))
return self.get_backup_job(job_id) or {"id": job_id, **payload, "created_at": created_at}
def get_backup_job(self, job_id: str) -> dict[str, Any] | None:
self.init_schema()
with self.connect() as conn:
row = conn.execute("SELECT * FROM backup_jobs WHERE id = ?", (job_id,)).fetchone()
return self._row_to_job(row) if row else None
def list_backup_jobs(self) -> list[dict[str, Any]]:
self.init_schema()
with self.connect() as conn:
rows = conn.execute("SELECT * FROM backup_jobs ORDER BY created_at DESC").fetchall()
return [self._row_to_job(row) for row in rows]
def create_backup_run(self, payload: dict[str, Any]) -> dict[str, Any]:
self.init_schema()
run_id = str(uuid.uuid4())
now = int(time.time())
with self.connect() as conn:
conn.execute("""
INSERT INTO backup_runs (id, job_id, started_at, ended_at, status, bytes_transferred, duration_ms, error_message, details_json, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (run_id, payload["job_id"], payload["started_at"], payload.get("ended_at"),
payload["status"], payload.get("bytes_transferred"), payload.get("duration_ms"),
payload.get("error_message"), json.dumps(payload.get("details_json") or {}), now))
return self.get_backup_run(run_id) or {"id": run_id, **payload, "created_at": now}
def get_backup_run(self, run_id: str) -> dict[str, Any] | None:
self.init_schema()
with self.connect() as conn:
row = conn.execute("SELECT * FROM backup_runs WHERE id = ?", (run_id,)).fetchone()
return self._row_to_run(row) if row else None
def list_backup_runs(self, job_id: str | None = None, status: str | None = None, limit: int = 50) -> list[dict[str, Any]]:
self.init_schema()
query = "SELECT * FROM backup_runs WHERE 1=1"
params: list[Any] = []
if job_id:
query += " AND job_id = ?"
params.append(job_id)
if status:
query += " AND status = ?"
params.append(status)
query += " ORDER BY started_at DESC LIMIT ?"
params.append(limit)
with self.connect() as conn:
rows = conn.execute(query, params).fetchall()
return [self._row_to_run(row) for row in rows]
def get_latest_backup_run(self, job_id: str) -> dict[str, Any] | None:
self.init_schema()
with self.connect() as conn:
row = conn.execute(
"SELECT * FROM backup_runs WHERE job_id = ? ORDER BY started_at DESC LIMIT 1",
(job_id,)
).fetchone()
return self._row_to_run(row) if row else None
def create_backup_alert(self, payload: dict[str, Any]) -> dict[str, Any]:
self.init_schema()
alert_id = str(uuid.uuid4())
now = int(time.time())
with self.connect() as conn:
conn.execute("""
INSERT INTO backup_alerts (id, job_id, run_id, alert_type, severity, message, acknowledged, resolved_at, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (alert_id, payload["job_id"], payload.get("run_id"), payload["alert_type"],
payload["severity"], payload["message"], 0, None, now))
return self.get_backup_alert(alert_id) or {"id": alert_id, **payload, "acknowledged": False, "created_at": now}
def get_backup_alert(self, alert_id: str) -> dict[str, Any] | None:
self.init_schema()
with self.connect() as conn:
row = conn.execute("SELECT * FROM backup_alerts WHERE id = ?", (alert_id,)).fetchone()
return self._row_to_alert(row) if row else None
def list_backup_alerts(self, job_id: str | None = None, acknowledged: bool | None = None, severity: str | None = None) -> list[dict[str, Any]]:
self.init_schema()
query = "SELECT * FROM backup_alerts WHERE 1=1"
params: list[Any] = []
if job_id:
query += " AND job_id = ?"
params.append(job_id)
if acknowledged is not None:
query += " AND acknowledged = ?"
params.append(1 if acknowledged else 0)
if severity:
query += " AND severity = ?"
params.append(severity)
query += " ORDER BY created_at DESC"
with self.connect() as conn:
rows = conn.execute(query, params).fetchall()
return [self._row_to_alert(row) for row in rows]
def acknowledge_backup_alert(self, alert_id: str) -> dict[str, Any] | None:
self.init_schema()
with self.connect() as conn:
conn.execute("UPDATE backup_alerts SET acknowledged = 1 WHERE id = ?", (alert_id,))
return self.get_backup_alert(alert_id)
def resolve_backup_alerts_for_job(self, job_id: str, alert_type: str | None = None) -> int:
self.init_schema()
now = int(time.time())
with self.connect() as conn:
if alert_type:
result = conn.execute(
"UPDATE backup_alerts SET resolved_at = ? WHERE job_id = ? AND alert_type = ? AND resolved_at IS NULL",
(now, job_id, alert_type)
)
else:
result = conn.execute(
"UPDATE backup_alerts SET resolved_at = ? WHERE job_id = ? AND resolved_at IS NULL",
(now, job_id)
)
return result.rowcount
def prune_backup_alerts(self, cutoff_ts: int) -> int:
self.init_schema()
with self.connect() as conn:
result = conn.execute("DELETE FROM backup_alerts WHERE created_at < ?", (cutoff_ts,))
return result.rowcount
```
- [ ] **Step 4: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_create_job_and_run -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/src/media_library_viewer_api/services/settings_store.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring CRUD operations"
```
---
## Task 4: Alert Engine
**Files:**
- Create: `backend/src/media_library_viewer_api/services/backup_alert_engine.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_alert_failed_status():
from media_library_viewer_api.services.backup_alert_engine import generate_alerts_for_run
run = {
"id": "run-1",
"job_id": "job-1",
"status": "failure",
"bytes_transferred": 0,
"duration_ms": 1000,
"started_at": 1715392800,
}
alerts = generate_alerts_for_run(run, [], None)
assert len(alerts) == 1
assert alerts[0]["alert_type"] == "failed_status"
assert alerts[0]["severity"] == "critical"
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_alert_failed_status -v`
Expected: FAIL — module not found
- [ ] **Step 3: Implement alert engine**
Create `backend/src/media_library_viewer_api/services/backup_alert_engine.py`:
```python
import statistics
from typing import Any
def generate_alerts_for_run(
run: dict[str, Any],
previous_runs: list[dict[str, Any]],
job: dict[str, Any] | None,
) -> list[dict[str, Any]]:
alerts = []
job_id = run["job_id"]
# 1. Failed status alert
if run["status"] == "failure":
alerts.append({
"job_id": job_id,
"run_id": run["id"],
"alert_type": "failed_status",
"severity": "critical",
"message": f"Backup job '{job['name'] if job else job_id}' failed: {run.get('error_message', 'No error details')}",
})
# 2. Anomaly size alert
bytes_transferred = run.get("bytes_transferred")
if bytes_transferred is not None and previous_runs:
successful_runs = [r for r in previous_runs if r["status"] == "success" and r.get("bytes_transferred") is not None]
if len(successful_runs) >= 3:
sizes = [r["bytes_transferred"] for r in successful_runs[-7:]]
median_size = statistics.median(sizes)
if median_size > 0:
ratio = bytes_transferred / median_size
if bytes_transferred == 0:
alerts.append({
"job_id": job_id,
"run_id": run["id"],
"alert_type": "anomaly_size",
"severity": "warning",
"message": f"Backup job '{job['name'] if job else job_id}' transferred 0 bytes (median: {median_size})",
})
elif ratio < 0.1 or ratio > 3.0:
alerts.append({
"job_id": job_id,
"run_id": run["id"],
"alert_type": "anomaly_size",
"severity": "warning",
"message": f"Backup job '{job['name'] if job else job_id}' size anomaly: {bytes_transferred} bytes (median: {median_size})",
})
# 3. Anomaly duration alert
duration_ms = run.get("duration_ms")
if duration_ms is not None and previous_runs:
successful_runs = [r for r in previous_runs if r["status"] == "success" and r.get("duration_ms") is not None]
if len(successful_runs) >= 3:
durations = [r["duration_ms"] for r in successful_runs[-7:]]
median_duration = statistics.median(durations)
if median_duration > 0 and duration_ms / median_duration > 3.0:
alerts.append({
"job_id": job_id,
"run_id": run["id"],
"alert_type": "anomaly_duration",
"severity": "warning",
"message": f"Backup job '{job['name'] if job else job_id}' duration anomaly: {duration_ms}ms (median: {median_duration}ms)",
})
return alerts
def check_missed_schedules(
jobs: list[dict[str, Any]],
get_latest_run: callable,
existing_alerts: list[dict[str, Any]],
) -> list[dict[str, Any]]:
alerts = []
now = int(__import__("time").time())
for job in jobs:
interval = job.get("schedule_interval_seconds")
if not interval:
continue
latest_run = get_latest_run(job["id"])
if not latest_run:
# No runs ever — alert if job is older than interval * 1.5
if now - job["created_at"] > interval * 1.5:
alerts.append({
"job_id": job["id"],
"run_id": None,
"alert_type": "missed_schedule",
"severity": "warning",
"message": f"Backup job '{job['name']}' has never run (expected every {interval}s)",
})
else:
last_run_time = latest_run["started_at"]
if now - last_run_time > interval * 1.5:
# Check if there's already an unresolved missed_schedule alert
has_open_alert = any(
a["alert_type"] == "missed_schedule" and a["resolved_at"] is None
for a in existing_alerts if a["job_id"] == job["id"]
)
if not has_open_alert:
alerts.append({
"job_id": job["id"],
"run_id": None,
"alert_type": "missed_schedule",
"severity": "warning",
"message": f"Backup job '{job['name']}' missed schedule: last run at {last_run_time} (expected every {interval}s)",
})
return alerts
```
- [ ] **Step 4: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_alert_failed_status -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/src/media_library_viewer_api/services/backup_alert_engine.py backend/tests/test_backups.py
git commit -m "feat: add backup alert generation engine"
```
---
## Task 5: Background Poller
**Files:**
- Create: `backend/src/media_library_viewer_api/services/backup_poller.py`
- Modify: `backend/src/media_library_viewer_api/main.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_backup_poller_detects_missed_schedule():
from media_library_viewer_api.services.backup_poller import BackupAlertPoller
store = get_settings_store()
store.init_schema()
job = store.upsert_backup_job({
"name": "old-job",
"schedule_interval_seconds": 60,
})
poller = BackupAlertPoller()
poller._run_cycle(store)
alerts = store.list_backup_alerts(job_id=job["id"])
assert len(alerts) == 1
assert alerts[0]["alert_type"] == "missed_schedule"
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_backup_poller_detects_missed_schedule -v`
Expected: FAIL — module not found
- [ ] **Step 3: Implement backup poller**
Create `backend/src/media_library_viewer_api/services/backup_poller.py`:
```python
import logging
import threading
import time
from typing import Any
from .backup_alert_engine import check_missed_schedules
from .settings_store import SettingsStore, get_settings_store
logger = logging.getLogger(__name__)
class BackupAlertPoller:
def __init__(self) -> None:
self._thread: threading.Thread | None = None
self._stop_event = threading.Event()
self._lock = threading.Lock()
self._last_run_at: float | None = None
self._last_success_at: float | None = None
self._last_error: str = ""
self._poll_count = 0
self._error_count = 0
def start(self) -> None:
with self._lock:
if self._thread and self._thread.is_alive():
return
self._stop_event.clear()
self._thread = threading.Thread(target=self._run, name="backup-alert-poller", daemon=True)
self._thread.start()
def stop(self, timeout: float = 5.0) -> None:
with self._lock:
thread = self._thread
if not thread:
return
self._stop_event.set()
thread.join(timeout=timeout)
@property
def status(self) -> dict[str, Any]:
with self._lock:
return {
"worker_running": self._thread is not None and self._thread.is_alive(),
"last_run_at": self._last_run_at,
"last_success_at": self._last_success_at,
"last_error": self._last_error,
"poll_count": self._poll_count,
"error_count": self._error_count,
}
def _run(self) -> None:
# Wait 30 seconds before first check
if self._stop_event.wait(30):
return
store = get_settings_store()
while not self._stop_event.is_set():
try:
self._run_cycle(store)
except Exception:
logger.exception("Backup alert poller cycle failed")
# Check every 5 minutes
if self._stop_event.wait(300):
break
def _run_cycle(self, store: SettingsStore) -> None:
start = time.perf_counter()
jobs = store.list_backup_jobs()
existing_alerts = store.list_backup_alerts(acknowledged=False)
new_alerts = check_missed_schedules(
jobs,
lambda job_id: store.get_latest_backup_run(job_id),
existing_alerts,
)
for alert in new_alerts:
store.create_backup_alert(alert)
# Prune old resolved alerts (90 days)
cutoff = int(time.time()) - (90 * 24 * 60 * 60)
store.prune_backup_alerts(cutoff)
duration_ms = int((time.perf_counter() - start) * 1000)
with self._lock:
self._last_run_at = time.time()
self._last_success_at = time.time()
self._poll_count += 1
_BACKUP_POLLER = BackupAlertPoller()
def get_backup_poller() -> BackupAlertPoller:
return _BACKUP_POLLER
```
- [ ] **Step 4: Wire up poller in main.py**
Modify `backend/src/media_library_viewer_api/main.py`:
```python
# Add import
from .services.backup_poller import get_backup_poller
# In lifespan:
@asynccontextmanager
async def lifespan(app: FastAPI):
settings = get_settings()
configure_logging(settings.log_level)
validate_auth_settings(settings)
mail_queue = get_mail_queue()
monitoring_poller = get_monitoring_poller()
backup_poller = get_backup_poller()
mail_queue.start()
monitoring_poller.start()
backup_poller.start()
yield
monitoring_poller.stop()
backup_poller.stop()
mail_queue.stop()
```
- [ ] **Step 5: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_backup_poller_detects_missed_schedule -v`
Expected: PASS
- [ ] **Step 6: Commit**
```bash
git add backend/src/media_library_viewer_api/services/backup_poller.py backend/src/media_library_viewer_api/main.py backend/tests/test_backups.py
git commit -m "feat: add backup alert background poller"
```
---
## Task 6: API Key Authentication
**Files:**
- Modify: `backend/src/media_library_viewer_api/auth.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_api_key_auth():
from media_library_viewer_api.auth import require_api_key
from fastapi import HTTPException
# Should raise 401 without key
try:
require_api_key("")
assert False, "Should have raised"
except HTTPException as e:
assert e.status_code == 401
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_api_key_auth -v`
Expected: FAIL — function doesn't exist
- [ ] **Step 3: Implement API key auth**
Modify `backend/src/media_library_viewer_api/auth.py`:
```python
# Add import
import secrets
# Add at module level
_API_KEY: str | None = None
def get_api_key() -> str:
global _API_KEY
if _API_KEY is None:
store = get_settings_store()
settings = store.get_settings()
_API_KEY = settings.get("backup_api_key")
if not _API_KEY:
_API_KEY = secrets.token_urlsafe(32)
store.update_setting("backup_api_key", _API_KEY)
return _API_KEY
def require_api_key(authorization: str = Header("", alias="Authorization")) -> str:
expected = f"Bearer {get_api_key()}"
if not authorization or not secrets.compare_digest(authorization, expected):
raise HTTPException(status_code=401, detail="Invalid or missing API key")
return authorization
```
- [ ] **Step 4: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_api_key_auth -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/src/media_library_viewer_api/auth.py backend/tests/test_backups.py
git commit -m "feat: add API key authentication for backup tool"
```
---
## Task 7: Backup Router (Backend API)
**Files:**
- Create: `backend/src/media_library_viewer_api/routers/backups.py`
- Modify: `backend/src/media_library_viewer_api/main.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_post_backup_report():
from fastapi.testclient import TestClient
from media_library_viewer_api.main import app
client = TestClient(app)
# Get API key
from media_library_viewer_api.auth import get_api_key
api_key = get_api_key()
response = client.post(
"/api/backups/report",
headers={"Authorization": f"Bearer {api_key}"},
json={
"name": "test-backup",
"started_at": "2026-05-11T02:00:00Z",
"status": "success",
"ended_at": "2026-05-11T02:15:00Z",
"duration_ms": 900000,
"bytes_transferred": 1024,
},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_post_backup_report -v`
Expected: FAIL — router not registered
- [ ] **Step 3: Implement backup router**
Create `backend/src/media_library_viewer_api/routers/backups.py`:
```python
import time
from typing import Any
from fastapi import APIRouter, Depends, HTTPException
from ..auth import require_api_key
from ..models.backups import (
BackupAlertResponse,
BackupDashboardSummary,
BackupJobResponse,
BackupReportRequest,
BackupRunResponse,
)
from ..services.backup_alert_engine import generate_alerts_for_run
from ..services.settings_store import SettingsStore, get_settings_store
router = APIRouter(prefix="/api/backups", tags=["backups"])
def _get_or_create_job(store: SettingsStore, report: BackupReportRequest) -> dict[str, Any]:
job = store.get_backup_job_by_name(report.name)
if not job:
job = store.upsert_backup_job({
"name": report.name,
"source": report.source,
"target": report.target,
"schedule_interval_seconds": report.schedule_interval_seconds,
})
elif report.schedule_interval_seconds:
# Update schedule if provided
store.upsert_backup_job({
"id": job["id"],
"name": report.name,
"source": report.source,
"target": report.target,
"schedule_interval_seconds": report.schedule_interval_seconds,
})
job = store.get_backup_job(job["id"])
return job
@router.post("/report")
def post_backup_report(
report: BackupReportRequest,
store: SettingsStore = Depends(get_settings_store),
_auth: str = Depends(require_api_key),
) -> BackupRunResponse:
job = _get_or_create_job(store, report)
# Check for duplicate (same job + started_at within 1s)
existing_runs = store.list_backup_runs(job_id=job["id"], limit=5)
started_at_ts = int(report.started_at.timestamp())
for existing in existing_runs:
if abs(existing["started_at"] - started_at_ts) <= 1:
# Update existing run
return store.get_backup_run(existing["id"])
run_data = {
"job_id": job["id"],
"started_at": started_at_ts,
"ended_at": int(report.ended_at.timestamp()) if report.ended_at else None,
"status": report.status,
"bytes_transferred": report.bytes_transferred,
"duration_ms": report.duration_ms,
"error_message": report.error_message,
"details_json": report.details,
}
run = store.create_backup_run(run_data)
# Generate alerts
previous_runs = store.list_backup_runs(job_id=job["id"], status="success", limit=20)
alerts = generate_alerts_for_run(run, previous_runs, job)
for alert in alerts:
store.create_backup_alert(alert)
# Resolve old alerts of the same type if this run is successful
if report.status == "success":
store.resolve_backup_alerts_for_job(job["id"], "failed_status")
store.resolve_backup_alerts_for_job(job["id"], "anomaly_size")
store.resolve_backup_alerts_for_job(job["id"], "anomaly_duration")
return BackupRunResponse(**run)
@router.post("/report/start")
def post_backup_start(
report: BackupReportRequest,
store: SettingsStore = Depends(get_settings_store),
_auth: str = Depends(require_api_key),
) -> BackupRunResponse:
job = _get_or_create_job(store, report)
run_data = {
"job_id": job["id"],
"started_at": int(report.started_at.timestamp()),
"status": "in_progress",
}
run = store.create_backup_run(run_data)
return BackupRunResponse(**run)
@router.get("/jobs")
def get_backup_jobs(
store: SettingsStore = Depends(get_settings_store),
) -> list[BackupJobResponse]:
jobs = store.list_backup_jobs()
return [BackupJobResponse(**job) for job in jobs]
@router.get("/jobs/{job_id}")
def get_backup_job(
job_id: str,
store: SettingsStore = Depends(get_settings_store),
) -> dict[str, Any]:
job = store.get_backup_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Backup job not found")
runs = store.list_backup_runs(job_id=job_id, limit=20)
return {
"job": BackupJobResponse(**job),
"runs": [BackupRunResponse(**run) for run in runs],
}
@router.get("/runs")
def get_backup_runs(
job_id: str | None = None,
status: str | None = None,
limit: int = 50,
store: SettingsStore = Depends(get_settings_store),
) -> list[BackupRunResponse]:
runs = store.list_backup_runs(job_id=job_id, status=status, limit=limit)
return [BackupRunResponse(**run) for run in runs]
@router.get("/runs/{run_id}")
def get_backup_run(
run_id: str,
store: SettingsStore = Depends(get_settings_store),
) -> BackupRunResponse:
run = store.get_backup_run(run_id)
if not run:
raise HTTPException(status_code=404, detail="Backup run not found")
return BackupRunResponse(**run)
@router.get("/alerts")
def get_backup_alerts(
job_id: str | None = None,
acknowledged: bool | None = None,
severity: str | None = None,
store: SettingsStore = Depends(get_settings_store),
) -> list[BackupAlertResponse]:
alerts = store.list_backup_alerts(job_id=job_id, acknowledged=acknowledged, severity=severity)
return [BackupAlertResponse(**alert) for alert in alerts]
@router.post("/alerts/{alert_id}/acknowledge")
def acknowledge_backup_alert(
alert_id: str,
store: SettingsStore = Depends(get_settings_store),
) -> BackupAlertResponse:
alert = store.acknowledge_backup_alert(alert_id)
if not alert:
raise HTTPException(status_code=404, detail="Alert not found")
return BackupAlertResponse(**alert)
```
- [ ] **Step 4: Register router in main.py**
Add to `backend/src/media_library_viewer_api/main.py`:
```python
from .routers import backups as backups_router
# In router registration section:
app.include_router(backups_router.router)
```
- [ ] **Step 5: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_post_backup_report -v`
Expected: PASS
- [ ] **Step 6: Commit**
```bash
git add backend/src/media_library_viewer_api/routers/backups.py backend/src/media_library_viewer_api/main.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring API endpoints"
```
---
## Task 8: Dashboard Endpoint
**Files:**
- Modify: `backend/src/media_library_viewer_api/routers/dashboard.py`
- Test: `backend/tests/test_backups.py`
- [ ] **Step 1: Write the failing test**
```python
def test_dashboard_backups():
from fastapi.testclient import TestClient
from media_library_viewer_api.main import app
client = TestClient(app)
response = client.get("/api/dashboard/backups")
assert response.status_code == 200
data = response.json()
assert "total_jobs" in data
assert "success_rate_24h" in data
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && pytest tests/test_backups.py::test_dashboard_backups -v`
Expected: FAIL — endpoint not found
- [ ] **Step 3: Implement dashboard endpoint**
Modify `backend/src/media_library_viewer_api/routers/dashboard.py`:
```python
# Add import
from ..models.backups import BackupDashboardSummary
# Add endpoint:
@router.get("/backups")
def get_backup_dashboard(
store: SettingsStore = Depends(get_settings_store),
) -> BackupDashboardSummary:
jobs = store.list_backup_jobs()
total_jobs = len(jobs)
# Calculate 24h success rate
cutoff = int(time.time()) - (24 * 60 * 60)
recent_runs = []
for job in jobs:
runs = store.list_backup_runs(job_id=job["id"], limit=1)
if runs and runs[0]["started_at"] >= cutoff:
recent_runs.append(runs[0])
successful = sum(1 for r in recent_runs if r["status"] == "success")
success_rate = (successful / len(recent_runs) * 100) if recent_runs else 100.0
# Active alerts
alerts = store.list_backup_alerts(acknowledged=False)
active_alerts = len(alerts)
# Last failed
failed_runs = []
for job in jobs:
runs = store.list_backup_runs(job_id=job["id"], status="failure", limit=1)
if runs:
failed_runs.append(runs[0])
last_failed_at = None
if failed_runs:
last_failed_at = max(r["started_at"] for r in failed_runs)
return BackupDashboardSummary(
total_jobs=total_jobs,
success_rate_24h=round(success_rate, 1),
active_alerts=active_alerts,
last_failed_at=last_failed_at,
)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `cd backend && pytest tests/test_backups.py::test_dashboard_backups -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/src/media_library_viewer_api/routers/dashboard.py backend/tests/test_backups.py
git commit -m "feat: add backup dashboard summary endpoint"
```
---
## Task 9: Frontend Types
**Files:**
- Create: `frontend/src/types/backups.ts`
- [ ] **Step 1: Write the type definitions**
Create `frontend/src/types/backups.ts`:
```typescript
export interface BackupJob {
id: string;
name: string;
source: string | null;
target: string | null;
schedule_interval_seconds: number | null;
created_at: number;
}
export interface BackupRun {
id: string;
job_id: string;
started_at: number;
ended_at: number | null;
status: "success" | "failure" | "in_progress";
bytes_transferred: number | null;
duration_ms: number | null;
error_message: string | null;
details_json: Record<string, unknown> | null;
created_at: number;
}
export interface BackupAlert {
id: string;
job_id: string;
run_id: string | null;
alert_type: "missed_schedule" | "failed_status" | "anomaly_size" | "anomaly_duration";
severity: "warning" | "critical";
message: string;
acknowledged: boolean;
resolved_at: number | null;
created_at: number;
}
export interface BackupDashboardSummary {
total_jobs: number;
success_rate_24h: number;
active_alerts: number;
last_failed_at: number | null;
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/types/backups.ts
git commit -m "feat: add backup monitoring TypeScript types"
```
---
## Task 10: Frontend API Client
**Files:**
- Create: `frontend/src/api/backups.ts`
- [ ] **Step 1: Write the API client**
Create `frontend/src/api/backups.ts`:
```typescript
import {
BackupAlert,
BackupDashboardSummary,
BackupJob,
BackupRun,
} from "../types/backups";
const API_BASE = "/api";
export async function fetchBackupJobs(): Promise<BackupJob[]> {
const res = await fetch(`${API_BASE}/backups/jobs`);
if (!res.ok) throw new Error("Failed to fetch backup jobs");
return res.json();
}
export async function fetchBackupJob(jobId: string): Promise<{ job: BackupJob; runs: BackupRun[] }> {
const res = await fetch(`${API_BASE}/backups/jobs/${jobId}`);
if (!res.ok) throw new Error("Failed to fetch backup job");
return res.json();
}
export async function fetchBackupRuns(jobId?: string, status?: string): Promise<BackupRun[]> {
const params = new URLSearchParams();
if (jobId) params.append("job_id", jobId);
if (status) params.append("status", status);
const res = await fetch(`${API_BASE}/backups/runs?${params}`);
if (!res.ok) throw new Error("Failed to fetch backup runs");
return res.json();
}
export async function fetchBackupRun(runId: string): Promise<BackupRun> {
const res = await fetch(`${API_BASE}/backups/runs/${runId}`);
if (!res.ok) throw new Error("Failed to fetch backup run");
return res.json();
}
export async function fetchBackupAlerts(
jobId?: string,
acknowledged?: boolean,
severity?: string
): Promise<BackupAlert[]> {
const params = new URLSearchParams();
if (jobId) params.append("job_id", jobId);
if (acknowledged !== undefined) params.append("acknowledged", String(acknowledged));
if (severity) params.append("severity", severity);
const res = await fetch(`${API_BASE}/backups/alerts?${params}`);
if (!res.ok) throw new Error("Failed to fetch backup alerts");
return res.json();
}
export async function acknowledgeBackupAlert(alertId: string): Promise<BackupAlert> {
const res = await fetch(`${API_BASE}/backups/alerts/${alertId}/acknowledge`, {
method: "POST",
});
if (!res.ok) throw new Error("Failed to acknowledge alert");
return res.json();
}
export async function fetchBackupDashboard(): Promise<BackupDashboardSummary> {
const res = await fetch(`${API_BASE}/dashboard/backups`);
if (!res.ok) throw new Error("Failed to fetch backup dashboard");
return res.json();
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/api/backups.ts
git commit -m "feat: add backup monitoring API client"
```
---
## Task 11: Frontend Hooks
**Files:**
- Create: `frontend/src/hooks/useBackups.ts`
- [ ] **Step 1: Write the hooks**
Create `frontend/src/hooks/useBackups.ts`:
```typescript
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import {
acknowledgeBackupAlert,
fetchBackupAlerts,
fetchBackupDashboard,
fetchBackupJob,
fetchBackupJobs,
fetchBackupRuns,
} from "../api/backups";
export function useBackupJobs() {
return useQuery({
queryKey: ["backups", "jobs"],
queryFn: fetchBackupJobs,
refetchInterval: 30_000,
});
}
export function useBackupJob(jobId: string) {
return useQuery({
queryKey: ["backups", "jobs", jobId],
queryFn: () => fetchBackupJob(jobId),
enabled: !!jobId,
});
}
export function useBackupRuns(jobId?: string, status?: string) {
return useQuery({
queryKey: ["backups", "runs", jobId, status],
queryFn: () => fetchBackupRuns(jobId, status),
refetchInterval: 15_000,
});
}
export function useBackupAlerts(jobId?: string, acknowledged?: boolean, severity?: string) {
return useQuery({
queryKey: ["backups", "alerts", jobId, acknowledged, severity],
queryFn: () => fetchBackupAlerts(jobId, acknowledged, severity),
refetchInterval: 30_000,
});
}
export function useAcknowledgeAlert() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: acknowledgeBackupAlert,
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ["backups", "alerts"] });
},
});
}
export function useBackupDashboard() {
return useQuery({
queryKey: ["dashboard", "backups"],
queryFn: fetchBackupDashboard,
refetchInterval: 30_000,
});
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/hooks/useBackups.ts
git commit -m "feat: add backup monitoring TanStack Query hooks"
```
---
## Task 12: Backup Jobs Table Component
**Files:**
- Create: `frontend/src/components/BackupJobsTable.tsx`
- [ ] **Step 1: Write the component**
Create `frontend/src/components/BackupJobsTable.tsx`:
```tsx
import {
Chip,
Paper,
Table,
TableBody,
TableCell,
TableContainer,
TableHead,
TableRow,
} from "@mui/material";
import { BackupJob, BackupRun } from "../types/backups";
interface Props {
jobs: BackupJob[];
latestRuns: Map<string, BackupRun>;
}
function formatInterval(seconds: number | null): string {
if (!seconds) return "N/A";
if (seconds < 60) return `${seconds}s`;
if (seconds < 3600) return `${Math.floor(seconds / 60)}m`;
if (seconds < 86400) return `${Math.floor(seconds / 3600)}h`;
return `${Math.floor(seconds / 86400)}d`;
}
function formatTimestamp(ts: number | null): string {
if (!ts) return "Never";
return new Date(ts * 1000).toLocaleString();
}
export default function BackupJobsTable({ jobs, latestRuns }: Props) {
return (
<TableContainer component={Paper}>
<Table>
<TableHead>
<TableRow>
<TableCell>Name</TableCell>
<TableCell>Source</TableCell>
<TableCell>Target</TableCell>
<TableCell>Schedule</TableCell>
<TableCell>Last Status</TableCell>
<TableCell>Last Run</TableCell>
<TableCell>Next Expected</TableCell>
</TableRow>
</TableHead>
<TableBody>
{jobs.map((job) => {
const run = latestRuns.get(job.id);
const status = run?.status ?? "unknown";
const nextExpected = run && job.schedule_interval_seconds
? run.started_at + job.schedule_interval_seconds
: null;
return (
<TableRow key={job.id} hover>
<TableCell>{job.name}</TableCell>
<TableCell>{job.source ?? "—"}</TableCell>
<TableCell>{job.target ?? "—"}</TableCell>
<TableCell>{formatInterval(job.schedule_interval_seconds)}</TableCell>
<TableCell>
<Chip
label={status}
color={
status === "success"
? "success"
: status === "failure"
? "error"
: status === "in_progress"
? "warning"
: "default"
}
size="small"
/>
</TableCell>
<TableCell>{formatTimestamp(run?.started_at ?? null)}</TableCell>
<TableCell>{formatTimestamp(nextExpected)}</TableCell>
</TableRow>
);
})}
</TableBody>
</Table>
</TableContainer>
);
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/components/BackupJobsTable.tsx
git commit -m "feat: add backup jobs table component"
```
---
## Task 13: Backup Runs Table Component
**Files:**
- Create: `frontend/src/components/BackupRunsTable.tsx`
- [ ] **Step 1: Write the component**
Create `frontend/src/components/BackupRunsTable.tsx`:
```tsx
import {
Chip,
FormControl,
InputLabel,
MenuItem,
Paper,
Select,
Table,
TableBody,
TableCell,
TableContainer,
TableHead,
TableRow,
} from "@mui/material";
import { useState } from "react";
import { BackupRun } from "../types/backups";
interface Props {
runs: BackupRun[];
}
function formatBytes(bytes: number | null): string {
if (bytes === null || bytes === undefined) return "—";
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`;
if (bytes < 1024 * 1024 * 1024) return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
return `${(bytes / (1024 * 1024 * 1024)).toFixed(1)} GB`;
}
function formatDuration(ms: number | null): string {
if (ms === null || ms === undefined) return "—";
if (ms < 1000) return `${ms}ms`;
if (ms < 60_000) return `${(ms / 1000).toFixed(1)}s`;
if (ms < 3600_000) return `${(ms / 60_000).toFixed(1)}m`;
return `${(ms / 3600_000).toFixed(1)}h`;
}
function formatTimestamp(ts: number): string {
return new Date(ts * 1000).toLocaleString();
}
export default function BackupRunsTable({ runs }: Props) {
const [statusFilter, setStatusFilter] = useState<string>("all");
const filteredRuns = statusFilter === "all"
? runs
: runs.filter((r) => r.status === statusFilter);
return (
<>
<FormControl sx={{ minWidth: 120, mb: 2 }}>
<InputLabel>Status</InputLabel>
<Select
value={statusFilter}
label="Status"
onChange={(e) => setStatusFilter(e.target.value)}
>
<MenuItem value="all">All</MenuItem>
<MenuItem value="success">Success</MenuItem>
<MenuItem value="failure">Failure</MenuItem>
<MenuItem value="in_progress">In Progress</MenuItem>
</Select>
</FormControl>
<TableContainer component={Paper}>
<Table>
<TableHead>
<TableRow>
<TableCell>Job</TableCell>
<TableCell>Status</TableCell>
<TableCell>Duration</TableCell>
<TableCell>Size</TableCell>
<TableCell>Started</TableCell>
</TableRow>
</TableHead>
<TableBody>
{filteredRuns.map((run) => (
<TableRow key={run.id} hover>
<TableCell>{run.job_id}</TableCell>
<TableCell>
<Chip
label={run.status}
color={
run.status === "success"
? "success"
: run.status === "failure"
? "error"
: "warning"
}
size="small"
/>
</TableCell>
<TableCell>{formatDuration(run.duration_ms)}</TableCell>
<TableCell>{formatBytes(run.bytes_transferred)}</TableCell>
<TableCell>{formatTimestamp(run.started_at)}</TableCell>
</TableRow>
))}
</TableBody>
</Table>
</TableContainer>
</>
);
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/components/BackupRunsTable.tsx
git commit -m "feat: add backup runs table component"
```
---
## Task 14: Backup Alerts Table Component
**Files:**
- Create: `frontend/src/components/BackupAlertsTable.tsx`
- [ ] **Step 1: Write the component**
Create `frontend/src/components/BackupAlertsTable.tsx`:
```tsx
import { Button, Chip, Paper, Table, TableBody, TableCell, TableContainer, TableHead, TableRow } from "@mui/material";
import { BackupAlert } from "../types/backups";
interface Props {
alerts: BackupAlert[];
onAcknowledge: (alertId: string) => void;
}
function formatTimestamp(ts: number): string {
return new Date(ts * 1000).toLocaleString();
}
export default function BackupAlertsTable({ alerts, onAcknowledge }: Props) {
return (
<TableContainer component={Paper}>
<Table>
<TableHead>
<TableRow>
<TableCell>Severity</TableCell>
<TableCell>Type</TableCell>
<TableCell>Message</TableCell>
<TableCell>Created</TableCell>
<TableCell>Actions</TableCell>
</TableRow>
</TableHead>
<TableBody>
{alerts.map((alert) => (
<TableRow key={alert.id} hover>
<TableCell>
<Chip
label={alert.severity}
color={alert.severity === "critical" ? "error" : "warning"}
size="small"
/>
</TableCell>
<TableCell>{alert.alert_type}</TableCell>
<TableCell>{alert.message}</TableCell>
<TableCell>{formatTimestamp(alert.created_at)}</TableCell>
<TableCell>
{!alert.acknowledged && (
<Button
size="small"
variant="outlined"
onClick={() => onAcknowledge(alert.id)}
>
Acknowledge
</Button>
)}
</TableCell>
</TableRow>
))}
</TableBody>
</Table>
</TableContainer>
);
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/components/BackupAlertsTable.tsx
git commit -m "feat: add backup alerts table component"
```
---
## Task 15: Backup Dashboard Widget
**Files:**
- Create: `frontend/src/components/BackupDashboardWidget.tsx`
- [ ] **Step 1: Write the component**
Create `frontend/src/components/BackupDashboardWidget.tsx`:
```tsx
import { Card, CardContent, Typography, Box, Chip } from "@mui/material";
import { useBackupDashboard } from "../hooks/useBackups";
export default function BackupDashboardWidget() {
const { data, isLoading } = useBackupDashboard();
if (isLoading || !data) {
return (
<Card>
<CardContent>
<Typography variant="h6">Backups</Typography>
<Typography color="text.secondary">Loading...</Typography>
</CardContent>
</Card>
);
}
return (
<Card>
<CardContent>
<Typography variant="h6" gutterBottom>Backups</Typography>
<Box sx={{ display: "flex", gap: 2, flexWrap: "wrap" }}>
<Box>
<Typography variant="h4">{data.total_jobs}</Typography>
<Typography variant="body2" color="text.secondary">Jobs</Typography>
</Box>
<Box>
<Typography variant="h4">{data.success_rate_24h}%</Typography>
<Typography variant="body2" color="text.secondary">24h Success</Typography>
</Box>
<Box>
<Typography variant="h4">
{data.active_alerts > 0 ? (
<Chip label={data.active_alerts} color="error" size="small" />
) : (
0
)}
</Typography>
<Typography variant="body2" color="text.secondary">Alerts</Typography>
</Box>
{data.last_failed_at && (
<Box>
<Typography variant="body2" color="error">
Last failed: {new Date(data.last_failed_at * 1000).toLocaleString()}
</Typography>
</Box>
)}
</Box>
</CardContent>
</Card>
);
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/components/BackupDashboardWidget.tsx
git commit -m "feat: add backup dashboard widget"
```
---
## Task 16: Main Backups Page
**Files:**
- Create: `frontend/src/components/BackupsPage.tsx`
- [ ] **Step 1: Write the component**
Create `frontend/src/components/BackupsPage.tsx`:
```tsx
import { Box, Tab, Tabs, Typography } from "@mui/material";
import { useState } from "react";
import {
useAcknowledgeAlert,
useBackupAlerts,
useBackupJobs,
useBackupRuns,
} from "../hooks/useBackups";
import BackupAlertsTable from "./BackupAlertsTable";
import BackupJobsTable from "./BackupJobsTable";
import BackupRunsTable from "./BackupRunsTable";
export default function BackupsPage() {
const [tab, setTab] = useState(0);
const { data: jobsData, isLoading: jobsLoading } = useBackupJobs();
const { data: runsData, isLoading: runsLoading } = useBackupRuns();
const { data: alertsData, isLoading: alertsLoading } = useBackupAlerts(undefined, false);
const acknowledgeMutation = useAcknowledgeAlert();
// Build a map of latest runs per job
const latestRuns = new Map();
if (runsData) {
for (const run of runsData) {
const existing = latestRuns.get(run.job_id);
if (!existing || run.started_at > existing.started_at) {
latestRuns.set(run.job_id, run);
}
}
}
return (
<Box sx={{ p: 3 }}>
<Typography variant="h4" gutterBottom>Backups</Typography>
<Tabs value={tab} onChange={(_, v) => setTab(v)} sx={{ mb: 2 }}>
<Tab label="Jobs" />
<Tab label="Runs" />
<Tab label={`Alerts ${alertsData ? `(${alertsData.length})` : ""}`} />
</Tabs>
{tab === 0 && (
jobsLoading ? (
<Typography>Loading jobs...</Typography>
) : (
<BackupJobsTable jobs={jobsData ?? []} latestRuns={latestRuns} />
)
)}
{tab === 1 && (
runsLoading ? (
<Typography>Loading runs...</Typography>
) : (
<BackupRunsTable runs={runsData ?? []} />
)
)}
{tab === 2 && (
alertsLoading ? (
<Typography>Loading alerts...</Typography>
) : (
<BackupAlertsTable
alerts={alertsData ?? []}
onAcknowledge={(id) => acknowledgeMutation.mutate(id)}
/>
)
)}
</Box>
);
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/components/BackupsPage.tsx
git commit -m "feat: add main backups page with tabs"
```
---
## Task 17: Wire Up Routes and Navigation
**Files:**
- Modify: `frontend/src/App.tsx`
- [ ] **Step 1: Add route and nav**
In `frontend/src/App.tsx`:
```tsx
// Add import
import BackupsPage from "./components/BackupsPage";
// Add route inside <Routes>:
<Route path="/backups" element={<BackupsPage />} />
// Add tab inside <Tabs>:
<Tab value="/backups" label="Backups" component={NavLink} to="/backups" />
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/App.tsx
git commit -m "feat: add backups route and navigation"
```
---
## Task 18: Wire Up Dashboard Widget
**Files:**
- Modify: `frontend/src/components/Dashboard.tsx` (or equivalent)
- [ ] **Step 1: Add widget to dashboard**
Find the dashboard component and add the backup widget:
```tsx
// Add import
import BackupDashboardWidget from "./BackupDashboardWidget";
// Add to dashboard layout (where other widgets are):
<BackupDashboardWidget />
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/components/Dashboard.tsx
git commit -m "feat: add backup widget to dashboard"
```
---
## Task 19: Update REQUIREMENTS.md
**Files:**
- Modify: `docs/REQUIREMENTS.md`
- [ ] **Step 1: Document backup monitoring**
Add a new section to `docs/REQUIREMENTS.md`:
```markdown
## Backup Monitoring
### Overview
The system receives backup execution reports from an external backup tool via HTTP API, stores job and run history, and provides alerting on failures, missed schedules, and anomalies.
### API
- `POST /api/backups/report` — Submit backup run (Bearer token auth)
- `POST /api/backups/report/start` — Mark backup as in_progress
- `GET /api/backups/jobs` — List jobs
- `GET /api/backups/runs` — List runs
- `GET /api/backups/alerts` — List alerts
- `POST /api/backups/alerts/{id}/acknowledge` — Acknowledge alert
- `GET /api/dashboard/backups` — Dashboard summary
### Data Model
- **BackupJob**: id, name, source, target, schedule_interval_seconds, created_at
- **BackupRun**: id, job_id, started_at, ended_at, status, bytes_transferred, duration_ms, error_message, details_json
- **BackupAlert**: id, job_id, run_id, alert_type, severity, message, acknowledged, resolved_at
### Alert Types
- `failed_status` — Backup reported failure (critical)
- `missed_schedule` — No run within 1.5x expected interval (warning)
- `anomaly_size` — Size is 0 or <10% / >300% of 7-day median (warning)
- `anomaly_duration` — Duration >300% of 7-day median (warning)
### Authentication
- Backup tool uses auto-generated Bearer API key
- Frontend uses existing OIDC/JWT auth
```
- [ ] **Step 2: Commit**
```bash
git add docs/REQUIREMENTS.md
git commit -m "docs: add backup monitoring requirements"
```
---
## Task 20: Run Full Test Suite
- [ ] **Step 1: Run backend tests**
```bash
cd backend && pytest tests/test_backups.py -v
```
Expected: All tests pass
- [ ] **Step 2: Run frontend build**
```bash
cd frontend && npm run build
```
Expected: Build succeeds with no errors
- [ ] **Step 3: Run backend lint**
```bash
cd backend && ruff check src/media_library_viewer_api/
```
Expected: No lint errors
- [ ] **Step 4: Commit**
```bash
git commit -m "test: verify backup monitoring implementation"
```
---
## Spec Coverage Check
| Spec Section | Implementing Task |
|-------------|-------------------|
| Data Model (BackupJob, BackupRun, BackupAlert) | Task 1, 3 |
| API Endpoints (report, start, jobs, runs, alerts) | Task 7 |
| Dashboard Endpoint | Task 8 |
| Validation Rules | Task 7 (endpoint validation) |
| Alert Types (failed, missed, anomaly) | Task 4, 5 |
| Alert State Machine | Task 4, 7 |
| Frontend Navigation | Task 17 |
| Dashboard Widget | Task 15, 18 |
| Backups Page (Jobs, Runs, Alerts tabs) | Task 12-16 |
| API Key Auth | Task 6 |
| Background Poller | Task 5 |
| Charts | Out of scope for MVP (future enhancement) |
## Placeholder Scan
- No "TBD", "TODO", "implement later" found
- No vague "add error handling" steps
- All code blocks contain actual code
- No "similar to Task N" references
## Type Consistency Check
- `BackupJob.schedule_interval_seconds` — used consistently across backend and frontend
- `BackupRun.status` — enum values match (`success`, `failure`, `in_progress`)
- `BackupAlert.alert_type` — values match spec
- `BackupAlert.severity` — values match spec (`warning`, `critical`)
- All Pydantic models match TypeScript interfaces
---
## Execution Handoff
**Plan complete and saved to `docs/superpowers/plans/2026-05-11-backup-monitoring.md`. Two execution options:**
**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration
**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints
**Which approach?**
Reference in New Issue View Git Blame Copy Permalink