# Backup Monitoring Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build a standalone backup monitoring system that receives HTTP reports from an external backup tool, stores job/run/alert data in SQLite, and provides a React frontend with validation, alerting, and charts.

**Architecture:** FastAPI backend with dedicated SQLite tables (`backup_jobs`, `backup_runs`, `backup_alerts`), background poller for missed schedule detection, React frontend with TanStack Query and D3 charts, Bearer token auth for the backup tool API.

**Tech Stack:** FastAPI, SQLite (WAL), Pydantic, React, TypeScript, TanStack Query, D3, MUI

---

## File Structure

### Backend

| File | Responsibility |
|------|--------------|
| `backend/src/media_library_viewer_api/services/settings_store.py` | Add CRUD methods for backup tables |
| `backend/src/media_library_viewer_api/models/backups.py` | Pydantic models for request/response validation |
| `backend/src/media_library_viewer_api/services/backup_alert_engine.py` | Alert generation logic (failed, missed, anomaly) |
| `backend/src/media_library_viewer_api/services/backup_poller.py` | Background poller for missed schedule checks |
| `backend/src/media_library_viewer_api/routers/backups.py` | FastAPI router with all backup endpoints |
| `backend/src/media_library_viewer_api/routers/dashboard.py` | Add `/api/dashboard/backups` endpoint |
| `backend/src/media_library_viewer_api/main.py` | Register backup router and start poller |
| `backend/src/media_library_viewer_api/auth.py` | Add API key auth dependency |
| `backend/tests/test_backups.py` | API and integration tests |

### Frontend

| File | Responsibility |
|------|--------------|
| `frontend/src/types/backups.ts` | TypeScript interfaces for backup data |
| `frontend/src/hooks/useBackups.ts` | TanStack Query hooks for backup API |
| `frontend/src/api/backups.ts` | API client functions |
| `frontend/src/components/BackupsPage.tsx` | Main backups page with tabs |
| `frontend/src/components/BackupJobsTable.tsx` | Jobs list table |
| `frontend/src/components/BackupRunsTable.tsx` | Runs list table with filters |
| `frontend/src/components/BackupAlertsTable.tsx` | Alerts table with acknowledge |
| `frontend/src/components/BackupDashboardWidget.tsx` | Dashboard summary widget |
| `frontend/src/App.tsx` | Add `/backups` route and nav tab |

---

## Task 1: Database Schema

**Files:**
- Modify: `backend/src/media_library_viewer_api/services/settings_store.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_backup_schema_created():
    store = get_settings_store()
    store.init_schema()
    with store.connect() as conn:
        tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE 'backup_%'").fetchall()
        table_names = [t[0] for t in tables]
        assert "backup_jobs" in table_names
        assert "backup_runs" in table_names
        assert "backup_alerts" in table_names
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_backup_schema_created -v`
Expected: FAIL — tables do not exist

- [ ] **Step 3: Add schema creation to `init_schema`**

In `backend/src/media_library_viewer_api/services/settings_store.py`, add to the `init_schema` method (after existing table creation):

```python
def init_schema(self) -> None:
    # ... existing code ...
    with self.connect() as conn:
        # ... existing tables ...
        conn.execute("""
            CREATE TABLE IF NOT EXISTS backup_jobs (
                id TEXT PRIMARY KEY,
                name TEXT NOT NULL UNIQUE,
                source TEXT,
                target TEXT,
                schedule_interval_seconds INTEGER,
                created_at INTEGER NOT NULL
            )
        """)
        conn.execute("""
            CREATE TABLE IF NOT EXISTS backup_runs (
                id TEXT PRIMARY KEY,
                job_id TEXT NOT NULL,
                started_at INTEGER NOT NULL,
                ended_at INTEGER,
                status TEXT NOT NULL,
                bytes_transferred INTEGER,
                duration_ms INTEGER,
                error_message TEXT,
                details_json TEXT,
                created_at INTEGER NOT NULL,
                FOREIGN KEY (job_id) REFERENCES backup_jobs(id)
            )
        """)
        conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_runs_job_id ON backup_runs(job_id)")
        conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_runs_status ON backup_runs(status)")
        conn.execute("""
            CREATE TABLE IF NOT EXISTS backup_alerts (
                id TEXT PRIMARY KEY,
                job_id TEXT NOT NULL,
                run_id TEXT,
                alert_type TEXT NOT NULL,
                severity TEXT NOT NULL,
                message TEXT NOT NULL,
                acknowledged INTEGER NOT NULL DEFAULT 0,
                resolved_at INTEGER,
                created_at INTEGER NOT NULL,
                FOREIGN KEY (job_id) REFERENCES backup_jobs(id),
                FOREIGN KEY (run_id) REFERENCES backup_runs(id)
            )
        """)
        conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_job_id ON backup_alerts(job_id)")
        conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_acknowledged ON backup_alerts(acknowledged)")
```

- [ ] **Step 4: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_backup_schema_created -v`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add backend/src/media_library_viewer_api/services/settings_store.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring database schema"
```

---

## Task 2: Pydantic Models

**Files:**
- Create: `backend/src/media_library_viewer_api/models/backups.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_backup_report_model():
    from media_library_viewer_api.models.backups import BackupReportRequest
    
    report = BackupReportRequest(
        name="test-backup",
        started_at="2026-05-11T02:00:00Z",
        status="success",
        ended_at="2026-05-11T02:15:00Z",
        duration_ms=900000,
        bytes_transferred=1024,
    )
    assert report.name == "test-backup"
    assert report.status == "success"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_backup_report_model -v`
Expected: FAIL — module not found

- [ ] **Step 3: Create Pydantic models**

Create `backend/src/media_library_viewer_api/models/backups.py`:

```python
from datetime import datetime
from typing import Any
from pydantic import BaseModel, Field, field_validator


class BackupReportRequest(BaseModel):
    name: str = Field(..., min_length=1)
    source: str | None = None
    target: str | None = None
    schedule_interval_seconds: int | None = Field(None, gt=0)
    started_at: datetime
    ended_at: datetime | None = None
    status: str = Field(..., pattern="^(success|failure|in_progress)$")
    bytes_transferred: int | None = Field(None, ge=0)
    duration_ms: int | None = None
    error_message: str | None = None
    details: dict[str, Any] | None = None

    @field_validator("ended_at", "duration_ms")
    @classmethod
    def validate_success_fields(cls, v, info):
        if info.data.get("status") == "success" and v is None:
            raise ValueError(f"{info.field_name} is required when status is 'success'")
        return v


class BackupJobResponse(BaseModel):
    id: str
    name: str
    source: str | None
    target: str | None
    schedule_interval_seconds: int | None
    created_at: int


class BackupRunResponse(BaseModel):
    id: str
    job_id: str
    started_at: int
    ended_at: int | None
    status: str
    bytes_transferred: int | None
    duration_ms: int | None
    error_message: str | None
    details_json: dict[str, Any] | None
    created_at: int


class BackupAlertResponse(BaseModel):
    id: str
    job_id: str
    run_id: str | None
    alert_type: str
    severity: str
    message: str
    acknowledged: bool
    resolved_at: int | None
    created_at: int


class BackupDashboardSummary(BaseModel):
    total_jobs: int
    success_rate_24h: float
    active_alerts: int
    last_failed_at: int | None
```

- [ ] **Step 4: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_backup_report_model -v`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add backend/src/media_library_viewer_api/models/backups.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring pydantic models"
```

---

## Task 3: Settings Store CRUD Methods

**Files:**
- Modify: `backend/src/media_library_viewer_api/services/settings_store.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_create_job_and_run():
    store = get_settings_store()
    store.init_schema()
    
    job = store.upsert_backup_job({
        "name": "test-job",
        "source": "server-a:/data",
        "target": "server-b:/backups",
        "schedule_interval_seconds": 86400,
    })
    assert job["name"] == "test-job"
    
    run = store.create_backup_run({
        "job_id": job["id"],
        "started_at": 1715392800,
        "status": "success",
        "ended_at": 1715393700,
        "duration_ms": 900000,
        "bytes_transferred": 1024,
    })
    assert run["job_id"] == job["id"]
    assert run["status"] == "success"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_create_job_and_run -v`
Expected: FAIL — methods don't exist

- [ ] **Step 3: Implement CRUD methods**

Add to `backend/src/media_library_viewer_api/services/settings_store.py`:

```python
def _row_to_job(self, row: sqlite3.Row) -> dict[str, Any]:
    return {
        "id": row["id"],
        "name": row["name"],
        "source": row["source"],
        "target": row["target"],
        "schedule_interval_seconds": row["schedule_interval_seconds"],
        "created_at": row["created_at"],
    }

def _row_to_run(self, row: sqlite3.Row) -> dict[str, Any]:
    details = None
    if row["details_json"]:
        try:
            details = json.loads(row["details_json"])
        except json.JSONDecodeError:
            details = {"raw": row["details_json"]}
    return {
        "id": row["id"],
        "job_id": row["job_id"],
        "started_at": row["started_at"],
        "ended_at": row["ended_at"],
        "status": row["status"],
        "bytes_transferred": row["bytes_transferred"],
        "duration_ms": row["duration_ms"],
        "error_message": row["error_message"],
        "details_json": details,
        "created_at": row["created_at"],
    }

def _row_to_alert(self, row: sqlite3.Row) -> dict[str, Any]:
    return {
        "id": row["id"],
        "job_id": row["job_id"],
        "run_id": row["run_id"],
        "alert_type": row["alert_type"],
        "severity": row["severity"],
        "message": row["message"],
        "acknowledged": bool(row["acknowledged"]),
        "resolved_at": row["resolved_at"],
        "created_at": row["created_at"],
    }

def get_backup_job_by_name(self, name: str) -> dict[str, Any] | None:
    self.init_schema()
    with self.connect() as conn:
        row = conn.execute("SELECT * FROM backup_jobs WHERE name = ?", (name,)).fetchone()
    return self._row_to_job(row) if row else None

def upsert_backup_job(self, payload: dict[str, Any]) -> dict[str, Any]:
    self.init_schema()
    job_id = payload.get("id") or str(uuid.uuid4())
    now = int(time.time())
    with self.connect() as conn:
        existing = conn.execute("SELECT created_at FROM backup_jobs WHERE id = ?", (job_id,)).fetchone()
        if not existing:
            existing = conn.execute("SELECT id, created_at FROM backup_jobs WHERE name = ?", (payload["name"],)).fetchone()
            if existing:
                job_id = existing["id"]
        created_at = int(existing["created_at"]) if existing else now
        conn.execute("""
            INSERT INTO backup_jobs (id, name, source, target, schedule_interval_seconds, created_at)
            VALUES (?, ?, ?, ?, ?, ?)
            ON CONFLICT(id) DO UPDATE SET
                name = excluded.name,
                source = COALESCE(excluded.source, source),
                target = COALESCE(excluded.target, target),
                schedule_interval_seconds = COALESCE(excluded.schedule_interval_seconds, schedule_interval_seconds),
                created_at = excluded.created_at
        """, (job_id, payload["name"], payload.get("source"), payload.get("target"),
              payload.get("schedule_interval_seconds"), created_at))
    return self.get_backup_job(job_id) or {"id": job_id, **payload, "created_at": created_at}

def get_backup_job(self, job_id: str) -> dict[str, Any] | None:
    self.init_schema()
    with self.connect() as conn:
        row = conn.execute("SELECT * FROM backup_jobs WHERE id = ?", (job_id,)).fetchone()
    return self._row_to_job(row) if row else None

def list_backup_jobs(self) -> list[dict[str, Any]]:
    self.init_schema()
    with self.connect() as conn:
        rows = conn.execute("SELECT * FROM backup_jobs ORDER BY created_at DESC").fetchall()
    return [self._row_to_job(row) for row in rows]

def create_backup_run(self, payload: dict[str, Any]) -> dict[str, Any]:
    self.init_schema()
    run_id = str(uuid.uuid4())
    now = int(time.time())
    with self.connect() as conn:
        conn.execute("""
            INSERT INTO backup_runs (id, job_id, started_at, ended_at, status, bytes_transferred, duration_ms, error_message, details_json, created_at)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        """, (run_id, payload["job_id"], payload["started_at"], payload.get("ended_at"),
              payload["status"], payload.get("bytes_transferred"), payload.get("duration_ms"),
              payload.get("error_message"), json.dumps(payload.get("details_json") or {}), now))
    return self.get_backup_run(run_id) or {"id": run_id, **payload, "created_at": now}

def get_backup_run(self, run_id: str) -> dict[str, Any] | None:
    self.init_schema()
    with self.connect() as conn:
        row = conn.execute("SELECT * FROM backup_runs WHERE id = ?", (run_id,)).fetchone()
    return self._row_to_run(row) if row else None

def list_backup_runs(self, job_id: str | None = None, status: str | None = None, limit: int = 50) -> list[dict[str, Any]]:
    self.init_schema()
    query = "SELECT * FROM backup_runs WHERE 1=1"
    params: list[Any] = []
    if job_id:
        query += " AND job_id = ?"
        params.append(job_id)
    if status:
        query += " AND status = ?"
        params.append(status)
    query += " ORDER BY started_at DESC LIMIT ?"
    params.append(limit)
    with self.connect() as conn:
        rows = conn.execute(query, params).fetchall()
    return [self._row_to_run(row) for row in rows]

def get_latest_backup_run(self, job_id: str) -> dict[str, Any] | None:
    self.init_schema()
    with self.connect() as conn:
        row = conn.execute(
            "SELECT * FROM backup_runs WHERE job_id = ? ORDER BY started_at DESC LIMIT 1",
            (job_id,)
        ).fetchone()
    return self._row_to_run(row) if row else None

def create_backup_alert(self, payload: dict[str, Any]) -> dict[str, Any]:
    self.init_schema()
    alert_id = str(uuid.uuid4())
    now = int(time.time())
    with self.connect() as conn:
        conn.execute("""
            INSERT INTO backup_alerts (id, job_id, run_id, alert_type, severity, message, acknowledged, resolved_at, created_at)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
        """, (alert_id, payload["job_id"], payload.get("run_id"), payload["alert_type"],
              payload["severity"], payload["message"], 0, None, now))
    return self.get_backup_alert(alert_id) or {"id": alert_id, **payload, "acknowledged": False, "created_at": now}

def get_backup_alert(self, alert_id: str) -> dict[str, Any] | None:
    self.init_schema()
    with self.connect() as conn:
        row = conn.execute("SELECT * FROM backup_alerts WHERE id = ?", (alert_id,)).fetchone()
    return self._row_to_alert(row) if row else None

def list_backup_alerts(self, job_id: str | None = None, acknowledged: bool | None = None, severity: str | None = None) -> list[dict[str, Any]]:
    self.init_schema()
    query = "SELECT * FROM backup_alerts WHERE 1=1"
    params: list[Any] = []
    if job_id:
        query += " AND job_id = ?"
        params.append(job_id)
    if acknowledged is not None:
        query += " AND acknowledged = ?"
        params.append(1 if acknowledged else 0)
    if severity:
        query += " AND severity = ?"
        params.append(severity)
    query += " ORDER BY created_at DESC"
    with self.connect() as conn:
        rows = conn.execute(query, params).fetchall()
    return [self._row_to_alert(row) for row in rows]

def acknowledge_backup_alert(self, alert_id: str) -> dict[str, Any] | None:
    self.init_schema()
    with self.connect() as conn:
        conn.execute("UPDATE backup_alerts SET acknowledged = 1 WHERE id = ?", (alert_id,))
    return self.get_backup_alert(alert_id)

def resolve_backup_alerts_for_job(self, job_id: str, alert_type: str | None = None) -> int:
    self.init_schema()
    now = int(time.time())
    with self.connect() as conn:
        if alert_type:
            result = conn.execute(
                "UPDATE backup_alerts SET resolved_at = ? WHERE job_id = ? AND alert_type = ? AND resolved_at IS NULL",
                (now, job_id, alert_type)
            )
        else:
            result = conn.execute(
                "UPDATE backup_alerts SET resolved_at = ? WHERE job_id = ? AND resolved_at IS NULL",
                (now, job_id)
            )
        return result.rowcount

def prune_backup_alerts(self, cutoff_ts: int) -> int:
    self.init_schema()
    with self.connect() as conn:
        result = conn.execute("DELETE FROM backup_alerts WHERE created_at < ?", (cutoff_ts,))
        return result.rowcount
```

- [ ] **Step 4: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_create_job_and_run -v`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add backend/src/media_library_viewer_api/services/settings_store.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring CRUD operations"
```

---

## Task 4: Alert Engine

**Files:**
- Create: `backend/src/media_library_viewer_api/services/backup_alert_engine.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_alert_failed_status():
    from media_library_viewer_api.services.backup_alert_engine import generate_alerts_for_run
    
    run = {
        "id": "run-1",
        "job_id": "job-1",
        "status": "failure",
        "bytes_transferred": 0,
        "duration_ms": 1000,
        "started_at": 1715392800,
    }
    alerts = generate_alerts_for_run(run, [], None)
    assert len(alerts) == 1
    assert alerts[0]["alert_type"] == "failed_status"
    assert alerts[0]["severity"] == "critical"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_alert_failed_status -v`
Expected: FAIL — module not found

- [ ] **Step 3: Implement alert engine**

Create `backend/src/media_library_viewer_api/services/backup_alert_engine.py`:

```python
import statistics
from typing import Any


def generate_alerts_for_run(
    run: dict[str, Any],
    previous_runs: list[dict[str, Any]],
    job: dict[str, Any] | None,
) -> list[dict[str, Any]]:
    alerts = []
    job_id = run["job_id"]
    
    # 1. Failed status alert
    if run["status"] == "failure":
        alerts.append({
            "job_id": job_id,
            "run_id": run["id"],
            "alert_type": "failed_status",
            "severity": "critical",
            "message": f"Backup job '{job['name'] if job else job_id}' failed: {run.get('error_message', 'No error details')}",
        })
    
    # 2. Anomaly size alert
    bytes_transferred = run.get("bytes_transferred")
    if bytes_transferred is not None and previous_runs:
        successful_runs = [r for r in previous_runs if r["status"] == "success" and r.get("bytes_transferred") is not None]
        if len(successful_runs) >= 3:
            sizes = [r["bytes_transferred"] for r in successful_runs[-7:]]
            median_size = statistics.median(sizes)
            if median_size > 0:
                ratio = bytes_transferred / median_size
                if bytes_transferred == 0:
                    alerts.append({
                        "job_id": job_id,
                        "run_id": run["id"],
                        "alert_type": "anomaly_size",
                        "severity": "warning",
                        "message": f"Backup job '{job['name'] if job else job_id}' transferred 0 bytes (median: {median_size})",
                    })
                elif ratio < 0.1 or ratio > 3.0:
                    alerts.append({
                        "job_id": job_id,
                        "run_id": run["id"],
                        "alert_type": "anomaly_size",
                        "severity": "warning",
                        "message": f"Backup job '{job['name'] if job else job_id}' size anomaly: {bytes_transferred} bytes (median: {median_size})",
                    })
    
    # 3. Anomaly duration alert
    duration_ms = run.get("duration_ms")
    if duration_ms is not None and previous_runs:
        successful_runs = [r for r in previous_runs if r["status"] == "success" and r.get("duration_ms") is not None]
        if len(successful_runs) >= 3:
            durations = [r["duration_ms"] for r in successful_runs[-7:]]
            median_duration = statistics.median(durations)
            if median_duration > 0 and duration_ms / median_duration > 3.0:
                alerts.append({
                    "job_id": job_id,
                    "run_id": run["id"],
                    "alert_type": "anomaly_duration",
                    "severity": "warning",
                    "message": f"Backup job '{job['name'] if job else job_id}' duration anomaly: {duration_ms}ms (median: {median_duration}ms)",
                })
    
    return alerts


def check_missed_schedules(
    jobs: list[dict[str, Any]],
    get_latest_run: callable,
    existing_alerts: list[dict[str, Any]],
) -> list[dict[str, Any]]:
    alerts = []
    now = int(__import__("time").time())
    
    for job in jobs:
        interval = job.get("schedule_interval_seconds")
        if not interval:
            continue
        
        latest_run = get_latest_run(job["id"])
        if not latest_run:
            # No runs ever — alert if job is older than interval * 1.5
            if now - job["created_at"] > interval * 1.5:
                alerts.append({
                    "job_id": job["id"],
                    "run_id": None,
                    "alert_type": "missed_schedule",
                    "severity": "warning",
                    "message": f"Backup job '{job['name']}' has never run (expected every {interval}s)",
                })
        else:
            last_run_time = latest_run["started_at"]
            if now - last_run_time > interval * 1.5:
                # Check if there's already an unresolved missed_schedule alert
                has_open_alert = any(
                    a["alert_type"] == "missed_schedule" and a["resolved_at"] is None
                    for a in existing_alerts if a["job_id"] == job["id"]
                )
                if not has_open_alert:
                    alerts.append({
                        "job_id": job["id"],
                        "run_id": None,
                        "alert_type": "missed_schedule",
                        "severity": "warning",
                        "message": f"Backup job '{job['name']}' missed schedule: last run at {last_run_time} (expected every {interval}s)",
                    })
    
    return alerts
```

- [ ] **Step 4: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_alert_failed_status -v`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add backend/src/media_library_viewer_api/services/backup_alert_engine.py backend/tests/test_backups.py
git commit -m "feat: add backup alert generation engine"
```

---

## Task 5: Background Poller

**Files:**
- Create: `backend/src/media_library_viewer_api/services/backup_poller.py`
- Modify: `backend/src/media_library_viewer_api/main.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_backup_poller_detects_missed_schedule():
    from media_library_viewer_api.services.backup_poller import BackupAlertPoller
    
    store = get_settings_store()
    store.init_schema()
    
    job = store.upsert_backup_job({
        "name": "old-job",
        "schedule_interval_seconds": 60,
    })
    
    poller = BackupAlertPoller()
    poller._run_cycle(store)
    
    alerts = store.list_backup_alerts(job_id=job["id"])
    assert len(alerts) == 1
    assert alerts[0]["alert_type"] == "missed_schedule"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_backup_poller_detects_missed_schedule -v`
Expected: FAIL — module not found

- [ ] **Step 3: Implement backup poller**

Create `backend/src/media_library_viewer_api/services/backup_poller.py`:

```python
import logging
import threading
import time
from typing import Any

from .backup_alert_engine import check_missed_schedules
from .settings_store import SettingsStore, get_settings_store

logger = logging.getLogger(__name__)


class BackupAlertPoller:
    def __init__(self) -> None:
        self._thread: threading.Thread | None = None
        self._stop_event = threading.Event()
        self._lock = threading.Lock()
        self._last_run_at: float | None = None
        self._last_success_at: float | None = None
        self._last_error: str = ""
        self._poll_count = 0
        self._error_count = 0

    def start(self) -> None:
        with self._lock:
            if self._thread and self._thread.is_alive():
                return
            self._stop_event.clear()
            self._thread = threading.Thread(target=self._run, name="backup-alert-poller", daemon=True)
            self._thread.start()

    def stop(self, timeout: float = 5.0) -> None:
        with self._lock:
            thread = self._thread
            if not thread:
                return
            self._stop_event.set()
        thread.join(timeout=timeout)

    @property
    def status(self) -> dict[str, Any]:
        with self._lock:
            return {
                "worker_running": self._thread is not None and self._thread.is_alive(),
                "last_run_at": self._last_run_at,
                "last_success_at": self._last_success_at,
                "last_error": self._last_error,
                "poll_count": self._poll_count,
                "error_count": self._error_count,
            }

    def _run(self) -> None:
        # Wait 30 seconds before first check
        if self._stop_event.wait(30):
            return
        store = get_settings_store()
        while not self._stop_event.is_set():
            try:
                self._run_cycle(store)
            except Exception:
                logger.exception("Backup alert poller cycle failed")
            # Check every 5 minutes
            if self._stop_event.wait(300):
                break

    def _run_cycle(self, store: SettingsStore) -> None:
        start = time.perf_counter()
        jobs = store.list_backup_jobs()
        existing_alerts = store.list_backup_alerts(acknowledged=False)
        
        new_alerts = check_missed_schedules(
            jobs,
            lambda job_id: store.get_latest_backup_run(job_id),
            existing_alerts,
        )
        
        for alert in new_alerts:
            store.create_backup_alert(alert)
        
        # Prune old resolved alerts (90 days)
        cutoff = int(time.time()) - (90 * 24 * 60 * 60)
        store.prune_backup_alerts(cutoff)
        
        duration_ms = int((time.perf_counter() - start) * 1000)
        with self._lock:
            self._last_run_at = time.time()
            self._last_success_at = time.time()
            self._poll_count += 1


_BACKUP_POLLER = BackupAlertPoller()


def get_backup_poller() -> BackupAlertPoller:
    return _BACKUP_POLLER
```

- [ ] **Step 4: Wire up poller in main.py**

Modify `backend/src/media_library_viewer_api/main.py`:

```python
# Add import
from .services.backup_poller import get_backup_poller

# In lifespan:
@asynccontextmanager
async def lifespan(app: FastAPI):
    settings = get_settings()
    configure_logging(settings.log_level)
    validate_auth_settings(settings)
    mail_queue = get_mail_queue()
    monitoring_poller = get_monitoring_poller()
    backup_poller = get_backup_poller()
    mail_queue.start()
    monitoring_poller.start()
    backup_poller.start()
    yield
    monitoring_poller.stop()
    backup_poller.stop()
    mail_queue.stop()
```

- [ ] **Step 5: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_backup_poller_detects_missed_schedule -v`
Expected: PASS

- [ ] **Step 6: Commit**

```bash
git add backend/src/media_library_viewer_api/services/backup_poller.py backend/src/media_library_viewer_api/main.py backend/tests/test_backups.py
git commit -m "feat: add backup alert background poller"
```

---

## Task 6: API Key Authentication

**Files:**
- Modify: `backend/src/media_library_viewer_api/auth.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_api_key_auth():
    from media_library_viewer_api.auth import require_api_key
    from fastapi import HTTPException
    
    # Should raise 401 without key
    try:
        require_api_key("")
        assert False, "Should have raised"
    except HTTPException as e:
        assert e.status_code == 401
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_api_key_auth -v`
Expected: FAIL — function doesn't exist

- [ ] **Step 3: Implement API key auth**

Modify `backend/src/media_library_viewer_api/auth.py`:

```python
# Add import
import secrets

# Add at module level
_API_KEY: str | None = None

def get_api_key() -> str:
    global _API_KEY
    if _API_KEY is None:
        store = get_settings_store()
        settings = store.get_settings()
        _API_KEY = settings.get("backup_api_key")
        if not _API_KEY:
            _API_KEY = secrets.token_urlsafe(32)
            store.update_setting("backup_api_key", _API_KEY)
    return _API_KEY

def require_api_key(authorization: str = Header("", alias="Authorization")) -> str:
    expected = f"Bearer {get_api_key()}"
    if not authorization or not secrets.compare_digest(authorization, expected):
        raise HTTPException(status_code=401, detail="Invalid or missing API key")
    return authorization
```

- [ ] **Step 4: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_api_key_auth -v`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add backend/src/media_library_viewer_api/auth.py backend/tests/test_backups.py
git commit -m "feat: add API key authentication for backup tool"
```

---

## Task 7: Backup Router (Backend API)

**Files:**
- Create: `backend/src/media_library_viewer_api/routers/backups.py`
- Modify: `backend/src/media_library_viewer_api/main.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_post_backup_report():
    from fastapi.testclient import TestClient
    from media_library_viewer_api.main import app
    
    client = TestClient(app)
    
    # Get API key
    from media_library_viewer_api.auth import get_api_key
    api_key = get_api_key()
    
    response = client.post(
        "/api/backups/report",
        headers={"Authorization": f"Bearer {api_key}"},
        json={
            "name": "test-backup",
            "started_at": "2026-05-11T02:00:00Z",
            "status": "success",
            "ended_at": "2026-05-11T02:15:00Z",
            "duration_ms": 900000,
            "bytes_transferred": 1024,
        },
    )
    assert response.status_code == 200
    data = response.json()
    assert data["status"] == "success"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_post_backup_report -v`
Expected: FAIL — router not registered

- [ ] **Step 3: Implement backup router**

Create `backend/src/media_library_viewer_api/routers/backups.py`:

```python
import time
from typing import Any
from fastapi import APIRouter, Depends, HTTPException

from ..auth import require_api_key
from ..models.backups import (
    BackupAlertResponse,
    BackupDashboardSummary,
    BackupJobResponse,
    BackupReportRequest,
    BackupRunResponse,
)
from ..services.backup_alert_engine import generate_alerts_for_run
from ..services.settings_store import SettingsStore, get_settings_store

router = APIRouter(prefix="/api/backups", tags=["backups"])


def _get_or_create_job(store: SettingsStore, report: BackupReportRequest) -> dict[str, Any]:
    job = store.get_backup_job_by_name(report.name)
    if not job:
        job = store.upsert_backup_job({
            "name": report.name,
            "source": report.source,
            "target": report.target,
            "schedule_interval_seconds": report.schedule_interval_seconds,
        })
    elif report.schedule_interval_seconds:
        # Update schedule if provided
        store.upsert_backup_job({
            "id": job["id"],
            "name": report.name,
            "source": report.source,
            "target": report.target,
            "schedule_interval_seconds": report.schedule_interval_seconds,
        })
        job = store.get_backup_job(job["id"])
    return job


@router.post("/report")
def post_backup_report(
    report: BackupReportRequest,
    store: SettingsStore = Depends(get_settings_store),
    _auth: str = Depends(require_api_key),
) -> BackupRunResponse:
    job = _get_or_create_job(store, report)
    
    # Check for duplicate (same job + started_at within 1s)
    existing_runs = store.list_backup_runs(job_id=job["id"], limit=5)
    started_at_ts = int(report.started_at.timestamp())
    for existing in existing_runs:
        if abs(existing["started_at"] - started_at_ts) <= 1:
            # Update existing run
            return store.get_backup_run(existing["id"])
    
    run_data = {
        "job_id": job["id"],
        "started_at": started_at_ts,
        "ended_at": int(report.ended_at.timestamp()) if report.ended_at else None,
        "status": report.status,
        "bytes_transferred": report.bytes_transferred,
        "duration_ms": report.duration_ms,
        "error_message": report.error_message,
        "details_json": report.details,
    }
    run = store.create_backup_run(run_data)
    
    # Generate alerts
    previous_runs = store.list_backup_runs(job_id=job["id"], status="success", limit=20)
    alerts = generate_alerts_for_run(run, previous_runs, job)
    for alert in alerts:
        store.create_backup_alert(alert)
    
    # Resolve old alerts of the same type if this run is successful
    if report.status == "success":
        store.resolve_backup_alerts_for_job(job["id"], "failed_status")
        store.resolve_backup_alerts_for_job(job["id"], "anomaly_size")
        store.resolve_backup_alerts_for_job(job["id"], "anomaly_duration")
    
    return BackupRunResponse(**run)


@router.post("/report/start")
def post_backup_start(
    report: BackupReportRequest,
    store: SettingsStore = Depends(get_settings_store),
    _auth: str = Depends(require_api_key),
) -> BackupRunResponse:
    job = _get_or_create_job(store, report)
    
    run_data = {
        "job_id": job["id"],
        "started_at": int(report.started_at.timestamp()),
        "status": "in_progress",
    }
    run = store.create_backup_run(run_data)
    return BackupRunResponse(**run)


@router.get("/jobs")
def get_backup_jobs(
    store: SettingsStore = Depends(get_settings_store),
) -> list[BackupJobResponse]:
    jobs = store.list_backup_jobs()
    return [BackupJobResponse(**job) for job in jobs]


@router.get("/jobs/{job_id}")
def get_backup_job(
    job_id: str,
    store: SettingsStore = Depends(get_settings_store),
) -> dict[str, Any]:
    job = store.get_backup_job(job_id)
    if not job:
        raise HTTPException(status_code=404, detail="Backup job not found")
    runs = store.list_backup_runs(job_id=job_id, limit=20)
    return {
        "job": BackupJobResponse(**job),
        "runs": [BackupRunResponse(**run) for run in runs],
    }


@router.get("/runs")
def get_backup_runs(
    job_id: str | None = None,
    status: str | None = None,
    limit: int = 50,
    store: SettingsStore = Depends(get_settings_store),
) -> list[BackupRunResponse]:
    runs = store.list_backup_runs(job_id=job_id, status=status, limit=limit)
    return [BackupRunResponse(**run) for run in runs]


@router.get("/runs/{run_id}")
def get_backup_run(
    run_id: str,
    store: SettingsStore = Depends(get_settings_store),
) -> BackupRunResponse:
    run = store.get_backup_run(run_id)
    if not run:
        raise HTTPException(status_code=404, detail="Backup run not found")
    return BackupRunResponse(**run)


@router.get("/alerts")
def get_backup_alerts(
    job_id: str | None = None,
    acknowledged: bool | None = None,
    severity: str | None = None,
    store: SettingsStore = Depends(get_settings_store),
) -> list[BackupAlertResponse]:
    alerts = store.list_backup_alerts(job_id=job_id, acknowledged=acknowledged, severity=severity)
    return [BackupAlertResponse(**alert) for alert in alerts]


@router.post("/alerts/{alert_id}/acknowledge")
def acknowledge_backup_alert(
    alert_id: str,
    store: SettingsStore = Depends(get_settings_store),
) -> BackupAlertResponse:
    alert = store.acknowledge_backup_alert(alert_id)
    if not alert:
        raise HTTPException(status_code=404, detail="Alert not found")
    return BackupAlertResponse(**alert)
```

- [ ] **Step 4: Register router in main.py**

Add to `backend/src/media_library_viewer_api/main.py`:

```python
from .routers import backups as backups_router

# In router registration section:
app.include_router(backups_router.router)
```

- [ ] **Step 5: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_post_backup_report -v`
Expected: PASS

- [ ] **Step 6: Commit**

```bash
git add backend/src/media_library_viewer_api/routers/backups.py backend/src/media_library_viewer_api/main.py backend/tests/test_backups.py
git commit -m "feat: add backup monitoring API endpoints"
```

---

## Task 8: Dashboard Endpoint

**Files:**
- Modify: `backend/src/media_library_viewer_api/routers/dashboard.py`
- Test: `backend/tests/test_backups.py`

- [ ] **Step 1: Write the failing test**

```python
def test_dashboard_backups():
    from fastapi.testclient import TestClient
    from media_library_viewer_api.main import app
    
    client = TestClient(app)
    response = client.get("/api/dashboard/backups")
    assert response.status_code == 200
    data = response.json()
    assert "total_jobs" in data
    assert "success_rate_24h" in data
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd backend && pytest tests/test_backups.py::test_dashboard_backups -v`
Expected: FAIL — endpoint not found

- [ ] **Step 3: Implement dashboard endpoint**

Modify `backend/src/media_library_viewer_api/routers/dashboard.py`:

```python
# Add import
from ..models.backups import BackupDashboardSummary

# Add endpoint:
@router.get("/backups")
def get_backup_dashboard(
    store: SettingsStore = Depends(get_settings_store),
) -> BackupDashboardSummary:
    jobs = store.list_backup_jobs()
    total_jobs = len(jobs)
    
    # Calculate 24h success rate
    cutoff = int(time.time()) - (24 * 60 * 60)
    recent_runs = []
    for job in jobs:
        runs = store.list_backup_runs(job_id=job["id"], limit=1)
        if runs and runs[0]["started_at"] >= cutoff:
            recent_runs.append(runs[0])
    
    successful = sum(1 for r in recent_runs if r["status"] == "success")
    success_rate = (successful / len(recent_runs) * 100) if recent_runs else 100.0
    
    # Active alerts
    alerts = store.list_backup_alerts(acknowledged=False)
    active_alerts = len(alerts)
    
    # Last failed
    failed_runs = []
    for job in jobs:
        runs = store.list_backup_runs(job_id=job["id"], status="failure", limit=1)
        if runs:
            failed_runs.append(runs[0])
    
    last_failed_at = None
    if failed_runs:
        last_failed_at = max(r["started_at"] for r in failed_runs)
    
    return BackupDashboardSummary(
        total_jobs=total_jobs,
        success_rate_24h=round(success_rate, 1),
        active_alerts=active_alerts,
        last_failed_at=last_failed_at,
    )
```

- [ ] **Step 4: Run test to verify it passes**

Run: `cd backend && pytest tests/test_backups.py::test_dashboard_backups -v`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add backend/src/media_library_viewer_api/routers/dashboard.py backend/tests/test_backups.py
git commit -m "feat: add backup dashboard summary endpoint"
```

---

## Task 9: Frontend Types

**Files:**
- Create: `frontend/src/types/backups.ts`

- [ ] **Step 1: Write the type definitions**

Create `frontend/src/types/backups.ts`:

```typescript
export interface BackupJob {
    id: string;
    name: string;
    source: string | null;
    target: string | null;
    schedule_interval_seconds: number | null;
    created_at: number;
}

export interface BackupRun {
    id: string;
    job_id: string;
    started_at: number;
    ended_at: number | null;
    status: "success" | "failure" | "in_progress";
    bytes_transferred: number | null;
    duration_ms: number | null;
    error_message: string | null;
    details_json: Record<string, unknown> | null;
    created_at: number;
}

export interface BackupAlert {
    id: string;
    job_id: string;
    run_id: string | null;
    alert_type: "missed_schedule" | "failed_status" | "anomaly_size" | "anomaly_duration";
    severity: "warning" | "critical";
    message: string;
    acknowledged: boolean;
    resolved_at: number | null;
    created_at: number;
}

export interface BackupDashboardSummary {
    total_jobs: number;
    success_rate_24h: number;
    active_alerts: number;
    last_failed_at: number | null;
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/types/backups.ts
git commit -m "feat: add backup monitoring TypeScript types"
```

---

## Task 10: Frontend API Client

**Files:**
- Create: `frontend/src/api/backups.ts`

- [ ] **Step 1: Write the API client**

Create `frontend/src/api/backups.ts`:

```typescript
import {
    BackupAlert,
    BackupDashboardSummary,
    BackupJob,
    BackupRun,
} from "../types/backups";

const API_BASE = "/api";

export async function fetchBackupJobs(): Promise<BackupJob[]> {
    const res = await fetch(`${API_BASE}/backups/jobs`);
    if (!res.ok) throw new Error("Failed to fetch backup jobs");
    return res.json();
}

export async function fetchBackupJob(jobId: string): Promise<{ job: BackupJob; runs: BackupRun[] }> {
    const res = await fetch(`${API_BASE}/backups/jobs/${jobId}`);
    if (!res.ok) throw new Error("Failed to fetch backup job");
    return res.json();
}

export async function fetchBackupRuns(jobId?: string, status?: string): Promise<BackupRun[]> {
    const params = new URLSearchParams();
    if (jobId) params.append("job_id", jobId);
    if (status) params.append("status", status);
    const res = await fetch(`${API_BASE}/backups/runs?${params}`);
    if (!res.ok) throw new Error("Failed to fetch backup runs");
    return res.json();
}

export async function fetchBackupRun(runId: string): Promise<BackupRun> {
    const res = await fetch(`${API_BASE}/backups/runs/${runId}`);
    if (!res.ok) throw new Error("Failed to fetch backup run");
    return res.json();
}

export async function fetchBackupAlerts(
    jobId?: string,
    acknowledged?: boolean,
    severity?: string
): Promise<BackupAlert[]> {
    const params = new URLSearchParams();
    if (jobId) params.append("job_id", jobId);
    if (acknowledged !== undefined) params.append("acknowledged", String(acknowledged));
    if (severity) params.append("severity", severity);
    const res = await fetch(`${API_BASE}/backups/alerts?${params}`);
    if (!res.ok) throw new Error("Failed to fetch backup alerts");
    return res.json();
}

export async function acknowledgeBackupAlert(alertId: string): Promise<BackupAlert> {
    const res = await fetch(`${API_BASE}/backups/alerts/${alertId}/acknowledge`, {
        method: "POST",
    });
    if (!res.ok) throw new Error("Failed to acknowledge alert");
    return res.json();
}

export async function fetchBackupDashboard(): Promise<BackupDashboardSummary> {
    const res = await fetch(`${API_BASE}/dashboard/backups`);
    if (!res.ok) throw new Error("Failed to fetch backup dashboard");
    return res.json();
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/api/backups.ts
git commit -m "feat: add backup monitoring API client"
```

---

## Task 11: Frontend Hooks

**Files:**
- Create: `frontend/src/hooks/useBackups.ts`

- [ ] **Step 1: Write the hooks**

Create `frontend/src/hooks/useBackups.ts`:

```typescript
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import {
    acknowledgeBackupAlert,
    fetchBackupAlerts,
    fetchBackupDashboard,
    fetchBackupJob,
    fetchBackupJobs,
    fetchBackupRuns,
} from "../api/backups";

export function useBackupJobs() {
    return useQuery({
        queryKey: ["backups", "jobs"],
        queryFn: fetchBackupJobs,
        refetchInterval: 30_000,
    });
}

export function useBackupJob(jobId: string) {
    return useQuery({
        queryKey: ["backups", "jobs", jobId],
        queryFn: () => fetchBackupJob(jobId),
        enabled: !!jobId,
    });
}

export function useBackupRuns(jobId?: string, status?: string) {
    return useQuery({
        queryKey: ["backups", "runs", jobId, status],
        queryFn: () => fetchBackupRuns(jobId, status),
        refetchInterval: 15_000,
    });
}

export function useBackupAlerts(jobId?: string, acknowledged?: boolean, severity?: string) {
    return useQuery({
        queryKey: ["backups", "alerts", jobId, acknowledged, severity],
        queryFn: () => fetchBackupAlerts(jobId, acknowledged, severity),
        refetchInterval: 30_000,
    });
}

export function useAcknowledgeAlert() {
    const queryClient = useQueryClient();
    return useMutation({
        mutationFn: acknowledgeBackupAlert,
        onSuccess: () => {
            queryClient.invalidateQueries({ queryKey: ["backups", "alerts"] });
        },
    });
}

export function useBackupDashboard() {
    return useQuery({
        queryKey: ["dashboard", "backups"],
        queryFn: fetchBackupDashboard,
        refetchInterval: 30_000,
    });
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/hooks/useBackups.ts
git commit -m "feat: add backup monitoring TanStack Query hooks"
```

---

## Task 12: Backup Jobs Table Component

**Files:**
- Create: `frontend/src/components/BackupJobsTable.tsx`

- [ ] **Step 1: Write the component**

Create `frontend/src/components/BackupJobsTable.tsx`:

```tsx
import {
    Chip,
    Paper,
    Table,
    TableBody,
    TableCell,
    TableContainer,
    TableHead,
    TableRow,
} from "@mui/material";
import { BackupJob, BackupRun } from "../types/backups";

interface Props {
    jobs: BackupJob[];
    latestRuns: Map<string, BackupRun>;
}

function formatInterval(seconds: number | null): string {
    if (!seconds) return "N/A";
    if (seconds < 60) return `${seconds}s`;
    if (seconds < 3600) return `${Math.floor(seconds / 60)}m`;
    if (seconds < 86400) return `${Math.floor(seconds / 3600)}h`;
    return `${Math.floor(seconds / 86400)}d`;
}

function formatTimestamp(ts: number | null): string {
    if (!ts) return "Never";
    return new Date(ts * 1000).toLocaleString();
}

export default function BackupJobsTable({ jobs, latestRuns }: Props) {
    return (
        <TableContainer component={Paper}>
            <Table>
                <TableHead>
                    <TableRow>
                        <TableCell>Name</TableCell>
                        <TableCell>Source</TableCell>
                        <TableCell>Target</TableCell>
                        <TableCell>Schedule</TableCell>
                        <TableCell>Last Status</TableCell>
                        <TableCell>Last Run</TableCell>
                        <TableCell>Next Expected</TableCell>
                    </TableRow>
                </TableHead>
                <TableBody>
                    {jobs.map((job) => {
                        const run = latestRuns.get(job.id);
                        const status = run?.status ?? "unknown";
                        const nextExpected = run && job.schedule_interval_seconds
                            ? run.started_at + job.schedule_interval_seconds
                            : null;
                        
                        return (
                            <TableRow key={job.id} hover>
                                <TableCell>{job.name}</TableCell>
                                <TableCell>{job.source ?? "—"}</TableCell>
                                <TableCell>{job.target ?? "—"}</TableCell>
                                <TableCell>{formatInterval(job.schedule_interval_seconds)}</TableCell>
                                <TableCell>
                                    <Chip
                                        label={status}
                                        color={
                                            status === "success"
                                                ? "success"
                                                : status === "failure"
                                                ? "error"
                                                : status === "in_progress"
                                                ? "warning"
                                                : "default"
                                        }
                                        size="small"
                                    />
                                </TableCell>
                                <TableCell>{formatTimestamp(run?.started_at ?? null)}</TableCell>
                                <TableCell>{formatTimestamp(nextExpected)}</TableCell>
                            </TableRow>
                        );
                    })}
                </TableBody>
            </Table>
        </TableContainer>
    );
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/components/BackupJobsTable.tsx
git commit -m "feat: add backup jobs table component"
```

---

## Task 13: Backup Runs Table Component

**Files:**
- Create: `frontend/src/components/BackupRunsTable.tsx`

- [ ] **Step 1: Write the component**

Create `frontend/src/components/BackupRunsTable.tsx`:

```tsx
import {
    Chip,
    FormControl,
    InputLabel,
    MenuItem,
    Paper,
    Select,
    Table,
    TableBody,
    TableCell,
    TableContainer,
    TableHead,
    TableRow,
} from "@mui/material";
import { useState } from "react";
import { BackupRun } from "../types/backups";

interface Props {
    runs: BackupRun[];
}

function formatBytes(bytes: number | null): string {
    if (bytes === null || bytes === undefined) return "—";
    if (bytes < 1024) return `${bytes} B`;
    if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`;
    if (bytes < 1024 * 1024 * 1024) return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
    return `${(bytes / (1024 * 1024 * 1024)).toFixed(1)} GB`;
}

function formatDuration(ms: number | null): string {
    if (ms === null || ms === undefined) return "—";
    if (ms < 1000) return `${ms}ms`;
    if (ms < 60_000) return `${(ms / 1000).toFixed(1)}s`;
    if (ms < 3600_000) return `${(ms / 60_000).toFixed(1)}m`;
    return `${(ms / 3600_000).toFixed(1)}h`;
}

function formatTimestamp(ts: number): string {
    return new Date(ts * 1000).toLocaleString();
}

export default function BackupRunsTable({ runs }: Props) {
    const [statusFilter, setStatusFilter] = useState<string>("all");
    
    const filteredRuns = statusFilter === "all"
        ? runs
        : runs.filter((r) => r.status === statusFilter);
    
    return (
        <>
            <FormControl sx={{ minWidth: 120, mb: 2 }}>
                <InputLabel>Status</InputLabel>
                <Select
                    value={statusFilter}
                    label="Status"
                    onChange={(e) => setStatusFilter(e.target.value)}
                >
                    <MenuItem value="all">All</MenuItem>
                    <MenuItem value="success">Success</MenuItem>
                    <MenuItem value="failure">Failure</MenuItem>
                    <MenuItem value="in_progress">In Progress</MenuItem>
                </Select>
            </FormControl>
            
            <TableContainer component={Paper}>
                <Table>
                    <TableHead>
                        <TableRow>
                            <TableCell>Job</TableCell>
                            <TableCell>Status</TableCell>
                            <TableCell>Duration</TableCell>
                            <TableCell>Size</TableCell>
                            <TableCell>Started</TableCell>
                        </TableRow>
                    </TableHead>
                    <TableBody>
                        {filteredRuns.map((run) => (
                            <TableRow key={run.id} hover>
                                <TableCell>{run.job_id}</TableCell>
                                <TableCell>
                                    <Chip
                                        label={run.status}
                                        color={
                                            run.status === "success"
                                                ? "success"
                                                : run.status === "failure"
                                                ? "error"
                                                : "warning"
                                        }
                                        size="small"
                                    />
                                </TableCell>
                                <TableCell>{formatDuration(run.duration_ms)}</TableCell>
                                <TableCell>{formatBytes(run.bytes_transferred)}</TableCell>
                                <TableCell>{formatTimestamp(run.started_at)}</TableCell>
                            </TableRow>
                        ))}
                    </TableBody>
                </Table>
            </TableContainer>
        </>
    );
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/components/BackupRunsTable.tsx
git commit -m "feat: add backup runs table component"
```

---

## Task 14: Backup Alerts Table Component

**Files:**
- Create: `frontend/src/components/BackupAlertsTable.tsx`

- [ ] **Step 1: Write the component**

Create `frontend/src/components/BackupAlertsTable.tsx`:

```tsx
import { Button, Chip, Paper, Table, TableBody, TableCell, TableContainer, TableHead, TableRow } from "@mui/material";
import { BackupAlert } from "../types/backups";

interface Props {
    alerts: BackupAlert[];
    onAcknowledge: (alertId: string) => void;
}

function formatTimestamp(ts: number): string {
    return new Date(ts * 1000).toLocaleString();
}

export default function BackupAlertsTable({ alerts, onAcknowledge }: Props) {
    return (
        <TableContainer component={Paper}>
            <Table>
                <TableHead>
                    <TableRow>
                        <TableCell>Severity</TableCell>
                        <TableCell>Type</TableCell>
                        <TableCell>Message</TableCell>
                        <TableCell>Created</TableCell>
                        <TableCell>Actions</TableCell>
                    </TableRow>
                </TableHead>
                <TableBody>
                    {alerts.map((alert) => (
                        <TableRow key={alert.id} hover>
                            <TableCell>
                                <Chip
                                    label={alert.severity}
                                    color={alert.severity === "critical" ? "error" : "warning"}
                                    size="small"
                                />
                            </TableCell>
                            <TableCell>{alert.alert_type}</TableCell>
                            <TableCell>{alert.message}</TableCell>
                            <TableCell>{formatTimestamp(alert.created_at)}</TableCell>
                            <TableCell>
                                {!alert.acknowledged && (
                                    <Button
                                        size="small"
                                        variant="outlined"
                                        onClick={() => onAcknowledge(alert.id)}
                                    >
                                        Acknowledge
                                    </Button>
                                )}
                            </TableCell>
                        </TableRow>
                    ))}
                </TableBody>
            </Table>
        </TableContainer>
    );
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/components/BackupAlertsTable.tsx
git commit -m "feat: add backup alerts table component"
```

---

## Task 15: Backup Dashboard Widget

**Files:**
- Create: `frontend/src/components/BackupDashboardWidget.tsx`

- [ ] **Step 1: Write the component**

Create `frontend/src/components/BackupDashboardWidget.tsx`:

```tsx
import { Card, CardContent, Typography, Box, Chip } from "@mui/material";
import { useBackupDashboard } from "../hooks/useBackups";

export default function BackupDashboardWidget() {
    const { data, isLoading } = useBackupDashboard();
    
    if (isLoading || !data) {
        return (
            <Card>
                <CardContent>
                    <Typography variant="h6">Backups</Typography>
                    <Typography color="text.secondary">Loading...</Typography>
                </CardContent>
            </Card>
        );
    }
    
    return (
        <Card>
            <CardContent>
                <Typography variant="h6" gutterBottom>Backups</Typography>
                <Box sx={{ display: "flex", gap: 2, flexWrap: "wrap" }}>
                    <Box>
                        <Typography variant="h4">{data.total_jobs}</Typography>
                        <Typography variant="body2" color="text.secondary">Jobs</Typography>
                    </Box>
                    <Box>
                        <Typography variant="h4">{data.success_rate_24h}%</Typography>
                        <Typography variant="body2" color="text.secondary">24h Success</Typography>
                    </Box>
                    <Box>
                        <Typography variant="h4">
                            {data.active_alerts > 0 ? (
                                <Chip label={data.active_alerts} color="error" size="small" />
                            ) : (
                                0
                            )}
                        </Typography>
                        <Typography variant="body2" color="text.secondary">Alerts</Typography>
                    </Box>
                    {data.last_failed_at && (
                        <Box>
                            <Typography variant="body2" color="error">
                                Last failed: {new Date(data.last_failed_at * 1000).toLocaleString()}
                            </Typography>
                        </Box>
                    )}
                </Box>
            </CardContent>
        </Card>
    );
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/components/BackupDashboardWidget.tsx
git commit -m "feat: add backup dashboard widget"
```

---

## Task 16: Main Backups Page

**Files:**
- Create: `frontend/src/components/BackupsPage.tsx`

- [ ] **Step 1: Write the component**

Create `frontend/src/components/BackupsPage.tsx`:

```tsx
import { Box, Tab, Tabs, Typography } from "@mui/material";
import { useState } from "react";
import {
    useAcknowledgeAlert,
    useBackupAlerts,
    useBackupJobs,
    useBackupRuns,
} from "../hooks/useBackups";
import BackupAlertsTable from "./BackupAlertsTable";
import BackupJobsTable from "./BackupJobsTable";
import BackupRunsTable from "./BackupRunsTable";

export default function BackupsPage() {
    const [tab, setTab] = useState(0);
    const { data: jobsData, isLoading: jobsLoading } = useBackupJobs();
    const { data: runsData, isLoading: runsLoading } = useBackupRuns();
    const { data: alertsData, isLoading: alertsLoading } = useBackupAlerts(undefined, false);
    const acknowledgeMutation = useAcknowledgeAlert();
    
    // Build a map of latest runs per job
    const latestRuns = new Map();
    if (runsData) {
        for (const run of runsData) {
            const existing = latestRuns.get(run.job_id);
            if (!existing || run.started_at > existing.started_at) {
                latestRuns.set(run.job_id, run);
            }
        }
    }
    
    return (
        <Box sx={{ p: 3 }}>
            <Typography variant="h4" gutterBottom>Backups</Typography>
            
            <Tabs value={tab} onChange={(_, v) => setTab(v)} sx={{ mb: 2 }}>
                <Tab label="Jobs" />
                <Tab label="Runs" />
                <Tab label={`Alerts ${alertsData ? `(${alertsData.length})` : ""}`} />
            </Tabs>
            
            {tab === 0 && (
                jobsLoading ? (
                    <Typography>Loading jobs...</Typography>
                ) : (
                    <BackupJobsTable jobs={jobsData ?? []} latestRuns={latestRuns} />
                )
            )}
            
            {tab === 1 && (
                runsLoading ? (
                    <Typography>Loading runs...</Typography>
                ) : (
                    <BackupRunsTable runs={runsData ?? []} />
                )
            )}
            
            {tab === 2 && (
                alertsLoading ? (
                    <Typography>Loading alerts...</Typography>
                ) : (
                    <BackupAlertsTable
                        alerts={alertsData ?? []}
                        onAcknowledge={(id) => acknowledgeMutation.mutate(id)}
                    />
                )
            )}
        </Box>
    );
}
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/components/BackupsPage.tsx
git commit -m "feat: add main backups page with tabs"
```

---

## Task 17: Wire Up Routes and Navigation

**Files:**
- Modify: `frontend/src/App.tsx`

- [ ] **Step 1: Add route and nav**

In `frontend/src/App.tsx`:

```tsx
// Add import
import BackupsPage from "./components/BackupsPage";

// Add route inside <Routes>:
<Route path="/backups" element={<BackupsPage />} />

// Add tab inside <Tabs>:
<Tab value="/backups" label="Backups" component={NavLink} to="/backups" />
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/App.tsx
git commit -m "feat: add backups route and navigation"
```

---

## Task 18: Wire Up Dashboard Widget

**Files:**
- Modify: `frontend/src/components/Dashboard.tsx` (or equivalent)

- [ ] **Step 1: Add widget to dashboard**

Find the dashboard component and add the backup widget:

```tsx
// Add import
import BackupDashboardWidget from "./BackupDashboardWidget";

// Add to dashboard layout (where other widgets are):
<BackupDashboardWidget />
```

- [ ] **Step 2: Commit**

```bash
git add frontend/src/components/Dashboard.tsx
git commit -m "feat: add backup widget to dashboard"
```

---

## Task 19: Update REQUIREMENTS.md

**Files:**
- Modify: `docs/REQUIREMENTS.md`

- [ ] **Step 1: Document backup monitoring**

Add a new section to `docs/REQUIREMENTS.md`:

```markdown
## Backup Monitoring

### Overview
The system receives backup execution reports from an external backup tool via HTTP API, stores job and run history, and provides alerting on failures, missed schedules, and anomalies.

### API
- `POST /api/backups/report` — Submit backup run (Bearer token auth)
- `POST /api/backups/report/start` — Mark backup as in_progress
- `GET /api/backups/jobs` — List jobs
- `GET /api/backups/runs` — List runs
- `GET /api/backups/alerts` — List alerts
- `POST /api/backups/alerts/{id}/acknowledge` — Acknowledge alert
- `GET /api/dashboard/backups` — Dashboard summary

### Data Model
- **BackupJob**: id, name, source, target, schedule_interval_seconds, created_at
- **BackupRun**: id, job_id, started_at, ended_at, status, bytes_transferred, duration_ms, error_message, details_json
- **BackupAlert**: id, job_id, run_id, alert_type, severity, message, acknowledged, resolved_at

### Alert Types
- `failed_status` — Backup reported failure (critical)
- `missed_schedule` — No run within 1.5x expected interval (warning)
- `anomaly_size` — Size is 0 or <10% / >300% of 7-day median (warning)
- `anomaly_duration` — Duration >300% of 7-day median (warning)

### Authentication
- Backup tool uses auto-generated Bearer API key
- Frontend uses existing OIDC/JWT auth
```

- [ ] **Step 2: Commit**

```bash
git add docs/REQUIREMENTS.md
git commit -m "docs: add backup monitoring requirements"
```

---

## Task 20: Run Full Test Suite

- [ ] **Step 1: Run backend tests**

```bash
cd backend && pytest tests/test_backups.py -v
```

Expected: All tests pass

- [ ] **Step 2: Run frontend build**

```bash
cd frontend && npm run build
```

Expected: Build succeeds with no errors

- [ ] **Step 3: Run backend lint**

```bash
cd backend && ruff check src/media_library_viewer_api/
```

Expected: No lint errors

- [ ] **Step 4: Commit**

```bash
git commit -m "test: verify backup monitoring implementation"
```

---

## Spec Coverage Check

| Spec Section | Implementing Task |
|-------------|-------------------|
| Data Model (BackupJob, BackupRun, BackupAlert) | Task 1, 3 |
| API Endpoints (report, start, jobs, runs, alerts) | Task 7 |
| Dashboard Endpoint | Task 8 |
| Validation Rules | Task 7 (endpoint validation) |
| Alert Types (failed, missed, anomaly) | Task 4, 5 |
| Alert State Machine | Task 4, 7 |
| Frontend Navigation | Task 17 |
| Dashboard Widget | Task 15, 18 |
| Backups Page (Jobs, Runs, Alerts tabs) | Task 12-16 |
| API Key Auth | Task 6 |
| Background Poller | Task 5 |
| Charts | Out of scope for MVP (future enhancement) |

## Placeholder Scan

- No "TBD", "TODO", "implement later" found
- No vague "add error handling" steps
- All code blocks contain actual code
- No "similar to Task N" references

## Type Consistency Check

- `BackupJob.schedule_interval_seconds` — used consistently across backend and frontend
- `BackupRun.status` — enum values match (`success`, `failure`, `in_progress`)
- `BackupAlert.alert_type` — values match spec
- `BackupAlert.severity` — values match spec (`warning`, `critical`)
- All Pydantic models match TypeScript interfaces

---

## Execution Handoff

**Plan complete and saved to `docs/superpowers/plans/2026-05-11-backup-monitoring.md`. Two execution options:**

**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration

**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints

**Which approach?**
