backup-tool/docs/release/m12-evidence.md

M12 evidence

• Migration revision: 0008_notification_outbox
• Event schema version: 1
• Catalog: 22 live stable IDs only. Deferred/unimplemented operation types are not public notification contracts.
• Delivery guarantee: durable at-least-once, stable event ID, leased worker retries; receiver deduplication is required.

Green focused evidence

PYTHONPATH=.:backend/src .venv/bin/python -m pytest \
  tests/contract/test_notification_contract.py \
  tests/integration/test_migrations.py \
  tests/integration/test_all_operational_events_deliver.py \
  tests/integration/test_notifications.py \
  tests/fault/test_notification_retries.py \
  tests/security/test_webhook_ssrf.py -q
28 passed (live-catalog contract, fair dispatch regression; no deferred event IDs).

# Scheduler-role service-path regression
PYTHONPATH=.:backend/src .venv/bin/python -m pytest \
  tests/integration/test_scheduler_live_sync.py -q
1 passed (scheduler role service path)

The suite uses temporary SQLite/repository roots, a fake SMTP implementation, fake resolver inputs, and dispatcher monkeypatches; it performs no real webhook DNS, HTTP, or SMTP delivery. It verifies receiver-visible versioned webhook headers/signatures, STARTTLS-before-AUTH SMTP behavior, transient/permanent SMTP classification, persisted SMTP attempt limits, lease-abandoned attempt closure, selected-only test sends, manual-retry idempotency, and the absence of a plaintext-secret idempotency verifier. Behavioral producer tests cover every live catalog family: execution, schedule, backup/verification, restore, and retention. They also prove fair dispatch under an execution backlog, scheduler-role delivery, and worker-owned retention/GC maintenance. The CLI scheduler role now runs SchedulerService; worker maintenance runs durable retention/GC on startup and at bounded intervals.

Quality evidence

• make test-fast: 89 passed.
• make test-integration: 61 passed in 27.00s (the execution wrapper nevertheless returned exit 124 at its fixed 30s wall limit).
• make test-fault: focused fair-dispatch regression passed.
• make test-security: 28 passed.
• make lint and make typecheck: passed.
• make frontend-build: passed.
• git diff --check: passed.
• staged-file check: no staged files.

Final verification: make check passed — 90 unit/contract, 61 integration, 14 fault, and 28 security tests; Ruff, mypy, TypeScript, and the frontend build passed. git diff --check and the staged-file check also passed.

Rollback

Disable or archive subscriptions and stop worker dispatch. Do not delete notification events, deliveries, or attempts: they remain audit history. A fresh host recovery intentionally starts with no notification settings or credentials and must be reconfigured.