backup-tool/docs/superpowers/plans/2026-07-27-backup-tool-v2-implementation.md

Backup Tool v2 Implementation Plan

Status: Ready for execution after branch kickoff
Date: 2026-07-27
Design authority: ../specs/2026-07-27-backup-tool-reimplementation-design.md
Replaces: 2026-05-11-backup-tool-implementation.md for all v2 work

1. Reason for Existence

This plan converts the decision-closed design into dependency-ordered, test-first, independently reviewable milestones. It prevents framework-first rebuilding: repository format, publication safety, restore, and crash recovery become executable contracts before scheduling, optimization, UI breadth, or v2.1 adapters.

A milestone is complete only when its named behavior tests pass, its failure path is proved, its design decisions and acceptance invariants are traceable, and the working tree contains no unrelated changes.

2. Scope and Guardrails

• V2 is a clean rewrite. It MUST NOT contain /api/v1, legacy DB readers, importers, converters, timestamp-directory parsers, or v1 payload access.
• Preserve v1 only as a Git tag/branch for historical reference before deleting executable v1 code. No v1 file or test is adapted into a compatibility layer.
• V2.0 includes local and SSH/SFTP sources, local repositories, restore, optional repository encryption, recovery bundles, webhooks, and email.
• PostgreSQL, MySQL, and generated TAR download are v2.1-only capabilities and MUST remain absent from v2.0 OpenAPI, UI, image, and capability metadata.
• Certified topology is one node, SQLite/WAL, one scheduler process, one worker process, and one or more web processes. Web processes MUST perform no backup, restore, scheduler, or repository I/O.
• Every task follows red → green → refactor. Preserve failing-then-passing command output in PR/CI evidence.
• Do not start a dependent milestone until its dependency gate is green. Do not split an atomic set from §6 across releases.

3. Target Project Layout

backend/
  pyproject.toml
  alembic/versions/
  src/backup_tool/
    api/{app,deps,errors,pagination,etag,idempotency,sse,routers/,schemas/}
    adapters/{base,local,sftp,postgres,mysql}.py
    db/{engine,models,repositories}.py
    domain/{manifest,paths,exclusions,transitions,cron,retention}.py
    execution/{enqueue,claims,leases,runner,retry,cancellation,progress}.py
    notifications/{events,dispatcher,webhook,email,retry}.py
    observability/{logging,metrics,health}.py
    repository/{layout,blob_store,manifest_store,staging,publisher,verifier,restore,reconcile,gc,encryption}.py
    security/{auth,secrets,redaction,repository_crypto,recovery_bundle,ssrf}.py
    services/{sources,jobs,repositories,backups,restores}.py
    scheduler/service.py
    {cli,config,ids,clock}.py
frontend/src/
  api/generated/
  app/
  pages/
  components/
contracts/repository/v1/
openapi/v2.json
tests/{unit,contract,integration,fault,security,e2e,performance,fixtures}/
tools/{export_openapi,forbidden_v1_scan,leakage_scan,check_traceability,run_scale_certification}.py
docs/{runbooks,security,release}/

Module rules: domain code imports no FastAPI, SQLAlchemy, filesystem, Paramiko, SMTP, or HTTP client; adapters and repository code implement ports; services own use cases; routers translate HTTP only; ORM and API schemas never act as domain models.

4. Dependency Spine

M0 contracts
 └─ M1 runtime/DB ─ M2 auth/API
        └─ M3 repository ─ M4 sources/jobs ─ M5 execution
             └─ M6 full backup/verify/restore
                  ├─ M7 incremental/exclusions
                  ├─ M8 scheduling
                  ├─ M9 retention/GC/reconcile
                  ├─ M10 SSH/SFTP
                  └─ M11 encryption/recovery
                       └─ M12 notifications ─ M13 UI/API ─ M14 operations ─ M15 v2.0
M15 ─ M16 PostgreSQL ─ M17 MySQL ─ M18 TAR/download ─ v2.1

M7-M11 MAY be developed on separate branches after M6, but merge in dependency order and never use concurrent writers in one worktree.

5. Milestones

M0 — Freeze protocol and test foundation

Depends on: approved design.
Deliver: lock Python/Node/tool versions; create package/test layout and Makefile; commit canonical repository/manifest JSON schemas, golden valid/invalid fixtures, normalized-path vectors, state-transition table, error-code catalog, capability schema, and deterministic fault-point names. Tag/archive v1, then remove backend/app, backend/backup, old tests, and handwritten frontend/src/api/client.ts.
Verify: python tools/forbidden_v1_scan.py . && python -m pytest tests/contract/test_repository_format.py tests/contract/test_no_v1.py -q
Commit: chore(v2): establish protocol and test foundation

M1 — Runtime, UUIDv7, SQLite/WAL, and Alembic baseline

Depends on: M0.
Deliver: typed config; canonical source/repository/restore allowlists; key-file owner/mode checks; UUIDv7 IDs; UTC clock port; role CLIs; SQLite foreign keys/WAL/busy timeout; complete explicit schema, constraints, indexes, and first hand-reviewed migration. Startup checks migration state and never calls create_all.
Verify: cd backend && python -m alembic upgrade head && python -m alembic downgrade base && python -m alembic upgrade head && python -m pytest ../tests/unit/test_config.py ../tests/integration/test_migrations.py -q
Commit: feat(v2): add validated runtime and metadata store

M2 — Admin setup, authentication, secrets, audit, and API conventions

Depends on: M1.
Deliver: guarded one-admin setup; Argon2id; secure session/CSRF; hashed scoped API tokens; envelope-encrypted write-only secrets; request IDs; append-only audit; RFC 9457 problems; stable cursor pagination/sort; ETags; request-digest idempotency; truthful /livez and /readyz.
Verify: python -m pytest tests/unit/test_redaction.py tests/contract/test_api_conventions.py tests/integration/test_auth.py tests/security/test_auth.py -q && python tools/leakage_scan.py
Commit: feat(v2): secure identity secrets and API conventions

M3 — Initialize and inspect repositories

Depends on: M2.
Deliver: allowlisted local repository init; repository.json; immutable compression/encryption choice; canonical JSON; plaintext SHA-256 blob identity; capacity probe; minimum free bytes/percent; safe partial-init cleanup; path/symlink containment. Encryption interfaces exist, but encrypted creation remains capability-disabled until M11.
Verify: python -m pytest tests/contract/test_repository_format.py tests/integration/test_repository_init.py tests/security/test_repository_paths.py -q
Commit: feat(v2): initialize immutable content repositories

M4 — Typed local sources and repository-targeted jobs

Depends on: M3.
Deliver: adapter protocol; normalized POSIX entries; bounded local streaming; symlink/cross-filesystem/special-file policy; source probes; typed source/job CRUD; immutable source kind; archive semantics; repository IDs instead of destination strings; enabled, mode, exclusions, retention, and allow_empty.
Verify: python -m pytest tests/unit/test_paths.py tests/unit/test_exclusions.py tests/integration/test_local_source_api.py tests/security/test_source_containment.py -q
Commit: feat(v2): add local sources and repository jobs

M5 — Durable execution lifecycle

Depends on: M4.
Deliver: monotonic states; 202 execution resource; 409 active execution; DB-enforced one nonterminal execution/job; disabled-job check inside enqueue; leases, heartbeats, retries using one execution ID, attempts, cancellation, reason codes, redacted progress/logs, SSE with polling fallback, graceful shutdown, stale-worker fencing, startup reconciliation.
Verify: python -m pytest tests/unit/test_execution_transitions.py tests/integration/test_queue_leases.py tests/fault/test_worker_loss.py -q
Commit: feat(v2): add durable leased execution lifecycle

M6 — Atomic full backup, verification, and restore

Depends on: M5.
Deliver: unique staging; capacity preflight; adapter-only reads; chunked hashing; immutable blob install; canonical complete manifest; fsync/atomic rename where supported; object verification; metadata visibility only after durable publish; reconciliation marker; metadata/full verification; durable dry-run/selected restore; fail|skip|replace; destination containment; result manifest; integrity states.
Verify: python -m pytest tests/integration/test_full_backup_restore.py tests/fault/test_publication_crashes.py tests/security/test_restore_paths.py -q
Commit: feat(v2): publish verified snapshots and durable restores

M7 — Exclusions, independent incrementals, and empty-source safety

Depends on: M6.
Deliver: versioned gitignore semantics shared by every adapter; compatible-baseline selection; requested/effective mode; baseline_missing; blob reuse; declared metadata-trust rules; independently restorable complete manifests; empty-source opt-in and vanished-source confirmation.
Verify: python -m pytest tests/unit/test_exclusions.py tests/integration/test_incremental.py tests/integration/test_empty_source.py -q
Commit: feat(v2): add safe deduplicated snapshots

M8 — Deterministic durable scheduling

Depends on: M6.
Deliver: one schedule/job; semantic five-field cron; IANA timezone; next run/last result; (schedule_id, nominal_utc) uniqueness; immediate create/update/delete synchronization; 15-minute grace; coalesce/no-overlap; exact DST gap/fold behavior; disabled job/schedule rejection; same enqueue service as manual execution.
Verify: python -m pytest tests/unit/test_cron_dst.py tests/integration/test_scheduler_live_sync.py tests/fault/test_schedule_delivery.py -q
Commit: feat(v2): schedule idempotent timezone-aware executions

M9 — Retention, tombstones, reconciliation, and safe GC

Depends on: M6 and M7.
Deliver: union policy for keep-last/days/daily/weekly/monthly/pins; newest protection; preview/apply; archive without payload deletion; tombstone/audit; grace plus second reference scan; deletion_failed; confirmed purge/report; bidirectional reconciliation; unknown-object quarantine; shared-blob safety.
Verify: python -m pytest tests/unit/test_retention.py tests/integration/test_gc.py tests/fault/test_deletion_failure.py tests/integration/test_reconcile.py -q
Commit: feat(v2): add retention reconciliation and safe GC

M10 — Strict SSH/SFTP parity

Depends on: M6 and M7.
Deliver: typed remote config; write-only auth; pinned known-host key; fail-closed probe and execution; bounded SFTP enumeration/streaming; timeouts/backpressure; common paths/exclusions; source-consistency metadata; transient classification; no local traversal fallback.
Verify: docker compose -f tests/compose.integration.yaml up -d sshd && python -m pytest tests/integration/test_sftp_backup_restore.py tests/security/test_sftp_host_keys.py -q
Commit: feat(v2): add pinned-host SFTP backups

M11 — Optional repository encryption and offline recovery

Depends on: M6.
Deliver: AEAD stored objects; associated metadata; key IDs and rotation; immutable policy; explicit confidentiality/equality-leakage threat model; separately passphrase-protected recovery bundle; non-disclosing validation; fresh-host metadata/key recovery and encrypted restore. Enable encrypted repository creation only after the host-loss test passes.
Verify: python -m pytest tests/integration/test_encrypted_repository.py tests/integration/test_recovery_bundle.py tests/security/test_crypto_leakage.py -q
Commit: feat(v2): encrypt repositories and recover keys offline

M12 — Signed webhooks and auditable email

Depends on: M2, M5, and M7-M11.
Deliver: complete operational event catalog covering execution, schedule, retention/GC, reconciliation, SSH, encryption/recovery, restore, verification, capacity, and security outcomes; typed filters/subscriptions; stable event IDs; durable outbox/deliveries/attempts; webhook signature/version and rotation; at-least-once bounded retry; SMTP transient retry; manual test/retry; history; rate limits; write-only credentials; redirect/DNS/IP/private-network/rebinding SSRF policy.
Verify: python -m pytest tests/contract/test_notification_contract.py tests/integration/test_all_operational_events_deliver.py tests/integration/test_notifications.py tests/fault/test_notification_retries.py tests/security/test_webhook_ssrf.py -q
Commit: feat(v2): add webhook and email delivery

M13 — OpenAPI-generated client and operator UI

Depends on: M7-M12.
Deliver: deterministic committed OpenAPI; generated TypeScript client only; drift CI; Setup/Login, Dashboard, Sources, Repositories, Jobs/Schedule, Executions/SSE, Backups/Verify/Restore/Delete Preview, Security/Recovery, Notifications/History, and Audit. Implement loading, empty, validation, partial failure, reconnect, session expiry, responsive, keyboard, focus, contrast, reduced-motion, and screen-reader states. Hide v2.1 controls completely.
Verify: python tools/export_openapi.py --check openapi/v2.json && npm --prefix frontend run api:generate && git diff --exit-code -- openapi/v2.json frontend/src/api/generated && npm --prefix frontend test -- --run && npm --prefix frontend run build && npx --prefix frontend playwright test
Commit: feat(v2): ship generated-client operator workflows

M14 — Production roles, observability, recovery, and packaging

Depends on: M13.
Deliver: one pinned non-root OCI image with web|scheduler|worker|migrate|admin; same-origin proxy; one scheduler and worker; no reload or embedded roles; role-aware readiness; structured logs; required metrics/alerts; metadata/repository/key/upgrade/disaster runbooks; restart persistence; safe shutdown; SBOM/provenance; v2.0 image contains no DB dump clients.
Verify: docker compose build --pull && docker compose run --rm migrate upgrade && docker compose up -d && python -m pytest tests/e2e/test_compose_v2.py tests/security/test_container.py -q && docker compose down
Commit: build(v2): package isolated roles and recovery operations

M15 — Certify and release v2.0

Depends on: M0-M14.
Deliver: reference-host definition; workloads for 100 jobs, 1M entries or 10 TiB logical/backup, and 100k cataloged backups; throughput/CPU/memory/DB/lag/pagination/restore/verify evidence; bounded-memory and accepted-regression gates; complete fault/security/leakage/E2E evidence; truthful capabilities/docs; removal of all v2.1 controls.
Verify: python tools/run_scale_certification.py --jobs 100 --entries 1000000 --backups 100000 && python tools/assert_capabilities.py --release v2.0 --include local,ssh,restore,webhook,email --exclude postgresql,mysql,tar-download
Commit: release: certify backup-tool v2.0.0

M16-M18 — V2.1 vertical slices

• M16 PostgreSQL (depends on M15): secret-safe bounded pg_dump, TLS, tool/server matrix, full-only manifest, cancellation/error classification, typed source CRUD/probe, capability metadata, OpenAPI/generated client/UI form, and v2.1 OCI dump-client packaging land atomically → verify: python -m pytest tests/contract/test_postgres_source_api.py tests/integration/test_postgres_adapter.py tests/security/test_dump_secrets.py tests/security/test_v21_container.py -q && make check-openapi && npm --prefix frontend run build && python tools/assert_capabilities.py --release v2.1 --include postgresql → feat(v2.1): add PostgreSQL logical snapshots.
• M17 MySQL (depends on M16): secret-safe bounded mysqldump, TLS/locking consistency, compatibility matrix, full-only manifest, typed source CRUD/probe, capability metadata, OpenAPI/generated client/UI form, and OCI client verification land atomically → verify: python -m pytest tests/contract/test_mysql_source_api.py tests/integration/test_mysql_adapter.py tests/security/test_dump_secrets.py tests/security/test_v21_container.py -q && make check-openapi && npm --prefix frontend run build && python tools/assert_capabilities.py --release v2.1 --include postgresql,mysql → feat(v2.1): add MySQL logical snapshots.
• M18 TAR/download and v2.1 release (depends on M17): verify every blob/tag before streaming manifest-derived safe entries; traversal/link/device/disconnect tests; generated API/client/UI capability; then rerun every v2.0 gate, both DB matrices, OpenAPI drift, Compose/E2E, leakage, and reference-host regression → verify: make check && make test-integration && make test-fault && make test-security && make test-e2e && make check-openapi && make check-traceability && make check-v1-absent && python -m pytest tests/integration/test_postgres_adapter.py tests/integration/test_mysql_adapter.py tests/security/test_dump_secrets.py tests/integration/test_tar_download.py tests/security/test_tar_paths.py -q && python tools/run_scale_certification.py --regression-from docs/release/v2.0-evidence.md && python tools/assert_capabilities.py --release v2.1 --include local,ssh,postgresql,mysql,restore,tar-download,webhook,email && npx --prefix frontend playwright test → release: certify backup-tool v2.1.0.

6. Atomic Sets and Stop Rules

The following land as indivisible reviewed sets: publication + reconciliation marker + crash matrix; execution state + uniqueness + leases + cancellation + recovery; auth + secrets + redaction + audit; restore record + containment + verification + atomic placement; retention + tombstone + reference scans + quarantine; scheduler parser + uniqueness + DST/misfire + live sync; encryption + recovery bundle + host-loss restore; notification outbox + signing + SSRF + SMTP + retry/history; OpenAPI + generated client + drift check + corresponding UI.

Stop and request an ADR/design amendment if implementation requires changing manifest canonicalization, blob identity, external IDs, process topology, delivery guarantee, schedule semantics, encryption confidentiality, retention precedence, v2.0/v2.1 capability split, or any D1-D21 decision. Stop a milestone when a failure test is nondeterministic, a destructive test can reach operator data, a zero-test selection can pass, or a secret/path canary appears in any output sink.

7. Traceability

Decisions	Owner and proof	Decisions	Owner and proof
D1	M0-M2: UUID/API/no-v1 contract	D2	M2: concurrent setup test
D3	M1, M5, M14: role/process test	D4	M1, M14: pragmas/topology test
D5	M2, M5: idempotency/lease tests	D6	M3-M4: repository-only job/path tests
D7	M0, M3, M6: manifest/independent restore	D8	M3, M11: immutable policy/AEAD tests
D9	M0, M4, M7, M10: shared matcher corpus	D10	M8: cron/timezone/DST tests
D11	M4-M5, M8: disabled enqueue tests	D12	M4, M9: archive/restorability tests
D13	M13: generated-client zero-diff test	D14	M1: Alembic/no-create_all test
D15	M10, M15-M17: capability gates	D16	M15: certified reference-host report
D17	M11: fresh-host recovery drill	D18	M0, M15: forbidden-v1 scan/runtime test
D19	M13, M15, M18: capability/route gates	D20	M12-M13, M15: channel/event E2E
D21	M12: stable-ID retry/history tests	—	—

Invariants	Owner and proof	Invariants	Owner and proof
I1	M6: publication crash matrix	I2	M8: replayed occurrence uniqueness
I3	M5: concurrent active-job starts	I4	M5-M6: transition/restart matrix
I5	M10, M16-M17: adapter-spy tests	I6	M6-M7: parent-free restore corpus
I7	M9: retain/GC/restore property tests	I8	M4, M9: archive inventory test
I9	M9: deletion failure/retry test	I10	M2, M11-M12, M14: canary scan
I11	M10: host-key fail-before-read test	I12	M7, M10, M16-M17: matcher parity
I13	M5, M8: all-trigger disable tests	I14	M8: live-sync/misfire/DST tests
I15	M0, M15: protected-v1 no-access test	I16	M13, M18: OpenAPI/client zero diff
I17	M6: full restore matrix	I18	M6, M11, M18: corruption matrix
I19	M7: empty/vanished-source test	I20	M3-M4, M6, M10, M18: containment
I21	M11: recovery-bundle host-loss drill	I22	M12: webhook/email retry history

tools/check_traceability.py MUST require every D1-D21 and I1-I22 row to name a milestone, test path, runnable command, and immutable evidence artifact.

8. Whole-Project Verification

make test-fast                 # unit + contract; nonempty marker assertion
make test-integration          # temp DB/repository roots only
make test-fault                # deterministic failpoints
make test-security             # canaries, containment, auth, SSRF
make test-e2e                  # production-like Compose
make check-openapi             # export, generate, zero diff, type/build
make check-traceability        # D1-D21 and I1-I22 evidence links
make check-v1-absent           # static + runtime protected-v1 fixture
make check                     # format, lint, type, all non-scale tests
git diff --check
git status --short

Fault/destructive tests MUST allocate hermetic temporary source, repository, restore, DB, key, SMTP, and webhook targets. Test teardown MUST reject paths outside its temp root. CI MUST assert each selected marker collected at least one test.

9. Release Gates

V2.0: M0-M15 green; every D1-D21 item applicable to v2.0 and I1-I22 has immutable evidence; no secret canary; fault matrix green; OpenAPI/client diff clean; local and SSH restore corpus green; encrypted fresh-host recovery green; notifications retry across process restart; Compose roles/non-root/persistence green; scale report approved; capabilities exclude PostgreSQL, MySQL, and TAR download.

V2.1: v2.0 gates remain green; PostgreSQL/MySQL compatibility matrices pass; dump credentials never enter argv/logs; TAR corruption/path/disconnect tests pass; generated API/client/UI exposes only installed capabilities; performance regression remains within the accepted threshold.

10. Execution Discipline

Use one feature branch/worktree per milestone. Keep commits small within the milestone but merge only when its atomic gate is closed. Conventional commit title matches the milestone; PR body links RED/GREEN evidence, design IDs, invariants, commands with exit codes, changed files, residual risks, and rollback. Run a fresh-context security/reliability review after M2, M6, M9, M11, M12, M14, and before each release.

Next action: start M0 by creating the contract fixtures and forbidden-v1 test before scaffolding application modules.

11. Plan Verification

python - <<'PY'
from pathlib import Path
p = Path('docs/superpowers/plans/2026-07-27-backup-tool-v2-implementation.md')
s = p.read_text()
for marker in ['M0', 'M15', 'M18', 'D1-D21', 'I1-I22', 'verify:', 'Release Gates']:
    assert marker in s, marker
assert s.count('**Verify:**') >= 16
print('v2 implementation plan: OK')
PY