Files
manage/openspec/changes/service-credential-tester/apply-progress.md
T
Developer 98bf496a98 spec(service-credential-tester): verify + strengthen no-secret-logs test + reconcile
Strengthen test_secrets_not_logged (N-2): now sends real-looking secrets
through prometheus + qbittorrent test_callables (mocked at network boundary),
asserts no fragments leak into caplog, verified non-vacuous. Write
apply-progress.md, tick all 29 tasks, add verify-report.md (21/21 PASS).
Gates green: 362+ pytest, ruff clean, npm build+lint 0 errors, 158 vitest.
2026-07-09 23:15:47 +00:00

4.7 KiB

Apply Progress: Service Credential Tester

Change: service-credential-tester Phase: apply-progress Date: 2026-07-09 Status: complete — all 29 tasks done, all gates green, verified (see verify-report.md)

Slices delivered

Two slices, each its own commit, each leaving pytest / npm run build / npm run lint / ruff green.

Slice 1 — Backend test endpoint + per-type routines (commit 3391fbc)

  • integrations/base.py — added frozen TestResult dataclass (ok, detail, evidence) + optional test_callable field on ServiceDefinition (placed last for dataclass ordering). Signature: (store, config, secrets) -> TestResult (CT-101).
  • Shared error-translation helper translate_connection_error — extracts the test_machine_ssh patterns (ConnectionError/Timeout/SSL/HTTP 401-403/5xx → friendly strings) (CT-112).
  • Per-type test_connection routines alongside each DEFINITION:
    • qbittorrentQbittorrentClient login + maindata(); "Fails." → "Authentication failed (qBittorrent rejected credentials)" — resolves the #3 log-only pain at the API layer (CT-104).
    • prometheusPOST {grafana_url}/api/ds/query with grafana_api_key + datasource_uid, expr "up"; evidence "Gateway reachable" (CT-105; gateway path per grafana-metric-gateway).
    • alertmanagerGET /api/v2/alerts (+ optional bearer); evidence cluster version (CT-106).
    • jellyfinJellyfinClient.users(); evidence " users" (CT-107).
    • authentik — directory endpoint GET; evidence slug/"connected" (CT-108).
    • ssh_tasks — reuses build_ssh_client(store, service) + connect(); banner/auth translation; no duplication of test_machine_ssh internals (CT-109).
    • nextcloudGET /status.php (unauth); evidence version (CT-110).
    • backupstest_callable=None{ok: True, detail: "No test needed"} (CT-111).
  • routers/services.pyPOST /api/services/test: validation-first (422 on malformed config, no network call), dispatch, zero persistence (no upsert/update_setting), INFO log only type+ok (sanitized; no secrets) (CT-102, CT-103, CT-113).
  • Tests: test_credential_tester.py (per-routine success + failure, dispatch, validation-before-test, no-persistence assertion test_no_persistence_after_test, qBit "Fails." → auth message).

Slice 2 — Frontend Test button + gating (commit 9972514, amended)

  • types/index.tsTestResult interface (CT-113).
  • api/services.ts + hooks/useServices.tstestServiceInstance + useTestServiceInstance mutation (no cache invalidation; test is side-effect-free).
  • NEW components/ServiceTestPanel.tsxpresentational shared component (cleaner than the design's stateful version — deviation N-6). Props: { input, onResult, disabled }. Renders Test button + Testing… state + result pill (✓ green evidence / ✗ red detail) + Save-anyway checkbox (only on failure). Parent owns testResult + saveAnyway state; store-previous pattern clears on input change (CT-114, CT-115, CT-116, CT-117).
  • Wired into BOTH surfaces: CreateServiceDialog (ServicesPage.tsx) AND ServiceConfigEditor (Settings.tsx — the correct edit surface per design source-finding, not ServicePage.tsx). Create/Save confirm gated on testPassed || saveAnyway (CT-118).
  • Tests: 7 ServiceTestPanel.test.tsx cases (button states, success/failure pills, checkbox toggle).

Deviations from tasks.md / design

  • N-6 (intentional improvement): ServiceTestPanel is presentational; the design's stateful version was simplified. Parent owns result + saveAnyway state. Cleaner; works identically in both surfaces.
  • Edit surface correction: spec CT-115 said ServicePage.tsx; the actual edit dialog is Settings.tsx::ServiceConfigEditor (design source-finding). Tasks targeted the right file.

Final gate results

Gate Result
backend && PYTHONPATH=src python3 -m pytest -q 362 passed (+31 new), 2 warnings (pre-existing pythonjsonlogger)
backend && PYTHONPATH=src python3 -m ruff check src tests All checks passed
frontend && npm run build exit 0 (pre-existing chunk-size warning)
frontend && npm run lint 0 errors, 1 pre-existing warning (WidgetConfigDialog.tsx, untouched)
frontend && npx vitest run 158 passed (+7 ServiceTestPanel)

Verification

See verify-report.md — adversarial fresh-context review: 21/21 PASS, all gates green. No blocking code findings. Archive blocker is doc-only (this file + ticked tasks clear it). Non-blocking notes: N-2 (no-secret-logs test sends empty secrets — weak coverage, not a defect), N-4 (edit flow requires re-typing secrets — inherent to no-persistence), N-5 (gating proven by source, not page-level test).