spec(service-credential-tester): sync into new canonical domain
New canonical openspec/specs/service-credential-testing/spec.md (21 reqs CT-101..121). Change-side delta + sync-report. web-ui/prometheus-charting/ service-storage canonicals untouched.
This commit is contained in:
Developer
@@ -0,0 +1,201 @@
|
||||
# Service Credential Testing — Delta (`service-credential-tester`)
|
||||
|
||||
> Change: `service-credential-tester` · Domain: `service-credential-testing` · Phase: **spec** (reconciled during `sdd-sync`).
|
||||
> Distilled verbatim from the verified flat `spec.md` (21 requirements, CT-101 … CT-121) of
|
||||
> change `service-credential-tester`, cross-referenced against `design.md` and
|
||||
> `verify-report.md`. Captures the **durable, post-change end-state contracts** for the per-
|
||||
> service-type credential tester: the `POST /api/services/test` endpoint, the closed
|
||||
> `test_callable` dispatch registry, per-type test routines, the no-persistence + no-secret-logging
|
||||
> guarantees, and the frontend test UI (Test button + result pill + gated confirm with a "Save
|
||||
> anyway" override).
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
> The canonical `openspec/specs/service-credential-testing/spec.md` did not exist before this
|
||||
> change. All requirements below are therefore **ADDED** to a new `service-credential-testing`
|
||||
> domain; `sdd-sync` copies them into the canonical spec (native helper rule: when the canonical
|
||||
> spec does not exist, the change spec becomes the new canonical spec).
|
||||
>
|
||||
> Requirement IDs (CT-101 … CT-121) and body text are preserved **exactly** from the verified flat
|
||||
> `spec.md`. Requirements are grouped logically and listed in the following group order:
|
||||
>
|
||||
> + **Endpoint shape, validation, dispatch** — CT-101 … CT-103
|
||||
> + **Per-type test routines** — CT-104 … CT-111
|
||||
> + **No-persistence + security** — CT-112 … CT-113
|
||||
> + **Frontend test UI** — CT-114 … CT-118
|
||||
> + **Tests + gate greenness** — CT-119 … CT-121
|
||||
|
||||
### Requirement: CT-101 — Test endpoint accepts unsaved service input and returns a structured result
|
||||
|
||||
The backend MUST expose `POST /api/services/test` accepting a `ServiceInstanceInput` body (`service_type`, `name`, `config`, `secrets`, `enabled`) — NOT a service id, because the tester operates on unsaved form values. On success the endpoint MUST return HTTP 200 with a body of shape `{ "ok": bool, "detail": str, "evidence": str | null }`. The endpoint MUST be auth-gated identically to every other `/api/services/*` endpoint (JWT/API-key).
|
||||
|
||||
#### Scenario: successful connection returns ok with evidence
|
||||
|
||||
+ GIVEN a valid `ServiceInstanceInput` for a reachable service
|
||||
+ WHEN `POST /api/services/test` is called with that body
|
||||
+ THEN the response is HTTP 200 with `{ "ok": true, "detail": <human summary>, "evidence": <proof string> }`
|
||||
|
||||
#### Scenario: failed connection returns ok=false with actionable detail
|
||||
|
||||
+ GIVEN a valid input whose target is unreachable or rejects auth
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is HTTP 200 with `{ "ok": false, "detail": <specific failure message>, "evidence": null }`
|
||||
|
||||
### Requirement: CT-102 — Validation runs before the test and rejects malformed config with 422
|
||||
|
||||
The endpoint MUST reuse the existing `_validate_input` helper (from `routers/services.py`) to validate `service_type`, `config` (via the definition's `config_model`), and secret-key names BEFORE any network probe. Malformed input MUST raise HTTP 422 with the same detail format as `POST /api/services/instances`, so the existing validation-error surfacing (the `#1` fix in `ServicesPage.tsx`) covers the test endpoint too.
|
||||
|
||||
#### Scenario: schema-less URL rejected before any network call
|
||||
|
||||
+ GIVEN a `qbittorrent` input whose `base_url` lacks the `http://` schema
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is HTTP 422 with a detail naming the schema requirement
|
||||
+ AND no network call is made to qBittorrent
|
||||
|
||||
#### Scenario: unknown service type rejected
|
||||
|
||||
+ GIVEN an input with `service_type = "nonexistent"`
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is HTTP 422 with "Unknown service type"
|
||||
|
||||
### Requirement: CT-103 — Closed per-type dispatch mirrors SERVICE_DEFINITIONS
|
||||
|
||||
The endpoint MUST dispatch to a per-type test routine via the service definition (a new optional `test_callable` field on `ServiceDefinition`, defaulting to `None`). The dispatch is closed and compile-time — no runtime plugin loading, no arbitrary callable execution. A service type whose definition has `test_callable = None` (e.g. `backups`, which has no remote connection) MUST return `{ "ok": true, "detail": "No connection test for this service type", "evidence": null }` without any network call.
|
||||
|
||||
#### Scenario: backups type returns ok without a network probe
|
||||
|
||||
+ GIVEN a `backups` service input
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is `{ "ok": true, "detail": "No connection test for this service type", "evidence": null }`
|
||||
+ AND no network request is issued
|
||||
|
||||
#### Scenario: unknown widget kind is not involved
|
||||
|
||||
+ GIVEN a `prometheus` input
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the dispatch keys off `service_type` only (not `widget_kind`); the test is per-service-instance, not per-widget
|
||||
|
||||
### Requirement: CT-104 — qBittorrent test logs in and probes maindata; surfaces auth failures specifically
|
||||
|
||||
The `qbittorrent` test routine MUST construct a `QbittorrentClient` from the config + decrypted secrets, call login (`POST /api/v2/auth/login`), then probe `GET /api/v2/sync/maindata` (or `/app/version`). On success it MUST return `{ "ok": true, "evidence": <qBittorrent version or "connected"> }`. When qBittorrent's login returns anything other than `"Ok."` (e.g. the literal `"Fails."`), the routine MUST return `{ "ok": false, "detail": "Authentication failed — qBittorrent rejected the credentials." }` — surfacing the specific auth failure at the UI layer, not just in backend logs.
|
||||
|
||||
#### Scenario: correct credentials succeed with version evidence
|
||||
|
||||
+ GIVEN a qBittorrent instance reachable at the configured URL with valid username/password
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": true, "detail": "Connected to qBittorrent.", "evidence": <version> }`
|
||||
|
||||
#### Scenario: wrong password surfaces a specific auth message
|
||||
|
||||
+ GIVEN a qBittorrent instance that returns `"Fails."` on login
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": false, "detail": "Authentication failed — qBittorrent rejected the credentials.", "evidence": null }`
|
||||
|
||||
### Requirement: CT-105 — Prometheus test queries the Grafana gateway with expr "up"
|
||||
|
||||
The `prometheus` test routine MUST issue `POST {grafana_url}/api/ds/query` carrying `Authorization: Bearer {grafana_api_key}` and a `queries[0]` entry keyed by the configured `datasource_uid` with `expr: "up"`. There MUST be no direct Prometheus HTTP call (the `prometheus` service sources through Grafana per `grafana-metric-gateway`). On success it returns `{ "ok": true, "evidence": "Gateway reachable; datasource responded." }`. On HTTP 401/403 it returns an auth-specific message; on connection error it returns an unreachable message.
|
||||
|
||||
#### Scenario: gateway reachable returns ok
|
||||
|
||||
+ GIVEN a prometheus service with valid Grafana gateway credentials
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": true, "detail": "Grafana gateway reachable.", "evidence": "Gateway reachable; datasource responded." }`
|
||||
|
||||
#### Scenario: wrong API key surfaces auth failure
|
||||
|
||||
+ GIVEN a prometheus service whose `grafana_api_key` is invalid
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": false, "detail": <message mentioning authentication/authorization>, "evidence": null }`
|
||||
|
||||
### Requirement: CT-106 — Alertmanager test probes /api/v2/alerts or /api/v2/status
|
||||
|
||||
The `alertmanager` test routine MUST issue an authenticated GET (using the optional `api_key` secret when set) to `/api/v2/alerts` (or `/api/v2/status`). On success it returns `{ "ok": true, "evidence": <cluster version> }`. On failure it translates the error per the shared helper.
|
||||
|
||||
### Requirement: CT-107 — Jellyfin test calls /Users via JellyfinClient
|
||||
|
||||
The `jellyfin` test routine MUST construct a `JellyfinClient` from config + secrets and call `.users()`. On success it returns `{ "ok": true, "evidence": "<N> users" }`. On failure it translates the error.
|
||||
|
||||
### Requirement: CT-108 — Authentik test probes its directory endpoint
|
||||
|
||||
The `authentik` test routine MUST issue a lightweight authenticated GET against the Authentik directory endpoint (the same one the existing Authentik users flow uses). On success it returns `{ "ok": true, "evidence": <instance slug or "connected"> }`. On failure it translates the error.
|
||||
|
||||
### Requirement: CT-109 — ssh_tasks test reuses the test_machine_ssh connect flow
|
||||
|
||||
The `ssh_tasks` test routine MUST reuse the existing SSH connection logic (construct an `RemoteSSHClient` from the service config + SSH key, call `.connect()`, translate banner/auth errors per the established `test_machine_ssh` patterns). On success it returns `{ "ok": true, "evidence": "Connected to <host>:<port>" }`. The known-host recording behavior (first successful connect records the host key) is preserved.
|
||||
|
||||
### Requirement: CT-110 — Nextcloud test probes /status.php
|
||||
|
||||
The `nextcloud` test routine MUST issue an unauthenticated GET to `{base_url}/status.php`. On success it returns `{ "ok": true, "evidence": <Nextcloud version> }`. On failure it translates the error.
|
||||
|
||||
### Requirement: CT-111 — Backups type has no remote test
|
||||
|
||||
The `backups` service type has no remote connection (its data is internal). Its definition MUST set `test_callable = None`, and the endpoint returns `{ "ok": true, "detail": "No connection test for this service type" }` (CT-103).
|
||||
|
||||
### Requirement: CT-112 — The test endpoint MUST NOT persist any service row or secret
|
||||
|
||||
The test endpoint MUST NOT call `store.upsert_service`, `store.update_setting`, or any other persistence method. It accepts plaintext secrets in the request body (over TLS, identical to the existing create endpoint), probes the target, and discards the secrets. A test MUST verify this by calling `POST /api/services/test` and asserting the service count in the store is unchanged afterward.
|
||||
|
||||
#### Scenario: calling /test does not create a row
|
||||
|
||||
+ GIVEN an empty service store
|
||||
+ WHEN `POST /api/services/test` is called with a valid `qbittorrent` input
|
||||
+ THEN the store's service count remains 0
|
||||
+ AND no new row exists for that service type
|
||||
|
||||
### Requirement: CT-113 — Secrets are never logged
|
||||
|
||||
The test endpoint MUST log at INFO level only `test requested type=%s ok=%s` — never the request body, config, or secrets. The log call MUST pass through `sanitize_log_extra` (the existing helper) so any accidental inclusion of secret-named keys is scrubbed.
|
||||
|
||||
#### Scenario: logs contain no secret values
|
||||
|
||||
+ GIVEN a test call carrying `secrets: { "password": "hunter2" }`
|
||||
+ WHEN the endpoint runs
|
||||
+ THEN no log line contains "hunter2"
|
||||
+ AND the INFO log line reads `test requested type=qbittorrent ok=false` (or similar), with no secret values
|
||||
|
||||
### Requirement: CT-114 — useTestServiceInstance mutation hook + testServiceInstance API client function
|
||||
|
||||
`frontend/src/api/services.ts` MUST gain `testServiceInstance(input: ServiceInstanceInput): Promise<TestResult>` posting to `/api/services/test`. `frontend/src/hooks/useServices.ts` MUST gain `useTestServiceInstance()` returning a TanStack mutation whose `mutateAsync` resolves to `{ ok, detail, evidence }`.
|
||||
|
||||
### Requirement: CT-115 — Test credentials button in CreateServiceDialog and the ServicePage edit dialog
|
||||
|
||||
Both the create dialog (`CreateServiceDialog` in `ServicesPage.tsx`) and the edit dialog on `ServicePage.tsx` MUST render a "Test credentials" button below the config/secret form fields. Clicking it fires the `useTestServiceInstance` mutation with the current draft values. While pending, the button shows a "Testing…" state and is disabled.
|
||||
|
||||
### Requirement: CT-116 — Result pill renders both states with evidence or detail
|
||||
|
||||
After a test completes, the dialog MUST render an inline status pill:
|
||||
|
||||
+ On success (`ok: true`): a green pill reading `✓ Connected` plus the `evidence` string.
|
||||
+ On failure (`ok: false`): a red/destructive pill reading `✗ <detail>`.
|
||||
|
||||
The pill is rendered in the same dialog, below the Test button, above the footer.
|
||||
|
||||
### Requirement: CT-117 — Create/Save confirm is gated on a passed test with a "Save anyway" override
|
||||
|
||||
The Create (in the add dialog) and Save (in the edit dialog) confirm buttons MUST be disabled by default until `testPassed` is true. A "Save anyway" checkbox MUST appear that, when checked, re-enables the confirm button — allowing an operator to pre-configure a service that is not yet online. The default state of the checkbox is unchecked (gating is on).
|
||||
|
||||
### Requirement: CT-118 — Editing a connectivity field clears the test result
|
||||
|
||||
When the user edits any field that affects connectivity (the URL/base_url config field, or any secret field), the dialog MUST clear the previous `testPassed` state and the result pill, requiring a fresh test before the confirm button is re-enabled (unless "Save anyway" is checked). This prevents a stale green result from masking a typo correction.
|
||||
|
||||
### Requirement: CT-119 — Backend tests cover routines, dispatch, validation, and no-persistence
|
||||
|
||||
`pytest` from `backend/` MUST pass, including new tests for:
|
||||
|
||||
+ Each type's `test_connection` routine (mocked client) returning the expected `{ok, detail, evidence}` shape for both success and at least one failure case.
|
||||
+ The `POST /api/services/test` endpoint dispatching correctly per type.
|
||||
+ Validation running before the test (422 on malformed config, no network call).
|
||||
+ No-persistence: calling `/test` leaves the store unchanged.
|
||||
|
||||
### Requirement: CT-120 — Frontend tests cover button, pill, gating, and field-clear
|
||||
|
||||
`npm run test` (vitest) from `frontend/` MUST pass, including new tests for:
|
||||
|
||||
+ The Test button fires the mutation.
|
||||
+ The result pill renders both the green (ok) and red (failed) states.
|
||||
+ The Create/Save confirm button is disabled until the test passes (and re-enabled by "Save anyway").
|
||||
+ Editing a connectivity field clears `testPassed`.
|
||||
|
||||
### Requirement: CT-121 — Build and lint stay green
|
||||
|
||||
`npm run build` (`tsc -b` + `vite build`) and `npm run lint` from `frontend/`, and `ruff check src tests` from `backend/`, MUST pass with 0 errors (pre-existing warnings acceptable).
|
||||
@@ -0,0 +1,183 @@
|
||||
# Sync Report — `service-credential-tester`
|
||||
|
||||
> Phase: **sync** · Change: `service-credential-tester` · Repo: `/home/user/manage`
|
||||
> Mode: file-backed (`artifactStore: openspec`). No source-code edits; only OpenSpec artifacts were
|
||||
> written. Not committed (parent owns the commit). The change folder was **not** moved (that is
|
||||
> `sdd-archive`'s job).
|
||||
|
||||
**Status: SYNCED.** A new canonical domain
|
||||
`openspec/specs/service-credential-testing/spec.md` was created from the verified change, and the
|
||||
change-side domain delta spec that unblocks the native status engine's `sync`/`archive` gates is
|
||||
also in place.
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
The `service-credential-tester` change shipped a **complete but flat**
|
||||
`openspec/changes/service-credential-tester/spec.md` (21 requirements, CT-101 … CT-121) with **no**
|
||||
per-domain delta spec under `openspec/changes/service-credential-tester/specs/<domain>/`. `sdd-sync`
|
||||
requires a domain delta spec; the flat spec alone does not satisfy the canonical-merge contract.
|
||||
|
||||
Verify already returned a **functional PASS** — verdict in `verify-report.md`: all 21 requirements
|
||||
CT-101 … CT-121 PASS against source, and all five gates are green (backend `pytest` 362 passed,
|
||||
`ruff` clean, frontend `npm run build`, `npm run lint` 0 errors, `npx vitest run` 158 passed). The
|
||||
verify report's two CRITICAL items are **archive-only** blockers (B-1 missing `apply-progress.md`,
|
||||
B-2 unchecked task checkboxes); the parent confirms `apply-progress.md` is now present and
|
||||
reconciled. These are lifecycle-hygiene items, not verification-integrity blockers, and they do not
|
||||
block `sdd-sync` of the green code.
|
||||
|
||||
This sync **reconciles** the flat-spec-vs-domain-spec gap:
|
||||
|
||||
1. Authored the missing **change-side domain delta spec** —
|
||||
`openspec/changes/service-credential-tester/specs/service-credential-testing/spec.md` — using a
|
||||
clean `## ADDED Requirements` structure that preserves the exact requirement IDs (CT-101 …
|
||||
CT-121) and text from the verified flat `spec.md`. This is what flips the native status engine's
|
||||
`specs` artifact from partial → done.
|
||||
2. **Synced** the end-state into the **canonical store** —
|
||||
`openspec/specs/service-credential-testing/spec.md` — the actual sync target. Because the
|
||||
canonical `service-credential-testing` domain did not previously exist, the native helper rule
|
||||
applies: *when the canonical spec does not exist, the change spec becomes the new canonical
|
||||
spec.* The two files therefore carry identical requirement bodies (delta under
|
||||
`## ADDED Requirements`; canonical under `## Requirements`).
|
||||
|
||||
Domain name **`service-credential-testing`** was chosen (per the dispatch brief) because it scopes
|
||||
the full new capability: the `POST /api/services/test` endpoint, the closed `test_callable`
|
||||
dispatch registry, the seven per-type connection-test routines, the no-persistence /
|
||||
no-secret-logging guarantees, and the frontend test UI. It is distinct from the existing canonical
|
||||
domains `web-ui` (MUI→shadcn migration), `prometheus-charting` (direct Prometheus metric
|
||||
visualization), and `service-storage` (per-service data lifecycle layer) — none of which was
|
||||
**touched**.
|
||||
|
||||
## 2. Structured status & actionContext findings
|
||||
|
||||
The native `gentle-pi.sdd-status` passed by the parent reports `changeName: null` with
|
||||
`blockedReasons: ["Change selection is ambiguous: per-instance-hook-scoping,
|
||||
service-credential-tester."]` because the engine auto-detected more than one active change. This
|
||||
sync task was **explicitly assigned** `service-credential-tester`; the ambiguity is a
|
||||
parent-resolution artifact and does not block this phase (`isNonAuthoritative: false`).
|
||||
|
||||
- `artifactStore: openspec`; change root `openspec/changes/service-credential-tester/`.
|
||||
- Artifacts present: `proposal.md`, `spec.md`, `design.md`, `tasks.md`, `apply-progress.md`,
|
||||
`verify-report.md`.
|
||||
- `verify: PASS` (functional verdict; gates green at `f6c67bd`).
|
||||
- `actionContext`: `mode: repo-local`, `workspaceRoot: /home/user/manage`,
|
||||
`allowedEditRoots: ["/home/user/manage"]`, `warnings: []`. All three files written are inside the
|
||||
authoritative workspace / allowed edit roots. ✓
|
||||
- `relationships.sameDomainActiveChanges: []`, `collisions: []` — **no active same-domain
|
||||
collisions**, so no archive/sync ordering decision was required.
|
||||
- The new `service-credential-testing` domain is distinct from the existing `web-ui`,
|
||||
`prometheus-charting`, and `service-storage` canonical domains; all three were left untouched
|
||||
(verified via `git status --porcelain`).
|
||||
|
||||
**Verify verdict nuance (archive-only blockers):** the verify report's verdict is "PASS
|
||||
(functionally) — every requirement CT-101 … CT-121 is met … ARCHIVE IS BLOCKED on a task-hygiene /
|
||||
missing-`apply-progress` issue." The two CRITICAL findings (B-1, B-2) are explicitly **archive**
|
||||
blockers, not verification failures: the code is functionally complete and all gates are green.
|
||||
`sdd-sync`'s stop conditions target *verification integrity* (unresolved FAIL/BLOCKED/CRITICAL
|
||||
**verification** blockers), and these are lifecycle-hygiene items. The parent states
|
||||
`apply-progress.md` is now reconciled. Sync therefore proceeds; the unchecked-tasks item is
|
||||
forwarded to `sdd-archive` (§7).
|
||||
|
||||
**Post-sync structural change:**
|
||||
`openspec/changes/service-credential-tester/specs/service-credential-testing/spec.md` now exists
|
||||
(`hasDomainSpecs` → true), resolving the missing-domain-spec condition that gated sync. The flat
|
||||
`spec.md` is intentionally **left in place** as the authoritative planning artifact the work was
|
||||
built against (the archive convention keeps flat specs too); it no longer triggers the "flat spec
|
||||
without domain specs" condition now that a domain delta sits alongside it.
|
||||
|
||||
## 3. Domains synced & canonical files updated
|
||||
|
||||
| Domain | Change-side delta (source) | Canonical (sync target) | Action |
|
||||
|---|---|---|---|
|
||||
| `service-credential-testing` | `openspec/changes/service-credential-tester/specs/service-credential-testing/spec.md` | `openspec/specs/service-credential-testing/spec.md` | **NEW domain** — `## ADDED Requirements` copied into canonical as a new spec |
|
||||
|
||||
- **Canonical file created:** `openspec/specs/service-credential-testing/spec.md` (21 requirements).
|
||||
- **Change-side delta created:**
|
||||
`openspec/changes/service-credential-tester/specs/service-credential-testing/spec.md`
|
||||
(21 requirements, all `## ADDED Requirements`).
|
||||
|
||||
## 4. Requirement delta (ADDED / MODIFIED / REMOVED)
|
||||
|
||||
- **ADDED (21)** — all to the new `service-credential-testing` domain (canonical did not exist
|
||||
pre-change). IDs and text preserved verbatim from the verified flat `spec.md`. Grouped logically:
|
||||
- *Endpoint shape, validation, dispatch* — CT-101, CT-102, CT-103
|
||||
- *Per-type test routines* — CT-104, CT-105, CT-106, CT-107, CT-108, CT-109, CT-110, CT-111
|
||||
- *No-persistence + security* — CT-112, CT-113
|
||||
- *Frontend test UI* — CT-114, CT-115, CT-116, CT-117, CT-118
|
||||
- *Tests + gate greenness* — CT-119, CT-120, CT-121
|
||||
- **MODIFIED (0)** — none (new domain; no pre-existing canonical requirements to replace).
|
||||
- **REMOVED (0)** — none.
|
||||
- **RENAMED (0)** — none (RENAMED is intentionally unsupported by the native delta helper; not used).
|
||||
|
||||
## 5. Guardrails, approvals & destructive-sync assessment
|
||||
|
||||
- **Same-domain collisions:** none (`sameDomainActiveChanges: []`, `collisions: []`). The new
|
||||
`service-credential-testing` domain does not overlap the existing `web-ui`,
|
||||
`prometheus-charting`, or `service-storage` canonical domains. No ordering decision was needed.
|
||||
- **Destructive sync:** **not applicable.** There are zero REMOVED requirements and zero large
|
||||
MODIFIED blocks (new domain; everything is ADDED). No destructive-sync parent approval was
|
||||
required beyond the explicit reconciliation instruction in the task.
|
||||
- **Legacy flat spec:** detected pre-sync; resolved by adding the domain delta spec alongside it
|
||||
(the block condition is specifically "flat spec *without* domain specs"). The flat spec was left
|
||||
in place as a planning artifact.
|
||||
- **`web-ui` / `prometheus-charting` / `service-storage` canonical isolation:** the existing
|
||||
`openspec/specs/web-ui/spec.md`, `openspec/specs/prometheus-charting/spec.md`, and
|
||||
`openspec/specs/service-storage/spec.md` were **not modified** — verified untouched by
|
||||
`git status --porcelain openspec/specs/web-ui openspec/specs/prometheus-charting
|
||||
openspec/specs/service-storage` (empty). The four domains are independent.
|
||||
|
||||
## 6. Validation / checks performed (file-backed, read-only)
|
||||
|
||||
Run from `/home/user/manage` (no source edits, no test re-runs — those are owned by verify and were
|
||||
already green at `f6c67bd`):
|
||||
|
||||
| Check | Command | Result |
|
||||
|---|---|---|
|
||||
| Canonical store populated | `ls openspec/specs/service-credential-testing/spec.md` | present ✓ |
|
||||
| Change-side domain spec present | `ls openspec/changes/service-credential-tester/specs/service-credential-testing/spec.md` | present ✓ |
|
||||
| Requirement-ID parity (flat ↔ delta ↔ canonical) | `grep -oE 'CT-[0-9]+' … \| sort -u \| wc -l` | **21 == 21 == 21**, identical IDs CT-101…CT-121 ✓ |
|
||||
| ID-set parity (flat ↔ canonical) | `diff <(…flat…) <(…canonical…)` | **PARITY OK** — identical ID sets ✓ |
|
||||
| Body-text parity (delta ↔ canonical) | `### Requirement:` regions compared | identical prose ✓ |
|
||||
| Delta is pure ADDED | count `## ADDED/MODIFIED/REMOVED Requirements` | ADDED=1, MODIFIED=0, REMOVED=0 ✓ (no destructive sync) |
|
||||
| Other canonicals untouched | `git status --porcelain openspec/specs/web-ui openspec/specs/prometheus-charting openspec/specs/service-storage` | empty (not modified) ✓ |
|
||||
| Flat spec left in place | `ls openspec/changes/service-credential-tester/spec.md` | present (not moved/deleted) ✓ |
|
||||
| No edits outside openspec | `git status --porcelain` (filtered) | only `openspec/specs/service-credential-testing/`, `openspec/changes/service-credential-tester/specs/`, and this report added ✓ |
|
||||
| Markdown validity | write-time lint | all three files "Markdown clean" ✓ |
|
||||
|
||||
## 7. Carry-over items for the archive summary
|
||||
|
||||
These verify-phase findings are non-blocking for sync and should land in the archive summary:
|
||||
|
||||
1. **[CRITICAL-process, archive-only] Unchecked task checkboxes.** At verify time, all 29 tasks in
|
||||
`tasks.md` were unchecked (`- [ ]`) and `apply-progress.md` was missing (verify findings B-1/B-2).
|
||||
The parent states `apply-progress.md` is now present and reconciled. `sdd-archive` should re-scan
|
||||
the native status engine to confirm `tasks: done` / `applyProgress: present` and tick any
|
||||
remaining unchecked boxes before moving the change to archive.
|
||||
2. **[INFO] Stale-proposal correction carried into the spec.** The flat spec documents that
|
||||
`jellyseerr` is no longer a distinct service type (merged into Jellyfin config by
|
||||
`services-as-hub-ia`); the 7 remote types + `backups` are the active coverage. No `jellyseerr`
|
||||
requirement exists.
|
||||
3. **[INFO] `prometheus` routes through the Grafana gateway.** CT-105's canonical text (verbatim
|
||||
from the flat spec) references `grafana-metric-gateway`. This is a pre-existing design assumption
|
||||
recorded faithfully; it does not affect sync.
|
||||
4. **[INFO] List-marker normalization.** The flat source uses `-` bullets; the linter normalized
|
||||
list markers in the delta/canonical files (cosmetic only — requirement prose is verbatim and the
|
||||
delta helper matches by `### Requirement:` blocks, not marker style).
|
||||
|
||||
## 8. Next recommended phase
|
||||
|
||||
→ **`sdd-archive`** (clean). Confirm the native status re-scan reports `specs: done` / `sync: ready`
|
||||
/ `archive: ready`, then move the change to
|
||||
`openspec/changes/archive/2026-07-09-service-credential-tester`, carrying over the items in §7 into
|
||||
the archive summary. Do **not** commit or push — the parent owns the commit with explicit paths.
|
||||
|
||||
---
|
||||
|
||||
### Appendix — Files written by this sync (OpenSpec only; no source code)
|
||||
|
||||
- `openspec/changes/service-credential-tester/specs/service-credential-testing/spec.md` —
|
||||
**change-side domain delta (`## ADDED Requirements`), 21 requirements CT-101…CT-121.**
|
||||
- `openspec/specs/service-credential-testing/spec.md` — **canonical spec (new domain), 21
|
||||
requirements.**
|
||||
- `openspec/changes/service-credential-tester/sync-report.md` — this report.
|
||||
@@ -0,0 +1,207 @@
|
||||
# Service Credential Testing
|
||||
|
||||
> Domain: `service-credential-testing` · **Canonical specification.** Synced from change `service-credential-tester`.
|
||||
>
|
||||
> This is the merged end-state of adding a per-service-type credential tester for the service
|
||||
> add/edit dialog. It captures the durable, post-change contracts for the `POST /api/services/test`
|
||||
> endpoint, the closed `test_callable` dispatch registry, the per-type connection-test routines,
|
||||
> the no-persistence + no-secret-logging guarantees, and the frontend test UI (Test button + result
|
||||
> pill + gated confirm with a "Save anyway" override) — not the per-slice delivery strategy (which
|
||||
> remains on record in the change's `spec.md` / `tasks.md` under
|
||||
> `openspec/changes/service-credential-tester/`).
|
||||
|
||||
## Purpose
|
||||
|
||||
Define WHAT must be true of Manage's service credential tester after the change: a backend endpoint
|
||||
(`POST /api/services/test`) that accepts the unsaved `ServiceInstanceInput` form values (never a
|
||||
stored service id), validates them first (reusing `_validate_input`, 422 on malformed config before
|
||||
any network probe), dispatches through a closed compile-time `test_callable` registry on
|
||||
`ServiceDefinition`, runs a minimal read-only probe per service type with a short timeout, and
|
||||
returns a structured `{ ok, detail, evidence }` result without persisting any row or secret. Each
|
||||
remote service type (qBittorrent, Prometheus via the Grafana gateway, Alertmanager, Jellyfin,
|
||||
Authentik, ssh_tasks, Nextcloud) carries its own `test_connection` routine alongside its integration
|
||||
module; `backups` has no remote test (`test_callable = None`) and returns an explicit no-test ok.
|
||||
|
||||
On the frontend, a shared `ServiceTestPanel` (Test credentials button + inline result pill) is wired
|
||||
into both the create dialog (`CreateServiceDialog` in `ServicesPage.tsx`) and the service edit
|
||||
surface (`ServiceConfigEditor` in `Settings.tsx`); the Create/Save confirm buttons are gated on a
|
||||
passing test, with a "Save anyway" override for pre-configuring an offline service, and editing any
|
||||
connectivity field clears the stale result. This spec is acceptance-focused and verifiable; it
|
||||
deliberately does not prescribe implementation.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: CT-101 — Test endpoint accepts unsaved service input and returns a structured result
|
||||
|
||||
The backend MUST expose `POST /api/services/test` accepting a `ServiceInstanceInput` body (`service_type`, `name`, `config`, `secrets`, `enabled`) — NOT a service id, because the tester operates on unsaved form values. On success the endpoint MUST return HTTP 200 with a body of shape `{ "ok": bool, "detail": str, "evidence": str | null }`. The endpoint MUST be auth-gated identically to every other `/api/services/*` endpoint (JWT/API-key).
|
||||
|
||||
#### Scenario: successful connection returns ok with evidence
|
||||
|
||||
+ GIVEN a valid `ServiceInstanceInput` for a reachable service
|
||||
+ WHEN `POST /api/services/test` is called with that body
|
||||
+ THEN the response is HTTP 200 with `{ "ok": true, "detail": <human summary>, "evidence": <proof string> }`
|
||||
|
||||
#### Scenario: failed connection returns ok=false with actionable detail
|
||||
|
||||
+ GIVEN a valid input whose target is unreachable or rejects auth
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is HTTP 200 with `{ "ok": false, "detail": <specific failure message>, "evidence": null }`
|
||||
|
||||
### Requirement: CT-102 — Validation runs before the test and rejects malformed config with 422
|
||||
|
||||
The endpoint MUST reuse the existing `_validate_input` helper (from `routers/services.py`) to validate `service_type`, `config` (via the definition's `config_model`), and secret-key names BEFORE any network probe. Malformed input MUST raise HTTP 422 with the same detail format as `POST /api/services/instances`, so the existing validation-error surfacing (the `#1` fix in `ServicesPage.tsx`) covers the test endpoint too.
|
||||
|
||||
#### Scenario: schema-less URL rejected before any network call
|
||||
|
||||
+ GIVEN a `qbittorrent` input whose `base_url` lacks the `http://` schema
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is HTTP 422 with a detail naming the schema requirement
|
||||
+ AND no network call is made to qBittorrent
|
||||
|
||||
#### Scenario: unknown service type rejected
|
||||
|
||||
+ GIVEN an input with `service_type = "nonexistent"`
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is HTTP 422 with "Unknown service type"
|
||||
|
||||
### Requirement: CT-103 — Closed per-type dispatch mirrors SERVICE_DEFINITIONS
|
||||
|
||||
The endpoint MUST dispatch to a per-type test routine via the service definition (a new optional `test_callable` field on `ServiceDefinition`, defaulting to `None`). The dispatch is closed and compile-time — no runtime plugin loading, no arbitrary callable execution. A service type whose definition has `test_callable = None` (e.g. `backups`, which has no remote connection) MUST return `{ "ok": true, "detail": "No connection test for this service type", "evidence": null }` without any network call.
|
||||
|
||||
#### Scenario: backups type returns ok without a network probe
|
||||
|
||||
+ GIVEN a `backups` service input
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the response is `{ "ok": true, "detail": "No connection test for this service type", "evidence": null }`
|
||||
+ AND no network request is issued
|
||||
|
||||
#### Scenario: unknown widget kind is not involved
|
||||
|
||||
+ GIVEN a `prometheus` input
|
||||
+ WHEN the endpoint is called
|
||||
+ THEN the dispatch keys off `service_type` only (not `widget_kind`); the test is per-service-instance, not per-widget
|
||||
|
||||
### Requirement: CT-104 — qBittorrent test logs in and probes maindata; surfaces auth failures specifically
|
||||
|
||||
The `qbittorrent` test routine MUST construct a `QbittorrentClient` from the config + decrypted secrets, call login (`POST /api/v2/auth/login`), then probe `GET /api/v2/sync/maindata` (or `/app/version`). On success it MUST return `{ "ok": true, "evidence": <qBittorrent version or "connected"> }`. When qBittorrent's login returns anything other than `"Ok."` (e.g. the literal `"Fails."`), the routine MUST return `{ "ok": false, "detail": "Authentication failed — qBittorrent rejected the credentials." }` — surfacing the specific auth failure at the UI layer, not just in backend logs.
|
||||
|
||||
#### Scenario: correct credentials succeed with version evidence
|
||||
|
||||
+ GIVEN a qBittorrent instance reachable at the configured URL with valid username/password
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": true, "detail": "Connected to qBittorrent.", "evidence": <version> }`
|
||||
|
||||
#### Scenario: wrong password surfaces a specific auth message
|
||||
|
||||
+ GIVEN a qBittorrent instance that returns `"Fails."` on login
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": false, "detail": "Authentication failed — qBittorrent rejected the credentials.", "evidence": null }`
|
||||
|
||||
### Requirement: CT-105 — Prometheus test queries the Grafana gateway with expr "up"
|
||||
|
||||
The `prometheus` test routine MUST issue `POST {grafana_url}/api/ds/query` carrying `Authorization: Bearer {grafana_api_key}` and a `queries[0]` entry keyed by the configured `datasource_uid` with `expr: "up"`. There MUST be no direct Prometheus HTTP call (the `prometheus` service sources through Grafana per `grafana-metric-gateway`). On success it returns `{ "ok": true, "evidence": "Gateway reachable; datasource responded." }`. On HTTP 401/403 it returns an auth-specific message; on connection error it returns an unreachable message.
|
||||
|
||||
#### Scenario: gateway reachable returns ok
|
||||
|
||||
+ GIVEN a prometheus service with valid Grafana gateway credentials
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": true, "detail": "Grafana gateway reachable.", "evidence": "Gateway reachable; datasource responded." }`
|
||||
|
||||
#### Scenario: wrong API key surfaces auth failure
|
||||
|
||||
+ GIVEN a prometheus service whose `grafana_api_key` is invalid
|
||||
+ WHEN the test routine runs
|
||||
+ THEN it returns `{ "ok": false, "detail": <message mentioning authentication/authorization>, "evidence": null }`
|
||||
|
||||
### Requirement: CT-106 — Alertmanager test probes /api/v2/alerts or /api/v2/status
|
||||
|
||||
The `alertmanager` test routine MUST issue an authenticated GET (using the optional `api_key` secret when set) to `/api/v2/alerts` (or `/api/v2/status`). On success it returns `{ "ok": true, "evidence": <cluster version> }`. On failure it translates the error per the shared helper.
|
||||
|
||||
### Requirement: CT-107 — Jellyfin test calls /Users via JellyfinClient
|
||||
|
||||
The `jellyfin` test routine MUST construct a `JellyfinClient` from config + secrets and call `.users()`. On success it returns `{ "ok": true, "evidence": "<N> users" }`. On failure it translates the error.
|
||||
|
||||
### Requirement: CT-108 — Authentik test probes its directory endpoint
|
||||
|
||||
The `authentik` test routine MUST issue a lightweight authenticated GET against the Authentik directory endpoint (the same one the existing Authentik users flow uses). On success it returns `{ "ok": true, "evidence": <instance slug or "connected"> }`. On failure it translates the error.
|
||||
|
||||
### Requirement: CT-109 — ssh_tasks test reuses the test_machine_ssh connect flow
|
||||
|
||||
The `ssh_tasks` test routine MUST reuse the existing SSH connection logic (construct an `RemoteSSHClient` from the service config + SSH key, call `.connect()`, translate banner/auth errors per the established `test_machine_ssh` patterns). On success it returns `{ "ok": true, "evidence": "Connected to <host>:<port>" }`. The known-host recording behavior (first successful connect records the host key) is preserved.
|
||||
|
||||
### Requirement: CT-110 — Nextcloud test probes /status.php
|
||||
|
||||
The `nextcloud` test routine MUST issue an unauthenticated GET to `{base_url}/status.php`. On success it returns `{ "ok": true, "evidence": <Nextcloud version> }`. On failure it translates the error.
|
||||
|
||||
### Requirement: CT-111 — Backups type has no remote test
|
||||
|
||||
The `backups` service type has no remote connection (its data is internal). Its definition MUST set `test_callable = None`, and the endpoint returns `{ "ok": true, "detail": "No connection test for this service type" }` (CT-103).
|
||||
|
||||
### Requirement: CT-112 — The test endpoint MUST NOT persist any service row or secret
|
||||
|
||||
The test endpoint MUST NOT call `store.upsert_service`, `store.update_setting`, or any other persistence method. It accepts plaintext secrets in the request body (over TLS, identical to the existing create endpoint), probes the target, and discards the secrets. A test MUST verify this by calling `POST /api/services/test` and asserting the service count in the store is unchanged afterward.
|
||||
|
||||
#### Scenario: calling /test does not create a row
|
||||
|
||||
+ GIVEN an empty service store
|
||||
+ WHEN `POST /api/services/test` is called with a valid `qbittorrent` input
|
||||
+ THEN the store's service count remains 0
|
||||
+ AND no new row exists for that service type
|
||||
|
||||
### Requirement: CT-113 — Secrets are never logged
|
||||
|
||||
The test endpoint MUST log at INFO level only `test requested type=%s ok=%s` — never the request body, config, or secrets. The log call MUST pass through `sanitize_log_extra` (the existing helper) so any accidental inclusion of secret-named keys is scrubbed.
|
||||
|
||||
#### Scenario: logs contain no secret values
|
||||
|
||||
+ GIVEN a test call carrying `secrets: { "password": "hunter2" }`
|
||||
+ WHEN the endpoint runs
|
||||
+ THEN no log line contains "hunter2"
|
||||
+ AND the INFO log line reads `test requested type=qbittorrent ok=false` (or similar), with no secret values
|
||||
|
||||
### Requirement: CT-114 — useTestServiceInstance mutation hook + testServiceInstance API client function
|
||||
|
||||
`frontend/src/api/services.ts` MUST gain `testServiceInstance(input: ServiceInstanceInput): Promise<TestResult>` posting to `/api/services/test`. `frontend/src/hooks/useServices.ts` MUST gain `useTestServiceInstance()` returning a TanStack mutation whose `mutateAsync` resolves to `{ ok, detail, evidence }`.
|
||||
|
||||
### Requirement: CT-115 — Test credentials button in CreateServiceDialog and the ServicePage edit dialog
|
||||
|
||||
Both the create dialog (`CreateServiceDialog` in `ServicesPage.tsx`) and the edit dialog on `ServicePage.tsx` MUST render a "Test credentials" button below the config/secret form fields. Clicking it fires the `useTestServiceInstance` mutation with the current draft values. While pending, the button shows a "Testing…" state and is disabled.
|
||||
|
||||
### Requirement: CT-116 — Result pill renders both states with evidence or detail
|
||||
|
||||
After a test completes, the dialog MUST render an inline status pill:
|
||||
|
||||
+ On success (`ok: true`): a green pill reading `✓ Connected` plus the `evidence` string.
|
||||
+ On failure (`ok: false`): a red/destructive pill reading `✗ <detail>`.
|
||||
|
||||
The pill is rendered in the same dialog, below the Test button, above the footer.
|
||||
|
||||
### Requirement: CT-117 — Create/Save confirm is gated on a passed test with a "Save anyway" override
|
||||
|
||||
The Create (in the add dialog) and Save (in the edit dialog) confirm buttons MUST be disabled by default until `testPassed` is true. A "Save anyway" checkbox MUST appear that, when checked, re-enables the confirm button — allowing an operator to pre-configure a service that is not yet online. The default state of the checkbox is unchecked (gating is on).
|
||||
|
||||
### Requirement: CT-118 — Editing a connectivity field clears the test result
|
||||
|
||||
When the user edits any field that affects connectivity (the URL/base_url config field, or any secret field), the dialog MUST clear the previous `testPassed` state and the result pill, requiring a fresh test before the confirm button is re-enabled (unless "Save anyway" is checked). This prevents a stale green result from masking a typo correction.
|
||||
|
||||
### Requirement: CT-119 — Backend tests cover routines, dispatch, validation, and no-persistence
|
||||
|
||||
`pytest` from `backend/` MUST pass, including new tests for:
|
||||
|
||||
+ Each type's `test_connection` routine (mocked client) returning the expected `{ok, detail, evidence}` shape for both success and at least one failure case.
|
||||
+ The `POST /api/services/test` endpoint dispatching correctly per type.
|
||||
+ Validation running before the test (422 on malformed config, no network call).
|
||||
+ No-persistence: calling `/test` leaves the store unchanged.
|
||||
|
||||
### Requirement: CT-120 — Frontend tests cover button, pill, gating, and field-clear
|
||||
|
||||
`npm run test` (vitest) from `frontend/` MUST pass, including new tests for:
|
||||
|
||||
+ The Test button fires the mutation.
|
||||
+ The result pill renders both the green (ok) and red (failed) states.
|
||||
+ The Create/Save confirm button is disabled until the test passes (and re-enabled by "Save anyway").
|
||||
+ Editing a connectivity field clears `testPassed`.
|
||||
|
||||
### Requirement: CT-121 — Build and lint stay green
|
||||
|
||||
`npm run build` (`tsc -b` + `vite build`) and `npm run lint` from `frontend/`, and `ruff check src tests` from `backend/`, MUST pass with 0 errors (pre-existing warnings acceptable).
|
||||
Reference in New Issue
Block a user