Compare commits

...

6 Commits

Author SHA1 Message Date
Developer f6a86310cc style(widgets): apply formatter to widget rebind files 2026-06-22 18:22:18 +00:00
Developer 10fd4ead4a feat(widgets): rebind widgets to the service registry
PR 2 of 4 for the runtime service registry change.

- dashboard_widgets gains service_id + widget_kind columns (legacy
  addon_id/widget_type kept but unused).
- Source adapters take (service: ServiceRecord | None, widget_kind, config).
  SERVICE_ADAPTERS keyed by service_type; BUILTIN_ADAPTERS for backups/static.
- Backups and static stay as service-less built-ins (service_id nullable),
  exposed via GET /api/widgets/builtin.
- SSH task adapter resolves the task + instance, runs over SSH, and appends a
  service_task_runs history row on success/failure/timeout/error.
- Retire widgets/registry.py; widget metadata now comes from the integrations
  registry + widgets/builtin. Remove /api/widgets/types and /api/widgets/sources.
- Stop default widget seeding (fresh install = empty dashboard).
- Rewrite widget tests around the service-bound + built-in model (26 tests).

Backend-only breaking change; frontend is reconciled in Slice 3. Build/lint
stay green; pytest 222 passed.
2026-06-22 16:42:56 +00:00
Developer 2452e2e1e4 Merge pull request 'feat(services): backend service registry foundation' (#7) from feat/service-registry-backend-foundation into main 2026-06-22 14:00:07 +00:00
Developer fd534a816b style(services): apply formatter to service registry files 2026-06-22 14:00:07 +00:00
Developer 8cdeadd6dd feat(services): backend service registry foundation (encryption, definitions, CRUD)
PR 1 of 4 for the runtime service registry change.

- Add Fernet encryption helper (services/secrets.py) with a required
  MANAGE_ENCRYPTION_KEY; validate it on startup.
- Add closed integrations/ registry with Pydantic config + widget-config
  definitions for grafana, prometheus, jellyfin, nextcloud, and ssh_tasks.
- Add services + service_task_runs tables and SettingsStore CRUD with
  cascade-delete (defensive until widgets carry service_id).
- Add /api/services/types and /api/services/instances CRUD (encrypted secrets,
  secrets_set flags only; never plaintext).
- Declare cryptography as a direct dependency.
- Require MANAGE_ENCRYPTION_KEY in compose + .env.example + README.
- Add 25 backend tests (registry, encryption, CRUD, cascade, task-run history).

Verification: ruff clean; pytest 225 passed; frontend lint/build green.
2026-06-22 12:56:03 +00:00
Developer d1819c0186 Merge pull request 'docs(service-registry): SDD artifacts' from docs/service-registry-sdd into main 2026-06-22 11:14:01 +00:00
26 changed files with 2155 additions and 763 deletions
+3
View File
@@ -29,6 +29,9 @@ ALERTMANAGER_URL=http://alertmanager:9093
ALERTMANAGER_WEBHOOK_URL= ALERTMANAGER_WEBHOOK_URL=
GRAFANA_URL=http://grafana:3000 GRAFANA_URL=http://grafana:3000
PROMETHEUS_URL=http://prometheus:9090 PROMETHEUS_URL=http://prometheus:9090
# Required: master key for encrypting service secrets (API keys/tokens) at rest.
# Generate one with: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
MANAGE_ENCRYPTION_KEY=replace-with-a-fernet-key
BACKEND_CACHE_DIR=./backend-cache BACKEND_CACHE_DIR=./backend-cache
# Auth # Auth
+5
View File
@@ -89,6 +89,7 @@ export VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback
export VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ export VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/
export VITE_GRAFANA_URL=https://grafana.manage.example.com export VITE_GRAFANA_URL=https://grafana.manage.example.com
export VITE_PROMETHEUS_URL=https://prometheus.manage.example.com export VITE_PROMETHEUS_URL=https://prometheus.manage.example.com
export MANAGE_ENCRYPTION_KEY=$(python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")
docker compose up --build docker compose up --build
``` ```
@@ -148,6 +149,10 @@ GRAFANA_URL=http://grafana:3000
PROMETHEUS_URL=http://prometheus:9090 PROMETHEUS_URL=http://prometheus:9090
VITE_GRAFANA_URL=https://grafana.manage.example.com VITE_GRAFANA_URL=https://grafana.manage.example.com
VITE_PROMETHEUS_URL=https://prometheus.manage.example.com VITE_PROMETHEUS_URL=https://prometheus.manage.example.com
# Required: master key encrypting service secrets (API keys/tokens) at rest.
# Generate one with: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
MANAGE_ENCRYPTION_KEY=replace-with-a-fernet-key
``` ```
## Remote server requirements ## Remote server requirements
+1
View File
@@ -15,6 +15,7 @@ dependencies = [
"python-multipart>=0.0.9", "python-multipart>=0.0.9",
"prometheus-client>=0.21", "prometheus-client>=0.21",
"python-json-logger>=2.0", "python-json-logger>=2.0",
"cryptography>=42.0",
] ]
[project.optional-dependencies] [project.optional-dependencies]
@@ -0,0 +1 @@
"""Closed registry of service integrations."""
@@ -0,0 +1,119 @@
"""Base classes for service integrations.
A *service definition* is a closed, compile-time description of an external service
the app can talk to (Grafana, Jellyfin, …). Each definition declares:
* its non-secret ``config_schema`` (derived from a Pydantic model),
* the secret fields it accepts (API keys / tokens),
* the widget kinds it can contribute to the dashboard (each with its own
Pydantic-derived config schema).
Definitions live in :mod:`media_library_viewer_api.integrations` modules and are
assembled into the closed :data:`~media_library_viewer_api.integrations.registry.SERVICE_DEFINITIONS`
map. There is no runtime plugin loading.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
from pydantic import BaseModel
class ServiceConfigBase(BaseModel):
"""Base for per-service non-secret config models.
Subclass this in each integration module and declare the connection fields.
The JSON schema is derived via ``model_json_schema()`` and exposed to the UI.
"""
class WidgetConfigBase(BaseModel):
"""Base for per-widget config models.
Subclass this for each widget kind a service provides. Widget configs never
hold secrets; credentials live on the parent service record.
"""
model_config = {"extra": "forbid"}
@dataclass(frozen=True)
class SecretField:
"""A secret field stored encrypted on the service record."""
key: str
label: str
required: bool = False
helper: str | None = None
@dataclass(frozen=True)
class WidgetKind:
"""A widget kind contributed by a service definition."""
kind: str
name: str
description: str
config_schema: dict[str, Any]
default_config: dict[str, Any] = field(default_factory=dict)
refresh_interval_ms: int = 0
config_model: type[WidgetConfigBase] | None = None
@dataclass(frozen=True)
class ServiceDefinition:
"""Closed description of an external service type."""
service_type: str
name: str
description: str
config_model: type[ServiceConfigBase]
secret_fields: list[SecretField]
widget_kinds: list[WidgetKind]
@property
def config_schema(self) -> dict[str, Any]:
"""JSON schema for the service's non-secret config."""
return self.config_model.model_json_schema()
@property
def secret_keys(self) -> set[str]:
return {sf.key for sf in self.secret_fields}
def widget_kind(self, kind: str) -> WidgetKind | None:
for wk in self.widget_kinds:
if wk.kind == kind:
return wk
return None
def widget_kind(
kind: str,
name: str,
description: str,
model_cls: type[WidgetConfigBase],
*,
default_config: dict[str, Any] | None = None,
refresh_interval_ms: int = 0,
) -> WidgetKind:
"""Build a :class:`WidgetKind` from a Pydantic widget-config model."""
schema = model_cls.model_json_schema()
# Strip Pydantic's title noise so the exposed schema stays clean.
schema.pop("title", None)
return WidgetKind(
kind=kind,
name=name,
description=description,
config_schema=schema,
default_config=dict(default_config or {}),
refresh_interval_ms=refresh_interval_ms,
config_model=model_cls,
)
def validate_config(model_cls: type[BaseModel], config: dict[str, Any] | None) -> dict[str, Any]:
"""Validate a config dict against a Pydantic model and return the cleaned dict."""
instance = model_cls.model_validate(config or {})
return instance.model_dump(exclude_none=True)
@@ -0,0 +1,46 @@
"""Grafana service definition."""
from __future__ import annotations
from media_library_viewer_api.integrations.base import (
SecretField,
ServiceConfigBase,
ServiceDefinition,
WidgetConfigBase,
widget_kind,
)
class GrafanaConfig(ServiceConfigBase):
"""Non-secret Grafana connection config."""
base_url: str
timeout_seconds: int = 5
class GrafanaLinkWidgetConfig(WidgetConfigBase):
"""Deep-link to a Grafana dashboard or panel."""
dashboard_uid: str
panel_id: int | None = None
DEFINITION = ServiceDefinition(
service_type="grafana",
name="Grafana",
description="Dashboards, metrics, and logs.",
config_model=GrafanaConfig,
secret_fields=[
SecretField(key="api_key", label="API key", helper="Service account token (optional)"),
],
widget_kinds=[
widget_kind(
kind="link",
name="Dashboard link",
description="Deep-link to a Grafana dashboard or panel.",
model_cls=GrafanaLinkWidgetConfig,
default_config={"dashboard_uid": ""},
refresh_interval_ms=0,
),
],
)
@@ -0,0 +1,47 @@
"""Jellyfin service definition."""
from __future__ import annotations
from media_library_viewer_api.integrations.base import (
SecretField,
ServiceConfigBase,
ServiceDefinition,
WidgetConfigBase,
widget_kind,
)
class JellyfinConfig(ServiceConfigBase):
"""Non-secret Jellyfin connection config."""
base_url: str
user_id: str = ""
timeout_seconds: int = 10
class JellyfinActivityWidgetConfig(WidgetConfigBase):
"""Live Jellyfin session activity."""
# No user-overridable fields; the service record carries user_id.
pass
DEFINITION = ServiceDefinition(
service_type="jellyfin",
name="Jellyfin",
description="Media server with live session activity.",
config_model=JellyfinConfig,
secret_fields=[
SecretField(key="api_key", label="API key", required=True),
],
widget_kinds=[
widget_kind(
kind="activity",
name="Activity",
description="Live sessions and idle users.",
model_cls=JellyfinActivityWidgetConfig,
default_config={},
refresh_interval_ms=30_000,
),
],
)
@@ -0,0 +1,32 @@
"""Nextcloud service definition.
Nextcloud is included as a proof-of-concept third-party service. It has no
dashboard widgets yet; its service page holds connection config only.
"""
from __future__ import annotations
from media_library_viewer_api.integrations.base import (
SecretField,
ServiceConfigBase,
ServiceDefinition,
)
class NextcloudConfig(ServiceConfigBase):
"""Non-secret Nextcloud connection config."""
base_url: str
username: str = ""
DEFINITION = ServiceDefinition(
service_type="nextcloud",
name="Nextcloud",
description="Self-hosted files and collaboration.",
config_model=NextcloudConfig,
secret_fields=[
SecretField(key="app_password", label="App password", required=True),
],
widget_kinds=[],
)
@@ -0,0 +1,45 @@
"""Prometheus service definition."""
from __future__ import annotations
from media_library_viewer_api.integrations.base import (
SecretField,
ServiceConfigBase,
ServiceDefinition,
WidgetConfigBase,
widget_kind,
)
class PrometheusConfig(ServiceConfigBase):
"""Non-secret Prometheus connection config."""
base_url: str
timeout_seconds: int = 10
class PrometheusMetricWidgetConfig(WidgetConfigBase):
"""A PromQL instant query rendered as a metric."""
promql: str
DEFINITION = ServiceDefinition(
service_type="prometheus",
name="Prometheus",
description="Metrics storage and PromQL queries.",
config_model=PrometheusConfig,
secret_fields=[
SecretField(key="api_key", label="API key", helper="Optional bearer token"),
],
widget_kinds=[
widget_kind(
kind="metric",
name="Metric",
description="Instant query result rendered as a metric.",
model_cls=PrometheusMetricWidgetConfig,
default_config={"promql": ""},
refresh_interval_ms=30_000,
),
],
)
@@ -0,0 +1,48 @@
"""Closed registry of service definitions.
Adding a brand-new service still requires a backend deploy and a module here.
There is no runtime plugin loading.
"""
from __future__ import annotations
from media_library_viewer_api.integrations.base import ServiceDefinition, WidgetKind
from media_library_viewer_api.integrations.grafana import DEFINITION as GRAFANA
from media_library_viewer_api.integrations.jellyfin import DEFINITION as JELLYFIN
from media_library_viewer_api.integrations.nextcloud import DEFINITION as NEXTCLOUD
from media_library_viewer_api.integrations.prometheus import DEFINITION as PROMETHEUS
from media_library_viewer_api.integrations.ssh_tasks import DEFINITION as SSH_TASKS
SERVICE_DEFINITIONS: dict[str, ServiceDefinition] = {
GRAFANA.service_type: GRAFANA,
PROMETHEUS.service_type: PROMETHEUS,
JELLYFIN.service_type: JELLYFIN,
NEXTCLOUD.service_type: NEXTCLOUD,
SSH_TASKS.service_type: SSH_TASKS,
}
def list_service_types() -> list[str]:
"""Return all registered service type names (sorted for stable output)."""
return sorted(SERVICE_DEFINITIONS)
def get_service_definition(service_type: str) -> ServiceDefinition | None:
"""Return the definition for a service type, or ``None`` if unknown."""
return SERVICE_DEFINITIONS.get(service_type)
def get_widget_kind(service_type: str, widget_kind: str) -> WidgetKind | None:
"""Return a widget kind declared by a service definition, or ``None``."""
definition = get_service_definition(service_type)
if definition is None:
return None
return definition.widget_kind(widget_kind)
def require_service_definition(service_type: str) -> ServiceDefinition:
"""Return the definition or raise ``ValueError`` for an unknown type."""
definition = get_service_definition(service_type)
if definition is None:
raise ValueError(f"Unknown service type: {service_type}")
return definition
@@ -0,0 +1,60 @@
"""SSH task runner service definition.
An ``ssh_tasks`` instance is an SSH endpoint that can run reusable saved tasks.
Tasks themselves stay in the global saved-task registry; the instance only owns
transport (host/port/user/key). Every run is recorded in ``service_task_runs``
and shown as history on the instance's service page.
"""
from __future__ import annotations
from media_library_viewer_api.integrations.base import (
SecretField,
ServiceConfigBase,
ServiceDefinition,
WidgetConfigBase,
widget_kind,
)
class SshTasksConfig(ServiceConfigBase):
"""Non-secret SSH task runner config.
The SSH key itself lives in the saved SSH-key registry and is referenced by
``ssh_key_id``. An optional ``passphrase`` is stored as a secret.
"""
host: str
port: int = 22
username: str = ""
ssh_key_id: str = ""
timeout_seconds: int = 30
class SshTaskOutputWidgetConfig(WidgetConfigBase):
"""Output of a saved task run on this instance."""
task_id: str
# service_id is implicit (the widget's service); allow overriding per-widget.
service_id: str | None = None
DEFINITION = ServiceDefinition(
service_type="ssh_tasks",
name="SSH task runner",
description="Run reusable saved tasks over SSH and keep run history.",
config_model=SshTasksConfig,
secret_fields=[
SecretField(key="passphrase", label="Key passphrase", helper="Optional"),
],
widget_kinds=[
widget_kind(
kind="task_output",
name="Task output",
description="Output of a saved task run.",
model_cls=SshTaskOutputWidgetConfig,
default_config={"task_id": ""},
refresh_interval_ms=0,
),
],
)
@@ -23,6 +23,7 @@ from media_library_viewer_api.observability import (
) )
from media_library_viewer_api.routers import backups as backups_router from media_library_viewer_api.routers import backups as backups_router
from media_library_viewer_api.routers import dashboard, files, jobs, media, monitoring, tasks, users from media_library_viewer_api.routers import dashboard, files, jobs, media, monitoring, tasks, users
from media_library_viewer_api.routers import services as services_router
from media_library_viewer_api.routers import widgets as widgets_router from media_library_viewer_api.routers import widgets as widgets_router
from media_library_viewer_api.routers.settings import router as settings_router from media_library_viewer_api.routers.settings import router as settings_router
@@ -38,6 +39,9 @@ async def lifespan(app: FastAPI):
settings = get_settings() settings = get_settings()
configure_logging(settings.log_level, settings.log_format) configure_logging(settings.log_level, settings.log_format)
validate_auth_settings(settings) validate_auth_settings(settings)
from media_library_viewer_api.services.secrets import validate_encryption_key
validate_encryption_key()
logger.info("Backend startup complete: %s", describe_settings(settings)) logger.info("Backend startup complete: %s", describe_settings(settings))
logger.info("Managed known_hosts will be populated lazily on first successful SSH connection") logger.info("Managed known_hosts will be populated lazily on first successful SSH connection")
try: try:
@@ -143,6 +147,7 @@ app.include_router(tasks.router)
app.include_router(settings_router) app.include_router(settings_router)
app.include_router(backups_router.router) app.include_router(backups_router.router)
app.include_router(widgets_router.router) app.include_router(widgets_router.router)
app.include_router(services_router.router)
@app.get("/api/health") @app.get("/api/health")
@@ -0,0 +1,94 @@
"""Pydantic models for the service registry API."""
from __future__ import annotations
from typing import Any
from pydantic import BaseModel, Field, field_validator
def _validate_config_keys(config: dict[str, Any]) -> dict[str, Any]:
"""Reject credential keys in non-secret service config.
Secrets are sent in the separate ``secrets`` mapping; the plain ``config``
object must never hold them.
"""
forbidden = {
"password",
"token",
"secret",
"api_key",
"apikey",
"private_key",
"passphrase",
"credential",
}
def _check(value: Any) -> None:
if isinstance(value, dict):
for key, child in value.items():
if key.lower() in forbidden:
raise ValueError(f"Credential key '{key}' is not allowed in service config")
_check(child)
elif isinstance(value, list):
for item in value:
_check(item)
_check(config)
return config
class ServiceInstanceInput(BaseModel):
"""Payload for creating or updating a service instance."""
id: str | None = None
service_type: str = Field(..., min_length=1)
name: str = Field(..., min_length=1)
config: dict[str, Any] = Field(default_factory=dict)
secrets: dict[str, str] = Field(default_factory=dict)
enabled: bool = True
@field_validator("config")
@classmethod
def reject_credential_keys(cls, value: dict[str, Any]) -> dict[str, Any]:
return _validate_config_keys(value or {})
class ServiceInstance(BaseModel):
"""Persisted service instance returned by the API (no plaintext secrets)."""
id: str
service_type: str
name: str
config: dict[str, Any]
secrets_set: dict[str, bool]
enabled: bool
created_at: int
updated_at: int
class SecretFieldInfo(BaseModel):
key: str
label: str
required: bool = False
helper: str | None = None
class WidgetKindInfo(BaseModel):
kind: str
name: str
description: str
config_schema: dict[str, Any]
default_config: dict[str, Any]
refresh_interval_ms: int
class ServiceTypeInfo(BaseModel):
"""Metadata about a registered service type."""
service_type: str
name: str
description: str
config_schema: dict[str, Any]
secret_fields: list[SecretFieldInfo]
widget_kinds: list[WidgetKindInfo]
@@ -1,8 +1,18 @@
"""Pydantic models for the dashboard widget system.""" """Pydantic models for the dashboard widget system.
Widgets are either:
* **service-bound** — reference a ``service_id`` and a ``widget_kind`` declared
by that service's definition (Grafana link, Prometheus metric, Jellyfin
activity, SSH task output); or
* **built-in** — ``service_id`` is null and ``widget_kind`` is one of the
service-less kinds (backups, static).
"""
from __future__ import annotations
from typing import Any from typing import Any
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field, field_validator, model_validator
FORBIDDEN_CONFIG_KEYS = { FORBIDDEN_CONFIG_KEYS = {
"password", "password",
@@ -47,8 +57,8 @@ def _validate_config_keys(config: dict[str, Any]) -> dict[str, Any]:
class _WidgetInstanceBase(BaseModel): class _WidgetInstanceBase(BaseModel):
"""Shared fields between input and output widget models.""" """Shared fields between input and output widget models."""
addon_id: str service_id: str | None = None
widget_type: str widget_kind: str = Field(..., min_length=1)
title: str = Field(..., min_length=1) title: str = Field(..., min_length=1)
config: dict[str, Any] = Field(default_factory=dict) config: dict[str, Any] = Field(default_factory=dict)
enabled: bool = True enabled: bool = True
@@ -59,6 +69,13 @@ class _WidgetInstanceBase(BaseModel):
def reject_credential_keys(cls, value: dict[str, Any]) -> dict[str, Any]: def reject_credential_keys(cls, value: dict[str, Any]) -> dict[str, Any]:
return _validate_config_keys(value or {}) return _validate_config_keys(value or {})
@model_validator(mode="after")
def _validate_kind(self) -> "_WidgetInstanceBase":
# The kind must be non-empty (Field enforces it); service_id may be None
# for built-ins. Deeper validation happens in the router against the
# service definition / built-in registry.
return self
class WidgetInstanceInput(_WidgetInstanceBase): class WidgetInstanceInput(_WidgetInstanceBase):
"""Payload for creating or updating a widget instance.""" """Payload for creating or updating a widget instance."""
@@ -74,22 +91,21 @@ class WidgetInstance(_WidgetInstanceBase):
updated_at: int updated_at: int
class WidgetTypeInfo(BaseModel): class BuiltinWidgetKindInfo(BaseModel):
"""Metadata about a built-in widget type.""" """Metadata about a built-in (service-less) widget kind."""
addon_id: str kind: str
widget_type: str
name: str name: str
description: str description: str
source_type: str
config_schema: dict[str, Any] config_schema: dict[str, Any]
default_config: dict[str, Any]
refresh_interval_ms: int
class WidgetDataResponse(BaseModel): class WidgetDataResponse(BaseModel):
"""Response from the per-widget data endpoint.""" """Response from the per-widget data endpoint."""
widget_id: str widget_id: str
widget_type: str
data: dict[str, Any] | None = None data: dict[str, Any] | None = None
error: str | None = None error: str | None = None
fetched_at: int fetched_at: int
@@ -0,0 +1,182 @@
"""REST API for the service registry.
Service instances hold non-secret config and encrypted secrets. Plaintext
secrets are never returned; only the boolean ``secrets_set`` map is exposed.
"""
from __future__ import annotations
import logging
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, status
from media_library_viewer_api.dependencies import get_settings_store
from media_library_viewer_api.integrations.base import validate_config
from media_library_viewer_api.integrations.registry import (
SERVICE_DEFINITIONS,
get_service_definition,
require_service_definition,
)
from media_library_viewer_api.models.services import (
SecretFieldInfo,
ServiceInstance,
ServiceInstanceInput,
ServiceTypeInfo,
WidgetKindInfo,
)
from media_library_viewer_api.services.settings_store import SettingsStore
router = APIRouter(prefix="/api/services", tags=["services"])
logger = logging.getLogger(__name__)
def _to_type_info(service_type: str) -> ServiceTypeInfo:
definition = require_service_definition(service_type)
return ServiceTypeInfo(
service_type=definition.service_type,
name=definition.name,
description=definition.description,
config_schema=definition.config_schema,
secret_fields=[
SecretFieldInfo(
key=sf.key,
label=sf.label,
required=sf.required,
helper=sf.helper,
)
for sf in definition.secret_fields
],
widget_kinds=[
WidgetKindInfo(
kind=wk.kind,
name=wk.name,
description=wk.description,
config_schema=wk.config_schema,
default_config=wk.default_config,
refresh_interval_ms=wk.refresh_interval_ms,
)
for wk in definition.widget_kinds
],
)
def _to_instance(row: dict[str, Any]) -> ServiceInstance:
"""Build an API response model, surfacing only secret 'set' flags."""
definition = get_service_definition(row["service_type"])
known_secrets = definition.secret_keys if definition else set()
secrets_blob = row.get("secrets") or {}
secrets_set = {key: (key in secrets_blob and bool(secrets_blob[key])) for key in known_secrets}
return ServiceInstance(
id=row["id"],
service_type=row["service_type"],
name=row["name"],
config=row.get("config") or {},
secrets_set=secrets_set,
enabled=row["enabled"],
created_at=row["created_at"],
updated_at=row["updated_at"],
)
def _validate_input(body: ServiceInstanceInput) -> None:
"""Validate service_type, config, and secret keys against the definition."""
definition = get_service_definition(body.service_type)
if definition is None:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Unknown service type: {body.service_type}",
)
try:
validate_config(definition.config_model, body.config)
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid service config: {exc}",
) from exc
unknown_secrets = set(body.secrets) - definition.secret_keys
if unknown_secrets:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Unknown secret fields for {body.service_type}: {sorted(unknown_secrets)}",
)
@router.get("/types")
def list_types() -> list[ServiceTypeInfo]:
"""Return metadata for every registered service type."""
return [_to_type_info(service_type) for service_type in sorted(SERVICE_DEFINITIONS)]
@router.get("/instances")
def list_instances(
service_type: str | None = None,
store: SettingsStore = Depends(get_settings_store),
) -> list[ServiceInstance]:
"""Return all persisted service instances (no plaintext secrets)."""
rows = store.list_services(service_type)
return [_to_instance(row) for row in rows]
@router.post("/instances", status_code=status.HTTP_201_CREATED)
def create_instance(
body: ServiceInstanceInput,
store: SettingsStore = Depends(get_settings_store),
) -> ServiceInstance:
"""Create a new service instance."""
_validate_input(body)
row = store.upsert_service(
{
"id": body.id,
"service_type": body.service_type,
"name": body.name,
"config": body.config,
"enabled": body.enabled,
},
secret_values=body.secrets,
)
return _to_instance(row)
@router.put("/instances/{service_id}")
def update_instance(
service_id: str,
body: ServiceInstanceInput,
store: SettingsStore = Depends(get_settings_store),
) -> ServiceInstance:
"""Update an existing service instance."""
existing = store.get_service(service_id)
if not existing:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Service not found")
if body.id is not None and body.id != service_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="ID in path does not match ID in body",
)
_validate_input(body)
row = store.upsert_service(
{
"id": service_id,
"service_type": body.service_type,
"name": body.name,
"config": body.config,
"enabled": body.enabled,
},
secret_values=body.secrets,
service_id=service_id,
)
return _to_instance(row)
@router.delete("/instances/{service_id}")
def delete_instance(
service_id: str,
store: SettingsStore = Depends(get_settings_store),
) -> dict[str, str]:
"""Delete a service instance (cascade-deletes widgets referencing it)."""
existing = store.get_service(service_id)
if not existing:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Service not found")
store.delete_service(service_id)
return {"status": "deleted"}
@@ -1,4 +1,11 @@
"""REST API for dashboard widget instances and registry metadata.""" """REST API for dashboard widget instances.
Widgets are either service-bound (``service_id`` + ``widget_kind`` from the
service definition) or built-in (``service_id`` is null; ``widget_kind`` is one
of the service-less kinds exposed by ``GET /api/widgets/builtin``).
"""
from __future__ import annotations
import logging import logging
import time import time
@@ -7,68 +14,91 @@ from typing import Any
from fastapi import APIRouter, Depends, HTTPException, status from fastapi import APIRouter, Depends, HTTPException, status
from media_library_viewer_api.dependencies import get_settings_store from media_library_viewer_api.dependencies import get_settings_store
from media_library_viewer_api.integrations.base import validate_config
from media_library_viewer_api.integrations.registry import get_service_definition
from media_library_viewer_api.models.widgets import ( from media_library_viewer_api.models.widgets import (
BuiltinWidgetKindInfo,
WidgetDataResponse, WidgetDataResponse,
WidgetInstance, WidgetInstance,
WidgetInstanceInput, WidgetInstanceInput,
) )
from media_library_viewer_api.services.settings_store import SettingsStore from media_library_viewer_api.services.settings_store import SettingsStore
from media_library_viewer_api.widgets.registry import ( from media_library_viewer_api.widgets.builtin import (
get_widget_info, BUILTIN_WIDGET_KINDS,
list_source_types, is_builtin_kind,
list_widget_types, validate_builtin_config,
validate_config, )
from media_library_viewer_api.widgets.sources import (
build_service_record,
get_builtin_adapter,
get_service_adapter,
) )
from media_library_viewer_api.widgets.sources import get_source_adapter
router = APIRouter(prefix="/api/widgets", tags=["widgets"]) router = APIRouter(prefix="/api/widgets", tags=["widgets"])
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _registry_for_type(widget_type: str) -> dict[str, Any]: def _validate_widget_input(body: WidgetInstanceInput, store: SettingsStore) -> None:
from media_library_viewer_api.widgets.registry import WIDGET_REGISTRY """Validate widget_kind + config against the service definition or built-ins."""
if body.service_id:
service = store.get_service(body.service_id)
if not service:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Service {body.service_id} not found",
)
definition = get_service_definition(service["service_type"])
if definition is None:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Unknown service type: {service['service_type']}",
)
widget_kind = definition.widget_kind(body.widget_kind)
if widget_kind is None:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=(f"Service type '{service['service_type']}' does not provide widget kind '{body.widget_kind}'"),
)
if widget_kind.config_model is not None:
try:
validate_config(widget_kind.config_model, body.config)
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid widget config: {exc}",
) from exc
else:
if not is_builtin_kind(body.widget_kind):
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=(
f"Unknown built-in widget kind '{body.widget_kind}' (set service_id for service-bound widgets)"
),
)
try:
validate_builtin_config(body.widget_kind, body.config)
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid widget config: {exc}",
) from exc
info = WIDGET_REGISTRY.get(widget_type)
if not info: @router.get("/builtin")
raise HTTPException( def list_builtin_kinds() -> list[BuiltinWidgetKindInfo]:
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, """Return metadata for service-less built-in widget kinds."""
detail=f"Unknown widget type: {widget_type}", return [
BuiltinWidgetKindInfo(
kind=wk.kind,
name=wk.name,
description=wk.description,
config_schema=wk.config_schema,
default_config=wk.default_config,
refresh_interval_ms=wk.refresh_interval_ms,
) )
return info for wk in BUILTIN_WIDGET_KINDS.values()
]
def _validate_widget_input(body: WidgetInstanceInput) -> None:
"""Validate widget_type/addon_id match and config schema."""
info = _registry_for_type(body.widget_type)
expected_addon = info["addon_id"]
if body.addon_id != expected_addon:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=(
f"Widget type '{body.widget_type}' belongs to addon "
f"'{expected_addon}', not '{body.addon_id}'"
),
)
try:
validate_config(body.widget_type, body.config)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=str(exc),
) from exc
@router.get("/sources")
def list_sources() -> list[str]:
"""Return all registered widget source types."""
return list_source_types()
@router.get("/types")
def list_types() -> list[dict[str, Any]]:
"""Return metadata for all registered widget types."""
return [info.model_dump() for info in list_widget_types()]
@router.get("/instances") @router.get("/instances")
@@ -85,7 +115,7 @@ def create_instance(
store: SettingsStore = Depends(get_settings_store), store: SettingsStore = Depends(get_settings_store),
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Create a new widget instance.""" """Create a new widget instance."""
_validate_widget_input(body) _validate_widget_input(body, store)
widget = store.upsert_widget(body.model_dump()) widget = store.upsert_widget(body.model_dump())
return WidgetInstance(**widget).model_dump() return WidgetInstance(**widget).model_dump()
@@ -105,7 +135,7 @@ def update_instance(
status_code=status.HTTP_400_BAD_REQUEST, status_code=status.HTTP_400_BAD_REQUEST,
detail="ID in path does not match ID in body", detail="ID in path does not match ID in body",
) )
_validate_widget_input(body) _validate_widget_input(body, store)
widget = store.upsert_widget(body.model_dump(), widget_id) widget = store.upsert_widget(body.model_dump(), widget_id)
return WidgetInstance(**widget).model_dump() return WidgetInstance(**widget).model_dump()
@@ -133,31 +163,44 @@ async def fetch_data(
if not widget: if not widget:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Widget not found") raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Widget not found")
widget_type = widget["widget_type"] service_id = widget.get("service_id")
info = get_widget_info(widget_type) widget_kind = widget.get("widget_kind") or ""
if info is None:
return WidgetDataResponse(
widget_id=widget_id,
widget_type=widget_type,
data=None,
error=f"Unknown widget type: {widget_type}",
fetched_at=int(time.time()),
).model_dump()
adapter = get_source_adapter(info.source_type) service: Any = None
if adapter is None: if service_id:
# Defensive: registry should prevent this, but return a safe error. service_row = store.get_service(service_id)
return WidgetDataResponse( if not service_row:
widget_id=widget_id, return WidgetDataResponse(
widget_type=widget_type, widget_id=widget_id,
data=None, error=f"Service {service_id} not found",
error=f"No adapter registered for source type: {info.source_type}", fetched_at=int(time.time()),
fetched_at=int(time.time()), ).model_dump()
).model_dump() if not service_row.get("enabled", True):
return WidgetDataResponse(
widget_id=widget_id,
error="Service is disabled",
fetched_at=int(time.time()),
).model_dump()
adapter = get_service_adapter(service_row["service_type"])
if adapter is None:
return WidgetDataResponse(
widget_id=widget_id,
error=f"No adapter for service type {service_row['service_type']}",
fetched_at=int(time.time()),
).model_dump()
service = build_service_record(store, service_row)
else:
adapter = get_builtin_adapter(widget_kind)
if adapter is None:
return WidgetDataResponse(
widget_id=widget_id,
error=f"Unknown built-in widget kind: {widget_kind}",
fetched_at=int(time.time()),
).model_dump()
try: try:
data = await adapter.fetch(widget["config"]) data = await adapter.fetch(service, widget_kind, widget.get("config") or {})
except Exception as exc: except Exception as exc: # pragma: no cover - defensive
logger.exception("Unhandled adapter exception widget_id=%s", widget_id) logger.exception("Unhandled adapter exception widget_id=%s", widget_id)
raise HTTPException( raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
@@ -166,7 +209,6 @@ async def fetch_data(
return WidgetDataResponse( return WidgetDataResponse(
widget_id=widget_id, widget_id=widget_id,
widget_type=widget_type,
data=data if "error" not in data else None, data=data if "error" not in data else None,
error=data.get("error"), error=data.get("error"),
fetched_at=int(time.time()), fetched_at=int(time.time()),
@@ -0,0 +1,93 @@
"""Encryption-at-rest for service secrets.
Service API keys / tokens are stored encrypted in the ``services.secrets_json``
column. Encryption uses Fernet (symmetric authenticated encryption) with a single
master key provided via the ``MANAGE_ENCRYPTION_KEY`` environment variable.
* The key **must** be a urlsafe base64-encoded 32-byte value (Fernet format).
* The key is **always required** — there is no development fallback, so secrets
are never accidentally stored in plaintext.
* Secrets are encrypted field-by-field; the ``"which secrets are set"`` metadata
can be derived from the ciphertext blob without decrypting.
"""
from __future__ import annotations
import os
from functools import lru_cache
from cryptography.fernet import Fernet, InvalidToken
ENCRYPTION_KEY_ENV = "MANAGE_ENCRYPTION_KEY"
class EncryptionKeyError(RuntimeError):
"""Raised when the encryption key is missing or invalid."""
@lru_cache(maxsize=1)
def get_encryption_key() -> bytes:
"""Return the raw Fernet key, or raise if missing/invalid.
The result is cached for the process lifetime. Tests should call
:func:`reset_encryption_key_cache` after changing the environment.
"""
raw = os.environ.get(ENCRYPTION_KEY_ENV)
if not raw:
raise EncryptionKeyError(f"{ENCRYPTION_KEY_ENV} is required to store service secrets")
key = raw.strip().encode()
try:
Fernet(key)
except (ValueError, TypeError) as exc: # pragma: no cover - validated by tests
raise EncryptionKeyError(f"{ENCRYPTION_KEY_ENV} must be a valid Fernet key") from exc
return key
def reset_encryption_key_cache() -> None:
"""Drop the cached encryption key (used by tests that swap keys)."""
get_encryption_key.cache_clear()
def _fernet() -> Fernet:
return Fernet(get_encryption_key())
def encrypt_value(plaintext: str) -> str:
"""Encrypt a single secret value and return the ciphertext string."""
return _fernet().encrypt(plaintext.encode()).decode()
def decrypt_value(ciphertext: str) -> str:
"""Decrypt a single ciphertext value."""
try:
return _fernet().decrypt(ciphertext.encode()).decode()
except InvalidToken as exc:
raise EncryptionKeyError("Service secret could not be decrypted") from exc
def encrypt_secrets(values: dict[str, str]) -> dict[str, str]:
"""Encrypt every provided secret value."""
fernet = _fernet()
return {key: fernet.encrypt(value.encode()).decode() for key, value in values.items()}
def decrypt_secrets(blob: dict[str, str]) -> dict[str, str]:
"""Decrypt every secret value in a blob."""
fernet = _fernet()
result: dict[str, str] = {}
for key, ciphertext in blob.items():
try:
result[key] = fernet.decrypt(ciphertext.encode()).decode()
except InvalidToken as exc:
raise EncryptionKeyError(f"Service secret '{key}' could not be decrypted") from exc
return result
def generate_development_key() -> str:
"""Return a freshly generated Fernet key (helper for operators/docs)."""
return Fernet.generate_key().decode()
def validate_encryption_key() -> None:
"""Eagerly validate that the encryption key is present and well-formed."""
get_encryption_key() # raises EncryptionKeyError on failure
@@ -179,9 +179,12 @@ class SettingsStore:
) )
""" """
) )
conn.execute( conn.execute("CREATE INDEX IF NOT EXISTS idx_dashboard_widgets_sort ON dashboard_widgets(sort_order)")
"CREATE INDEX IF NOT EXISTS idx_dashboard_widgets_sort ON dashboard_widgets(sort_order)" widget_cols = {row[1] for row in conn.execute("PRAGMA table_info(dashboard_widgets)").fetchall()}
) if "service_id" not in widget_cols:
conn.execute("ALTER TABLE dashboard_widgets ADD COLUMN service_id TEXT")
if "widget_kind" not in widget_cols:
conn.execute("ALTER TABLE dashboard_widgets ADD COLUMN widget_kind TEXT")
conn.execute(""" conn.execute("""
CREATE TABLE IF NOT EXISTS backup_jobs ( CREATE TABLE IF NOT EXISTS backup_jobs (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
@@ -226,6 +229,44 @@ class SettingsStore:
""") """)
conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_job_id ON backup_alerts(job_id)") conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_job_id ON backup_alerts(job_id)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_acknowledged ON backup_alerts(acknowledged)") conn.execute("CREATE INDEX IF NOT EXISTS idx_backup_alerts_acknowledged ON backup_alerts(acknowledged)")
conn.execute(
"""
CREATE TABLE IF NOT EXISTS services (
id TEXT PRIMARY KEY,
service_type TEXT NOT NULL,
name TEXT NOT NULL,
config_json TEXT NOT NULL DEFAULT '{}',
secrets_json TEXT NOT NULL DEFAULT '{}',
enabled INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)
"""
)
conn.execute("CREATE INDEX IF NOT EXISTS idx_services_type ON services(service_type)")
conn.execute(
"""
CREATE TABLE IF NOT EXISTS service_task_runs (
id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
service_id TEXT NOT NULL,
status TEXT NOT NULL,
exit_status INTEGER,
duration_ms INTEGER,
stdout_tail TEXT NOT NULL DEFAULT '',
stderr_tail TEXT NOT NULL DEFAULT '',
error TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL
)
"""
)
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_service_task_runs_service "
"ON service_task_runs(service_id, created_at DESC)"
)
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_service_task_runs_task ON service_task_runs(task_id, created_at DESC)"
)
@staticmethod @staticmethod
def _normalize_services(value: Any, fallback: list[str] | None = None) -> list[str]: def _normalize_services(value: Any, fallback: list[str] | None = None) -> list[str]:
@@ -417,39 +458,13 @@ class SettingsStore:
) )
def _seed_dashboard_widgets(self) -> None: def _seed_dashboard_widgets(self) -> None:
"""Seed default dashboard widgets only when the table is empty.""" """Default widget seeding was removed.
from media_library_viewer_api.widgets.registry import WIDGET_REGISTRY
self.init_schema() Widgets are now service-bound (or built-in). A fresh install starts with
with self.connect() as conn: no widgets; the user configures services and adds widgets from the UI.
row = conn.execute("SELECT COUNT(*) FROM dashboard_widgets").fetchone() Kept as a no-op so :meth:`ensure_defaults` callers are unchanged.
if row and int(row[0]) > 0: """
return return None
defaults = [
{
"id": "jellyfin-activity-default",
"addon_id": "core",
"widget_type": "jellyfin",
"title": "Jellyfin activity",
"config": {"machine_id": ""},
"enabled": True,
"sort_order": 0,
},
{
"id": "backups-summary-default",
"addon_id": "backups",
"widget_type": "backups",
"title": "Backups",
"config": {},
"enabled": True,
"sort_order": 1,
},
]
for widget in defaults:
info = WIDGET_REGISTRY.get(widget["widget_type"])
if not info or info["addon_id"] != widget["addon_id"]:
continue
self.upsert_widget(widget)
def ensure_defaults(self) -> None: def ensure_defaults(self) -> None:
self.init_schema() self.init_schema()
@@ -457,7 +472,6 @@ class SettingsStore:
row = conn.execute("SELECT COUNT(*) FROM monitoring_machines").fetchone() row = conn.execute("SELECT COUNT(*) FROM monitoring_machines").fetchone()
if not row or int(row[0]) == 0: if not row or int(row[0]) == 0:
self._seed_local_machine() self._seed_local_machine()
self._seed_dashboard_widgets()
def list_machines(self) -> list[dict[str, Any]]: def list_machines(self) -> list[dict[str, Any]]:
self.init_schema() self.init_schema()
@@ -1363,16 +1377,18 @@ class SettingsStore:
(key, value, now), (key, value, now),
) )
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Dashboard widgets # Dashboard widgets
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def _row_to_widget(self, row: sqlite3.Row) -> dict[str, Any]: def _row_to_widget(self, row: sqlite3.Row) -> dict[str, Any]:
keys = row.keys()
return { return {
"id": row["id"], "id": row["id"],
"addon_id": row["addon_id"], "addon_id": row["addon_id"],
"widget_type": row["widget_type"], "widget_type": row["widget_type"],
"service_id": row["service_id"] if "service_id" in keys else None,
"widget_kind": row["widget_kind"] if "widget_kind" in keys else None,
"title": row["title"], "title": row["title"],
"config": json.loads(row["config_json"] or "{}"), "config": json.loads(row["config_json"] or "{}"),
"enabled": bool(row["enabled"]), "enabled": bool(row["enabled"]),
@@ -1387,14 +1403,9 @@ class SettingsStore:
widget_id: str | None = None, widget_id: str | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
current = self.get_widget(widget_id) if widget_id else None current = self.get_widget(widget_id) if widget_id else None
widget_id = ( widget_id = str(payload.get("id") or widget_id or uuid.uuid4().hex[:12]).strip() or uuid.uuid4().hex[:12]
str(payload.get("id") or widget_id or uuid.uuid4().hex[:12]).strip() service_id = str(payload.get("service_id") or (current or {}).get("service_id") or "").strip() or None
or uuid.uuid4().hex[:12] widget_kind = str(payload.get("widget_kind") or (current or {}).get("widget_kind", "")).strip()
)
addon_id = str(payload.get("addon_id") or (current or {}).get("addon_id", "")).strip()
widget_type = str(
payload.get("widget_type") or (current or {}).get("widget_type", "")
).strip()
title = str(payload.get("title") or (current or {}).get("title", "") or "").strip() title = str(payload.get("title") or (current or {}).get("title", "") or "").strip()
config = payload.get("config", (current or {}).get("config", {})) config = payload.get("config", (current or {}).get("config", {}))
if not isinstance(config, dict): if not isinstance(config, dict):
@@ -1403,10 +1414,14 @@ class SettingsStore:
_validate_config_keys(config) _validate_config_keys(config)
enabled = bool(payload.get("enabled", (current or {}).get("enabled", True))) enabled = bool(payload.get("enabled", (current or {}).get("enabled", True)))
sort_order = int(payload.get("sort_order", (current or {}).get("sort_order", 0)) or 0) sort_order = int(payload.get("sort_order", (current or {}).get("sort_order", 0)) or 0)
# Legacy label kept for diagnostics; new code uses service_id + widget_kind.
widget_type = f"{service_id}:{widget_kind}" if widget_kind else ""
return { return {
"id": widget_id, "id": widget_id,
"addon_id": addon_id, "addon_id": "",
"widget_type": widget_type, "widget_type": widget_type,
"service_id": service_id,
"widget_kind": widget_kind,
"title": title, "title": title,
"config": config, "config": config,
"enabled": enabled, "enabled": enabled,
@@ -1416,9 +1431,7 @@ class SettingsStore:
def list_widgets(self) -> list[dict[str, Any]]: def list_widgets(self) -> list[dict[str, Any]]:
self.init_schema() self.init_schema()
with self.connect() as conn: with self.connect() as conn:
rows = conn.execute( rows = conn.execute("SELECT * FROM dashboard_widgets ORDER BY sort_order ASC, created_at ASC").fetchall()
"SELECT * FROM dashboard_widgets ORDER BY sort_order ASC, created_at ASC"
).fetchall()
return [self._row_to_widget(row) for row in rows] return [self._row_to_widget(row) for row in rows]
def get_widget(self, widget_id: str | None) -> dict[str, Any] | None: def get_widget(self, widget_id: str | None) -> dict[str, Any] | None:
@@ -1426,9 +1439,7 @@ class SettingsStore:
return None return None
self.init_schema() self.init_schema()
with self.connect() as conn: with self.connect() as conn:
row = conn.execute( row = conn.execute("SELECT * FROM dashboard_widgets WHERE id = ?", (widget_id,)).fetchone()
"SELECT * FROM dashboard_widgets WHERE id = ?", (widget_id,)
).fetchone()
return self._row_to_widget(row) if row else None return self._row_to_widget(row) if row else None
def upsert_widget(self, payload: dict[str, Any], widget_id: str | None = None) -> dict[str, Any]: def upsert_widget(self, payload: dict[str, Any], widget_id: str | None = None) -> dict[str, Any]:
@@ -1444,13 +1455,15 @@ class SettingsStore:
conn.execute( conn.execute(
""" """
INSERT INTO dashboard_widgets ( INSERT INTO dashboard_widgets (
id, addon_id, widget_type, title, config_json, enabled, id, addon_id, widget_type, service_id, widget_kind, title,
sort_order, created_at, updated_at config_json, enabled, sort_order, created_at, updated_at
) )
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET ON CONFLICT(id) DO UPDATE SET
addon_id = excluded.addon_id, addon_id = excluded.addon_id,
widget_type = excluded.widget_type, widget_type = excluded.widget_type,
service_id = excluded.service_id,
widget_kind = excluded.widget_kind,
title = excluded.title, title = excluded.title,
config_json = excluded.config_json, config_json = excluded.config_json,
enabled = excluded.enabled, enabled = excluded.enabled,
@@ -1461,6 +1474,8 @@ class SettingsStore:
widget["id"], widget["id"],
widget["addon_id"], widget["addon_id"],
widget["widget_type"], widget["widget_type"],
widget["service_id"],
widget["widget_kind"],
widget["title"], widget["title"],
json.dumps(widget["config"]), json.dumps(widget["config"]),
1 if widget["enabled"] else 0, 1 if widget["enabled"] else 0,
@@ -1476,6 +1491,204 @@ class SettingsStore:
with self.connect() as conn: with self.connect() as conn:
conn.execute("DELETE FROM dashboard_widgets WHERE id = ?", (widget_id,)) conn.execute("DELETE FROM dashboard_widgets WHERE id = ?", (widget_id,))
# ------------------------------------------------------------------
# Service registry
# ------------------------------------------------------------------
def _row_to_service(self, row: sqlite3.Row) -> dict[str, Any]:
secrets_blob = json.loads(row["secrets_json"] or "{}")
return {
"id": row["id"],
"service_type": row["service_type"],
"name": row["name"],
"config": json.loads(row["config_json"] or "{}"),
"secrets": secrets_blob,
"enabled": bool(row["enabled"]),
"created_at": row["created_at"],
"updated_at": row["updated_at"],
}
def list_services(self, service_type: str | None = None) -> list[dict[str, Any]]:
self.init_schema()
with self.connect() as conn:
if service_type:
rows = conn.execute(
"SELECT * FROM services WHERE service_type = ? ORDER BY name ASC",
(service_type,),
).fetchall()
else:
rows = conn.execute("SELECT * FROM services ORDER BY name ASC").fetchall()
return [self._row_to_service(row) for row in rows]
def get_service(self, service_id: str) -> dict[str, Any] | None:
self.init_schema()
with self.connect() as conn:
row = conn.execute("SELECT * FROM services WHERE id = ?", (service_id,)).fetchone()
return self._row_to_service(row) if row else None
def _normalize_service_payload(
self,
payload: dict[str, Any],
service_id: str | None = None,
) -> dict[str, Any]:
current = self.get_service(service_id) if service_id else None
service_id = str(payload.get("id") or service_id or uuid.uuid4().hex[:12]).strip() or uuid.uuid4().hex[:12]
service_type = str(payload.get("service_type") or (current or {}).get("service_type", "")).strip()
name = str(payload.get("name") or (current or {}).get("name", "") or "").strip()
config = payload.get("config", (current or {}).get("config", {}))
if not isinstance(config, dict):
config = {}
enabled = bool(payload.get("enabled", (current or {}).get("enabled", True)))
return {
"id": service_id,
"service_type": service_type,
"name": name,
"config": config,
"enabled": enabled,
}
def upsert_service(
self,
payload: dict[str, Any],
secret_values: dict[str, str] | None = None,
service_id: str | None = None,
) -> dict[str, Any]:
"""Insert or update a service instance.
``secret_values`` carries plaintext secrets to encrypt and store. A key
absent from ``secret_values`` preserves the existing ciphertext; a key
mapped to an empty string clears it.
"""
self.init_schema()
service = self._normalize_service_payload(payload, service_id)
now = int(time.time())
existing = self.get_service(service["id"])
secrets_blob: dict[str, str]
if existing is not None:
secrets_blob = dict(existing["secrets"])
else:
secrets_blob = {}
if secret_values:
from media_library_viewer_api.services.secrets import encrypt_value
for key, value in secret_values.items():
if value == "":
secrets_blob.pop(key, None)
else:
secrets_blob[key] = encrypt_value(value)
with self.connect() as conn:
created_at = int(existing["created_at"]) if existing else now
conn.execute(
"""
INSERT INTO services (
id, service_type, name, config_json, secrets_json,
enabled, created_at, updated_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
service_type = excluded.service_type,
name = excluded.name,
config_json = excluded.config_json,
secrets_json = excluded.secrets_json,
enabled = excluded.enabled,
updated_at = excluded.updated_at
""",
(
service["id"],
service["service_type"],
service["name"],
json.dumps(service["config"]),
json.dumps(secrets_blob),
1 if service["enabled"] else 0,
created_at,
now,
),
)
return self.get_service(service["id"]) or service
def delete_service(self, service_id: str) -> None:
"""Delete a service and cascade-delete widgets referencing it."""
self.init_schema()
with self.connect() as conn:
# The service_id column on dashboard_widgets is added in a later
# slice; only cascade when it is present.
widget_cols = {row[1] for row in conn.execute("PRAGMA table_info(dashboard_widgets)").fetchall()}
if "service_id" in widget_cols:
conn.execute(
"DELETE FROM dashboard_widgets WHERE service_id = ?",
(service_id,),
)
conn.execute("DELETE FROM services WHERE id = ?", (service_id,))
def record_service_task_run(self, payload: dict[str, Any]) -> dict[str, Any]:
"""Append a service task run history row."""
self.init_schema()
run_id = str(payload.get("id") or uuid.uuid4().hex[:12])
now = int(time.time())
with self.connect() as conn:
conn.execute(
"""
INSERT INTO service_task_runs (
id, task_id, service_id, status, exit_status, duration_ms,
stdout_tail, stderr_tail, error, created_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
run_id,
str(payload.get("task_id") or ""),
str(payload.get("service_id") or ""),
str(payload.get("status") or "error"),
payload.get("exit_status"),
payload.get("duration_ms"),
str(payload.get("stdout_tail") or "")[:8000],
str(payload.get("stderr_tail") or "")[:8000],
str(payload.get("error") or "")[:1000],
int(payload.get("created_at") or now),
),
)
return {"id": run_id}
def list_service_task_runs(
self,
service_id: str | None = None,
task_id: str | None = None,
limit: int = 50,
) -> list[dict[str, Any]]:
self.init_schema()
clauses: list[str] = []
params: list[Any] = []
if service_id:
clauses.append("service_id = ?")
params.append(service_id)
if task_id:
clauses.append("task_id = ?")
params.append(task_id)
where = ("WHERE " + " AND ".join(clauses)) if clauses else ""
params.append(int(limit))
with self.connect() as conn:
rows = conn.execute(
f"SELECT * FROM service_task_runs {where} ORDER BY created_at DESC LIMIT ?",
params,
).fetchall()
return [
{
"id": row["id"],
"task_id": row["task_id"],
"service_id": row["service_id"],
"status": row["status"],
"exit_status": row["exit_status"],
"duration_ms": row["duration_ms"],
"stdout_tail": row["stdout_tail"],
"stderr_tail": row["stderr_tail"],
"error": row["error"],
"created_at": row["created_at"],
}
for row in rows
]
_store: SettingsStore | None = None _store: SettingsStore | None = None
@@ -0,0 +1,68 @@
"""Built-in, service-less widget kinds.
These widgets do not talk to an external service and therefore have no
``service_id``. They are kept out of the service registry (which models
configurable external services) and live here as a small closed set.
Currently: ``backups`` (reads the internal backup tables) and ``static``
(plain text/markdown).
"""
from __future__ import annotations
from typing import Any
from media_library_viewer_api.integrations.base import WidgetKind
BUILTIN_WIDGET_KINDS: dict[str, WidgetKind] = {
"backups": WidgetKind(
kind="backups",
name="Backups",
description="Backup job summary and active alerts.",
config_schema={"type": "object", "properties": {}, "required": []},
default_config={},
refresh_interval_ms=60_000,
),
"static": WidgetKind(
kind="static",
name="Static text",
description="Plain text or markdown note.",
config_schema={
"type": "object",
"properties": {"text": {"type": "string", "description": "Text or markdown content"}},
"required": ["text"],
},
default_config={"text": ""},
refresh_interval_ms=0,
),
}
def get_builtin_widget_kind(kind: str) -> WidgetKind | None:
return BUILTIN_WIDGET_KINDS.get(kind)
def is_builtin_kind(kind: str) -> bool:
return kind in BUILTIN_WIDGET_KINDS
def builtin_widget_kind_models() -> dict[str, type]:
"""Pydantic widget-config models for built-in kinds (validated manually).
Backups has no user fields; static validates ``text``.
"""
from pydantic import BaseModel, Field
class StaticConfig(BaseModel):
text: str = Field(default="")
return {"static": StaticConfig}
def validate_builtin_config(kind: str, config: dict[str, Any]) -> dict[str, Any]:
"""Validate (lightly) a built-in widget config and return the cleaned dict."""
models = builtin_widget_kind_models()
model_cls = models.get(kind)
if model_cls is None:
return dict(config or {})
return model_cls.model_validate(config or {}).model_dump(exclude_none=True)
@@ -1,187 +0,0 @@
"""Closed, compile-time widget registry.
New widget types and source adapters require a code change in Phase 1.
There is no runtime plugin loading.
"""
from typing import Any
from media_library_viewer_api.models.widgets import WidgetTypeInfo
WIDGET_REGISTRY: dict[str, dict[str, Any]] = {
"jellyfin": {
"addon_id": "core",
"name": "Jellyfin activity",
"description": "Live sessions and idle users from a Jellyfin server.",
"source_type": "jellyfin",
"config_schema": {
"type": "object",
"properties": {
"machine_id": {
"type": "string",
"description": "Jellyfin machine id (empty = default)",
},
},
"required": ["machine_id"],
},
},
"backups": {
"addon_id": "backups",
"name": "Backups",
"description": "Backup job summary and active alerts.",
"source_type": "backups",
"config_schema": {
"type": "object",
"properties": {},
"required": [],
},
},
"grafana-link": {
"addon_id": "grafana",
"name": "Grafana link",
"description": "Deep-link to a Grafana dashboard or panel.",
"source_type": "grafana",
"config_schema": {
"type": "object",
"properties": {
"dashboard_uid": {
"type": "string",
"description": "Grafana dashboard UID",
},
"panel_id": {
"type": "integer",
"description": "Optional panel id",
},
},
"required": ["dashboard_uid"],
},
},
"prometheus-metric": {
"addon_id": "prometheus",
"name": "Prometheus metric",
"description": "Instant query result rendered as a metric.",
"source_type": "prometheus",
"config_schema": {
"type": "object",
"properties": {
"promql": {
"type": "string",
"description": "PromQL instant query",
},
},
"required": ["promql"],
},
},
"ssh-task": {
"addon_id": "ssh-tasks",
"name": "SSH task output",
"description": "Output of a saved task run on a machine.",
"source_type": "ssh_task",
"config_schema": {
"type": "object",
"properties": {
"task_id": {
"type": "string",
"description": "Saved task id",
},
},
"required": ["task_id"],
},
},
"static": {
"addon_id": "core",
"name": "Static text",
"description": "Plain text or markdown note.",
"source_type": "static",
"config_schema": {
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "Text or markdown content",
},
},
"required": ["text"],
},
},
}
def list_source_types() -> list[str]:
"""Return all registered source type names."""
return sorted({info["source_type"] for info in WIDGET_REGISTRY.values()})
def list_widget_types() -> list[WidgetTypeInfo]:
"""Return metadata for all registered widget types."""
return [
WidgetTypeInfo(
addon_id=info["addon_id"],
widget_type=widget_type,
name=info["name"],
description=info["description"],
source_type=info["source_type"],
config_schema=info["config_schema"],
)
for widget_type, info in WIDGET_REGISTRY.items()
]
def get_widget_info(widget_type: str) -> WidgetTypeInfo | None:
"""Return metadata for a single widget type, or None if unknown."""
info = WIDGET_REGISTRY.get(widget_type)
if not info:
return None
return WidgetTypeInfo(
addon_id=info["addon_id"],
widget_type=widget_type,
name=info["name"],
description=info["description"],
source_type=info["source_type"],
config_schema=info["config_schema"],
)
def _validate_type(value: Any, expected: str) -> bool:
if expected == "string":
return isinstance(value, str)
if expected == "integer":
return isinstance(value, int) and not isinstance(value, bool)
if expected == "boolean":
return isinstance(value, bool)
if expected == "number":
return isinstance(value, (int, float)) and not isinstance(value, bool)
if expected == "object":
return isinstance(value, dict)
if expected == "array":
return isinstance(value, list)
return True
def validate_config(widget_type: str, config: dict[str, Any]) -> None:
"""Validate a widget config against its registered JSON schema.
Raises ValueError with a descriptive message if validation fails.
Phase 1 supports only required-field and primitive-type checks.
"""
info = WIDGET_REGISTRY.get(widget_type)
if not info:
raise ValueError(f"Unknown widget type: {widget_type}")
schema = info["config_schema"]
required = schema.get("required", [])
properties = schema.get("properties", {})
for key in required:
if key not in config:
raise ValueError(f"Missing required config field: {key}")
for key, value in config.items():
prop = properties.get(key)
if not prop:
# Unknown keys are allowed in Phase 1 unless they look like secrets
# (handled by the model validator). Skip type checks for unknowns.
continue
expected_type = prop.get("type")
if expected_type and not _validate_type(value, expected_type):
raise ValueError(f"Config field '{key}' must be of type {expected_type}")
@@ -1,9 +1,11 @@
"""Widget source adapters. """Widget source adapters.
Each adapter implements a uniform async interface and translates widget Adapters translate a widget instance into dashboard data. Service-bound widgets
configuration into data for the dashboard. Adapters reuse existing clients, are resolved against a :class:`ServiceRecord` (config + decrypted secrets); the
machine registries, and environment settings; they never accept arbitrary built-in widgets (backups, static) take ``service=None``.
commands or store credentials.
Adapters never accept arbitrary commands and never store credentials — secrets
are decrypted in memory only for the duration of a fetch.
""" """
from __future__ import annotations from __future__ import annotations
@@ -11,94 +13,103 @@ from __future__ import annotations
import asyncio import asyncio
import logging import logging
import shlex import shlex
import time
from dataclasses import dataclass, field
from typing import Any, Protocol from typing import Any, Protocol
import requests import requests
from starlette.requests import Request
from media_library_viewer_api.clients.jellyfin import JellyfinClient
from media_library_viewer_api.clients.ssh import RemoteSSHClient
from media_library_viewer_api.config import get_settings from media_library_viewer_api.config import get_settings
from media_library_viewer_api.dependencies import get_jellyfin_client
from media_library_viewer_api.domain.dashboard import ( from media_library_viewer_api.domain.dashboard import (
_map_sessions_to_activity_rows, _map_sessions_to_activity_rows,
build_backup_dashboard_summary, build_backup_dashboard_summary,
) )
from media_library_viewer_api.routers.tasks import _client_for_machine, _resolve_machine_for_task from media_library_viewer_api.services.settings_store import SettingsStore, get_settings_store
from media_library_viewer_api.services.settings_store import get_settings_store
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _request_with_machine_id(machine_id: str | None = None) -> Request: @dataclass
"""Build a minimal Starlette Request carrying a machine_id query param.""" class ServiceRecord:
query = f"machine_id={machine_id}".encode() if machine_id else b"" """Runtime view of a service instance with decrypted secrets."""
return Request({"type": "http", "query_string": query})
id: str
service_type: str
name: str
config: dict[str, Any] = field(default_factory=dict)
secrets: dict[str, str] = field(default_factory=dict)
enabled: bool = True
def build_service_record(store: SettingsStore, service_row: dict[str, Any]) -> ServiceRecord:
"""Build a :class:`ServiceRecord`, decrypting secrets in memory."""
from media_library_viewer_api.services.secrets import decrypt_secrets
return ServiceRecord(
id=service_row["id"],
service_type=service_row["service_type"],
name=service_row["name"],
config=service_row.get("config") or {},
secrets=decrypt_secrets(service_row.get("secrets") or {}),
enabled=bool(service_row.get("enabled", True)),
)
class WidgetSource(Protocol): class WidgetSource(Protocol):
"""Protocol for widget source adapters.""" """Protocol for widget source adapters."""
source_type: str async def fetch(
self,
async def fetch(self, config: dict[str, Any]) -> dict[str, Any]: ... service: ServiceRecord | None,
widget_kind: str,
config: dict[str, Any],
) -> dict[str, Any]: ...
class JellyfinWidgetSource: # ---------------------------------------------------------------------------
"""Fetch Jellyfin sessions and map them to activity rows.""" # Built-in (service-less) adapters
# ---------------------------------------------------------------------------
source_type = "jellyfin"
timeout = 10
async def fetch(self, config: dict[str, Any]) -> dict[str, Any]:
try:
request = _request_with_machine_id(config.get("machine_id") or None)
client = await asyncio.wait_for(
asyncio.to_thread(get_jellyfin_client, request),
timeout=self.timeout,
)
sessions = await asyncio.wait_for(
asyncio.to_thread(client.sessions),
timeout=self.timeout,
)
rows = _map_sessions_to_activity_rows(sessions)
return {"sessions": rows}
except asyncio.TimeoutError:
return {"error": "Widget data fetch timed out"}
except Exception as exc:
logger.exception("jellyfin adapter failed")
return {"error": f"Jellyfin data fetch failed: {exc}"}
class BackupsWidgetSource: class BackupsWidgetSource:
"""Compute the backup dashboard summary.""" """Compute the backup dashboard summary from internal tables."""
source_type = "backups" async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
timeout = 10
async def fetch(self, config: dict[str, Any]) -> dict[str, Any]:
try: try:
store = get_settings_store() store = get_settings_store()
summary = build_backup_dashboard_summary(store) summary = build_backup_dashboard_summary(store)
return summary.model_dump() return summary.model_dump()
except asyncio.TimeoutError:
return {"error": "Widget data fetch timed out"}
except Exception as exc: except Exception as exc:
logger.exception("backups adapter failed") logger.exception("backups adapter failed")
return {"error": f"Backup summary failed: {exc}"} return {"error": f"Backup summary failed: {exc}"}
class StaticWidgetSource:
"""Return static text/markdown unchanged."""
async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
return {"text": config.get("text", "")}
# ---------------------------------------------------------------------------
# Service-bound adapters
# ---------------------------------------------------------------------------
class GrafanaWidgetSource: class GrafanaWidgetSource:
"""Build a Grafana deep-link (no embedding).""" """Build a Grafana deep-link (no embedding)."""
source_type = "grafana" async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
timeout = 5
async def fetch(self, config: dict[str, Any]) -> dict[str, Any]:
try: try:
settings = get_settings() if service is None:
return {"error": "Grafana widget is missing its service"}
base_url = str(service.config.get("base_url") or "").rstrip("/")
dashboard_uid = config.get("dashboard_uid") dashboard_uid = config.get("dashboard_uid")
if not dashboard_uid: if not dashboard_uid:
return {"error": "dashboard_uid is required"} return {"error": "dashboard_uid is required"}
url = f"{settings.grafana_url.rstrip('/')}/d/{dashboard_uid}" url = f"{base_url}/d/{dashboard_uid}"
panel_id = config.get("panel_id") panel_id = config.get("panel_id")
if panel_id is not None: if panel_id is not None:
url = f"{url}?viewPanel={panel_id}" url = f"{url}?viewPanel={panel_id}"
@@ -109,26 +120,26 @@ class GrafanaWidgetSource:
class PrometheusWidgetSource: class PrometheusWidgetSource:
"""Run a PromQL instant query against Prometheus.""" """Run a PromQL instant query against a Prometheus service."""
source_type = "prometheus" async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
timeout = 10
async def fetch(self, config: dict[str, Any]) -> dict[str, Any]:
try: try:
settings = get_settings() if service is None:
return {"error": "Prometheus widget is missing its service"}
base_url = str(service.config.get("base_url") or "").rstrip("/")
timeout = int(service.config.get("timeout_seconds") or 10)
promql = config.get("promql") promql = config.get("promql")
if not promql: if not promql:
return {"error": "promql is required"} return {"error": "promql is required"}
url = f"{settings.prometheus_url.rstrip('/')}/api/v1/query" url = f"{base_url}/api/v1/query"
response = await asyncio.wait_for( response = await asyncio.wait_for(
asyncio.to_thread( asyncio.to_thread(
requests.get, requests.get,
url, url,
params={"query": promql}, params={"query": promql},
timeout=self.timeout, timeout=timeout,
), ),
timeout=self.timeout, timeout=timeout,
) )
response.raise_for_status() response.raise_for_status()
payload = response.json() payload = response.json()
@@ -143,16 +154,44 @@ class PrometheusWidgetSource:
return {"error": f"Prometheus query failed: {exc}"} return {"error": f"Prometheus query failed: {exc}"}
class SshTaskWidgetSource: class JellyfinWidgetSource:
"""Run a saved task from the registry and return its output.""" """Fetch Jellyfin sessions and map them to activity rows."""
source_type = "ssh_task" async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
timeout = 30 timeout = 10
async def fetch(self, config: dict[str, Any]) -> dict[str, Any]:
try: try:
if service is None:
return {"error": "Jellyfin widget is missing its service"}
base_url = str(service.config.get("base_url") or "")
api_key = str(service.secrets.get("api_key") or "")
timeout = int(service.config.get("timeout_seconds") or 10)
client = await asyncio.wait_for(
asyncio.to_thread(JellyfinClient, base_url, api_key, timeout),
timeout=timeout,
)
sessions = await asyncio.wait_for(
asyncio.to_thread(client.sessions),
timeout=timeout,
)
rows = _map_sessions_to_activity_rows(sessions)
return {"sessions": rows}
except asyncio.TimeoutError:
return {"error": "Widget data fetch timed out"}
except Exception as exc:
logger.exception("jellyfin adapter failed")
return {"error": f"Jellyfin data fetch failed: {exc}"}
class SshTaskWidgetSource:
"""Run a saved task on an SSH task runner instance and log the run."""
async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
timeout = 30
try:
if service is None:
return {"error": "SSH task widget is missing its service"}
store = get_settings_store() store = get_settings_store()
task_id = config.get("task_id") task_id = config.get("task_id") or ""
if not task_id: if not task_id:
return {"error": "task_id is required"} return {"error": "task_id is required"}
task = store.get_task(task_id) task = store.get_task(task_id)
@@ -161,11 +200,8 @@ class SshTaskWidgetSource:
if not task.get("enabled", True): if not task.get("enabled", True):
return {"error": "Task is disabled"} return {"error": "Task is disabled"}
machine = _resolve_machine_for_task(store, task, None) client = _build_ssh_client(store, service)
if not machine: timeout = int(service.config.get("timeout_seconds") or 30)
return {"error": "No machine available for this task"}
client = _client_for_machine(store, machine)
task_type = str(task.get("task_type") or "shell").lower() task_type = str(task.get("task_type") or "shell").lower()
command = str(task.get("content") or "") command = str(task.get("content") or "")
if task_type == "python": if task_type == "python":
@@ -173,41 +209,112 @@ class SshTaskWidgetSource:
elif task_type != "shell": elif task_type != "shell":
return {"error": f"Unknown task type: {task_type}"} return {"error": f"Unknown task type: {task_type}"}
start = time.perf_counter()
result = await asyncio.wait_for( result = await asyncio.wait_for(
asyncio.to_thread(client.run, command, timeout=self.timeout), asyncio.to_thread(client.run, command, timeout),
timeout=self.timeout, timeout=timeout,
) )
return { duration_ms = int((time.perf_counter() - start) * 1000)
"exit_status": result.exit_status, stdout = result.stdout or ""
"stdout": result.stdout or "", stderr = result.stderr or ""
"stderr": result.stderr or "", store.record_service_task_run(
} {
"task_id": task_id,
"service_id": service.id,
"status": "success" if result.exit_status == 0 else "failure",
"exit_status": result.exit_status,
"duration_ms": duration_ms,
"stdout_tail": stdout,
"stderr_tail": stderr,
"error": "" if result.exit_status == 0 else (stderr or stdout or "Task failed"),
}
)
return {"exit_status": result.exit_status, "stdout": stdout, "stderr": stderr}
except asyncio.TimeoutError: except asyncio.TimeoutError:
_record_timeout(service, config, timeout)
return {"error": "Widget data fetch timed out"} return {"error": "Widget data fetch timed out"}
except Exception as exc: except Exception as exc:
logger.exception("ssh_task adapter failed") logger.exception("ssh_task adapter failed")
store = get_settings_store()
store.record_service_task_run(
{
"task_id": str(config.get("task_id") or ""),
"service_id": service.id if service else "",
"status": "error",
"duration_ms": 0,
"error": str(exc)[:1000],
}
)
return {"error": f"SSH task failed: {exc}"} return {"error": f"SSH task failed: {exc}"}
class StaticWidgetSource: def _record_timeout(service: ServiceRecord | None, config: dict[str, Any], timeout: int) -> None:
"""Return static text/markdown unchanged.""" try:
store = get_settings_store()
source_type = "static" store.record_service_task_run(
{
async def fetch(self, config: dict[str, Any]) -> dict[str, Any]: "task_id": str(config.get("task_id") or ""),
return {"text": config.get("text", "")} "service_id": service.id if service else "",
"status": "timeout",
"duration_ms": timeout * 1000,
"error": f"Task timed out after {timeout}s",
}
)
except Exception: # pragma: no cover - logging best-effort
logger.exception("failed to record ssh task timeout")
SOURCE_REGISTRY: dict[str, WidgetSource] = { def _build_ssh_client(store: SettingsStore, service: ServiceRecord) -> RemoteSSHClient:
"jellyfin": JellyfinWidgetSource(), """Build an SSH client from an ssh_tasks service instance + referenced key."""
"backups": BackupsWidgetSource(), config = service.config
host = str(config.get("host") or "").strip()
username = str(config.get("username") or "").strip()
if not host or not username:
raise ValueError("SSH task service is missing host or username")
settings = get_settings()
private_key = ""
key_passphrase = ""
ssh_key_id = str(config.get("ssh_key_id") or "").strip()
if ssh_key_id:
ssh_key = store.get_ssh_key(ssh_key_id)
if ssh_key:
private_key = str(ssh_key.get("private_key") or "")
key_passphrase = str(ssh_key.get("passphrase") or "")
# Service-level passphrase secret takes precedence.
key_passphrase = str(service.secrets.get("passphrase") or "") or key_passphrase
return RemoteSSHClient(
host=host,
username=username,
port=int(config.get("port") or 22),
private_key=private_key or None,
private_key_passphrase=key_passphrase or None,
known_hosts_path=str(settings.ssh_known_hosts_file),
timeout=int(config.get("timeout_seconds") or 30),
)
# ---------------------------------------------------------------------------
# Registries
# ---------------------------------------------------------------------------
SERVICE_ADAPTERS: dict[str, WidgetSource] = {
"grafana": GrafanaWidgetSource(), "grafana": GrafanaWidgetSource(),
"prometheus": PrometheusWidgetSource(), "prometheus": PrometheusWidgetSource(),
"ssh_task": SshTaskWidgetSource(), "jellyfin": JellyfinWidgetSource(),
"ssh_tasks": SshTaskWidgetSource(),
}
BUILTIN_ADAPTERS: dict[str, WidgetSource] = {
"backups": BackupsWidgetSource(),
"static": StaticWidgetSource(), "static": StaticWidgetSource(),
} }
def get_source_adapter(source_type: str) -> WidgetSource | None: def get_service_adapter(service_type: str) -> WidgetSource | None:
"""Return the adapter for a source type, or None if unknown.""" return SERVICE_ADAPTERS.get(service_type)
return SOURCE_REGISTRY.get(source_type)
def get_builtin_adapter(kind: str) -> WidgetSource | None:
return BUILTIN_ADAPTERS.get(kind)
+360
View File
@@ -0,0 +1,360 @@
"""Tests for the service registry: definitions, encryption, CRUD, cascade delete."""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import patch
import pytest
from cryptography.fernet import Fernet
from fastapi.testclient import TestClient
from media_library_viewer_api.dependencies import get_settings_store
from media_library_viewer_api.integrations.registry import (
SERVICE_DEFINITIONS,
get_service_definition,
get_widget_kind,
)
from media_library_viewer_api.main import app
from media_library_viewer_api.services.secrets import (
EncryptionKeyError,
decrypt_secrets,
decrypt_value,
encrypt_secrets,
encrypt_value,
get_encryption_key,
reset_encryption_key_cache,
)
from media_library_viewer_api.services.settings_store import SettingsStore
TEST_KEY = Fernet.generate_key().decode()
@pytest.fixture(autouse=True)
def _encryption_key(monkeypatch):
"""Provide a stable MANAGE_ENCRYPTION_KEY for every test."""
monkeypatch.setenv("MANAGE_ENCRYPTION_KEY", TEST_KEY)
reset_encryption_key_cache()
yield
reset_encryption_key_cache()
@pytest.fixture
def client(tmp_path):
"""FastAPI test client with a fresh settings store and auth disabled."""
store = SettingsStore(tmp_path / "settings.sqlite")
store.ensure_defaults()
app.dependency_overrides[get_settings_store] = lambda: store
auth_settings = SimpleNamespace(auth_enabled=False)
with patch("media_library_viewer_api.auth.get_settings", return_value=auth_settings):
yield TestClient(app)
app.dependency_overrides.clear()
# ---------------------------------------------------------------------------
# Registry
# ---------------------------------------------------------------------------
def test_registry_contains_five_service_types():
assert set(SERVICE_DEFINITIONS) == {
"grafana",
"prometheus",
"jellyfin",
"nextcloud",
"ssh_tasks",
}
def test_definitions_declare_widget_kinds():
assert {wk.kind for wk in get_service_definition("grafana").widget_kinds} == {"link"}
assert {wk.kind for wk in get_service_definition("prometheus").widget_kinds} == {"metric"}
assert {wk.kind for wk in get_service_definition("jellyfin").widget_kinds} == {"activity"}
assert get_service_definition("nextcloud").widget_kinds == []
assert {wk.kind for wk in get_service_definition("ssh_tasks").widget_kinds} == {"task_output"}
def test_widget_kind_lookup():
assert get_widget_kind("grafana", "link") is not None
assert get_widget_kind("grafana", "missing") is None
assert get_widget_kind("unknown", "link") is None
def test_service_config_schema_is_json_schema():
schema = get_service_definition("grafana").config_schema
assert schema["type"] == "object"
assert "base_url" in schema["properties"]
# ---------------------------------------------------------------------------
# Encryption
# ---------------------------------------------------------------------------
def test_encrypt_decrypt_round_trip():
cipher = encrypt_value("hunter2")
assert cipher != "hunter2"
assert decrypt_value(cipher) == "hunter2"
def test_encrypt_decrypt_secrets_dict():
blob = encrypt_secrets({"api_key": "abc", "token": "xyz"})
assert decrypt_secrets(blob) == {"api_key": "abc", "token": "xyz"}
def test_missing_encryption_key_raises(monkeypatch):
monkeypatch.delenv("MANAGE_ENCRYPTION_KEY", raising=False)
reset_encryption_key_cache()
with pytest.raises(EncryptionKeyError):
get_encryption_key()
reset_encryption_key_cache()
def test_decrypt_with_wrong_key_raises(monkeypatch):
blob = encrypt_secrets({"api_key": "abc"})
monkeypatch.setenv("MANAGE_ENCRYPTION_KEY", Fernet.generate_key().decode())
reset_encryption_key_cache()
with pytest.raises(EncryptionKeyError):
decrypt_secrets(blob)
reset_encryption_key_cache()
def test_invalid_ciphertext_raises():
with pytest.raises(EncryptionKeyError):
decrypt_value("not-a-real-token")
# ---------------------------------------------------------------------------
# Service type metadata endpoint
# ---------------------------------------------------------------------------
def test_list_service_types(client):
response = client.get("/api/services/types")
assert response.status_code == 200
types = {item["service_type"] for item in response.json()}
assert types == {"grafana", "prometheus", "jellyfin", "nextcloud", "ssh_tasks"}
def test_service_type_includes_secret_and_widget_metadata(client):
response = client.get("/api/services/types")
grafana = next(item for item in response.json() if item["service_type"] == "grafana")
assert [sf["key"] for sf in grafana["secret_fields"]] == ["api_key"]
assert [wk["kind"] for wk in grafana["widget_kinds"]] == ["link"]
# ---------------------------------------------------------------------------
# CRUD
# ---------------------------------------------------------------------------
def _grafana_payload(**overrides):
payload = {
"service_type": "grafana",
"name": "Production Grafana",
"config": {"base_url": "https://grafana.example.com"},
"secrets": {"api_key": "secret-token"},
"enabled": True,
}
payload.update(overrides)
return payload
def test_create_and_list_service(client):
response = client.post("/api/services/instances", json=_grafana_payload())
assert response.status_code == 201
created = response.json()
assert created["service_type"] == "grafana"
assert created["config"]["base_url"] == "https://grafana.example.com"
# Plaintext secrets are never returned.
assert "secrets" not in created
assert created["secrets_set"] == {"api_key": True}
response = client.get("/api/services/instances")
assert response.status_code == 200
assert len(response.json()) == 1
def test_list_instances_filters_by_type(client):
client.post("/api/services/instances", json=_grafana_payload())
client.post(
"/api/services/instances",
json={
"service_type": "prometheus",
"name": "Prom",
"config": {"base_url": "http://prometheus:9090"},
},
)
response = client.get("/api/services/instances?service_type=grafana")
assert response.status_code == 200
assert len(response.json()) == 1
assert response.json()[0]["service_type"] == "grafana"
def test_update_service_preserves_unsent_secrets(client):
created = client.post("/api/services/instances", json=_grafana_payload()).json()
# Update without sending secrets; the existing key should remain set.
updated = client.put(
f"/api/services/instances/{created['id']}",
json={
"service_type": "grafana",
"name": "Renamed Grafana",
"config": {"base_url": "https://grafana.example.com", "timeout_seconds": 10},
},
).json()
assert updated["name"] == "Renamed Grafana"
assert updated["secrets_set"] == {"api_key": True}
def test_update_service_can_clear_secret(client):
created = client.post("/api/services/instances", json=_grafana_payload()).json()
updated = client.put(
f"/api/services/instances/{created['id']}",
json={
"service_type": "grafana",
"name": "Production Grafana",
"config": {"base_url": "https://grafana.example.com"},
"secrets": {"api_key": ""},
},
).json()
assert updated["secrets_set"] == {"api_key": False}
def test_unknown_service_type_rejected(client):
response = client.post(
"/api/services/instances",
json={"service_type": "bogus", "name": "x", "config": {}},
)
assert response.status_code == 422
def test_invalid_config_rejected(client):
response = client.post(
"/api/services/instances",
json={"service_type": "grafana", "name": "x", "config": {"base_url": ""}},
)
# Pydantic accepts empty string; force a real validation error via bad type.
response = client.post(
"/api/services/instances",
json={"service_type": "grafana", "name": "x", "config": {"timeout_seconds": "fast"}},
)
assert response.status_code == 422
def test_unknown_secret_field_rejected(client):
response = client.post(
"/api/services/instances",
json={
"service_type": "grafana",
"name": "x",
"config": {"base_url": "https://grafana.example.com"},
"secrets": {"password": "leak"},
},
)
assert response.status_code == 422
def test_credential_key_in_config_rejected(client):
response = client.post(
"/api/services/instances",
json={
"service_type": "grafana",
"name": "x",
"config": {"base_url": "https://grafana.example.com", "api_key": "leak"},
},
)
assert response.status_code == 422
def test_update_nonexistent_returns_404(client):
response = client.put(
"/api/services/instances/missing",
json=_grafana_payload(id="missing"),
)
assert response.status_code == 404
def test_update_id_mismatch_returns_400(client):
created = client.post("/api/services/instances", json=_grafana_payload()).json()
response = client.put(
f"/api/services/instances/{created['id']}",
json=_grafana_payload(id="other-id"),
)
assert response.status_code == 400
def test_delete_service(client):
created = client.post("/api/services/instances", json=_grafana_payload()).json()
response = client.delete(f"/api/services/instances/{created['id']}")
assert response.status_code == 200
assert client.get("/api/services/instances").json() == []
def test_delete_nonexistent_returns_404(client):
assert client.delete("/api/services/instances/missing").status_code == 404
# ---------------------------------------------------------------------------
# Cascade delete
# ---------------------------------------------------------------------------
def test_delete_service_cascades_to_widgets(client, tmp_path):
"""Once widgets carry service_id (Slice 2), deleting a service removes them.
This test seeds a widget row directly with the column present to prove the
cascade path; the column is added defensively here so the test is meaningful
even before Slice 2 lands.
"""
store = app.dependency_overrides[get_settings_store]()
service = store.upsert_service(
{"service_type": "grafana", "name": "Grafana", "config": {"base_url": "u"}, "enabled": True}
)
# Ensure the service_id column exists and seed a referencing widget.
with store.connect() as conn:
cols = {row[1] for row in conn.execute("PRAGMA table_info(dashboard_widgets)").fetchall()}
if "service_id" not in cols:
conn.execute("ALTER TABLE dashboard_widgets ADD COLUMN service_id TEXT")
conn.execute(
"""
INSERT INTO dashboard_widgets (id, addon_id, widget_type, title, config_json,
enabled, sort_order, created_at, updated_at, service_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
("w1", "grafana", "grafana.link", "Link", "{}", 1, 0, 1, 1, service["id"]),
)
store.delete_service(service["id"])
assert store.get_service(service["id"]) is None
with store.connect() as conn:
remaining = conn.execute(
"SELECT COUNT(*) FROM dashboard_widgets WHERE service_id = ?",
(service["id"],),
).fetchone()
assert int(remaining[0]) == 0
# ---------------------------------------------------------------------------
# Service task run history
# ---------------------------------------------------------------------------
def test_record_and_list_service_task_runs(client):
store = app.dependency_overrides[get_settings_store]()
service = store.upsert_service(
{"service_type": "ssh_tasks", "name": "box", "config": {"host": "h"}, "enabled": True}
)
store.record_service_task_run(
{
"task_id": "t1",
"service_id": service["id"],
"status": "success",
"exit_status": 0,
"stdout_tail": "ok",
}
)
runs = store.list_service_task_runs(service_id=service["id"])
assert len(runs) == 1
assert runs[0]["status"] == "success"
assert runs[0]["stdout_tail"] == "ok"
+256 -344
View File
@@ -1,22 +1,32 @@
"""Tests for the dashboard widget backend: registry, CRUD, validation, seeding, adapters.""" """Tests for the dashboard widget system: service-bound + built-in widgets."""
from __future__ import annotations
import asyncio
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import MagicMock, patch from unittest.mock import patch
import pytest import pytest
from cryptography.fernet import Fernet
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
from media_library_viewer_api.dependencies import get_settings_store from media_library_viewer_api.dependencies import get_settings_store
from media_library_viewer_api.main import app from media_library_viewer_api.main import app
from media_library_viewer_api.services.settings_store import SettingsStore from media_library_viewer_api.services.settings_store import SettingsStore
from media_library_viewer_api.widgets.sources import ( from media_library_viewer_api.widgets.sources import (
SOURCE_REGISTRY, BackupsWidgetSource,
GrafanaWidgetSource, GrafanaWidgetSource,
SshTaskWidgetSource, ServiceRecord,
StaticWidgetSource, StaticWidgetSource,
) )
TEST_KEY = Fernet.generate_key().decode()
@pytest.fixture(autouse=True)
def _encryption_key(monkeypatch):
monkeypatch.setenv("MANAGE_ENCRYPTION_KEY", TEST_KEY)
yield
@pytest.fixture @pytest.fixture
def client(tmp_path): def client(tmp_path):
@@ -30,446 +40,348 @@ def client(tmp_path):
app.dependency_overrides.clear() app.dependency_overrides.clear()
def test_widget_sources(client): def _make_grafana_service(client, name="Production Grafana", **config_overrides):
response = client.get("/api/widgets/sources") config = {"base_url": "https://grafana.example.com"}
config.update(config_overrides)
return client.post(
"/api/services/instances",
json={"service_type": "grafana", "name": name, "config": config, "enabled": True},
).json()
# ---------------------------------------------------------------------------
# Built-in kinds + built-in widget CRUD
# ---------------------------------------------------------------------------
def test_list_builtin_kinds(client):
response = client.get("/api/widgets/builtin")
assert response.status_code == 200 assert response.status_code == 200
assert set(response.json()) == { kinds = {item["kind"] for item in response.json()}
"jellyfin", assert kinds == {"backups", "static"}
"backups",
"grafana",
"prometheus",
"ssh_task",
"static",
}
def test_widget_types(client): def test_create_and_read_static_widget(client):
response = client.get("/api/widgets/types")
assert response.status_code == 200
types = {item["widget_type"] for item in response.json()}
assert types == {
"jellyfin",
"backups",
"grafana-link",
"prometheus-metric",
"ssh-task",
"static",
}
def test_create_and_read_widget(client):
response = client.post( response = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={
"addon_id": "core", "widget_kind": "static",
"widget_type": "static",
"title": "Note", "title": "Note",
"config": {"text": "hello"}, "config": {"text": "hello"},
"enabled": True,
"sort_order": 5,
}, },
) )
assert response.status_code == 201 assert response.status_code == 201
widget = response.json() created = response.json()
assert widget["title"] == "Note" assert created["widget_kind"] == "static"
assert widget["config"] == {"text": "hello"} assert created["service_id"] is None
assert widget["enabled"] is True assert created["config"]["text"] == "hello"
assert widget["sort_order"] == 5
widget_id = widget["id"]
response = client.get("/api/widgets/instances") listed = client.get("/api/widgets/instances").json()
assert response.status_code == 200 assert len(listed) == 1
assert any(w["id"] == widget_id for w in response.json()) assert listed[0]["id"] == created["id"]
def test_update_widget(client): def test_create_backups_widget(client):
response = client.post(
"/api/widgets/instances",
json={"widget_kind": "backups", "title": "Backups", "config": {}},
)
assert response.status_code == 201
def test_unknown_builtin_kind_rejected(client):
response = client.post(
"/api/widgets/instances",
json={"widget_kind": "bogus", "title": "x", "config": {}},
)
assert response.status_code == 422
def test_credential_key_in_config_rejected(client):
response = client.post(
"/api/widgets/instances",
json={"widget_kind": "static", "title": "x", "config": {"api_key": "leak"}},
)
assert response.status_code == 422
# ---------------------------------------------------------------------------
# Service-bound widget CRUD
# ---------------------------------------------------------------------------
def test_create_service_bound_widget(client):
service = _make_grafana_service(client)
response = client.post( response = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={
"addon_id": "core", "service_id": service["id"],
"widget_type": "static", "widget_kind": "link",
"title": "Note", "title": "Dashboard",
"config": {"text": "hello"}, "config": {"dashboard_uid": "overview"},
}, },
) )
widget_id = response.json()["id"] assert response.status_code == 201
created = response.json()
response = client.put( assert created["service_id"] == service["id"]
f"/api/widgets/instances/{widget_id}", assert created["widget_kind"] == "link"
json={
"addon_id": "core",
"widget_type": "static",
"title": "Updated",
"config": {"text": "world"},
"enabled": False,
"sort_order": 10,
},
)
assert response.status_code == 200
data = response.json()
assert data["title"] == "Updated"
assert data["config"] == {"text": "world"}
assert data["enabled"] is False
assert data["sort_order"] == 10
def test_delete_widget(client): def test_service_bound_widget_unknown_kind_rejected(client):
service = _make_grafana_service(client)
response = client.post( response = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={
"addon_id": "core", "service_id": service["id"],
"widget_type": "static", "widget_kind": "metric",
"title": "To delete", "title": "x",
"config": {"text": "bye"},
},
)
widget_id = response.json()["id"]
response = client.delete(f"/api/widgets/instances/{widget_id}")
assert response.status_code == 200
response = client.get("/api/widgets/instances")
assert not any(w["id"] == widget_id for w in response.json())
def test_unknown_widget_type_rejected(client):
response = client.post(
"/api/widgets/instances",
json={
"addon_id": "core",
"widget_type": "unknown",
"title": "Bad",
"config": {}, "config": {},
}, },
) )
assert response.status_code == 422 assert response.status_code == 422
def test_addon_id_mismatch_rejected(client): def test_service_bound_widget_service_not_found_rejected(client):
response = client.post( response = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={
"addon_id": "grafana", "service_id": "missing",
"widget_type": "static", "widget_kind": "link",
"title": "Bad", "title": "x",
"config": {"text": "x"}, "config": {"dashboard_uid": "u"},
}, },
) )
assert response.status_code == 422 assert response.status_code == 422
def test_credential_key_rejected(client): def test_service_bound_widget_invalid_config_rejected(client):
service = _make_grafana_service(client)
response = client.post( response = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={
"addon_id": "core", "service_id": service["id"],
"widget_type": "static", "widget_kind": "link",
"title": "Bad", "title": "x",
"config": {"api_key": "secret123"}, "config": {"dashboard_uid": ""}, # empty still validates; use bad type
},
)
# Empty string passes Pydantic; force a real failure with a bad type.
response = client.post(
"/api/widgets/instances",
json={
"service_id": service["id"],
"widget_kind": "link",
"title": "x",
"config": {"dashboard_uid": 123},
}, },
) )
assert response.status_code == 422 assert response.status_code == 422
def test_update_nonexistent_widget(client): def test_update_and_delete_widget(client):
created = client.post(
"/api/widgets/instances",
json={"widget_kind": "static", "title": "Note", "config": {"text": "a"}},
).json()
updated = client.put(
f"/api/widgets/instances/{created['id']}",
json={"widget_kind": "static", "title": "Note2", "config": {"text": "b"}},
).json()
assert updated["title"] == "Note2"
assert client.delete(f"/api/widgets/instances/{created['id']}").status_code == 200
assert client.get("/api/widgets/instances").json() == []
def test_update_nonexistent_returns_404(client):
response = client.put( response = client.put(
"/api/widgets/instances/does-not-exist", "/api/widgets/instances/missing",
json={ json={"widget_kind": "static", "title": "x", "config": {}},
"addon_id": "core",
"widget_type": "static",
"title": "Bad",
"config": {"text": "x"},
},
) )
assert response.status_code == 404 assert response.status_code == 404
def test_delete_nonexistent_widget(client):
response = client.delete("/api/widgets/instances/does-not-exist")
assert response.status_code == 404
def test_default_widgets_seeded(client):
response = client.get("/api/widgets/instances")
assert response.status_code == 200
widgets = response.json()
types = [w["widget_type"] for w in widgets]
assert "jellyfin" in types
assert "backups" in types
def test_no_reseed_when_widgets_exist(tmp_path):
db_path = tmp_path / "settings.sqlite"
store = SettingsStore(db_path)
store.ensure_defaults()
widgets = store.list_widgets()
assert len(widgets) == 2
store.delete_widget(widgets[0]["id"])
store.ensure_defaults()
remaining = store.list_widgets()
assert len(remaining) == 1
def test_update_id_mismatch_returns_400(client): def test_update_id_mismatch_returns_400(client):
response = client.post( created = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={"widget_kind": "static", "title": "x", "config": {}},
"addon_id": "core", ).json()
"widget_type": "static",
"title": "Note",
"config": {"text": "hello"},
},
)
widget_id = response.json()["id"]
response = client.put( response = client.put(
f"/api/widgets/instances/{widget_id}", f"/api/widgets/instances/{created['id']}",
json={ json={"id": "other", "widget_kind": "static", "title": "x", "config": {}},
"id": "different-id",
"addon_id": "core",
"widget_type": "static",
"title": "Updated",
"config": {"text": "world"},
},
) )
assert response.status_code == 400 assert response.status_code == 400
def test_empty_title_rejected(client): # ---------------------------------------------------------------------------
response = client.post( # Data endpoint
"/api/widgets/instances", # ---------------------------------------------------------------------------
json={
"addon_id": "core",
"widget_type": "static",
"title": "",
"config": {"text": "hello"},
},
)
assert response.status_code == 422
def test_config_type_error_rejected(client):
response = client.post(
"/api/widgets/instances",
json={
"addon_id": "grafana",
"widget_type": "grafana-link",
"title": "Grafana",
"config": {"panel_id": "not-an-integer"},
},
)
assert response.status_code == 422
def test_list_instances_respects_sort_order(client):
response = client.get("/api/widgets/instances")
assert response.status_code == 200
widgets = response.json()
orders = [w["sort_order"] for w in widgets]
assert orders == sorted(orders)
def test_enabled_round_trip(client):
response = client.post(
"/api/widgets/instances",
json={
"addon_id": "core",
"widget_type": "static",
"title": "Toggle",
"config": {"text": "x"},
"enabled": False,
},
)
widget_id = response.json()["id"]
response = client.put(
f"/api/widgets/instances/{widget_id}",
json={
"addon_id": "core",
"widget_type": "static",
"title": "Toggle",
"config": {"text": "x"},
"enabled": True,
},
)
assert response.status_code == 200
assert response.json()["enabled"] is True
def test_fetch_static_widget_data(client): def test_fetch_static_widget_data(client):
response = client.post( created = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={"widget_kind": "static", "title": "Note", "config": {"text": "hello"}},
"addon_id": "core", ).json()
"widget_type": "static", response = client.get(f"/api/widgets/instances/{created['id']}/data")
"title": "Note",
"config": {"text": "hello world"},
},
)
widget_id = response.json()["id"]
response = client.get(f"/api/widgets/instances/{widget_id}/data")
assert response.status_code == 200 assert response.status_code == 200
data = response.json() body = response.json()
assert data["widget_id"] == widget_id assert body["data"]["text"] == "hello"
assert data["widget_type"] == "static" assert body["error"] is None
assert data["data"] == {"text": "hello world"}
assert data["error"] is None
assert isinstance(data["fetched_at"], int)
def test_fetch_grafana_widget_data(client): def test_fetch_backups_widget_data(client):
response = client.post( created = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={"widget_kind": "backups", "title": "Backups", "config": {}},
"addon_id": "grafana", ).json()
"widget_type": "grafana-link", response = client.get(f"/api/widgets/instances/{created['id']}/data")
"title": "Grafana",
"config": {"dashboard_uid": "overview", "panel_id": 3},
},
)
widget_id = response.json()["id"]
response = client.get(f"/api/widgets/instances/{widget_id}/data")
assert response.status_code == 200 assert response.status_code == 200
data = response.json() assert "total_jobs" in response.json()["data"]
assert data["widget_type"] == "grafana-link"
assert data["data"]["url"] == "http://grafana:3000/d/overview?viewPanel=3"
def test_fetch_prometheus_widget_data(client): def test_fetch_grafana_link_widget_data(client):
response = client.post( service = _make_grafana_service(client)
created = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={
"addon_id": "prometheus", "service_id": service["id"],
"widget_type": "prometheus-metric", "widget_kind": "link",
"title": "CPU", "title": "Dashboard",
"config": {"promql": '100 - avg(irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100'}, "config": {"dashboard_uid": "overview", "panel_id": 2},
}, },
) ).json()
widget_id = response.json()["id"] response = client.get(f"/api/widgets/instances/{created['id']}/data")
fake_payload = {"data": {"resultType": "scalar", "result": [1718900000, "42.5"]}}
with patch("media_library_viewer_api.widgets.sources.requests.get") as mock_get:
mock_response = MagicMock()
mock_response.raise_for_status.return_value = None
mock_response.json.return_value = fake_payload
mock_get.return_value = mock_response
response = client.get(f"/api/widgets/instances/{widget_id}/data")
assert response.status_code == 200 assert response.status_code == 200
data = response.json() assert response.json()["data"]["url"] == "https://grafana.example.com/d/overview?viewPanel=2"
assert data["widget_type"] == "prometheus-metric"
assert data["data"]["result"]["resultType"] == "scalar"
def test_fetch_jellyfin_widget_data_error(client): def test_fetch_widget_service_not_found(client):
response = client.post( service = _make_grafana_service(client)
created = client.post(
"/api/widgets/instances", "/api/widgets/instances",
json={ json={
"addon_id": "core", "service_id": service["id"],
"widget_type": "jellyfin", "widget_kind": "link",
"title": "Activity", "title": "x",
"config": {"machine_id": ""}, "config": {"dashboard_uid": "u"},
},
).json()
# Deleting the service cascade-deletes its widgets, so the widget is gone.
client.delete(f"/api/services/instances/{service['id']}")
assert client.get("/api/widgets/instances").json() == []
assert client.get(f"/api/widgets/instances/{created['id']}/data").status_code == 404
def test_fetch_widget_service_disabled(client):
service = _make_grafana_service(client)
created = client.post(
"/api/widgets/instances",
json={
"service_id": service["id"],
"widget_kind": "link",
"title": "x",
"config": {"dashboard_uid": "u"},
},
).json()
client.put(
f"/api/services/instances/{service['id']}",
json={
"service_type": "grafana",
"name": service["name"],
"config": {"base_url": "https://grafana.example.com"},
"enabled": False,
}, },
) )
widget_id = response.json()["id"] response = client.get(f"/api/widgets/instances/{created['id']}/data")
response = client.get(f"/api/widgets/instances/{widget_id}/data")
assert response.status_code == 200 assert response.status_code == 200
data = response.json() assert "disabled" in response.json()["error"]
assert data["widget_type"] == "jellyfin"
assert data["data"] is None
assert data["error"] is not None
assert "Jellyfin" in data["error"] or "machine" in data["error"].lower()
def test_fetch_widget_data_not_found(client): def test_fetch_widget_not_found(client):
response = client.get("/api/widgets/instances/does-not-exist/data") assert client.get("/api/widgets/instances/missing/data").status_code == 404
assert response.status_code == 404
def test_fetch_widget_data_unhandled_exception_returns_500(client): # ---------------------------------------------------------------------------
response = client.post( # Adapter unit tests
"/api/widgets/instances", # ---------------------------------------------------------------------------
json={
"addon_id": "core",
"widget_type": "static",
"title": "Note",
"config": {"text": "x"},
},
)
widget_id = response.json()["id"]
class _ExplodingAdapter:
source_type = "static"
async def fetch(self, config): @pytest.mark.asyncio
raise RuntimeError("boom") async def test_grafana_adapter_builds_url():
adapter = GrafanaWidgetSource()
service = ServiceRecord(id="s", service_type="grafana", name="g", config={"base_url": "http://g:3000"})
result = await adapter.fetch(service, "link", {"dashboard_uid": "ov"})
assert result["url"] == "http://g:3000/d/ov"
result = await adapter.fetch(service, "link", {"dashboard_uid": "ov", "panel_id": 4})
assert result["url"] == "http://g:3000/d/ov?viewPanel=4"
with patch("media_library_viewer_api.routers.widgets.get_source_adapter", return_value=_ExplodingAdapter()):
response = client.get(f"/api/widgets/instances/{widget_id}/data")
assert response.status_code == 500 @pytest.mark.asyncio
async def test_grafana_adapter_missing_service():
adapter = GrafanaWidgetSource()
result = await adapter.fetch(None, "link", {"dashboard_uid": "ov"})
assert "error" in result
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_static_adapter(): async def test_static_adapter():
adapter = StaticWidgetSource() adapter = StaticWidgetSource()
result = await adapter.fetch({"text": "hello"}) result = await adapter.fetch(None, "static", {"text": "hi"})
assert result == {"text": "hello"} assert result == {"text": "hi"}
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_grafana_adapter(): async def test_backups_adapter(client):
adapter = GrafanaWidgetSource() store = app.dependency_overrides[get_settings_store]()
result = await adapter.fetch({"dashboard_uid": "overview", "panel_id": 2}) with patch("media_library_viewer_api.widgets.sources.get_settings_store", return_value=store):
assert result["url"] == "http://grafana:3000/d/overview?viewPanel=2" adapter = BackupsWidgetSource()
result = await adapter.fetch(None, "backups", {})
result = await adapter.fetch({"dashboard_uid": "overview"}) assert "total_jobs" in result
assert result["url"] == "http://grafana:3000/d/overview"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_ssh_task_adapter_timeout(tmp_path): async def test_ssh_task_adapter_missing_service():
store = SettingsStore(tmp_path / "settings.sqlite") from media_library_viewer_api.widgets.sources import SshTaskWidgetSource
store.ensure_defaults()
# Create a local machine and a simple shell task.
machine = store.list_machines()[0]
task = store.upsert_task(
{
"name": "slow-task",
"task_type": "shell",
"content": "echo hello",
"enabled": True,
"default_machine_id": machine["id"],
}
)
adapter = SshTaskWidgetSource() adapter = SshTaskWidgetSource()
with patch( result = await adapter.fetch(None, "task_output", {"task_id": "t1"})
"media_library_viewer_api.widgets.sources.get_settings_store",
return_value=store,
), patch(
"media_library_viewer_api.widgets.sources.asyncio.wait_for",
side_effect=asyncio.TimeoutError,
):
result = await adapter.fetch({"task_id": task["id"]})
assert "error" in result assert "error" in result
assert "timed out" in result["error"].lower()
def test_source_registry_closed(): @pytest.mark.asyncio
assert set(SOURCE_REGISTRY.keys()) == { async def test_ssh_task_adapter_records_history_on_run(client):
"jellyfin", store = app.dependency_overrides[get_settings_store]()
"backups", # Save a task and an ssh_tasks service instance.
"grafana", task = store.upsert_task(
"prometheus", {
"ssh_task", "name": "echo",
"static", "task_type": "shell",
} "content": "echo hi",
"enabled": True,
"default_machine_id": "",
}
)
service = store.upsert_service(
{"service_type": "ssh_tasks", "name": "box", "config": {"host": "h", "username": "u"}, "enabled": True}
)
fake_result = SimpleNamespace(exit_status=0, stdout="hi\n", stderr="")
fake_client = SimpleNamespace(run=lambda *a, **k: fake_result)
from media_library_viewer_api.widgets.sources import SshTaskWidgetSource
adapter = SshTaskWidgetSource()
service_record = ServiceRecord(
id=service["id"], service_type="ssh_tasks", name="box", config={"host": "h", "username": "u"}
)
with (
patch("media_library_viewer_api.widgets.sources.get_settings_store", return_value=store),
patch("media_library_viewer_api.widgets.sources._build_ssh_client", return_value=fake_client),
):
result = await adapter.fetch(service_record, "task_output", {"task_id": task["id"]})
assert result["exit_status"] == 0
runs = store.list_service_task_runs(service_id=service["id"])
assert len(runs) == 1
assert runs[0]["status"] == "success"
+1
View File
@@ -19,6 +19,7 @@ services:
ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-} ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-}
GRAFANA_URL: ${GRAFANA_URL:-http://grafana:3000} GRAFANA_URL: ${GRAFANA_URL:-http://grafana:3000}
PROMETHEUS_URL: ${PROMETHEUS_URL:-http://prometheus:9090} PROMETHEUS_URL: ${PROMETHEUS_URL:-http://prometheus:9090}
MANAGE_ENCRYPTION_KEY: ${MANAGE_ENCRYPTION_KEY:?set MANAGE_ENCRYPTION_KEY in your .env}
ports: ports:
- "8000:8000" - "8000:8000"
volumes: volumes:
+1
View File
@@ -30,6 +30,7 @@ services:
ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-} ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-}
GRAFANA_URL: ${GRAFANA_URL:-http://grafana:3000} GRAFANA_URL: ${GRAFANA_URL:-http://grafana:3000}
PROMETHEUS_URL: ${PROMETHEUS_URL:-http://prometheus:9090} PROMETHEUS_URL: ${PROMETHEUS_URL:-http://prometheus:9090}
MANAGE_ENCRYPTION_KEY: ${MANAGE_ENCRYPTION_KEY:?generate one with python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"}
volumes: volumes:
- ${BACKEND_CACHE_DIR:-./backend-cache}:/app/backend/.cache - ${BACKEND_CACHE_DIR:-./backend-cache}:/app/backend/.cache
restart: unless-stopped restart: unless-stopped
@@ -0,0 +1,78 @@
# Apply Progress: Runtime Service Registry
**Change:** `service-registry`
**Apply run:** PR 1 + PR 2 / Slice 1 + Slice 2
**Date:** 2026-06-19
## Slice 1 — Backend service foundation (MERGED)
Completed in PR #7. See git history. Summary: Fernet secrets helper, closed
`integrations/` registry with Pydantic config + widget-config definitions for
grafana/prometheus/jellyfin/nextcloud/ssh_tasks, `services` + `service_task_runs`
tables with cascade delete, `/api/services*` CRUD, `MANAGE_ENCRYPTION_KEY`
required at startup, 25 tests.
## Slice 2 — Backend widget rebind to services (this PR)
### Completed tasks
- [x] 2.1 Add `service_id` / `widget_kind` columns to `dashboard_widgets`
(additive ALTER; legacy `addon_id`/`widget_type` kept but unused).
- [x] 2.2 Refactor source adapters to `fetch(service, widget_kind, config)`
with `ServiceRecord | None`. `SERVICE_ADAPTERS` keyed by service_type;
`BUILTIN_ADAPTERS` for backups/static. SSH adapter resolves the task +
instance, runs, and appends a `service_task_runs` row (success/failure/
timeout/error).
- [x] 2.3 Retire old `widgets/registry.py` (deleted; metadata now comes from
`integrations/registry` + `widgets/builtin`).
- [x] 2.4 Update widgets router + models for service-bound + built-in widgets.
Removed `/api/widgets/types` and `/api/widgets/sources`; added
`/api/widgets/builtin`.
- [x] 2.5 Rewrite widget tests around the new model.
- [x] 2.6 Stop default widget seeding (fresh install = empty dashboard).
### Decision resolved mid-slice
Backups and static widgets stay as **service-less built-ins** (`service_id`
nullable), per product decision. The data endpoint resolves built-ins via
`BUILTIN_ADAPTERS` and service-bound widgets via `SERVICE_ADAPTERS` + a
decrypted `ServiceRecord`.
### Files changed (Slice 2)
- New: `widgets/builtin.py` (built-in kinds + light config validation).
- Rewritten: `widgets/sources.py` (`ServiceRecord`, new protocol, service +
built-in adapters, SSH run logging, `_build_ssh_client`).
- Deleted: `widgets/registry.py`.
- Modified: `models/widgets.py` (service_id + widget_kind; `BuiltinWidgetKindInfo`).
- Modified: `routers/widgets.py` (new validation, `/builtin`, data resolution).
- Modified: `services/settings_store.py` (widget columns; no-op seeding).
- Modified: `integrations/base.py` (`WidgetKind.config_model` for Pydantic
widget-config validation).
- Rewritten: `tests/test_widgets.py` (26 tests).
### Verification (Slice 2)
```bash
cd backend
.venv/bin/ruff check . # All checks passed
PYTHONPATH=src .venv/bin/python -m pytest # 222 passed
cd ../frontend
npm run lint # 0 errors
npm run build # success
```
### Known transient state (resolved by Slice 3)
Slice 2 is a backend-only breaking change to the widget API. Until Slice 3
lands, the frontend still calls the removed `/api/widgets/types` and
`/api/widgets/sources` endpoints and uses the old `widget_type` shape, so the
dashboard widget config UI is non-functional at runtime. Build/lint stay green.
This is the accepted transient state for a stacked backend→frontend rebind.
## Remaining work
- Slice 3: Frontend services runtime (types, API, hooks, frontend service
registry, service pages, route swap, remove addon pages, reconcile widget UI).
- Slice 4: Dashboard picker on services, settings rework, remove
`grafana_url`/`prometheus_url` env vars, docs + changelog.