chore(observability): externalize stack from root compose files

Manage now connects to existing Grafana/Prometheus/Alertmanager instances
and never deploys its own stack.

- docker-compose.yml / docker-compose.dev.yml: removed prometheus, loki,
  alloy, grafana, alertmanager, node-exporter services, the monitoring
  network, and observability named volumes; they now ship only backend +
  frontend. Dev frontend now joins the web network so the Vite dev proxy
  can reach the backend.
- backend: alertmanager_url default is now empty; /api/monitoring/alerts
  and /alertmanager-status return graceful "not configured" responses
  when ALERTMANAGER_URL is unset. Added not-configured tests.
- docker-compose.observability.yml: kept as the optional standalone
  example; header clarifies Manage does not deploy it.
- Removed orphaned combined monitoring/prometheus/prometheus.yml
  (standalone stack uses prometheus.standalone.yml).
- Docs (README, REQUIREMENTS decision log, monitoring-logging-design,
  observability-runbooks, context.md, MIGRATION_PLAN, frontend/README,
  CHANGELOG) updated to the connect-to-existing model.

VITE_GRAFANA_URL / VITE_PROMETHEUS_URL remain as optional frontend
deep-link overrides. .env.example still needs a manual update (safety
policy blocks assistant edits): set ALERTMANAGER_URL empty/optional and
move standalone-only vars out of the root file.

CHANGELOG.md

@@ -4,6 +4,25 @@ All notable changes to Manage. Breaking changes are marked with **BREAKING**.
 
 ## [Unreleased]
 
+### Changed — Observability is now external only
+
+- **Removed** all observability services from `docker-compose.yml` and
+  `docker-compose.dev.yml`. They now deploy **only** the backend and frontend.
+  The `monitoring` network and the `prometheus`/`loki`/`alloy`/`grafana`/
+  `alertmanager`/`node-exporter` services and their named volumes were deleted,
+  and the `GRAFANA_APP_HOST` Traefik rule was removed.
+- Manage now connects to **existing** Grafana/Prometheus/Alertmanager instances
+  and never ships its own stack. The previous in-compose stack is preserved as
+  an optional, deploy-it-yourself example in `docker-compose.observability.yml`
+  (config under `monitoring/`, documented in `docs/observability-runbooks.md`).
+- The backend `alertmanager_url` default is now empty. The
+  `/api/monitoring/alerts` and `/alertmanager-status` endpoints return a
+  graceful "not configured" response when `ALERTMANAGER_URL` is unset.
+- Removed the now-orphaned combined `monitoring/prometheus/prometheus.yml`; the
+  standalone stack uses `monitoring/prometheus/prometheus.standalone.yml`.
+- `VITE_GRAFANA_URL`/`VITE_PROMETHEUS_URL` remain as optional frontend deep-link
+  overrides. `ALERTMANAGER_URL`/`ALERTMANAGER_WEBHOOK_URL` are optional.
+
 ### Added — Service registry
 
 - Runtime **service registry** persisted in the backend SQLite database. External

README.md

@@ -44,6 +44,8 @@ Open the app at <http://localhost:8080>.
 
 The production Compose file requires OIDC and Traefik variables; see [Configuration](#configuration) below. Copy `.env.example` to `.env`, fill in the required values, and export them in your shell before running `docker compose up`.
 
+> **Observability is external.** Manage only ships its **backend** and **frontend**. It does **not** deploy Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter. The backend exposes a `/metrics` endpoint and optional Alertmanager proxy endpoints so an *existing* observability deployment can scrape and consume them. For a ready-to-run example stack you can deploy alongside Manage, see [`docker-compose.observability.yml`](docker-compose.observability.yml) and [`docs/observability-runbooks.md`](docs/observability-runbooks.md).
+
 Local development with hot reload:
 
 ```bash
@@ -81,7 +83,6 @@ Production-style example with shell exports:
 ```bash
 export BACKEND_APP_HOST=api.manage.example.com
 export FRONTEND_APP_HOST=manage.example.com
-export GRAFANA_APP_HOST=grafana.manage.example.com
 export CERT_RESOLVER=letsencrypt
 export VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/
 export VITE_OIDC_CLIENT_ID=manage
@@ -97,7 +98,7 @@ docker compose up --build
 Inline one-liner example:
 
 ```bash
-BACKEND_APP_HOST=api.manage.example.com FRONTEND_APP_HOST=manage.example.com GRAFANA_APP_HOST=grafana.manage.example.com CERT_RESOLVER=letsencrypt VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ VITE_OIDC_CLIENT_ID=manage VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ VITE_GRAFANA_URL=https://grafana.manage.example.com VITE_PROMETHEUS_URL=https://prometheus.manage.example.com docker compose up --build
+BACKEND_APP_HOST=api.manage.example.com FRONTEND_APP_HOST=manage.example.com CERT_RESOLVER=letsencrypt VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ VITE_OIDC_CLIENT_ID=manage VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ VITE_GRAFANA_URL=https://grafana.manage.example.com VITE_PROMETHEUS_URL=https://prometheus.manage.example.com docker compose up --build
 ```
 
 For local development, no SSH key is required unless you want to connect to remote SSH machines later:
@@ -182,6 +183,7 @@ cd frontend && npx tsc --noEmit && npm run build
 - Jellyfin server root URL required (not `/web`). The client strips trailing `/web` defensively.
 - SSH commands run through `/bin/sh -c` regardless of remote login shell.
 - Job templates are shell-quoted. Add new templates in `backend/src/media_library_viewer_api/jobs.py`.
-- Root-level Docker Compose files are provided for production (`docker-compose.yml`) and local development (`docker-compose.dev.yml`), and both rely on Compose interpolation rather than `env_file` entries.
+- Root-level Docker Compose files are provided for production (`docker-compose.yml`) and local development (`docker-compose.dev.yml`), and both rely on Compose interpolation rather than `env_file` entries. They deploy **only** the backend and frontend; Manage never deploys its own observability stack (see `docker-compose.observability.yml` for an optional standalone example).
 - The configurable dashboard stores widget instances in the backend SQLite settings database. New installs seed default Jellyfin activity and Backups widgets automatically.
 - Grafana and Prometheus widget adapters resolve URLs from service records configured in the app; `VITE_GRAFANA_URL` / `VITE_PROMETHEUS_URL` are only used for frontend deep-links. No credentials are stored in widget config; service API keys are encrypted at rest with `MANAGE_ENCRYPTION_KEY`.
+- `ALERTMANAGER_URL` is optional. When unset, the Alertmanager proxy endpoints return graceful "not configured" responses instead of erroring.

backend/src/media_library_viewer_api/config.py

@@ -55,7 +55,7 @@ class Settings(BaseSettings):
     # Observability
     prometheus_enabled: bool = True
     prometheus_file_sd_dir: str = "/app/backend/.cache/prometheus-file-sd"
-    alertmanager_url: str = "http://alertmanager:9093"
+    alertmanager_url: str = ""
     alertmanager_webhook_url: str = ""  # Optional receiver for alertmanager webhook notifications
 
     # Remote paths

backend/src/media_library_viewer_api/routers/monitoring.py

@@ -88,11 +88,13 @@ def get_alertmanager_alerts() -> dict[str, Any]:
     """Return a summary of active Alertmanager alerts for the UI.
 
     Proxies the Alertmanager `/api/v1/alerts` endpoint and reshapes the payload
-    into a stable, UI-friendly format. If Alertmanager is unreachable, the
-    endpoint returns an empty summary and logs the failure so the UI can still
-    render a health card instead of an error page.
+    into a stable, UI-friendly format. If Alertmanager is unreachable (or not
+    configured via ``ALERTMANAGER_URL``), the endpoint returns an empty summary
+    so the UI can still render a health card instead of an error page.
     """
     session, base_url = _alertmanager_client()
+    if not base_url:
+        return {"total": 0, "by_severity": {}, "alerts": [], "error": "alertmanager_not_configured"}
     try:
         response = session.get(f"{base_url}/api/v1/alerts", timeout=5)
         response.raise_for_status()
@@ -115,9 +117,12 @@ def get_alertmanager_status() -> dict[str, Any]:
 
     Uses the Alertmanager `/api/v2/status` endpoint and exposes only the high-
     level fields the UI needs: uptime, version, and whether the cluster is
-    healthy.
+    healthy. Returns ``up=False`` when Alertmanager is unreachable or not
+    configured via ``ALERTMANAGER_URL``.
     """
     session, base_url = _alertmanager_client()
+    if not base_url:
+        return {"up": False, "version": "", "uptime": ""}
     try:
         response = session.get(f"{base_url}/api/v2/status", timeout=5)
         response.raise_for_status()

backend/tests/test_api.py

@@ -677,6 +677,17 @@ class TestAlertmanager:
         assert data["total"] == 0
         assert data["error"] == "alertmanager_unreachable"
 
+    def test_alerts_endpoint_when_not_configured(self, test_client):
+        with patch("media_library_viewer_api.routers.monitoring._alertmanager_client") as mock_client:
+            session = MagicMock()
+            mock_client.return_value = (session, "")
+            response = test_client.get("/api/monitoring/alerts")
+        assert response.status_code == 200
+        data = response.json()
+        assert data["total"] == 0
+        assert data["error"] == "alertmanager_not_configured"
+        session.get.assert_not_called()
+
     def test_alerts_endpoint_returns_summary(self, test_client):
         with patch("media_library_viewer_api.routers.monitoring._alertmanager_client") as mock_client:
             session = MagicMock()
@@ -700,6 +711,16 @@ class TestAlertmanager:
         assert data["by_severity"]["critical"] == 1
         assert data["alerts"][0]["name"] == "BackupJobFailed"
 
+    def test_alertmanager_status_endpoint_when_not_configured(self, test_client):
+        with patch("media_library_viewer_api.routers.monitoring._alertmanager_client") as mock_client:
+            session = MagicMock()
+            mock_client.return_value = (session, "")
+            response = test_client.get("/api/monitoring/alertmanager-status")
+        assert response.status_code == 200
+        data = response.json()
+        assert data["up"] is False
+        session.get.assert_not_called()
+
     def test_alertmanager_status_endpoint_when_unreachable(self, test_client):
         with patch("media_library_viewer_api.routers.monitoring._alertmanager_client") as mock_client:
             session = MagicMock()

backend/tests/test_widgets.py

@@ -369,7 +369,6 @@ async def test_ssh_task_adapter_records_history_on_run(client):
     fake_result = SimpleNamespace(exit_status=0, stdout="hi\n", stderr="")
     fake_client = SimpleNamespace(run=lambda *a, **k: fake_result)
 
-    from media_library_viewer_api.services.task_runner import build_ssh_client
     from media_library_viewer_api.widgets.sources import SshTaskWidgetSource
 
     adapter = SshTaskWidgetSource()

context.md

@@ -1,6 +1,16 @@
 # Code Context
 
+> **Status (2026-06-23):** Manage no longer deploys an observability stack.
+> The root `docker-compose.yml` / `docker-compose.dev.yml` ship **only** the
+> backend and frontend; Grafana, Prometheus, Loki, Alertmanager, Alloy, and Node
+> Exporter were removed from them. Manage connects to **existing** instances.
+> The standalone example stack lives in `docker-compose.observability.yml`. Some
+> snippets below still reference the former in-compose services and are kept as
+> historical context; treat `docker-compose.observability.yml` as authoritative
+> for the stack layout.
+
 ## Files Retrieved
+
 1. `docker-compose.yml` (lines 1–262) – production Compose stack; defines observability services and Traefik routing.
 2. `docker-compose.dev.yml` (lines 1–234) – development Compose stack; same observability services but with host ports exposed and auth disabled.
 3. `.env.example` (lines 1–55) – template with all required environment variables for the stack, including Prometheus/Grafana/Alertmanager/Alloy/Loki and Node Exporter settings.
@@ -20,7 +30,9 @@
 ## Key Code
 
 ### Backend `/metrics` endpoint
+
 `backend/src/media_library_viewer_api/main.py`:
+
 ```python
 @app.middleware("http")
 async def enforce_jwt_auth(request: Request, call_next):
@@ -36,7 +48,9 @@ def metrics() -> Response:
 ```
 
 ### Metrics emitted by the backend
+
 `backend/src/media_library_viewer_api/observability.py`:
+
 ```python
 REQUESTS_TOTAL = Counter("manage_api_requests_total", "Total API requests", ["method", "path", "status_code"])
 REQUEST_DURATION = Histogram("manage_api_request_duration_seconds", "API request duration", ["method", "path"], ...)
@@ -48,7 +62,9 @@ MAIL_QUEUE_SIZE = Counter("manage_mail_queue_messages_total", "Total messages en
 ```
 
 ### Prometheus scrape configuration
+
 `monitoring/prometheus/prometheus.yml`:
+
 ```yaml
 scrape_configs:
   - job_name: manage-backend
@@ -71,7 +87,9 @@ scrape_configs:
 ```
 
 ### Backend-managed remote Node Exporter targets
+
 `backend/src/media_library_viewer_api/services/targets.py`:
+
 ```python
 def build_node_exporter_targets(store: SettingsStore) -> list[dict[str, Any]]:
     ...
@@ -96,6 +114,7 @@ The observability stack is a standard self-hosted Prometheus/Grafana/Loki/Alertm
 - **Alertmanager** routes alerts by severity (critical vs warning) and delivers email notifications (and optionally a webhook back to the backend).
 
 The backend bridges the stack with the application:
+
 - It exposes `/metrics` (unauthenticated, along with `/api/health` and `/api/version`).
 - On startup it writes `${PROMETHEUS_FILE_SD_DIR}/node_exporter_targets.json` based on enabled SSH machines in the settings store.
 - It provides proxy endpoints (`/api/monitoring/alerts`, `/api/monitoring/alertmanager-status`, `/api/monitoring/prometheus-targets`) consumed by the frontend.
@@ -105,6 +124,7 @@ The backend bridges the stack with the application:
 Open `monitoring/prometheus/prometheus.yml` first to understand what is scraped and how the backend is wired, then read `backend/src/media_library_viewer_api/observability.py` to see the metric names and labels. For environment requirements, read `.env.example`.
 
 ## Supervisor coordination
+
 Not needed — this is a read-only scouting summary.
 
 ---
@@ -113,7 +133,7 @@ Not needed — this is a read-only scouting summary.
 
 ## 1. Observability services defined in Compose
 
-Both `docker-compose.yml` and `docker-compose.dev.yml` define the following services:
+Both `docker-compose.yml` and `docker-compose.dev.yml` define **only the backend and frontend**. The observability services (Prometheus, Loki, Grafana, Alertmanager, Alloy, Node Exporter) were extracted to the standalone `docker-compose.observability.yml` example stack and are **no longer** deployed by Manage. Summary of what remains in the app compose files:
 
 | Service | Image | Internal endpoint | Purpose |
 |---------|-------|-------------------|---------|
@@ -126,6 +146,7 @@ Both `docker-compose.yml` and `docker-compose.dev.yml` define the following serv
 | `backend` | Build from `backend/Dockerfile` | `http://backend:8000` | FastAPI app exposing `/metrics` |
 
 Differences:
+
 - Production (`docker-compose.yml`): services attach to an external `web` network for Traefik, use `expose` instead of host ports for most services, and require OIDC/auth variables.
 - Development (`docker-compose.dev.yml`): Prometheus/Grafana/Loki/Alertmanager/Node Exporter are published on host ports `9090`, `3000`, `3100`, `9093`, `9100`; auth is disabled (`AUTH_ENABLED=false`).
 
@@ -134,6 +155,7 @@ Differences:
 From `.env.example` and the Compose files, the variables relevant to the observability stack are:
 
 ### Backend / metrics
+
 - `PROMETHEUS_ENABLED` – enable metrics endpoint (set to `"true"` in both compose files).
 - `PROMETHEUS_FILE_SD_DIR` – directory where the backend writes `node_exporter_targets.json` (default `/app/backend/.cache/prometheus-file-sd`).
 - `ALERTMANAGER_URL` – backend proxy target (default `http://alertmanager:9093`).
@@ -141,6 +163,7 @@ From `.env.example` and the Compose files, the variables relevant to the observa
 - `BACKEND_CACHE_DIR` – host directory mounted into backend and Prometheus for file-SD.
 
 ### Grafana
+
 - `GRAFANA_APP_HOST` – public hostname for Grafana (production; required).
 - `GRAFANA_APP_PORT` – defaults to `3000`.
 - `GRAFANA_APP_NAME` – defaults to `grafana`.
@@ -152,16 +175,19 @@ From `.env.example` and the Compose files, the variables relevant to the observa
 - `GF_AUTH_GENERIC_OAUTH_API_URL`
 
 ### Alertmanager
+
 - `SMTP_HOST` / `SMTP_PORT`
 - `SMTP_USERNAME` / `SMTP_PASSWORD`
 - `SMTP_FROM_ADDRESS`
 - `ALERT_EMAIL_TO`
 
 ### Traefik / network (production)
+
 - `BACKEND_APP_HOST` / `FRONTEND_APP_HOST` / `GRAFANA_APP_HOST`
 - `CERT_RESOLVER` – e.g. `letsencrypt`
 
 ### General
+
 - `LOG_LEVEL` / `LOG_FORMAT` – also passed to Grafana and backend.
 
 ## 3. Monitoring config files
@@ -193,7 +219,7 @@ Remote Node Exporter targets are not static: the backend reads machine settings
 
 ## 5. Setup steps and gotchas
 
-- The observability stack is brought up with the app itself:
+- Manage's own compose stack does **not** include observability services. To run a full local stack, bring up the app and the standalone observability example separately:
   - Production: `docker compose -f docker-compose.yml up --build`
   - Development: `docker compose -f docker-compose.dev.yml up --build`
 - Production requires the external `web` network and Traefik already configured; `docker-compose.dev.yml` does not use Traefik and binds ports directly.

docker-compose.dev.yml

@@ -1,3 +1,12 @@
+# Manage development stack — backend + frontend only.
+#
+# Manage does NOT deploy its own Grafana/Prometheus/Loki/Alertmanager stack.
+# It exposes a /metrics endpoint and optional Alertmanager proxy endpoints so
+# an *existing* observability deployment can scrape/consume them.
+#
+# For a ready-to-use example observability stack, see
+# docker-compose.observability.yml and docs/observability-runbooks.md.
+
 services:
   backend:
     build:
@@ -15,7 +24,7 @@ services:
       PROMETHEUS_ENABLED: "true"
       SSH_KNOWN_HOSTS_PATH: /app/backend/.cache/known_hosts
       PROMETHEUS_FILE_SD_DIR: /app/backend/.cache/prometheus-file-sd
-      ALERTMANAGER_URL: ${ALERTMANAGER_URL:-http://alertmanager:9093}
+      ALERTMANAGER_URL: ${ALERTMANAGER_URL:-}
       ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-}
       MANAGE_ENCRYPTION_KEY: ${MANAGE_ENCRYPTION_KEY:?set MANAGE_ENCRYPTION_KEY in your .env}
     ports:
@@ -25,7 +34,6 @@ services:
       - ${BACKEND_CACHE_DIR:-./backend-cache}:/app/backend/.cache
     networks:
       - web
-      - monitoring
     restart: unless-stopped
 
   frontend:
@@ -38,8 +46,8 @@ services:
       VITE_API_URL: "/api"
       VITE_OIDC_ENABLED: "false"
       VITE_DEV_API_PROXY_TARGET: "http://backend:8000"
-      VITE_GRAFANA_URL: "http://localhost:3000"
-      VITE_PROMETHEUS_URL: "http://localhost:9090"
+      VITE_GRAFANA_URL: ${VITE_GRAFANA_URL:-http://localhost:3000}
+      VITE_PROMETHEUS_URL: ${VITE_PROMETHEUS_URL:-http://localhost:9090}
     ports:
       - "5173:5173"
     volumes:
@@ -47,211 +55,12 @@ services:
       - frontend_node_modules:/app/frontend/node_modules
     depends_on:
       - backend
-    restart: unless-stopped
-
-  prometheus:
-    image: prom/prometheus:v2.55.1
-    container_name: prometheus
-    command:
-      - "--config.file=/etc/prometheus/prometheus.yml"
-      - "--storage.tsdb.path=/prometheus"
-      - "--storage.tsdb.retention.time=30d"
-      - "--web.console.libraries=/usr/share/prometheus/console_libraries"
-      - "--web.console.templates=/usr/share/prometheus/consoles"
-      - "--web.enable-lifecycle"
-    volumes:
-      - ./monitoring/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
-      - ./monitoring/prometheus/rules:/etc/prometheus/rules:ro
-      - ${BACKEND_CACHE_DIR:-./backend-cache}/prometheus-file-sd:/etc/prometheus/file-sd:ro
-      - prometheus_data:/prometheus
-    ports:
-      - "9090:9090"
     networks:
-      - monitoring
+      - web
     restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9090/-/healthy"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "1.00"
-          memory: 1G
-        reservations:
-          cpus: "0.25"
-          memory: 256M
-
-  loki:
-    image: grafana/loki:3.1.1
-    container_name: loki
-    command: -config.file=/etc/loki/loki.yml
-    volumes:
-      - ./monitoring/loki/loki.yml:/etc/loki/loki.yml:ro
-      - loki_data:/loki
-    ports:
-      - "3100:3100"
-    networks:
-      - monitoring
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3100/ready"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "1.00"
-          memory: 1G
-        reservations:
-          cpus: "0.25"
-          memory: 256M
-
-  alloy:
-    image: grafana/alloy:v1.5.0
-    container_name: alloy
-    command:
-      - run
-      - /etc/alloy/config.alloy
-      - --storage.path=/var/lib/alloy
-    volumes:
-      - ./monitoring/alloy/config.alloy:/etc/alloy/config.alloy:ro
-      - /var/run/docker.sock:/var/run/docker.sock:ro
-      - /var/lib/docker/containers:/var/lib/docker/containers:ro
-    networks:
-      - monitoring
-    depends_on:
-      loki:
-        condition: service_healthy
-    restart: unless-stopped
-    user: root
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:12345/-/healthy"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "0.50"
-          memory: 512M
-        reservations:
-          cpus: "0.10"
-          memory: 128M
-
-  grafana:
-    image: grafana/grafana:11.3.1
-    container_name: grafana
-    environment:
-      GF_SECURITY_ADMIN_USER: ${GRAFANA_ADMIN_USER:-admin}
-      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD:-admin}
-      GF_AUTH_GENERIC_OAUTH_CLIENT_ID: ${GF_AUTH_GENERIC_OAUTH_CLIENT_ID:-}
-      GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET:-}
-      GF_AUTH_GENERIC_OAUTH_AUTH_URL: ${GF_AUTH_GENERIC_OAUTH_AUTH_URL:-}
-      GF_AUTH_GENERIC_OAUTH_TOKEN_URL: ${GF_AUTH_GENERIC_OAUTH_TOKEN_URL:-}
-      GF_AUTH_GENERIC_OAUTH_API_URL: ${GF_AUTH_GENERIC_OAUTH_API_URL:-}
-      LOG_LEVEL: ${LOG_LEVEL:-INFO}
-    volumes:
-      - ./monitoring/grafana/grafana.ini:/etc/grafana/grafana.ini:ro
-      - ./monitoring/grafana/provisioning:/etc/grafana/provisioning:ro
-      - grafana_data:/var/lib/grafana
-    ports:
-      - "3000:3000"
-    networks:
-      - monitoring
-    depends_on:
-      prometheus:
-        condition: service_healthy
-      loki:
-        condition: service_healthy
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3000/api/health"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "1.00"
-          memory: 512M
-        reservations:
-          cpus: "0.25"
-          memory: 128M
-
-  alertmanager:
-    image: prom/alertmanager:v0.27.0
-    container_name: alertmanager
-    command:
-      - "--config.file=/etc/alertmanager/alertmanager.yml"
-      - "--storage.path=/alertmanager"
-    environment:
-      SMTP_HOST: ${SMTP_HOST:-smtp.example.com}
-      SMTP_PORT: ${SMTP_PORT:-587}
-      SMTP_USERNAME: ${SMTP_USERNAME:-}
-      SMTP_PASSWORD: ${SMTP_PASSWORD:-}
-      SMTP_FROM_ADDRESS: ${SMTP_FROM_ADDRESS:-no-reply@example.com}
-      ALERT_EMAIL_TO: ${ALERT_EMAIL_TO:-admin@example.com}
-    volumes:
-      - ./monitoring/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
-      - alertmanager_data:/alertmanager
-    ports:
-      - "9093:9093"
-    networks:
-      - monitoring
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9093/-/healthy"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "0.50"
-          memory: 256M
-        reservations:
-          cpus: "0.10"
-          memory: 64M
-
-  node-exporter:
-    image: prom/node-exporter:v1.8.2
-    container_name: node-exporter
-    command:
-      - "--path.rootfs=/host"
-    volumes:
-      - /:/host:ro,rslave
-    networks:
-      - monitoring
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9100/"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 10s
-    deploy:
-      resources:
-        limits:
-          cpus: "0.25"
-          memory: 128M
-        reservations:
-          cpus: "0.05"
-          memory: 32M
 
 networks:
-  monitoring:
+  web:
 
 volumes:
   frontend_node_modules:
-  prometheus_data:
-  loki_data:
-  grafana_data:
-  alertmanager_data:

docker-compose.observability.yml

@@ -1,4 +1,13 @@
-# Standalone observability stack.
+# Standalone observability stack (OPTIONAL EXAMPLE).
+#
+# Manage itself does NOT deploy Grafana/Prometheus/Loki/Alertmanager/Alloy/Node
+# Exporter. The root docker-compose.yml / docker-compose.dev.yml ship only the
+# backend and frontend, and Manage connects to EXISTING Grafana/Prometheus/
+# Alertmanager instances.
+#
+# This file is a ready-to-use example you can run on a host where you want the
+# full Grafana, Prometheus, Loki, Alertmanager, Alloy and Node Exporter stack
+# without the rest of Manage. It is not required to run Manage.
 #
 # Run this on a host where you want Grafana, Prometheus, Loki, Alertmanager,
 # Alloy and Node Exporter without the rest of Manage.

docker-compose.yml

@@ -1,3 +1,12 @@
+# Manage production stack — backend + frontend only.
+#
+# Manage does NOT deploy its own Grafana/Prometheus/Loki/Alertmanager stack.
+# It exposes a /metrics endpoint and optional Alertmanager proxy endpoints so
+# an *existing* observability deployment can scrape/consume them.
+#
+# For a ready-to-use example observability stack, see
+# docker-compose.observability.yml and docs/observability-runbooks.md.
+
 services:
   backend:
     build:
@@ -26,7 +35,7 @@ services:
       SMTP_TIMEOUT: ${SMTP_TIMEOUT:-30}
       SSH_KNOWN_HOSTS_PATH: /app/backend/.cache/known_hosts
       PROMETHEUS_FILE_SD_DIR: ${PROMETHEUS_FILE_SD_DIR:-/app/backend/.cache/prometheus-file-sd}
-      ALERTMANAGER_URL: ${ALERTMANAGER_URL:-http://alertmanager:9093}
+      ALERTMANAGER_URL: ${ALERTMANAGER_URL:-}
       ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-}
       MANAGE_ENCRYPTION_KEY: ${MANAGE_ENCRYPTION_KEY:?generate one with python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"}
     volumes:
@@ -34,7 +43,6 @@ services:
     restart: unless-stopped
     networks:
       - web
-      - monitoring
     expose:
       - "8000"
     labels:
@@ -71,7 +79,7 @@ services:
         VITE_OIDC_POST_LOGOUT_REDIRECT_URI: ${VITE_OIDC_POST_LOGOUT_REDIRECT_URI:?set VITE_OIDC_POST_LOGOUT_REDIRECT_URI}
         VITE_DEV_API_PROXY_TARGET: ${VITE_DEV_API_PROXY_TARGET:-http://backend:8000}
         VITE_GRAFANA_URL: ${VITE_GRAFANA_URL:-https://grafana.example.com}
-        VITE_PROMETHEUS_URL: ${VITE_PROMETHEUS_URL:-http://localhost:9090}
+        VITE_PROMETHEUS_URL: ${VITE_PROMETHEUS_URL:-http://prometheus.example.com}
         VITE_APP_VERSION: ${APP_VERSION:-0.1.0}
         VITE_APP_BUILD_INFO: ${APP_BUILD_INFO:-dev}
     depends_on:
@@ -89,218 +97,6 @@ services:
       - "8080:80"
     restart: unless-stopped
 
-  prometheus:
-    image: prom/prometheus:v2.55.1
-    container_name: prometheus
-    command:
-      - "--config.file=/etc/prometheus/prometheus.yml"
-      - "--storage.tsdb.path=/prometheus"
-      - "--storage.tsdb.retention.time=30d"
-      - "--web.console.libraries=/usr/share/prometheus/console_libraries"
-      - "--web.console.templates=/usr/share/prometheus/consoles"
-      - "--web.enable-lifecycle"
-    volumes:
-      - ./monitoring/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
-      - ./monitoring/prometheus/rules:/etc/prometheus/rules:ro
-      - ${BACKEND_CACHE_DIR:-./backend-cache}/prometheus-file-sd:/etc/prometheus/file-sd:ro
-      - prometheus_data:/prometheus
-    expose:
-      - "9090"
-    networks:
-      - web
-      - monitoring
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9090/-/healthy"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "1.00"
-          memory: 1G
-        reservations:
-          cpus: "0.25"
-          memory: 256M
-
-  loki:
-    image: grafana/loki:3.1.1
-    container_name: loki
-    command: -config.file=/etc/loki/loki.yml
-    volumes:
-      - ./monitoring/loki/loki.yml:/etc/loki/loki.yml:ro
-      - loki_data:/loki
-    expose:
-      - "3100"
-    networks:
-      - monitoring
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3100/ready"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "1.00"
-          memory: 1G
-        reservations:
-          cpus: "0.25"
-          memory: 256M
-
-  alloy:
-    image: grafana/alloy:v1.5.0
-    container_name: alloy
-    command:
-      - run
-      - /etc/alloy/config.alloy
-      - --storage.path=/var/lib/alloy
-    volumes:
-      - ./monitoring/alloy/config.alloy:/etc/alloy/config.alloy:ro
-      - /var/run/docker.sock:/var/run/docker.sock:ro
-      - /var/lib/docker/containers:/var/lib/docker/containers:ro
-    networks:
-      - monitoring
-    depends_on:
-      loki:
-        condition: service_healthy
-    restart: unless-stopped
-    user: root
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:12345/-/healthy"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "0.50"
-          memory: 512M
-        reservations:
-          cpus: "0.10"
-          memory: 128M
-
-  grafana:
-    image: grafana/grafana:11.3.1
-    container_name: grafana
-    environment:
-      GF_SECURITY_ADMIN_USER: ${GRAFANA_ADMIN_USER:-admin}
-      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD:-admin}
-      GF_AUTH_GENERIC_OAUTH_CLIENT_ID: ${GF_AUTH_GENERIC_OAUTH_CLIENT_ID:-}
-      GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET:-}
-      GF_AUTH_GENERIC_OAUTH_AUTH_URL: ${GF_AUTH_GENERIC_OAUTH_AUTH_URL:-}
-      GF_AUTH_GENERIC_OAUTH_TOKEN_URL: ${GF_AUTH_GENERIC_OAUTH_TOKEN_URL:-}
-      GF_AUTH_GENERIC_OAUTH_API_URL: ${GF_AUTH_GENERIC_OAUTH_API_URL:-}
-      LOG_LEVEL: ${LOG_LEVEL:-INFO}
-    volumes:
-      - ./monitoring/grafana/grafana.ini:/etc/grafana/grafana.ini:ro
-      - ./monitoring/grafana/provisioning:/etc/grafana/provisioning:ro
-      - grafana_data:/var/lib/grafana
-    expose:
-      - "3000"
-    networks:
-      - web
-      - monitoring
-    depends_on:
-      prometheus:
-        condition: service_healthy
-      loki:
-        condition: service_healthy
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3000/api/health"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    labels:
-      - "traefik.enable=true"
-      - "traefik.http.routers.${GRAFANA_APP_NAME:-grafana}.rule=Host(`${GRAFANA_APP_HOST:?set GRAFANA_APP_HOST}`)"
-      - "traefik.http.routers.${GRAFANA_APP_NAME:-grafana}.entrypoints=websecure"
-      - "traefik.http.routers.${GRAFANA_APP_NAME:-grafana}.tls.certresolver=${CERT_RESOLVER:?set CERT_RESOLVER}"
-      - "traefik.http.services.${GRAFANA_APP_NAME:-grafana}.loadbalancer.server.port=${GRAFANA_APP_PORT:-3000}"
-    deploy:
-      resources:
-        limits:
-          cpus: "1.00"
-          memory: 512M
-        reservations:
-          cpus: "0.25"
-          memory: 128M
-
-  alertmanager:
-    image: prom/alertmanager:v0.27.0
-    container_name: alertmanager
-    command:
-      - "--config.file=/etc/alertmanager/alertmanager.yml"
-      - "--storage.path=/alertmanager"
-    environment:
-      SMTP_HOST: ${SMTP_HOST:-smtp.example.com}
-      SMTP_PORT: ${SMTP_PORT:-587}
-      SMTP_USERNAME: ${SMTP_USERNAME:-}
-      SMTP_PASSWORD: ${SMTP_PASSWORD:-}
-      SMTP_FROM_ADDRESS: ${SMTP_FROM_ADDRESS:-no-reply@example.com}
-      ALERT_EMAIL_TO: ${ALERT_EMAIL_TO:-admin@example.com}
-    volumes:
-      - ./monitoring/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
-      - alertmanager_data:/alertmanager
-    expose:
-      - "9093"
-    networks:
-      - monitoring
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9093/-/healthy"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 30s
-    deploy:
-      resources:
-        limits:
-          cpus: "0.50"
-          memory: 256M
-        reservations:
-          cpus: "0.10"
-          memory: 64M
-
-  node-exporter:
-    image: prom/node-exporter:v1.8.2
-    container_name: node-exporter
-    command:
-      - "--path.rootfs=/host"
-    volumes:
-      - /:/host:ro,rslave
-    networks:
-      - monitoring
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9100/"]
-      interval: 30s
-      timeout: 5s
-      retries: 3
-      start_period: 10s
-    deploy:
-      resources:
-        limits:
-          cpus: "0.25"
-          memory: 128M
-        reservations:
-          cpus: "0.05"
-          memory: 32M
-
 networks:
   web:
     external: true
-  monitoring:
-
-volumes:
-  prometheus_data:
-  loki_data:
-  grafana_data:
-  alertmanager_data:

docs/MIGRATION_PLAN.md

@@ -92,7 +92,7 @@ repo/
 │       └── types/
 │           └── index.ts
 │
-├── docker-compose.yml          # Optional unified deployment
+├── docker-compose.yml          # Backend + frontend deployment (observability is external)
 └── README.md
 ```
 

docs/REQUIREMENTS.md

@@ -197,7 +197,7 @@ fully removed (web-ui-rework; see decision log 2026-06-17).
 - Dashboard shortcuts should support a small icon/preview field so cards can be visually recognizable without changing the underlying model later.
 - Support OIDC login in the frontend using an OIDC client library, with backend JWT validation for protected API requests.
 - Persist frontend OIDC auth state across tab reloads by storing the OIDC user and request state in browser localStorage.
-- Provide Docker Compose deployment files at the repository root for production and local development.
+- Provide Docker Compose deployment files at the repository root for production and local development. These deploy **only** the backend and frontend; Manage connects to *existing* Grafana/Prometheus/Alertmanager instances and never ships its own observability stack (see `docker-compose.observability.yml` for an optional standalone example).
 - SSH private keys should be managed as reusable saved secrets in Settings, independent of any one machine, and SSH machines should select from that saved-key list.
 - The web UI should allow both importing an existing private key and generating a new SSH keypair for that saved-key list.
 - Saved SSH keys should display their derived public key, fingerprint, and machine usage count so administrators can audit them at a glance.
@@ -331,6 +331,7 @@ the widget/addon-pages model were removed. `MANAGE_ENCRYPTION_KEY` is now requir
 
 ## Decision Log
 
+- 2026-06-23: Removed all observability services from the root Compose files. `docker-compose.yml` and `docker-compose.dev.yml` now deploy **only** the backend and frontend; the `monitoring` network and the `prometheus`/`loki`/`alloy`/`grafana`/`alertmanager`/`node-exporter` services and their named volumes were deleted, and the `GRAFANA_APP_HOST` Traefik rule was removed. Manage connects to **existing** Grafana/Prometheus/Alertmanager instances and does not deploy them itself. The backend `alertmanager_url` default is now empty, and the `/api/monitoring/alerts` and `/alertmanager-status` endpoints return graceful "not configured" responses when `ALERTMANAGER_URL` is unset. The previous in-compose stack is preserved as an optional standalone example in `docker-compose.observability.yml` (config under `monitoring/`, documented in `docs/observability-runbooks.md`). `VITE_GRAFANA_URL`/`VITE_PROMETHEUS_URL` remain as optional frontend deep-link overrides.
 - 2026-06-17: Decommissioned the legacy Manage-side system-metric scraping. Removed the backend `MonitoringPoller` (SSH-ran `df` on every machine every 5 min into a local SQLite `monitoring_machine_actions` table), the entire `services/monitoring_actions.py` module, the `/api/monitoring/poller`, `/api/monitoring/machines/{id}/actions`, and `/api/monitoring/disk` endpoints, the `monitoring_machine_actions` table (DROP on startup), the three `monitoring_poll_*` / `monitoring_action_retention_days` config knobs, and the orphaned frontend `DiskSpaceCard` + `DiskSpace` type. System metrics are now owned exclusively by Prometheus + node_exporter + Grafana. Kept the Alertmanager proxy (`/alerts`, `/alertmanager-status`, `/alertmanager-webhook`), `/prometheus-targets`, `/machines`, the `node_exporter_*` machine fields, and the on-demand `disk_usage` job template.
 - 2026-06-17: Completed the web UI rework to a single design system. The frontend now uses **shadcn/ui + Tailwind CSS v4 + lucide-react** exclusively, with CSS `@theme` tokens in `src/index.css` (primary `#4f8cff`; `chart-1..5` repurposed as status/Grafana-link cues). Removed `@mui/material`, `@mui/icons-material`, `@mui/x-data-grid`, `@emotion/react`, `@emotion/styled`, `recharts`, `d3`, and the `src/theme.ts` shim. Tables moved from `@mui/x-data-grid`/AG Grid to a visibility-only TanStack `DataTable` wrapper (pagination, row selection, row click, column visibility — no sorting/resizing). Adopted the thin-dashboard observability model (no in-app charts; Alertmanager alerts + Prometheus target health + Grafana deep-links). Reconciled the information architecture: Backups is a top-level nav item at `/backups`, and the media surface is named Media at `/media` with `/applications` redirecting to `/media` (mirroring `/monitoring` → `/observability`). Frontend tests moved to Vitest + @testing-library/react (`npm test`), with legacy node suites in `frontend/tests`.
 - 2026-06-13: Adopted a dedicated, self-hosted observability subsystem based on Prometheus, Grafana Loki, Grafana, and Alertmanager. Metrics will be pulled from Node Exporter on machines and from application exporters in containers; logs will be structured JSON shipped by Promtail/Grafana Alloy. The existing POSIX remote collector will be removed and backup alerts migrated to Alertmanager rules. See `docs/monitoring-logging-design.md`.

docs/monitoring-logging-design.md

@@ -1,5 +1,16 @@
 # Monitoring and Logging Design — Manage
 
+> **Status (2026-06-23):** This document is the **reference architecture** for the
+> self-hosted observability stack Manage integrates with. Manage itself **does
+> not deploy** Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter.
+> The root `docker-compose.yml` / `docker-compose.dev.yml` ship **only** the
+> backend and frontend; Manage connects to **existing** Grafana / Prometheus /
+> Alertmanager instances. The stack described here is available as an optional,
+> deploy-it-yourself example in `docker-compose.observability.yml` (see
+> `docs/observability-runbooks.md`). The "Add services to `docker-compose.yml`"
+> phase notes below describe the **original** in-compose deployment, which has
+> since been extracted to the standalone compose file.
+
 ## Executive Summary
 
 Manage currently uses ad-hoc observability: plain-text Python logs, a custom POSIX shell metrics collector on remote machines, and a background poller that stores snapshots in SQLite. This works for a single-instance homelab but becomes painful as the fleet grows and as users need faster incident response.

docs/observability-runbooks.md

@@ -1,6 +1,10 @@
 # Observability Runbooks
 
-Operational playbooks for the Manage self-hosted observability stack (Prometheus, Grafana, Loki, Alertmanager).
+Operational playbooks for the **standalone example observability stack**
+(`docker-compose.observability.yml`) that can be deployed alongside Manage.
+Manage itself does **not** deploy these services; it connects to existing
+Grafana / Prometheus / Alertmanager instances. These runbooks cover operating
+the standalone stack shipped under `monitoring/`.
 
 ## Service Overview
 

frontend/README.md

@@ -8,10 +8,9 @@ React + TypeScript SPA for Manage, consuming the FastAPI backend.
 - **React 18+** — UI framework
 - **TypeScript** — Type safety
 - **@tanstack/react-query** — Data fetching/caching
-- **ag-grid-react** — Data tables (media, file browser)
-- **recharts** — Monitoring charts
+- **@tanstack/react-table** — Data tables (media, file browser)
 - **react-router-dom** — Client-side routing
-- **Tailwind CSS** — Styling
+- **Tailwind CSS** + **shadcn/ui** — Styling
 
 ## Setup
 
@@ -26,7 +25,7 @@ npm install
 npm run dev
 ```
 
-Runs on http://localhost:5173 with API requests proxied to http://localhost:8000.
+Runs on <http://localhost:5173> with API requests proxied to <http://localhost:8000>.
 
 Make sure the backend is running:
 
@@ -45,12 +44,13 @@ Output goes to `frontend/dist/`.
 
 ## Pages
 
-- **Dashboard** (`/`) — Now playing, backend-collected per-machine monitoring table with 10-minute averages/min/max, library stats, frontend/backend version chips in the shell header
-- **Monitoring** (`/monitoring`) — Per-machine CPU/IO wait/RAM/network/disk charts, collector controls, and backend-collected recent action history
+- **Dashboard** (`/`) — Now playing, library stats, configurable widgets and shortcuts, frontend/backend version chips in the shell header
+- **Observability** (`/observability`) — Thin dashboard: Alertmanager alerts, Prometheus target health, machine status, and Grafana deep-links (no in-app charting)
 - **Media** (`/media`) — Full-library table with sort/filter/search
 - **Users** (`/users`) — Read-only Jellyfin user list with optional Jellyseerr enrichment
 - **File Browser** (`/files`) — Remote directory browsing, ffprobe preview, jobs
-- **Settings** (`/settings`) — Persistent monitoring machine definitions and setup workflow
+- **Settings** (`/settings`) — Persistent monitoring machine definitions, SSH keys, and setup workflow
+- **Actions** (`/actions`) — Saved server tasks (shell/python) targeting ssh_tasks services
 
 ## Environment Variables
 
@@ -65,7 +65,6 @@ VITE_API_URL=http://your-backend-host:8000
 
 - **Local development**: run `docker compose -f docker-compose.dev.yml up --build`, then open the app and add monitoring machines in the **Settings** tab.
 - **Production**: export the required Compose variables in your shell, run `docker compose up --build`, and manage local/remote machines from **Settings**.
-- **Dashboard monitoring UI**: the dashboard shows a compact, sortable table with one row per configured machine plus poller status.
-- **Monitoring UI**: the **Monitoring** tab shows one card per configured machine, including a recent action-history table populated by the backend poller. A machine can be `local` (the API host itself) or `ssh` (a remote host), and the UI treats both the same after configuration. The page also shows the backend poller health badge.
+- **Observability**: Manage only deploys backend + frontend. It connects to **existing** Grafana/Prometheus/Alertmanager instances; see `docker-compose.observability.yml` for an optional standalone example stack. A machine can be `local` (the API host itself) or `ssh` (a remote host), and the UI treats both the same after configuration.
 
-In development, the Vite proxy handles `/api` requests automatically.
+In development, the Vite proxy handles `/api` requests automatically. `VITE_GRAFANA_URL` / `VITE_PROMETHEUS_URL` are optional frontend deep-link overrides for your existing instances.

monitoring/prometheus/prometheus.yml

@@ -1,48 +0,0 @@
-global:
-  scrape_interval: 15s
-  evaluation_interval: 15s
-  external_labels:
-    monitor: manage
-
-alerting:
-  alertmanagers:
-    - static_configs:
-        - targets:
-            - alertmanager:9093
-
-rule_files:
-  - /etc/prometheus/rules/*.yml
-
-scrape_configs:
-  - job_name: prometheus
-    static_configs:
-      - targets:
-          - localhost:9090
-
-  - job_name: manage-backend
-    static_configs:
-      - targets:
-          - backend:8000
-    metrics_path: /metrics
-    scrape_interval: 15s
-
-  - job_name: node-exporter
-    static_configs:
-      - targets:
-          - node-exporter:9100
-
-  - job_name: loki
-    static_configs:
-      - targets:
-          - loki:3100
-
-  - job_name: alertmanager
-    static_configs:
-      - targets:
-          - alertmanager:9093
-
-  - job_name: node-exporter-remote
-    file_sd_configs:
-      - files:
-          - /etc/prometheus/file-sd/node_exporter_targets.json
-        refresh_interval: 30s