chore(observability): externalize stack from root compose files

Manage now connects to existing Grafana/Prometheus/Alertmanager instances
and never deploys its own stack.

- docker-compose.yml / docker-compose.dev.yml: removed prometheus, loki,
  alloy, grafana, alertmanager, node-exporter services, the monitoring
  network, and observability named volumes; they now ship only backend +
  frontend. Dev frontend now joins the web network so the Vite dev proxy
  can reach the backend.
- backend: alertmanager_url default is now empty; /api/monitoring/alerts
  and /alertmanager-status return graceful "not configured" responses
  when ALERTMANAGER_URL is unset. Added not-configured tests.
- docker-compose.observability.yml: kept as the optional standalone
  example; header clarifies Manage does not deploy it.
- Removed orphaned combined monitoring/prometheus/prometheus.yml
  (standalone stack uses prometheus.standalone.yml).
- Docs (README, REQUIREMENTS decision log, monitoring-logging-design,
  observability-runbooks, context.md, MIGRATION_PLAN, frontend/README,
  CHANGELOG) updated to the connect-to-existing model.

VITE_GRAFANA_URL / VITE_PROMETHEUS_URL remain as optional frontend
deep-link overrides. .env.example still needs a manual update (safety
policy blocks assistant edits): set ALERTMANAGER_URL empty/optional and
move standalone-only vars out of the root file.

docs/MIGRATION_PLAN.md

@@ -92,7 +92,7 @@ repo/
 │       └── types/
 │           └── index.ts
 │
-├── docker-compose.yml          # Optional unified deployment
+├── docker-compose.yml          # Backend + frontend deployment (observability is external)
 └── README.md
 ```
 

docs/REQUIREMENTS.md

@@ -197,7 +197,7 @@ fully removed (web-ui-rework; see decision log 2026-06-17).
 - Dashboard shortcuts should support a small icon/preview field so cards can be visually recognizable without changing the underlying model later.
 - Support OIDC login in the frontend using an OIDC client library, with backend JWT validation for protected API requests.
 - Persist frontend OIDC auth state across tab reloads by storing the OIDC user and request state in browser localStorage.
-- Provide Docker Compose deployment files at the repository root for production and local development.
+- Provide Docker Compose deployment files at the repository root for production and local development. These deploy **only** the backend and frontend; Manage connects to *existing* Grafana/Prometheus/Alertmanager instances and never ships its own observability stack (see `docker-compose.observability.yml` for an optional standalone example).
 - SSH private keys should be managed as reusable saved secrets in Settings, independent of any one machine, and SSH machines should select from that saved-key list.
 - The web UI should allow both importing an existing private key and generating a new SSH keypair for that saved-key list.
 - Saved SSH keys should display their derived public key, fingerprint, and machine usage count so administrators can audit them at a glance.
@@ -331,6 +331,7 @@ the widget/addon-pages model were removed. `MANAGE_ENCRYPTION_KEY` is now requir
 
 ## Decision Log
 
+- 2026-06-23: Removed all observability services from the root Compose files. `docker-compose.yml` and `docker-compose.dev.yml` now deploy **only** the backend and frontend; the `monitoring` network and the `prometheus`/`loki`/`alloy`/`grafana`/`alertmanager`/`node-exporter` services and their named volumes were deleted, and the `GRAFANA_APP_HOST` Traefik rule was removed. Manage connects to **existing** Grafana/Prometheus/Alertmanager instances and does not deploy them itself. The backend `alertmanager_url` default is now empty, and the `/api/monitoring/alerts` and `/alertmanager-status` endpoints return graceful "not configured" responses when `ALERTMANAGER_URL` is unset. The previous in-compose stack is preserved as an optional standalone example in `docker-compose.observability.yml` (config under `monitoring/`, documented in `docs/observability-runbooks.md`). `VITE_GRAFANA_URL`/`VITE_PROMETHEUS_URL` remain as optional frontend deep-link overrides.
 - 2026-06-17: Decommissioned the legacy Manage-side system-metric scraping. Removed the backend `MonitoringPoller` (SSH-ran `df` on every machine every 5 min into a local SQLite `monitoring_machine_actions` table), the entire `services/monitoring_actions.py` module, the `/api/monitoring/poller`, `/api/monitoring/machines/{id}/actions`, and `/api/monitoring/disk` endpoints, the `monitoring_machine_actions` table (DROP on startup), the three `monitoring_poll_*` / `monitoring_action_retention_days` config knobs, and the orphaned frontend `DiskSpaceCard` + `DiskSpace` type. System metrics are now owned exclusively by Prometheus + node_exporter + Grafana. Kept the Alertmanager proxy (`/alerts`, `/alertmanager-status`, `/alertmanager-webhook`), `/prometheus-targets`, `/machines`, the `node_exporter_*` machine fields, and the on-demand `disk_usage` job template.
 - 2026-06-17: Completed the web UI rework to a single design system. The frontend now uses **shadcn/ui + Tailwind CSS v4 + lucide-react** exclusively, with CSS `@theme` tokens in `src/index.css` (primary `#4f8cff`; `chart-1..5` repurposed as status/Grafana-link cues). Removed `@mui/material`, `@mui/icons-material`, `@mui/x-data-grid`, `@emotion/react`, `@emotion/styled`, `recharts`, `d3`, and the `src/theme.ts` shim. Tables moved from `@mui/x-data-grid`/AG Grid to a visibility-only TanStack `DataTable` wrapper (pagination, row selection, row click, column visibility — no sorting/resizing). Adopted the thin-dashboard observability model (no in-app charts; Alertmanager alerts + Prometheus target health + Grafana deep-links). Reconciled the information architecture: Backups is a top-level nav item at `/backups`, and the media surface is named Media at `/media` with `/applications` redirecting to `/media` (mirroring `/monitoring` → `/observability`). Frontend tests moved to Vitest + @testing-library/react (`npm test`), with legacy node suites in `frontend/tests`.
 - 2026-06-13: Adopted a dedicated, self-hosted observability subsystem based on Prometheus, Grafana Loki, Grafana, and Alertmanager. Metrics will be pulled from Node Exporter on machines and from application exporters in containers; logs will be structured JSON shipped by Promtail/Grafana Alloy. The existing POSIX remote collector will be removed and backup alerts migrated to Alertmanager rules. See `docs/monitoring-logging-design.md`.

docs/monitoring-logging-design.md

@@ -1,5 +1,16 @@
 # Monitoring and Logging Design — Manage
 
+> **Status (2026-06-23):** This document is the **reference architecture** for the
+> self-hosted observability stack Manage integrates with. Manage itself **does
+> not deploy** Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter.
+> The root `docker-compose.yml` / `docker-compose.dev.yml` ship **only** the
+> backend and frontend; Manage connects to **existing** Grafana / Prometheus /
+> Alertmanager instances. The stack described here is available as an optional,
+> deploy-it-yourself example in `docker-compose.observability.yml` (see
+> `docs/observability-runbooks.md`). The "Add services to `docker-compose.yml`"
+> phase notes below describe the **original** in-compose deployment, which has
+> since been extracted to the standalone compose file.
+
 ## Executive Summary
 
 Manage currently uses ad-hoc observability: plain-text Python logs, a custom POSIX shell metrics collector on remote machines, and a background poller that stores snapshots in SQLite. This works for a single-instance homelab but becomes painful as the fleet grows and as users need faster incident response.

docs/observability-runbooks.md

@@ -1,6 +1,10 @@
 # Observability Runbooks
 
-Operational playbooks for the Manage self-hosted observability stack (Prometheus, Grafana, Loki, Alertmanager).
+Operational playbooks for the **standalone example observability stack**
+(`docker-compose.observability.yml`) that can be deployed alongside Manage.
+Manage itself does **not** deploy these services; it connects to existing
+Grafana / Prometheus / Alertmanager instances. These runbooks cover operating
+the standalone stack shipped under `monitoring/`.
 
 ## Service Overview