diff --git a/README.md b/README.md index e439499..7d6c17d 100644 --- a/README.md +++ b/README.md @@ -1,62 +1,67 @@ # Manage -Manage is a media and server operations tool with Jellyfin integration, SSH file inspection, server monitoring, and safe remote job templates. +Manage is a media and server-operations application with Jellyfin integration, SSH file inspection, monitoring integrations, safe remote-job templates, a FastAPI backend, and a React single-page application. -See `docs/REQUIREMENTS.md` for the living requirements, decisions, and planning history. -See `docs/MIGRATION_PLAN.md` for the FastAPI + React architecture plan. +It includes a configurable dashboard, service registry, per-machine settings, a SQLite-indexed media library, a read-only Users view with optional Jellyseerr enrichment, remote file browsing with `ffprobe`, and SSH-based job execution. -Project policy/docs: +## Architecture and scope -- License: `LICENSE` (MIT) -- Contributing guide: `CONTRIBUTING.md` +- `backend/` is the FastAPI API. +- `frontend/` is the React and TypeScript SPA. +- `archive/` retains the original Streamlit prototype for reference. -## Architecture +The root Compose files deploy **only** Manage's backend and frontend. Manage can expose `/metrics` and optional Alertmanager proxy endpoints, but it does not deploy Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter as part of its normal stack. Configure service instances in the app's Services page. -The project consists of two subprojects: +## Prerequisites -- **`backend/`** — FastAPI Python API (see `backend/README.md`) -- **`frontend/`** — React + TypeScript SPA (see `frontend/README.md`) -- **`archive/`** — Original Streamlit prototype (preserved for reference) +- Docker and Docker Compose for the supplied Compose stacks. +- Python 3.11 or newer for manual backend development. +- Node.js and npm for manual frontend development. +- A valid Fernet key for `MANAGE_ENCRYPTION_KEY`, including in development Compose. +- For production: an existing external Docker network named `web`, Traefik, DNS/TLS configuration, and an OIDC provider. -## Features +## Local development with Compose -- Configurable dashboard with persisted widgets (Jellyfin activity, backups summary, Grafana deep-links, Prometheus metrics, Alertmanager alerts, SSH task output, static text) and shortcuts -- Thin-dashboard observability: Alertmanager alerts, Prometheus target health, machine status, and Grafana deep-links (no in-app charting) -- Service registry: configure Jellyfin, Jellyseerr, Alertmanager, Grafana, Prometheus, Nextcloud, and SSH task runner instances in the UI -- Per-machine settings for SSH, monitoring targets, and file browsing -- SQLite-indexed media table with full-library sort/filter -- Read-only Users tab with Jellyfin as the base source and optional Jellyseerr enrichment -- Remote file browser with ffprobe preview and job execution -- Jellyfin API integration for library metadata and user identity data -- SSH-based file inspection and safe remote job templates +1. Create `.env` from the template and set a valid `MANAGE_ENCRYPTION_KEY`. Docker Compose automatically reads `.env` for interpolation; alternatively, export the same variables in the shell. -## Quick Start + ```bash + cp .env.example .env + ``` -### Docker Compose (recommended) + Generate a Fernet key if needed: -Production-style deployment with the frontend serving the SPA and proxying `/api` to the backend. The compose files rely on environment-variable interpolation, so export the required values in your shell before running them (no `env_file` is needed): + ```bash + python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())" + ``` + +2. Start the development stack: + + ```bash + docker compose -f docker-compose.dev.yml up --build + ``` + +The development frontend is available at and the backend at . Development Compose sets `AUTH_ENABLED=false` and `VITE_OIDC_ENABLED=false`, but still requires `MANAGE_ENCRYPTION_KEY`. The backend cache, settings database, media index, saved SSH keys, tasks, and dashboard widgets persist outside rebuilt containers. + +## Production-style deployment + +The root [`docker-compose.yml`](docker-compose.yml) is designed for deployment behind Traefik; it does not publish localhost ports. Before starting it, configure `.env` (or shell variables) with the required values: + +- `BACKEND_APP_HOST`, `FRONTEND_APP_HOST`, and `CERT_RESOLVER` for Traefik routing and certificates. +- `OIDC_ISSUER_URL` and `OIDC_AUDIENCE` for backend authentication. +- `VITE_OIDC_ISSUER`, `VITE_OIDC_CLIENT_ID`, `VITE_OIDC_REDIRECT_URI`, and `VITE_OIDC_POST_LOGOUT_REDIRECT_URI` for the frontend build. +- `MANAGE_ENCRYPTION_KEY`, a valid Fernet key used to encrypt service secrets at rest. + +Then run: ```bash docker compose up --build ``` -Open the app at . +The production Compose file requires its external `web` network to exist. It is not a standalone local deployment; access is through the configured Traefik hostnames. -The production Compose file requires OIDC and Traefik variables; see [Configuration](#configuration) below. Copy `.env.example` to `.env`, fill in the required values, and export them in your shell before running `docker compose up`. +## Manual development -> **Observability is external.** Manage only ships its **backend** and **frontend**. It does **not** deploy Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter. The backend exposes a `/metrics` endpoint and optional Alertmanager proxy endpoints so an *existing* observability deployment can scrape and consume them. For a ready-to-run example stack you can deploy alongside Manage, see [`docker-compose.observability.yml`](docker-compose.observability.yml) and [`docs/observability-runbooks.md`](docs/observability-runbooks.md). - -Local development with hot reload: - -```bash -docker compose -f docker-compose.dev.yml up --build -``` - -Frontend runs on and the backend on . Dev compose disables OIDC by default (`AUTH_ENABLED=false`), so you can open it directly without an identity provider. - -The backend media index and settings database (including monitoring machines, SSH keys, saved tasks, and dashboard widgets) are persisted in Docker volumes so rebuilds and container restarts do not reset state. - -### Manual backend/frontend development +### Backend ```bash cd backend @@ -66,126 +71,75 @@ pip install -e '.[dev]' uvicorn media_library_viewer_api.main:app --reload --port 8000 ``` +### Frontend + ```bash cd frontend npm install npm run dev ``` -## Configuration - -The Compose files use environment-variable interpolation. Export the required variables in your shell or pass them inline; a `.env` file is optional, not required. - -### Compose examples - -Production-style example with shell exports: +## Tests and quality checks ```bash -export BACKEND_APP_HOST=api.manage.example.com -export FRONTEND_APP_HOST=manage.example.com -export CERT_RESOLVER=letsencrypt -export VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ -export VITE_OIDC_CLIENT_ID=manage -export VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback -export VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ -export MANAGE_ENCRYPTION_KEY=$(python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())") +# Backend +cd backend +ruff check . +python -m pytest -docker compose up --build +# Frontend +cd ../frontend +npm run lint +npm run build +npm run test ``` -> Observability services (Grafana, Prometheus, Alertmanager) are configured in -> the app on the **Services** page — no env vars for them. +A focused frontend typecheck can be run with `npx tsc --noEmit` from `frontend/`. -Inline one-liner example: +## Configuration and operations -```bash -BACKEND_APP_HOST=api.manage.example.com FRONTEND_APP_HOST=manage.example.com CERT_RESOLVER=letsencrypt VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ VITE_OIDC_CLIENT_ID=manage VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ docker compose up --build -``` +[`.env.example`](.env.example) is a template; do not commit real credentials or encryption keys. The Compose files interpolate environment values directly. Some template entries are for the optional observability example and are not consumed by the normal Manage Compose stack. -For local development, no SSH key is required unless you want to connect to remote SSH machines later: +Optional SMTP settings (`SMTP_HOST`, `SMTP_PORT`, `SMTP_USERNAME`, `SMTP_PASSWORD`, `SMTP_FROM_ADDRESS`, `SMTP_FROM_NAME`, `SMTP_USE_TLS`, `SMTP_USE_SSL`, and `SMTP_TIMEOUT`) support the Users message popup. -```bash -docker compose -f docker-compose.dev.yml up --build -``` +### Remote servers -Example environment variables: - -```bash -# Optional backend logging level -LOG_LEVEL=INFO - -# Optional SMTP settings for the Users -> message popup -SMTP_HOST=smtp.example.com -SMTP_PORT=587 -SMTP_USERNAME=your-smtp-username -SMTP_PASSWORD=your-smtp-password -SMTP_FROM_ADDRESS=no-reply@example.com -SMTP_FROM_NAME=Manage -SMTP_USE_TLS=true -SMTP_USE_SSL=false -SMTP_TIMEOUT=30 - -# Jellyfin, Jellyseerr, and SSH targets are now configured per machine in the app's Settings tab. -# The backend seeds a local machine automatically, so no global Jellyfin or SSH env vars are required. -# -# Remote SSH machines can store their private key and optional passphrase directly in Settings, -# so no SSH key mount is required for normal use. - -# Authentik / OIDC -AUTH_ENABLED=true -OIDC_ISSUER_URL=https://auth.example.com/application/o/manage/ -OIDC_AUDIENCE=manage -OIDC_JWKS_URL= -OIDC_CLOCK_SKEW_SECONDS=30 - -# Frontend OIDC settings -VITE_OIDC_ENABLED=true -VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ -VITE_OIDC_CLIENT_ID=manage -VITE_OIDC_SCOPE=openid profile email -VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback -VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ - -# Observability services (Grafana, Prometheus, Alertmanager) are configured in -# the app on the Services page. The only observability env var is the optional -# PROMETHEUS_ENABLED toggle (defaults on) for Manage's own /metrics endpoint. - -# Required: master key encrypting service secrets (API keys/tokens) at rest. -# Generate one with: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())" -MANAGE_ENCRYPTION_KEY=replace-with-a-fernet-key -``` - -## Remote server requirements - -The remote server needs: - -- `/bin/sh` (POSIX shell) -- `python3`, `ffprobe`, `find`, `stat`, `df`, `awk` for file inspection and job templates -- SSH access with a key configured in the app's Settings tab - -The SSH client rejects unknown host keys. Connect manually once first: +A managed remote server needs a POSIX `/bin/sh`, `python3`, `ffprobe`, `find`, `stat`, `df`, and `awk`. Configure its SSH credentials in Manage's Settings. Unknown SSH host keys are rejected; establish trust first, for example: ```bash ssh user@host ``` -## Development +SSH commands run through `/bin/sh -c` regardless of the remote login shell. + +### Optional observability example + +[`docker-compose.observability.yml`](docker-compose.observability.yml) is a separate, optional stack for Grafana, Prometheus, Loki, Alertmanager, Alloy, and Node Exporter. It is not required by Manage. Its header documents required `*_ROOT` persistence directories, `CERT_RESOLVER`, and Grafana/Prometheus/Alertmanager host variables. With those prepared, run: ```bash -# Backend (lint + tests) -cd backend && .venv/bin/ruff check . && .venv/bin/python -m pytest - -# Frontend (lint + typecheck/build + tests) -cd frontend && npm run lint && npm run build && npm run test +docker compose -f docker-compose.observability.yml up -d ``` -Focused frontend typecheck: `npx tsc --noEmit`. +Set a non-default `GRAFANA_ADMIN_USER` and a strong, secret `GRAFANA_ADMIN_PASSWORD` before deploying this stack. Do not expose the example observability services with their defaults. -## Notes +See [`docs/observability-runbooks.md`](docs/observability-runbooks.md) for its operational documentation. -- Jellyfin server root URL required (not `/web`). The client strips trailing `/web` defensively. -- SSH commands run through `/bin/sh -c` regardless of remote login shell. -- Job templates are shell-quoted. Add new templates in `backend/src/media_library_viewer_api/jobs.py`. -- Root-level Docker Compose files are provided for production (`docker-compose.yml`) and local development (`docker-compose.dev.yml`), and both rely on Compose interpolation rather than `env_file` entries. They deploy **only** the backend and frontend; Manage never deploys its own observability stack (see `docker-compose.observability.yml` for an optional standalone example). -- The configurable dashboard stores widget instances in the backend SQLite settings database. New installs seed default Jellyfin activity and Backups widgets automatically. -- Grafana, Prometheus, and Alertmanager are configured as **service instances** in the app (Services page); their widget adapters resolve URLs from service records, and no observability URLs/credentials live in env vars. No credentials are stored in widget config; service API keys are encrypted at rest with `MANAGE_ENCRYPTION_KEY`. When no alertmanager service is configured, the alert proxy endpoints return graceful "not configured" responses. +## Repository layout + +```text +. +├── backend/ # FastAPI API and tests +├── frontend/ # React/TypeScript SPA and tests +├── archive/ # Preserved Streamlit prototype +├── docs/ # Requirements, migration, and operations docs +├── docker-compose.yml # Traefik-backed production-style stack +├── docker-compose.dev.yml # Local hot-reload development stack +└── docker-compose.observability.yml # Optional standalone observability example +``` + +## Project documents + +- [Requirements and planning history](docs/REQUIREMENTS.md) +- [FastAPI + React migration plan](docs/MIGRATION_PLAN.md) +- [Contributing guide](CONTRIBUTING.md) +- [MIT license](LICENSE)