|
|
@@ -1,67 +1,62 @@
|
|
|
|
# Manage
|
|
|
|
# Manage
|
|
|
|
|
|
|
|
|
|
|
|
Manage is a media and server-operations application with Jellyfin integration, SSH file inspection, monitoring integrations, safe remote-job templates, a FastAPI backend, and a React single-page application.
|
|
|
|
Manage is a media and server operations tool with Jellyfin integration, SSH file inspection, server monitoring, and safe remote job templates.
|
|
|
|
|
|
|
|
|
|
|
|
It includes a configurable dashboard, service registry, per-machine settings, a SQLite-indexed media library, a read-only Users view with optional Jellyseerr enrichment, remote file browsing with `ffprobe`, and SSH-based job execution.
|
|
|
|
See `docs/REQUIREMENTS.md` for the living requirements, decisions, and planning history.
|
|
|
|
|
|
|
|
See `docs/MIGRATION_PLAN.md` for the FastAPI + React architecture plan.
|
|
|
|
|
|
|
|
|
|
|
|
## Architecture and scope
|
|
|
|
Project policy/docs:
|
|
|
|
|
|
|
|
|
|
|
|
- `backend/` is the FastAPI API.
|
|
|
|
- License: `LICENSE` (MIT)
|
|
|
|
- `frontend/` is the React and TypeScript SPA.
|
|
|
|
- Contributing guide: `CONTRIBUTING.md`
|
|
|
|
- `archive/` retains the original Streamlit prototype for reference.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The root Compose files deploy **only** Manage's backend and frontend. Manage can expose `/metrics` and optional Alertmanager proxy endpoints, but it does not deploy Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter as part of its normal stack. Configure service instances in the app's Services page.
|
|
|
|
## Architecture
|
|
|
|
|
|
|
|
|
|
|
|
## Prerequisites
|
|
|
|
The project consists of two subprojects:
|
|
|
|
|
|
|
|
|
|
|
|
- Docker and Docker Compose for the supplied Compose stacks.
|
|
|
|
- **`backend/`** — FastAPI Python API (see `backend/README.md`)
|
|
|
|
- Python 3.11 or newer for manual backend development.
|
|
|
|
- **`frontend/`** — React + TypeScript SPA (see `frontend/README.md`)
|
|
|
|
- Node.js and npm for manual frontend development.
|
|
|
|
- **`archive/`** — Original Streamlit prototype (preserved for reference)
|
|
|
|
- A valid Fernet key for `MANAGE_ENCRYPTION_KEY`, including in development Compose.
|
|
|
|
|
|
|
|
- For production: an existing external Docker network named `web`, Traefik, DNS/TLS configuration, and an OIDC provider.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Local development with Compose
|
|
|
|
## Features
|
|
|
|
|
|
|
|
|
|
|
|
1. Create `.env` from the template and set a valid `MANAGE_ENCRYPTION_KEY`. Docker Compose automatically reads `.env` for interpolation; alternatively, export the same variables in the shell.
|
|
|
|
- Configurable dashboard with persisted widgets (Jellyfin activity, backups summary, Grafana deep-links, Prometheus metrics, Alertmanager alerts, SSH task output, static text) and shortcuts
|
|
|
|
|
|
|
|
- Thin-dashboard observability: Alertmanager alerts, Prometheus target health, machine status, and Grafana deep-links (no in-app charting)
|
|
|
|
|
|
|
|
- Service registry: configure Jellyfin, Jellyseerr, Alertmanager, Grafana, Prometheus, Nextcloud, and SSH task runner instances in the UI
|
|
|
|
|
|
|
|
- Per-machine settings for SSH, monitoring targets, and file browsing
|
|
|
|
|
|
|
|
- SQLite-indexed media table with full-library sort/filter
|
|
|
|
|
|
|
|
- Read-only Users tab with Jellyfin as the base source and optional Jellyseerr enrichment
|
|
|
|
|
|
|
|
- Remote file browser with ffprobe preview and job execution
|
|
|
|
|
|
|
|
- Jellyfin API integration for library metadata and user identity data
|
|
|
|
|
|
|
|
- SSH-based file inspection and safe remote job templates
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
## Quick Start
|
|
|
|
cp .env.example .env
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Generate a Fernet key if needed:
|
|
|
|
### Docker Compose (recommended)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
Production-style deployment with the frontend serving the SPA and proxying `/api` to the backend. The compose files rely on environment-variable interpolation, so export the required values in your shell before running them (no `env_file` is needed):
|
|
|
|
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2. Start the development stack:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
|
|
docker compose -f docker-compose.dev.yml up --build
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The development frontend is available at <http://localhost:5173> and the backend at <http://localhost:8000>. Development Compose sets `AUTH_ENABLED=false` and `VITE_OIDC_ENABLED=false`, but still requires `MANAGE_ENCRYPTION_KEY`. The backend cache, settings database, media index, saved SSH keys, tasks, and dashboard widgets persist outside rebuilt containers.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Production-style deployment
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The root [`docker-compose.yml`](docker-compose.yml) is designed for deployment behind Traefik; it does not publish localhost ports. Before starting it, configure `.env` (or shell variables) with the required values:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- `BACKEND_APP_HOST`, `FRONTEND_APP_HOST`, and `CERT_RESOLVER` for Traefik routing and certificates.
|
|
|
|
|
|
|
|
- `OIDC_ISSUER_URL` and `OIDC_AUDIENCE` for backend authentication.
|
|
|
|
|
|
|
|
- `VITE_OIDC_ISSUER`, `VITE_OIDC_CLIENT_ID`, `VITE_OIDC_REDIRECT_URI`, and `VITE_OIDC_POST_LOGOUT_REDIRECT_URI` for the frontend build.
|
|
|
|
|
|
|
|
- `MANAGE_ENCRYPTION_KEY`, a valid Fernet key used to encrypt service secrets at rest.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Then run:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
```bash
|
|
|
|
docker compose up --build
|
|
|
|
docker compose up --build
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
The production Compose file requires its external `web` network to exist. It is not a standalone local deployment; access is through the configured Traefik hostnames.
|
|
|
|
Open the app at <http://localhost:8080>.
|
|
|
|
|
|
|
|
|
|
|
|
## Manual development
|
|
|
|
The production Compose file requires OIDC and Traefik variables; see [Configuration](#configuration) below. Copy `.env.example` to `.env`, fill in the required values, and export them in your shell before running `docker compose up`.
|
|
|
|
|
|
|
|
|
|
|
|
### Backend
|
|
|
|
> **Observability is external.** Manage only ships its **backend** and **frontend**. It does **not** deploy Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter. The backend exposes a `/metrics` endpoint and optional Alertmanager proxy endpoints so an *existing* observability deployment can scrape and consume them. For a ready-to-run example stack you can deploy alongside Manage, see [`docker-compose.observability.yml`](docker-compose.observability.yml) and [`docs/observability-runbooks.md`](docs/observability-runbooks.md).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Local development with hot reload:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
|
|
docker compose -f docker-compose.dev.yml up --build
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Frontend runs on <http://localhost:5173> and the backend on <http://localhost:8000>. Dev compose disables OIDC by default (`AUTH_ENABLED=false`), so you can open it directly without an identity provider.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The backend media index and settings database (including monitoring machines, SSH keys, saved tasks, and dashboard widgets) are persisted in Docker volumes so rebuilds and container restarts do not reset state.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Manual backend/frontend development
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
```bash
|
|
|
|
cd backend
|
|
|
|
cd backend
|
|
|
@@ -71,75 +66,126 @@ pip install -e '.[dev]'
|
|
|
|
uvicorn media_library_viewer_api.main:app --reload --port 8000
|
|
|
|
uvicorn media_library_viewer_api.main:app --reload --port 8000
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### Frontend
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
```bash
|
|
|
|
cd frontend
|
|
|
|
cd frontend
|
|
|
|
npm install
|
|
|
|
npm install
|
|
|
|
npm run dev
|
|
|
|
npm run dev
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## Tests and quality checks
|
|
|
|
## Configuration
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The Compose files use environment-variable interpolation. Export the required variables in your shell or pass them inline; a `.env` file is optional, not required.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Compose examples
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Production-style example with shell exports:
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
```bash
|
|
|
|
# Backend
|
|
|
|
export BACKEND_APP_HOST=api.manage.example.com
|
|
|
|
cd backend
|
|
|
|
export FRONTEND_APP_HOST=manage.example.com
|
|
|
|
ruff check .
|
|
|
|
export CERT_RESOLVER=letsencrypt
|
|
|
|
python -m pytest
|
|
|
|
export VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/
|
|
|
|
|
|
|
|
export VITE_OIDC_CLIENT_ID=manage
|
|
|
|
|
|
|
|
export VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback
|
|
|
|
|
|
|
|
export VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/
|
|
|
|
|
|
|
|
export MANAGE_ENCRYPTION_KEY=$(python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")
|
|
|
|
|
|
|
|
|
|
|
|
# Frontend
|
|
|
|
docker compose up --build
|
|
|
|
cd ../frontend
|
|
|
|
|
|
|
|
npm run lint
|
|
|
|
|
|
|
|
npm run build
|
|
|
|
|
|
|
|
npm run test
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
A focused frontend typecheck can be run with `npx tsc --noEmit` from `frontend/`.
|
|
|
|
> Observability services (Grafana, Prometheus, Alertmanager) are configured in
|
|
|
|
|
|
|
|
> the app on the **Services** page — no env vars for them.
|
|
|
|
|
|
|
|
|
|
|
|
## Configuration and operations
|
|
|
|
Inline one-liner example:
|
|
|
|
|
|
|
|
|
|
|
|
[`.env.example`](.env.example) is a template; do not commit real credentials or encryption keys. The Compose files interpolate environment values directly. Some template entries are for the optional observability example and are not consumed by the normal Manage Compose stack.
|
|
|
|
```bash
|
|
|
|
|
|
|
|
BACKEND_APP_HOST=api.manage.example.com FRONTEND_APP_HOST=manage.example.com CERT_RESOLVER=letsencrypt VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ VITE_OIDC_CLIENT_ID=manage VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ docker compose up --build
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Optional SMTP settings (`SMTP_HOST`, `SMTP_PORT`, `SMTP_USERNAME`, `SMTP_PASSWORD`, `SMTP_FROM_ADDRESS`, `SMTP_FROM_NAME`, `SMTP_USE_TLS`, `SMTP_USE_SSL`, and `SMTP_TIMEOUT`) support the Users message popup.
|
|
|
|
For local development, no SSH key is required unless you want to connect to remote SSH machines later:
|
|
|
|
|
|
|
|
|
|
|
|
### Remote servers
|
|
|
|
```bash
|
|
|
|
|
|
|
|
docker compose -f docker-compose.dev.yml up --build
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
A managed remote server needs a POSIX `/bin/sh`, `python3`, `ffprobe`, `find`, `stat`, `df`, and `awk`. Configure its SSH credentials in Manage's Settings. Unknown SSH host keys are rejected; establish trust first, for example:
|
|
|
|
Example environment variables:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
|
|
# Optional backend logging level
|
|
|
|
|
|
|
|
LOG_LEVEL=INFO
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Optional SMTP settings for the Users -> message popup
|
|
|
|
|
|
|
|
SMTP_HOST=smtp.example.com
|
|
|
|
|
|
|
|
SMTP_PORT=587
|
|
|
|
|
|
|
|
SMTP_USERNAME=your-smtp-username
|
|
|
|
|
|
|
|
SMTP_PASSWORD=your-smtp-password
|
|
|
|
|
|
|
|
SMTP_FROM_ADDRESS=no-reply@example.com
|
|
|
|
|
|
|
|
SMTP_FROM_NAME=Manage
|
|
|
|
|
|
|
|
SMTP_USE_TLS=true
|
|
|
|
|
|
|
|
SMTP_USE_SSL=false
|
|
|
|
|
|
|
|
SMTP_TIMEOUT=30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Jellyfin, Jellyseerr, and SSH targets are now configured per machine in the app's Settings tab.
|
|
|
|
|
|
|
|
# The backend seeds a local machine automatically, so no global Jellyfin or SSH env vars are required.
|
|
|
|
|
|
|
|
#
|
|
|
|
|
|
|
|
# Remote SSH machines can store their private key and optional passphrase directly in Settings,
|
|
|
|
|
|
|
|
# so no SSH key mount is required for normal use.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Authentik / OIDC
|
|
|
|
|
|
|
|
AUTH_ENABLED=true
|
|
|
|
|
|
|
|
OIDC_ISSUER_URL=https://auth.example.com/application/o/manage/
|
|
|
|
|
|
|
|
OIDC_AUDIENCE=manage
|
|
|
|
|
|
|
|
OIDC_JWKS_URL=
|
|
|
|
|
|
|
|
OIDC_CLOCK_SKEW_SECONDS=30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Frontend OIDC settings
|
|
|
|
|
|
|
|
VITE_OIDC_ENABLED=true
|
|
|
|
|
|
|
|
VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/
|
|
|
|
|
|
|
|
VITE_OIDC_CLIENT_ID=manage
|
|
|
|
|
|
|
|
VITE_OIDC_SCOPE=openid profile email
|
|
|
|
|
|
|
|
VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback
|
|
|
|
|
|
|
|
VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Observability services (Grafana, Prometheus, Alertmanager) are configured in
|
|
|
|
|
|
|
|
# the app on the Services page. The only observability env var is the optional
|
|
|
|
|
|
|
|
# PROMETHEUS_ENABLED toggle (defaults on) for Manage's own /metrics endpoint.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Required: master key encrypting service secrets (API keys/tokens) at rest.
|
|
|
|
|
|
|
|
# Generate one with: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
|
|
|
|
|
|
|
|
MANAGE_ENCRYPTION_KEY=replace-with-a-fernet-key
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Remote server requirements
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The remote server needs:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- `/bin/sh` (POSIX shell)
|
|
|
|
|
|
|
|
- `python3`, `ffprobe`, `find`, `stat`, `df`, `awk` for file inspection and job templates
|
|
|
|
|
|
|
|
- SSH access with a key configured in the app's Settings tab
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The SSH client rejects unknown host keys. Connect manually once first:
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
```bash
|
|
|
|
ssh user@host
|
|
|
|
ssh user@host
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
SSH commands run through `/bin/sh -c` regardless of the remote login shell.
|
|
|
|
## Development
|
|
|
|
|
|
|
|
|
|
|
|
### Optional observability example
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[`docker-compose.observability.yml`](docker-compose.observability.yml) is a separate, optional stack for Grafana, Prometheus, Loki, Alertmanager, Alloy, and Node Exporter. It is not required by Manage. Its header documents required `*_ROOT` persistence directories, `CERT_RESOLVER`, and Grafana/Prometheus/Alertmanager host variables. With those prepared, run:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
```bash
|
|
|
|
docker compose -f docker-compose.observability.yml up -d
|
|
|
|
# Backend (lint + tests)
|
|
|
|
|
|
|
|
cd backend && .venv/bin/ruff check . && .venv/bin/python -m pytest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Frontend (lint + typecheck/build + tests)
|
|
|
|
|
|
|
|
cd frontend && npm run lint && npm run build && npm run test
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Set a non-default `GRAFANA_ADMIN_USER` and a strong, secret `GRAFANA_ADMIN_PASSWORD` before deploying this stack. Do not expose the example observability services with their defaults.
|
|
|
|
Focused frontend typecheck: `npx tsc --noEmit`.
|
|
|
|
|
|
|
|
|
|
|
|
See [`docs/observability-runbooks.md`](docs/observability-runbooks.md) for its operational documentation.
|
|
|
|
## Notes
|
|
|
|
|
|
|
|
|
|
|
|
## Repository layout
|
|
|
|
- Jellyfin server root URL required (not `/web`). The client strips trailing `/web` defensively.
|
|
|
|
|
|
|
|
- SSH commands run through `/bin/sh -c` regardless of remote login shell.
|
|
|
|
```text
|
|
|
|
- Job templates are shell-quoted. Add new templates in `backend/src/media_library_viewer_api/jobs.py`.
|
|
|
|
.
|
|
|
|
- Root-level Docker Compose files are provided for production (`docker-compose.yml`) and local development (`docker-compose.dev.yml`), and both rely on Compose interpolation rather than `env_file` entries. They deploy **only** the backend and frontend; Manage never deploys its own observability stack (see `docker-compose.observability.yml` for an optional standalone example).
|
|
|
|
├── backend/ # FastAPI API and tests
|
|
|
|
- The configurable dashboard stores widget instances in the backend SQLite settings database. New installs seed default Jellyfin activity and Backups widgets automatically.
|
|
|
|
├── frontend/ # React/TypeScript SPA and tests
|
|
|
|
- Grafana, Prometheus, and Alertmanager are configured as **service instances** in the app (Services page); their widget adapters resolve URLs from service records, and no observability URLs/credentials live in env vars. No credentials are stored in widget config; service API keys are encrypted at rest with `MANAGE_ENCRYPTION_KEY`. When no alertmanager service is configured, the alert proxy endpoints return graceful "not configured" responses.
|
|
|
|
├── archive/ # Preserved Streamlit prototype
|
|
|
|
|
|
|
|
├── docs/ # Requirements, migration, and operations docs
|
|
|
|
|
|
|
|
├── docker-compose.yml # Traefik-backed production-style stack
|
|
|
|
|
|
|
|
├── docker-compose.dev.yml # Local hot-reload development stack
|
|
|
|
|
|
|
|
└── docker-compose.observability.yml # Optional standalone observability example
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Project documents
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- [Requirements and planning history](docs/REQUIREMENTS.md)
|
|
|
|
|
|
|
|
- [FastAPI + React migration plan](docs/MIGRATION_PLAN.md)
|
|
|
|
|
|
|
|
- [Contributing guide](CONTRIBUTING.md)
|
|
|
|
|
|
|
|
- [MIT license](LICENSE)
|
|
|
|
|
|
|
|
Developer