Compare commits
48 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| b583d5a365 | |||
| 32fa01cc12 | |||
| ac703eecd2 | |||
| d05de0aacd | |||
| 09b9c45665 | |||
| 32516f6e3b | |||
| 30f1b6e6db | |||
| 7808822a55 | |||
| e805c624b2 | |||
| f7b63fead5 | |||
| 2eb649eceb | |||
| 2076ab76fa | |||
| 2e3e7b3850 | |||
| c447dfe68d | |||
| 688a18af22 | |||
| 18ee77a4e4 | |||
| 3d331e4c72 | |||
| eebc86a52b | |||
| 56b919ea1f | |||
| 648320abfd | |||
| 7d252489de | |||
| 04319025de | |||
| 8bc209b27e | |||
| cbc2740e37 | |||
| 6919158012 | |||
| fd12e921fd | |||
| 7107815a5c | |||
| 38b2de54ff | |||
| c1610c93a1 | |||
| b1a66a1ab7 | |||
| b200025daa | |||
| 0c5698c903 | |||
| 14771ae990 | |||
| 7d49df3e7d | |||
| c13e274ca4 | |||
| d4f95b64d4 | |||
| 4d520ab0e3 | |||
| ca8927834e | |||
| a39dbf272c | |||
| 50eb76a10d | |||
| d7ad933b2a | |||
| 8c69911252 | |||
| 7b3e2ebace | |||
| cfb9977532 | |||
| 802a9202e9 | |||
| 7ab9b1ac59 | |||
| cbb703341e | |||
| a13f560df2 |
@@ -0,0 +1,20 @@
|
||||
# .claude (index)
|
||||
dir: .claude
|
||||
|
||||
## role
|
||||
Configuration directory for the Claude AI assistant, storing project-specific settings, instructions, and behavioral guidelines.
|
||||
## parent
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
## children
|
||||
- .claude/skills
|
||||
index: .claude/skills/.pi-map.index.md
|
||||
map: .claude/skills/.pi-map.md
|
||||
## files
|
||||
## links
|
||||
index: .claude/.pi-map.index.md
|
||||
map: .claude/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,18 @@
|
||||
# .claude
|
||||
dir: .claude
|
||||
|
||||
index: .claude/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Configuration directory for the Claude AI assistant, storing project-specific settings, instructions, and behavioral guidelines.
|
||||
## files
|
||||
## arch
|
||||
Flat configuration structure containing markdown/YAML files that define custom commands, project context, and operational rules for Claude's interactions with the codebase.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,20 @@
|
||||
# .claude/skills (index)
|
||||
dir: .claude/skills
|
||||
|
||||
## role
|
||||
Directory containing custom skill definitions and capability instructions for the Claude AI assistant integration.
|
||||
## parent
|
||||
index: .claude/.pi-map.index.md
|
||||
map: .claude/.pi-map.md
|
||||
## children
|
||||
- .claude/skills/sift-backlog
|
||||
index: .claude/skills/sift-backlog/.pi-map.index.md
|
||||
map: .claude/skills/sift-backlog/.pi-map.md
|
||||
## files
|
||||
## links
|
||||
index: .claude/skills/.pi-map.index.md
|
||||
map: .claude/skills/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,18 @@
|
||||
# .claude/skills
|
||||
dir: .claude/skills
|
||||
|
||||
index: .claude/skills/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Directory containing custom skill definitions and capability instructions for the Claude AI assistant integration.
|
||||
## files
|
||||
## arch
|
||||
Flat configuration file structure defining modular skill behaviors and prompts used to extend Claude's domain-specific abilities.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .claude/skills/sift-backlog (index)
|
||||
dir: .claude/skills/sift-backlog
|
||||
|
||||
## role
|
||||
Defines a Claude skill workflow for triaging, organizing, and activating backlog tasks into actionable plans using the `sf` CLI tool.
|
||||
## parent
|
||||
index: .claude/skills/.pi-map.index.md
|
||||
map: .claude/skills/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- SKILL.md
|
||||
## links
|
||||
index: .claude/skills/sift-backlog/.pi-map.index.md
|
||||
map: .claude/skills/sift-backlog/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .claude/skills/sift-backlog
|
||||
dir: .claude/skills/sift-backlog
|
||||
|
||||
index: .claude/skills/sift-backlog/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Defines a Claude skill workflow for triaging, organizing, and activating backlog tasks into actionable plans using the `sf` CLI tool.
|
||||
## files
|
||||
- SKILL.md | Defines a workflow skill for triaging, organizing, and activating backlog tasks into actionable plans using the `sf` CLI tool. | dep: sf CLI (task, plan, dependency, update subcommands)
|
||||
## arch
|
||||
Single-file declarative skill definition following a prompt-driven workflow pattern with structured triage and activation instructions for Claude to execute.
|
||||
## tags
|
||||
skill, defines, workflow, triaging, organizing, activating, backlog, tasks
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -55,5 +55,3 @@ frontend/dist/
|
||||
.superpowers/
|
||||
# Local Pi runtime state
|
||||
.atl/
|
||||
.pi-map.md
|
||||
.pi-map.index.md
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# .opencode (index)
|
||||
dir: .opencode
|
||||
|
||||
## role
|
||||
Configuration directory for the opencode tool, managing project-specific settings and preferences.
|
||||
## parent
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
## children
|
||||
- .opencode/commands
|
||||
index: .opencode/commands/.pi-map.index.md
|
||||
map: .opencode/commands/.pi-map.md
|
||||
- .opencode/skills
|
||||
index: .opencode/skills/.pi-map.index.md
|
||||
map: .opencode/skills/.pi-map.md
|
||||
## files
|
||||
## links
|
||||
index: .opencode/.pi-map.index.md
|
||||
map: .opencode/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,18 @@
|
||||
# .opencode
|
||||
dir: .opencode
|
||||
|
||||
index: .opencode/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Configuration directory for the opencode tool, managing project-specific settings and preferences.
|
||||
## files
|
||||
## arch
|
||||
Flat directory structure containing configuration files that define opencode behavior for the associated project.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,22 @@
|
||||
# .opencode/commands (index)
|
||||
dir: .opencode/commands
|
||||
|
||||
## role
|
||||
Defines slash-command workflows and assistant personas for an OpenSpec-based development process (explore, propose, apply, archive).
|
||||
## parent
|
||||
index: .opencode/.pi-map.index.md
|
||||
map: .opencode/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- opsx-apply.md
|
||||
- opsx-archive.md
|
||||
- opsx-explore.md
|
||||
- opsx-propose.md
|
||||
## links
|
||||
index: .opencode/commands/.pi-map.index.md
|
||||
map: .opencode/commands/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,22 @@
|
||||
# .opencode/commands
|
||||
dir: .opencode/commands
|
||||
|
||||
index: .opencode/commands/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Defines slash-command workflows and assistant personas for an OpenSpec-based development process (explore, propose, apply, archive).
|
||||
## files
|
||||
- opsx-apply.md | Defines a workflow for implementing tasks from an OpenSpec change in a structured, iterative manner with pause points for blockers and ambiguity. | dep: openspec CLI, AskUserQuestion tool, filesystem access
|
||||
- opsx-archive.md | Defines a workflow for archiving completed changes in an experimental openspec-based development process, including validation, spec sync assessment, and user confirmation steps. | dep: openspec CLI, AskUserQuestion tool, Task tool, Skill tool, filesystem (mkdir, mv), tasks.md
|
||||
- opsx-explore.md | Defines the explore mode stance for a thinking/discussion assistant that investigates problems and clarifies requirements without implementing code | dep: OpenSpec system (openspec CLI, change artifacts like proposal.md/design.md/tasks.md/spec.md)
|
||||
- opsx-propose.md | Defines a workflow for creating a new change with all required planning artifacts (proposal, design, tasks) in a single step using the openspec CLI tool. | dep: openspec CLI, AskUserQuestion tool, TodoWrite tool, file system
|
||||
## arch
|
||||
Markdown-based declarative templates serving as structured prompts/playbooks that guide an AI assistant through specific operational phases of a spec-driven lifecycle.
|
||||
## tags
|
||||
opsx, defines, tasks, md, workflow, openspec, openspec cli, askuserquestion tool
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,29 @@
|
||||
# .opencode/skills (index)
|
||||
dir: .opencode/skills
|
||||
|
||||
## role
|
||||
Directory for defining custom agent skills, capabilities, and behavioral instructions within the opencode configuration framework.
|
||||
## parent
|
||||
index: .opencode/.pi-map.index.md
|
||||
map: .opencode/.pi-map.md
|
||||
## children
|
||||
- .opencode/skills/openspec-apply-change
|
||||
index: .opencode/skills/openspec-apply-change/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-apply-change/.pi-map.md
|
||||
- .opencode/skills/openspec-archive-change
|
||||
index: .opencode/skills/openspec-archive-change/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-archive-change/.pi-map.md
|
||||
- .opencode/skills/openspec-explore
|
||||
index: .opencode/skills/openspec-explore/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-explore/.pi-map.md
|
||||
- .opencode/skills/openspec-propose
|
||||
index: .opencode/skills/openspec-propose/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-propose/.pi-map.md
|
||||
## files
|
||||
## links
|
||||
index: .opencode/skills/.pi-map.index.md
|
||||
map: .opencode/skills/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,18 @@
|
||||
# .opencode/skills
|
||||
dir: .opencode/skills
|
||||
|
||||
index: .opencode/skills/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Directory for defining custom agent skills, capabilities, and behavioral instructions within the opencode configuration framework.
|
||||
## files
|
||||
## arch
|
||||
Configuration-based skill definition directory; skills are declared as individual files consumed by the opencode agent runtime to extend or specialize assistant behavior.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-apply-change (index)
|
||||
dir: .opencode/skills/openspec-apply-change
|
||||
|
||||
## role
|
||||
Provides a structured skill definition for implementing OpenSpec changes through a schema-driven workflow with progress tracking.
|
||||
## parent
|
||||
index: .opencode/skills/.pi-map.index.md
|
||||
map: .opencode/skills/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- SKILL.md
|
||||
## links
|
||||
index: .opencode/skills/openspec-apply-change/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-apply-change/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-apply-change
|
||||
dir: .opencode/skills/openspec-apply-change
|
||||
|
||||
index: .opencode/skills/openspec-apply-change/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides a structured skill definition for implementing OpenSpec changes through a schema-driven workflow with progress tracking.
|
||||
## files
|
||||
- SKILL.md | Defines a skill for implementing tasks from an OpenSpec change using a schema-driven workflow with progress tracking and contextual file reading. | dep: openspec CLI, AskUserQuestion tool
|
||||
## arch
|
||||
Documentation-based skill specification using markdown with defined workflow steps, schema references, and contextual file reading rules.
|
||||
## tags
|
||||
skill, defines, implementing, tasks, openspec, change, schema, driven
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-archive-change (index)
|
||||
dir: .opencode/skills/openspec-archive-change
|
||||
|
||||
## role
|
||||
Provides a structured skill definition for archiving completed changes in the openspec experimental workflow with validation and user confirmation steps.
|
||||
## parent
|
||||
index: .opencode/skills/.pi-map.index.md
|
||||
map: .opencode/skills/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- SKILL.md
|
||||
## links
|
||||
index: .opencode/skills/openspec-archive-change/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-archive-change/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-archive-change
|
||||
dir: .opencode/skills/openspec-archive-change
|
||||
|
||||
index: .opencode/skills/openspec-archive-change/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides a structured skill definition for archiving completed changes in the openspec experimental workflow with validation and user confirmation steps.
|
||||
## files
|
||||
- SKILL.md | Defines a skill for archiving a completed change in the openspec experimental workflow, including validation, sync assessment, and user confirmation steps. | dep: openspec CLI, AskUserQuestion tool, Task tool (subagent_type: general-purpose), openspec-sync-specs skill
|
||||
## arch
|
||||
Single-document declarative skill specification following a procedural checklist pattern (validate, assess sync, confirm) designed for an AI agent to execute.
|
||||
## tags
|
||||
skill, openspec, sync, defines, archiving, completed, change, experimental
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-explore (index)
|
||||
dir: .opencode/skills/openspec-explore
|
||||
|
||||
## role
|
||||
Provides a conversational "explore mode" skill for the OpenSpec CLI that serves as a thinking partner for brainstorming ideas, investigating problems, and clarifying requirements.
|
||||
## parent
|
||||
index: .opencode/skills/.pi-map.index.md
|
||||
map: .opencode/skills/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- SKILL.md
|
||||
## links
|
||||
index: .opencode/skills/openspec-explore/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-explore/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-explore
|
||||
dir: .opencode/skills/openspec-explore
|
||||
|
||||
index: .opencode/skills/openspec-explore/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides a conversational "explore mode" skill for the OpenSpec CLI that serves as a thinking partner for brainstorming ideas, investigating problems, and clarifying requirements.
|
||||
## files
|
||||
- SKILL.md | Defines a conversational "explore mode" skill for the OpenSpec CLI that acts as a thinking partner for exploring ideas, investigating problems, and clarifying requirements without implementing code. | dep: openspec CLI
|
||||
## arch
|
||||
Skill-definition pattern using a single Markdown file (SKILL.md) that declaratively specifies the assistant's behavioral constraints, workflow, and operational guidelines.
|
||||
## tags
|
||||
skill, defines, conversational, explore, mode, openspec, cli, acts
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-propose (index)
|
||||
dir: .opencode/skills/openspec-propose
|
||||
|
||||
## role
|
||||
Provides an AI assistant skill that automates the openspec proposal workflow by scaffolding directories and generating structured artifacts (proposals, designs, tasks).
|
||||
## parent
|
||||
index: .opencode/skills/.pi-map.index.md
|
||||
map: .opencode/skills/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- SKILL.md
|
||||
## links
|
||||
index: .opencode/skills/openspec-propose/.pi-map.index.md
|
||||
map: .opencode/skills/openspec-propose/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# .opencode/skills/openspec-propose
|
||||
dir: .opencode/skills/openspec-propose
|
||||
|
||||
index: .opencode/skills/openspec-propose/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides an AI assistant skill that automates the openspec proposal workflow by scaffolding directories and generating structured artifacts (proposals, designs, tasks).
|
||||
## files
|
||||
- SKILL.md | Defines an AI assistant skill that automates proposing new changes by scaffolding a directory, generating dependent artifacts (proposal, design, tasks), and tracking progress through a structured workflow using the openspec CLI. | dep: openspec CLI, AskUserQuestion tool, TodoWrite tool
|
||||
## arch
|
||||
Skill-definition pattern using a declarative markdown document (SKILL.md) that encodes a step-by-step procedural workflow with CLI integration conventions.
|
||||
## tags
|
||||
skill, defines, assistant, automates, proposing, new, changes, scaffolding
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,77 @@
|
||||
# . (index)
|
||||
dir: .
|
||||
|
||||
## Project Map Protocol
|
||||
|
||||
1. Read this protocol and the root `.pi-map.index.md` first.
|
||||
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
|
||||
3. Load indexes before rich maps during task-start navigation.
|
||||
4. Read the local rich map and actual source before editing.
|
||||
5. Treat non-empty `## dirty` sections in either artifact as stale.
|
||||
6. If source and generated artifacts disagree, trust source.
|
||||
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
|
||||
8. After editing source, run `project_map_patch` for each changed file.
|
||||
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
|
||||
|
||||
Trust boundary: index routes, map orients, source decides.
|
||||
|
||||
## role
|
||||
Root project configuration and orchestration package for a media library management application with observability, defining Docker deployment stacks, environment templates, and project documentation.
|
||||
## parent
|
||||
-
|
||||
## children
|
||||
- .atl
|
||||
index: .atl/.pi-map.index.md
|
||||
map: .atl/.pi-map.md
|
||||
- .claude
|
||||
index: .claude/.pi-map.index.md
|
||||
map: .claude/.pi-map.md
|
||||
- .opencode
|
||||
index: .opencode/.pi-map.index.md
|
||||
map: .opencode/.pi-map.md
|
||||
- .pi
|
||||
index: .pi/.pi-map.index.md
|
||||
map: .pi/.pi-map.md
|
||||
- .ruff_cache
|
||||
index: .ruff_cache/.pi-map.index.md
|
||||
map: .ruff_cache/.pi-map.md
|
||||
- archive
|
||||
index: archive/.pi-map.index.md
|
||||
map: archive/.pi-map.md
|
||||
- backend
|
||||
index: backend/.pi-map.index.md
|
||||
map: backend/.pi-map.md
|
||||
- docs
|
||||
index: docs/.pi-map.index.md
|
||||
map: docs/.pi-map.md
|
||||
- frontend
|
||||
index: frontend/.pi-map.index.md
|
||||
map: frontend/.pi-map.md
|
||||
- monitoring
|
||||
index: monitoring/.pi-map.index.md
|
||||
map: monitoring/.pi-map.md
|
||||
- openspec
|
||||
index: openspec/.pi-map.index.md
|
||||
map: openspec/.pi-map.md
|
||||
## files
|
||||
- .dockerignore
|
||||
- .env.example
|
||||
- .gitignore
|
||||
- AGENTS.md
|
||||
- CHANGELOG.md
|
||||
- CONTRIBUTING.md
|
||||
- LICENSE
|
||||
- README.md
|
||||
- context.md
|
||||
- docker-compose.dev.yml
|
||||
- docker-compose.observability.yml
|
||||
- docker-compose.yml
|
||||
- swap-pane
|
||||
- token-usage-output.txt
|
||||
## links
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# .
|
||||
dir: .
|
||||
|
||||
index: ./.pi-map.index.md
|
||||
|
||||
## Project Map Protocol
|
||||
|
||||
1. Read this protocol and the root `.pi-map.index.md` first.
|
||||
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
|
||||
3. Load indexes before rich maps during task-start navigation.
|
||||
4. Read the local rich map and actual source before editing.
|
||||
5. Treat non-empty `## dirty` sections in either artifact as stale.
|
||||
6. If source and generated artifacts disagree, trust source.
|
||||
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
|
||||
8. After editing source, run `project_map_patch` for each changed file.
|
||||
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
|
||||
|
||||
Trust boundary: index routes, map orients, source decides.
|
||||
|
||||
## role
|
||||
Root project configuration and orchestration package for a media library management application with observability, defining Docker deployment stacks, environment templates, and project documentation.
|
||||
## files
|
||||
- .dockerignore | Specifies files and directories to exclude from Docker build context to reduce image size and improve build performance | dep: Docker
|
||||
- .env.example | Provides a template of environment variables for configuring application hosts, backend settings, OIDC authentication, SMTP, Grafana, and alerting across a Docker Compose deployment.
|
||||
- .gitignore | Configures Git to ignore Python artifacts, virtual environments, secrets, editor files, frontend builds, and tool-specific metadata from version control.
|
||||
- AGENTS.md | Provides project-specific guidance for AI agents working on a media library viewer application with FastAPI backend and Vite React frontend | dep: FastAPI, Vite, React, Docker Compose, uvicorn, pytest, Ruff, TypeScript, Python 3.11
|
||||
- CHANGELOG.md | Documents notable changes, breaking changes, and migration steps for the Manage application across recent versions.
|
||||
- CONTRIBUTING.md | Provides contribution guidelines and setup instructions for the Manage project's backend (FastAPI) and frontend (React) codebases. | dep: FastAPI, React, Vite, TypeScript, Ruff, pytest, Docker Compose, Tailwind CSS, TanStack Query
|
||||
- LICENSE | Provides the MIT open-source software license terms for the project
|
||||
- README.md | Project README documenting a media and server operations tool with Jellyfin integration, SSH file inspection, and server monitoring capabilities. | dep: FastAPI, React, TypeScript, Docker Compose, SQLite, Traefik, OIDC/Authentik, Jellyfin, Prometheus, Grafana, Alertmanager
|
||||
- context.md | Documentation file providing a historical and architectural overview of an observability stack (Prometheus, Grafana, Loki, Alertmanager) for a containerized media management application. | dep: Prometheus, Grafana, Loki, Alertmanager, Grafana Alloy, Node Exporter, Docker Compose, FastAPI
|
||||
- docker-compose.dev.yml | Defines a development Docker Compose stack for a backend (FastAPI/Uvicorn) and frontend (Vite) application with hot-reload and disabled authentication. | dep: uvicorn, Docker
|
||||
- docker-compose.observability.yml | Defines an optional standalone Docker Compose observability stack with Prometheus, Loki, Grafana, Alertmanager, Alloy, and Node Exporter for monitoring hosts without the main Manage application. | dep: prom/prometheus, grafana/loki, grafana/alloy, grafana/grafana, prom/alertmanager, prom/node-exporter, Traefik
|
||||
- docker-compose.yml | Defines a production Docker Compose stack for a backend-frontend application with OIDC authentication, Traefik routing, TLS, and Prometheus metrics exposure. | dep: Traefik, OIDC provider, Docker, Vite, external observability stack
|
||||
- swap-pane | Swaps the position of two tmux panes within a window or between windows | dep: tmux, sh
|
||||
- token-usage-output.txt | Displays a detailed token usage and cost analysis report for an AI coding session, including breakdowns by category, tool usage, cache efficiency, subagent costs, and pricing comparisons.
|
||||
## arch
|
||||
Containerized full-stack architecture using Docker Compose for orchestration, Traefik for production routing/TLS, dual dev/production environments, and an optional standalone observability stack (Prometheus/Grafana/Loki/Alertmanager).
|
||||
## tags
|
||||
docker, grafana, application, fastapi, compose, prometheus, backend, frontend
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
+75
-3
@@ -4,6 +4,48 @@ All notable changes to Manage. Breaking changes are marked with **BREAKING**.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added — Observability service registry
|
||||
|
||||
- **Alertmanager is now a service type.** Configure Alertmanager, Grafana, and
|
||||
Prometheus instances in the UI on the Services page; all three are first-class
|
||||
service-registry entries with dashboard widgets (`active_alerts`, Grafana link,
|
||||
Prometheus metric).
|
||||
- New monitoring endpoints resolve the configured service instance and probe its
|
||||
health: `GET /api/monitoring/grafana-status`, `/prometheus-status`. The
|
||||
`/alerts` and `/alertmanager-status` endpoints now take an optional
|
||||
`service_id` and pick the first enabled alertmanager instance by default.
|
||||
- The Observability page discovers Grafana/Prometheus/Alertmanager from the
|
||||
registry and renders health cards; the dashboard `active_alerts` widget sums
|
||||
firing alerts by severity.
|
||||
|
||||
### Changed — Observability is now external only
|
||||
|
||||
- **Removed** all observability services from `docker-compose.yml` and
|
||||
`docker-compose.dev.yml`. They now deploy **only** the backend and frontend.
|
||||
The `monitoring` network and the `prometheus`/`loki`/`alloy`/`grafana`/
|
||||
`alertmanager`/`node-exporter` services and their named volumes were deleted,
|
||||
and the `GRAFANA_APP_HOST` Traefik rule was removed.
|
||||
- Manage now connects to **existing** Grafana/Prometheus/Alertmanager instances
|
||||
and never ships its own stack. The previous in-compose stack is preserved as
|
||||
an optional, deploy-it-yourself example in `docker-compose.observability.yml`
|
||||
(config under `monitoring/`, documented in `docs/observability-runbooks.md`).
|
||||
- Removed the now-orphaned combined `monitoring/prometheus/prometheus.yml`; the
|
||||
standalone stack uses `monitoring/prometheus/prometheus.standalone.yml`.
|
||||
- Removed the Prometheus file-SD bridge (`PROMETHEUS_FILE_SD_DIR` + the
|
||||
`write_prometheus_targets` file writer). External Prometheus instances now
|
||||
consume node-exporter targets via `http_sd_configs` against
|
||||
`GET /api/monitoring/prometheus-targets`. The webhook receiver is log-only.
|
||||
|
||||
### **BREAKING**
|
||||
|
||||
- Observability is configured entirely via the service registry; the backend
|
||||
`alertmanager_url`/`alertmanager_webhook_url` and frontend
|
||||
`VITE_GRAFANA_URL`/`VITE_PROMETHEUS_URL` environment variables, plus
|
||||
`PROMETHEUS_FILE_SD_DIR`, were **removed**. Re-create your Alertmanager /
|
||||
Grafana / Prometheus instances on the Services page after upgrading. The only
|
||||
observability env var remaining is `PROMETHEUS_ENABLED` (toggles Manage's own
|
||||
`/metrics` endpoint).
|
||||
|
||||
### Added — Service registry
|
||||
|
||||
- Runtime **service registry** persisted in the backend SQLite database. External
|
||||
@@ -32,6 +74,12 @@ All notable changes to Manage. Breaking changes are marked with **BREAKING**.
|
||||
|
||||
### **BREAKING**
|
||||
|
||||
- Saved Actions (server tasks) now target `ssh_tasks` service instances instead
|
||||
of monitoring machines. The `default_machine_id` field on saved tasks was
|
||||
replaced with `default_service_id`; the legacy `saved_task_runs` table was
|
||||
dropped and run history now lives in `service_task_runs`. Re-create SSH task
|
||||
runner services on the Services page and re-link saved actions after
|
||||
upgrading.
|
||||
- **`MANAGE_ENCRYPTION_KEY` is now required** to start the backend. Generate one
|
||||
with:
|
||||
|
||||
@@ -51,6 +99,30 @@ All notable changes to Manage. Breaking changes are marked with **BREAKING**.
|
||||
|
||||
### Notes / follow-ups
|
||||
|
||||
- Machine-level Jellyfin/Jellyseerr app config still powers the Media/Users/Files
|
||||
pages. Migrating those onto the service registry is a separate follow-up change
|
||||
(see `openspec/changes/service-registry/design.md` §12.5).
|
||||
- ~~Machine-level Jellyfin/Jellyseerr app config still powers the Media/Users/Files
|
||||
pages. Migrating those onto the service registry is a separate follow-up change.~~
|
||||
**Done (2026-06-23):** Jellyfin is no longer a machine service, and the dead
|
||||
machine-level `media_root`/`path_prefix` fields were removed. See the
|
||||
Jellyfin migration entry in `docs/REQUIREMENTS.md`.
|
||||
|
||||
## Follow-up #2 — remove dead machine `media_root`/`path_prefix` + Jellyfin service
|
||||
|
||||
Completes the Jellyfin migration onto the service registry. Jellyfin is no
|
||||
longer a machine `services` tag (`DEFAULT_SERVICES` is now `["monitoring",
|
||||
"files"]`), and the dead machine-level `media_root`/`path_prefix` fields were
|
||||
removed from the settings store, `MonitoringMachineInput`, frontend types, and
|
||||
the Settings UI. Jellyfin is configured exclusively as a service-registry
|
||||
instance. The global `REMOTE_MEDIA_ROOT`/`REMOTE_PATH_PREFIX` config properties
|
||||
and `path_utils.py` remain (files/media-index still use them for Jellyfin→SSH
|
||||
path resolution). Existing DB rows may still carry these keys in `config_json`;
|
||||
they are inert and get dropped on the next machine save.
|
||||
|
||||
## Follow-up #1 — remove dead machine Jellyfin/Jellyseerr fields
|
||||
|
||||
With Jellyfin/Jellyseerr now resolved from the service registry, the machine-level
|
||||
Jellyfin/Jellyseerr fields are dead config. Removed from `dependencies.py` (dead
|
||||
`_jellyseerr_client_for`; `_resolve_machine` simplified to SSH-only),
|
||||
`services/settings_store.py`, `routers/settings.py` (`MachineInput`), frontend
|
||||
types, the `Settings.tsx` form, and frontend test fixtures. Existing DB rows may
|
||||
still carry these keys in `config_json`; they are inert and get dropped on the
|
||||
next machine save. No data migration required.
|
||||
|
||||
+77
-20
@@ -1,55 +1,114 @@
|
||||
# Contributing
|
||||
|
||||
Thanks for considering a contribution.
|
||||
Thanks for considering a contribution to Manage.
|
||||
|
||||
Manage is a media and server-operations dashboard built from two subprojects:
|
||||
|
||||
- **`backend/`** — FastAPI (Python 3.11) REST API using a `src/` layout.
|
||||
- **`frontend/`** — Vite + React + TypeScript SPA.
|
||||
- **`archive/`** — the original Streamlit prototype, preserved for reference only. Do **not** use it as a guide; the app is FastAPI + React now.
|
||||
|
||||
The authoritative contributor quick-reference is [`AGENTS.md`](./AGENTS.md). This document mirrors it for human contributors.
|
||||
|
||||
## Setup
|
||||
|
||||
### Backend
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e '.[dev]'
|
||||
```
|
||||
|
||||
Copy env template:
|
||||
### Frontend
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
cd frontend
|
||||
npm install
|
||||
```
|
||||
|
||||
Then set real values in `.env` and run:
|
||||
### Local stack (optional)
|
||||
|
||||
For a full local dev stack with hot reload (auth disabled):
|
||||
|
||||
```bash
|
||||
streamlit run app.py
|
||||
docker compose -f docker-compose.dev.yml up --build
|
||||
```
|
||||
|
||||
## Development guidelines
|
||||
The dev compose deploys only the backend and frontend; Manage never deploys an
|
||||
observability stack. For the optional standalone observability example, see
|
||||
`docker-compose.observability.yml` and `docs/observability-runbooks.md`.
|
||||
|
||||
## Development commands
|
||||
|
||||
Run backend checks from `backend/` and frontend checks from `frontend/`.
|
||||
|
||||
```bash
|
||||
# Backend: lint + tests
|
||||
cd backend && ruff check . && python -m pytest
|
||||
|
||||
# Run the API locally (if the package is installed as above)
|
||||
uvicorn media_library_viewer_api.main:app --reload --port 8000
|
||||
# Otherwise, without installing: PYTHONPATH=src uvicorn media_library_viewer_api.main:app --reload --port 8000
|
||||
|
||||
# Focused backend tests
|
||||
pytest tests/test_api.py
|
||||
pytest -k <expr>
|
||||
```
|
||||
|
||||
```bash
|
||||
# Frontend: dev server (proxies /api to http://localhost:8000)
|
||||
cd frontend && npm run dev
|
||||
|
||||
# Frontend: lint + typecheck/build (build runs tsc -b + vite build) + tests
|
||||
npm run lint
|
||||
npm run build
|
||||
npm run test
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
|
||||
- Keep architecture boundaries clear:
|
||||
- `clients/` for external integrations
|
||||
- `domain/` for normalization/business logic
|
||||
- `services/` for app services/indexing
|
||||
- `ui/` for Streamlit rendering
|
||||
- `clients/` for external service transports (Jellyfin, Jellyseerr, SSH, local shell).
|
||||
- `integrations/` for service-registry definitions (config schema, secrets, widget kinds).
|
||||
- `domain/` for normalization/business logic.
|
||||
- `services/` for app services, indexing, persistence, and background workers.
|
||||
- `routers/` for FastAPI route handlers.
|
||||
- `models/` for Pydantic request/response schemas.
|
||||
- Prefer small, focused functions and explicit names.
|
||||
- Preserve safe SSH behavior and shell quoting — job templates must quote all interpolated values.
|
||||
- External services (Jellyfin, Grafana, Prometheus, Alertmanager, …) are configured at runtime via the **service registry** in the UI, not environment variables. The only observability env var is `PROMETHEUS_ENABLED` (Manage's own `/metrics` toggle).
|
||||
- Avoid introducing optional fallback paths unless required.
|
||||
- Prefer small, focused functions and explicit session-state keys.
|
||||
- Preserve safe SSH behavior and path quoting.
|
||||
|
||||
## Validation
|
||||
### Backend style
|
||||
|
||||
Before opening a merge request, run:
|
||||
Backend linting/format is Ruff (line length 120, Python 3.11); config lives in `backend/pyproject.toml`.
|
||||
|
||||
### Frontend style
|
||||
|
||||
The frontend uses **shadcn/ui + Tailwind CSS v4 + lucide-react + TanStack Query + TanStack Table**. Do not introduce MUI, Emotion, recharts, d3, or AG Grid — those were removed and are not coming back.
|
||||
|
||||
## Validation before opening a merge request
|
||||
|
||||
Before opening a merge request, run and ensure green:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=src python -m py_compile app.py src/media_library_viewer/*.py src/media_library_viewer/clients/*.py src/media_library_viewer/domain/*.py src/media_library_viewer/services/*.py src/media_library_viewer/ui/*.py
|
||||
cd backend && ruff check . && python -m pytest
|
||||
cd frontend && npm run lint && npm run build && npm run test
|
||||
```
|
||||
|
||||
If behavior, UX, or architecture changed, also update `docs/REQUIREMENTS.md`.
|
||||
|
||||
## Security / secrets
|
||||
|
||||
Never commit:
|
||||
|
||||
- `.env`
|
||||
- `.streamlit/secrets.toml`
|
||||
- private keys or API tokens
|
||||
- service secrets
|
||||
|
||||
Use `.env.example` for documented placeholders only.
|
||||
Service secrets are encrypted at rest with `MANAGE_ENCRYPTION_KEY` (required to start the backend). Use `.env.example` for documented placeholders only.
|
||||
|
||||
## Pull requests
|
||||
|
||||
@@ -57,6 +116,4 @@ Please include:
|
||||
|
||||
- what changed
|
||||
- why it changed
|
||||
- how it was tested
|
||||
|
||||
If behavior/requirements changed, also update `docs/REQUIREMENTS.md`.
|
||||
- how it was tested (commands run / tests added)
|
||||
|
||||
@@ -20,15 +20,15 @@ The project consists of two subprojects:
|
||||
|
||||
## Features
|
||||
|
||||
- Configurable dashboard with persisted widgets (Jellyfin activity, backups summary, Grafana deep-links, Prometheus metrics, SSH task output, static text) and shortcuts
|
||||
- Configurable dashboard with persisted widgets (Jellyfin activity, backups summary, Grafana deep-links, Prometheus metrics, Alertmanager alerts, SSH task output, static text) and shortcuts
|
||||
- Thin-dashboard observability: Alertmanager alerts, Prometheus target health, machine status, and Grafana deep-links (no in-app charting)
|
||||
- Per-machine settings for Jellyfin, Jellyseerr, SSH, and monitoring targets
|
||||
- Service registry: configure Jellyfin, Jellyseerr, Alertmanager, Grafana, Prometheus, Nextcloud, and SSH task runner instances in the UI
|
||||
- Per-machine settings for SSH, monitoring targets, and file browsing
|
||||
- SQLite-indexed media table with full-library sort/filter
|
||||
- Read-only Users tab with Jellyfin as the base source and optional Jellyseerr enrichment
|
||||
- Remote file browser with ffprobe preview and job execution
|
||||
- Jellyfin API integration for library metadata and user identity data
|
||||
- SSH-based file inspection and safe remote job templates
|
||||
- Addon pages for Grafana, Prometheus, and SSH tasks at `/addons/:addonId`
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -44,6 +44,8 @@ Open the app at <http://localhost:8080>.
|
||||
|
||||
The production Compose file requires OIDC and Traefik variables; see [Configuration](#configuration) below. Copy `.env.example` to `.env`, fill in the required values, and export them in your shell before running `docker compose up`.
|
||||
|
||||
> **Observability is external.** Manage only ships its **backend** and **frontend**. It does **not** deploy Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter. The backend exposes a `/metrics` endpoint and optional Alertmanager proxy endpoints so an *existing* observability deployment can scrape and consume them. For a ready-to-run example stack you can deploy alongside Manage, see [`docker-compose.observability.yml`](docker-compose.observability.yml) and [`docs/observability-runbooks.md`](docs/observability-runbooks.md).
|
||||
|
||||
Local development with hot reload:
|
||||
|
||||
```bash
|
||||
@@ -81,23 +83,23 @@ Production-style example with shell exports:
|
||||
```bash
|
||||
export BACKEND_APP_HOST=api.manage.example.com
|
||||
export FRONTEND_APP_HOST=manage.example.com
|
||||
export GRAFANA_APP_HOST=grafana.manage.example.com
|
||||
export CERT_RESOLVER=letsencrypt
|
||||
export VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/
|
||||
export VITE_OIDC_CLIENT_ID=manage
|
||||
export VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback
|
||||
export VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/
|
||||
export VITE_GRAFANA_URL=https://grafana.manage.example.com
|
||||
export VITE_PROMETHEUS_URL=https://prometheus.manage.example.com
|
||||
export MANAGE_ENCRYPTION_KEY=$(python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")
|
||||
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
> Observability services (Grafana, Prometheus, Alertmanager) are configured in
|
||||
> the app on the **Services** page — no env vars for them.
|
||||
|
||||
Inline one-liner example:
|
||||
|
||||
```bash
|
||||
BACKEND_APP_HOST=api.manage.example.com FRONTEND_APP_HOST=manage.example.com GRAFANA_APP_HOST=grafana.manage.example.com CERT_RESOLVER=letsencrypt VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ VITE_OIDC_CLIENT_ID=manage VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ VITE_GRAFANA_URL=https://grafana.manage.example.com VITE_PROMETHEUS_URL=https://prometheus.manage.example.com docker compose up --build
|
||||
BACKEND_APP_HOST=api.manage.example.com FRONTEND_APP_HOST=manage.example.com CERT_RESOLVER=letsencrypt VITE_OIDC_ISSUER=https://auth.example.com/application/o/manage/ VITE_OIDC_CLIENT_ID=manage VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/ docker compose up --build
|
||||
```
|
||||
|
||||
For local development, no SSH key is required unless you want to connect to remote SSH machines later:
|
||||
@@ -144,9 +146,9 @@ VITE_OIDC_SCOPE=openid profile email
|
||||
VITE_OIDC_REDIRECT_URI=https://manage.example.com/oidc/callback
|
||||
VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/
|
||||
|
||||
# Grafana / Prometheus public URLs for frontend deep-links (service adapters read URLs from service records)
|
||||
VITE_GRAFANA_URL=https://grafana.manage.example.com
|
||||
VITE_PROMETHEUS_URL=https://prometheus.manage.example.com
|
||||
# Observability services (Grafana, Prometheus, Alertmanager) are configured in
|
||||
# the app on the Services page. The only observability env var is the optional
|
||||
# PROMETHEUS_ENABLED toggle (defaults on) for Manage's own /metrics endpoint.
|
||||
|
||||
# Required: master key encrypting service secrets (API keys/tokens) at rest.
|
||||
# Generate one with: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
|
||||
@@ -170,18 +172,20 @@ ssh user@host
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Backend
|
||||
cd backend && PYTHONPATH=src python -m py_compile src/media_library_viewer_api/main.py
|
||||
# Backend (lint + tests)
|
||||
cd backend && .venv/bin/ruff check . && .venv/bin/python -m pytest
|
||||
|
||||
# Frontend
|
||||
cd frontend && npx tsc --noEmit && npm run build
|
||||
# Frontend (lint + typecheck/build + tests)
|
||||
cd frontend && npm run lint && npm run build && npm run test
|
||||
```
|
||||
|
||||
Focused frontend typecheck: `npx tsc --noEmit`.
|
||||
|
||||
## Notes
|
||||
|
||||
- Jellyfin server root URL required (not `/web`). The client strips trailing `/web` defensively.
|
||||
- SSH commands run through `/bin/sh -c` regardless of remote login shell.
|
||||
- Job templates are shell-quoted. Add new templates in `backend/src/media_library_viewer_api/jobs.py`.
|
||||
- Root-level Docker Compose files are provided for production (`docker-compose.yml`) and local development (`docker-compose.dev.yml`), and both rely on Compose interpolation rather than `env_file` entries.
|
||||
- Root-level Docker Compose files are provided for production (`docker-compose.yml`) and local development (`docker-compose.dev.yml`), and both rely on Compose interpolation rather than `env_file` entries. They deploy **only** the backend and frontend; Manage never deploys its own observability stack (see `docker-compose.observability.yml` for an optional standalone example).
|
||||
- The configurable dashboard stores widget instances in the backend SQLite settings database. New installs seed default Jellyfin activity and Backups widgets automatically.
|
||||
- Grafana and Prometheus widget adapters resolve URLs from service records configured in the app; `VITE_GRAFANA_URL` / `VITE_PROMETHEUS_URL` are only used for frontend deep-links. No credentials are stored in widget config; service API keys are encrypted at rest with `MANAGE_ENCRYPTION_KEY`.
|
||||
- Grafana, Prometheus, and Alertmanager are configured as **service instances** in the app (Services page); their widget adapters resolve URLs from service records, and no observability URLs/credentials live in env vars. No credentials are stored in widget config; service API keys are encrypted at rest with `MANAGE_ENCRYPTION_KEY`. When no alertmanager service is configured, the alert proxy endpoints return graceful "not configured" responses.
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
# archive (index)
|
||||
dir: archive
|
||||
|
||||
## role
|
||||
Archive of an earlier project structure for a Streamlit-based Jellyfin media library browser with SSH remote file inspection capabilities.
|
||||
## parent
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
## children
|
||||
- archive/src
|
||||
index: archive/src/.pi-map.index.md
|
||||
map: archive/src/.pi-map.md
|
||||
- archive/tests
|
||||
index: archive/tests/.pi-map.index.md
|
||||
map: archive/tests/.pi-map.md
|
||||
## files
|
||||
- app.py
|
||||
- pyproject.toml
|
||||
- requirements.txt
|
||||
## links
|
||||
index: archive/.pi-map.index.md
|
||||
map: archive/.pi-map.md
|
||||
## workflows
|
||||
- change archive behavior
|
||||
read: app.py, pyproject.toml, requirements.txt
|
||||
- change archive config
|
||||
read: pyproject.toml
|
||||
- explore archive subdirectories
|
||||
index: archive/src/.pi-map.index.md, archive/tests/.pi-map.index.md
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,26 @@
|
||||
# archive
|
||||
dir: archive
|
||||
|
||||
index: archive/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Archive of an earlier project structure for a Streamlit-based Jellyfin media library browser with SSH remote file inspection capabilities.
|
||||
## files
|
||||
- app.py | Provides a minimal Streamlit entrypoint that adds the src directory to Python's path and delegates to the actual application in media_library_viewer.app. | dep: sys, pathlib, media_library_viewer.app
|
||||
- pyproject.toml | Defines Python package metadata, dependencies, and tool configurations for a Streamlit-based Jellyfin media library browser with SSH remote file inspection. | dep: hatchling, streamlit, streamlit-aggrid, requests, paramiko, python-dotenv, pandas, ruff, pytest
|
||||
- requirements.txt | Installs the current package in editable/development mode using pip | dep: pip, setuptools
|
||||
## arch
|
||||
Thin entrypoint pattern using a bootstrap app.py that manipulates sys.path to delegate execution to a nested media_library_viewer package, managed via standard Python packaging (pyproject.toml).
|
||||
## tags
|
||||
streamlit, app, python, media, library, package, pyproject, pip
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
- change archive behavior
|
||||
read: app.py, pyproject.toml, requirements.txt
|
||||
- change archive config
|
||||
read: pyproject.toml
|
||||
- explore archive subdirectories
|
||||
index: archive/src/.pi-map.index.md, archive/tests/.pi-map.index.md
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,20 @@
|
||||
# archive/src (index)
|
||||
dir: archive/src
|
||||
|
||||
## role
|
||||
No files provided — directory appears to be empty or contents were not included, so the package's role cannot be determined.
|
||||
## parent
|
||||
index: archive/.pi-map.index.md
|
||||
map: archive/.pi-map.md
|
||||
## children
|
||||
- archive/src/media_library_viewer
|
||||
index: archive/src/media_library_viewer/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/.pi-map.md
|
||||
## files
|
||||
## links
|
||||
index: archive/src/.pi-map.index.md
|
||||
map: archive/src/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,18 @@
|
||||
# archive/src
|
||||
dir: archive/src
|
||||
|
||||
index: archive/src/.pi-map.index.md
|
||||
|
||||
## role
|
||||
No files provided — directory appears to be empty or contents were not included, so the package's role cannot be determined.
|
||||
## files
|
||||
## arch
|
||||
Cannot be assessed due to missing file contents; please provide the file listing for analysis.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,39 @@
|
||||
# archive/src/media_library_viewer (index)
|
||||
dir: archive/src/media_library_viewer
|
||||
|
||||
## role
|
||||
Streamlit-based media library viewer that provides a unified dashboard for browsing and monitoring Jellyfin media alongside remote SSH file systems.
|
||||
## parent
|
||||
index: archive/src/.pi-map.index.md
|
||||
map: archive/src/.pi-map.md
|
||||
## children
|
||||
- archive/src/media_library_viewer/clients
|
||||
index: archive/src/media_library_viewer/clients/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/clients/.pi-map.md
|
||||
- archive/src/media_library_viewer/domain
|
||||
index: archive/src/media_library_viewer/domain/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/domain/.pi-map.md
|
||||
- archive/src/media_library_viewer/services
|
||||
index: archive/src/media_library_viewer/services/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/services/.pi-map.md
|
||||
- archive/src/media_library_viewer/ui
|
||||
index: archive/src/media_library_viewer/ui/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/ui/.pi-map.md
|
||||
## files
|
||||
- __init__.py
|
||||
- app.py
|
||||
- config.py
|
||||
- jobs.py
|
||||
- utils.py
|
||||
## links
|
||||
index: archive/src/media_library_viewer/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/.pi-map.md
|
||||
## workflows
|
||||
- change media_library_viewer behavior
|
||||
read: __init__.py, app.py, config.py
|
||||
- change media_library_viewer config
|
||||
read: config.py
|
||||
- explore media_library_viewer subdirectories
|
||||
index: archive/src/media_library_viewer/clients/.pi-map.index.md, archive/src/media_library_viewer/domain/.pi-map.index.md, archive/src/media_library_viewer/services/.pi-map.index.md
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,35 @@
|
||||
# archive/src/media_library_viewer
|
||||
dir: archive/src/media_library_viewer
|
||||
|
||||
index: archive/src/media_library_viewer/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Streamlit-based media library viewer that provides a unified dashboard for browsing and monitoring Jellyfin media alongside remote SSH file systems.
|
||||
## files
|
||||
- __init__.py | Package initialization file that defines the Media Library Viewer package metadata and exports the version string.
|
||||
- app.py | Streamlit UI entrypoint for a Media Library Viewer that connects to Jellyfin and SSH backends, providing dashboard, monitoring, media browsing, and file browser tabs with cached data and path resolution between systems. | exp: func:get_jellyfin_client(base_url: str, api_key: str) → JellyfinClient, call:JellyfinClient, func:cached_users(base_url: str, api_key: str), call:get_jellyfin_client(base_url, api_key).users, func:get_ssh_client(host: str, username: str, port: int, key_filename: str, password: str) → RemoteSSHClient, call:RemoteSSHClient, call:client.connect, func:cached_libraries(base_url: str, api_key: str, user_id: str), call:get_jellyfin_client(base_url, api_key).libraries, func:cached_media_counts(base_url: str, api_key: str, user_id: str), call:get_jellyfin_client(base_url, api_key).media_counts, func:cached_library_counts(base_url: str, api_key: str, user_id: str), call:get_jellyfin_client, call:client.libraries, call:client.library_item_counts, func:cached_active_sessions(base_url: str, api_key: str), call:get_jellyfin_client(base_url, api_key).active_sessions, func:cached_dir_listing(host: str, username: str, port: int, key_filename: str, password: str, path: str), call:get_ssh_client, call:ssh.list_dir, call:json.loads, raise:RuntimeError, func:cached_ffprobe_preview(host: str, username: str, port: int, key_filename: str, password: str, path: str), call:get_ssh_client, call:ssh.ffprobe_json, func:apply_remote_path_prefix(path: str, prefix: str) → str, call:(prefix or "").strip, call:normalized_prefix.rstrip, call:path.startswith, call:posixpath.normpath, call:posixpath.join, func:map_path_to_media_root(path: str, media_root: str) → str, call:(media_root or "").strip, call:posixpath.normpath, call:str(path).split, call:"/".join, call:path_absolute.startswith, call:posixpath.basename, call:raw_parts.index, call:posixpath.join, func:resolve_remote_media_path(path: str, media_root: str, fallback_prefix: str) → str, call:map_path_to_media_root, call:apply_remote_path_prefix, func:credentials_panel(), call:load_config, call:st.header, call:st.expander, call:st.text_input, call:st.number_input, call:int, func:main(), call:st.set_page_config, call:st.title, call:st.caption, call:credentials_panel, call:st.info, call:get_jellyfin_client, call:cached_users, call:st.error, call:user.get, call:st.selectbox, call:list, call:user_options.keys, call:st.tabs, call:render_now_playing, call:st.divider, call:render_resource_dashboard, call:render_media_overview, call:cached_libraries, call:set_file_browser_path, call:resolve_remote_media_path, call:render_media_tab, call:render_file_browser, call:get_ssh_client, call:render_ssh_tools, func:set_prefixed_file_browser_path(path: str, selected_path, reset_filters) → None, call:set_file_browser_path, call:resolve_remote_media_path | dep: json, posixpath, typing, media_library_viewer.clients.jellyfin, media_library_viewer.clients.ssh, media_library_viewer.config, media_library_viewer.ui.dashboard, media_library_viewer.ui.file_browser, media_library_viewer.ui.media, media_library_viewer.ui.preview, streamlit
|
||||
- config.py | Loads application configuration from environment variables and .env files using immutable dataclasses for Jellyfin and SSH settings. | exp: class:JellyfinConfig, class:SSHConfig, class:AppConfig, func:load_config() → AppConfig, call:AppConfig | dep: os, dataclasses, pathlib, dotenv
|
||||
- jobs.py | Defines safe, template-based remote SSH jobs with shell-quoted parameter rendering. | exp: class:JobTemplate, method:render(self, values: Mapping[str, str]) → str, call:shlex.quote, call:values.items, call:self.command_template.format, func:run_job(ssh: RemoteSSHClient, job_key: str, path: str, timeout) → CommandResult, call:template.render, call:ssh.run | dep: shlex, dataclasses, typing, media_library_viewer.clients.ssh, typing.Mapping
|
||||
- utils.py | Provides UI-independent formatting helpers and ffprobe output summarizers for video/audio/subtitle stream metadata. | exp: func:ticks_to_minutes(ticks: int | None) → int | None, call:round, func:human_size(num: int | float | None) → str, call:float, call:int, func:timestamp_to_local(ts: float | None) → str, call:datetime.fromtimestamp(ts).strftime, func:is_known_video_file(path: str | None) → bool, call:PurePosixPath(path).suffix.lower, func:format_duration(seconds: str | int | float | None) → str, call:float, call:str, call:int, func:format_bitrate(bit_rate: str | int | float | None) → str, call:float, call:str, func:_tags(stream: dict[str, Any]) → dict[str, Any], call:stream.get, func:_disposition(stream: dict[str, Any], key: str) → str, call:(stream.get("disposition") or {}).get, call:stream.get, func:_side_data_types(stream: dict[str, Any]) → str, call:stream.get, call:item.get, call:values.append, call:", ".join, func:ffprobe_format_summary(ffprobe: dict[str, Any]) → dict[str, str], call:ffprobe.get, call:fmt.get, call:format_duration, call:human_size, call:float, call:format_bitrate, call:str, func:summarize_video_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:stream.get, call:_tags, call:rows.append, call:format_bitrate, call:_side_data_types, call:tags.get, call:_disposition, func:summarize_audio_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:stream.get, call:_tags, call:rows.append, call:format_bitrate, call:tags.get, call:_disposition, func:summarize_subtitle_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:stream.get, call:_tags, call:rows.append, call:tags.get, call:_disposition, func:summarize_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:rows.append, call:format_bitrate, call:stream.get("tags", {}).get | dep: datetime, pathlib, typing
|
||||
## arch
|
||||
Layered Streamlit application using immutable dataclass configuration, template-based remote job execution, cached data access, and separated utility functions following a tab-based modular UI pattern.
|
||||
## tags
|
||||
client, path, media, call:, jellyfin, call:get, ssh, cached
|
||||
## symbols
|
||||
- JellyfinConfig
|
||||
- SSHConfig
|
||||
- AppConfig
|
||||
- JobTemplate
|
||||
- get_jellyfin_client
|
||||
- cached_users
|
||||
- get_ssh_client
|
||||
- cached_libraries
|
||||
## workflows
|
||||
- change media_library_viewer behavior
|
||||
read: __init__.py, app.py, config.py
|
||||
- change media_library_viewer config
|
||||
read: config.py
|
||||
- explore media_library_viewer subdirectories
|
||||
index: archive/src/media_library_viewer/clients/.pi-map.index.md, archive/src/media_library_viewer/domain/.pi-map.index.md, archive/src/media_library_viewer/services/.pi-map.index.md
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,23 @@
|
||||
# archive/src/media_library_viewer/clients (index)
|
||||
dir: archive/src/media_library_viewer/clients
|
||||
|
||||
## role
|
||||
External service and system integration layer providing HTTP API clients for Jellyfin/Emby media servers and SSH-based remote system metrics collection.
|
||||
## parent
|
||||
index: archive/src/media_library_viewer/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- jellyfin.py
|
||||
- resources.py
|
||||
- ssh.py
|
||||
## links
|
||||
index: archive/src/media_library_viewer/clients/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/clients/.pi-map.md
|
||||
## workflows
|
||||
- change clients behavior
|
||||
read: __init__.py, jellyfin.py, resources.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,30 @@
|
||||
# archive/src/media_library_viewer/clients
|
||||
dir: archive/src/media_library_viewer/clients
|
||||
|
||||
index: archive/src/media_library_viewer/clients/.pi-map.index.md
|
||||
|
||||
## role
|
||||
External service and system integration layer providing HTTP API clients for Jellyfin/Emby media servers and SSH-based remote system metrics collection.
|
||||
## files
|
||||
- __init__.py | Package initialization file that defines external service clients module boundaries and constraints
|
||||
- jellyfin.py | HTTP API client for Jellyfin/Emby media servers providing user, library, item, and session management with plain Python return types for frontend agnosticism. | exp: class:JellyfinClient, method:__init__(self, base_url: str, api_key: str, timeout), call:base_url.rstrip, call:self.base_url.endswith, call:requests.Session, call:self.session.headers.update, raise:ValueError, method:get(self, path: str, **params: Any) → dict[str, Any], call:params.items, call:self.session.get, call:response.raise_for_status, call:response.json, raise:requests.HTTPError, method:users(self) → list[dict[str, Any]], call:self.get, method:libraries(self, user_id: str) → list[dict[str, Any]], call:self.get(f"/Users/{user_id}/Views").get, method:items(self, user_id: str, parent_id, start_index, limit, search, include_item_types, recursive, sort_by, sort_order) → dict[str, Any], call:self.get, call:str(recursive).lower, method:item_count(self, user_id: str, include_item_types: str, parent_id) → int, call:self.get, call:int, call:response.get, method:media_counts(self, user_id: str) → dict[str, int], call:self.item_count, method:library_item_counts(self, user_id: str, libraries: list[dict[str, Any]]) → list[dict[str, Any]], call:lib.get, call:self.item_count, call:results.append, method:active_sessions(self, active_within_seconds) → list[dict[str, Any]], call:self.get, call:isinstance, call:session.get, method:image_url(self, item_id: str, image_type) → str | dep: typing, requests
|
||||
- resources.py | Manages a lightweight POSIX shell-based remote system metrics collector that samples CPU, memory, network, and disk statistics via SSH and reads the resulting JSONL data. | exp: class:ResourceMonitorPaths, func:start_resource_collector(ssh: RemoteSSHClient, interval_seconds, retention_seconds, max_lines, paths) → str, call:shlex.quote, call:int, call:ssh.run, call:result.stdout.strip, raise:RuntimeError, func:stop_resource_collector(ssh: RemoteSSHClient, paths) → str, call:shlex.quote, call:ssh.run, call:result.stdout.strip, raise:RuntimeError, func:restart_resource_collector(ssh: RemoteSSHClient, interval_seconds, retention_seconds, max_lines, paths) → str, call:stop_resource_collector, call:start_resource_collector, func:resource_collector_status(ssh: RemoteSSHClient, paths) → str, call:shlex.quote, call:ssh.run, call:result.stdout.strip, raise:RuntimeError, func:resource_collector_debug_info(ssh: RemoteSSHClient, paths) → str, call:shlex.quote, call:ssh.run, func:read_resource_metrics(ssh: RemoteSSHClient, max_lines, paths) → list[dict[str, Any]], call:shlex.quote, call:int, call:ssh.run, call:result.stdout.splitlines, call:line.strip, call:rows.append, call:json.loads, raise:RuntimeError, func:disk_space(ssh: RemoteSSHClient, path) → dict[str, Any], call:shlex.quote, call:ssh.run, call:result.stdout.strip, call:json.loads, raise:RuntimeError | dep: json, shlex, dataclasses, typing, media_library_viewer.clients.ssh
|
||||
- ssh.py | Provides an SSH client wrapper around Paramiko for remote filesystem inspection and media analysis, ensuring POSIX shell compatibility regardless of the user's login shell. | exp: class:CommandResult, class:RemoteSSHClient, method:__init__(self, host: str, username: str, port, key_filename, password, timeout), raise:ValueError, method:connect(self) → paramiko.SSHClient, call:paramiko.SSHClient, call:client.load_system_host_keys, call:client.set_missing_host_key_policy, call:paramiko.RejectPolicy, call:client.connect, method:close(self) → None, call:self._client.close, method:run(self, command: str, timeout) → CommandResult, call:self.connect, call:shlex.quote, call:client.exec_command, call:stdout.channel.recv_exit_status, call:CommandResult, call:stdout.read().decode, call:stderr.read().decode, method:list_dir(self, path: str) → CommandResult, call:shlex.quote, call:self.run, method:stat_path(self, path: str) → CommandResult, call:shlex.quote, call:self.run, method:ffprobe_json(self, path: str) → dict[str, Any], call:shlex.quote, call:self.run, call:json.loads, raise:RuntimeError | dep: json, posixpath, shlex, dataclasses, typing, paramiko
|
||||
## arch
|
||||
Client-wrapper pattern with each module encapsulating a specific integration concern (Jellyfin HTTP API, SSH filesystem access, remote resource monitoring), returning plain Python types for frontend agnosticism.
|
||||
## tags
|
||||
call:shlex.quote, resource, error, collector, call:ssh.run, raise:runtime, call:self.get, call:result.stdout.strip
|
||||
## symbols
|
||||
- JellyfinClient
|
||||
- ResourceMonitorPaths
|
||||
- CommandResult
|
||||
- RemoteSSHClient
|
||||
- __init__
|
||||
- get
|
||||
- users
|
||||
- libraries
|
||||
## workflows
|
||||
- change clients behavior
|
||||
read: __init__.py, jellyfin.py, resources.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,21 @@
|
||||
# archive/src/media_library_viewer/domain (index)
|
||||
dir: archive/src/media_library_viewer/domain
|
||||
|
||||
## role
|
||||
Provides domain-level normalization logic that transforms inconsistent Jellyfin API responses into stable, flattened data structures for storage and display.
|
||||
## parent
|
||||
index: archive/src/media_library_viewer/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- media.py
|
||||
## links
|
||||
index: archive/src/media_library_viewer/domain/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/domain/.pi-map.md
|
||||
## workflows
|
||||
- change domain behavior
|
||||
read: __init__.py, media.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,28 @@
|
||||
# archive/src/media_library_viewer/domain
|
||||
dir: archive/src/media_library_viewer/domain
|
||||
|
||||
index: archive/src/media_library_viewer/domain/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides domain-level normalization logic that transforms inconsistent Jellyfin API responses into stable, flattened data structures for storage and display.
|
||||
## files
|
||||
- __init__.py | Serves as the package docstring for a domain-level helpers/normalization module that converts external data into stable app concepts.
|
||||
- media.py | Flattens inconsistent Jellyfin API item JSON into stable, normalized dictionaries for SQLite storage and frontend display. | exp: func:first_media_source(item: dict[str, Any]) → dict[str, Any], call:item.get, func:media_streams(item: dict[str, Any], stream_type) → list[dict[str, Any]], call:item.get, call:streams.extend, call:source.get, call:str(stream.get("Type") or stream.get("codec_type") or "").lower, call:stream.get, call:stream_type.lower, func:stream_value(stream: dict[str, Any], *keys: str) → Any, func:is_hdr_item(item: dict[str, Any]) → bool, call:media_streams, call:stream_value, call:" ".join, call:str(value).lower, call:any, func:format_date_added(value: str | None) → str, call:pd.to_datetime(value).strftime, call:str, func:timestamp_date_added(value: str | None) → int | None, call:int, call:pd.to_datetime(value).timestamp, func:format_rate_bits_decimal(bits_per_second: float | int | str | None) → str, call:float, call:str, func:normalize_media_item(item: dict[str, Any], library_id, library_name) → dict[str, Any], call:first_media_source, call:media_streams, call:source.get, call:item.get, call:stream_value, call:is_hdr_item, call:int, call:ticks_to_minutes, call:human_size, call:format_rate_bits_decimal, call:video.get, call:format_date_added, call:timestamp_date_added, func:display_media_row(row: dict[str, Any]) → dict[str, Any], call:row.get, call:human_size, call:format_rate_bits_decimal | dep: typing, media_library_viewer.utils, pandas, media_library_viewer.utils (human_size, ticks_to_minutes)
|
||||
## arch
|
||||
Functional transformation layer pattern mapping raw external API JSON directly into normalized flat dictionaries without intermediate ORM or complex object hierarchies.
|
||||
## tags
|
||||
media, call:str, date, added, item, call:item.get, streams, call:stream
|
||||
## symbols
|
||||
- first_media_source
|
||||
- media_streams
|
||||
- stream_value
|
||||
- is_hdr_item
|
||||
- format_date_added
|
||||
- timestamp_date_added
|
||||
- format_rate_bits_decimal
|
||||
- normalize_media_item
|
||||
## workflows
|
||||
- change domain behavior
|
||||
read: __init__.py, media.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,21 @@
|
||||
# archive/src/media_library_viewer/services (index)
|
||||
dir: archive/src/media_library_viewer/services
|
||||
|
||||
## role
|
||||
Application service layer that coordinates domain logic and external clients into reusable, UI-agnostic media library operations.
|
||||
## parent
|
||||
index: archive/src/media_library_viewer/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- media_index.py
|
||||
## links
|
||||
index: archive/src/media_library_viewer/services/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/services/.pi-map.md
|
||||
## workflows
|
||||
- change services behavior
|
||||
read: __init__.py, media_index.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,28 @@
|
||||
# archive/src/media_library_viewer/services
|
||||
dir: archive/src/media_library_viewer/services
|
||||
|
||||
index: archive/src/media_library_viewer/services/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Application service layer that coordinates domain logic and external clients into reusable, UI-agnostic media library operations.
|
||||
## files
|
||||
- __init__.py | Marks the directory as a Python package and documents it as the application services layer for coordinating clients/domain logic into reusable operations.
|
||||
- media_index.py | Provides a UI-agnostic SQLite-backed media inventory service that indexes, queries, and manages Jellyfin media metadata with filtering, sorting, and pagination capabilities. | exp: class:MediaIndexStatus, class:MediaIndex, method:__init__(self, db_path), call:Path, call:self.db_path.parent.mkdir, method:connect(self) → sqlite3.Connection, call:sqlite3.connect, method:init_schema(self) → None, call:self.connect, call:conn.executescript, method:set_metadata(self, key: str, value: str | int | float) → None, call:self.init_schema, call:self.connect, call:conn.execute, call:str, method:replace_items(self, rows: Iterable[dict[str, Any]]) → int, call:self.init_schema, call:list, call:",".join, call:len, call:self.connect, call:conn.execute, call:conn.executemany, call:','.join, call:row.get, call:str, call:int, call:time.time, method:status(self) → MediaIndexStatus, call:self.db_path.exists, call:MediaIndexStatus, call:self.connect, call:int, call:conn.execute("SELECT COUNT(*) FROM media_items").fetchone, call:conn.execute("SELECT value FROM index_metadata WHERE key='updated_at'").fetchone, call:conn.execute("SELECT value FROM index_metadata WHERE key='build_duration_seconds'").fetchone, call:str(updated_row[0]).isdigit, call:time.strftime, call:time.localtime, call:float, method:query(self, library_id, library_ids, media_types, search, hdr_filter, sort_key, sort_order, limit, offset) → tuple[list[dict[str, Any]], int], call:self.init_schema, call:where.append, call:",".join, call:len, call:params.extend, call:params.append, call:search.lower, call:" AND ".join, call:SORT_COLUMNS.get, call:self.connect, call:int, call:conn.execute("SELECT COUNT(*) FROM media_items" + where_sql, params).fetchone, call:conn.execute( "SELECT * FROM media_items" + where_sql + order_sql + " LIMIT ? OFFSET ?", [*params, int(limit), int(offset)], ).fetchall, call:display_media_row, call:dict, func:build_media_index(client: JellyfinClient, user_id: str, libraries: list[dict[str, Any]], index, page_size) → int, call:MediaIndex, call:time.perf_counter, call:library.get, call:client.items, call:response.get, call:normalized_rows.extend, call:normalize_media_item, call:len, call:int, call:index.replace_items, call:index.set_metadata | dep: sqlite3, time, dataclasses, pathlib, typing, media_library_viewer.clients.jellyfin, media_library_viewer.domain.media, media_library_viewer.clients.jellyfin.JellyfinClient, media_library_viewer.domain.media.display_media_row, media_library_viewer.domain.media.normalize_media_item
|
||||
## arch
|
||||
Service-oriented pattern with SQLite-backed indexing, query filtering, and pagination encapsulated behind a single cohesive media index service module.
|
||||
## tags
|
||||
media, call:conn.execute, index, call:self.connect, schema, call:int, init, status
|
||||
## symbols
|
||||
- MediaIndexStatus
|
||||
- MediaIndex
|
||||
- __init__
|
||||
- connect
|
||||
- init_schema
|
||||
- set_metadata
|
||||
- replace_items
|
||||
- status
|
||||
## workflows
|
||||
- change services behavior
|
||||
read: __init__.py, media_index.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,24 @@
|
||||
# archive/src/media_library_viewer/ui (index)
|
||||
dir: archive/src/media_library_viewer/ui
|
||||
|
||||
## role
|
||||
Streamlit UI rendering layer for the media library viewer application, providing dashboard monitoring, file browsing, media indexing, and preview capabilities.
|
||||
## parent
|
||||
index: archive/src/media_library_viewer/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- dashboard.py
|
||||
- file_browser.py
|
||||
- media.py
|
||||
- preview.py
|
||||
## links
|
||||
index: archive/src/media_library_viewer/ui/.pi-map.index.md
|
||||
map: archive/src/media_library_viewer/ui/.pi-map.md
|
||||
## workflows
|
||||
- change ui behavior
|
||||
read: __init__.py, dashboard.py, file_browser.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,31 @@
|
||||
# archive/src/media_library_viewer/ui
|
||||
dir: archive/src/media_library_viewer/ui
|
||||
|
||||
index: archive/src/media_library_viewer/ui/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Streamlit UI rendering layer for the media library viewer application, providing dashboard monitoring, file browsing, media indexing, and preview capabilities.
|
||||
## files
|
||||
- __init__.py | Package initialization file for Streamlit UI modules that documents the architectural pattern of splitting the application into separate render modules.
|
||||
- dashboard.py | Implements a Streamlit dashboard for monitoring a Jellyfin media server, displaying media library statistics, active playback sessions, and server resource metrics via SSH. | exp: func:format_rate_bytes(bytes_per_second: float | int | None) → str, call:human_size, func:rate_scale(max_value: float | int | None) → tuple[float, str], call:abs, call:float, func:scaled_rate_chart_df(chart_df: pd.DataFrame, columns: list[str], labels: list[str]) → tuple[pd.DataFrame, str], call:chart_df[columns].max(numeric_only=True).max, call:rate_scale, call:chart_df[columns].copy, func:format_elapsed(seconds: float | int | None) → str, call:float, call:int, func:render_media_overview(cached_media_counts, cached_library_counts, base_url: str, api_key: str, user_id: str) → None, call:st.subheader, call:cached_media_counts, call:st.warning, call:counts.get, call:st.columns, call:top_cols[0].metric, call:top_cols[1].metric, call:top_cols[2].metric, call:top_cols[3].metric, call:cached_library_counts, call:st.caption, call:st.markdown, call:e.get, call:st.container, call:m_cols[0].metric, call:m_cols[1].metric, func:render_now_playing(cached_active_sessions, base_url: str, api_key: str) → None, call:st.subheader, call:cached_active_sessions, call:st.warning, call:st.caption, call:session.get, call:bool, call:play_state.get, call:item.get, call:transcoding.get, call:transcode_type.append, call:rows.append, call:", ".join, call:st.dataframe, call:pd.DataFrame, func:render_resource_dashboard(get_ssh_client, ssh_args: tuple, media_root: str, detailed) → None, call:st.subheader, call:get_ssh_client, call:resource_collector_status, call:st.error, call:st.columns, call:control_col.caption, call:start_col.button, call:st.success, call:start_resource_collector, call:restart_col.button, call:restart_resource_collector, call:stop_col.button, call:st.info, call:stop_resource_collector, call:refresh_col.button, call:st.rerun, call:st.caption, call:read_resource_metrics, call:disk_space, call:float, call:str(space.get("used_pct", "0")).rstrip, call:space.get, call:disk_cols[0].metric, call:human_size, call:disk_cols[1].metric, call:disk_cols[2].metric, call:disk_cols[3].metric, call:st.progress, call:min, call:max, call:st.warning, call:st.expander, call:st.code, call:resource_collector_debug_info, call:pd.DataFrame, call:pd.to_numeric, call:df.dropna, call:pd.to_datetime(df["ts"], unit="s", utc=True).dt.tz_convert, call:time.time, call:len, call:st.write, call:raw_df['ts'].astype(float).max, call:st.dataframe, call:raw_df.tail, call:df.sort_values, call:df["cpu_pct"].mean, call:df["cpu_pct"].max, call:df["iowait_pct"].mean, call:df["iowait_pct"].max, call:df["mem_pct"].mean, call:df["mem_pct"].max, call:df["net_rx_bytes_per_sec"].mean, call:df["net_rx_bytes_per_sec"].max, call:df["net_tx_bytes_per_sec"].mean, call:df["net_tx_bytes_per_sec"].max, call:df["disk_read_bps"].mean, call:df["disk_read_bps"].max, call:df["disk_write_bps"].mean, call:df["disk_write_bps"].max, call:metric_cols[0].metric, call:metric_cols[0].caption, call:metric_cols[1].metric, call:latest.get, call:metric_cols[1].caption, call:metric_cols[2].metric, call:metric_cols[2].caption, call:metric_cols[3].metric, call:format_rate_bytes, call:metric_cols[3].caption, call:metric_cols[4].metric, call:metric_cols[4].caption, call:metric_cols[5].metric, call:metric_cols[5].caption, call:metric_cols[6].metric, call:metric_cols[6].caption, call:df.set_index, call:st.markdown, call:st.line_chart, call:scaled_rate_chart_df | dep: time, typing, media_library_viewer.clients.resources, media_library_viewer.utils, pandas, streamlit
|
||||
- file_browser.py | Renders an interactive SSH remote file browser UI in Streamlit with filtering, sorting, pagination, and directory navigation using ag-grid. | exp: func:reset_file_browser_filters() → None, call:st.session_state.pop, func:set_file_browser_path(path: str, selected_path, reset_filters) → None, func:aggrid_selected_rows(response: dict) → list[dict], call:response.get, call:isinstance, call:selected_rows.to_dict, call:list, func:render_file_browser(cached_dir_listing: Callable[..., list[dict]], ssh_args: tuple, initial_path: str) → str, call:st.subheader, call:st.session_state.pop, call:reset_file_browser_filters, call:st.session_state.get, call:st.columns, call:status_col.caption, call:selected_col.caption, call:path_col.text_input, call:set_file_browser_path, call:st.rerun, call:refresh_col.button, call:cached_dir_listing.clear, call:st.error, call:PurePosixPath(name).suffix.lower, call:str, call:display_rows.append, call:int, call:human_size, call:float, call:timestamp_to_local, call:len, call:sum, call:st.caption, call:st.container, call:filter_col.selectbox, call:search_col.text_input, call:sorted, call:ext_col.selectbox, call:sort_col.selectbox, call:order_col.toggle, call:page_size_col.selectbox, call:search_term.lower, call:r["name"].lower, call:filtered_rows.sort, call:max, call:page_col.number_input, call:summary_col.caption, call:min, call:visible_rows.append, call:visible_rows.extend, call:st.info, call:st.expander, call:st.write, call:pd.DataFrame, call:GridOptionsBuilder.from_dataframe, call:grid_builder.configure_default_column, call:grid_builder.configure_column, call:grid_builder.configure_selection, call:grid_builder.build, call:JsCode, call:AgGrid, call:aggrid_selected_rows, call:picked_row.get | dep: json, pathlib, typing, st_aggrid, media_library_viewer.utils, streamlit, pandas
|
||||
- media.py | Renders a Streamlit UI tab for browsing and filtering a local SQLite-backed media index with ag-grid table display and automatic file browser synchronization. | exp: func:aggrid_selected_rows(response: dict[str, Any]) → list[dict[str, Any]], call:response.get, call:isinstance, call:selected_rows.to_dict, call:list, func:format_elapsed(seconds: float | int | None) → str, call:float, call:int, func:render_media_tab(client, user_id: str, libraries: list[dict[str, Any]], set_file_browser_path: Callable[[str, str | None, bool], None]) → None, call:st.subheader, call:st.caption, call:MediaIndex, call:index.status, call:st.columns, call:status_parts.append, call:format_elapsed, call:status_col.caption, call:" | ".join, call:status_col.warning, call:build_col.button, call:st.spinner, call:build_media_index, call:st.success, call:st.rerun, call:refresh_col.button, call:st.info, call:filter_col.multiselect, call:list, call:library_options.keys, call:type_col.multiselect, call:search_col.text_input, call:page_size_col.selectbox, call:page_col.number_input, call:sort_col.selectbox, call:sort_options.keys, call:order_col.selectbox, call:hdr_col.selectbox, call:index.query, call:int, call:len, call:pd.DataFrame(rows)[columns].fillna, call:st.session_state.get, call:GridOptionsBuilder.from_dataframe, call:grid_builder.configure_default_column, call:grid_builder.configure_column, call:grid_builder.configure_selection, call:grid_builder.build, call:JsCode, call:AgGrid, call:min, call:aggrid_selected_rows, call:selected_rows[0].get, call:set_file_browser_path, call:str, call:PurePosixPath, call:st.expander, call:st.write | dep: pathlib, typing, st_aggrid, media_library_viewer.services.media_index, pandas, streamlit
|
||||
- preview.py | Renders a Streamlit UI for previewing selected media file metadata via ffprobe and executing remote SSH diagnostic tools/jobs. | exp: func:render_ffprobe_sections(ffprobe_data: dict[str, Any]) → None, call:ffprobe_format_summary, call:summarize_video_streams, call:summarize_audio_streams, call:summarize_subtitle_streams, call:st.markdown, call:st.dataframe, call:pd.DataFrame, call:st.caption, func:render_selected_file_preview(ssh_args: tuple, selected_path: str | None, cached_ffprobe_preview: Callable[..., dict[str, Any]]) → None, call:st.container, call:st.markdown, call:st.caption, call:is_known_video_file, call:st.columns, call:refresh_col.button, call:cached_ffprobe_preview.clear, call:st.rerun, call:st.spinner, call:status_col.error, call:status_col.success, call:render_ffprobe_sections, call:st.expander, call:st.json, func:render_ssh_tools(ssh, ssh_args: tuple, selected_path: str | None, cached_ffprobe_preview: Callable[..., dict[str, Any]]) → None, call:render_selected_file_preview, call:st.subheader, call:st.tabs, call:st.button, call:ssh.ffprobe_json, call:render_ffprobe_sections, call:st.expander, call:st.dataframe, call:pd.DataFrame, call:summarize_streams, call:st.json, call:st.error, call:str, call:ssh.stat_path, call:st.code, call:st.warning, call:st.selectbox, call:list, call:JOB_TEMPLATES.keys, call:st.caption, call:JOB_TEMPLATES[job_key].render, call:run_job, call:st.write | dep: typing, media_library_viewer.jobs, media_library_viewer.utils, pandas, streamlit
|
||||
## arch
|
||||
Module-based render pattern where each UI tab/view is isolated in its own module, sharing session state for cross-component synchronization (e.g., file browser auto-sync) and leveraging ag-grid for interactive data tables.
|
||||
## tags
|
||||
call:metric, call:grid, render, call:st.caption, col.button, browser, col.selectbox, media
|
||||
## symbols
|
||||
- format_rate_bytes
|
||||
- rate_scale
|
||||
- scaled_rate_chart_df
|
||||
- format_elapsed
|
||||
- render_media_overview
|
||||
- render_now_playing
|
||||
- render_resource_dashboard
|
||||
- reset_file_browser_filters
|
||||
## workflows
|
||||
- change ui behavior
|
||||
read: __init__.py, dashboard.py, file_browser.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# archive/tests (index)
|
||||
dir: archive/tests
|
||||
|
||||
## role
|
||||
Legacy or archived test directory currently containing only a placeholder file with no active test code.
|
||||
## parent
|
||||
index: archive/.pi-map.index.md
|
||||
map: archive/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- .gitkeep
|
||||
## links
|
||||
index: archive/tests/.pi-map.index.md
|
||||
map: archive/tests/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,19 @@
|
||||
# archive/tests
|
||||
dir: archive/tests
|
||||
|
||||
index: archive/tests/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Legacy or archived test directory currently containing only a placeholder file with no active test code.
|
||||
## files
|
||||
- .gitkeep | Swaps the position of two tmux panes within a window or between windows | dep: tmux, sh
|
||||
## arch
|
||||
Empty placeholder structure using a `.gitkeep` file to preserve the directory in version control for potential future use.
|
||||
## tags
|
||||
tmux, swaps, position, two, panes, within, window, windows
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,32 @@
|
||||
# backend (index)
|
||||
dir: backend
|
||||
|
||||
## role
|
||||
FastAPI backend service providing Jellyfin media browsing, SSH file inspection, server monitoring, and JWT-protected API endpoints.
|
||||
## parent
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
## children
|
||||
- backend/.pytest_cache
|
||||
index: backend/.pytest_cache/.pi-map.index.md
|
||||
map: backend/.pytest_cache/.pi-map.md
|
||||
- backend/.ruff_cache
|
||||
index: backend/.ruff_cache/.pi-map.index.md
|
||||
map: backend/.ruff_cache/.pi-map.md
|
||||
- backend/src
|
||||
index: backend/src/.pi-map.index.md
|
||||
map: backend/src/.pi-map.md
|
||||
- backend/tests
|
||||
index: backend/tests/.pi-map.index.md
|
||||
map: backend/tests/.pi-map.md
|
||||
## files
|
||||
- Dockerfile
|
||||
- README.md
|
||||
- pyproject.toml
|
||||
## links
|
||||
index: backend/.pi-map.index.md
|
||||
map: backend/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,21 @@
|
||||
# backend
|
||||
dir: backend
|
||||
|
||||
index: backend/.pi-map.index.md
|
||||
|
||||
## role
|
||||
FastAPI backend service providing Jellyfin media browsing, SSH file inspection, server monitoring, and JWT-protected API endpoints.
|
||||
## files
|
||||
- Dockerfile | Builds a Docker container for a Python 3.11 backend API service using uvicorn | dep: python:3.11-slim, pip, uvicorn, pyproject.toml-based package
|
||||
- README.md | Documentation describing the setup, configuration, Docker deployment, and API endpoints for a FastAPI backend that provides Jellyfin media browsing, SSH file inspection, server monitoring, and JWT-protected access. | dep: FastAPI, uvicorn, pydantic-settings, Docker Compose
|
||||
- pyproject.toml | Project configuration file defining dependencies, build system, linting, and testing settings for a FastAPI media library viewer backend. | dep: FastAPI, uvicorn, pydantic-settings, paramiko, requests, python-dotenv, pandas, PyJWT, prometheus-client, python-json-logger, cryptography, hatchling, ruff, pytest, httpx
|
||||
## arch
|
||||
Containerized Python 3.11 REST API using FastAPI/uvicorn with JWT authentication, configured via pyproject.toml with linting and testing support.
|
||||
## tags
|
||||
uvicorn, fastapi, python, backend, pyproject, settings, docker, api
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
+36
-21
@@ -13,29 +13,46 @@ backend/
|
||||
│ ├── __init__.py
|
||||
│ ├── main.py # FastAPI app entrypoint
|
||||
│ ├── config.py # pydantic-settings config
|
||||
│ ├── auth.py # OIDC/JWT + API key auth
|
||||
│ ├── dependencies.py # Dependency injection
|
||||
│ ├── observability.py # Prometheus metrics + request IDs
|
||||
│ ├── logging_utils.py # Structured JSON/text logging
|
||||
│ ├── path_utils.py # Jellyfin→SSH path resolution
|
||||
│ ├── jobs.py # Job templates
|
||||
│ ├── utils.py # Formatting helpers
|
||||
│ ├── routers/
|
||||
│ │ ├── backups.py
|
||||
│ │ ├── dashboard.py
|
||||
│ │ ├── monitoring.py
|
||||
│ │ ├── media.py
|
||||
│ │ ├── users.py
|
||||
│ │ ├── settings.py
|
||||
│ │ ├── files.py
|
||||
│ │ └── jobs.py
|
||||
│ │ ├── jobs.py
|
||||
│ │ ├── media.py
|
||||
│ │ ├── monitoring.py
|
||||
│ │ ├── services.py
|
||||
│ │ ├── settings.py
|
||||
│ │ ├── tasks.py
|
||||
│ │ ├── users.py (+ users_impl.py)
|
||||
│ │ └── widgets.py
|
||||
│ ├── clients/
|
||||
│ │ ├── jellyfin.py
|
||||
│ │ ├── jellyseerr.py
|
||||
│ │ ├── local.py
|
||||
│ │ ├── resources.py
|
||||
│ │ └── ssh.py
|
||||
│ ├── integrations/ # Service-registry definitions
|
||||
│ ├── domain/
|
||||
│ │ └── media.py
|
||||
│ └── services/
|
||||
│ ├── media_index.py
|
||||
│ └── settings_store.py
|
||||
│ ├── models/ # Pydantic request/response models
|
||||
│ ├── services/
|
||||
│ │ ├── media_index.py (+ _impl.py)
|
||||
│ │ ├── settings_store.py
|
||||
│ │ ├── secrets.py # Fernet encryption at rest
|
||||
│ │ ├── targets.py # Node Exporter target discovery
|
||||
│ │ ├── task_runner.py
|
||||
│ │ ├── mail_queue.py (+ mailer.py/_impl.py)
|
||||
│ │ ├── backup_alert_engine.py (+ backup_poller.py)
|
||||
│ │ ├── known_hosts.py
|
||||
│ │ └── db_maintenance.py
|
||||
│ ├── widgets/ # Widget sources (dashboard data adapters)
|
||||
│ └── workers/ # Background workers (media index)
|
||||
└── tests/
|
||||
```
|
||||
|
||||
@@ -99,7 +116,7 @@ Or with PYTHONPATH if not installed:
|
||||
PYTHONPATH=src uvicorn media_library_viewer_api.main:app --reload --port 8000
|
||||
```
|
||||
|
||||
API docs available at: http://localhost:8000/docs
|
||||
API docs available at: <http://localhost:8000/docs>
|
||||
|
||||
## Docker
|
||||
|
||||
@@ -123,16 +140,16 @@ export VITE_OIDC_POST_LOGOUT_REDIRECT_URI=https://manage.example.com/
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
2. After the API is running, open the app, go to **Settings**, and add machine entries:
|
||||
1. After the API is running, open the app, go to **Settings**, and add machine entries:
|
||||
- **Local**: monitors the API host itself without SSH.
|
||||
- **SSH**: monitors another machine using a host, username, and a private key pasted directly into the machine settings, with an optional passphrase.
|
||||
- The machine editor groups Connection, Monitoring / Files, Jellyfin, Jellyseerr, and Notes under separate headings so each service area is easier to scan.
|
||||
- Saving a monitoring machine now validates the banner/auth flow, records the first trusted host key into the backend-managed `known_hosts` file, and starts the collector so charts populate without a separate manual step.
|
||||
- Saving a monitoring machine validates the banner/auth flow and records the first trusted host key into the backend-managed `known_hosts` file.
|
||||
- If the host cannot be reached or authenticated, the save flow surfaces the SSH error directly in the dialog.
|
||||
- Use **Validate SSH + trust host** in the machine editor before saving if you want to test the banner/auth flow explicitly.
|
||||
- The first successful SSH connection uses trust-on-first-use: the backend records that machine's host key into its managed `known_hosts` file automatically, then continues verifying it strictly on later connects.
|
||||
|
||||
3. Open **Monitoring** to see one section per configured machine. Each section uses its own collector state, disk path, metrics queries, and recent action history, which are populated automatically by the backend poller.
|
||||
2. Open **Observability** to see Alertmanager alerts, Prometheus scrape targets, and Grafana deep-links for configured machines. Alertmanager, Grafana, and Prometheus are configured as service instances on the **Services** page; system metrics (disk, CPU, memory) are owned by the external observability stack (Prometheus + node_exporter + Grafana), not by the Manage backend.
|
||||
|
||||
For local development, `docker compose -f docker-compose.dev.yml up --build` does not require an SSH key unless you configure remote SSH machines in the Settings tab.
|
||||
|
||||
@@ -142,14 +159,12 @@ For local development, `docker compose -f docker-compose.dev.yml up --build` doe
|
||||
- `GET /api/dashboard/libraries` — Per-library breakdown
|
||||
- `GET /api/dashboard/now-playing` — Active playback sessions
|
||||
- `GET /api/monitoring/machines` — Persistent monitoring machine definitions
|
||||
- `GET /api/monitoring/status?machine_id=` — Collector status for a machine
|
||||
- `GET /api/monitoring/metrics?machine_id=` — Resource samples (last hour)
|
||||
- `GET /api/monitoring/disk?machine_id=` — Disk space
|
||||
- `POST /api/monitoring/start|stop|restart?machine_id=` — Collector controls
|
||||
- `GET /api/monitoring/diagnostics?machine_id=` — Collector debug info
|
||||
- `GET /api/monitoring/poller` — Backend poller status and configuration
|
||||
- `GET /api/monitoring/machines/{machine_id}/actions` — Recent machine action history
|
||||
- `GET /api/dashboard/monitoring` — Dashboard-wide per-machine monitoring summary table with 10-minute averages and min/max subtext
|
||||
- `GET /api/monitoring/prometheus-targets` — Prometheus scrape targets for remote Node Exporters (consumed by external Prometheus via `http_sd_configs`)
|
||||
- `GET /api/monitoring/alerts` — Active Alertmanager alerts summary (resolves the configured alertmanager service)
|
||||
- `GET /api/monitoring/alertmanager-status` — Alertmanager cluster/status
|
||||
- `GET /api/monitoring/grafana-status` — Grafana service health
|
||||
- `GET /api/monitoring/prometheus-status` — Prometheus service health
|
||||
- `POST /api/monitoring/alertmanager-webhook` — Receive Alertmanager webhooks (log-only)
|
||||
- `GET /api/settings/machines` — Manage machine definitions
|
||||
- `GET /api/media/status` — Index status
|
||||
- `POST /api/media/build` — Rebuild index
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# backend/src (index)
|
||||
dir: backend/src
|
||||
|
||||
## role
|
||||
Root source directory serving as the main entry point and organizational container for the backend application.
|
||||
## parent
|
||||
index: backend/.pi-map.index.md
|
||||
map: backend/.pi-map.md
|
||||
## children
|
||||
- backend/src/media_library_viewer_api
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## files
|
||||
## links
|
||||
index: backend/src/.pi-map.index.md
|
||||
map: backend/src/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,18 @@
|
||||
# backend/src
|
||||
dir: backend/src
|
||||
|
||||
index: backend/src/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Root source directory serving as the main entry point and organizational container for the backend application.
|
||||
## files
|
||||
## arch
|
||||
Standard layered architecture entry point, typically initializing the application, wiring up configurations, modules, routes, and services (e.g., MVC, modular monolith, or Clean Architecture).
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,57 @@
|
||||
# backend/src/media_library_viewer_api (index)
|
||||
dir: backend/src/media_library_viewer_api
|
||||
|
||||
## role
|
||||
FastAPI backend service that provides authenticated, observable APIs for viewing and managing media library data across Jellyfin, Jellyseerr, and remote SSH/local systems.
|
||||
## parent
|
||||
index: backend/src/.pi-map.index.md
|
||||
map: backend/src/.pi-map.md
|
||||
## children
|
||||
- backend/src/media_library_viewer_api/clients
|
||||
index: backend/src/media_library_viewer_api/clients/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/clients/.pi-map.md
|
||||
- backend/src/media_library_viewer_api/domain
|
||||
index: backend/src/media_library_viewer_api/domain/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/domain/.pi-map.md
|
||||
- backend/src/media_library_viewer_api/integrations
|
||||
index: backend/src/media_library_viewer_api/integrations/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/integrations/.pi-map.md
|
||||
- backend/src/media_library_viewer_api/models
|
||||
index: backend/src/media_library_viewer_api/models/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/models/.pi-map.md
|
||||
- backend/src/media_library_viewer_api/routers
|
||||
index: backend/src/media_library_viewer_api/routers/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/routers/.pi-map.md
|
||||
- backend/src/media_library_viewer_api/services
|
||||
index: backend/src/media_library_viewer_api/services/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/services/.pi-map.md
|
||||
- backend/src/media_library_viewer_api/widgets
|
||||
index: backend/src/media_library_viewer_api/widgets/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/widgets/.pi-map.md
|
||||
- backend/src/media_library_viewer_api/workers
|
||||
index: backend/src/media_library_viewer_api/workers/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/workers/.pi-map.md
|
||||
## files
|
||||
- __init__.py
|
||||
- auth.py
|
||||
- config.py
|
||||
- dependencies.py
|
||||
- jobs.py
|
||||
- logging_utils.py
|
||||
- main.py
|
||||
- observability.py
|
||||
- path_utils.py
|
||||
- utils.py
|
||||
- version.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## workflows
|
||||
- change media_library_viewer_api behavior
|
||||
read: __init__.py, auth.py, config.py
|
||||
- change media_library_viewer_api config
|
||||
read: config.py
|
||||
- explore media_library_viewer_api subdirectories
|
||||
index: backend/src/media_library_viewer_api/clients/.pi-map.index.md, backend/src/media_library_viewer_api/domain/.pi-map.index.md, backend/src/media_library_viewer_api/integrations/.pi-map.index.md
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,41 @@
|
||||
# backend/src/media_library_viewer_api
|
||||
dir: backend/src/media_library_viewer_api
|
||||
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
|
||||
## role
|
||||
FastAPI backend service that provides authenticated, observable APIs for viewing and managing media library data across Jellyfin, Jellyseerr, and remote SSH/local systems.
|
||||
## files
|
||||
- __init__.py | Swaps the position of two tmux panes within a window or between windows | dep: tmux, sh
|
||||
- auth.py | Implements OIDC/JWT and API key authentication for a FastAPI backend with middleware-based route protection. | exp: func:_normalize_issuer_url(issuer_url: str) → str, call:issuer_url.rstrip, func:get_oidc_metadata(issuer_url: str) → dict[str, Any], call:_normalize_issuer_url, call:urljoin, call:requests.get, call:response.raise_for_status, call:response.json, call:isinstance, raise:RuntimeError, func:get_jwk_client(jwks_url: str) → PyJWKClient, call:PyJWKClient, func:_split_audience(audience: str) → list[str], call:item.strip, call:audience.split, func:validate_auth_settings(settings: Settings) → None, raise:RuntimeError, func:validate_bearer_jwt(authorization: str | None, settings) → dict[str, Any], call:get_settings, call:validate_auth_settings, call:authorization.partition, call:scheme.lower, call:token.strip, call:_normalize_issuer_url, call:get_oidc_metadata, call:settings.oidc_jwks_url.strip, call:str, call:metadata.get, call:get_jwk_client, call:jwk_client.get_signing_key_from_jwt, call:_split_audience, call:jwt.decode, call:list, call:len, call:int, raise:PermissionError, raise:RuntimeError, func:require_jwt_auth(request: Request, call_next), call:get_settings, call:path.startswith, call:call_next, call:validate_bearer_jwt, call:request.headers.get, call:logger.warning, call:JSONResponse, call:str, call:logger.exception, call:claims.get, call:isinstance, func:get_api_key() → str, call:get_settings_store, call:store.get_settings, call:settings.get, call:secrets.token_urlsafe, call:store.update_setting, func:require_api_key(authorization) → str, call:get_api_key, call:secrets.compare_digest, raise:HTTPException | dep: logging, secrets, functools, typing, urllib.parse, jwt, requests, fastapi, fastapi.responses, jwt.exceptions, media_library_viewer_api.config, media_library_viewer_api.dependencies
|
||||
- config.py | Defines a flat pydantic-settings configuration model that loads application settings from environment variables and .env files with cached access. | exp: class:Settings, func:_find_env_file() → str | None, call:Path.cwd, call:candidate.is_file, call:str, call:(directory / ".git").exists, func:get_settings() → Settings, call:_find_env_file, call:Settings, call:logger.info, call:describe_settings | dep: logging, functools, pathlib, pydantic_settings, media_library_viewer_api.logging_utils, functools.lru_cache, pathlib.Path, pydantic_settings.BaseSettings
|
||||
- dependencies.py | Provides FastAPI dependency injection functions for resolving and caching service clients (Jellyfin, Jellyseerr, SSH/Local) and settings based on request query parameters. | exp: func:_request_machine_id(request: Request | None) → str | None, call:request.query_params.get, func:_request_jellyfin_service_id(request: Request | None) → str | None, call:request.query_params.get, func:_service_record(store: SettingsStore, service_type: str, service_id: str | None) → dict[str, Any] | None, call:store.get_service, call:candidate.get, call:store.list_services, call:s.get, call:row.get, call:decrypt_secrets, call:logger.exception, func:_jellyfin_client_for(cache_key: tuple[str, str, str]) → JellyfinClient, call:logger.info, call:url.rstrip, call:JellyfinClient, func:_ssh_client_for(cache_key: tuple[str, str, str, int, str, str | None, str | None, str | None, str | None]) → RemoteSSHClient, call:logger.info, call:RemoteSSHClient, call:client.connect, call:str, call:message.lower, call:logger.exception, raise:HTTPException, func:_resolve_machine(service: str, request) → dict[str, Any] | None, call:get_settings_store, call:_request_machine_id, call:store.get_machine, call:machine.get, call:store.list_machines_for_service, func:get_jellyfin_client(request) → JellyfinClient, call:get_settings_store, call:_request_jellyfin_service_id, call:_service_record, call:str, call:service.get("config", {}).get, call:service.get("secrets", {}).get, call:_jellyfin_client_for, raise:HTTPException, func:get_jellyseerr_client(request) → JellyseerrClient | None, call:get_settings_store, call:_request_jellyfin_service_id, call:_service_record, call:logger.info, call:str, call:service.get("config", {}).get, call:service.get("secrets", {}).get, call:JellyseerrClient, func:_ssh_client_from_machine_config(machine: dict[str, Any], store) → RemoteSSHClient, call:get_settings_store, call:get_settings, call:str(machine.get("ssh_key_id") or "").strip, call:machine.get, call:store.get_ssh_key, call:ssh_key.get, call:int, call:_ssh_client_for, func:get_ssh_client(request), call:get_settings_store, call:_request_machine_id, call:store.get_machine_config, call:_resolve_machine, call:str(machine.get("mode") or "local").strip().lower, call:machine.get, call:logger.info, call:LocalCommandClient, call:_ssh_client_from_machine_config, call:get_settings, call:_ssh_client_for, raise:HTTPException, func:get_mail_queue() → MailQueue, call:_get_mail_queue, func:get_settings_store() → SettingsStore, call:_get_settings_store, func:get_user_id(request) → str, call:get_settings_store, call:_request_jellyfin_service_id, call:_service_record, call:service.get("config", {}).get, call:str, call:get_jellyfin_client, call:client.users, raise:HTTPException | dep: logging, functools, typing, fastapi, media_library_viewer_api.clients.jellyfin, media_library_viewer_api.clients.jellyseerr, media_library_viewer_api.clients.local, media_library_viewer_api.clients.ssh, media_library_viewer_api.config, media_library_viewer_api.services.mail_queue, media_library_viewer_api.services.settings_store, media_library_viewer_api.services.secrets
|
||||
- jobs.py | Defines template-based remote SSH jobs with shell-safe rendering for a media library viewer API. | exp: class:JobTemplate, method:render(self, values: Mapping[str, str]) → str, call:shlex.quote, call:values.items, call:self.command_template.format, func:run_job(ssh: RemoteSSHClient, job_key: str, path: str, timeout) → CommandResult, call:template.render, call:logger.info, call:ssh.run | dep: logging, shlex, dataclasses, typing, media_library_viewer_api.clients.ssh
|
||||
- logging_utils.py | Configures structured JSON/text logging with secret-safe settings introspection and log field sanitization for a backend application. | exp: func:_json_formatter() → logging.Formatter, call:jsonlogger.JsonFormatter, func:_text_formatter() → logging.Formatter, call:logging.Formatter, func:configure_logging(level_name, log_format) → int, call:(level_name or os.getenv("LOG_LEVEL", "INFO")).upper, call:os.getenv, call:getattr, call:(log_format or os.getenv("LOG_FORMAT", "text")).lower, call:logging.StreamHandler, call:handler.setFormatter, call:_json_formatter, call:_text_formatter, call:logging.basicConfig, call:root.setLevel, call:logging.getLogger("media_library_viewer_api").setLevel, call:logging.getLogger("uvicorn").setLevel, call:logging.getLogger("uvicorn.error").setLevel, call:logging.getLogger("uvicorn.access").setLevel, call:logging.getLogger("paramiko").setLevel, call:logging.getLogger("urllib3").setLevel, func:_sanitize_url(url: str | None) → str, call:urlsplit, call:url.strip, call:url.rstrip, func:describe_settings(settings: object) → dict[str, str], call:str(getattr(settings, "log_level", "INFO") or "INFO").upper, call:getattr, call:str(getattr(settings, "log_format", "text") or "text").lower, call:bool, call:_sanitize_url, func:sanitize_log_extra(extra: dict[str, Any] | None) → dict[str, Any], call:extra.items, call:key.lower, call:any, call:lower_key.endswith | dep: logging, os, typing, urllib.parse, pythonjsonlogger
|
||||
- main.py | FastAPI application entrypoint that configures middleware, registers routers, manages startup/shutdown lifecycle, and exposes health/version/metrics endpoints. | exp: func:lifespan(app: FastAPI), call:get_settings, call:configure_logging, call:validate_auth_settings, call:validate_encryption_key, call:logger.info, call:describe_settings, call:get_settings_store().ensure_defaults, call:logger.exception, call:get_mail_queue, call:get_backup_poller, call:mail_queue.start, call:backup_poller.start, call:backup_poller.stop, call:mail_queue.stop, func:enforce_jwt_auth(request: Request, call_next), call:call_next, call:require_jwt_auth, func:log_requests(request: Request, call_next), call:time.perf_counter, call:get_request_id, call:set_current_request_id, call:sanitize_log_extra, call:logger.info, call:call_next, call:logger.exception, call:record_request, call:round, func:health_check() → dict[str, str], call:logger.debug, func:version_info() → dict[str, str], call:logger.debug, call:get_version_info, func:metrics() → Response, call:metrics_payload, call:FastAPIResponse | dep: logging, time, contextlib, uvicorn, fastapi, fastapi.middleware.cors, fastapi.responses, media_library_viewer_api.auth, media_library_viewer_api.config, media_library_viewer_api.dependencies, media_library_viewer_api.logging_utils, media_library_viewer_api.observability, media_library_viewer_api.routers, media_library_viewer_api.routers.settings, .services.backup_poller, .version, media_library_viewer_api.services.secrets, media_library_viewer_api.services.backup_poller, media_library_viewer_api.version
|
||||
- observability.py | Provides Prometheus metrics collection, request ID generation/correlation, and structured logging helpers for application observability. | exp: func:set_current_request_id(request_id: str | None) → None, call:_current_request_id.set, func:get_current_request_id() → str | None, call:_current_request_id.get, func:generate_request_id() → str, call:uuid.uuid4, func:get_request_id(request) → str, call:request.headers.get, call:header.strip, call:_current_request_id.get, call:generate_request_id, call:_current_request_id.set, func:metrics_payload() → tuple[bytes, str], call:generate_latest, func:record_request(request: Request, response: Response, duration_seconds: float) → None, call:str, call:REQUESTS_TOTAL.labels(method=method, path=path, status_code=status).inc, call:REQUEST_DURATION.labels(method=method, path=path).observe, func:record_ssh_command(machine_id: str, action: str, status: str, duration_seconds: float) → None, call:SSH_COMMANDS_TOTAL.labels(machine_id=machine_id or "unknown", action=action, status=status).inc, call:SSH_COMMAND_DURATION.labels(machine_id=machine_id or "unknown", action=action).observe, func:record_media_index_build(status: str, duration_seconds) → None, call:MEDIA_INDEX_BUILDS_TOTAL.labels(status=status).inc, call:MEDIA_INDEX_BUILD_DURATION.observe, func:record_backup_run(job_name: str, status: str, success) → None, call:BACKUP_RUNS_TOTAL.labels(job_name=job_name, status=status).inc, call:BACKUP_RUNS_LAST_SUCCESS.labels(job_name=job_name).set_to_current_time, func:record_mail_queue(status: str) → None, call:MAIL_QUEUE_SIZE.labels(status=status).inc, func:log_extra(request, **kwargs: Any) → dict[str, Any], call:get_request_id, call:extra.update | dep: uuid, contextvars, typing, fastapi, prometheus_client
|
||||
- path_utils.py | Maps Jellyfin media paths to SSH-accessible paths using media root anchoring or fallback prefixing. | exp: func:apply_remote_path_prefix(path: str, prefix: str) → str, call:(prefix or "").strip, call:normalized_prefix.rstrip, call:path.startswith, call:posixpath.normpath, call:logger.debug, call:posixpath.join, func:map_path_to_media_root(path: str, media_root: str) → str, call:(media_root or "").strip, call:posixpath.normpath, call:str(path).split, call:"/".join, call:path_absolute.startswith, call:logger.debug, call:posixpath.basename, call:raw_parts.index, call:posixpath.join, func:resolve_remote_media_path(path: str, media_root: str, fallback_prefix: str) → str, call:map_path_to_media_root, call:logger.debug, call:apply_remote_path_prefix | dep: logging, posixpath
|
||||
- utils.py | Provides UI-framework-independent formatting helpers and ffprobe output summarizers for video, audio, and subtitle streams. | exp: func:ticks_to_minutes(ticks: int | None) → int | None, call:round, func:human_size(num: int | float | None) → str, call:float, call:int, func:timestamp_to_local(ts: float | None) → str, call:datetime.fromtimestamp(ts).strftime, func:is_known_video_file(path: str | None) → bool, call:PurePosixPath(path).suffix.lower, func:format_duration(seconds: str | int | float | None) → str, call:float, call:str, call:int, func:format_bitrate(bit_rate: str | int | float | None) → str, call:float, call:str, func:_tags(stream: dict[str, Any]) → dict[str, Any], call:stream.get, func:_disposition(stream: dict[str, Any], key: str) → str, call:(stream.get("disposition") or {}).get, call:stream.get, func:_side_data_types(stream: dict[str, Any]) → str, call:stream.get, call:item.get, call:values.append, call:", ".join, func:ffprobe_format_summary(ffprobe: dict[str, Any]) → dict[str, str], call:ffprobe.get, call:fmt.get, call:format_duration, call:human_size, call:float, call:format_bitrate, call:str, func:summarize_video_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:stream.get, call:_tags, call:rows.append, call:format_bitrate, call:_side_data_types, call:tags.get, call:_disposition, func:summarize_audio_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:stream.get, call:_tags, call:rows.append, call:format_bitrate, call:tags.get, call:_disposition, func:summarize_subtitle_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:stream.get, call:_tags, call:rows.append, call:tags.get, call:_disposition, func:summarize_streams(ffprobe: dict[str, Any]) → list[dict[str, Any]], call:ffprobe.get, call:rows.append, call:format_bitrate, call:stream.get("tags", {}).get | dep: datetime, pathlib, typing
|
||||
- version.py | Provides version retrieval and formatting utilities for a backend service, falling back through environment variables, package metadata, and default values. | exp: func:get_backend_version() → str, call:os.getenv("APP_VERSION", "").strip, call:package_version, func:get_backend_build_info() → str, call:os.getenv("APP_BUILD_INFO", "").strip, call:os.getenv("GIT_COMMIT", "").strip, call:os.getenv("BUILD_COMMIT", "").strip, func:format_version_label(version: str, build_info: str) → str, call:version.strip, call:build_info.strip, func:get_version_info() → dict[str, str], call:get_backend_version, call:get_backend_build_info, call:format_version_label | dep: os, importlib.metadata
|
||||
## arch
|
||||
Layered FastAPI architecture using dependency injection for cached service clients, Pydantic settings configuration, middleware-based OIDC/JWT/API-key authentication, Prometheus observability with structured logging, and template-based remote job execution.
|
||||
## tags
|
||||
call:, settings, call:get, request, get, client, call:str, id
|
||||
## symbols
|
||||
- Settings
|
||||
- JobTemplate
|
||||
- _normalize_issuer_url
|
||||
- get_oidc_metadata
|
||||
- get_jwk_client
|
||||
- _split_audience
|
||||
- validate_auth_settings
|
||||
- validate_bearer_jwt
|
||||
## workflows
|
||||
- change media_library_viewer_api behavior
|
||||
read: __init__.py, auth.py, config.py
|
||||
- change media_library_viewer_api config
|
||||
read: config.py
|
||||
- explore media_library_viewer_api subdirectories
|
||||
index: backend/src/media_library_viewer_api/clients/.pi-map.index.md, backend/src/media_library_viewer_api/domain/.pi-map.index.md, backend/src/media_library_viewer_api/integrations/.pi-map.index.md
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,24 @@
|
||||
# backend/src/media_library_viewer_api/clients (index)
|
||||
dir: backend/src/media_library_viewer_api/clients
|
||||
|
||||
## role
|
||||
Provides HTTP and command execution client wrappers for integrating with external media services (Jellyfin, Jellyseerr) and performing remote/local filesystem inspection.
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- jellyfin.py
|
||||
- jellyseerr.py
|
||||
- local.py
|
||||
- ssh.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/clients/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/clients/.pi-map.md
|
||||
## workflows
|
||||
- change clients behavior
|
||||
read: __init__.py, jellyfin.py, jellyseerr.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,31 @@
|
||||
# backend/src/media_library_viewer_api/clients
|
||||
dir: backend/src/media_library_viewer_api/clients
|
||||
|
||||
index: backend/src/media_library_viewer_api/clients/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides HTTP and command execution client wrappers for integrating with external media services (Jellyfin, Jellyseerr) and performing remote/local filesystem inspection.
|
||||
## files
|
||||
- __init__.py | Swaps the position of two tmux panes within a window or between windows | dep: tmux, sh
|
||||
- jellyfin.py | Provides a reusable, framework-agnostic HTTP client wrapper for the Jellyfin/Emby API with methods for browsing users, libraries, media items, and sessions. | exp: class:JellyfinClient, method:__init__(self, base_url: str, api_key: str, timeout), call:base_url.rstrip, call:self.base_url.endswith, call:requests.Session, call:self.session.headers.update, raise:ValueError, method:get(self, path: str, **params: Any) → Any, call:params.items, call:logger.debug, call:sorted, call:clean_params.keys, call:self.session.get, call:response.raise_for_status, call:logger.warning, call:response.json, raise:requests.HTTPError, method:users(self) → list[dict[str, Any]], call:self.get, call:logger.info, call:len, method:libraries(self, user_id: str) → list[dict[str, Any]], call:self.get(f"/Users/{user_id}/Views").get, call:logger.info, call:len, method:items(self, user_id: str, parent_id, start_index, limit, search, include_item_types, recursive, sort_by, sort_order) → dict[str, Any], call:logger.debug, call:self.get, call:str(recursive).lower, method:item_count(self, user_id: str, include_item_types: str, parent_id) → int, call:self.get, call:int, call:response.get, call:logger.debug, method:media_counts(self, user_id: str) → dict[str, int], call:self.item_count, method:library_item_counts(self, user_id: str, libraries: list[dict[str, Any]]) → list[dict[str, Any]], call:lib.get, call:self.item_count, call:results.append, method:sessions(self, active_within_seconds) → list[dict[str, Any]], call:self.get, call:cast, call:isinstance, method:active_sessions(self, active_within_seconds) → list[dict[str, Any]], call:self.sessions, call:session.get, call:logger.info, call:len, method:image_url(self, item_id: str, image_type) → str | dep: logging, typing, requests
|
||||
- jellyseerr.py | HTTP client wrapper for the Jellyseerr REST API to fetch user data and enrich Jellyfin user information | exp: class:JellyseerrClient, method:__init__(self, base_url: str, api_key: str, timeout), call:base_url.rstrip, call:self.base_url.endswith, call:requests.Session, call:self.session.headers.update, raise:ValueError, method:get(self, path: str, **params: Any) → Any, call:params.items, call:logger.debug, call:sorted, call:clean_params.keys, call:self.session.get, call:response.raise_for_status, call:logger.warning, call:response.json, raise:requests.HTTPError, method:absolute_url(self, path: str | None) → str, call:path.startswith, method:jellyfin_users(self) → list[dict[str, Any]], call:self.get, call:isinstance, call:logger.info, call:len, call:payload.get, method:users(self, page_size) → list[dict[str, Any]], call:max, call:int, call:self.get, call:isinstance, call:payload.get, call:results.extend, call:page_info.get, call:logger.debug, call:len, call:logger.info | dep: logging, typing, requests
|
||||
- local.py | Provides a local command execution client that mirrors remote SSH helpers to run POSIX shell commands, list directories, stat paths, and run ffprobe on the API host for built-in local monitoring. | exp: class:CommandResult, class:LocalCommandClient, method:__init__(self, timeout), method:run(self, command: str, timeout) → CommandResult, call:logger.debug, call:subprocess.run, call:CommandResult, call:logger.warning, call:result.stderr.strip, call:result.stdout.strip, method:list_dir(self, path: str) → CommandResult, call:shlex.quote, call:self.run, method:stat_path(self, path: str) → CommandResult, call:shlex.quote, call:self.run, method:ffprobe_json(self, path: str) → dict[str, object], call:shlex.quote, call:self.run, call:json.loads, raise:RuntimeError | dep: json, logging, posixpath, shlex, subprocess, dataclasses
|
||||
- ssh.py | Provides an SSH client wrapper for remote filesystem inspection and media analysis using paramiko, with POSIX shell command execution and host key management. | exp: class:CommandResult, class:RemoteSSHClient, method:__init__(self, host: str, username: str, port, key_filename, private_key, private_key_passphrase, password, known_hosts_path, timeout), raise:ValueError, method:connect(self) → paramiko.SSHClient, call:paramiko.SSHClient, call:client.load_system_host_keys, call:Path, call:bool, call:has_known_host, call:known_hosts_file.is_file, call:client.load_host_keys, call:client.set_missing_host_key_policy, call:paramiko.RejectPolicy, call:paramiko.AutoAddPolicy, call:self._load_private_key, call:client.connect, call:str(exc).lower, call:known_hosts_file.parent.mkdir, call:client.save_host_keys, raise:RuntimeError, method:close(self) → None, call:self._client.close, method:run(self, command: str, timeout) → CommandResult, call:self.connect, call:shlex.quote, call:logger.debug, call:client.exec_command, call:stdout.channel.recv_exit_status, call:CommandResult, call:stdout.read().decode, call:stderr.read().decode, call:logger.warning, call:result.stderr.strip, call:result.stdout.strip, method:list_dir(self, path: str) → CommandResult, call:shlex.quote, call:self.run, call:logger.info, method:stat_path(self, path: str) → CommandResult, call:shlex.quote, call:self.run, call:logger.info, method:ffprobe_json(self, path: str) → dict[str, Any], call:shlex.quote, call:self.run, call:logger.info, call:json.loads, raise:RuntimeError | dep: json, logging, posixpath, shlex, dataclasses, io, pathlib, typing, paramiko, media_library_viewer_api.services.known_hosts
|
||||
## arch
|
||||
Client-wrapper pattern with framework-agnostic abstractions; parallel local/remote execution strategies via paramiko SSH and local subprocess; centralized REST API communication modules.
|
||||
## tags
|
||||
call:logger.info, call:logger.debug, call:self.get, call:shlex.quote, error, host, call:self.run, init
|
||||
## symbols
|
||||
- JellyfinClient
|
||||
- JellyseerrClient
|
||||
- CommandResult
|
||||
- LocalCommandClient
|
||||
- RemoteSSHClient
|
||||
- __init__
|
||||
- get
|
||||
- users
|
||||
## workflows
|
||||
- change clients behavior
|
||||
read: __init__.py, jellyfin.py, jellyseerr.py
|
||||
## dirty
|
||||
-
|
||||
@@ -54,9 +54,6 @@ class Settings(BaseSettings):
|
||||
|
||||
# Observability
|
||||
prometheus_enabled: bool = True
|
||||
prometheus_file_sd_dir: str = "/app/backend/.cache/prometheus-file-sd"
|
||||
alertmanager_url: str = "http://alertmanager:9093"
|
||||
alertmanager_webhook_url: str = "" # Optional receiver for alertmanager webhook notifications
|
||||
|
||||
# Remote paths
|
||||
remote_media_root: str = ""
|
||||
|
||||
@@ -81,21 +81,6 @@ def _jellyfin_client_for(cache_key: tuple[str, str, str]) -> JellyfinClient:
|
||||
return JellyfinClient(url, api_key)
|
||||
|
||||
|
||||
@lru_cache(maxsize=32)
|
||||
def _jellyseerr_client_for(cache_key: tuple[str, str]) -> JellyseerrClient | None:
|
||||
machine_id, url = cache_key
|
||||
if not url:
|
||||
return None
|
||||
settings = get_settings_store().get_machine_config(machine_id) if machine_id else None
|
||||
api_key = (settings or {}).get("jellyseerr_api_key") if settings else ""
|
||||
if not api_key:
|
||||
return None
|
||||
logger.info(
|
||||
"Creating Jellyseerr client machine_id=%s url=%s", machine_id or "<default>", url.rstrip("/") or "<unset>"
|
||||
)
|
||||
return JellyseerrClient(url, api_key)
|
||||
|
||||
|
||||
@lru_cache(maxsize=32)
|
||||
def _ssh_client_for(
|
||||
cache_key: tuple[str, str, str, int, str, str | None, str | None, str | None, str | None],
|
||||
@@ -154,6 +139,10 @@ def _ssh_client_for(
|
||||
|
||||
|
||||
def _resolve_machine(service: str, request: Request | None = None) -> dict[str, Any] | None:
|
||||
"""Resolve an SSH/Files machine for the given transport service.
|
||||
|
||||
Jellyfin/Jellyseerr are resolved against the service registry, not here.
|
||||
"""
|
||||
store = get_settings_store()
|
||||
machine_id = _request_machine_id(request)
|
||||
if machine_id:
|
||||
@@ -161,11 +150,7 @@ def _resolve_machine(service: str, request: Request | None = None) -> dict[str,
|
||||
if machine and (service in machine.get("services", []) or service == "ssh"):
|
||||
return machine
|
||||
return machine
|
||||
if service == "jellyfin":
|
||||
machines = store.list_machines_for_service("jellyfin")
|
||||
elif service == "jellyseerr":
|
||||
machines = [m for m in store.list_machines_for_service("jellyfin") if m.get("jellyseerr_url")]
|
||||
elif service == "ssh":
|
||||
if service == "ssh":
|
||||
machines = store.list_machines_for_service("files") or store.list_machines_for_service("monitoring")
|
||||
else:
|
||||
machines = store.list_machines_for_service(service)
|
||||
@@ -178,11 +163,17 @@ def get_jellyfin_client(request: Request = None) -> JellyfinClient:
|
||||
service_id = _request_jellyfin_service_id(request)
|
||||
service = _service_record(store, "jellyfin", service_id)
|
||||
if service is None:
|
||||
raise RuntimeError("No Jellyfin service is configured. Add a Jellyfin service on the Services page.")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="No Jellyfin service is configured. Add a Jellyfin service on the Services page.",
|
||||
)
|
||||
base_url = str(service.get("config", {}).get("base_url") or "")
|
||||
api_key = str(service.get("secrets", {}).get("api_key") or "")
|
||||
if not base_url or not api_key:
|
||||
raise RuntimeError("Jellyfin service is missing base_url or api_key. Edit it on the Services page.")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="Jellyfin service is missing base_url or api_key. Edit it on the Services page.",
|
||||
)
|
||||
cache_key = (service["id"], base_url, api_key)
|
||||
return _jellyfin_client_for(cache_key)
|
||||
|
||||
@@ -259,7 +250,10 @@ def get_ssh_client(request: Request = None):
|
||||
"set" if settings.ssh_password else "missing",
|
||||
)
|
||||
if not settings.ssh_key_path:
|
||||
raise RuntimeError("No SSH machine is configured and SSH key settings must be configured")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="No SSH machine is configured and SSH key settings must be configured",
|
||||
)
|
||||
return _ssh_client_for(
|
||||
(
|
||||
"legacy",
|
||||
@@ -295,5 +289,8 @@ def get_user_id(request: Request = None) -> str:
|
||||
client = get_jellyfin_client(request)
|
||||
users = client.users()
|
||||
if not users:
|
||||
raise RuntimeError("No Jellyfin users found and no user_id configured on the service")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="No Jellyfin users found and no user_id configured on the service",
|
||||
)
|
||||
return users[0]["Id"]
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
# backend/src/media_library_viewer_api/domain (index)
|
||||
dir: backend/src/media_library_viewer_api/domain
|
||||
|
||||
## role
|
||||
Domain layer providing data normalization and transformation helpers for Jellyfin media data and dashboard summaries.
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- dashboard.py
|
||||
- media.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/domain/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/domain/.pi-map.md
|
||||
## workflows
|
||||
- change domain behavior
|
||||
read: __init__.py, dashboard.py, media.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,29 @@
|
||||
# backend/src/media_library_viewer_api/domain
|
||||
dir: backend/src/media_library_viewer_api/domain
|
||||
|
||||
index: backend/src/media_library_viewer_api/domain/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Domain layer providing data normalization and transformation helpers for Jellyfin media data and dashboard summaries.
|
||||
## files
|
||||
- __init__.py | Swaps the position of two tmux panes within a window or between windows | dep: tmux, sh
|
||||
- dashboard.py | Provides domain helper functions for building dashboard data, specifically normalizing Jellyfin session activity rows and computing backup job summaries. | exp: func:_map_sessions_to_activity_rows(sessions: list[dict[str, Any]]) → list[dict[str, Any]], call:session.get, call:bool, call:item.get, call:play_state.get, call:transcoding.get, call:transcode_type.append, call:results.append, call:", ".join, func:build_backup_dashboard_summary(store: SettingsStore) → BackupDashboardSummary, call:store.list_backup_jobs, call:len, call:int, call:time.time, call:store.list_backup_runs, call:recent_runs.append, call:sum, call:store.list_backup_alerts, call:failed_runs.append, call:max, call:BackupDashboardSummary, call:round | dep: time, typing, media_library_viewer_api.models.backups, media_library_viewer_api.services.settings_store
|
||||
- media.py | Flattens inconsistent Jellyfin API JSON into normalized dictionaries for SQLite indexing and frontend display. | exp: func:first_media_source(item: dict[str, Any]) → dict[str, Any], call:item.get, func:media_streams(item: dict[str, Any], stream_type) → list[dict[str, Any]], call:item.get, call:streams.extend, call:source.get, call:str(stream.get("Type") or stream.get("codec_type") or "").lower, call:stream.get, call:stream_type.lower, func:stream_value(stream: dict[str, Any], *keys: str) → Any, func:is_hdr_item(item: dict[str, Any]) → bool, call:media_streams, call:stream_value, call:" ".join, call:str(value).lower, call:any, func:format_date_added(value: str | None) → str, call:pd.to_datetime(value).strftime, call:str, func:timestamp_date_added(value: str | None) → int | None, call:int, call:pd.to_datetime(value).timestamp, func:format_rate_bits_decimal(bits_per_second: float | int | str | None) → str, call:float, call:str, func:normalize_media_item(item: dict[str, Any], library_id, library_name) → dict[str, Any], call:first_media_source, call:media_streams, call:source.get, call:item.get, call:stream_value, call:is_hdr_item, call:int, call:ticks_to_minutes, call:human_size, call:format_rate_bits_decimal, call:video.get, call:format_date_added, call:timestamp_date_added, func:display_media_row(row: dict[str, Any]) → dict[str, Any], call:row.get, call:human_size, call:format_rate_bits_decimal | dep: typing, media_library_viewer_api.utils, pandas
|
||||
## arch
|
||||
Stateless functional modules that transform inconsistent upstream API JSON into normalized dictionaries for persistence and display.
|
||||
## tags
|
||||
media, backup, call:item.get, call:str, date, added, dashboard, call:store.list
|
||||
## symbols
|
||||
- _map_sessions_to_activity_rows
|
||||
- build_backup_dashboard_summary
|
||||
- first_media_source
|
||||
- media_streams
|
||||
- stream_value
|
||||
- is_hdr_item
|
||||
- format_date_added
|
||||
- timestamp_date_added
|
||||
## workflows
|
||||
- change domain behavior
|
||||
read: __init__.py, dashboard.py, media.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,29 @@
|
||||
# backend/src/media_library_viewer_api/integrations (index)
|
||||
dir: backend/src/media_library_viewer_api/integrations
|
||||
|
||||
## role
|
||||
Provides a plugin-style integration framework for declaring and registering external service connections (e.g., Grafana, Jellyfin, Prometheus) with config schemas, secrets, and widget definitions for the media library viewer API.
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- alertmanager.py
|
||||
- base.py
|
||||
- grafana.py
|
||||
- jellyfin.py
|
||||
- jellyseerr.py
|
||||
- nextcloud.py
|
||||
- prometheus.py
|
||||
- registry.py
|
||||
- ssh_tasks.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/integrations/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/integrations/.pi-map.md
|
||||
## workflows
|
||||
- change integrations behavior
|
||||
read: __init__.py, alertmanager.py, base.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,36 @@
|
||||
# backend/src/media_library_viewer_api/integrations
|
||||
dir: backend/src/media_library_viewer_api/integrations
|
||||
|
||||
index: backend/src/media_library_viewer_api/integrations/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides a plugin-style integration framework for declaring and registering external service connections (e.g., Grafana, Jellyfin, Prometheus) with config schemas, secrets, and widget definitions for the media library viewer API.
|
||||
## files
|
||||
- __init__.py | Defines a closed registry module for service integrations.
|
||||
- alertmanager.py | Defines the Alertmanager service integration configuration, widget definitions, and alert summarization logic for a media library viewer API. | exp: class:AlertmanagerConfig, class:AlertmanagerAlertsWidgetConfig, func:summarize_alerts(alerts: list[dict[str, Any]], severity_filter) → dict[str, Any], call:alert.get, call:labels.get, call:by_severity.get, call:open_alerts.append, call:annotations.get, call:open_alerts.sort, call:len | dep: typing, media_library_viewer_api.integrations.base
|
||||
- base.py | Provides abstract base classes and dataclass definitions for declaring external service integrations with config schemas, secret fields, and widget kinds. | exp: class:ServiceConfigBase, class:WidgetConfigBase, class:SecretField, class:WidgetKind, class:ServiceDefinition, method:widget_kind(self, kind: str) → WidgetKind | None, func:_validate_service_base_url(value: Any) → str, call:isinstance, call:value.strip, call:text.lower, call:lowered.startswith, raise:ValueError, func:widget_kind(kind: str, name: str, description: str, model_cls: type[WidgetConfigBase], default_config, refresh_interval_ms) → WidgetKind, call:model_cls.model_json_schema, call:schema.pop, call:WidgetKind, call:dict, func:validate_config(model_cls: type[BaseModel], config: dict[str, Any] | None) → dict[str, Any], call:model_cls.model_validate, call:instance.model_dump | dep: dataclasses, typing, pydantic
|
||||
- grafana.py | Defines the Grafana service integration configuration, including connection settings, API key secrets, and dashboard link widget support. | exp: class:GrafanaConfig, class:GrafanaLinkWidgetConfig | dep: media_library_viewer_api.integrations.base
|
||||
- jellyfin.py | Defines the Jellyfin service configuration and activity widget for a media library viewer API integration. | exp: class:JellyfinConfig, class:JellyfinActivityWidgetConfig | dep: media_library_viewer_api.integrations.base
|
||||
- jellyseerr.py | Defines the Jellyseerr service configuration and its service definition schema for integration as a request management companion to Jellyfin. | exp: class:JellyseerrConfig | dep: media_library_viewer_api.integrations.base
|
||||
- nextcloud.py | Defines the Nextcloud service configuration model and service definition for integration into the media library viewer API. | exp: class:NextcloudConfig | dep: media_library_viewer_api.integrations.base
|
||||
- prometheus.py | Defines the service definition and configuration models for integrating Prometheus as a metrics data source with PromQL query widgets. | exp: class:PrometheusConfig, class:PrometheusMetricWidgetConfig | dep: media_library_viewer_api.integrations.base
|
||||
- registry.py | Provides a closed registry of service definitions with lookup and enumeration functions. | exp: func:list_service_types() → list[str], call:sorted, func:get_service_definition(service_type: str) → ServiceDefinition | None, call:SERVICE_DEFINITIONS.get, func:get_widget_kind(service_type: str, widget_kind: str) → WidgetKind | None, call:get_service_definition, call:definition.widget_kind, func:require_service_definition(service_type: str) → ServiceDefinition, call:get_service_definition, raise:ValueError | dep: media_library_viewer_api.integrations.alertmanager, media_library_viewer_api.integrations.base, media_library_viewer_api.integrations.grafana, media_library_viewer_api.integrations.jellyfin, media_library_viewer_api.integrations.jellyseerr, media_library_viewer_api.integrations.nextcloud, media_library_viewer_api.integrations.prometheus, media_library_viewer_api.integrations.ssh_tasks
|
||||
- ssh_tasks.py | Defines a service configuration for an SSH task runner that executes reusable saved tasks over SSH and records run history. | exp: class:SshTasksConfig, class:SshTaskOutputWidgetConfig | dep: media_library_viewer_api.integrations.base
|
||||
## arch
|
||||
Registry pattern with abstract base classes and dataclass-driven configuration models; each integration is a self-contained module registered in a closed registry that supports lookup, enumeration, and declarative widget/kind definitions.
|
||||
## tags
|
||||
config, service, widget, integrations, base, media_library_viewer_api, definition, kind
|
||||
## symbols
|
||||
- AlertmanagerConfig
|
||||
- AlertmanagerAlertsWidgetConfig
|
||||
- ServiceConfigBase
|
||||
- WidgetConfigBase
|
||||
- SecretField
|
||||
- WidgetKind
|
||||
- ServiceDefinition
|
||||
- GrafanaConfig
|
||||
## workflows
|
||||
- change integrations behavior
|
||||
read: __init__.py, alertmanager.py, base.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,89 @@
|
||||
"""Alertmanager service definition."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from media_library_viewer_api.integrations.base import (
|
||||
SecretField,
|
||||
ServiceBaseUrl,
|
||||
ServiceConfigBase,
|
||||
ServiceDefinition,
|
||||
WidgetConfigBase,
|
||||
widget_kind,
|
||||
)
|
||||
|
||||
|
||||
class AlertmanagerConfig(ServiceConfigBase):
|
||||
"""Non-secret Alertmanager connection config."""
|
||||
|
||||
base_url: ServiceBaseUrl
|
||||
timeout_seconds: int = 5
|
||||
|
||||
|
||||
class AlertmanagerAlertsWidgetConfig(WidgetConfigBase):
|
||||
"""Active-alerts summary for an Alertmanager instance."""
|
||||
|
||||
severity_filter: str | None = None
|
||||
|
||||
|
||||
def summarize_alerts(
|
||||
alerts: list[dict[str, Any]],
|
||||
*,
|
||||
severity_filter: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Build a UI-friendly summary from an Alertmanager ``/api/v1/alerts`` list.
|
||||
|
||||
Reshapes the raw alert objects into a stable summary (``total``,
|
||||
``by_severity``, top-50 ``alerts``). When ``severity_filter`` is given, only
|
||||
alerts whose ``labels.severity`` matches are counted.
|
||||
"""
|
||||
by_severity: dict[str, int] = {}
|
||||
open_alerts: list[dict[str, Any]] = []
|
||||
for alert in alerts:
|
||||
labels = alert.get("labels") or {}
|
||||
annotations = alert.get("annotations") or {}
|
||||
severity = labels.get("severity", "unknown")
|
||||
if severity_filter and severity != severity_filter:
|
||||
continue
|
||||
by_severity[severity] = by_severity.get(severity, 0) + 1
|
||||
open_alerts.append(
|
||||
{
|
||||
"name": labels.get("alertname", "unknown"),
|
||||
"severity": severity,
|
||||
"category": labels.get("category", ""),
|
||||
"job_name": labels.get("job_name", labels.get("job", "")),
|
||||
"summary": annotations.get("summary", ""),
|
||||
"description": annotations.get("description", ""),
|
||||
"active_since": alert.get("startsAt"),
|
||||
"state": alert.get("status", "firing"),
|
||||
"labels": labels,
|
||||
}
|
||||
)
|
||||
open_alerts.sort(key=lambda a: (a["severity"] not in {"critical", "warning"}, a["severity"], a["name"]))
|
||||
return {
|
||||
"total": len(open_alerts),
|
||||
"by_severity": by_severity,
|
||||
"alerts": open_alerts[:50],
|
||||
}
|
||||
|
||||
|
||||
DEFINITION = ServiceDefinition(
|
||||
service_type="alertmanager",
|
||||
name="Alertmanager",
|
||||
description="Alertmanager alerts and status.",
|
||||
config_model=AlertmanagerConfig,
|
||||
secret_fields=[
|
||||
SecretField(key="api_key", label="API key", helper="Optional bearer token"),
|
||||
],
|
||||
widget_kinds=[
|
||||
widget_kind(
|
||||
kind="active_alerts",
|
||||
name="Active alerts",
|
||||
description="Firing alerts summary from Alertmanager.",
|
||||
model_cls=AlertmanagerAlertsWidgetConfig,
|
||||
default_config={},
|
||||
refresh_interval_ms=30_000,
|
||||
),
|
||||
],
|
||||
)
|
||||
@@ -16,9 +16,37 @@ map. There is no runtime plugin loading.
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
from typing import Annotated, Any
|
||||
|
||||
from pydantic import BaseModel
|
||||
from pydantic import BaseModel, BeforeValidator, Field
|
||||
|
||||
|
||||
def _validate_service_base_url(value: Any) -> str:
|
||||
"""Require an absolute http(s) URL for service ``base_url`` fields.
|
||||
|
||||
Relative hosts (e.g. ``grafana.example.com``) break downstream HTTP clients
|
||||
because ``requests`` treats them as relative paths, so we fail fast with a
|
||||
clear error instead of letting the call silently malfunction.
|
||||
"""
|
||||
if not isinstance(value, str):
|
||||
raise ValueError("base_url must be a string starting with http:// or https://")
|
||||
text = value.strip()
|
||||
if not text:
|
||||
raise ValueError("base_url must not be empty")
|
||||
lowered = text.lower()
|
||||
if not (lowered.startswith("http://") or lowered.startswith("https://")):
|
||||
raise ValueError("base_url must start with http:// or https:// (include the schema)")
|
||||
return text
|
||||
|
||||
|
||||
#: Shared annotated type for service ``base_url`` fields. applying the validator
|
||||
#: uniformly across every integration so missing schemas are rejected at the
|
||||
#: config boundary with a helpful message.
|
||||
ServiceBaseUrl = Annotated[
|
||||
str,
|
||||
Field(description="Absolute URL including the http:// or https:// schema."),
|
||||
BeforeValidator(_validate_service_base_url),
|
||||
]
|
||||
|
||||
|
||||
class ServiceConfigBase(BaseModel):
|
||||
@@ -26,6 +54,9 @@ class ServiceConfigBase(BaseModel):
|
||||
|
||||
Subclass this in each integration module and declare the connection fields.
|
||||
The JSON schema is derived via ``model_json_schema()`` and exposed to the UI.
|
||||
|
||||
Connection URLs should use the :data:`ServiceBaseUrl` type so the
|
||||
``http(s)://`` schema is enforced consistently across integrations.
|
||||
"""
|
||||
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from media_library_viewer_api.integrations.base import (
|
||||
SecretField,
|
||||
ServiceBaseUrl,
|
||||
ServiceConfigBase,
|
||||
ServiceDefinition,
|
||||
WidgetConfigBase,
|
||||
@@ -14,7 +15,7 @@ from media_library_viewer_api.integrations.base import (
|
||||
class GrafanaConfig(ServiceConfigBase):
|
||||
"""Non-secret Grafana connection config."""
|
||||
|
||||
base_url: str
|
||||
base_url: ServiceBaseUrl
|
||||
timeout_seconds: int = 5
|
||||
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from media_library_viewer_api.integrations.base import (
|
||||
SecretField,
|
||||
ServiceBaseUrl,
|
||||
ServiceConfigBase,
|
||||
ServiceDefinition,
|
||||
WidgetConfigBase,
|
||||
@@ -14,7 +15,7 @@ from media_library_viewer_api.integrations.base import (
|
||||
class JellyfinConfig(ServiceConfigBase):
|
||||
"""Non-secret Jellyfin connection config."""
|
||||
|
||||
base_url: str
|
||||
base_url: ServiceBaseUrl
|
||||
user_id: str = ""
|
||||
timeout_seconds: int = 10
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
from media_library_viewer_api.integrations.base import (
|
||||
SecretField,
|
||||
ServiceBaseUrl,
|
||||
ServiceConfigBase,
|
||||
ServiceDefinition,
|
||||
)
|
||||
@@ -17,7 +18,7 @@ from media_library_viewer_api.integrations.base import (
|
||||
class JellyseerrConfig(ServiceConfigBase):
|
||||
"""Non-secret Jellyseerr connection config."""
|
||||
|
||||
base_url: str
|
||||
base_url: ServiceBaseUrl
|
||||
|
||||
|
||||
DEFINITION = ServiceDefinition(
|
||||
|
||||
@@ -8,6 +8,7 @@ from __future__ import annotations
|
||||
|
||||
from media_library_viewer_api.integrations.base import (
|
||||
SecretField,
|
||||
ServiceBaseUrl,
|
||||
ServiceConfigBase,
|
||||
ServiceDefinition,
|
||||
)
|
||||
@@ -16,7 +17,7 @@ from media_library_viewer_api.integrations.base import (
|
||||
class NextcloudConfig(ServiceConfigBase):
|
||||
"""Non-secret Nextcloud connection config."""
|
||||
|
||||
base_url: str
|
||||
base_url: ServiceBaseUrl
|
||||
username: str = ""
|
||||
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from media_library_viewer_api.integrations.base import (
|
||||
SecretField,
|
||||
ServiceBaseUrl,
|
||||
ServiceConfigBase,
|
||||
ServiceDefinition,
|
||||
WidgetConfigBase,
|
||||
@@ -14,7 +15,7 @@ from media_library_viewer_api.integrations.base import (
|
||||
class PrometheusConfig(ServiceConfigBase):
|
||||
"""Non-secret Prometheus connection config."""
|
||||
|
||||
base_url: str
|
||||
base_url: ServiceBaseUrl
|
||||
timeout_seconds: int = 10
|
||||
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ There is no runtime plugin loading.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from media_library_viewer_api.integrations.alertmanager import DEFINITION as ALERTMANAGER
|
||||
from media_library_viewer_api.integrations.base import ServiceDefinition, WidgetKind
|
||||
from media_library_viewer_api.integrations.grafana import DEFINITION as GRAFANA
|
||||
from media_library_viewer_api.integrations.jellyfin import DEFINITION as JELLYFIN
|
||||
@@ -17,6 +18,7 @@ from media_library_viewer_api.integrations.ssh_tasks import DEFINITION as SSH_TA
|
||||
SERVICE_DEFINITIONS: dict[str, ServiceDefinition] = {
|
||||
GRAFANA.service_type: GRAFANA,
|
||||
PROMETHEUS.service_type: PROMETHEUS,
|
||||
ALERTMANAGER.service_type: ALERTMANAGER,
|
||||
JELLYFIN.service_type: JELLYFIN,
|
||||
JELLYSEERR.service_type: JELLYSEERR,
|
||||
NEXTCLOUD.service_type: NEXTCLOUD,
|
||||
|
||||
@@ -44,12 +44,6 @@ async def lifespan(app: FastAPI):
|
||||
validate_encryption_key()
|
||||
logger.info("Backend startup complete: %s", describe_settings(settings))
|
||||
logger.info("Managed known_hosts will be populated lazily on first successful SSH connection")
|
||||
try:
|
||||
from media_library_viewer_api.services.targets import write_prometheus_targets
|
||||
|
||||
write_prometheus_targets(get_settings_store())
|
||||
except Exception:
|
||||
logger.exception("Failed to write Prometheus file-SD targets during startup")
|
||||
try:
|
||||
get_settings_store().ensure_defaults()
|
||||
except Exception:
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
# backend/src/media_library_viewer_api/models (index)
|
||||
dir: backend/src/media_library_viewer_api/models
|
||||
|
||||
## role
|
||||
Defines Pydantic data models for request/response validation across backup management, service registry, and dashboard widget APIs.
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- backups.py
|
||||
- services.py
|
||||
- widgets.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/models/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/models/.pi-map.md
|
||||
## workflows
|
||||
- change models behavior
|
||||
read: backups.py, services.py, widgets.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,29 @@
|
||||
# backend/src/media_library_viewer_api/models
|
||||
dir: backend/src/media_library_viewer_api/models
|
||||
|
||||
index: backend/src/media_library_viewer_api/models/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Defines Pydantic data models for request/response validation across backup management, service registry, and dashboard widget APIs.
|
||||
## files
|
||||
- backups.py | Defines Pydantic data models for backup system API requests and responses including reports, jobs, runs, alerts, and dashboard summaries. | exp: class:BackupReportRequest, class:BackupJobResponse, class:BackupRunResponse, class:BackupAlertResponse, class:BackupDashboardSummary | dep: datetime, typing, pydantic
|
||||
- services.py | Defines Pydantic models for a service registry API, including validation to prevent credential keys in non-secret configuration. | exp: class:ServiceInstanceInput, class:ServiceInstance, class:SecretFieldInfo, class:WidgetKindInfo, class:ServiceTypeInfo, func:_validate_config_keys(config: dict[str, Any]) → dict[str, Any], call:isinstance, call:value.items, call:key.lower, call:_check, raise:ValueError, func:_check(value: Any) → None, call:isinstance, call:value.items, call:key.lower, call:_check, raise:ValueError | dep: typing, pydantic
|
||||
- widgets.py | Defines Pydantic models for a dashboard widget system with validation to prevent secrets/credentials in widget configuration. | exp: class:_WidgetInstanceBase, class:WidgetInstanceInput, class:WidgetInstance, class:BuiltinWidgetKindInfo, class:WidgetDataResponse, func:_looks_secret(value: Any) → bool, call:isinstance, call:value.strip, call:value.lower, call:value.startswith, call:len, call:lowered.isalnum, func:_validate_config_keys(config: dict[str, Any]) → dict[str, Any], call:config.items, call:key.lower, call:_looks_secret, call:isinstance, call:_validate_config_keys, raise:ValueError | dep: typing, pydantic
|
||||
## arch
|
||||
Pydantic-based model layer implementing data validation, serialization contracts, and custom validators enforcing security constraints (e.g., blocking credentials in non-secret configs).
|
||||
## tags
|
||||
widget, backup, instance, response, info, call:isinstance, call:, service
|
||||
## symbols
|
||||
- BackupReportRequest
|
||||
- BackupJobResponse
|
||||
- BackupRunResponse
|
||||
- BackupAlertResponse
|
||||
- BackupDashboardSummary
|
||||
- ServiceInstanceInput
|
||||
- ServiceInstance
|
||||
- SecretFieldInfo
|
||||
## workflows
|
||||
- change models behavior
|
||||
read: backups.py, services.py, widgets.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,32 @@
|
||||
# backend/src/media_library_viewer_api/routers (index)
|
||||
dir: backend/src/media_library_viewer_api/routers
|
||||
|
||||
## role
|
||||
FastAPI router package that exposes all REST API endpoints for the media library viewer backend, organized by domain (files, media, jobs, backups, dashboard, monitoring, services, settings, tasks, users, widgets).
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- backups.py
|
||||
- dashboard.py
|
||||
- files.py
|
||||
- jobs.py
|
||||
- media.py
|
||||
- monitoring.py
|
||||
- services.py
|
||||
- settings.py
|
||||
- tasks.py
|
||||
- users.py
|
||||
- users_impl.py
|
||||
- widgets.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/routers/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/routers/.pi-map.md
|
||||
## workflows
|
||||
- change routers behavior
|
||||
read: __init__.py, backups.py, dashboard.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,39 @@
|
||||
# backend/src/media_library_viewer_api/routers
|
||||
dir: backend/src/media_library_viewer_api/routers
|
||||
|
||||
index: backend/src/media_library_viewer_api/routers/.pi-map.index.md
|
||||
|
||||
## role
|
||||
FastAPI router package that exposes all REST API endpoints for the media library viewer backend, organized by domain (files, media, jobs, backups, dashboard, monitoring, services, settings, tasks, users, widgets).
|
||||
## files
|
||||
- __init__.py | Marks the directory as a Python package for routers.
|
||||
- backups.py | FastAPI router that provides REST endpoints for reporting, tracking, and alerting on backup jobs and runs. | exp: func:_get_or_create_job(store: SettingsStore, report: BackupReportRequest) → dict[str, Any], call:store.get_backup_job_by_name, call:store.upsert_backup_job, call:store.get_backup_job, func:post_backup_report(report: BackupReportRequest, store, _auth) → BackupRunResponse, call:_get_or_create_job, call:store.list_backup_runs, call:int, call:report.started_at.timestamp, call:abs, call:BackupRunResponse, call:report.ended_at.timestamp, call:store.create_backup_run, call:record_backup_run, call:generate_alerts_for_run, call:store.create_backup_alert, call:store.resolve_backup_alerts_for_job, call:run.pop, func:post_backup_start(report: BackupReportRequest, store, _auth) → BackupRunResponse, call:_get_or_create_job, call:int, call:report.started_at.timestamp, call:store.create_backup_run, call:record_backup_run, call:run.pop, call:BackupRunResponse, func:get_backup_jobs(store) → list[dict[str, Any]], call:store.list_backup_jobs, func:get_backup_job(job_id: str, store) → dict[str, Any], call:store.get_backup_job, call:store.list_backup_runs, raise:HTTPException, func:get_backup_runs(job_id, status, limit, store) → list[BackupRunResponse], call:store.list_backup_runs, call:BackupRunResponse, func:get_backup_run(run_id: str, store) → BackupRunResponse, call:store.get_backup_run, call:BackupRunResponse, raise:HTTPException, func:get_backup_alerts(job_id, acknowledged, severity, store) → list[BackupAlertResponse], call:store.list_backup_alerts, call:BackupAlertResponse, func:acknowledge_backup_alert(alert_id: str, store) → BackupAlertResponse, call:store.acknowledge_backup_alert, call:BackupAlertResponse, raise:HTTPException | dep: typing, fastapi, ..auth, ..models.backups, ..observability, ..services.backup_alert_engine, ..services.settings_store
|
||||
- dashboard.py | FastAPI router providing dashboard endpoints for media counts, library breakdowns, shortcuts CRUD, activity sessions, and backup summaries. | exp: func:get_counts(client, user_id) → dict[str, int], call:client.media_counts, call:logger.info, func:get_library_counts(client, user_id) → list[dict[str, Any]], call:client.libraries, call:logger.info, call:len, call:client.library_item_counts, func:get_shortcuts() → list[dict[str, Any]], call:store.list_shortcuts, call:logger.info, call:len, func:create_shortcut(payload: dict[str, Any]) → dict[str, Any], call:store.upsert_shortcut, call:logger.info, call:shortcut.get, func:update_shortcut(shortcut_id: str, payload: dict[str, Any]) → dict[str, Any], call:store.upsert_shortcut, call:logger.info, call:shortcut.get, func:delete_shortcut(shortcut_id: str) → dict[str, str], call:store.delete_shortcut, call:logger.info, func:get_activity(client) → list[dict[str, Any]], call:client.sessions, call:_map_sessions_to_activity_rows, call:rows.sort, call:state_rank.get, call:r.get, call:str(r.get("user", "")).lower, call:logger.info, call:len, func:get_now_playing(client) → list[dict[str, Any]], call:get_activity, func:get_backup_dashboard(store) → BackupDashboardSummary, call:build_backup_dashboard_summary | dep: logging, typing, fastapi, media_library_viewer_api.clients.jellyfin, media_library_viewer_api.dependencies, media_library_viewer_api.domain.dashboard, media_library_viewer_api.models.backups, media_library_viewer_api.services.settings_store
|
||||
- files.py | FastAPI router providing endpoints for remote file operations including directory listing, ffprobe media analysis, stat, and path resolution via SSH. | exp: func:list_directory(path, ssh) → dict[str, Any], call:ssh.list_dir, call:logger.warning, call:json.loads, call:logger.info, call:len, raise:HTTPException, func:get_ffprobe(path, ssh) → dict[str, Any], call:ssh.ffprobe_json, call:logger.warning, call:logger.info, raise:HTTPException, func:get_stat(path, ssh) → dict[str, str], call:ssh.stat_path, call:logger.warning, call:logger.info, raise:HTTPException, func:resolve_path(path) → dict[str, str], call:get_settings, call:resolve_remote_media_path, call:logger.info | dep: json, logging, typing, fastapi, media_library_viewer_api.clients.ssh, media_library_viewer_api.config, media_library_viewer_api.dependencies, media_library_viewer_api.path_utils
|
||||
- jobs.py | FastAPI router that exposes endpoints to list available job templates and execute them on remote paths via SSH. | exp: class:RunJobRequest, func:get_templates() → list[dict[str, str]], call:JOB_TEMPLATES.items, call:logger.info, call:len, func:post_run_job(request: RunJobRequest, ssh) → dict[str, Any], call:logger.warning, call:logger.info, call:run_job, raise:HTTPException | dep: logging, typing, fastapi, pydantic, media_library_viewer_api.clients.ssh, media_library_viewer_api.dependencies, media_library_viewer_api.jobs
|
||||
- media.py | FastAPI router that manages media index lifecycle (status, build, stop, query) with subprocess worker orchestration and cooperative/force cancellation. | exp: func:get_media_index() → MediaIndex, call:MediaIndex, func:_set_build_metadata(index: MediaIndex, state: dict[str, Any]) → None, call:state.items, call:index.set_metadata, func:_staging_db_path(index: MediaIndex) → Path, call:index.db_path.with_name, func:_pid_is_alive(pid: int | None) → bool, call:os.kill, func:_clean_stale_build_state(index: MediaIndex) → Any, call:index.status, call:_pid_is_alive, call:logger.warning, call:_set_build_metadata, func:_serialize_status(status: Any) → dict[str, Any], func:_worker_command(final_db_path: Path, staging_db_path: Path) → list[str], call:str, func:_start_worker(index: MediaIndex) → subprocess.Popen[bytes], call:_staging_db_path, call:staging_path.unlink, call:subprocess.Popen, call:_worker_command, call:os.environ.copy, func:get_index_status(index) → dict[str, Any], call:_clean_stale_build_state, call:logger.info, call:_serialize_status, func:post_build_index(client, user_id, index) → dict[str, Any], call:_clean_stale_build_state, call:_pid_is_alive, call:logger.warning, call:client.libraries, call:logger.info, call:len, call:_start_worker, call:_set_build_metadata, call:index.status, call:record_media_index_build, call:_serialize_status, raise:HTTPException, func:stop_build(index) → dict[str, Any], call:_clean_stale_build_state, call:logger.warning, call:logger.info, call:_set_build_metadata, call:index.status, call:_serialize_status, raise:HTTPException, func:force_stop_build(index) → dict[str, Any], call:_clean_stale_build_state, call:logger.warning, call:_pid_is_alive, call:_set_build_metadata, call:index.status, call:_serialize_status, call:logger.info, call:os.killpg, call:time.time, call:time.sleep, call:record_media_index_build, raise:HTTPException, func:query_media(libraries, types, search, hdr_filter, sort_key, sort_order, limit, offset, client, user_id, index) → dict[str, Any], call:lid.strip, call:libraries.split, call:client.libraries, call:t.strip, call:types.split, call:logger.info, call:len, call:",".join, call:index.query | dep: logging, os, signal, subprocess, sys, threading, time, pathlib, typing, fastapi, media_library_viewer_api.clients.jellyfin, media_library_viewer_api.dependencies, media_library_viewer_api.observability, media_library_viewer_api.services.media_index, media_library_viewer_api.workers.media_index_worker
|
||||
- monitoring.py | FastAPI router providing endpoints to check the health/status of Alertmanager, Grafana, and Prometheus services and expose Prometheus scrape targets. | exp: func:_resolve_service_record(store: SettingsStore, service_type: str, service_id) → ServiceRecord | None, call:store.get_service, call:row.get, call:build_service_record, call:store.list_services, func:_base_url(service: ServiceRecord) → str, call:str(service.config.get("base_url") or "").rstrip, call:service.config.get, func:_timeout(service: ServiceRecord, default: int) → int, call:int, call:service.config.get, func:_auth_headers(service: ServiceRecord) → dict[str, str], call:str, call:service.secrets.get, func:_status_response(service: ServiceRecord | None, version, error) → dict[str, Any], func:_summary_from_alerts(alerts: list[dict[str, Any]]) → dict[str, Any], call:summarize_alerts, func:get_machines(store) → list[dict[str, Any]], call:store.list_machines, call:m.get, func:get_prometheus_targets(store) → list[dict[str, Any]], call:build_node_exporter_targets, call:logger.info, call:len, func:get_alertmanager_alerts(service_id, store) → dict[str, Any], call:_resolve_service_record, call:requests.get, call:_base_url, call:_auth_headers, call:_timeout, call:response.raise_for_status, call:response.json, call:logger.exception, call:data.get, call:_summary_from_alerts, call:logger.info, func:get_alertmanager_status(service_id, store) → dict[str, Any], call:_resolve_service_record, call:requests.get, call:_base_url, call:_auth_headers, call:_timeout, call:response.raise_for_status, call:response.json, call:logger.exception, call:data.get("versionInfo", {}).get, call:status.get, call:p.get, call:cluster.get, func:get_grafana_status(service_id, store) → dict[str, Any], call:_resolve_service_record, call:_status_response, call:requests.get, call:_base_url, call:_auth_headers, call:_timeout, call:response.raise_for_status, call:response.json, call:logger.exception, call:data.get, func:get_prometheus_status(service_id, store) → dict[str, Any], call:_resolve_service_record, call:_status_response, call:_base_url, call:_timeout, call:_auth_headers, call:requests.get, call:health.raise_for_status, call:build_info.raise_for_status, call:build_info.json().get("data", {}).get, call:logger.exception, func:receive_alertmanager_webhook(payload) → dict[str, str], call:payload.get, call:logger.info, call:len | dep: logging, typing, requests, fastapi, media_library_viewer_api.dependencies, media_library_viewer_api.services.settings_store, media_library_viewer_api.services.targets, media_library_viewer_api.widgets.sources, media_library_viewer_api.integrations.alertmanager
|
||||
- services.py | Provides REST API endpoints for managing service instances in a service registry, including listing service types and CRUD operations for instances while ensuring plaintext secrets are never exposed. | exp: func:_to_type_info(service_type: str) → ServiceTypeInfo, call:require_service_definition, call:ServiceTypeInfo, call:SecretFieldInfo, call:WidgetKindInfo, func:_to_instance(row: dict[str, Any]) → ServiceInstance, call:get_service_definition, call:set, call:row.get, call:bool, call:ServiceInstance, func:_validate_input(body: ServiceInstanceInput) → None, call:get_service_definition, call:validate_config, call:set, raise:HTTPException, func:list_types() → list[ServiceTypeInfo], call:_to_type_info, call:sorted, func:list_instances(service_type, store) → list[ServiceInstance], call:store.list_services, call:_to_instance, func:create_instance(body: ServiceInstanceInput, store) → ServiceInstance, call:_validate_input, call:store.upsert_service, call:_to_instance, func:update_instance(service_id: str, body: ServiceInstanceInput, store) → ServiceInstance, call:store.get_service, call:_validate_input, call:store.upsert_service, call:_to_instance, raise:HTTPException, func:delete_instance(service_id: str, store) → dict[str, str], call:store.get_service, call:store.delete_service, raise:HTTPException | dep: logging, typing, fastapi, media_library_viewer_api.dependencies, media_library_viewer_api.integrations.base, media_library_viewer_api.integrations.registry, media_library_viewer_api.models.services, media_library_viewer_api.services.settings_store
|
||||
- settings.py | FastAPI router for managing machine definitions, SSH keys, SSH connection validation, and local database resets. | exp: class:MonitoringMachineInput, class:SSHKeyInput, class:SSHKeyGenerateInput, class:ResetLocalDatabaseInput, func:get_machines(store) → list[dict[str, Any]], call:store.list_machines, func:_resolve_ssh_client(machine: MonitoringMachineInput, store: SettingsStore) → tuple[RemoteSSHClient, str, int], call:machine.host.strip, call:machine.username.strip, call:int, call:store.get_ssh_key, call:str, call:ssh_key.get, call:get_settings, call:RemoteSSHClient, raise:HTTPException, func:_raise_ssh_validation_error(host: str, port: int, exc: Exception) → None, call:str, call:message.lower, raise:HTTPException, func:_validate_saved_machine_ssh(machine: MonitoringMachineInput, store: SettingsStore) → None, call:str(machine.mode or "").strip().lower, call:_resolve_ssh_client, call:client.connect, call:_raise_ssh_validation_error, call:client.close, func:test_machine_ssh(machine: MonitoringMachineInput, store) → dict[str, Any], call:str(machine.mode or "").strip().lower, call:_resolve_ssh_client, call:get_settings, call:has_known_host, call:client.connect, call:message.lower, call:client.close, raise:HTTPException, func:post_machine(machine: MonitoringMachineInput, store) → dict[str, Any], call:store.upsert_machine, call:machine.model_dump, call:MonitoringMachineInput.model_validate, call:_validate_saved_machine_ssh, func:put_machine(machine_id: str, machine: MonitoringMachineInput, store) → dict[str, Any], call:store.get_machine, call:store.upsert_machine, call:machine.model_dump, call:MonitoringMachineInput.model_validate, call:_validate_saved_machine_ssh, raise:HTTPException, func:delete_machine(machine_id: str, store) → dict[str, str], call:store.get_machine, call:store.delete_machine, raise:HTTPException, func:generate_ssh_key(payload: SSHKeyGenerateInput) → dict[str, Any], call:paramiko.RSAKey.generate, call:StringIO, call:key.write_private_key, call:private_buffer.getvalue, call:key.get_name, call:key.get_base64, call:":".join, call:key.get_fingerprint, func:get_ssh_keys(store) → list[dict[str, Any]], call:store.list_ssh_keys, func:post_ssh_key(key: SSHKeyInput, store) → dict[str, Any], call:store.upsert_ssh_key, call:key.model_dump, func:put_ssh_key(key_id: str, key: SSHKeyInput, store) → dict[str, Any], call:store.get_ssh_key, call:store.upsert_ssh_key, call:key.model_dump, raise:HTTPException, func:delete_ssh_key(key_id: str, store) → dict[str, str], call:store.get_ssh_key, call:store.delete_ssh_key, raise:HTTPException, func:reset_local_database(payload: ResetLocalDatabaseInput, store) → dict[str, Any], call:payload.confirm_phrase.strip().upper, call:remove_sqlite_database, call:MediaIndex, call:bool, raise:HTTPException | dep: logging, io, typing, paramiko, fastapi, pydantic, media_library_viewer_api.clients.ssh, media_library_viewer_api.config, media_library_viewer_api.dependencies, media_library_viewer_api.services.db_maintenance, media_library_viewer_api.services.known_hosts, media_library_viewer_api.services.media_index, media_library_viewer_api.services.settings_store
|
||||
- tasks.py | FastAPI router providing CRUD endpoints and execution for saved server tasks with SSH service resolution | exp: class:TaskInput, class:RunTaskRequest, func:_service_label(service: dict[str, Any] | None) → str, call:str, call:service.get, func:_resolve_service_for_task(store: SettingsStore, task: dict[str, Any], service_id: str | None) → dict[str, Any] | None, call:store.get_service, call:str(task.get("default_service_id") or "").strip, call:task.get, call:store.list_services, call:svc.get, func:_service_row_to_record(service_row: dict[str, Any]) → ServiceRecord, call:build_service_record, call:get_settings_store, func:list_tasks(store) → list[dict[str, Any]], call:store.list_tasks, func:create_task(task: TaskInput, store) → dict[str, Any], call:store.upsert_task, call:task.model_dump, func:update_task(task_id: str, task: TaskInput, store) → dict[str, Any], call:store.get_task, call:store.upsert_task, call:task.model_dump, raise:HTTPException, func:delete_task(task_id: str, store) → dict[str, str], call:store.get_task, call:store.delete_task, raise:HTTPException, func:list_task_runs(task_id: str, limit, store) → dict[str, Any], call:store.get_task, call:store.list_service_task_runs, call:len, raise:HTTPException, func:run_task(request: RunTaskRequest, service_id, store) → dict[str, Any], call:store.get_task, call:task.get, call:_resolve_service_for_task, call:service_row.get, call:_service_row_to_record, call:run_saved_task, call:_service_label, raise:HTTPException | dep: logging, typing, fastapi, pydantic, media_library_viewer_api.dependencies, media_library_viewer_api.services.settings_store, media_library_viewer_api.services.task_runner, media_library_viewer_api.widgets.sources
|
||||
- users.py | Re-exports all public names from the users_impl module to provide a unified public API for user-related functionality. | dep: .users_impl, users_impl
|
||||
- users_impl.py | FastAPI router that fetches and merges Jellyfin users with optional Jellyseerr enrichment, plus endpoints for queueing user emails via a background mail worker. | exp: func:_safe_int(value: Any) → int, call:int, func:_permission_labels(permissions: int) → list[str], func:_role_label(permissions: int) → str, func:_account_type(user_type: Any) → str, call:_USER_TYPES.get, call:_safe_int, func:_merge_users(jellyfin_users: list[dict[str, Any]], jellyseerr_jellyfin_users: list[dict[str, Any]] | None, jellyseerr_users: list[dict[str, Any]] | None, jellyseerr_client: JellyseerrClient | None) → dict[str, Any], call:str(value or "").strip().lower, call:bool, call:_looks_like_email, call:str(value).strip, call:", ".join, call:_normalize, call:item.get, call:_lookup_keys, call:user.get, call:linked_by_jellyfin_id.get, call:(jf_link or {}).get, call:seerr_by_key.get, call:_pick_source_and_value, call:(seerr_user or {}).get, call:_first_value, call:jellyseerr_client.absolute_url, call:_safe_int, call:_role_label, call:_source_summary, call:items.append, call:_account_type, call:_permission_labels, call:logger.info, call:len, func:_normalize(value: Any) → str, call:str(value or "").strip().lower, func:_looks_like_email(value: Any) → bool, call:str(value or "").strip, call:bool, func:_pick_source_and_value(candidates: list[tuple[str, Any]]) → tuple[str, str], call:_looks_like_email, call:str(value).strip, func:_first_value(candidates: list[tuple[str, Any]]) → tuple[str, str], call:str(value or "").strip, func:_source_summary(name_source: str, email_source: str, avatar_source: str, access_source: str) → str, call:", ".join, func:_lookup_keys(item: dict[str, Any]) → list[str], call:_normalize, call:item.get, func:get_users(jellyfin, jellyseerr) → dict[str, Any], call:jellyfin.users, call:logger.info, call:len, call:jellyseerr.jellyfin_users, call:logger.exception, call:jellyseerr.users, call:_merge_users, call:bool, func:get_user_message_status() → dict[str, Any], call:mail_queue.status, func:post_user_message(recipient_ids, subject, html_body, text_body, attachments, jellyfin, jellyseerr) → dict[str, Any], call:json.loads, call:isinstance, call:str(item).strip, call:subject.strip, call:get_users, call:item.get, call:directory.get, call:users_by_id.get, call:skipped.append, call:str(item.get("email") or "").strip, call:recipients.append, call:recipient_labels.append, call:get_settings, call:validate_smtp_settings, call:mail_queue.status, call:upload.read, call:attachment_payloads.append, call:EmailAttachment, call:mail_queue.enqueue, call:str(getattr(settings, "smtp_from_address", "") or "").strip, call:getattr, call:str(getattr(settings, "smtp_username", "") or "").strip, call:logger.info, call:len, raise:HTTPException | dep: json, logging, typing, fastapi, media_library_viewer_api.clients.jellyfin, media_library_viewer_api.clients.jellyseerr, media_library_viewer_api.config, media_library_viewer_api.dependencies, media_library_viewer_api.services.mailer
|
||||
- widgets.py | Provides a FastAPI REST API for managing dashboard widget instances, including CRUD operations, built-in widget discovery, and data fetching through service or built-in adapters. | exp: func:_validate_widget_input(body: WidgetInstanceInput, store: SettingsStore) → None, call:store.get_service, call:get_service_definition, call:definition.widget_kind, call:validate_config, call:is_builtin_kind, call:validate_builtin_config, raise:HTTPException, func:list_builtin_kinds() → list[BuiltinWidgetKindInfo], call:BuiltinWidgetKindInfo, call:BUILTIN_WIDGET_KINDS.values, func:list_instances(store) → list[dict[str, Any]], call:WidgetInstance(**widget).model_dump, call:store.list_widgets, func:create_instance(body: WidgetInstanceInput, store) → dict[str, Any], call:_validate_widget_input, call:store.upsert_widget, call:body.model_dump, call:WidgetInstance(**widget).model_dump, func:update_instance(widget_id: str, body: WidgetInstanceInput, store) → dict[str, Any], call:store.get_widget, call:_validate_widget_input, call:store.upsert_widget, call:body.model_dump, call:WidgetInstance(**widget).model_dump, raise:HTTPException, func:delete_instance(widget_id: str, store) → dict[str, str], call:store.get_widget, call:store.delete_widget, raise:HTTPException, func:fetch_data(widget_id: str, store) → dict[str, Any], call:store.get_widget, call:widget.get, call:store.get_service, call:WidgetDataResponse( widget_id=widget_id, error=f"Service {service_id} not found", fetched_at=int(time.time()), ).model_dump, call:int, call:time.time, call:service_row.get, call:WidgetDataResponse( widget_id=widget_id, error="Service is disabled", fetched_at=int(time.time()), ).model_dump, call:get_service_adapter, call:WidgetDataResponse( widget_id=widget_id, error=f"No adapter for service type {service_row['service_type']}", fetched_at=int(time.time()), ).model_dump, call:build_service_record, call:get_builtin_adapter, call:WidgetDataResponse( widget_id=widget_id, error=f"Unknown built-in widget kind: {widget_kind}", fetched_at=int(time.time()), ).model_dump, call:adapter.fetch, call:logger.exception, call:WidgetDataResponse( widget_id=widget_id, data=data if "error" not in data else None, error=data.get("error"), fetched_at=int(time.time()), ).model_dump, call:data.get, raise:HTTPException | dep: logging, time, typing, fastapi, media_library_viewer_api.dependencies, media_library_viewer_api.integrations.base, media_library_viewer_api.integrations.registry, media_library_viewer_api.models.widgets, media_library_viewer_api.services.settings_store, media_library_viewer_api.widgets.builtin, media_library_viewer_api.widgets.sources
|
||||
## arch
|
||||
Modular router-per-domain pattern where each file defines an isolated FastAPI APIRouter; routers are registered by the parent application and share common dependencies for SSH orchestration, database access, and service resolution.
|
||||
## tags
|
||||
call:, service, raise:httpexception, get, backup, media_library_viewer_api, ssh, call:logger.info
|
||||
## symbols
|
||||
- RunJobRequest
|
||||
- MonitoringMachineInput
|
||||
- SSHKeyInput
|
||||
- SSHKeyGenerateInput
|
||||
- ResetLocalDatabaseInput
|
||||
- TaskInput
|
||||
- RunTaskRequest
|
||||
- _get_or_create_job
|
||||
## workflows
|
||||
- change routers behavior
|
||||
read: __init__.py, backups.py, dashboard.py
|
||||
## dirty
|
||||
-
|
||||
@@ -1,64 +1,76 @@
|
||||
"""Monitoring router — observability stack status (Alertmanager + Prometheus)."""
|
||||
"""Monitoring router — observability service status.
|
||||
|
||||
Observability components (Alertmanager, Grafana, Prometheus) are resolved from
|
||||
the service registry, not environment variables. The endpoints pick the first
|
||||
enabled instance of a type when no ``service_id`` is given, and return graceful
|
||||
"not configured" / "unreachable" payloads so the UI always renders a health card.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
import requests
|
||||
from fastapi import APIRouter, Body, Depends
|
||||
|
||||
from media_library_viewer_api.config import get_settings
|
||||
from media_library_viewer_api.dependencies import get_settings_store
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
from media_library_viewer_api.services.targets import build_node_exporter_targets
|
||||
from media_library_viewer_api.widgets.sources import ServiceRecord, build_service_record
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _alertmanager_client() -> Any:
|
||||
"""Return a simple HTTP client for the configured Alertmanager URL."""
|
||||
import requests
|
||||
def _resolve_service_record(
|
||||
store: SettingsStore, service_type: str, service_id: str | None = None
|
||||
) -> ServiceRecord | None:
|
||||
"""Return the requested service instance, else the first enabled one.
|
||||
|
||||
settings = get_settings()
|
||||
return requests.Session(), settings.alertmanager_url
|
||||
Returns ``None`` when the instance does not exist / is the wrong type, or
|
||||
when no enabled instance of ``service_type`` is configured.
|
||||
"""
|
||||
if service_id:
|
||||
row = store.get_service(service_id)
|
||||
if not row or row.get("service_type") != service_type:
|
||||
return None
|
||||
if not row.get("enabled", True):
|
||||
return None
|
||||
return build_service_record(store, row)
|
||||
for row in store.list_services(service_type):
|
||||
if row.get("enabled", True):
|
||||
return build_service_record(store, row)
|
||||
return None
|
||||
|
||||
|
||||
def _webhook_client() -> Any:
|
||||
"""Return a simple HTTP client for the optional webhook receiver URL."""
|
||||
import requests
|
||||
def _base_url(service: ServiceRecord) -> str:
|
||||
return str(service.config.get("base_url") or "").rstrip("/")
|
||||
|
||||
settings = get_settings()
|
||||
return requests.Session(), settings.alertmanager_webhook_url
|
||||
|
||||
def _timeout(service: ServiceRecord, default: int) -> int:
|
||||
return int(service.config.get("timeout_seconds") or default)
|
||||
|
||||
|
||||
def _auth_headers(service: ServiceRecord) -> dict[str, str]:
|
||||
api_key = str(service.secrets.get("api_key") or "")
|
||||
return {"Authorization": f"Bearer {api_key}"} if api_key else {}
|
||||
|
||||
|
||||
def _status_response(service: ServiceRecord | None, *, version: str = "", error: str | None = None) -> dict[str, Any]:
|
||||
return {
|
||||
"up": error is None,
|
||||
"version": version or "",
|
||||
"service_id": service.id if service else "",
|
||||
"name": service.name if service else "",
|
||||
"error": error,
|
||||
}
|
||||
|
||||
|
||||
def _summary_from_alerts(alerts: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
"""Build a UI-friendly summary from Alertmanager /api/v1/alerts payload."""
|
||||
by_severity: dict[str, int] = {}
|
||||
open_alerts: list[dict[str, Any]] = []
|
||||
for alert in alerts:
|
||||
labels = alert.get("labels") or {}
|
||||
annotations = alert.get("annotations") or {}
|
||||
severity = labels.get("severity", "unknown")
|
||||
by_severity[severity] = by_severity.get(severity, 0) + 1
|
||||
open_alerts.append(
|
||||
{
|
||||
"name": labels.get("alertname", "unknown"),
|
||||
"severity": severity,
|
||||
"category": labels.get("category", ""),
|
||||
"job_name": labels.get("job_name", labels.get("job", "")),
|
||||
"summary": annotations.get("summary", ""),
|
||||
"description": annotations.get("description", ""),
|
||||
"active_since": alert.get("startsAt"),
|
||||
"state": alert.get("status", "firing"),
|
||||
"labels": labels,
|
||||
}
|
||||
)
|
||||
open_alerts.sort(key=lambda a: (a["severity"] not in {"critical", "warning"}, a["severity"], a["name"]))
|
||||
return {
|
||||
"total": len(open_alerts),
|
||||
"by_severity": by_severity,
|
||||
"alerts": open_alerts[:50],
|
||||
}
|
||||
from media_library_viewer_api.integrations.alertmanager import summarize_alerts
|
||||
|
||||
return summarize_alerts(alerts)
|
||||
|
||||
|
||||
router = APIRouter(prefix="/api/monitoring", tags=["monitoring"])
|
||||
@@ -72,11 +84,9 @@ def get_machines(store: SettingsStore = Depends(get_settings_store)) -> list[dic
|
||||
|
||||
@router.get("/prometheus-targets")
|
||||
def get_prometheus_targets(store: SettingsStore = Depends(get_settings_store)) -> list[dict[str, Any]]:
|
||||
"""Return Prometheus file-SD targets for remote Node Exporters.
|
||||
"""Return Prometheus scrape targets for remote Node Exporters.
|
||||
|
||||
The backend writes these targets to a JSON file that Prometheus reads via
|
||||
file_sd_configs. This endpoint returns the same list live from the store so
|
||||
the UI can preview which machines will be scraped.
|
||||
External Prometheus instances consume this list via ``http_sd_configs``.
|
||||
"""
|
||||
targets = build_node_exporter_targets(store)
|
||||
logger.info("Prometheus targets requested count=%s", len(targets))
|
||||
@@ -84,47 +94,90 @@ def get_prometheus_targets(store: SettingsStore = Depends(get_settings_store)) -
|
||||
|
||||
|
||||
@router.get("/alerts")
|
||||
def get_alertmanager_alerts() -> dict[str, Any]:
|
||||
def get_alertmanager_alerts(
|
||||
service_id: str | None = None,
|
||||
store: SettingsStore = Depends(get_settings_store),
|
||||
) -> dict[str, Any]:
|
||||
"""Return a summary of active Alertmanager alerts for the UI.
|
||||
|
||||
Proxies the Alertmanager `/api/v1/alerts` endpoint and reshapes the payload
|
||||
into a stable, UI-friendly format. If Alertmanager is unreachable, the
|
||||
endpoint returns an empty summary and logs the failure so the UI can still
|
||||
render a health card instead of an error page.
|
||||
Resolves an ``alertmanager`` service instance from the registry. When none
|
||||
is configured the endpoint returns an empty summary with an
|
||||
``alertmanager_not_configured`` error so the UI can render a health card.
|
||||
"""
|
||||
session, base_url = _alertmanager_client()
|
||||
service = _resolve_service_record(store, "alertmanager", service_id)
|
||||
if service is None:
|
||||
return {"total": 0, "by_severity": {}, "alerts": [], "error": "alertmanager_not_configured"}
|
||||
try:
|
||||
response = session.get(f"{base_url}/api/v1/alerts", timeout=5)
|
||||
response = requests.get(
|
||||
f"{_base_url(service)}/api/v1/alerts",
|
||||
headers=_auth_headers(service),
|
||||
timeout=_timeout(service, 5),
|
||||
)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
except Exception:
|
||||
logger.exception("Failed to fetch Alertmanager alerts from %s", base_url)
|
||||
return {"total": 0, "by_severity": {}, "alerts": [], "error": "alertmanager_unreachable"}
|
||||
logger.exception("Failed to fetch Alertmanager alerts")
|
||||
return {
|
||||
"total": 0,
|
||||
"by_severity": {},
|
||||
"alerts": [],
|
||||
"error": "alertmanager_unreachable",
|
||||
"service_id": service.id,
|
||||
"name": service.name,
|
||||
}
|
||||
|
||||
if data.get("status") != "success":
|
||||
return {"total": 0, "by_severity": {}, "alerts": [], "error": data.get("error", "unknown")}
|
||||
return {
|
||||
"total": 0,
|
||||
"by_severity": {},
|
||||
"alerts": [],
|
||||
"error": data.get("error", "unknown"),
|
||||
"service_id": service.id,
|
||||
"name": service.name,
|
||||
}
|
||||
|
||||
summary = _summary_from_alerts(data.get("data", []))
|
||||
summary["service_id"] = service.id
|
||||
summary["name"] = service.name
|
||||
logger.info("Alertmanager alerts requested total=%s", summary["total"])
|
||||
return summary
|
||||
|
||||
|
||||
@router.get("/alertmanager-status")
|
||||
def get_alertmanager_status() -> dict[str, Any]:
|
||||
"""Return Alertmanager cluster/status for the UI health card.
|
||||
|
||||
Uses the Alertmanager `/api/v2/status` endpoint and exposes only the high-
|
||||
level fields the UI needs: uptime, version, and whether the cluster is
|
||||
healthy.
|
||||
"""
|
||||
session, base_url = _alertmanager_client()
|
||||
def get_alertmanager_status(
|
||||
service_id: str | None = None,
|
||||
store: SettingsStore = Depends(get_settings_store),
|
||||
) -> dict[str, Any]:
|
||||
"""Return Alertmanager cluster/status for the UI health card."""
|
||||
service = _resolve_service_record(store, "alertmanager", service_id)
|
||||
if service is None:
|
||||
return {
|
||||
"up": False,
|
||||
"version": "",
|
||||
"uptime": "",
|
||||
"name": "",
|
||||
"peers": [],
|
||||
"error": "alertmanager_not_configured",
|
||||
}
|
||||
try:
|
||||
response = session.get(f"{base_url}/api/v2/status", timeout=5)
|
||||
response = requests.get(
|
||||
f"{_base_url(service)}/api/v2/status",
|
||||
headers=_auth_headers(service),
|
||||
timeout=_timeout(service, 5),
|
||||
)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
except Exception:
|
||||
logger.exception("Failed to fetch Alertmanager status from %s", base_url)
|
||||
return {"up": False, "version": "", "uptime": ""}
|
||||
logger.exception("Failed to fetch Alertmanager status")
|
||||
return {
|
||||
"up": False,
|
||||
"version": "",
|
||||
"uptime": "",
|
||||
"name": service.name,
|
||||
"peers": [],
|
||||
"service_id": service.id,
|
||||
"error": "alertmanager_unreachable",
|
||||
}
|
||||
|
||||
cluster = data.get("cluster") or {}
|
||||
status = data.get("clusterStatus") or {}
|
||||
@@ -132,20 +185,68 @@ def get_alertmanager_status() -> dict[str, Any]:
|
||||
"up": True,
|
||||
"version": data.get("versionInfo", {}).get("version", ""),
|
||||
"uptime": status.get("createdAt", ""),
|
||||
"name": "",
|
||||
"name": service.name,
|
||||
"peers": [p.get("name", "") for p in cluster.get("peers", [])],
|
||||
"service_id": service.id,
|
||||
"error": None,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/grafana-status")
|
||||
def get_grafana_status(
|
||||
service_id: str | None = None,
|
||||
store: SettingsStore = Depends(get_settings_store),
|
||||
) -> dict[str, Any]:
|
||||
"""Probe a Grafana service instance's ``/api/health`` endpoint."""
|
||||
service = _resolve_service_record(store, "grafana", service_id)
|
||||
if service is None:
|
||||
return _status_response(None, error="no_service_configured")
|
||||
try:
|
||||
response = requests.get(
|
||||
f"{_base_url(service)}/api/health",
|
||||
headers=_auth_headers(service),
|
||||
timeout=_timeout(service, 5),
|
||||
)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
except Exception:
|
||||
logger.exception("Failed to fetch Grafana status")
|
||||
return _status_response(service, error="grafana_unreachable")
|
||||
return _status_response(service, version=data.get("version", ""))
|
||||
|
||||
|
||||
@router.get("/prometheus-status")
|
||||
def get_prometheus_status(
|
||||
service_id: str | None = None,
|
||||
store: SettingsStore = Depends(get_settings_store),
|
||||
) -> dict[str, Any]:
|
||||
"""Probe a Prometheus service instance's health and build info."""
|
||||
service = _resolve_service_record(store, "prometheus", service_id)
|
||||
if service is None:
|
||||
return _status_response(None, error="no_service_configured")
|
||||
base = _base_url(service)
|
||||
timeout = _timeout(service, 10)
|
||||
headers = _auth_headers(service)
|
||||
try:
|
||||
health = requests.get(f"{base}/-/healthy", headers=headers, timeout=timeout)
|
||||
health.raise_for_status()
|
||||
build_info = requests.get(f"{base}/api/v1/status/buildinfo", headers=headers, timeout=timeout)
|
||||
build_info.raise_for_status()
|
||||
version = build_info.json().get("data", {}).get("version", "")
|
||||
except Exception:
|
||||
logger.exception("Failed to fetch Prometheus status")
|
||||
return _status_response(service, error="prometheus_unreachable")
|
||||
return _status_response(service, version=version)
|
||||
|
||||
|
||||
@router.post("/alertmanager-webhook")
|
||||
def receive_alertmanager_webhook(payload: dict[str, Any] = Body(...)) -> dict[str, str]:
|
||||
"""Receive alerts from Alertmanager and optionally forward to a webhook URL.
|
||||
"""Receive alerts from Alertmanager and log them for audit/debug.
|
||||
|
||||
This endpoint is the receiver referenced by the optional `webhook_configs`
|
||||
block in Alertmanager. It logs the payload for audit/debug purposes and, if
|
||||
`ALERTMANAGER_WEBHOOK_URL` is configured, forwards the alert JSON verbatim.
|
||||
Forwarding is best-effort: a failure to reach the downstream webhook does
|
||||
not fail this endpoint, so Alertmanager sees a successful delivery.
|
||||
This endpoint is the receiver referenced by the optional ``webhook_configs``
|
||||
block in Alertmanager. It is log-only: received payloads are recorded but not
|
||||
forwarded anywhere. (The previous outbound relay to ``ALERTMANAGER_WEBHOOK_URL``
|
||||
was removed when observability became service-registry configured.)
|
||||
"""
|
||||
alerts = payload.get("alerts", [])
|
||||
logger.info(
|
||||
@@ -153,16 +254,4 @@ def receive_alertmanager_webhook(payload: dict[str, Any] = Body(...)) -> dict[st
|
||||
len(alerts),
|
||||
payload.get("status", "unknown"),
|
||||
)
|
||||
|
||||
session, webhook_url = _webhook_client()
|
||||
if webhook_url:
|
||||
try:
|
||||
response = session.post(webhook_url, json=payload, timeout=10)
|
||||
response.raise_for_status()
|
||||
logger.info("Forwarded Alertmanager webhook to %s", webhook_url)
|
||||
except Exception:
|
||||
logger.exception("Failed to forward Alertmanager webhook to %s", webhook_url)
|
||||
else:
|
||||
logger.debug("No ALERTMANAGER_WEBHOOK_URL configured; webhook stored in logs only")
|
||||
|
||||
return {"status": "received"}
|
||||
|
||||
@@ -17,7 +17,6 @@ from media_library_viewer_api.services.db_maintenance import remove_sqlite_datab
|
||||
from media_library_viewer_api.services.known_hosts import has_known_host
|
||||
from media_library_viewer_api.services.media_index import MediaIndex
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
from media_library_viewer_api.services.targets import write_prometheus_targets
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -41,13 +40,6 @@ class MonitoringMachineInput(BaseModel):
|
||||
ssh_private_key: str = ""
|
||||
ssh_private_key_passphrase: str = ""
|
||||
password: str = ""
|
||||
media_root: str = ""
|
||||
path_prefix: str = ""
|
||||
jellyfin_url: str = ""
|
||||
jellyfin_user_id: str = ""
|
||||
jellyfin_api_key: str = ""
|
||||
jellyseerr_url: str = ""
|
||||
jellyseerr_api_key: str = ""
|
||||
notes: str = ""
|
||||
|
||||
|
||||
@@ -126,14 +118,6 @@ def _validate_saved_machine_ssh(machine: MonitoringMachineInput, store: Settings
|
||||
client.close()
|
||||
|
||||
|
||||
def _write_prometheus_targets(store: SettingsStore) -> None:
|
||||
"""Regenerate Prometheus file-SD targets after machine changes."""
|
||||
try:
|
||||
write_prometheus_targets(store)
|
||||
except Exception:
|
||||
logger.exception("Failed to write Prometheus file-SD targets")
|
||||
|
||||
|
||||
@router.post("/machines/test-ssh")
|
||||
def test_machine_ssh(
|
||||
machine: MonitoringMachineInput,
|
||||
@@ -194,7 +178,6 @@ def post_machine(
|
||||
store: SettingsStore = Depends(get_settings_store),
|
||||
) -> dict[str, Any]:
|
||||
saved = store.upsert_machine(machine.model_dump(exclude_none=True), machine.id)
|
||||
_write_prometheus_targets(store)
|
||||
saved_machine = MonitoringMachineInput.model_validate(saved)
|
||||
_validate_saved_machine_ssh(saved_machine, store)
|
||||
return saved
|
||||
@@ -209,7 +192,6 @@ def put_machine(
|
||||
if not store.get_machine(machine_id):
|
||||
raise HTTPException(status_code=404, detail="Machine not found")
|
||||
saved = store.upsert_machine(machine.model_dump(exclude_none=True), machine_id)
|
||||
_write_prometheus_targets(store)
|
||||
saved_machine = MonitoringMachineInput.model_validate(saved)
|
||||
_validate_saved_machine_ssh(saved_machine, store)
|
||||
return saved
|
||||
@@ -220,7 +202,6 @@ def delete_machine(machine_id: str, store: SettingsStore = Depends(get_settings_
|
||||
if not store.get_machine(machine_id):
|
||||
raise HTTPException(status_code=404, detail="Machine not found")
|
||||
store.delete_machine(machine_id)
|
||||
_write_prometheus_targets(store)
|
||||
return {"status": "deleted"}
|
||||
|
||||
|
||||
|
||||
@@ -3,18 +3,15 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import shlex
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from media_library_viewer_api.clients.local import LocalCommandClient
|
||||
from media_library_viewer_api.clients.ssh import RemoteSSHClient
|
||||
from media_library_viewer_api.config import get_settings
|
||||
from media_library_viewer_api.dependencies import get_settings_store
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
from media_library_viewer_api.services.task_runner import run_saved_task
|
||||
from media_library_viewer_api.widgets.sources import ServiceRecord, build_service_record
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -27,7 +24,7 @@ class TaskInput(BaseModel):
|
||||
task_type: str = Field(default="shell", description="shell or python")
|
||||
content: str = Field(default="")
|
||||
enabled: bool = True
|
||||
default_machine_id: str = ""
|
||||
default_service_id: str = ""
|
||||
notes: str = ""
|
||||
|
||||
|
||||
@@ -35,61 +32,31 @@ class RunTaskRequest(BaseModel):
|
||||
task_id: str
|
||||
|
||||
|
||||
def _machine_label(machine: dict[str, Any] | None) -> str:
|
||||
if not machine:
|
||||
def _service_label(service: dict[str, Any] | None) -> str:
|
||||
if not service:
|
||||
return ""
|
||||
return str(machine.get("name") or machine.get("host") or machine.get("id") or "")
|
||||
return str(service.get("name") or service.get("id") or "")
|
||||
|
||||
|
||||
def _resolve_machine_for_task(
|
||||
def _resolve_service_for_task(
|
||||
store: SettingsStore,
|
||||
task: dict[str, Any],
|
||||
machine_id: str | None,
|
||||
service_id: str | None,
|
||||
) -> dict[str, Any] | None:
|
||||
if machine_id:
|
||||
return store.get_machine_config(machine_id) or store.get_machine(machine_id)
|
||||
default_machine_id = str(task.get("default_machine_id") or "").strip()
|
||||
if default_machine_id:
|
||||
return store.get_machine_config(default_machine_id) or store.get_machine(default_machine_id)
|
||||
machines = [machine for machine in store.list_machines() if machine.get("enabled")]
|
||||
return machines[0] if machines else None
|
||||
if service_id:
|
||||
return store.get_service(service_id)
|
||||
default_service_id = str(task.get("default_service_id") or "").strip()
|
||||
if default_service_id:
|
||||
return store.get_service(default_service_id)
|
||||
services = [svc for svc in store.list_services("ssh_tasks") if svc.get("enabled")]
|
||||
return services[0] if services else None
|
||||
|
||||
|
||||
def _client_for_machine(store: SettingsStore, machine: dict[str, Any]):
|
||||
mode = str(machine.get("mode") or "local").lower()
|
||||
if mode == "local":
|
||||
return LocalCommandClient()
|
||||
def _service_row_to_record(service_row: dict[str, Any]) -> ServiceRecord:
|
||||
"""Build a ServiceRecord from a raw settings_store service row."""
|
||||
from media_library_viewer_api.services.settings_store import get_settings_store
|
||||
|
||||
host = str(machine.get("host") or "").strip()
|
||||
username = str(machine.get("username") or "").strip()
|
||||
if not host or not username:
|
||||
raise HTTPException(status_code=400, detail="SSH machine is missing host or username")
|
||||
|
||||
settings = get_settings()
|
||||
|
||||
private_key = str(machine.get("ssh_private_key") or "")
|
||||
passphrase = str(machine.get("ssh_private_key_passphrase") or "")
|
||||
ssh_key_id = str(machine.get("ssh_key_id") or "").strip()
|
||||
if ssh_key_id:
|
||||
ssh_key = store.get_ssh_key(ssh_key_id)
|
||||
if ssh_key:
|
||||
private_key = str(ssh_key.get("private_key") or private_key)
|
||||
passphrase = str(ssh_key.get("passphrase") or passphrase)
|
||||
|
||||
key_filename = ""
|
||||
if machine.get("key_directory") and machine.get("key_name"):
|
||||
key_filename = f"{machine.get('key_directory')}/{machine.get('key_name')}"
|
||||
|
||||
return RemoteSSHClient(
|
||||
host=host,
|
||||
username=username,
|
||||
port=int(machine.get("port") or 22),
|
||||
key_filename=key_filename or None,
|
||||
private_key=private_key or None,
|
||||
private_key_passphrase=passphrase or None,
|
||||
password=str(machine.get("password") or "") or None,
|
||||
known_hosts_path=str(settings.ssh_known_hosts_file),
|
||||
)
|
||||
return build_service_record(get_settings_store(), service_row)
|
||||
|
||||
|
||||
@router.get("")
|
||||
@@ -125,14 +92,14 @@ def list_task_runs(
|
||||
) -> dict[str, Any]:
|
||||
if not store.get_task(task_id):
|
||||
raise HTTPException(status_code=404, detail="Task not found")
|
||||
runs = store.list_task_runs(task_id, limit=limit)
|
||||
runs = store.list_service_task_runs(task_id=task_id, limit=limit)
|
||||
return {"items": runs, "total": len(runs)}
|
||||
|
||||
|
||||
@router.post("/run")
|
||||
def run_task(
|
||||
request: RunTaskRequest,
|
||||
machine_id: str | None = Query(default=None),
|
||||
service_id: str | None = Query(default=None),
|
||||
store: SettingsStore = Depends(get_settings_store),
|
||||
) -> dict[str, Any]:
|
||||
task = store.get_task(request.task_id)
|
||||
@@ -141,68 +108,22 @@ def run_task(
|
||||
if not task.get("enabled", True):
|
||||
raise HTTPException(status_code=400, detail="Task is disabled")
|
||||
|
||||
machine = _resolve_machine_for_task(store, task, machine_id)
|
||||
if not machine:
|
||||
raise HTTPException(status_code=400, detail="No machine is available for this action")
|
||||
service_row = _resolve_service_for_task(store, task, service_id)
|
||||
if not service_row:
|
||||
raise HTTPException(status_code=400, detail="No SSH task service is available for this action")
|
||||
if not service_row.get("enabled", True):
|
||||
raise HTTPException(status_code=400, detail="Selected SSH task service is disabled")
|
||||
|
||||
task_type = str(task.get("task_type") or "shell").lower()
|
||||
command = str(task.get("content") or "")
|
||||
if task_type == "python":
|
||||
command = f"python3 -c {shlex.quote(command)}"
|
||||
elif task_type != "shell":
|
||||
raise HTTPException(status_code=400, detail=f"Unknown task type: {task_type}")
|
||||
service = _service_row_to_record(service_row)
|
||||
result = run_saved_task(store, task, service)
|
||||
|
||||
client = _client_for_machine(store, machine)
|
||||
start = time.perf_counter()
|
||||
machine_name = _machine_label(machine)
|
||||
try:
|
||||
result = client.run(command, timeout=1200)
|
||||
stdout = result.stdout or ""
|
||||
stderr = result.stderr or ""
|
||||
status_text = "success" if result.exit_status == 0 else "error"
|
||||
store.record_task_run(
|
||||
task,
|
||||
status_text,
|
||||
machine_id=str(machine.get("id") or ""),
|
||||
machine_name=machine_name,
|
||||
task_type=task_type,
|
||||
duration_ms=int((time.perf_counter() - start) * 1000),
|
||||
stdout_tail=stdout[-4000:],
|
||||
stderr_tail=stderr[-4000:],
|
||||
error="" if result.exit_status == 0 else (stderr or stdout or "Task failed"),
|
||||
)
|
||||
return {
|
||||
"task_id": task["id"],
|
||||
"task_name": task["name"],
|
||||
"machine_id": str(machine.get("id") or ""),
|
||||
"machine_name": machine_name,
|
||||
"task_type": task_type,
|
||||
"exit_status": result.exit_status,
|
||||
"stdout": stdout,
|
||||
"stderr": stderr,
|
||||
}
|
||||
except Exception as exc:
|
||||
duration_ms = int((time.perf_counter() - start) * 1000)
|
||||
error_text = str(exc)
|
||||
store.record_task_run(
|
||||
task,
|
||||
"error",
|
||||
machine_id=str(machine.get("id") or ""),
|
||||
machine_name=machine_name,
|
||||
task_type=task_type,
|
||||
duration_ms=duration_ms,
|
||||
stdout_tail="",
|
||||
stderr_tail=error_text[-4000:],
|
||||
error=error_text,
|
||||
)
|
||||
logger.exception("Task execution failed task_id=%s", task["id"])
|
||||
return {
|
||||
"task_id": task["id"],
|
||||
"task_name": task["name"],
|
||||
"machine_id": str(machine.get("id") or ""),
|
||||
"machine_name": machine_name,
|
||||
"task_type": task_type,
|
||||
"exit_status": 1,
|
||||
"stdout": "",
|
||||
"stderr": error_text,
|
||||
}
|
||||
return {
|
||||
"task_id": task["id"],
|
||||
"task_name": task["name"],
|
||||
"service_id": service.id,
|
||||
"service_name": _service_label(service_row),
|
||||
"task_type": task.get("task_type", "shell"),
|
||||
"exit_status": result.exit_status,
|
||||
"stdout": result.stdout,
|
||||
"stderr": result.stderr,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
# backend/src/media_library_viewer_api/services (index)
|
||||
dir: backend/src/media_library_viewer_api/services
|
||||
|
||||
## role
|
||||
Backend service layer providing business logic for media library management, backup monitoring, email notifications, SSH task execution, encryption, and persistent settings storage.
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- backup_alert_engine.py
|
||||
- backup_poller.py
|
||||
- db_maintenance.py
|
||||
- known_hosts.py
|
||||
- mail_queue.py
|
||||
- mailer.py
|
||||
- mailer_impl.py
|
||||
- media_index.py
|
||||
- media_index_impl.py
|
||||
- secrets.py
|
||||
- settings_store.py
|
||||
- targets.py
|
||||
- task_runner.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/services/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/services/.pi-map.md
|
||||
## workflows
|
||||
- change services behavior
|
||||
read: __init__.py, backup_alert_engine.py, backup_poller.py
|
||||
## dirty
|
||||
-
|
||||
File diff suppressed because one or more lines are too long
@@ -17,16 +17,14 @@ from typing import Any
|
||||
|
||||
import paramiko
|
||||
|
||||
from media_library_viewer_api.config import get_settings
|
||||
from media_library_viewer_api.models.widgets import _validate_config_keys
|
||||
|
||||
DEFAULT_SETTINGS_PATH = Path(".cache/media_library_viewer/settings.sqlite")
|
||||
LOCAL_MACHINE_ID = "local"
|
||||
DEFAULT_SERVICES = ["monitoring", "files", "jellyfin"]
|
||||
DEFAULT_SERVICES = ["monitoring", "files"]
|
||||
|
||||
|
||||
def _default_local_machine() -> dict[str, Any]:
|
||||
settings = get_settings()
|
||||
return {
|
||||
"id": LOCAL_MACHINE_ID,
|
||||
"name": "This machine",
|
||||
@@ -42,13 +40,6 @@ def _default_local_machine() -> dict[str, Any]:
|
||||
"ssh_private_key": "",
|
||||
"ssh_private_key_passphrase": "",
|
||||
"password": "",
|
||||
"media_root": settings.media_root,
|
||||
"path_prefix": settings.path_prefix,
|
||||
"jellyfin_url": "",
|
||||
"jellyfin_user_id": "",
|
||||
"jellyfin_api_key": "",
|
||||
"jellyseerr_url": "",
|
||||
"jellyseerr_api_key": "",
|
||||
"node_exporter_enabled": False,
|
||||
"node_exporter_port": 9100,
|
||||
"node_exporter_scrape_host": "",
|
||||
@@ -118,7 +109,7 @@ class SettingsStore:
|
||||
task_type TEXT NOT NULL,
|
||||
content TEXT NOT NULL,
|
||||
enabled INTEGER NOT NULL,
|
||||
default_machine_id TEXT NOT NULL,
|
||||
default_service_id TEXT NOT NULL,
|
||||
notes TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
updated_at INTEGER NOT NULL
|
||||
@@ -126,28 +117,14 @@ class SettingsStore:
|
||||
"""
|
||||
)
|
||||
conn.execute("CREATE INDEX IF NOT EXISTS idx_saved_tasks_name ON saved_tasks(name)")
|
||||
conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS saved_task_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
task_id TEXT NOT NULL,
|
||||
task_name TEXT NOT NULL,
|
||||
machine_id TEXT NOT NULL,
|
||||
machine_name TEXT NOT NULL,
|
||||
task_type TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
duration_ms INTEGER NOT NULL,
|
||||
request_id TEXT NOT NULL,
|
||||
stdout_tail TEXT NOT NULL,
|
||||
stderr_tail TEXT NOT NULL,
|
||||
error TEXT NOT NULL
|
||||
)
|
||||
"""
|
||||
)
|
||||
conn.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_saved_task_runs_task_time ON saved_task_runs(task_id, created_at DESC)"
|
||||
)
|
||||
# saved_tasks.default_machine_id → default_service_id (saved tasks now
|
||||
# target ssh_tasks service instances). Migrate existing columns.
|
||||
saved_tasks_cols = {row[1] for row in conn.execute("PRAGMA table_info(saved_tasks)").fetchall()}
|
||||
if "default_service_id" not in saved_tasks_cols and "default_machine_id" in saved_tasks_cols:
|
||||
conn.execute("ALTER TABLE saved_tasks RENAME COLUMN default_machine_id TO default_service_id")
|
||||
# Run history for saved tasks now lives in service_task_runs; the
|
||||
# legacy machine-based table is dropped.
|
||||
conn.execute("DROP TABLE IF EXISTS saved_task_runs")
|
||||
conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS dashboard_shortcuts (
|
||||
@@ -304,13 +281,6 @@ class SettingsStore:
|
||||
"ssh_private_key_set": bool(data.get("ssh_private_key")),
|
||||
"ssh_private_key_passphrase_set": bool(data.get("ssh_private_key_passphrase")),
|
||||
"password_set": bool(data.get("password")),
|
||||
"media_root": data.get("media_root", ""),
|
||||
"path_prefix": data.get("path_prefix", ""),
|
||||
"jellyfin_url": data.get("jellyfin_url", ""),
|
||||
"jellyfin_user_id": data.get("jellyfin_user_id", ""),
|
||||
"jellyfin_api_key_set": bool(data.get("jellyfin_api_key")),
|
||||
"jellyseerr_url": data.get("jellyseerr_url", ""),
|
||||
"jellyseerr_api_key_set": bool(data.get("jellyseerr_api_key")),
|
||||
"node_exporter_enabled": bool(data.get("node_exporter_enabled", False)),
|
||||
"node_exporter_port": int(data.get("node_exporter_port", 9100) or 9100),
|
||||
"node_exporter_scrape_host": data.get("node_exporter_scrape_host", ""),
|
||||
@@ -358,19 +328,6 @@ class SettingsStore:
|
||||
if password in (None, ""):
|
||||
password = (current or {}).get("password", "")
|
||||
password = str(password or "")
|
||||
media_root = _current_str("media_root")
|
||||
path_prefix = _current_str("path_prefix")
|
||||
jellyfin_url = _current_str("jellyfin_url")
|
||||
jellyfin_user_id = _current_str("jellyfin_user_id")
|
||||
jellyfin_api_key = payload.get("jellyfin_api_key")
|
||||
if jellyfin_api_key in (None, ""):
|
||||
jellyfin_api_key = (current or {}).get("jellyfin_api_key", "")
|
||||
jellyfin_api_key = str(jellyfin_api_key or "")
|
||||
jellyseerr_url = _current_str("jellyseerr_url")
|
||||
jellyseerr_api_key = payload.get("jellyseerr_api_key")
|
||||
if jellyseerr_api_key in (None, ""):
|
||||
jellyseerr_api_key = (current or {}).get("jellyseerr_api_key", "")
|
||||
jellyseerr_api_key = str(jellyseerr_api_key or "")
|
||||
node_exporter_enabled = bool(
|
||||
payload.get("node_exporter_enabled")
|
||||
if payload.get("node_exporter_enabled") is not None
|
||||
@@ -400,13 +357,6 @@ class SettingsStore:
|
||||
"ssh_private_key": ssh_private_key,
|
||||
"ssh_private_key_passphrase": ssh_private_key_passphrase,
|
||||
"password": password,
|
||||
"media_root": media_root,
|
||||
"path_prefix": path_prefix,
|
||||
"jellyfin_url": jellyfin_url,
|
||||
"jellyfin_user_id": jellyfin_user_id,
|
||||
"jellyfin_api_key": jellyfin_api_key,
|
||||
"jellyseerr_url": jellyseerr_url,
|
||||
"jellyseerr_api_key": jellyseerr_api_key,
|
||||
"node_exporter_enabled": node_exporter_enabled,
|
||||
"node_exporter_port": node_exporter_port,
|
||||
"node_exporter_scrape_host": node_exporter_scrape_host,
|
||||
@@ -428,13 +378,6 @@ class SettingsStore:
|
||||
"ssh_private_key": "",
|
||||
"ssh_private_key_passphrase": "",
|
||||
"password": "",
|
||||
"media_root": machine["media_root"],
|
||||
"path_prefix": machine["path_prefix"],
|
||||
"jellyfin_url": machine["jellyfin_url"],
|
||||
"jellyfin_user_id": machine["jellyfin_user_id"],
|
||||
"jellyfin_api_key": machine["jellyfin_api_key"],
|
||||
"jellyseerr_url": machine["jellyseerr_url"],
|
||||
"jellyseerr_api_key": machine["jellyseerr_api_key"],
|
||||
"node_exporter_enabled": machine["node_exporter_enabled"],
|
||||
"node_exporter_port": machine["node_exporter_port"],
|
||||
"node_exporter_scrape_host": machine["node_exporter_scrape_host"],
|
||||
@@ -518,13 +461,6 @@ class SettingsStore:
|
||||
"ssh_private_key": data.get("ssh_private_key", ""),
|
||||
"ssh_private_key_passphrase": data.get("ssh_private_key_passphrase", ""),
|
||||
"password": data.get("password", ""),
|
||||
"media_root": data.get("media_root", ""),
|
||||
"path_prefix": data.get("path_prefix", ""),
|
||||
"jellyfin_url": data.get("jellyfin_url", ""),
|
||||
"jellyfin_user_id": data.get("jellyfin_user_id", ""),
|
||||
"jellyfin_api_key": data.get("jellyfin_api_key", ""),
|
||||
"jellyseerr_url": data.get("jellyseerr_url", ""),
|
||||
"jellyseerr_api_key": data.get("jellyseerr_api_key", ""),
|
||||
"node_exporter_enabled": bool(data.get("node_exporter_enabled", False)),
|
||||
"node_exporter_port": int(data.get("node_exporter_port", 9100) or 9100),
|
||||
"node_exporter_scrape_host": data.get("node_exporter_scrape_host", ""),
|
||||
@@ -562,13 +498,6 @@ class SettingsStore:
|
||||
"ssh_private_key": machine["ssh_private_key"],
|
||||
"ssh_private_key_passphrase": machine["ssh_private_key_passphrase"],
|
||||
"password": machine["password"],
|
||||
"media_root": machine["media_root"],
|
||||
"path_prefix": machine["path_prefix"],
|
||||
"jellyfin_url": machine["jellyfin_url"],
|
||||
"jellyfin_user_id": machine["jellyfin_user_id"],
|
||||
"jellyfin_api_key": machine["jellyfin_api_key"],
|
||||
"jellyseerr_url": machine["jellyseerr_url"],
|
||||
"jellyseerr_api_key": machine["jellyseerr_api_key"],
|
||||
"node_exporter_enabled": machine["node_exporter_enabled"],
|
||||
"node_exporter_port": machine["node_exporter_port"],
|
||||
"node_exporter_scrape_host": machine["node_exporter_scrape_host"],
|
||||
@@ -755,7 +684,7 @@ class SettingsStore:
|
||||
"task_type": row["task_type"],
|
||||
"content": row["content"],
|
||||
"enabled": bool(row["enabled"]),
|
||||
"default_machine_id": row["default_machine_id"],
|
||||
"default_service_id": row["default_service_id"],
|
||||
"notes": row["notes"],
|
||||
"created_at": row["created_at"],
|
||||
"updated_at": row["updated_at"],
|
||||
@@ -772,10 +701,10 @@ class SettingsStore:
|
||||
payload.get("content") if payload.get("content") is not None else (current or {}).get("content", "") or ""
|
||||
)
|
||||
enabled = bool(payload.get("enabled", (current or {}).get("enabled", True)))
|
||||
default_machine_id = str(
|
||||
payload.get("default_machine_id")
|
||||
if payload.get("default_machine_id") is not None
|
||||
else (current or {}).get("default_machine_id", "") or ""
|
||||
default_service_id = str(
|
||||
payload.get("default_service_id")
|
||||
if payload.get("default_service_id") is not None
|
||||
else (current or {}).get("default_service_id", "") or ""
|
||||
).strip()
|
||||
notes = str(
|
||||
payload.get("notes") if payload.get("notes") is not None else (current or {}).get("notes", "") or ""
|
||||
@@ -786,7 +715,7 @@ class SettingsStore:
|
||||
"task_type": task_type,
|
||||
"content": content,
|
||||
"enabled": enabled,
|
||||
"default_machine_id": default_machine_id,
|
||||
"default_service_id": default_service_id,
|
||||
"notes": notes,
|
||||
}
|
||||
|
||||
@@ -814,7 +743,7 @@ class SettingsStore:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO saved_tasks (
|
||||
id, name, task_type, content, enabled, default_machine_id,
|
||||
id, name, task_type, content, enabled, default_service_id,
|
||||
notes, created_at, updated_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
@@ -823,7 +752,7 @@ class SettingsStore:
|
||||
task_type = excluded.task_type,
|
||||
content = excluded.content,
|
||||
enabled = excluded.enabled,
|
||||
default_machine_id = excluded.default_machine_id,
|
||||
default_service_id = excluded.default_service_id,
|
||||
notes = excluded.notes,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
@@ -833,7 +762,7 @@ class SettingsStore:
|
||||
task["task_type"],
|
||||
task["content"],
|
||||
1 if task["enabled"] else 0,
|
||||
task["default_machine_id"],
|
||||
task["default_service_id"],
|
||||
task["notes"],
|
||||
created_at,
|
||||
now,
|
||||
@@ -846,58 +775,6 @@ class SettingsStore:
|
||||
with self.connect() as conn:
|
||||
conn.execute("DELETE FROM saved_tasks WHERE id = ?", (task_id,))
|
||||
|
||||
def list_task_runs(self, task_id: str, *, limit: int = 10) -> list[dict[str, Any]]:
|
||||
self.init_schema()
|
||||
with self.connect() as conn:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM saved_task_runs WHERE task_id = ? ORDER BY created_at DESC LIMIT ?",
|
||||
(task_id, max(1, min(int(limit), 50))),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def record_task_run(
|
||||
self,
|
||||
task: dict[str, Any],
|
||||
status: str,
|
||||
*,
|
||||
machine_id: str,
|
||||
machine_name: str,
|
||||
task_type: str,
|
||||
duration_ms: int,
|
||||
request_id: str = "",
|
||||
stdout_tail: str = "",
|
||||
stderr_tail: str = "",
|
||||
error: str = "",
|
||||
) -> None:
|
||||
self.init_schema()
|
||||
now = int(time.time())
|
||||
with self.connect() as conn:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO saved_task_runs (
|
||||
id, task_id, task_name, machine_id, machine_name, task_type,
|
||||
status, created_at, duration_ms, request_id, stdout_tail,
|
||||
stderr_tail, error
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
uuid.uuid4().hex,
|
||||
str(task.get("id") or ""),
|
||||
str(task.get("name") or ""),
|
||||
machine_id,
|
||||
machine_name,
|
||||
task_type,
|
||||
status,
|
||||
now,
|
||||
duration_ms,
|
||||
request_id,
|
||||
stdout_tail,
|
||||
stderr_tail,
|
||||
error,
|
||||
),
|
||||
)
|
||||
|
||||
def _row_to_shortcut(self, row: sqlite3.Row) -> dict[str, Any]:
|
||||
target = json.loads(row["target_json"] or "{}")
|
||||
return {
|
||||
|
||||
@@ -1,19 +1,16 @@
|
||||
"""Prometheus file-based service discovery target management.
|
||||
"""Prometheus Node Exporter target discovery.
|
||||
|
||||
The backend owns the list of remote Node Exporter targets so that operators can
|
||||
enable scraping per machine from the Manage UI. Prometheus reads the generated
|
||||
JSON file via `file_sd_configs`; this keeps Prometheus config static and pushes
|
||||
machine-specific changes into a file it can reload.
|
||||
enable scraping per machine from the Manage UI. The list is exposed over HTTP at
|
||||
``GET /api/monitoring/prometheus-targets`` and consumed by an external Prometheus
|
||||
via ``http_sd_configs`` (no shared volume required).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from media_library_viewer_api.config import get_settings
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -34,10 +31,9 @@ def _scrape_address(machine: dict[str, Any]) -> str | None:
|
||||
|
||||
|
||||
def build_node_exporter_targets(store: SettingsStore) -> list[dict[str, Any]]:
|
||||
"""Build a file-SD target list for all enabled SSH machines.
|
||||
"""Build an http-SD target list for all enabled SSH machines.
|
||||
|
||||
Local machines are excluded because the Compose-managed node-exporter
|
||||
service already covers the Docker host.
|
||||
Local machines are excluded because the Docker host is scraped directly.
|
||||
"""
|
||||
targets: list[dict[str, Any]] = []
|
||||
for machine in store.list_machines():
|
||||
@@ -60,18 +56,3 @@ def build_node_exporter_targets(store: SettingsStore) -> list[dict[str, Any]]:
|
||||
}
|
||||
)
|
||||
return targets
|
||||
|
||||
|
||||
def write_prometheus_targets(store: SettingsStore, file_sd_dir: Path | None = None) -> Path:
|
||||
"""Render and persist Prometheus file-SD targets.
|
||||
|
||||
Returns the path written so callers can log or expose it.
|
||||
"""
|
||||
settings = get_settings()
|
||||
file_sd_dir = file_sd_dir or Path(settings.prometheus_file_sd_dir)
|
||||
file_sd_dir.mkdir(parents=True, exist_ok=True)
|
||||
file_path = file_sd_dir / "node_exporter_targets.json"
|
||||
targets = build_node_exporter_targets(store)
|
||||
file_path.write_text(json.dumps(targets, indent=2), encoding="utf-8")
|
||||
logger.info("Wrote %s node_exporter targets to %s", len(targets), file_path)
|
||||
return file_path
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
"""Shared runner for saved tasks over SSH task services.
|
||||
|
||||
Both the Actions page (``routers/tasks.py``) and the SSH task widget
|
||||
(``widgets/sources.py``) run saved tasks against ``ssh_tasks`` service instances.
|
||||
This module is the single execution path: build the client from the service
|
||||
record, render the command, run it with the service timeout, append a
|
||||
``service_task_runs`` row, and return the result.
|
||||
|
||||
There is intentionally no local execution mode — tasks are SSH-only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import shlex
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from media_library_viewer_api.clients.ssh import RemoteSSHClient
|
||||
from media_library_viewer_api.config import get_settings
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from media_library_viewer_api.widgets.sources import ServiceRecord
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class TaskRunResult:
|
||||
"""Outcome of a single saved-task run."""
|
||||
|
||||
exit_status: int
|
||||
stdout: str
|
||||
stderr: str
|
||||
duration_ms: int
|
||||
status: str # "success" | "failure" | "error"
|
||||
error: str = ""
|
||||
|
||||
|
||||
def build_ssh_client(store: SettingsStore, service: "ServiceRecord") -> RemoteSSHClient:
|
||||
"""Build an SSH client from an ssh_tasks service instance + referenced key."""
|
||||
config = service.config
|
||||
host = str(config.get("host") or "").strip()
|
||||
username = str(config.get("username") or "").strip()
|
||||
if not host or not username:
|
||||
raise ValueError("SSH task service is missing host or username")
|
||||
|
||||
settings = get_settings()
|
||||
private_key = ""
|
||||
key_passphrase = ""
|
||||
ssh_key_id = str(config.get("ssh_key_id") or "").strip()
|
||||
if ssh_key_id:
|
||||
ssh_key = store.get_ssh_key(ssh_key_id)
|
||||
if ssh_key:
|
||||
private_key = str(ssh_key.get("private_key") or "")
|
||||
key_passphrase = str(ssh_key.get("passphrase") or "")
|
||||
# Service-level passphrase secret takes precedence.
|
||||
key_passphrase = str(service.secrets.get("passphrase") or "") or key_passphrase
|
||||
|
||||
return RemoteSSHClient(
|
||||
host=host,
|
||||
username=username,
|
||||
port=int(config.get("port") or 22),
|
||||
private_key=private_key or None,
|
||||
private_key_passphrase=key_passphrase or None,
|
||||
known_hosts_path=str(settings.ssh_known_hosts_file),
|
||||
timeout=int(config.get("timeout_seconds") or 30),
|
||||
)
|
||||
|
||||
|
||||
def _render_command(task: dict[str, Any]) -> str:
|
||||
"""Render a saved task into a shell command (shell or python3 -c)."""
|
||||
task_type = str(task.get("task_type") or "shell").lower()
|
||||
command = str(task.get("content") or "")
|
||||
if task_type == "python":
|
||||
return f"python3 -c {shlex.quote(command)}"
|
||||
if task_type == "shell":
|
||||
return command
|
||||
raise ValueError(f"Unknown task type: {task_type}")
|
||||
|
||||
|
||||
def run_saved_task(
|
||||
store: SettingsStore,
|
||||
task: dict[str, Any],
|
||||
service: "ServiceRecord",
|
||||
*,
|
||||
timeout: int | None = None,
|
||||
) -> TaskRunResult:
|
||||
"""Run a saved task on an ssh_tasks service instance and log the run.
|
||||
|
||||
The ``timeout`` defaults to the service's ``timeout_seconds`` config. The run
|
||||
is recorded in ``service_task_runs`` regardless of outcome (success, failure,
|
||||
error). Raises ``ValueError`` for an unsupported task type or an incomplete
|
||||
service config (propagated from ``build_ssh_client`` / ``_render_command``).
|
||||
"""
|
||||
timeout = int(timeout if timeout is not None else service.config.get("timeout_seconds") or 30)
|
||||
client = build_ssh_client(store, service)
|
||||
command = _render_command(task)
|
||||
|
||||
start = time.perf_counter()
|
||||
try:
|
||||
result = client.run(command, timeout=timeout)
|
||||
except Exception as exc:
|
||||
duration_ms = int((time.perf_counter() - start) * 1000)
|
||||
_record(store, task, service, "error", duration_ms=duration_ms, error=str(exc)[:1000])
|
||||
logger.exception("saved task run failed task_id=%s", task.get("id"))
|
||||
return TaskRunResult(
|
||||
exit_status=1,
|
||||
stdout="",
|
||||
stderr=str(exc),
|
||||
duration_ms=duration_ms,
|
||||
status="error",
|
||||
error=str(exc),
|
||||
)
|
||||
|
||||
duration_ms = int((time.perf_counter() - start) * 1000)
|
||||
stdout = result.stdout or ""
|
||||
stderr = result.stderr or ""
|
||||
status = "success" if result.exit_status == 0 else "failure"
|
||||
_record(
|
||||
store,
|
||||
task,
|
||||
service,
|
||||
status,
|
||||
exit_status=result.exit_status,
|
||||
duration_ms=duration_ms,
|
||||
stdout_tail=stdout,
|
||||
stderr_tail=stderr,
|
||||
error="" if result.exit_status == 0 else (stderr or stdout or "Task failed"),
|
||||
)
|
||||
return TaskRunResult(
|
||||
exit_status=result.exit_status,
|
||||
stdout=stdout,
|
||||
stderr=stderr,
|
||||
duration_ms=duration_ms,
|
||||
status=status,
|
||||
)
|
||||
|
||||
|
||||
def _record(
|
||||
store: SettingsStore,
|
||||
task: dict[str, Any],
|
||||
service: "ServiceRecord",
|
||||
status: str,
|
||||
*,
|
||||
exit_status: int | None = None,
|
||||
duration_ms: int = 0,
|
||||
stdout_tail: str = "",
|
||||
stderr_tail: str = "",
|
||||
error: str = "",
|
||||
) -> None:
|
||||
"""Append a service_task_runs row for a saved-task run."""
|
||||
store.record_service_task_run(
|
||||
{
|
||||
"task_id": str(task.get("id") or ""),
|
||||
"service_id": service.id,
|
||||
"status": status,
|
||||
"exit_status": exit_status,
|
||||
"duration_ms": duration_ms,
|
||||
"stdout_tail": stdout_tail,
|
||||
"stderr_tail": stderr_tail,
|
||||
"error": error,
|
||||
}
|
||||
)
|
||||
@@ -0,0 +1,22 @@
|
||||
# backend/src/media_library_viewer_api/widgets (index)
|
||||
dir: backend/src/media_library_viewer_api/widgets
|
||||
|
||||
## role
|
||||
Provides widget definitions, schemas, and data source adapters for rendering configurable dashboard widgets from both built-in and external service data.
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- builtin.py
|
||||
- sources.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/widgets/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/widgets/.pi-map.md
|
||||
## workflows
|
||||
- change widgets behavior
|
||||
read: __init__.py, builtin.py, sources.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,29 @@
|
||||
# backend/src/media_library_viewer_api/widgets
|
||||
dir: backend/src/media_library_viewer_api/widgets
|
||||
|
||||
index: backend/src/media_library_viewer_api/widgets/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Provides widget definitions, schemas, and data source adapters for rendering configurable dashboard widgets from both built-in and external service data.
|
||||
## files
|
||||
- __init__.py | Marks the directory as a Python package for the widget subsystem.
|
||||
- builtin.py | Defines built-in widget kinds that don't require external services, providing their configurations, schemas, and validation. | exp: class:StaticConfig, func:get_builtin_widget_kind(kind: str) → WidgetKind | None, call:BUILTIN_WIDGET_KINDS.get, func:is_builtin_kind(kind: str) → bool, func:builtin_widget_kind_models() → dict[str, type], call:Field, func:validate_builtin_config(kind: str, config: dict[str, Any]) → dict[str, Any], call:builtin_widget_kind_models, call:models.get, call:dict, call:model_cls.model_validate(config or {}).model_dump | dep: typing, media_library_viewer_api.integrations.base, pydantic
|
||||
- sources.py | Defines widget source adapters that fetch and transform data from various external services (Grafana, Prometheus, Jellyfin, etc.) into dashboard widget payloads. | exp: class:ServiceRecord, class:WidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], class:BackupsWidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], call:get_settings_store, call:build_backup_dashboard_summary, call:summary.model_dump, call:logger.exception, class:StaticWidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], call:config.get, class:GrafanaWidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], call:str(service.config.get("base_url") or "").rstrip, call:service.config.get, call:config.get, call:logger.exception, class:PrometheusWidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], call:str(service.config.get("base_url") or "").rstrip, call:service.config.get, call:int, call:config.get, call:asyncio.wait_for, call:asyncio.to_thread, call:response.raise_for_status, call:response.json, call:payload.get, call:logger.exception, class:AlertmanagerWidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], call:str(service.config.get("base_url") or "").rstrip, call:service.config.get, call:int, call:config.get, call:service.secrets.get, call:asyncio.wait_for, call:asyncio.to_thread, call:response.raise_for_status, call:response.json, call:payload.get, call:isinstance, call:summarize_alerts, call:logger.exception, class:JellyfinWidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], call:str, call:service.config.get, call:service.secrets.get, call:int, call:asyncio.wait_for, call:asyncio.to_thread, call:_map_sessions_to_activity_rows, call:logger.exception, class:SshTaskWidgetSource, method:fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) → dict[str, Any], call:get_settings_store, call:config.get, call:store.get_task, call:task.get, call:int, call:service.config.get, call:asyncio.wait_for, call:asyncio.to_thread, call:_record_timeout, call:logger.exception, func:build_service_record(store: SettingsStore, service_row: dict[str, Any]) → ServiceRecord, call:ServiceRecord, call:service_row.get, call:decrypt_secrets, call:bool, func:_record_timeout(service: ServiceRecord | None, config: dict[str, Any], timeout: int) → None, call:get_settings_store, call:store.record_service_task_run, call:str, call:config.get, call:logger.exception, func:get_service_adapter(service_type: str) → WidgetSource | None, call:SERVICE_ADAPTERS.get, func:get_builtin_adapter(kind: str) → WidgetSource | None, call:BUILTIN_ADAPTERS.get | dep: asyncio, logging, dataclasses, typing, requests, media_library_viewer_api.clients.jellyfin, media_library_viewer_api.domain.dashboard, media_library_viewer_api.integrations.alertmanager, media_library_viewer_api.services.settings_store, media_library_viewer_api.services.task_runner, media_library_viewer_api.services.secrets
|
||||
## arch
|
||||
Adapter pattern with modular source integrations that normalize heterogeneous external API responses into a unified widget payload format, complemented by schema-validated built-in widget configurations.
|
||||
## tags
|
||||
widget, source, fetch, call:logger.exception, builtin, call:config.get, record, call:str
|
||||
## symbols
|
||||
- StaticConfig
|
||||
- ServiceRecord
|
||||
- WidgetSource
|
||||
- BackupsWidgetSource
|
||||
- StaticWidgetSource
|
||||
- GrafanaWidgetSource
|
||||
- PrometheusWidgetSource
|
||||
- AlertmanagerWidgetSource
|
||||
## workflows
|
||||
- change widgets behavior
|
||||
read: __init__.py, builtin.py, sources.py
|
||||
## dirty
|
||||
-
|
||||
@@ -12,21 +12,19 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import shlex
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Protocol
|
||||
|
||||
import requests
|
||||
|
||||
from media_library_viewer_api.clients.jellyfin import JellyfinClient
|
||||
from media_library_viewer_api.clients.ssh import RemoteSSHClient
|
||||
from media_library_viewer_api.config import get_settings
|
||||
from media_library_viewer_api.domain.dashboard import (
|
||||
_map_sessions_to_activity_rows,
|
||||
build_backup_dashboard_summary,
|
||||
)
|
||||
from media_library_viewer_api.integrations.alertmanager import summarize_alerts
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore, get_settings_store
|
||||
from media_library_viewer_api.services.task_runner import run_saved_task
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -154,6 +152,43 @@ class PrometheusWidgetSource:
|
||||
return {"error": f"Prometheus query failed: {exc}"}
|
||||
|
||||
|
||||
class AlertmanagerWidgetSource:
|
||||
"""Fetch firing alerts from an Alertmanager service and summarize them."""
|
||||
|
||||
async def fetch(self, service: ServiceRecord | None, widget_kind: str, config: dict[str, Any]) -> dict[str, Any]:
|
||||
try:
|
||||
if service is None:
|
||||
return {"error": "Alertmanager widget is missing its service"}
|
||||
base_url = str(service.config.get("base_url") or "").rstrip("/")
|
||||
timeout = int(service.config.get("timeout_seconds") or 5)
|
||||
severity_filter = config.get("severity_filter") or None
|
||||
headers: dict[str, str] = {}
|
||||
api_key = str(service.secrets.get("api_key") or "")
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
response = await asyncio.wait_for(
|
||||
asyncio.to_thread(
|
||||
requests.get,
|
||||
f"{base_url}/api/v1/alerts",
|
||||
headers=headers,
|
||||
timeout=timeout,
|
||||
),
|
||||
timeout=timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
payload = response.json()
|
||||
alerts = payload.get("data", []) if isinstance(payload, dict) else []
|
||||
return summarize_alerts(alerts, severity_filter=severity_filter)
|
||||
except asyncio.TimeoutError:
|
||||
return {"error": "Widget data fetch timed out"}
|
||||
except requests.RequestException as exc:
|
||||
logger.exception("alertmanager adapter failed")
|
||||
return {"error": f"Alertmanager query failed: {exc}"}
|
||||
except Exception as exc:
|
||||
logger.exception("alertmanager adapter failed")
|
||||
return {"error": f"Alertmanager query failed: {exc}"}
|
||||
|
||||
|
||||
class JellyfinWidgetSource:
|
||||
"""Fetch Jellyfin sessions and map them to activity rows."""
|
||||
|
||||
@@ -200,51 +235,17 @@ class SshTaskWidgetSource:
|
||||
if not task.get("enabled", True):
|
||||
return {"error": "Task is disabled"}
|
||||
|
||||
client = _build_ssh_client(store, service)
|
||||
timeout = int(service.config.get("timeout_seconds") or 30)
|
||||
task_type = str(task.get("task_type") or "shell").lower()
|
||||
command = str(task.get("content") or "")
|
||||
if task_type == "python":
|
||||
command = f"python3 -c {shlex.quote(command)}"
|
||||
elif task_type != "shell":
|
||||
return {"error": f"Unknown task type: {task_type}"}
|
||||
|
||||
start = time.perf_counter()
|
||||
result = await asyncio.wait_for(
|
||||
asyncio.to_thread(client.run, command, timeout),
|
||||
asyncio.to_thread(run_saved_task, store, task, service),
|
||||
timeout=timeout,
|
||||
)
|
||||
duration_ms = int((time.perf_counter() - start) * 1000)
|
||||
stdout = result.stdout or ""
|
||||
stderr = result.stderr or ""
|
||||
store.record_service_task_run(
|
||||
{
|
||||
"task_id": task_id,
|
||||
"service_id": service.id,
|
||||
"status": "success" if result.exit_status == 0 else "failure",
|
||||
"exit_status": result.exit_status,
|
||||
"duration_ms": duration_ms,
|
||||
"stdout_tail": stdout,
|
||||
"stderr_tail": stderr,
|
||||
"error": "" if result.exit_status == 0 else (stderr or stdout or "Task failed"),
|
||||
}
|
||||
)
|
||||
return {"exit_status": result.exit_status, "stdout": stdout, "stderr": stderr}
|
||||
return {"exit_status": result.exit_status, "stdout": result.stdout, "stderr": result.stderr}
|
||||
except asyncio.TimeoutError:
|
||||
_record_timeout(service, config, timeout)
|
||||
return {"error": "Widget data fetch timed out"}
|
||||
except Exception as exc:
|
||||
logger.exception("ssh_task adapter failed")
|
||||
store = get_settings_store()
|
||||
store.record_service_task_run(
|
||||
{
|
||||
"task_id": str(config.get("task_id") or ""),
|
||||
"service_id": service.id if service else "",
|
||||
"status": "error",
|
||||
"duration_ms": 0,
|
||||
"error": str(exc)[:1000],
|
||||
}
|
||||
)
|
||||
return {"error": f"SSH task failed: {exc}"}
|
||||
|
||||
|
||||
@@ -264,37 +265,6 @@ def _record_timeout(service: ServiceRecord | None, config: dict[str, Any], timeo
|
||||
logger.exception("failed to record ssh task timeout")
|
||||
|
||||
|
||||
def _build_ssh_client(store: SettingsStore, service: ServiceRecord) -> RemoteSSHClient:
|
||||
"""Build an SSH client from an ssh_tasks service instance + referenced key."""
|
||||
config = service.config
|
||||
host = str(config.get("host") or "").strip()
|
||||
username = str(config.get("username") or "").strip()
|
||||
if not host or not username:
|
||||
raise ValueError("SSH task service is missing host or username")
|
||||
|
||||
settings = get_settings()
|
||||
private_key = ""
|
||||
key_passphrase = ""
|
||||
ssh_key_id = str(config.get("ssh_key_id") or "").strip()
|
||||
if ssh_key_id:
|
||||
ssh_key = store.get_ssh_key(ssh_key_id)
|
||||
if ssh_key:
|
||||
private_key = str(ssh_key.get("private_key") or "")
|
||||
key_passphrase = str(ssh_key.get("passphrase") or "")
|
||||
# Service-level passphrase secret takes precedence.
|
||||
key_passphrase = str(service.secrets.get("passphrase") or "") or key_passphrase
|
||||
|
||||
return RemoteSSHClient(
|
||||
host=host,
|
||||
username=username,
|
||||
port=int(config.get("port") or 22),
|
||||
private_key=private_key or None,
|
||||
private_key_passphrase=key_passphrase or None,
|
||||
known_hosts_path=str(settings.ssh_known_hosts_file),
|
||||
timeout=int(config.get("timeout_seconds") or 30),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Registries
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -302,6 +272,7 @@ def _build_ssh_client(store: SettingsStore, service: ServiceRecord) -> RemoteSSH
|
||||
SERVICE_ADAPTERS: dict[str, WidgetSource] = {
|
||||
"grafana": GrafanaWidgetSource(),
|
||||
"prometheus": PrometheusWidgetSource(),
|
||||
"alertmanager": AlertmanagerWidgetSource(),
|
||||
"jellyfin": JellyfinWidgetSource(),
|
||||
"ssh_tasks": SshTaskWidgetSource(),
|
||||
}
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# backend/src/media_library_viewer_api/workers (index)
|
||||
dir: backend/src/media_library_viewer_api/workers
|
||||
|
||||
## role
|
||||
Background worker package providing subprocess-based media index building from Jellyfin libraries with progress tracking and cancellation support.
|
||||
## parent
|
||||
index: backend/src/media_library_viewer_api/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- media_index_worker.py
|
||||
## links
|
||||
index: backend/src/media_library_viewer_api/workers/.pi-map.index.md
|
||||
map: backend/src/media_library_viewer_api/workers/.pi-map.md
|
||||
## workflows
|
||||
- change workers behavior
|
||||
read: __init__.py, media_index_worker.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,28 @@
|
||||
# backend/src/media_library_viewer_api/workers
|
||||
dir: backend/src/media_library_viewer_api/workers
|
||||
|
||||
index: backend/src/media_library_viewer_api/workers/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Background worker package providing subprocess-based media index building from Jellyfin libraries with progress tracking and cancellation support.
|
||||
## files
|
||||
- __init__.py | Marks the directory as a Python package for worker entrypoints used in background task processing.
|
||||
- media_index_worker.py | Subprocess worker that builds a media index from Jellyfin libraries with progress tracking and cooperative cancellation support. | exp: func:_set_build_metadata(index: MediaIndex, state: dict[str, Any]) → None, call:state.items, call:index.set_metadata, func:_cancel_requested(index: MediaIndex) → bool, call:index.status, func:_start_state(index: MediaIndex, pid: int, library_count: int) → None, call:_set_build_metadata, func:_progress_callback(index: MediaIndex, pid: int, state: dict[str, Any]) → None, call:_set_build_metadata, call:state.get, func:run_build(final_index_path: str | Path, staging_index_path: str | Path) → int, call:get_settings, call:configure_logging, call:logger.info, call:describe_settings, call:get_jellyfin_client, call:get_user_id, call:client.libraries, call:MediaIndex, call:os.getpid, call:time.perf_counter, call:Path, call:staging_path.unlink, call:len, call:_start_state, call:build_media_index, call:_progress_callback, call:_cancel_requested, call:os.replace, call:completed_index.status, call:_set_build_metadata, call:logger.exception, call:str, call:staging_path.exists, func:main() → int, call:argparse.ArgumentParser, call:parser.add_argument, call:parser.parse_args, call:run_build | dep: argparse, logging, os, time, pathlib, typing, media_library_viewer_api.config, media_library_viewer_api.dependencies, media_library_viewer_api.logging_utils, media_library_viewer_api.services.media_index
|
||||
## arch
|
||||
Subprocess worker pattern with cooperative cancellation, designed to be spawned and monitored by a parent process for long-running media indexing tasks.
|
||||
## tags
|
||||
call:, metadata, set, index, call:get, media, progress, worker
|
||||
## symbols
|
||||
- _set_build_metadata
|
||||
- _cancel_requested
|
||||
- _start_state
|
||||
- _progress_callback
|
||||
- run_build
|
||||
- main
|
||||
- call:state.items
|
||||
- call:index.set_metadata
|
||||
## workflows
|
||||
- change workers behavior
|
||||
read: __init__.py, media_index_worker.py
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,39 @@
|
||||
# backend/tests (index)
|
||||
dir: backend/tests
|
||||
|
||||
## role
|
||||
Test suite providing unit and integration tests that validate API endpoints, configuration, external clients, utilities, and service logic across the backend.
|
||||
## parent
|
||||
index: backend/.pi-map.index.md
|
||||
map: backend/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- __init__.py
|
||||
- test_api.py
|
||||
- test_backups.py
|
||||
- test_config.py
|
||||
- test_domain_media.py
|
||||
- test_jellyseerr_client.py
|
||||
- test_jobs.py
|
||||
- test_mailer.py
|
||||
- test_media_index.py
|
||||
- test_observability.py
|
||||
- test_path_utils.py
|
||||
- test_services.py
|
||||
- test_ssh_client.py
|
||||
- test_targets.py
|
||||
- test_utils.py
|
||||
- test_widgets.py
|
||||
## links
|
||||
index: backend/tests/.pi-map.index.md
|
||||
map: backend/tests/.pi-map.md
|
||||
## workflows
|
||||
- change tests behavior
|
||||
read: __init__.py, test_api.py, test_backups.py
|
||||
- change tests CLI
|
||||
read: test_jellyseerr_client.py, test_ssh_client.py
|
||||
- change tests config
|
||||
read: test_config.py
|
||||
## dirty
|
||||
-
|
||||
File diff suppressed because one or more lines are too long
+290
-67
@@ -24,6 +24,10 @@ from media_library_viewer_api.main import app
|
||||
from media_library_viewer_api.routers.media import get_media_index
|
||||
from media_library_viewer_api.services.media_index import MediaIndex
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
from media_library_viewer_api.widgets.sources import ServiceRecord
|
||||
|
||||
# Short alias for the monitoring router module under test.
|
||||
_MON = "media_library_viewer_api.routers.monitoring"
|
||||
|
||||
# --- Fixtures ---
|
||||
|
||||
@@ -207,6 +211,22 @@ class TestDashboard:
|
||||
data = response.json()
|
||||
assert len(data) == 2
|
||||
|
||||
def test_jellyfin_endpoints_return_503_when_not_configured(self, test_client):
|
||||
# Remove the mocked Jellyfin dependency so the real one runs; with no
|
||||
# Jellyfin service seeded, endpoints must degrade to 503, not 500.
|
||||
app.dependency_overrides.pop(get_jellyfin_client, None)
|
||||
app.dependency_overrides.pop(get_user_id, None)
|
||||
try:
|
||||
for path in ("/api/dashboard/counts", "/api/dashboard/activity"):
|
||||
response = test_client.get(path)
|
||||
assert response.status_code == 503, path
|
||||
detail = response.json()["detail"]
|
||||
assert "configured" in detail, path
|
||||
finally:
|
||||
# Restore the mocks for subsequent tests in this fixture session.
|
||||
app.dependency_overrides[get_jellyfin_client] = lambda: MagicMock()
|
||||
app.dependency_overrides[get_user_id] = lambda: "user123"
|
||||
|
||||
|
||||
# --- Settings reset ---
|
||||
|
||||
@@ -626,28 +646,98 @@ class TestMonitoring:
|
||||
assert data[0]["labels"]["job"] == "node-exporter-remote"
|
||||
|
||||
|
||||
class TestSettingsMachines:
|
||||
def test_post_machine_rewrites_prometheus_targets(self, test_client):
|
||||
with patch("media_library_viewer_api.routers.settings.write_prometheus_targets") as write_targets:
|
||||
with patch("media_library_viewer_api.routers.settings._validate_saved_machine_ssh"):
|
||||
response = test_client.post(
|
||||
"/api/settings/machines",
|
||||
json={
|
||||
"name": "remote1",
|
||||
"mode": "ssh",
|
||||
"enabled": True,
|
||||
"services": ["monitoring"],
|
||||
"host": "10.0.0.5",
|
||||
"username": "u",
|
||||
"node_exporter_enabled": True,
|
||||
"node_exporter_port": 9200,
|
||||
"node_exporter_scrape_host": "1.2.3.4",
|
||||
},
|
||||
)
|
||||
assert response.status_code == 201
|
||||
write_targets.assert_called_once()
|
||||
class TestResolveServiceRecord:
|
||||
"""Unit tests for _resolve_service_record (service_id + first-enabled paths)."""
|
||||
|
||||
def test_delete_machine_rewrites_prometheus_targets(self, test_client):
|
||||
def _store(self, rows):
|
||||
store = MagicMock()
|
||||
store.get_service = lambda sid: next((r for r in rows if r["id"] == sid), None)
|
||||
|
||||
def list_filtered(service_type=None):
|
||||
return [r for r in rows if r["service_type"] == (service_type or r["service_type"])]
|
||||
|
||||
store.list_services = list_filtered
|
||||
return store
|
||||
|
||||
def test_service_id_match_returns_record(self):
|
||||
from media_library_viewer_api.routers.monitoring import _resolve_service_record
|
||||
|
||||
row = {"id": "am1", "service_type": "alertmanager", "name": "AM", "enabled": True, "config": {}, "secrets": {}}
|
||||
store = self._store([row])
|
||||
with patch(f"{_MON}.build_service_record", return_value="RECORD") as mock_build:
|
||||
result = _resolve_service_record(store, "alertmanager", "am1")
|
||||
assert result == "RECORD"
|
||||
mock_build.assert_called_once_with(store, row)
|
||||
|
||||
def test_service_id_type_mismatch_returns_none(self):
|
||||
from media_library_viewer_api.routers.monitoring import _resolve_service_record
|
||||
|
||||
row = {"id": "x1", "service_type": "grafana", "name": "G", "enabled": True, "config": {}, "secrets": {}}
|
||||
store = self._store([row])
|
||||
assert _resolve_service_record(store, "alertmanager", "x1") is None
|
||||
|
||||
def test_service_id_disabled_returns_none(self):
|
||||
from media_library_viewer_api.routers.monitoring import _resolve_service_record
|
||||
|
||||
row = {"id": "am1", "service_type": "alertmanager", "name": "AM", "enabled": False, "config": {}, "secrets": {}}
|
||||
store = self._store([row])
|
||||
assert _resolve_service_record(store, "alertmanager", "am1") is None
|
||||
|
||||
def test_no_service_id_returns_first_enabled(self):
|
||||
from media_library_viewer_api.routers.monitoring import _resolve_service_record
|
||||
|
||||
rows = [
|
||||
{
|
||||
"id": "am1",
|
||||
"service_type": "alertmanager",
|
||||
"name": "Disabled",
|
||||
"enabled": False,
|
||||
"config": {},
|
||||
"secrets": {},
|
||||
},
|
||||
{
|
||||
"id": "am2",
|
||||
"service_type": "alertmanager",
|
||||
"name": "Active",
|
||||
"enabled": True,
|
||||
"config": {},
|
||||
"secrets": {},
|
||||
},
|
||||
]
|
||||
store = self._store(rows)
|
||||
with patch(f"{_MON}.build_service_record", return_value="RECORD") as mock_build:
|
||||
result = _resolve_service_record(store, "alertmanager", None)
|
||||
assert result == "RECORD"
|
||||
mock_build.assert_called_once_with(store, rows[1])
|
||||
|
||||
def test_no_service_id_and_none_enabled_returns_none(self):
|
||||
from media_library_viewer_api.routers.monitoring import _resolve_service_record
|
||||
|
||||
store = self._store([])
|
||||
assert _resolve_service_record(store, "alertmanager", None) is None
|
||||
|
||||
|
||||
class TestSettingsMachines:
|
||||
def test_machine_appears_in_prometheus_targets(self, test_client):
|
||||
store = app.dependency_overrides[get_settings_store]()
|
||||
store.upsert_machine(
|
||||
{
|
||||
"name": "remote1",
|
||||
"mode": "ssh",
|
||||
"enabled": True,
|
||||
"services": ["monitoring"],
|
||||
"host": "10.0.0.5",
|
||||
"username": "u",
|
||||
"node_exporter_enabled": True,
|
||||
"node_exporter_port": 9200,
|
||||
"node_exporter_scrape_host": "1.2.3.4",
|
||||
}
|
||||
)
|
||||
targets = test_client.get("/api/monitoring/prometheus-targets").json()
|
||||
assert len(targets) == 1
|
||||
assert targets[0]["targets"] == ["1.2.3.4:9200"]
|
||||
|
||||
def test_delete_machine_removed_from_prometheus_targets(self, test_client):
|
||||
store = app.dependency_overrides[get_settings_store]()
|
||||
machine = store.upsert_machine(
|
||||
{
|
||||
@@ -657,62 +747,130 @@ class TestSettingsMachines:
|
||||
"services": ["monitoring"],
|
||||
"host": "10.0.0.5",
|
||||
"username": "u",
|
||||
"node_exporter_enabled": True,
|
||||
"node_exporter_port": 9200,
|
||||
"node_exporter_scrape_host": "1.2.3.4",
|
||||
}
|
||||
)
|
||||
with patch("media_library_viewer_api.routers.settings.write_prometheus_targets") as write_targets:
|
||||
response = test_client.delete(f"/api/settings/machines/{machine['id']}")
|
||||
response = test_client.delete(f"/api/settings/machines/{machine['id']}")
|
||||
assert response.status_code == 200
|
||||
write_targets.assert_called_once()
|
||||
assert test_client.get("/api/monitoring/prometheus-targets").json() == []
|
||||
|
||||
|
||||
def _am_service(name="Alertmanager", **config):
|
||||
cfg = {"base_url": "http://alertmanager:9093", "timeout_seconds": 5}
|
||||
cfg.update(config)
|
||||
return ServiceRecord(id="am1", service_type="alertmanager", name=name, config=cfg)
|
||||
|
||||
|
||||
class TestAlertmanager:
|
||||
def test_alerts_endpoint_when_alertmanager_unreachable(self, test_client):
|
||||
with patch("media_library_viewer_api.routers.monitoring._alertmanager_client") as mock_client:
|
||||
session = MagicMock()
|
||||
session.get.side_effect = Exception("connection refused")
|
||||
mock_client.return_value = (session, "http://alertmanager:9093")
|
||||
def test_alerts_endpoint_when_not_configured(self, test_client):
|
||||
# No alertmanager service seeded -> registry resolves None.
|
||||
response = test_client.get("/api/monitoring/alerts")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["total"] == 0
|
||||
assert data["error"] == "alertmanager_not_configured"
|
||||
|
||||
def test_alerts_endpoint_when_unreachable(self, test_client):
|
||||
service = _am_service()
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", side_effect=Exception("connection refused")),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/alerts")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["total"] == 0
|
||||
assert data["error"] == "alertmanager_unreachable"
|
||||
assert data["service_id"] == "am1"
|
||||
assert data["name"] == "Alertmanager"
|
||||
|
||||
def test_alerts_endpoint_returns_summary(self, test_client):
|
||||
with patch("media_library_viewer_api.routers.monitoring._alertmanager_client") as mock_client:
|
||||
session = MagicMock()
|
||||
session.get.return_value.json.return_value = {
|
||||
"status": "success",
|
||||
"data": [
|
||||
{
|
||||
"labels": {"alertname": "BackupJobFailed", "severity": "critical", "job_name": "test"},
|
||||
"annotations": {"summary": "Backup failed", "description": "details"},
|
||||
"startsAt": "2026-05-11T02:00:00Z",
|
||||
"status": "firing",
|
||||
}
|
||||
],
|
||||
}
|
||||
session.get.return_value.raise_for_status = MagicMock()
|
||||
mock_client.return_value = (session, "http://alertmanager:9093")
|
||||
service = _am_service()
|
||||
resp = MagicMock()
|
||||
resp.json.return_value = {
|
||||
"status": "success",
|
||||
"data": [
|
||||
{
|
||||
"labels": {"alertname": "BackupJobFailed", "severity": "critical", "job_name": "test"},
|
||||
"annotations": {"summary": "Backup failed", "description": "details"},
|
||||
"startsAt": "2026-05-11T02:00:00Z",
|
||||
"status": "firing",
|
||||
}
|
||||
],
|
||||
}
|
||||
resp.raise_for_status = MagicMock()
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", return_value=resp),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/alerts")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["total"] == 1
|
||||
assert data["by_severity"]["critical"] == 1
|
||||
assert data["alerts"][0]["name"] == "BackupJobFailed"
|
||||
assert data["service_id"] == "am1"
|
||||
|
||||
def test_alertmanager_status_endpoint_when_unreachable(self, test_client):
|
||||
with patch("media_library_viewer_api.routers.monitoring._alertmanager_client") as mock_client:
|
||||
session = MagicMock()
|
||||
session.get.side_effect = Exception("connection refused")
|
||||
mock_client.return_value = (session, "http://alertmanager:9093")
|
||||
def test_alerts_endpoint_sends_bearer_token(self, test_client):
|
||||
service = _am_service()
|
||||
service.secrets = {"api_key": "tok"}
|
||||
resp = MagicMock()
|
||||
resp.json.return_value = {"status": "success", "data": []}
|
||||
resp.raise_for_status = MagicMock()
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", return_value=resp) as mock_get,
|
||||
):
|
||||
test_client.get("/api/monitoring/alerts")
|
||||
_, kwargs = mock_get.call_args
|
||||
assert kwargs["headers"]["Authorization"] == "Bearer tok"
|
||||
|
||||
def test_alertmanager_status_when_not_configured(self, test_client):
|
||||
response = test_client.get("/api/monitoring/alertmanager-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is False
|
||||
assert data["error"] == "alertmanager_not_configured"
|
||||
assert data["peers"] == []
|
||||
|
||||
def test_alertmanager_status_when_unreachable(self, test_client):
|
||||
service = _am_service()
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", side_effect=Exception("refused")),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/alertmanager-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is False
|
||||
assert data["error"] == "alertmanager_unreachable"
|
||||
assert data["name"] == "Alertmanager"
|
||||
|
||||
def test_alertmanager_status_returns_cluster_info(self, test_client):
|
||||
service = _am_service()
|
||||
resp = MagicMock()
|
||||
resp.json.return_value = {
|
||||
"versionInfo": {"version": "0.27.0"},
|
||||
"clusterStatus": {"createdAt": "2026-06-01T00:00:00Z"},
|
||||
"cluster": {"peers": [{"name": "am-1"}, {"name": "am-2"}]},
|
||||
}
|
||||
resp.raise_for_status = MagicMock()
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", return_value=resp),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/alertmanager-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is True
|
||||
assert data["version"] == "0.27.0"
|
||||
assert data["peers"] == ["am-1", "am-2"]
|
||||
|
||||
|
||||
class TestAlertmanagerWebhook:
|
||||
"""Tests for POST /api/monitoring/alertmanager-webhook."""
|
||||
"""Tests for POST /api/monitoring/alertmanager-webhook (log-only receiver)."""
|
||||
|
||||
def test_webhook_receives_payload_and_logs(self, test_client, caplog):
|
||||
payload = {
|
||||
@@ -732,22 +890,87 @@ class TestAlertmanagerWebhook:
|
||||
assert response.json()["status"] == "received"
|
||||
assert "Received Alertmanager webhook with 1 alert(s)" in caplog.text
|
||||
|
||||
def test_webhook_forwards_when_configured(self, test_client, caplog, monkeypatch):
|
||||
payload = {"status": "resolved", "alerts": []}
|
||||
forwarded = {"captured": False}
|
||||
|
||||
class FakeResponse:
|
||||
def raise_for_status(self):
|
||||
pass
|
||||
|
||||
def fake_post(url, json, timeout):
|
||||
forwarded["captured"] = True
|
||||
forwarded["url"] = url
|
||||
forwarded["payload"] = json
|
||||
return FakeResponse()
|
||||
|
||||
monkeypatch.setattr("requests.Session.post", lambda _self, url, json, timeout: fake_post(url, json, timeout))
|
||||
with caplog.at_level("INFO", logger="media_library_viewer_api.routers.monitoring"):
|
||||
response = test_client.post("/api/monitoring/alertmanager-webhook", json=payload)
|
||||
class TestGrafanaStatus:
|
||||
def test_grafana_status_when_not_configured(self, test_client):
|
||||
response = test_client.get("/api/monitoring/grafana-status")
|
||||
assert response.status_code == 200
|
||||
assert forwarded["captured"] is False
|
||||
data = response.json()
|
||||
assert data["up"] is False
|
||||
assert data["error"] == "no_service_configured"
|
||||
|
||||
def test_grafana_status_when_unreachable(self, test_client):
|
||||
service = ServiceRecord(
|
||||
id="g1", service_type="grafana", name="Grafana", config={"base_url": "http://grafana:3000"}
|
||||
)
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", side_effect=Exception("refused")),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/grafana-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is False
|
||||
assert data["error"] == "grafana_unreachable"
|
||||
assert data["name"] == "Grafana"
|
||||
|
||||
def test_grafana_status_returns_version(self, test_client):
|
||||
service = ServiceRecord(
|
||||
id="g1", service_type="grafana", name="Grafana", config={"base_url": "http://grafana:3000"}
|
||||
)
|
||||
resp = MagicMock()
|
||||
resp.json.return_value = {"version": "11.3.1", "database": "ok"}
|
||||
resp.raise_for_status = MagicMock()
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", return_value=resp),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/grafana-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is True
|
||||
assert data["version"] == "11.3.1"
|
||||
assert data["service_id"] == "g1"
|
||||
|
||||
|
||||
class TestPrometheusStatus:
|
||||
def test_prometheus_status_when_not_configured(self, test_client):
|
||||
response = test_client.get("/api/monitoring/prometheus-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is False
|
||||
assert data["error"] == "no_service_configured"
|
||||
|
||||
def test_prometheus_status_when_unreachable(self, test_client):
|
||||
service = ServiceRecord(
|
||||
id="p1", service_type="prometheus", name="Prometheus", config={"base_url": "http://prometheus:9090"}
|
||||
)
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", side_effect=Exception("refused")),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/prometheus-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is False
|
||||
assert data["error"] == "prometheus_unreachable"
|
||||
|
||||
def test_prometheus_status_returns_version(self, test_client):
|
||||
service = ServiceRecord(
|
||||
id="p1", service_type="prometheus", name="Prometheus", config={"base_url": "http://prometheus:9090"}
|
||||
)
|
||||
health = MagicMock()
|
||||
health.raise_for_status = MagicMock()
|
||||
build_info = MagicMock()
|
||||
build_info.raise_for_status = MagicMock()
|
||||
build_info.json.return_value = {"status": "success", "data": {"version": "2.55.1"}}
|
||||
with (
|
||||
patch(f"{_MON}._resolve_service_record", return_value=service),
|
||||
patch(f"{_MON}.requests.get", side_effect=[health, build_info]),
|
||||
):
|
||||
response = test_client.get("/api/monitoring/prometheus-status")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["up"] is True
|
||||
assert data["version"] == "2.55.1"
|
||||
assert data["service_id"] == "p1"
|
||||
|
||||
@@ -8,6 +8,7 @@ from unittest.mock import patch
|
||||
import pytest
|
||||
from cryptography.fernet import Fernet
|
||||
from fastapi.testclient import TestClient
|
||||
from pydantic import ValidationError
|
||||
|
||||
from media_library_viewer_api.dependencies import get_settings_store
|
||||
from media_library_viewer_api.integrations.registry import (
|
||||
@@ -56,10 +57,11 @@ def client(tmp_path):
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_registry_contains_five_service_types():
|
||||
def test_registry_contains_seven_service_types():
|
||||
assert set(SERVICE_DEFINITIONS) == {
|
||||
"grafana",
|
||||
"prometheus",
|
||||
"alertmanager",
|
||||
"jellyfin",
|
||||
"jellyseerr",
|
||||
"nextcloud",
|
||||
@@ -70,6 +72,7 @@ def test_registry_contains_five_service_types():
|
||||
def test_definitions_declare_widget_kinds():
|
||||
assert {wk.kind for wk in get_service_definition("grafana").widget_kinds} == {"link"}
|
||||
assert {wk.kind for wk in get_service_definition("prometheus").widget_kinds} == {"metric"}
|
||||
assert {wk.kind for wk in get_service_definition("alertmanager").widget_kinds} == {"active_alerts"}
|
||||
assert {wk.kind for wk in get_service_definition("jellyfin").widget_kinds} == {"activity"}
|
||||
assert get_service_definition("nextcloud").widget_kinds == []
|
||||
assert {wk.kind for wk in get_service_definition("ssh_tasks").widget_kinds} == {"task_output"}
|
||||
@@ -135,6 +138,7 @@ def test_list_service_types(client):
|
||||
assert response.status_code == 200
|
||||
types = {item["service_type"] for item in response.json()}
|
||||
assert types == {
|
||||
"alertmanager",
|
||||
"grafana",
|
||||
"jellyfin",
|
||||
"jellyseerr",
|
||||
@@ -241,7 +245,8 @@ def test_invalid_config_rejected(client):
|
||||
"/api/services/instances",
|
||||
json={"service_type": "grafana", "name": "x", "config": {"base_url": ""}},
|
||||
)
|
||||
# Pydantic accepts empty string; force a real validation error via bad type.
|
||||
assert response.status_code == 422
|
||||
# Force a real validation error via bad type.
|
||||
response = client.post(
|
||||
"/api/services/instances",
|
||||
json={"service_type": "grafana", "name": "x", "config": {"timeout_seconds": "fast"}},
|
||||
@@ -249,6 +254,25 @@ def test_invalid_config_rejected(client):
|
||||
assert response.status_code == 422
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad_url", ["grafana.example.com", "localhost:3000", "//grafana.example.com", "ftp://grafana.example.com"]
|
||||
)
|
||||
def test_service_base_url_requires_http_schema(bad_url):
|
||||
"""Every service base_url must include an http:// or https:// schema."""
|
||||
model = get_service_definition("grafana").config_model
|
||||
with pytest.raises(ValidationError):
|
||||
model.model_validate({"base_url": bad_url, "timeout_seconds": 5})
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"service_type", ["grafana", "prometheus", "alertmanager", "jellyfin", "jellyseerr", "nextcloud"]
|
||||
)
|
||||
def test_service_base_url_accepts_absolute_urls(service_type):
|
||||
model = get_service_definition(service_type).config_model
|
||||
instance = model.model_validate({"base_url": "https://example.com"})
|
||||
assert instance.base_url == "https://example.com"
|
||||
|
||||
|
||||
def test_unknown_secret_field_rejected(client):
|
||||
response = client.post(
|
||||
"/api/services/instances",
|
||||
|
||||
@@ -1,14 +1,11 @@
|
||||
"""Tests for Prometheus file-based service discovery target generation."""
|
||||
"""Tests for Prometheus Node Exporter target discovery."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
from media_library_viewer_api.services.targets import (
|
||||
build_node_exporter_targets,
|
||||
write_prometheus_targets,
|
||||
)
|
||||
from media_library_viewer_api.services.targets import build_node_exporter_targets
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -88,22 +85,3 @@ class TestBuildNodeExporterTargets:
|
||||
}
|
||||
)
|
||||
assert build_node_exporter_targets(store) == []
|
||||
|
||||
|
||||
class TestWritePrometheusTargets:
|
||||
def test_writes_valid_json(self, store: SettingsStore, tmp_path: Path):
|
||||
store.upsert_machine(
|
||||
{
|
||||
"name": "remote1",
|
||||
"mode": "ssh",
|
||||
"host": "10.0.0.5",
|
||||
"username": "u",
|
||||
"node_exporter_enabled": True,
|
||||
"node_exporter_port": 9200,
|
||||
}
|
||||
)
|
||||
file_path = write_prometheus_targets(store, tmp_path)
|
||||
assert file_path.exists()
|
||||
assert file_path.name == "node_exporter_targets.json"
|
||||
targets = build_node_exporter_targets(store)
|
||||
assert len(targets) == 1
|
||||
|
||||
@@ -13,6 +13,7 @@ from media_library_viewer_api.dependencies import get_settings_store
|
||||
from media_library_viewer_api.main import app
|
||||
from media_library_viewer_api.services.settings_store import SettingsStore
|
||||
from media_library_viewer_api.widgets.sources import (
|
||||
AlertmanagerWidgetSource,
|
||||
BackupsWidgetSource,
|
||||
GrafanaWidgetSource,
|
||||
ServiceRecord,
|
||||
@@ -324,6 +325,76 @@ async def test_grafana_adapter_missing_service():
|
||||
assert "error" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_alertmanager_adapter_summarizes_alerts():
|
||||
adapter = AlertmanagerWidgetSource()
|
||||
service = ServiceRecord(id="s", service_type="alertmanager", name="am", config={"base_url": "http://am:9093"})
|
||||
payload = SimpleNamespace(
|
||||
raise_for_status=lambda: None,
|
||||
json=lambda: {
|
||||
"status": "success",
|
||||
"data": [
|
||||
{
|
||||
"labels": {"alertname": "DiskFull", "severity": "critical"},
|
||||
"annotations": {"summary": "disk full"},
|
||||
"startsAt": "2026-06-23T00:00:00Z",
|
||||
"status": "firing",
|
||||
}
|
||||
],
|
||||
},
|
||||
)
|
||||
with patch("media_library_viewer_api.widgets.sources.requests.get", return_value=payload):
|
||||
result = await adapter.fetch(service, "active_alerts", {})
|
||||
assert result["total"] == 1
|
||||
assert result["by_severity"]["critical"] == 1
|
||||
assert result["alerts"][0]["name"] == "DiskFull"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_alertmanager_adapter_applies_severity_filter():
|
||||
adapter = AlertmanagerWidgetSource()
|
||||
service = ServiceRecord(id="s", service_type="alertmanager", name="am", config={"base_url": "http://am:9093"})
|
||||
payload = SimpleNamespace(
|
||||
raise_for_status=lambda: None,
|
||||
json=lambda: {
|
||||
"status": "success",
|
||||
"data": [
|
||||
{"labels": {"alertname": "A", "severity": "critical"}, "annotations": {}, "status": "firing"},
|
||||
{"labels": {"alertname": "B", "severity": "warning"}, "annotations": {}, "status": "firing"},
|
||||
],
|
||||
},
|
||||
)
|
||||
with patch("media_library_viewer_api.widgets.sources.requests.get", return_value=payload):
|
||||
result = await adapter.fetch(service, "active_alerts", {"severity_filter": "critical"})
|
||||
assert result["total"] == 1
|
||||
assert result["alerts"][0]["name"] == "A"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_alertmanager_adapter_sends_bearer_token():
|
||||
adapter = AlertmanagerWidgetSource()
|
||||
service = ServiceRecord(
|
||||
id="s",
|
||||
service_type="alertmanager",
|
||||
name="am",
|
||||
config={"base_url": "http://am:9093"},
|
||||
secrets={"api_key": "tok"},
|
||||
)
|
||||
payload = SimpleNamespace(raise_for_status=lambda: None, json=lambda: {"status": "success", "data": []})
|
||||
with patch("media_library_viewer_api.widgets.sources.requests.get", return_value=payload) as mock_get:
|
||||
result = await adapter.fetch(service, "active_alerts", {})
|
||||
assert result["total"] == 0
|
||||
_, kwargs = mock_get.call_args
|
||||
assert kwargs["headers"]["Authorization"] == "Bearer tok"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_alertmanager_adapter_missing_service():
|
||||
adapter = AlertmanagerWidgetSource()
|
||||
result = await adapter.fetch(None, "active_alerts", {})
|
||||
assert "error" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_static_adapter():
|
||||
adapter = StaticWidgetSource()
|
||||
@@ -359,7 +430,7 @@ async def test_ssh_task_adapter_records_history_on_run(client):
|
||||
"task_type": "shell",
|
||||
"content": "echo hi",
|
||||
"enabled": True,
|
||||
"default_machine_id": "",
|
||||
"default_service_id": "",
|
||||
}
|
||||
)
|
||||
service = store.upsert_service(
|
||||
@@ -377,7 +448,7 @@ async def test_ssh_task_adapter_records_history_on_run(client):
|
||||
)
|
||||
with (
|
||||
patch("media_library_viewer_api.widgets.sources.get_settings_store", return_value=store),
|
||||
patch("media_library_viewer_api.widgets.sources._build_ssh_client", return_value=fake_client),
|
||||
patch("media_library_viewer_api.services.task_runner.build_ssh_client", return_value=fake_client),
|
||||
):
|
||||
result = await adapter.fetch(service_record, "task_output", {"task_id": task["id"]})
|
||||
|
||||
|
||||
+34
-2
@@ -1,6 +1,22 @@
|
||||
# Code Context
|
||||
|
||||
> **Status (2026-06-23):** Manage no longer deploys an observability stack,
|
||||
> and observability is no longer env-configured. The root `docker-compose.yml` /
|
||||
> `docker-compose.dev.yml` ship **only** the backend and frontend; Grafana,
|
||||
> Prometheus, Loki, Alertmanager, Alloy, and Node Exporter were removed from
|
||||
> them. Manage connects to **existing** instances, and Alertmanager/Grafana/
|
||||
> Prometheus are now **service-registry instances** configured in the UI (no
|
||||
> `ALERTMANAGER_URL`, `VITE_GRAFANA_URL`, `VITE_PROMETHEUS_URL`, or
|
||||
> `PROMETHEUS_FILE_SD_DIR` env vars). The file-SD writer was removed; external
|
||||
> Prometheus uses `http_sd_configs` against `GET /api/monitoring/prometheus-targets`.
|
||||
> The standalone example stack lives in `docker-compose.observability.yml`. The
|
||||
> detailed snippets below describe the **former** in-compose, env-configured
|
||||
> model and are kept as historical context only; treat the service registry
|
||||
> (Services page UI + `openspec/changes/observability-service-registry/`) and
|
||||
> `docker-compose.observability.yml` as authoritative.
|
||||
|
||||
## Files Retrieved
|
||||
|
||||
1. `docker-compose.yml` (lines 1–262) – production Compose stack; defines observability services and Traefik routing.
|
||||
2. `docker-compose.dev.yml` (lines 1–234) – development Compose stack; same observability services but with host ports exposed and auth disabled.
|
||||
3. `.env.example` (lines 1–55) – template with all required environment variables for the stack, including Prometheus/Grafana/Alertmanager/Alloy/Loki and Node Exporter settings.
|
||||
@@ -20,7 +36,9 @@
|
||||
## Key Code
|
||||
|
||||
### Backend `/metrics` endpoint
|
||||
|
||||
`backend/src/media_library_viewer_api/main.py`:
|
||||
|
||||
```python
|
||||
@app.middleware("http")
|
||||
async def enforce_jwt_auth(request: Request, call_next):
|
||||
@@ -36,7 +54,9 @@ def metrics() -> Response:
|
||||
```
|
||||
|
||||
### Metrics emitted by the backend
|
||||
|
||||
`backend/src/media_library_viewer_api/observability.py`:
|
||||
|
||||
```python
|
||||
REQUESTS_TOTAL = Counter("manage_api_requests_total", "Total API requests", ["method", "path", "status_code"])
|
||||
REQUEST_DURATION = Histogram("manage_api_request_duration_seconds", "API request duration", ["method", "path"], ...)
|
||||
@@ -48,7 +68,9 @@ MAIL_QUEUE_SIZE = Counter("manage_mail_queue_messages_total", "Total messages en
|
||||
```
|
||||
|
||||
### Prometheus scrape configuration
|
||||
|
||||
`monitoring/prometheus/prometheus.yml`:
|
||||
|
||||
```yaml
|
||||
scrape_configs:
|
||||
- job_name: manage-backend
|
||||
@@ -71,7 +93,9 @@ scrape_configs:
|
||||
```
|
||||
|
||||
### Backend-managed remote Node Exporter targets
|
||||
|
||||
`backend/src/media_library_viewer_api/services/targets.py`:
|
||||
|
||||
```python
|
||||
def build_node_exporter_targets(store: SettingsStore) -> list[dict[str, Any]]:
|
||||
...
|
||||
@@ -96,6 +120,7 @@ The observability stack is a standard self-hosted Prometheus/Grafana/Loki/Alertm
|
||||
- **Alertmanager** routes alerts by severity (critical vs warning) and delivers email notifications (and optionally a webhook back to the backend).
|
||||
|
||||
The backend bridges the stack with the application:
|
||||
|
||||
- It exposes `/metrics` (unauthenticated, along with `/api/health` and `/api/version`).
|
||||
- On startup it writes `${PROMETHEUS_FILE_SD_DIR}/node_exporter_targets.json` based on enabled SSH machines in the settings store.
|
||||
- It provides proxy endpoints (`/api/monitoring/alerts`, `/api/monitoring/alertmanager-status`, `/api/monitoring/prometheus-targets`) consumed by the frontend.
|
||||
@@ -105,6 +130,7 @@ The backend bridges the stack with the application:
|
||||
Open `monitoring/prometheus/prometheus.yml` first to understand what is scraped and how the backend is wired, then read `backend/src/media_library_viewer_api/observability.py` to see the metric names and labels. For environment requirements, read `.env.example`.
|
||||
|
||||
## Supervisor coordination
|
||||
|
||||
Not needed — this is a read-only scouting summary.
|
||||
|
||||
---
|
||||
@@ -113,7 +139,7 @@ Not needed — this is a read-only scouting summary.
|
||||
|
||||
## 1. Observability services defined in Compose
|
||||
|
||||
Both `docker-compose.yml` and `docker-compose.dev.yml` define the following services:
|
||||
Both `docker-compose.yml` and `docker-compose.dev.yml` define **only the backend and frontend**. The observability services (Prometheus, Loki, Grafana, Alertmanager, Alloy, Node Exporter) were extracted to the standalone `docker-compose.observability.yml` example stack and are **no longer** deployed by Manage. Summary of what remains in the app compose files:
|
||||
|
||||
| Service | Image | Internal endpoint | Purpose |
|
||||
|---------|-------|-------------------|---------|
|
||||
@@ -126,6 +152,7 @@ Both `docker-compose.yml` and `docker-compose.dev.yml` define the following serv
|
||||
| `backend` | Build from `backend/Dockerfile` | `http://backend:8000` | FastAPI app exposing `/metrics` |
|
||||
|
||||
Differences:
|
||||
|
||||
- Production (`docker-compose.yml`): services attach to an external `web` network for Traefik, use `expose` instead of host ports for most services, and require OIDC/auth variables.
|
||||
- Development (`docker-compose.dev.yml`): Prometheus/Grafana/Loki/Alertmanager/Node Exporter are published on host ports `9090`, `3000`, `3100`, `9093`, `9100`; auth is disabled (`AUTH_ENABLED=false`).
|
||||
|
||||
@@ -134,6 +161,7 @@ Differences:
|
||||
From `.env.example` and the Compose files, the variables relevant to the observability stack are:
|
||||
|
||||
### Backend / metrics
|
||||
|
||||
- `PROMETHEUS_ENABLED` – enable metrics endpoint (set to `"true"` in both compose files).
|
||||
- `PROMETHEUS_FILE_SD_DIR` – directory where the backend writes `node_exporter_targets.json` (default `/app/backend/.cache/prometheus-file-sd`).
|
||||
- `ALERTMANAGER_URL` – backend proxy target (default `http://alertmanager:9093`).
|
||||
@@ -141,6 +169,7 @@ From `.env.example` and the Compose files, the variables relevant to the observa
|
||||
- `BACKEND_CACHE_DIR` – host directory mounted into backend and Prometheus for file-SD.
|
||||
|
||||
### Grafana
|
||||
|
||||
- `GRAFANA_APP_HOST` – public hostname for Grafana (production; required).
|
||||
- `GRAFANA_APP_PORT` – defaults to `3000`.
|
||||
- `GRAFANA_APP_NAME` – defaults to `grafana`.
|
||||
@@ -152,16 +181,19 @@ From `.env.example` and the Compose files, the variables relevant to the observa
|
||||
- `GF_AUTH_GENERIC_OAUTH_API_URL`
|
||||
|
||||
### Alertmanager
|
||||
|
||||
- `SMTP_HOST` / `SMTP_PORT`
|
||||
- `SMTP_USERNAME` / `SMTP_PASSWORD`
|
||||
- `SMTP_FROM_ADDRESS`
|
||||
- `ALERT_EMAIL_TO`
|
||||
|
||||
### Traefik / network (production)
|
||||
|
||||
- `BACKEND_APP_HOST` / `FRONTEND_APP_HOST` / `GRAFANA_APP_HOST`
|
||||
- `CERT_RESOLVER` – e.g. `letsencrypt`
|
||||
|
||||
### General
|
||||
|
||||
- `LOG_LEVEL` / `LOG_FORMAT` – also passed to Grafana and backend.
|
||||
|
||||
## 3. Monitoring config files
|
||||
@@ -193,7 +225,7 @@ Remote Node Exporter targets are not static: the backend reads machine settings
|
||||
|
||||
## 5. Setup steps and gotchas
|
||||
|
||||
- The observability stack is brought up with the app itself:
|
||||
- Manage's own compose stack does **not** include observability services. To run a full local stack, bring up the app and the standalone observability example separately:
|
||||
- Production: `docker compose -f docker-compose.yml up --build`
|
||||
- Development: `docker compose -f docker-compose.dev.yml up --build`
|
||||
- Production requires the external `web` network and Traefik already configured; `docker-compose.dev.yml` does not use Traefik and binds ports directly.
|
||||
|
||||
+11
-207
@@ -1,3 +1,12 @@
|
||||
# Manage development stack — backend + frontend only.
|
||||
#
|
||||
# Manage does NOT deploy its own Grafana/Prometheus/Loki/Alertmanager stack.
|
||||
# It exposes a /metrics endpoint and optional Alertmanager proxy endpoints so
|
||||
# an *existing* observability deployment can scrape/consume them.
|
||||
#
|
||||
# For a ready-to-use example observability stack, see
|
||||
# docker-compose.observability.yml and docs/observability-runbooks.md.
|
||||
|
||||
services:
|
||||
backend:
|
||||
build:
|
||||
@@ -14,9 +23,6 @@ services:
|
||||
LOG_FORMAT: ${LOG_FORMAT:-text}
|
||||
PROMETHEUS_ENABLED: "true"
|
||||
SSH_KNOWN_HOSTS_PATH: /app/backend/.cache/known_hosts
|
||||
PROMETHEUS_FILE_SD_DIR: /app/backend/.cache/prometheus-file-sd
|
||||
ALERTMANAGER_URL: ${ALERTMANAGER_URL:-http://alertmanager:9093}
|
||||
ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-}
|
||||
MANAGE_ENCRYPTION_KEY: ${MANAGE_ENCRYPTION_KEY:?set MANAGE_ENCRYPTION_KEY in your .env}
|
||||
ports:
|
||||
- "8000:8000"
|
||||
@@ -25,7 +31,6 @@ services:
|
||||
- ${BACKEND_CACHE_DIR:-./backend-cache}:/app/backend/.cache
|
||||
networks:
|
||||
- web
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
|
||||
frontend:
|
||||
@@ -38,8 +43,6 @@ services:
|
||||
VITE_API_URL: "/api"
|
||||
VITE_OIDC_ENABLED: "false"
|
||||
VITE_DEV_API_PROXY_TARGET: "http://backend:8000"
|
||||
VITE_GRAFANA_URL: "http://localhost:3000"
|
||||
VITE_PROMETHEUS_URL: "http://localhost:9090"
|
||||
ports:
|
||||
- "5173:5173"
|
||||
volumes:
|
||||
@@ -47,211 +50,12 @@ services:
|
||||
- frontend_node_modules:/app/frontend/node_modules
|
||||
depends_on:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
|
||||
prometheus:
|
||||
image: prom/prometheus:v2.55.1
|
||||
container_name: prometheus
|
||||
command:
|
||||
- "--config.file=/etc/prometheus/prometheus.yml"
|
||||
- "--storage.tsdb.path=/prometheus"
|
||||
- "--storage.tsdb.retention.time=30d"
|
||||
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
|
||||
- "--web.console.templates=/usr/share/prometheus/consoles"
|
||||
- "--web.enable-lifecycle"
|
||||
volumes:
|
||||
- ./monitoring/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
|
||||
- ./monitoring/prometheus/rules:/etc/prometheus/rules:ro
|
||||
- ${BACKEND_CACHE_DIR:-./backend-cache}/prometheus-file-sd:/etc/prometheus/file-sd:ro
|
||||
- prometheus_data:/prometheus
|
||||
ports:
|
||||
- "9090:9090"
|
||||
networks:
|
||||
- monitoring
|
||||
- web
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9090/-/healthy"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "1.00"
|
||||
memory: 1G
|
||||
reservations:
|
||||
cpus: "0.25"
|
||||
memory: 256M
|
||||
|
||||
loki:
|
||||
image: grafana/loki:3.1.1
|
||||
container_name: loki
|
||||
command: -config.file=/etc/loki/loki.yml
|
||||
volumes:
|
||||
- ./monitoring/loki/loki.yml:/etc/loki/loki.yml:ro
|
||||
- loki_data:/loki
|
||||
ports:
|
||||
- "3100:3100"
|
||||
networks:
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3100/ready"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "1.00"
|
||||
memory: 1G
|
||||
reservations:
|
||||
cpus: "0.25"
|
||||
memory: 256M
|
||||
|
||||
alloy:
|
||||
image: grafana/alloy:v1.5.0
|
||||
container_name: alloy
|
||||
command:
|
||||
- run
|
||||
- /etc/alloy/config.alloy
|
||||
- --storage.path=/var/lib/alloy
|
||||
volumes:
|
||||
- ./monitoring/alloy/config.alloy:/etc/alloy/config.alloy:ro
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
||||
networks:
|
||||
- monitoring
|
||||
depends_on:
|
||||
loki:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
user: root
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:12345/-/healthy"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "0.50"
|
||||
memory: 512M
|
||||
reservations:
|
||||
cpus: "0.10"
|
||||
memory: 128M
|
||||
|
||||
grafana:
|
||||
image: grafana/grafana:11.3.1
|
||||
container_name: grafana
|
||||
environment:
|
||||
GF_SECURITY_ADMIN_USER: ${GRAFANA_ADMIN_USER:-admin}
|
||||
GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD:-admin}
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_ID: ${GF_AUTH_GENERIC_OAUTH_CLIENT_ID:-}
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET:-}
|
||||
GF_AUTH_GENERIC_OAUTH_AUTH_URL: ${GF_AUTH_GENERIC_OAUTH_AUTH_URL:-}
|
||||
GF_AUTH_GENERIC_OAUTH_TOKEN_URL: ${GF_AUTH_GENERIC_OAUTH_TOKEN_URL:-}
|
||||
GF_AUTH_GENERIC_OAUTH_API_URL: ${GF_AUTH_GENERIC_OAUTH_API_URL:-}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||
volumes:
|
||||
- ./monitoring/grafana/grafana.ini:/etc/grafana/grafana.ini:ro
|
||||
- ./monitoring/grafana/provisioning:/etc/grafana/provisioning:ro
|
||||
- grafana_data:/var/lib/grafana
|
||||
ports:
|
||||
- "3000:3000"
|
||||
networks:
|
||||
- monitoring
|
||||
depends_on:
|
||||
prometheus:
|
||||
condition: service_healthy
|
||||
loki:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3000/api/health"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "1.00"
|
||||
memory: 512M
|
||||
reservations:
|
||||
cpus: "0.25"
|
||||
memory: 128M
|
||||
|
||||
alertmanager:
|
||||
image: prom/alertmanager:v0.27.0
|
||||
container_name: alertmanager
|
||||
command:
|
||||
- "--config.file=/etc/alertmanager/alertmanager.yml"
|
||||
- "--storage.path=/alertmanager"
|
||||
environment:
|
||||
SMTP_HOST: ${SMTP_HOST:-smtp.example.com}
|
||||
SMTP_PORT: ${SMTP_PORT:-587}
|
||||
SMTP_USERNAME: ${SMTP_USERNAME:-}
|
||||
SMTP_PASSWORD: ${SMTP_PASSWORD:-}
|
||||
SMTP_FROM_ADDRESS: ${SMTP_FROM_ADDRESS:-no-reply@example.com}
|
||||
ALERT_EMAIL_TO: ${ALERT_EMAIL_TO:-admin@example.com}
|
||||
volumes:
|
||||
- ./monitoring/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
|
||||
- alertmanager_data:/alertmanager
|
||||
ports:
|
||||
- "9093:9093"
|
||||
networks:
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9093/-/healthy"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "0.50"
|
||||
memory: 256M
|
||||
reservations:
|
||||
cpus: "0.10"
|
||||
memory: 64M
|
||||
|
||||
node-exporter:
|
||||
image: prom/node-exporter:v1.8.2
|
||||
container_name: node-exporter
|
||||
command:
|
||||
- "--path.rootfs=/host"
|
||||
volumes:
|
||||
- /:/host:ro,rslave
|
||||
networks:
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9100/"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "0.25"
|
||||
memory: 128M
|
||||
reservations:
|
||||
cpus: "0.05"
|
||||
memory: 32M
|
||||
|
||||
networks:
|
||||
monitoring:
|
||||
web:
|
||||
|
||||
volumes:
|
||||
frontend_node_modules:
|
||||
prometheus_data:
|
||||
loki_data:
|
||||
grafana_data:
|
||||
alertmanager_data:
|
||||
|
||||
@@ -1,4 +1,13 @@
|
||||
# Standalone observability stack.
|
||||
# Standalone observability stack (OPTIONAL EXAMPLE).
|
||||
#
|
||||
# Manage itself does NOT deploy Grafana/Prometheus/Loki/Alertmanager/Alloy/Node
|
||||
# Exporter. The root docker-compose.yml / docker-compose.dev.yml ship only the
|
||||
# backend and frontend, and Manage connects to EXISTING Grafana/Prometheus/
|
||||
# Alertmanager instances.
|
||||
#
|
||||
# This file is a ready-to-use example you can run on a host where you want the
|
||||
# full Grafana, Prometheus, Loki, Alertmanager, Alloy and Node Exporter stack
|
||||
# without the rest of Manage. It is not required to run Manage.
|
||||
#
|
||||
# Run this on a host where you want Grafana, Prometheus, Loki, Alertmanager,
|
||||
# Alloy and Node Exporter without the rest of Manage.
|
||||
|
||||
+9
-222
@@ -1,3 +1,12 @@
|
||||
# Manage production stack — backend + frontend only.
|
||||
#
|
||||
# Manage does NOT deploy its own Grafana/Prometheus/Loki/Alertmanager stack.
|
||||
# It exposes a /metrics endpoint and optional Alertmanager proxy endpoints so
|
||||
# an *existing* observability deployment can scrape/consume them.
|
||||
#
|
||||
# For a ready-to-use example observability stack, see
|
||||
# docker-compose.observability.yml and docs/observability-runbooks.md.
|
||||
|
||||
services:
|
||||
backend:
|
||||
build:
|
||||
@@ -25,18 +34,12 @@ services:
|
||||
SMTP_USE_SSL: ${SMTP_USE_SSL:-false}
|
||||
SMTP_TIMEOUT: ${SMTP_TIMEOUT:-30}
|
||||
SSH_KNOWN_HOSTS_PATH: /app/backend/.cache/known_hosts
|
||||
PROMETHEUS_FILE_SD_DIR: ${PROMETHEUS_FILE_SD_DIR:-/app/backend/.cache/prometheus-file-sd}
|
||||
ALERTMANAGER_URL: ${ALERTMANAGER_URL:-http://alertmanager:9093}
|
||||
ALERTMANAGER_WEBHOOK_URL: ${ALERTMANAGER_WEBHOOK_URL:-}
|
||||
MANAGE_ENCRYPTION_KEY: ${MANAGE_ENCRYPTION_KEY:?generate one with python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"}
|
||||
volumes:
|
||||
- ${BACKEND_CACHE_DIR:-./backend-cache}:/app/backend/.cache
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- web
|
||||
- monitoring
|
||||
expose:
|
||||
- "8000"
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.${BACKEND_APP_NAME:-manage-backend}.rule=Host(`${BACKEND_APP_HOST:?set BACKEND_APP_HOST}`)"
|
||||
@@ -70,8 +73,6 @@ services:
|
||||
VITE_OIDC_REDIRECT_URI: ${VITE_OIDC_REDIRECT_URI:?set VITE_OIDC_REDIRECT_URI}
|
||||
VITE_OIDC_POST_LOGOUT_REDIRECT_URI: ${VITE_OIDC_POST_LOGOUT_REDIRECT_URI:?set VITE_OIDC_POST_LOGOUT_REDIRECT_URI}
|
||||
VITE_DEV_API_PROXY_TARGET: ${VITE_DEV_API_PROXY_TARGET:-http://backend:8000}
|
||||
VITE_GRAFANA_URL: ${VITE_GRAFANA_URL:-https://grafana.example.com}
|
||||
VITE_PROMETHEUS_URL: ${VITE_PROMETHEUS_URL:-http://localhost:9090}
|
||||
VITE_APP_VERSION: ${APP_VERSION:-0.1.0}
|
||||
VITE_APP_BUILD_INFO: ${APP_BUILD_INFO:-dev}
|
||||
depends_on:
|
||||
@@ -85,222 +86,8 @@ services:
|
||||
- "traefik.http.routers.${FRONTEND_APP_NAME:-manage-frontend}.entrypoints=websecure"
|
||||
- "traefik.http.routers.${FRONTEND_APP_NAME:-manage-frontend}.tls.certresolver=${CERT_RESOLVER:?set CERT_RESOLVER}"
|
||||
- "traefik.http.services.${FRONTEND_APP_NAME:-manage-frontend}.loadbalancer.server.port=${FRONTEND_APP_PORT:-80}"
|
||||
ports:
|
||||
- "8080:80"
|
||||
restart: unless-stopped
|
||||
|
||||
prometheus:
|
||||
image: prom/prometheus:v2.55.1
|
||||
container_name: prometheus
|
||||
command:
|
||||
- "--config.file=/etc/prometheus/prometheus.yml"
|
||||
- "--storage.tsdb.path=/prometheus"
|
||||
- "--storage.tsdb.retention.time=30d"
|
||||
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
|
||||
- "--web.console.templates=/usr/share/prometheus/consoles"
|
||||
- "--web.enable-lifecycle"
|
||||
volumes:
|
||||
- ./monitoring/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
|
||||
- ./monitoring/prometheus/rules:/etc/prometheus/rules:ro
|
||||
- ${BACKEND_CACHE_DIR:-./backend-cache}/prometheus-file-sd:/etc/prometheus/file-sd:ro
|
||||
- prometheus_data:/prometheus
|
||||
expose:
|
||||
- "9090"
|
||||
networks:
|
||||
- web
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9090/-/healthy"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "1.00"
|
||||
memory: 1G
|
||||
reservations:
|
||||
cpus: "0.25"
|
||||
memory: 256M
|
||||
|
||||
loki:
|
||||
image: grafana/loki:3.1.1
|
||||
container_name: loki
|
||||
command: -config.file=/etc/loki/loki.yml
|
||||
volumes:
|
||||
- ./monitoring/loki/loki.yml:/etc/loki/loki.yml:ro
|
||||
- loki_data:/loki
|
||||
expose:
|
||||
- "3100"
|
||||
networks:
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3100/ready"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "1.00"
|
||||
memory: 1G
|
||||
reservations:
|
||||
cpus: "0.25"
|
||||
memory: 256M
|
||||
|
||||
alloy:
|
||||
image: grafana/alloy:v1.5.0
|
||||
container_name: alloy
|
||||
command:
|
||||
- run
|
||||
- /etc/alloy/config.alloy
|
||||
- --storage.path=/var/lib/alloy
|
||||
volumes:
|
||||
- ./monitoring/alloy/config.alloy:/etc/alloy/config.alloy:ro
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
||||
networks:
|
||||
- monitoring
|
||||
depends_on:
|
||||
loki:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
user: root
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:12345/-/healthy"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "0.50"
|
||||
memory: 512M
|
||||
reservations:
|
||||
cpus: "0.10"
|
||||
memory: 128M
|
||||
|
||||
grafana:
|
||||
image: grafana/grafana:11.3.1
|
||||
container_name: grafana
|
||||
environment:
|
||||
GF_SECURITY_ADMIN_USER: ${GRAFANA_ADMIN_USER:-admin}
|
||||
GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD:-admin}
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_ID: ${GF_AUTH_GENERIC_OAUTH_CLIENT_ID:-}
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET:-}
|
||||
GF_AUTH_GENERIC_OAUTH_AUTH_URL: ${GF_AUTH_GENERIC_OAUTH_AUTH_URL:-}
|
||||
GF_AUTH_GENERIC_OAUTH_TOKEN_URL: ${GF_AUTH_GENERIC_OAUTH_TOKEN_URL:-}
|
||||
GF_AUTH_GENERIC_OAUTH_API_URL: ${GF_AUTH_GENERIC_OAUTH_API_URL:-}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||
volumes:
|
||||
- ./monitoring/grafana/grafana.ini:/etc/grafana/grafana.ini:ro
|
||||
- ./monitoring/grafana/provisioning:/etc/grafana/provisioning:ro
|
||||
- grafana_data:/var/lib/grafana
|
||||
expose:
|
||||
- "3000"
|
||||
networks:
|
||||
- web
|
||||
- monitoring
|
||||
depends_on:
|
||||
prometheus:
|
||||
condition: service_healthy
|
||||
loki:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3000/api/health"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.${GRAFANA_APP_NAME:-grafana}.rule=Host(`${GRAFANA_APP_HOST:?set GRAFANA_APP_HOST}`)"
|
||||
- "traefik.http.routers.${GRAFANA_APP_NAME:-grafana}.entrypoints=websecure"
|
||||
- "traefik.http.routers.${GRAFANA_APP_NAME:-grafana}.tls.certresolver=${CERT_RESOLVER:?set CERT_RESOLVER}"
|
||||
- "traefik.http.services.${GRAFANA_APP_NAME:-grafana}.loadbalancer.server.port=${GRAFANA_APP_PORT:-3000}"
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "1.00"
|
||||
memory: 512M
|
||||
reservations:
|
||||
cpus: "0.25"
|
||||
memory: 128M
|
||||
|
||||
alertmanager:
|
||||
image: prom/alertmanager:v0.27.0
|
||||
container_name: alertmanager
|
||||
command:
|
||||
- "--config.file=/etc/alertmanager/alertmanager.yml"
|
||||
- "--storage.path=/alertmanager"
|
||||
environment:
|
||||
SMTP_HOST: ${SMTP_HOST:-smtp.example.com}
|
||||
SMTP_PORT: ${SMTP_PORT:-587}
|
||||
SMTP_USERNAME: ${SMTP_USERNAME:-}
|
||||
SMTP_PASSWORD: ${SMTP_PASSWORD:-}
|
||||
SMTP_FROM_ADDRESS: ${SMTP_FROM_ADDRESS:-no-reply@example.com}
|
||||
ALERT_EMAIL_TO: ${ALERT_EMAIL_TO:-admin@example.com}
|
||||
volumes:
|
||||
- ./monitoring/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
|
||||
- alertmanager_data:/alertmanager
|
||||
expose:
|
||||
- "9093"
|
||||
networks:
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9093/-/healthy"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "0.50"
|
||||
memory: 256M
|
||||
reservations:
|
||||
cpus: "0.10"
|
||||
memory: 64M
|
||||
|
||||
node-exporter:
|
||||
image: prom/node-exporter:v1.8.2
|
||||
container_name: node-exporter
|
||||
command:
|
||||
- "--path.rootfs=/host"
|
||||
volumes:
|
||||
- /:/host:ro,rslave
|
||||
networks:
|
||||
- monitoring
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9100/"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "0.25"
|
||||
memory: 128M
|
||||
reservations:
|
||||
cpus: "0.05"
|
||||
memory: 32M
|
||||
|
||||
networks:
|
||||
web:
|
||||
external: true
|
||||
monitoring:
|
||||
|
||||
volumes:
|
||||
prometheus_data:
|
||||
loki_data:
|
||||
grafana_data:
|
||||
alertmanager_data:
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
# docs (index)
|
||||
dir: docs
|
||||
|
||||
## role
|
||||
Documentation directory containing architecture, planning, and operational reference materials for the project.
|
||||
## parent
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
## children
|
||||
- docs/superpowers
|
||||
index: docs/superpowers/.pi-map.index.md
|
||||
map: docs/superpowers/.pi-map.md
|
||||
## files
|
||||
- MIGRATION_PLAN.md
|
||||
- REQUIREMENTS.md
|
||||
- monitoring-logging-design.md
|
||||
- observability-runbooks.md
|
||||
## links
|
||||
index: docs/.pi-map.index.md
|
||||
map: docs/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,22 @@
|
||||
# docs
|
||||
dir: docs
|
||||
|
||||
index: docs/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Documentation directory containing architecture, planning, and operational reference materials for the project.
|
||||
## files
|
||||
- MIGRATION_PLAN.md | This file documents the architecture, API design, and step-by-step migration plan for transitioning an application from a Streamlit monolith to a FastAPI and React SPA. | dep: FastAPI, React, Vite, TypeScript, pydantic-settings, @tanstack/react-query, ag-grid-react, recharts, tailwindcss
|
||||
- REQUIREMENTS.md | This file is a living requirements and decision log detailing the product goals, architecture, and feature specifications for a web application that manages a remote Jellyfin media library and inspects server files over SSH.
|
||||
- monitoring-logging-design.md | Design document detailing a self-hosted observability architecture (metrics, logs, dashboards, alerting) for integration with a platform called Manage. | dep: Prometheus, Grafana, Node Exporter, Grafana Loki, Grafana Alloy, Alertmanager, Authentik, Traefik
|
||||
- observability-runbooks.md | Provides operational runbooks, configuration, and maintenance procedures for deploying and managing a standalone observability stack. | dep: Prometheus, Grafana, Loki, Alloy, Alertmanager, Node Exporter, Docker Compose, Traefik
|
||||
## arch
|
||||
Flat collection of standalone Markdown documents covering requirements tracking, migration planning, and observability/operational runbooks.
|
||||
## tags
|
||||
design, react, observability, architecture, migration, plan, requirements, runbooks
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -92,7 +92,7 @@ repo/
|
||||
│ └── types/
|
||||
│ └── index.ts
|
||||
│
|
||||
├── docker-compose.yml # Optional unified deployment
|
||||
├── docker-compose.yml # Backend + frontend deployment (observability is external)
|
||||
└── README.md
|
||||
```
|
||||
|
||||
|
||||
+70
-3
@@ -197,7 +197,7 @@ fully removed (web-ui-rework; see decision log 2026-06-17).
|
||||
- Dashboard shortcuts should support a small icon/preview field so cards can be visually recognizable without changing the underlying model later.
|
||||
- Support OIDC login in the frontend using an OIDC client library, with backend JWT validation for protected API requests.
|
||||
- Persist frontend OIDC auth state across tab reloads by storing the OIDC user and request state in browser localStorage.
|
||||
- Provide Docker Compose deployment files at the repository root for production and local development.
|
||||
- Provide Docker Compose deployment files at the repository root for production and local development. These deploy **only** the backend and frontend; Manage connects to *existing* Grafana/Prometheus/Alertmanager instances and never ships its own observability stack (see `docker-compose.observability.yml` for an optional standalone example).
|
||||
- SSH private keys should be managed as reusable saved secrets in Settings, independent of any one machine, and SSH machines should select from that saved-key list.
|
||||
- The web UI should allow both importing an existing private key and generating a new SSH keypair for that saved-key list.
|
||||
- Saved SSH keys should display their derived public key, fingerprint, and machine usage count so administrators can audit them at a glance.
|
||||
@@ -249,8 +249,8 @@ fully removed (web-ui-rework; see decision log 2026-06-17).
|
||||
|
||||
- Provide an Actions tab for predefined server tasks that users can save and run later.
|
||||
- The initial task types should support shell commands and Python scripts, while keeping the design flexible for future task types.
|
||||
- Avoid arbitrary free-form command execution for ad-hoc execution; tasks should be stored records with an explicit name, type, content, enabled flag, default machine, and notes.
|
||||
- Support running tasks against either the local API host or a configured SSH machine using the same machine registry used by Monitoring.
|
||||
- Avoid arbitrary free-form command execution for ad-hoc execution; tasks should be stored records with an explicit name, type, content, enabled flag, default SSH task service, and notes.
|
||||
- Support running tasks against `ssh_tasks` service instances only; local execution on the API host is no longer supported.
|
||||
- Command/script content should be executed through the existing safe process helpers and shell-quoted where applicable.
|
||||
- Future destructive actions should require explicit confirmations or dry-run style safeguards.
|
||||
- Job templates should remain centralized in `jobs.py` for future extension.
|
||||
@@ -271,6 +271,10 @@ Service definitions live as Pydantic modules in the backend
|
||||
(`integrations/`); they declare the service config schema, secret fields, and
|
||||
the widget kinds the service provides. There is no runtime plugin loading.
|
||||
|
||||
Every service `base_url` uses the shared `ServiceBaseUrl` type, which rejects
|
||||
values missing an `http://` or `https://` schema with a clear validation error
|
||||
(relative hosts break downstream HTTP clients).
|
||||
|
||||
### Services
|
||||
|
||||
- **Grafana** — base URL + optional API key; provides a dashboard-link widget.
|
||||
@@ -328,9 +332,18 @@ the widget/addon-pages model were removed. `MANAGE_ENCRYPTION_KEY` is now requir
|
||||
> config still powers the Media/Users/Files pages. Migrating those onto the
|
||||
> service registry (and removing the machine app fields) is a separate change;
|
||||
> see `openspec/changes/service-registry/design.md` §12.5.
|
||||
>
|
||||
> **Update (2026-06-23):** this follow-up is now complete — Jellyfin is no
|
||||
> longer a machine service, and the dead machine-level `media_root`/`path_prefix`
|
||||
> fields were removed. Jellyfin is configured exclusively via the service
|
||||
> registry. The global `REMOTE_MEDIA_ROOT`/`REMOTE_PATH_PREFIX` config and
|
||||
> `path_utils.py` remain for Jellyfin→SSH path resolution.
|
||||
|
||||
## Decision Log
|
||||
|
||||
- 2026-06-23: Completed the Jellyfin migration onto the service registry. Jellyfin is no longer a machine `services` tag (`DEFAULT_SERVICES` is now `["monitoring", "files"]`), and the dead machine-level `media_root`/`path_prefix` fields were removed from the settings store, `MonitoringMachineInput`, frontend types, and the Settings UI. Jellyfin is configured exclusively as a service-registry instance. The global `REMOTE_MEDIA_ROOT`/`REMOTE_PATH_PREFIX` config properties and `path_utils.py` are unchanged — `routers/files.py` and the media index still use them for Jellyfin→SSH path resolution. See `openspec/changes/jellyfin-service-registry/`.
|
||||
- 2026-06-23: Made all observability integration service-registry first-class with zero service env config. Alertmanager is now a 6th service type (`integrations/alertmanager.py`) with an `active_alerts` widget, mirroring Grafana/Prometheus. The monitoring router resolves alertmanager/grafana/prometheus from service records (first-enabled-instance default, optional `?service_id=`); added `/api/monitoring/grafana-status` and `/prometheus-status` health probes. The Observability page discovers services from the registry (dropped `import.meta.env.VITE_GRAFANA_URL`) and renders health cards; the dashboard gained an `active_alerts` widget. Removed the backend `alertmanager_url`/`alertmanager_webhook_url` fields and the frontend `VITE_GRAFANA_URL`/`VITE_PROMETHEUS_URL` build args — observability services are configured in the UI only. Removed the `PROMETHEUS_FILE_SD_DIR` shared-file bridge + `write_prometheus_targets` file writer; external Prometheus now uses `http_sd_configs` against `GET /api/monitoring/prometheus-targets` (`build_node_exporter_targets` + that endpoint stay). The webhook receiver is log-only (dropped the outbound forward). The only observability env var remaining is `PROMETHEUS_ENABLED` (Manage's own `/metrics` toggle). See `openspec/changes/observability-service-registry/`.
|
||||
- 2026-06-23: Removed all observability services from the root Compose files. `docker-compose.yml` and `docker-compose.dev.yml` now deploy **only** the backend and frontend; the `monitoring` network and the `prometheus`/`loki`/`alloy`/`grafana`/`alertmanager`/`node-exporter` services and their named volumes were deleted, and the `GRAFANA_APP_HOST` Traefik rule was removed. Manage connects to **existing** Grafana/Prometheus/Alertmanager instances and does not deploy them itself. The backend `alertmanager_url` default is now empty, and the `/api/monitoring/alerts` and `/alertmanager-status` endpoints return graceful "not configured" responses when `ALERTMANAGER_URL` is unset. The previous in-compose stack is preserved as an optional standalone example in `docker-compose.observability.yml` (config under `monitoring/`, documented in `docs/observability-runbooks.md`). (The `VITE_GRAFANA_URL`/`VITE_PROMETHEUS_URL` overrides were later removed entirely — see the service-registry entry above.)
|
||||
- 2026-06-17: Decommissioned the legacy Manage-side system-metric scraping. Removed the backend `MonitoringPoller` (SSH-ran `df` on every machine every 5 min into a local SQLite `monitoring_machine_actions` table), the entire `services/monitoring_actions.py` module, the `/api/monitoring/poller`, `/api/monitoring/machines/{id}/actions`, and `/api/monitoring/disk` endpoints, the `monitoring_machine_actions` table (DROP on startup), the three `monitoring_poll_*` / `monitoring_action_retention_days` config knobs, and the orphaned frontend `DiskSpaceCard` + `DiskSpace` type. System metrics are now owned exclusively by Prometheus + node_exporter + Grafana. Kept the Alertmanager proxy (`/alerts`, `/alertmanager-status`, `/alertmanager-webhook`), `/prometheus-targets`, `/machines`, the `node_exporter_*` machine fields, and the on-demand `disk_usage` job template.
|
||||
- 2026-06-17: Completed the web UI rework to a single design system. The frontend now uses **shadcn/ui + Tailwind CSS v4 + lucide-react** exclusively, with CSS `@theme` tokens in `src/index.css` (primary `#4f8cff`; `chart-1..5` repurposed as status/Grafana-link cues). Removed `@mui/material`, `@mui/icons-material`, `@mui/x-data-grid`, `@emotion/react`, `@emotion/styled`, `recharts`, `d3`, and the `src/theme.ts` shim. Tables moved from `@mui/x-data-grid`/AG Grid to a visibility-only TanStack `DataTable` wrapper (pagination, row selection, row click, column visibility — no sorting/resizing). Adopted the thin-dashboard observability model (no in-app charts; Alertmanager alerts + Prometheus target health + Grafana deep-links). Reconciled the information architecture: Backups is a top-level nav item at `/backups`, and the media surface is named Media at `/media` with `/applications` redirecting to `/media` (mirroring `/monitoring` → `/observability`). Frontend tests moved to Vitest + @testing-library/react (`npm test`), with legacy node suites in `frontend/tests`.
|
||||
- 2026-06-13: Adopted a dedicated, self-hosted observability subsystem based on Prometheus, Grafana Loki, Grafana, and Alertmanager. Metrics will be pulled from Node Exporter on machines and from application exporters in containers; logs will be structured JSON shipped by Promtail/Grafana Alloy. The existing POSIX remote collector will be removed and backup alerts migrated to Alertmanager rules. See `docs/monitoring-logging-design.md`.
|
||||
@@ -455,3 +468,57 @@ The system receives backup execution reports from an external backup tool via HT
|
||||
|
||||
- Backup tool uses auto-generated Bearer API key
|
||||
- Frontend uses existing OIDC/JWT auth
|
||||
|
||||
## Mobile Responsive Design
|
||||
|
||||
The frontend is fully operable in phone portrait (≥360px) at a single `md:`
|
||||
(768px) breakpoint. Tablets and wider viewports use the desktop layout
|
||||
unchanged.
|
||||
|
||||
### Breakpoint policy
|
||||
|
||||
- Single responsive cut: `md:` (768px). Below is "mobile"; at-or-above is
|
||||
"desktop" (existing layout, unchanged).
|
||||
- `useIsMobile()` hook (`frontend/src/hooks/useIsMobile.ts`) is the single
|
||||
source of truth; it wraps `matchMedia("(max-width: 768px)")` and is SSR-safe.
|
||||
- No `sm:` intermediate cut. No PWA, manifest, or service worker.
|
||||
|
||||
### Data tables (hybrid)
|
||||
|
||||
- The four wide tables (Media, FileBrowser, Users, Backups) render stacked
|
||||
**cards per row** below `md` via `MobileCardRow`, each showing a primary
|
||||
title plus 3–5 key fields. Narrow tables (SessionActivity) keep horizontal
|
||||
scroll. The TanStack column-visibility toggle is hidden below `md`.
|
||||
- At `md:` and above, all tables render as the existing `<DataTable>` unchanged.
|
||||
|
||||
### Edit forms (Sheet)
|
||||
|
||||
- Below `md`, ServicePage, Settings (machine editor), message compose, and
|
||||
WidgetConfigDialog open inside a full-height `SheetForm` (side=bottom,
|
||||
`h-[100dvh]`) with sticky header + sticky save bar instead of a centered
|
||||
Dialog.
|
||||
- At `md:` and above, the existing Dialog-based forms are unchanged.
|
||||
|
||||
### Touch targets
|
||||
|
||||
- All interactive elements below `md` have a minimum 44×44px hit area via the
|
||||
`.mobile-touch-target` CSS utility (applied only below 768px). This covers
|
||||
icon buttons, checkboxes, switches, and small text buttons. The class is a
|
||||
no-op at `md:` and above.
|
||||
|
||||
### Dashboard
|
||||
|
||||
- Below `md`, the widget grid collapses to a single column with a section
|
||||
anchor bar (Observability / Media / Backups / Custom) for quick navigation.
|
||||
- At `md:` and above, the existing multi-widget grid is unchanged.
|
||||
|
||||
### Polling
|
||||
|
||||
- Widget refresh intervals and the message-queue poll interval are identical
|
||||
on mobile and desktop. A follow-up to pause refetch when the tab is hidden
|
||||
(`document.visibilityState`) is tracked as a future battery optimization.
|
||||
|
||||
### `HoverEditButton`
|
||||
|
||||
- Below `md`, edit affordances are always visible (not hover-gated). At `md:`
|
||||
and above, the desktop hover-reveal aesthetic is preserved.
|
||||
|
||||
@@ -1,5 +1,16 @@
|
||||
# Monitoring and Logging Design — Manage
|
||||
|
||||
> **Status (2026-06-23):** This document is the **reference architecture** for the
|
||||
> self-hosted observability stack Manage integrates with. Manage itself **does
|
||||
> not deploy** Grafana, Prometheus, Loki, Alertmanager, Alloy, or Node Exporter.
|
||||
> The root `docker-compose.yml` / `docker-compose.dev.yml` ship **only** the
|
||||
> backend and frontend; Manage connects to **existing** Grafana / Prometheus /
|
||||
> Alertmanager instances. The stack described here is available as an optional,
|
||||
> deploy-it-yourself example in `docker-compose.observability.yml` (see
|
||||
> `docs/observability-runbooks.md`). The "Add services to `docker-compose.yml`"
|
||||
> phase notes below describe the **original** in-compose deployment, which has
|
||||
> since been extracted to the standalone compose file.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Manage currently uses ad-hoc observability: plain-text Python logs, a custom POSIX shell metrics collector on remote machines, and a background poller that stores snapshots in SQLite. This works for a single-instance homelab but becomes painful as the fleet grows and as users need faster incident response.
|
||||
@@ -296,6 +307,17 @@ The existing POSIX remote collector will be removed, and the Python backup alert
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
> **Superseded mechanisms (2026-06-23):** Phases 2 and 3 originally wired Manage
|
||||
> to a co-deployed Prometheus/Alertmanager via a shared-file **file-SD bridge**
|
||||
> (`PROMETHEUS_FILE_SD_DIR` + `write_prometheus_targets`) and an
|
||||
> `alertmanager_url` env var. Both have been removed. Manage now resolves
|
||||
> Alertmanager/Grafana/Prometheus from the **service registry**, and external
|
||||
> Prometheus instances consume node-exporter targets via **`http_sd_configs`**
|
||||
> against `GET /api/monitoring/prometheus-targets`. Treat the Phase 2/3
|
||||
> file/config details below as the historical implementation, not the current
|
||||
> wiring. See `openspec/changes/observability-service-registry/` and the
|
||||
> decision log in `docs/REQUIREMENTS.md`.
|
||||
|
||||
### Phase 0 — Foundation and Cleanup
|
||||
|
||||
- [x] Add `prometheus-client` and `python-json-logger` to `backend/pyproject.toml`.
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
# Observability Runbooks
|
||||
|
||||
Operational playbooks for the Manage self-hosted observability stack (Prometheus, Grafana, Loki, Alertmanager).
|
||||
Operational playbooks for the **standalone example observability stack**
|
||||
(`docker-compose.observability.yml`) that can be deployed alongside Manage.
|
||||
Manage itself does **not** deploy these services; it connects to existing
|
||||
Grafana / Prometheus / Alertmanager instances. These runbooks cover operating
|
||||
the standalone stack shipped under `monitoring/`.
|
||||
|
||||
## Service Overview
|
||||
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# docs/superpowers (index)
|
||||
dir: docs/superpowers
|
||||
|
||||
## role
|
||||
Documentation directory for advanced features, plugins, or capabilities (currently empty).
|
||||
## parent
|
||||
index: docs/.pi-map.index.md
|
||||
map: docs/.pi-map.md
|
||||
## children
|
||||
- docs/superpowers/plans
|
||||
index: docs/superpowers/plans/.pi-map.index.md
|
||||
map: docs/superpowers/plans/.pi-map.md
|
||||
- docs/superpowers/specs
|
||||
index: docs/superpowers/specs/.pi-map.index.md
|
||||
map: docs/superpowers/specs/.pi-map.md
|
||||
## files
|
||||
## links
|
||||
index: docs/superpowers/.pi-map.index.md
|
||||
map: docs/superpowers/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user