manage/docs/REQUIREMENTS.md

# Media Library Viewer - Requirements and Decision Log

This is a living document for the project. Update it whenever requirements, UX expectations, architecture decisions, constraints, or implementation plans change.

## Product Goal

Build a compact Streamlit application for browsing a remote Jellyfin media library and inspecting the corresponding media files on disk over SSH. The app should support library metadata review, direct file-system navigation, detailed media metadata inspection, and safe remote maintenance/job workflows.

## Current Phase

Phase 1: Jellyfin library browser plus SSH-based remote filesystem inspection and safe job templates.

## Core Requirements

### Jellyfin Library

- Connect to a remote Jellyfin server using an API key.
- Use the Jellyfin server root URL, not the `/web` UI URL.
- Handle API-key auth correctly:
  - use `GET /Users` to list available users;
  - allow a manual `JELLYFIN_USER_ID` override;
  - do not rely on `/Users/Me` for API-key auth.
- List Jellyfin libraries for the selected user.
- Browse library items with search, pagination, media type filtering, and poster cards.
- Show item details including overview, genres, ratings where available, file path, media sources/streams, and raw Jellyfin JSON.
- Provide a Media tab with a paginated inventory table for large libraries.
- Media inventory should show title, series name, season, episode number, type, year, runtime, file size, bitrate, explicit HDR yes/no flag, video codec, resolution, date added, and path where available.
- Media inventory should use Jellyfin metadata only for now; full ffprobe enrichment for all media should be deferred to a cached/background scan to avoid expensive per-item SSH probing.
- Media inventory should support multi-library selection, type, search, full-index sort/order, HDR filter, page size, and page controls.
- Media inventory should use a local SQLite index so sorting/filtering by nested/derived fields such as size, bitrate, HDR, codec, resolution, series, season, and episode can apply to the whole indexed library instead of only one Jellyfin page.
- Media inventory table should be read-only; full-index sorting/filtering should be handled by the service/query layer rather than relying on frontend table sorting.
- Media inventory should use row-based table selection and automatically sync the File browser tab to the selected row's containing directory.
- Media row selection should update both Media selected-row state and File browser location without requiring an extra action button.
- File browser interaction should stay explicit and simple: read-only listing plus explicit Open/Select actions rather than another row-selection grid.
- Use valid Jellyfin `Fields` query values only, because invalid field names can cause `400 Bad Request` responses.

### Remote Filesystem over SSH

- Connect to a remote media server via SSH.
- Use strict SSH host key behavior; users should connect manually once to populate `known_hosts`.
- Browse remote directories and files rooted at a configurable default media path.
- File browser handoff should map Jellyfin paths to `REMOTE_MEDIA_ROOT` when possible (for example `/media/...` -> `/srv/media/...` when root is `/srv/media`).
- Support a configurable Jellyfin-to-SSH fallback path prefix for cases where `REMOTE_MEDIA_ROOT` mapping alone is not sufficient.
- Support manual path entry and refresh.
- Remote file listing must be compact, structured, and navigable.
- The file table should be read-only.
- The file table should use row selection (single-select) in an AG Grid format consistent with the Media tab.
- The file table should not expose a visible checkbox selection column.
- The file table should not show a visible `selected` column.
- Include a top `[UP] ..` row, when not at `/`, to navigate to the parent directory.
- The selected path should be visibly shown outside the table.
- Selecting a directory row (including `[UP] ..`) should open it immediately.
- Selecting a file row should set the selected file target for metadata/jobs.
- Avoid emojis and hard-to-render characters in labels.

### Remote Listing Controls

- Show compact listing summary information:
  - total entries;
  - directories;
  - files;
  - total file size for files in the current directory.
- Support filtering by:
  - all entries;
  - directories;
  - files;
  - file extension.
- Support case-insensitive filename search.
- Support sorting by:
  - name;
  - kind/type;
  - size;
  - modified time.
- Support ascending/descending sort order.
- Support pagination and configurable rows per page.

### Disk-Level Metadata

- Run `ffprobe` on selected remote files to inspect authoritative media metadata directly from disk.
- When a known video/movie file is selected in the remote file browser, automatically run a blocking `ffprobe` preview and show a spinner while it completes.
- Cache preview results briefly to keep repeated Streamlit reruns responsive; allow users to reload the preview manually.
- Display `ffprobe` results in separate sections instead of one sparse all-streams table:
  - container/format summary;
  - video streams;
  - audio streams;
  - subtitle streams.
- Video metadata should include codec, profile, resolution, pixel format, bitrate, frame rate, color range/space/transfer/primaries, side data/HDR-related metadata where available, language, title, and default flag.
- Audio metadata should include codec, profile, channels/layout, sample rate, bitrate, language, title, default, and forced flags.
- Subtitle metadata should include codec, language, title, default, forced, and hearing-impaired flags where available.
- Expose raw `ffprobe` JSON for deeper inspection.
- Keep a manual `ffprobe` action available for selected paths.
- Support `stat` on selected paths.

### Dashboard / Server Resources

- Provide a dashboard tab with a compact Jellyfin media library overview and server resource overview.
- Show Jellyfin media counts for movies, series, and series episodes on the dashboard.
- Show currently playing Jellyfin sessions on the dashboard, including user, media title, playback state, and whether transcoding is active.
- Provide a dashboard tab with a compact server resource overview over SSH.
- Provide a separate Resources tab for detailed resource charts, collector controls, diagnostics, and raw samples.
- Show CPU and RAM usage for the last hour.
- Show average and spike/peak values for network throughput and disk I/O.
- Show used, available, and total disk space for the configured media root, falling back to `/`.
- Use a lightweight remote collector that reads Linux `/proc`, `/sys/block`, and `df` data into a JSONL file under `/tmp`.
- The collector should rotate/prune its JSONL metrics file so it does not grow unbounded; default retention is 7 days with a 70,000-line safety cap.
- The collector should be startable/stoppable/restartable from the dashboard and should not require installing a full monitoring stack.
- Last-hour charts require the collector to have been running long enough to collect samples.
- Network throughput should be shown split into down/download and up/upload.
- Network throughput should use bytes-per-second display units such as KB/s, MB/s, and GB/s to avoid bit/byte ambiguity.
- Disk throughput should be shown split into read and write.
- Network and disk throughput charts should scale values into readable units such as KB/s, MB/s, and GB/s.

### Remote Jobs

- Support running remote jobs over SSH using explicit templates.
- Phase 1 jobs should be safe/read-only by default.
- Avoid arbitrary free-form command execution in the UI.
- Job templates should be centralized in `jobs.py` for future extension.
- Command template values must be shell-quoted before execution.
- Future destructive jobs should require explicit confirmation.

## Current Architecture

- `app.py` - Thin root Streamlit entrypoint for `streamlit run app.py`.
- `pyproject.toml` - Authoritative package metadata, dependencies, and tool configuration.
- `requirements.txt` - Convenience install file that installs the local package editable.
- `src/media_library_viewer/app.py` - Thin Streamlit orchestration layer.
- `src/media_library_viewer/ui/` - Streamlit UI modules split by feature area (dashboard, media, file browser, library, preview/tools).
- `src/media_library_viewer/clients/jellyfin.py` - Jellyfin API wrapper.
- `src/media_library_viewer/clients/ssh.py` - SSH command execution, directory listing, `stat`, and `ffprobe` helpers.
- `src/media_library_viewer/domain/` - UI-independent normalization/domain helpers.
- `src/media_library_viewer/services/` - UI-independent application services such as the SQLite media index.
- `src/media_library_viewer/jobs.py` - Remote job template definitions and runner.
- `src/media_library_viewer/utils.py` - Formatting and media metadata summarization helpers.
- `src/media_library_viewer/config.py` - Environment variable and `.env` configuration loading.
- `docs/REQUIREMENTS.md` - Living requirements and decision log.
- `tests/` - Reserved for future test coverage.

## Key Implementation Decisions

- Prefer the Jellyfin API for library and server metadata.
- Prefer SSH plus `ffprobe` for disk-authoritative stream/container metadata.
- Use API-key auth for Jellyfin, but select a user explicitly for user-scoped endpoints.
- Use `streamlit-aggrid` as a required dependency for Media table row selection. Avoid optional frontend fallbacks that create multiple interaction models.
- Keep remote jobs template-based to reduce accidental destructive actions.
- Keep the Phase 1 UI compact and structured rather than using large per-row buttons.
- Use a `src/` package layout so the project can grow without accumulating many root-level modules.
- Keep root `app.py` as a compatibility/convenience wrapper for Streamlit.
- Keep clients, domain normalization, and application services independent from Streamlit so the frontend can later be replaced by React/FastAPI or another UI.
- Keep Streamlit rendering split into small UI modules so interaction bugs can be debugged in feature-local code instead of one monolithic app file.

## Security and Safety Requirements

- Do not hardcode secrets.
- Use `.env`, environment variables, or Streamlit secrets for credentials.
- Keep `.env` and Streamlit secrets out of version control.
- Reject unknown SSH host keys by default.
- Treat SSH jobs as potentially dangerous and keep them explicit/template-based.
- Add confirmation steps before implementing cleanup, delete, transcode-replace, or other destructive workflows.

## Configuration Requirements

Supported environment variables:

```bash
JELLYFIN_URL=
JELLYFIN_API_KEY=
JELLYFIN_USER_ID=
SSH_HOST=
SSH_USERNAME=
SSH_PORT=22
SSH_KEY_FILENAME=
SSH_PASSWORD=
REMOTE_MEDIA_ROOT=
REMOTE_PATH_PREFIX=
```

## Known External Requirements

Remote server should have:

- Linux `/proc` and `/sys/block` for resource metrics
- `/bin/sh` for POSIX command execution, even when the user's login shell is fish or another non-POSIX shell
- POSIX shell utilities including `awk`, `date`, `tail`, `df`, `kill`, and `nohup`
- `python3`
- GNU/coreutils-compatible `find` and `stat`
- `ffprobe` for media metadata inspection

Local app dependencies are declared in `pyproject.toml`; `requirements.txt` installs the package editable for convenience. Runtime dependencies include:

- `streamlit`
- `streamlit-aggrid`
- `requests`
- `paramiko`
- `python-dotenv`
- `pandas`

## Backlog / Future Extensions

- Add transcode job templates.
- Add cleanup job templates with dry-run and explicit confirmation.
- Add subtitle/audio-track diagnostics.
- Add sidecar file inspection for `.nfo`, `.srt`, images, and metadata files.
- Compare Jellyfin metadata against disk metadata and sidecars.
- Add long-running job tracking/log streaming.
- Add saved presets for common media roots and job templates.
- Add file previews for text sidecars.
- Add richer HDR/Dolby Vision/bit-depth summaries from `ffprobe`.
- Add optional integration with existing monitoring stacks such as Prometheus/node_exporter, Netdata, or sysstat/sar.

## Decision Log

### 2026-04-30 - Initial app plan

- Planned a Streamlit app that uses the Jellyfin API as the primary metadata source.
- Decided SSH should be used for disk inspection and future maintenance jobs.

### 2026-04-30 - Phase 1 implementation

- Created the initial app structure with Jellyfin, SSH, jobs, config, and utility modules.
- Added safe/read-only remote job templates.
- Added `ffprobe` and `stat` inspection.

### 2026-04-30 - Jellyfin API fixes

- Replaced `/Users/Me` usage with `GET /Users` plus user selection.
- Added `JELLYFIN_USER_ID` override.
- Cleaned Jellyfin `Fields` values to avoid 400 responses.
- Added defensive stripping of trailing `/web` from Jellyfin URLs.

### 2026-04-30 - Remote file browser evolution

- Added interactive remote listing.
- Removed emoji and hard-to-render characters.
- Added search, filtering, sorting, pagination, and compact listing summary.
- Reworked listing from large button rows into a compact table.
- Switched to `streamlit-aggrid` for file-browser-like row click behavior.
- Removed visible checkbox/selection column behavior.
- Added `[UP] ..` top row for parent directory navigation.

### 2026-04-30 - Selected-file metadata preview

- Added a requirement for automatic `ffprobe` preview when known video files are selected.
- Initially explored asynchronous/non-blocking preview, then changed to a blocking call with a spinner because it is more streamlined for this app.
- Decided to cache preview results briefly and provide a manual reload action.
- Decided `ffprobe` output should be separated into container, video, audio, and subtitle sections to avoid sparse mixed-stream tables.

### 2026-04-30 - Process requirement

- Added this living requirements and decision log document.
- Added a global agent skill to encourage maintaining such a document for future projects.

### 2026-04-30 - Repository restructuring

- Restructured the project into a larger-project-ready `src/media_library_viewer/` package layout.
- Kept a thin root `app.py` entrypoint so `streamlit run app.py` remains the primary launch command.
- Moved service clients into `src/media_library_viewer/clients/`.
- Added `pyproject.toml` with runtime dependencies, development extras, Ruff configuration, and pytest configuration.
- Simplified `requirements.txt` to install the local project editable.
- Expanded `.gitignore` for Python caches, build artifacts, virtual environments, local secrets, editor files, and logs.

### 2026-04-30 - Resource dashboard

- Added a dashboard requirement for CPU, RAM, network, disk I/O, and disk space overview.
- Decided that true last-hour metrics require collection over time; implemented a lightweight SSH-started remote collector instead of requiring Prometheus, Netdata, or sysstat.
- The collector stores JSONL samples in `/tmp` and can be started/stopped from the dashboard.
- Charts show the last hour of collected samples; the dashboard becomes more useful once the collector has been running for a while.
- Last-hour filtering uses epoch seconds rather than local naive datetimes to avoid timezone-offset issues between the app host and remote sample timestamps.
- Fixed SSH command execution to explicitly use `/bin/sh -c` so POSIX resource commands work even when the remote user's login shell is fish.
- Added explicit Streamlit keys to dashboard/file/tool buttons to avoid duplicate auto-generated element IDs as the UI grows.
- Changed the resource collector script from bash-specific syntax to POSIX `/bin/sh` syntax and added dashboard diagnostics/restart controls for collector troubleshooting.
- Added 7-day metrics file pruning plus a 70,000-line safety cap to prevent the JSONL file from growing without bound.
- Split network charts and metrics into download and upload, and disk charts and metrics into read and write.
- Changed network display units from bits per second to bytes per second to avoid Kbps/KB/s ambiguity; the collector still stores bit-rate compatibility fields for old/debug consumers.
- Scaled network and disk throughput charts into readable units such as KB/s, MB/s, and GB/s instead of plotting raw base units.
- Updated collector startup to remove old temporary metrics/log files when a new collector process is started after a schema/display change.
- Moved detailed resource charts, raw samples, diagnostics, and collector controls into a dedicated Resources tab; the Dashboard now keeps a compact overview.
- Removed the CPU/RAM chart from the Dashboard and kept detailed charts in the Resources tab.
- Renamed the remote files tab to File browser.
- Added Jellyfin media counts for movies, series, and episodes to the Dashboard using lightweight count queries.
- Added a Dashboard now-playing section sourced from Jellyfin sessions, showing who is currently playing what and whether each session is transcoding.

### 2026-04-30 - Media inventory tab

- Added a paginated Media tab for file-oriented Jellyfin metadata.
- Decided not to fetch all media at once because large libraries can make API responses and Streamlit rendering slow.
- Decided to derive length, size, bitrate, HDR flag, date added, codec, and resolution from Jellyfin metadata for now.
- Added series name, season, and episode number for episode rows.
- Added server-side sort/order controls and read-only AG Grid column sorting/filtering for the loaded page.
- Reworked the Media tab to use a local SQLite media index for full-library sorting/filtering, including numeric sorting for size and bitrate.
- Added last index build duration metadata to the Media tab status line.
- Replaced single-library selection with multi-library selection so users can include/exclude multiple libraries in the indexed table.
- Changed HDR display from blank/no-value to explicit yes/no.
- Added row selection plus an Open folder action in the Media tab that sets the File browser to the containing folder.
- Restored row-based Media table selection while keeping File browser state changes limited to the explicit Open folder button.
- Updated Media tab behavior so selecting a row automatically syncs the File browser folder to that item's containing directory; removed the extra Open folder button step.
- Restored File browser table row selection with AG Grid (single-select), using a table interaction style consistent with the Media tab.
- Reintroduced open-on-select behavior in File browser: selecting a directory row (including `[UP] ..`) opens it immediately, while file rows update selected target path.
- Refined Media table column presentation with explicit user-friendly headers and null-safe display formatting to keep the grid readable and consistent.
- Added broad inline/module documentation across clients, domain, services, and Streamlit adapter modules to make debugging and future frontend extraction easier.
- Simplified the File browser by removing its interactive AG Grid and using a read-only listing with explicit Open/Select controls, reducing cross-tab state interactions with the Media grid.
- Removed optional/compatibility code paths around the Media table grid and old file-browser state aliases to keep the interaction model easier to reason about during debugging.
- Split the Streamlit frontend into dedicated UI modules (dashboard, media, file browser, library, preview/tools) and reduced `app.py` to orchestration glue.
- Reviewed remote path handling and kept shell interactions routed through quoted paths (`shlex.quote`) while UI/path-parent operations use POSIX path handling, preserving paths with spaces.
- Fixed file browser handoff/navigation to reset stale search/filter/page state when changing folders, preventing old filters from hiding all entries in the newly opened folder.
- Reworked file browser state to separate current directory from selected path. Selecting a file no longer changes the directory being listed, while opening a folder updates the current directory and keeps path input synchronized.
- Made remote directory listing fail explicitly when the current path is not a directory and recover by listing the parent, preventing file paths from appearing as empty directories.
- Made File browser Refresh/Select folder apply a manually typed path if it differs from the current folder, reducing confusion when manually navigating.
- Moved media normalization into `domain/media.py` and index/query logic into `services/media_index.py` to make the project less Streamlit-specific and easier to expose through a future API/React frontend.
- Deferred full ffprobe enrichment for every item to a future cached/background scan.
- Fixed network byte parsing to split `/proc/net/dev` lines at the colon first, so interface indentation differences do not shift fields and accidentally report packet counts instead of byte counts.
- Fixed a follow-up `/proc/net/dev` parsing issue where leading whitespace after the colon could produce an empty first split field in some `awk` implementations, resulting in zero network rates. Added `/proc/net/dev` snapshots to collector diagnostics.
- Simplified File browser directory error behavior: stopped automatic parent-directory fallback and now show the direct listing error for the current path.
- Added configurable `REMOTE_PATH_PREFIX` support so Jellyfin paths can be mapped to SSH-visible paths when opening folders from Media/Library tabs (for example `/media/...` -> `/srv/media/...`).
- Updated path handoff logic to prefer mapping through `REMOTE_MEDIA_ROOT` (anchor replacement using the root basename, e.g. `media`) and use `REMOTE_PATH_PREFIX` as fallback.
- Added a public-repo readiness note in README describing what local/sensitive files must stay out of version control.
- Added `LICENSE` (MIT) and `CONTRIBUTING.md` for public-repo baseline documentation.