manage/docs/REQUIREMENTS.md

Media Library Viewer - Requirements and Decision Log

This is a living document for the project. Update it whenever requirements, UX expectations, architecture decisions, constraints, or implementation plans change.

Product Goal

Build a compact Streamlit application for browsing a remote Jellyfin media library and inspecting the corresponding media files on disk over SSH. The app should support library metadata review, direct file-system navigation, detailed media metadata inspection, and safe remote maintenance/job workflows.

Current Phase

Phase 1: Jellyfin media index, SSH-based remote filesystem inspection, server monitoring, and safe job templates.

Core Requirements

Jellyfin Library

• Connect to a remote Jellyfin server using an API key.
• Use the Jellyfin server root URL, not the /web UI URL.
• Handle API-key auth correctly:
  • use GET /Users to list available users;
  • allow a manual JELLYFIN_USER_ID override;
  • do not rely on /Users/Me for API-key auth.
• List Jellyfin libraries for the selected user.
• Browse library items with search, pagination, media type filtering, and poster cards.
• Show item details including overview, genres, ratings where available, file path, media sources/streams, and raw Jellyfin JSON.
• Provide a Media tab with a paginated inventory table for large libraries.
• Media inventory should show title, series name, season, episode number, type, year, runtime, file size, bitrate, explicit HDR yes/no flag, video codec, resolution, date added, and path where available.
• Media inventory should use Jellyfin metadata only for now; full ffprobe enrichment for all media should be deferred to a cached/background scan to avoid expensive per-item SSH probing.
• Media inventory should support multi-library selection, type, search, full-index sort/order, HDR filter, page size, and page controls.
• Media inventory should use a local SQLite index so sorting/filtering by nested/derived fields such as size, bitrate, HDR, codec, resolution, series, season, and episode can apply to the whole indexed library instead of only one Jellyfin page.
• Media inventory table should be read-only; full-index sorting/filtering should be handled by the service/query layer rather than relying on frontend table sorting.
• Media inventory should use row-based table selection and automatically sync the File browser tab to the selected row's containing directory.
• Media row selection should update both Media selected-row state and File browser location without requiring an extra action button.
• File browser interaction should stay explicit and simple: read-only listing plus explicit Open/Select actions rather than another row-selection grid.
• Use valid Jellyfin Fields query values only, because invalid field names can cause 400 Bad Request responses.

Remote Filesystem over SSH

• Connect to a remote media server via SSH.
• Use strict SSH host key behavior; users should connect manually once to populate known_hosts.
• Browse remote directories and files rooted at a configurable default media path.
• File browser handoff should map Jellyfin paths to REMOTE_MEDIA_ROOT when possible (for example /media/... -> /srv/media/... when root is /srv/media).
• Support a configurable Jellyfin-to-SSH fallback path prefix for cases where REMOTE_MEDIA_ROOT mapping alone is not sufficient.
• Support manual path entry and refresh.
• Remote file listing must be compact, structured, and navigable.
• The file table should be read-only.
• The file table should use row selection (single-select) in an AG Grid format consistent with the Media tab.
• The file table should not expose a visible checkbox selection column.
• The file table should not show a visible selected column.
• Include a top [UP] .. row, when not at /, to navigate to the parent directory.
• The selected path should be visibly shown outside the table.
• Selecting a directory row (including [UP] ..) should open it immediately.
• Selecting a file row should set the selected file target for metadata/jobs.
• Avoid emojis and hard-to-render characters in labels.

Remote Listing Controls

• Show compact listing summary information:
  • total entries;
  • directories;
  • files;
  • total file size for files in the current directory.
• Support filtering by:
  • all entries;
  • directories;
  • files;
  • file extension.
• Support case-insensitive filename search.
• Support sorting by:
  • name;
  • kind/type;
  • size;
  • modified time.
• Support ascending/descending sort order.
• Support pagination and configurable rows per page.

Disk-Level Metadata

• Run ffprobe on selected remote files to inspect authoritative media metadata directly from disk.
• When a known video/movie file is selected in the remote file browser, automatically run a blocking ffprobe preview and show a spinner while it completes.
• Cache preview results briefly to keep repeated Streamlit reruns responsive; allow users to reload the preview manually.
• Display ffprobe results in separate sections instead of one sparse all-streams table:
  • container/format summary;
  • video streams;
  • audio streams;
  • subtitle streams.
• Video metadata should include codec, profile, resolution, pixel format, bitrate, frame rate, color range/space/transfer/primaries, side data/HDR-related metadata where available, language, title, and default flag.
• Audio metadata should include codec, profile, channels/layout, sample rate, bitrate, language, title, default, and forced flags.
• Subtitle metadata should include codec, language, title, default, forced, and hearing-impaired flags where available.
• Expose raw ffprobe JSON for deeper inspection.
• Keep a manual ffprobe action available for selected paths.
• Support stat on selected paths.

Dashboard / Server Monitoring

• Provide a dashboard tab with a compact Jellyfin media library overview and server resource overview.
• Show Jellyfin media counts for movies, series, and series episodes on the dashboard.
• Show currently playing Jellyfin sessions on the dashboard, including user, media title, playback state, and whether transcoding is active.
• Provide a dashboard tab with a compact server resource overview over SSH.
• Provide a separate Monitoring tab for detailed resource charts, collector controls, diagnostics, and raw samples.
• Show CPU and RAM usage for the last hour.
• Show IO wait percentage for the last hour.
• Show average and spike/peak values for network throughput and disk I/O.
• Show used, available, and total disk space for the configured media root, falling back to /.
• Use a lightweight remote collector that reads Linux /proc, /sys/block, and df data into a JSONL file under /tmp.
• The collector should rotate/prune its JSONL metrics file so it does not grow unbounded; default retention is 7 days with a 70,000-line safety cap.
• The collector should be startable/stoppable/restartable from the dashboard and should not require installing a full monitoring stack.
• Last-hour charts require the collector to have been running long enough to collect samples.
• Network throughput should be shown split into down/download and up/upload.
• Network throughput should use bytes-per-second display units such as KB/s, MB/s, and GB/s to avoid bit/byte ambiguity.
• Disk throughput should be shown split into read and write.
• Network and disk throughput charts should scale values into readable units such as KB/s, MB/s, and GB/s.

Remote Jobs

• Support running remote jobs over SSH using explicit templates.
• Phase 1 jobs should be safe/read-only by default.
• Avoid arbitrary free-form command execution in the UI.
• Job templates should be centralized in jobs.py for future extension.
• Command template values must be shell-quoted before execution.
• Future destructive jobs should require explicit confirmation.

Current Architecture

• app.py - Thin root Streamlit entrypoint for streamlit run app.py.
• pyproject.toml - Authoritative package metadata, dependencies, and tool configuration.
• requirements.txt - Convenience install file that installs the local package editable.
• src/media_library_viewer/app.py - Thin Streamlit orchestration layer.
• src/media_library_viewer/ui/ - Streamlit UI modules split by feature area (dashboard, media, file browser, library, preview/tools).
• src/media_library_viewer/clients/jellyfin.py - Jellyfin API wrapper.
• src/media_library_viewer/clients/ssh.py - SSH command execution, directory listing, stat, and ffprobe helpers.
• src/media_library_viewer/domain/ - UI-independent normalization/domain helpers.
• src/media_library_viewer/services/ - UI-independent application services such as the SQLite media index.
• src/media_library_viewer/jobs.py - Remote job template definitions and runner.
• src/media_library_viewer/utils.py - Formatting and media metadata summarization helpers.
• src/media_library_viewer/config.py - Environment variable and .env configuration loading.
• docs/REQUIREMENTS.md - Living requirements and decision log.
• tests/ - Reserved for future test coverage.

Key Implementation Decisions

• Prefer the Jellyfin API for library and server metadata.
• Prefer SSH plus ffprobe for disk-authoritative stream/container metadata.
• Use API-key auth for Jellyfin, but select a user explicitly for user-scoped endpoints.
• Use streamlit-aggrid as a required dependency for Media table row selection. Avoid optional frontend fallbacks that create multiple interaction models.
• Keep remote jobs template-based to reduce accidental destructive actions.
• Keep the Phase 1 UI compact and structured rather than using large per-row buttons.
• Use a src/ package layout so the project can grow without accumulating many root-level modules.
• Keep root app.py as a compatibility/convenience wrapper for Streamlit.
• Keep clients, domain normalization, and application services independent from Streamlit so the frontend can later be replaced by React/FastAPI or another UI.
• Keep Streamlit rendering split into small UI modules so interaction bugs can be debugged in feature-local code instead of one monolithic app file.

Security and Safety Requirements

• Do not hardcode secrets.
• Use .env, environment variables, or Streamlit secrets for credentials.
• Keep .env and Streamlit secrets out of version control.
• Reject unknown SSH host keys by default.
• Treat SSH jobs as potentially dangerous and keep them explicit/template-based.
• Add confirmation steps before implementing cleanup, delete, transcode-replace, or other destructive workflows.

Configuration Requirements

Supported environment variables:

JELLYFIN_URL=
JELLYFIN_API_KEY=
JELLYFIN_USER_ID=
SSH_HOST=
SSH_USERNAME=
SSH_PORT=22
SSH_KEY_FILENAME=
SSH_PASSWORD=
REMOTE_MEDIA_ROOT=
REMOTE_PATH_PREFIX=

Known External Requirements

Remote server should have:

• Linux /proc and /sys/block for resource metrics
• /bin/sh for POSIX command execution, even when the user's login shell is fish or another non-POSIX shell
• POSIX shell utilities including awk, date, tail, df, kill, and nohup
• python3
• GNU/coreutils-compatible find and stat
• ffprobe for media metadata inspection

Local app dependencies are declared in pyproject.toml; requirements.txt installs the package editable for convenience. Runtime dependencies include:

• streamlit
• streamlit-aggrid
• requests
• paramiko
• python-dotenv
• pandas

Backlog / Future Extensions

• Add transcode job templates.
• Add cleanup job templates with dry-run and explicit confirmation.
• Add subtitle/audio-track diagnostics.
• Add sidecar file inspection for .nfo, .srt, images, and metadata files.
• Compare Jellyfin metadata against disk metadata and sidecars.
• Add long-running job tracking/log streaming.
• Add saved presets for common media roots and job templates.
• Add file previews for text sidecars.
• Add richer HDR/Dolby Vision/bit-depth summaries from ffprobe.
• Add optional integration with existing monitoring stacks such as Prometheus/node_exporter, Netdata, or sysstat/sar.

Decision Log

2026-04-30 - Initial app plan

• Planned a Streamlit app that uses the Jellyfin API as the primary metadata source.
• Decided SSH should be used for disk inspection and future maintenance jobs.

2026-04-30 - Phase 1 implementation

• Created the initial app structure with Jellyfin, SSH, jobs, config, and utility modules.
• Added safe/read-only remote job templates.
• Added ffprobe and stat inspection.

2026-04-30 - Jellyfin API fixes

• Replaced /Users/Me usage with GET /Users plus user selection.
• Added JELLYFIN_USER_ID override.
• Cleaned Jellyfin Fields values to avoid 400 responses.
• Added defensive stripping of trailing /web from Jellyfin URLs.

2026-04-30 - Remote file browser evolution

• Added interactive remote listing.
• Removed emoji and hard-to-render characters.
• Added search, filtering, sorting, pagination, and compact listing summary.
• Reworked listing from large button rows into a compact table.
• Switched to streamlit-aggrid for file-browser-like row click behavior.
• Removed visible checkbox/selection column behavior.
• Added [UP] .. top row for parent directory navigation.

2026-04-30 - Selected-file metadata preview

• Added a requirement for automatic ffprobe preview when known video files are selected.
• Initially explored asynchronous/non-blocking preview, then changed to a blocking call with a spinner because it is more streamlined for this app.
• Decided to cache preview results briefly and provide a manual reload action.
• Decided ffprobe output should be separated into container, video, audio, and subtitle sections to avoid sparse mixed-stream tables.

2026-04-30 - Process requirement

• Added this living requirements and decision log document.
• Added a global agent skill to encourage maintaining such a document for future projects.

2026-04-30 - Repository restructuring

• Restructured the project into a larger-project-ready src/media_library_viewer/ package layout.
• Kept a thin root app.py entrypoint so streamlit run app.py remains the primary launch command.
• Moved service clients into src/media_library_viewer/clients/.
• Added pyproject.toml with runtime dependencies, development extras, Ruff configuration, and pytest configuration.
• Simplified requirements.txt to install the local project editable.
• Expanded .gitignore for Python caches, build artifacts, virtual environments, local secrets, editor files, and logs.

2026-04-30 - Resource dashboard

• Added a dashboard requirement for CPU, RAM, network, disk I/O, and disk space overview.
• Decided that true last-hour metrics require collection over time; implemented a lightweight SSH-started remote collector instead of requiring Prometheus, Netdata, or sysstat.
• The collector stores JSONL samples in /tmp and can be started/stopped from the dashboard.
• Charts show the last hour of collected samples; the dashboard becomes more useful once the collector has been running for a while.
• Last-hour filtering uses epoch seconds rather than local naive datetimes to avoid timezone-offset issues between the app host and remote sample timestamps.
• Fixed SSH command execution to explicitly use /bin/sh -c so POSIX resource commands work even when the remote user's login shell is fish.
• Added explicit Streamlit keys to dashboard/file/tool buttons to avoid duplicate auto-generated element IDs as the UI grows.
• Changed the resource collector script from bash-specific syntax to POSIX /bin/sh syntax and added dashboard diagnostics/restart controls for collector troubleshooting.
• Added 7-day metrics file pruning plus a 70,000-line safety cap to prevent the JSONL file from growing without bound.
• Split network charts and metrics into download and upload, and disk charts and metrics into read and write.
• Changed network display units from bits per second to bytes per second to avoid Kbps/KB/s ambiguity; the collector still stores bit-rate compatibility fields for old/debug consumers.
• Scaled network and disk throughput charts into readable units such as KB/s, MB/s, and GB/s instead of plotting raw base units.
• Updated collector startup to remove old temporary metrics/log files when a new collector process is started after a schema/display change.
• Moved detailed resource charts, raw samples, diagnostics, and collector controls into a dedicated Monitoring tab; the Dashboard now keeps a compact overview.
• Removed the CPU/RAM chart from the Dashboard and kept detailed charts in the Monitoring tab.
• Renamed the remote files tab to File browser.
• Added Jellyfin media counts for movies, series, and episodes to the Dashboard using lightweight count queries.
• Added a Dashboard now-playing section sourced from Jellyfin sessions, showing who is currently playing what and whether each session is transcoding.

2026-04-30 - Media inventory tab

• Added a paginated Media tab for file-oriented Jellyfin metadata.
• Decided not to fetch all media at once because large libraries can make API responses and Streamlit rendering slow.
• Decided to derive length, size, bitrate, HDR flag, date added, codec, and resolution from Jellyfin metadata for now.
• Added series name, season, and episode number for episode rows.
• Added server-side sort/order controls and read-only AG Grid column sorting/filtering for the loaded page.
• Reworked the Media tab to use a local SQLite media index for full-library sorting/filtering, including numeric sorting for size and bitrate.
• Added last index build duration metadata to the Media tab status line.
• Replaced single-library selection with multi-library selection so users can include/exclude multiple libraries in the indexed table.
• Changed HDR display from blank/no-value to explicit yes/no.
• Added row selection plus an Open folder action in the Media tab that sets the File browser to the containing folder.
• Restored row-based Media table selection while keeping File browser state changes limited to the explicit Open folder button.
• Updated Media tab behavior so selecting a row automatically syncs the File browser folder to that item's containing directory; removed the extra Open folder button step.
• Restored File browser table row selection with AG Grid (single-select), using a table interaction style consistent with the Media tab.
• Reintroduced open-on-select behavior in File browser: selecting a directory row (including [UP] ..) opens it immediately, while file rows update selected target path.
• Refined Media table column presentation with explicit user-friendly headers and null-safe display formatting to keep the grid readable and consistent.
• Renamed Resources tab to Monitoring; added IO wait (iowait) percentage to the collector script, metrics, dashboard summary, and detailed charts.
• Removed the Jellyfin library poster-grid tab and its associated cached API calls and UI module; the Media index tab now covers library browsing needs.
• Simplified File browser navigation: removed Up/Go/Select folder buttons; pressing Enter in the path text input navigates directly.
• Added broad inline/module documentation across clients, domain, services, and Streamlit adapter modules to make debugging and future frontend extraction easier.
• Simplified the File browser by removing its interactive AG Grid and using a read-only listing with explicit Open/Select controls, reducing cross-tab state interactions with the Media grid.
• Removed optional/compatibility code paths around the Media table grid and old file-browser state aliases to keep the interaction model easier to reason about during debugging.
• Split the Streamlit frontend into dedicated UI modules (dashboard, media, file browser, library, preview/tools) and reduced app.py to orchestration glue.
• Reviewed remote path handling and kept shell interactions routed through quoted paths (shlex.quote) while UI/path-parent operations use POSIX path handling, preserving paths with spaces.
• Fixed file browser handoff/navigation to reset stale search/filter/page state when changing folders, preventing old filters from hiding all entries in the newly opened folder.
• Reworked file browser state to separate current directory from selected path. Selecting a file no longer changes the directory being listed, while opening a folder updates the current directory and keeps path input synchronized.
• Made remote directory listing fail explicitly when the current path is not a directory and recover by listing the parent, preventing file paths from appearing as empty directories.
• Made File browser Refresh/Select folder apply a manually typed path if it differs from the current folder, reducing confusion when manually navigating.
• Moved media normalization into domain/media.py and index/query logic into services/media_index.py to make the project less Streamlit-specific and easier to expose through a future API/React frontend.
• Deferred full ffprobe enrichment for every item to a future cached/background scan.
• Fixed network byte parsing to split /proc/net/dev lines at the colon first, so interface indentation differences do not shift fields and accidentally report packet counts instead of byte counts.
• Fixed a follow-up /proc/net/dev parsing issue where leading whitespace after the colon could produce an empty first split field in some awk implementations, resulting in zero network rates. Added /proc/net/dev snapshots to collector diagnostics.
• Simplified File browser directory error behavior: stopped automatic parent-directory fallback and now show the direct listing error for the current path.
• Added configurable REMOTE_PATH_PREFIX support so Jellyfin paths can be mapped to SSH-visible paths when opening folders from Media/Library tabs (for example /media/... -> /srv/media/...).
• Updated path handoff logic to prefer mapping through REMOTE_MEDIA_ROOT (anchor replacement using the root basename, e.g. media) and use REMOTE_PATH_PREFIX as fallback.
• Added a public-repo readiness note in README describing what local/sensitive files must stay out of version control.
• Added LICENSE (MIT) and CONTRIBUTING.md for public-repo baseline documentation.