alex/backup-tool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite v2 operator documentation

Browse Source
This commit is contained in:
Alex
2026-07-31 14:34:34 +02:00
parent bd107d6a30
commit f97daa35d0
2 changed files with 125 additions and 196 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+83 -196
View File
@@ -1,232 +1,119 @@
# Backup Tool # Backup Tool v2
A modern backup management application with a FastAPI backend and React frontend. > A self-hosted, integrity-first backup appliance for local and hardened SSH sources.
## Features Backup Tool creates signed, content-addressed backups, supports encrypted repositories and offline recovery, and runs as isolated web, scheduler, worker, migration, and admin roles. v2.0 supports **local** and **SSH** sources, local repositories, restore, encrypted recovery, email, and signed webhooks.
- **Source Type**: Local filesystem ## What is supported
- **Backup Strategies**: Full and incremental backups
- **Scheduled Backups**: Cron-based scheduling with APScheduler
- **Retention Policies**: Count-based and days-based backup retention
- **Web Dashboard**: React-based UI for managing backups
- **REST API**: Full REST API for programmatic access
## Architecture - Signed full and incremental backups, restore, retention, verification, and recovery.
- AES-256-GCM repository encryption with key epochs and offline Argon2id recovery bundles.
- Local sources and SSH sources using **private-key-only, pinned-host-key, forced-SFTP chroot** access.
- Durable, at-least-once webhook and STARTTLS SMTP notifications.
- A generated OpenAPI TypeScript client and operator UI.
``` Never use an SSH source with a shell, password, agent forwarding, remote commands, or an unchrooted account. See [SSH source requirements](docs/runbooks/ssh-sources.md).
backup-tool/
├── backend/ # FastAPI backend ## Quick local verification
│ ├── app/ # FastAPI application
│ │ ├── routers/ # API endpoints Prerequisites: Docker Compose, Python 3.12–3.14, Node/npm, and a working virtual environment.
│ │ ├── models.py # SQLAlchemy models
│ │ ├── schemas.py # Pydantic schemas ```sh
│ │ └── main.py # Application entry point make setup
│ ├── backup/ # Backup engine make check
│ │ ├── adapters/ # Local source adapter make test-e2e
│ │ ├── engine.py # Backup execution engine make test-ssh-integration
│ │ ├── scheduler.py # Job scheduler
│ │ └── retention.py # Retention policies
│ └── requirements.txt # Python dependencies
├── frontend/ # React frontend
│ ├── src/ # Source code
│ └── package.json # Node dependencies
└── docs/ # Documentation
``` ```
## Quick Start `make test-e2e` creates temporary test-only keys and source data, starts the complete Compose stack, checks readiness/metrics/restart behavior, and tears it down. `make test-ssh-integration` builds a separate forced-SFTP fixture and verifies key-authenticated SSH probe, backup, verification, and restore.
### Option 1: Docker (Recommended) ## Run locally with Docker Compose
The easiest way to run the backup tool is using Docker Compose: The production Compose topology binds the proxy to `127.0.0.1:8080`; put a TLS reverse proxy in front of it for remote access.
```bash ```sh
# Start the backend mkdir -p secrets sources
head -c 32 /dev/urandom > secrets/master.key
chmod 600 secrets/master.key
# The container runs as UID 10001; grant that UID read access to the key.
chown 10001:10001 secrets/master.key
# Put test source data below ./sources, then start the isolated roles.
docker compose build --pull
docker compose run --rm migrate upgrade
docker compose up -d docker compose up -d
curl -fsS http://127.0.0.1:8080/readyz
# Start with frontend (production) curl -fsS http://127.0.0.1:8080/metrics
docker compose --profile prod up -d
# Start with frontend (development with hot reload)
docker compose --profile dev up -d
``` ```
Access the application: On a fresh non-loopback deployment, set `BACKUP_TOOL_PUBLIC_BASE_URL` to the externally reachable HTTPS URL and set `BACKUP_TOOL_BOOTSTRAP_SECRET` before first setup. Never expose a fresh setup endpoint without bootstrap protection.
- Backend API: <http://localhost:8000> Stop the local stack with:
- Frontend: <http://localhost:3000>
- API Docs: <http://localhost:8000/docs>
### Option 2: Manual Setup ```sh
docker compose down
#### Prerequisites # Add --volumes only when intentionally discarding local metadata and repositories.
- Python 3.11+
- Node.js 18+
- PostgreSQL or MySQL (for database backups)
#### Backend Setup
```bash
cd backend
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -e ".[dev]"
# Run the server
uvicorn app.main:app --reload --port 8000
``` ```
#### Frontend Setup ## Operator workflow
```bash 1. Open `http://127.0.0.1:8080` and create the first administrator.
cd frontend 2. Create a local or encrypted repository.
npm install 3. Create a local source, or configure a hardened SSH source as documented in [SSH sources](docs/runbooks/ssh-sources.md).
npm run dev 4. Create a job, run a probe, enqueue a backup, verify it, then perform a test restore.
``` 5. Configure notification subscriptions and test them before relying on delivery.
6. Export and validate a recovery bundle after encrypted repository creation and each key rotation.
#### Production Build The browser exposes recovery status and the CLI procedure only; it never transfers recovery bundles or passphrases.
```bash
cd frontend
npm run build
cd ../backend
uvicorn app.main:app --host 0.0.0.0 --port 8000
```
## API Documentation
Once the backend is running, visit:
- Swagger UI: <http://localhost:8000/docs>
- ReDoc: <http://localhost:8000/redoc>
## Configuration ## Configuration
### Environment Variables Compose supplies the core runtime variables. The important host paths are:
| Variable | Description | Default | | Setting | Compose value | Purpose |
| ---------- | ------------- | --------- | | --- | --- | --- |
| `DATABASE_URL` | Database connection string | `sqlite+aiosqlite:///./backup_tool.db` | | `BACKUP_TOOL_MASTER_KEY_FILE` | `/run/backup-tool-secrets/master.key` | Service-owned `0600` master key |
| `CORS_ORIGINS` | Comma-separated allowed CORS origins | `http://localhost:3000` | | `BACKUP_TOOL_SOURCES_DIR` | `./sources` | Read-only local-source bind mount |
| `SQL_ECHO` | Enable SQL query logging | `false` | | `BACKUP_TOOL_PUBLIC_BASE_URL` | `http://localhost:8080` | External URL and bootstrap policy |
| `BACKUP_STORAGE_PATH` | Path for storing backups | `/app/backups` | | `BACKUP_TOOL_PORT` | `8080` | Loopback proxy port |
### Docker-Specific Configuration Use absolute, allowlisted paths for repositories, sources, and restores. The service rejects unsafe paths, symlinks where they violate the contract, missing key files, and non-current schemas.
When running with Docker Compose, the following volumes are mounted: ## Operations and security
- `backup-data`: Persisted SQLite database at `/app/data` - [Observability](docs/runbooks/observability.md): readiness, liveness, metrics, and alert response.
- `backup-storage`: Backup files at `/app/backups` - [Upgrade and rollback](docs/runbooks/upgrade.md)
- [Disaster recovery](docs/runbooks/disaster-recovery.md)
### Development vs Production - [Recovery bundles](docs/runbooks/recovery-bundle.md)
- [Repository keys](docs/runbooks/keys.md)
**Development Mode** (`docker compose --profile dev up -d`): - [Notifications](docs/runbooks/notifications.md)
- [SSH sources](docs/runbooks/ssh-sources.md)
- Backend hot reload enabled - [Security policies](docs/README.md#security-boundaries)
- Frontend Vite dev server with HMR
- Source code mounted as volumes
**Production Mode** (`docker compose --profile prod up -d`):
- Optimized frontend build served via nginx
- Backend without reload
- Static assets compiled
## Troubleshooting
### Docker Issues
**Port already in use**
```bash
# Check what's using port 8000
lsof -i :8000
# Or use different ports in docker-compose.yml
```
**Container fails to start**
```bash
# Check logs
docker logs backup-tool-backend
# Rebuild with no cache
docker compose build --no-cache
```
**Permission denied on data directory**
```bash
# Fix permissions
docker compose exec backend chown -R backup-tool:backup-tool /app/data
```
**Tests fail in Docker**
Tests require development dependencies. Install with:
```bash
docker compose exec backend pip install -e ".[dev]"
```
### Manual Setup Issues
**Python version incompatibility**
Ensure Python 3.11+ is installed:
```bash
python --version
```
**Node modules conflicts**
```bash
cd frontend
rm -rf node_modules package-lock.json
npm install
```
## Deployment
### Docker Deployment
1. Clone the repository
2. Run `docker compose --profile prod up -d`
3. Access at <http://localhost:3000>
### Manual Deployment
1. Install Python 3.11+ and Node.js 18+
2. Install backend: `cd backend && pip install -e ".[prod]"`
3. Build frontend: `cd frontend && npm run build`
4. Start backend: `cd backend && uvicorn app.main:app --host 0.0.0.0`
### Production Considerations
- Use a reverse proxy (nginx, traefik) for SSL termination
- Set strong credentials for database sources
- Configure backup retention policies
- Monitor disk usage for backup storage
- Use `docker compose -f docker-compose.yml up -d` for production without dev tools
## Development ## Development
### Running Tests ```sh
make setup
```bash make test-fast
cd backend make test-integration
pytest make test-fault
make test-security
npm --prefix frontend test -- --run
npm --prefix frontend exec playwright test
``` ```
### Database Migrations OpenAPI and the generated browser client are committed artifacts:
```bash ```sh
cd backend .venv/bin/python tools/export_openapi.py --check openapi/v2.json
alembic revision --autogenerate -m "Description" npm --prefix frontend run api:generate
alembic upgrade head git diff --exit-code -- openapi/v2.json frontend/src/api/generated
``` ```
## Release evidence
Milestone evidence, the SBOM, provenance, and synthetic scale report live in [docs/release](docs/release/). M15 certification is synthetic metadata certification on the reference CI host; it does not claim a physical 10 TiB transfer.
## License ## License
MIT MIT — see [LICENSE](LICENSE).
docs/README.md
+42
View File
@@ -0,0 +1,42 @@
# Backup Tool v2 documentation
## Reason for existence
This index points operators and contributors to the authoritative v2.0 procedures. Do not duplicate runbook steps in release evidence or issue comments.
## Start here
- [Project quick start](../README.md)
- [Upgrade and rollback](runbooks/upgrade.md)
- [Observability and alert response](runbooks/observability.md)
- [Disaster recovery](runbooks/disaster-recovery.md)
## Runbooks
| Need | Authoritative guide |
| --- | --- |
| Repository lifecycle | [repositories](runbooks/repositories.md) |
| Metadata protection | [metadata](runbooks/metadata.md) |
| Master and repository keys | [keys](runbooks/keys.md) |
| Encrypted recovery bundle | [recovery bundle](runbooks/recovery-bundle.md) |
| SSH source hardening | [SSH sources](runbooks/ssh-sources.md) |
| Notification operations | [notifications](runbooks/notifications.md) |
| Upgrade safety | [upgrade](runbooks/upgrade.md) |
| Host loss | [disaster recovery](runbooks/disaster-recovery.md) |
## Security boundaries
- [Repository encryption threat model](security/repository-encryption.md)
- [Notification egress and delivery policy](security/notifications.md)
- SSH sources require private-key authentication, exact host-key pinning, and a forced-SFTP chrooted account. No password, shell, agent, tunnel, or remote-command mode exists.
## Release records
[Release evidence](release/) records verification rather than replacing runbooks. v2.0 scale certification is documented in `release/m15-evidence.md` and `release/m15-scale-report.json`.
## Verify
```sh
grep -c '^## ' README.md
make check
```