4989652c5d
- 20 tasks covering backend, frontend, adapters, scheduler - File structure with exact paths - Complete code in every step - Self-review with spec coverage verification
3096 lines
88 KiB
Markdown
3096 lines
88 KiB
Markdown
# Backup Tool Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Build a web-based backup management tool with FastAPI backend and React frontend
|
|
|
|
**Architecture:** Python FastAPI backend with async SQLite database, React frontend with TypeScript, source adapter pattern for extensible backup sources
|
|
|
|
**Tech Stack:** Python 3.11+, FastAPI, SQLAlchemy 2.0, Alembic, APScheduler, React 18+, TypeScript, TanStack Query, Tailwind CSS
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
```
|
|
backup-tool/
|
|
├── backend/
|
|
│ ├── alembic/
|
|
│ │ ├── versions/
|
|
│ │ └── env.py
|
|
│ ├── app/
|
|
│ │ ├── __init__.py
|
|
│ │ ├── main.py # FastAPI entry point
|
|
│ │ ├── database.py # SQLAlchemy setup
|
|
│ │ ├── models.py # Database models
|
|
│ │ ├── schemas.py # Pydantic schemas
|
|
│ │ ├── config.py # App configuration
|
|
│ │ └── routers/
|
|
│ │ ├── __init__.py
|
|
│ │ ├── dashboard.py # Dashboard stats
|
|
│ │ ├── sources.py # Source CRUD
|
|
│ │ ├── jobs.py # Job CRUD + run
|
|
│ │ ├── executions.py # Execution monitoring
|
|
│ │ ├── backups.py # Backup management
|
|
│ │ └── settings.py # Settings management
|
|
│ ├── backup/
|
|
│ │ ├── __init__.py
|
|
│ │ ├── engine.py # Backup execution engine
|
|
│ │ ├── scheduler.py # APScheduler setup
|
|
│ │ ├── retention.py # Retention policy
|
|
│ │ └── adapters/
|
|
│ │ ├── __init__.py
|
|
│ │ ├── base.py # Abstract adapter
|
|
│ │ ├── local.py # Local filesystem
|
|
│ │ ├── ssh.py # SSH/SFTP
|
|
│ │ └── database.py # Database dump
|
|
│ ├── tests/
|
|
│ │ ├── __init__.py
|
|
│ │ ├── conftest.py # Pytest fixtures
|
|
│ │ ├── test_sources.py
|
|
│ │ ├── test_jobs.py
|
|
│ │ └── test_engine.py
|
|
│ ├── requirements.txt
|
|
│ └── alembic.ini
|
|
├── frontend/
|
|
│ ├── package.json
|
|
│ ├── tsconfig.json
|
|
│ ├── vite.config.ts
|
|
│ ├── tailwind.config.js
|
|
│ ├── index.html
|
|
│ └── src/
|
|
│ ├── main.tsx
|
|
│ ├── App.tsx
|
|
│ ├── api/
|
|
│ │ └── client.ts # API client
|
|
│ ├── components/
|
|
│ │ ├── Layout.tsx # Main layout
|
|
│ │ ├── JobForm.tsx # Create/edit job
|
|
│ │ └── SourceForm.tsx # Create/edit source
|
|
│ └── pages/
|
|
│ ├── Dashboard.tsx
|
|
│ ├── Backups.tsx
|
|
│ └── Settings.tsx
|
|
└── README.md
|
|
```
|
|
|
|
---
|
|
|
|
## Tasks
|
|
|
|
### Task 1: Project Setup
|
|
|
|
**Files:**
|
|
- Create: `backend/requirements.txt`
|
|
- Create: `frontend/package.json`
|
|
- Create: `frontend/tsconfig.json`
|
|
- Create: `frontend/vite.config.ts`
|
|
- Create: `frontend/tailwind.config.js`
|
|
- Create: `frontend/index.html`
|
|
|
|
- [ ] **Step 1: Create backend requirements**
|
|
|
|
```txt
|
|
fastapi==0.109.0
|
|
uvicorn[standard]==0.27.0
|
|
sqlalchemy[asyncio]==2.0.25
|
|
aiosqlite==0.19.0
|
|
alembic==1.13.1
|
|
pydantic==2.5.3
|
|
pydantic-settings==2.1.0
|
|
apscheduler==3.10.4
|
|
paramiko==3.4.0
|
|
pytest==7.4.4
|
|
pytest-asyncio==0.21.1
|
|
httpx==0.26.0
|
|
```
|
|
|
|
- [ ] **Step 2: Create frontend package.json**
|
|
|
|
```json
|
|
{
|
|
"name": "backup-tool-frontend",
|
|
"private": true,
|
|
"version": "0.0.1",
|
|
"type": "module",
|
|
"scripts": {
|
|
"dev": "vite",
|
|
"build": "tsc && vite build",
|
|
"preview": "vite preview"
|
|
},
|
|
"dependencies": {
|
|
"react": "^18.2.0",
|
|
"react-dom": "^18.2.0",
|
|
"react-router-dom": "^6.21.0",
|
|
"@tanstack/react-query": "^5.17.0",
|
|
"axios": "^1.6.5",
|
|
"react-hook-form": "^7.49.0",
|
|
"recharts": "^2.10.0",
|
|
"lucide-react": "^0.303.0",
|
|
"clsx": "^2.1.0"
|
|
},
|
|
"devDependencies": {
|
|
"@types/react": "^18.2.43",
|
|
"@types/react-dom": "^18.2.17",
|
|
"@vitejs/plugin-react": "^4.2.1",
|
|
"autoprefixer": "^10.4.16",
|
|
"postcss": "^8.4.32",
|
|
"tailwindcss": "^3.4.0",
|
|
"typescript": "^5.2.2",
|
|
"vite": "^5.0.8"
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Create frontend config files**
|
|
|
|
`frontend/tsconfig.json`:
|
|
```json
|
|
{
|
|
"compilerOptions": {
|
|
"target": "ES2020",
|
|
"useDefineForClassFields": true,
|
|
"lib": ["ES2020", "DOM", "DOM.Iterable"],
|
|
"module": "ESNext",
|
|
"skipLibCheck": true,
|
|
"moduleResolution": "bundler",
|
|
"allowImportingTsExtensions": true,
|
|
"resolveJsonModule": true,
|
|
"isolatedModules": true,
|
|
"noEmit": true,
|
|
"jsx": "react-jsx",
|
|
"strict": true,
|
|
"noUnusedLocals": true,
|
|
"noUnusedParameters": true,
|
|
"noFallthroughCasesInSwitch": true
|
|
},
|
|
"include": ["src"],
|
|
"references": [{ "path": "./tsconfig.node.json" }]
|
|
}
|
|
```
|
|
|
|
`frontend/vite.config.ts`:
|
|
```typescript
|
|
import { defineConfig } from 'vite'
|
|
import react from '@vitejs/plugin-react'
|
|
|
|
export default defineConfig({
|
|
plugins: [react()],
|
|
server: {
|
|
port: 3000,
|
|
proxy: {
|
|
'/api': 'http://localhost:8000'
|
|
}
|
|
}
|
|
})
|
|
```
|
|
|
|
`frontend/tailwind.config.js`:
|
|
```javascript
|
|
/** @type {import('tailwindcss').Config} */
|
|
export default {
|
|
content: ["./index.html", "./src/**/*.{js,ts,jsx,tsx}"],
|
|
theme: {
|
|
extend: {},
|
|
},
|
|
plugins: [],
|
|
}
|
|
```
|
|
|
|
`frontend/index.html`:
|
|
```html
|
|
<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8" />
|
|
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
|
<title>Backup Tool</title>
|
|
</head>
|
|
<body>
|
|
<div id="root"></div>
|
|
<script type="module" src="/src/main.tsx"></script>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add backend/requirements.txt frontend/package.json frontend/tsconfig.json frontend/vite.config.ts frontend/tailwind.config.js frontend/index.html
|
|
git commit -m "chore: setup project structure with dependencies"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Database Models
|
|
|
|
**Files:**
|
|
- Create: `backend/app/database.py`
|
|
- Create: `backend/app/models.py`
|
|
- Create: `backend/alembic.ini`
|
|
- Create: `backend/alembic/env.py`
|
|
|
|
- [ ] **Step 1: Create database setup**
|
|
|
|
`backend/app/database.py`:
|
|
```python
|
|
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
|
|
from sqlalchemy.orm import declarative_base
|
|
|
|
DATABASE_URL = "sqlite+aiosqlite:///./backup_tool.db"
|
|
|
|
engine = create_async_engine(DATABASE_URL, echo=True)
|
|
AsyncSessionLocal = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
|
|
|
Base = declarative_base()
|
|
|
|
async def get_db():
|
|
async with AsyncSessionLocal() as session:
|
|
try:
|
|
yield session
|
|
finally:
|
|
await session.close()
|
|
```
|
|
|
|
- [ ] **Step 2: Create models**
|
|
|
|
`backend/app/models.py`:
|
|
```python
|
|
from sqlalchemy import Column, Integer, String, DateTime, Boolean, Text, ForeignKey, JSON
|
|
from sqlalchemy.orm import relationship
|
|
from datetime import datetime
|
|
from .database import Base
|
|
|
|
class Source(Base):
|
|
__tablename__ = "sources"
|
|
|
|
id = Column(Integer, primary_key=True, index=True)
|
|
name = Column(String, nullable=False)
|
|
type = Column(String, nullable=False) # local, ssh, database
|
|
config = Column(JSON, default=dict)
|
|
created_at = Column(DateTime, default=datetime.utcnow)
|
|
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
|
|
|
|
jobs = relationship("Job", back_populates="source", cascade="all, delete-orphan")
|
|
|
|
class Job(Base):
|
|
__tablename__ = "jobs"
|
|
|
|
id = Column(Integer, primary_key=True, index=True)
|
|
name = Column(String, nullable=False)
|
|
source_id = Column(Integer, ForeignKey("sources.id"), nullable=False)
|
|
strategy = Column(String, nullable=False, default="full") # full, incremental
|
|
destination_path = Column(String, nullable=False)
|
|
exclude_patterns = Column(JSON, default=list)
|
|
enabled = Column(Boolean, default=True)
|
|
created_at = Column(DateTime, default=datetime.utcnow)
|
|
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
|
|
|
|
source = relationship("Source", back_populates="jobs")
|
|
schedule = relationship("Schedule", back_populates="job", uselist=False, cascade="all, delete-orphan")
|
|
executions = relationship("JobExecution", back_populates="job", cascade="all, delete-orphan")
|
|
|
|
class Schedule(Base):
|
|
__tablename__ = "schedules"
|
|
|
|
id = Column(Integer, primary_key=True, index=True)
|
|
job_id = Column(Integer, ForeignKey("jobs.id"), unique=True, nullable=False)
|
|
cron_expression = Column(String, nullable=False)
|
|
enabled = Column(Boolean, default=True)
|
|
created_at = Column(DateTime, default=datetime.utcnow)
|
|
|
|
job = relationship("Job", back_populates="schedule")
|
|
|
|
class JobExecution(Base):
|
|
__tablename__ = "job_executions"
|
|
|
|
id = Column(Integer, primary_key=True, index=True)
|
|
job_id = Column(Integer, ForeignKey("jobs.id"), nullable=False)
|
|
status = Column(String, nullable=False, default="pending") # pending, running, success, failed, cancelled
|
|
started_at = Column(DateTime, nullable=True)
|
|
completed_at = Column(DateTime, nullable=True)
|
|
bytes_processed = Column(Integer, default=0)
|
|
bytes_backed_up = Column(Integer, default=0)
|
|
error_message = Column(Text, nullable=True)
|
|
triggered_by = Column(String, nullable=False) # manual, schedule
|
|
|
|
job = relationship("Job", back_populates="executions")
|
|
backups = relationship("Backup", back_populates="execution", cascade="all, delete-orphan")
|
|
|
|
class Backup(Base):
|
|
__tablename__ = "backups"
|
|
|
|
id = Column(Integer, primary_key=True, index=True)
|
|
execution_id = Column(Integer, ForeignKey("job_executions.id"), nullable=False)
|
|
storage_path = Column(String, nullable=False)
|
|
size_bytes = Column(Integer, default=0)
|
|
checksum = Column(String, nullable=True)
|
|
type = Column(String, nullable=False) # full, incremental
|
|
parent_backup_id = Column(Integer, ForeignKey("backups.id"), nullable=True)
|
|
created_at = Column(DateTime, default=datetime.utcnow)
|
|
|
|
execution = relationship("JobExecution", back_populates="backups")
|
|
parent_backup = relationship("Backup", remote_side=[id])
|
|
|
|
class Setting(Base):
|
|
__tablename__ = "settings"
|
|
|
|
key = Column(String, primary_key=True)
|
|
value = Column(Text, nullable=True)
|
|
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
|
|
```
|
|
|
|
- [ ] **Step 3: Create alembic config**
|
|
|
|
`backend/alembic.ini`:
|
|
```ini
|
|
[alembic]
|
|
script_location = alembic
|
|
prepend_sys_path = .
|
|
version_path_separator = os
|
|
sqlalchemy.url = sqlite+aiosqlite:///./backup_tool.db
|
|
```
|
|
|
|
`backend/alembic/env.py`:
|
|
```python
|
|
import asyncio
|
|
from logging.config import fileConfig
|
|
from sqlalchemy import pool
|
|
from sqlalchemy.engine import Connection
|
|
from sqlalchemy.ext.asyncio import async_engine_from_config
|
|
from alembic import context
|
|
from app.models import Base
|
|
|
|
config = context.config
|
|
if config.config_file_name is not None:
|
|
fileConfig(config.config_file_name)
|
|
|
|
target_metadata = Base.metadata
|
|
|
|
def run_migrations_offline() -> None:
|
|
url = config.get_main_option("sqlalchemy.url")
|
|
context.configure(
|
|
url=url,
|
|
target_metadata=target_metadata,
|
|
literal_binds=True,
|
|
dialect_opts={"paramstyle": "named"},
|
|
)
|
|
with context.begin_transaction():
|
|
context.run_migrations()
|
|
|
|
def do_run_migrations(connection: Connection) -> None:
|
|
context.configure(connection=connection, target_metadata=target_metadata)
|
|
with context.begin_transaction():
|
|
context.run_migrations()
|
|
|
|
async def run_async_migrations() -> None:
|
|
connectable = async_engine_from_config(
|
|
config.get_section(config.config_ini_section, {}),
|
|
prefix="sqlalchemy.",
|
|
poolclass=pool.NullPool,
|
|
)
|
|
async with connectable.connect() as connection:
|
|
await connection.run_sync(do_run_migrations)
|
|
await connectable.dispose()
|
|
|
|
def run_migrations_online() -> None:
|
|
asyncio.run(run_async_migrations())
|
|
|
|
if context.is_offline_mode():
|
|
run_migrations_offline()
|
|
else:
|
|
run_migrations_online()
|
|
```
|
|
|
|
- [ ] **Step 4: Initialize alembic and create first migration**
|
|
|
|
```bash
|
|
cd backend
|
|
alembic init alembic
|
|
# Update alembic/env.py with the content from Step 3
|
|
alembic revision --autogenerate -m "Initial migration"
|
|
alembic upgrade head
|
|
```
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add backend/app/database.py backend/app/models.py backend/alembic.ini backend/alembic/
|
|
git commit -m "feat: add database models and alembic setup
|
|
|
|
- SQLAlchemy async models for sources, jobs, schedules, executions, backups, settings
|
|
- Alembic migration configuration
|
|
- Initial migration created"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Pydantic Schemas
|
|
|
|
**Files:**
|
|
- Create: `backend/app/schemas.py`
|
|
|
|
- [ ] **Step 1: Create Pydantic schemas**
|
|
|
|
`backend/app/schemas.py`:
|
|
```python
|
|
from pydantic import BaseModel, Field
|
|
from typing import Optional, List, Dict, Any
|
|
from datetime import datetime
|
|
|
|
# Source schemas
|
|
class SourceBase(BaseModel):
|
|
name: str
|
|
type: str = Field(..., pattern="^(local|ssh|database)$")
|
|
config: Dict[str, Any] = Field(default_factory=dict)
|
|
|
|
class SourceCreate(SourceBase):
|
|
pass
|
|
|
|
class SourceUpdate(BaseModel):
|
|
name: Optional[str] = None
|
|
config: Optional[Dict[str, Any]] = None
|
|
|
|
class Source(SourceBase):
|
|
id: int
|
|
created_at: datetime
|
|
updated_at: datetime
|
|
|
|
class Config:
|
|
from_attributes = True
|
|
|
|
# Job schemas
|
|
class JobBase(BaseModel):
|
|
name: str
|
|
source_id: int
|
|
strategy: str = Field(default="full", pattern="^(full|incremental)$")
|
|
destination_path: str
|
|
exclude_patterns: List[str] = Field(default_factory=list)
|
|
enabled: bool = True
|
|
|
|
class JobCreate(JobBase):
|
|
pass
|
|
|
|
class JobUpdate(BaseModel):
|
|
name: Optional[str] = None
|
|
strategy: Optional[str] = None
|
|
destination_path: Optional[str] = None
|
|
exclude_patterns: Optional[List[str]] = None
|
|
enabled: Optional[bool] = None
|
|
|
|
class Job(JobBase):
|
|
id: int
|
|
created_at: datetime
|
|
updated_at: datetime
|
|
|
|
class Config:
|
|
from_attributes = True
|
|
|
|
# Schedule schemas
|
|
class ScheduleBase(BaseModel):
|
|
job_id: int
|
|
cron_expression: str
|
|
enabled: bool = True
|
|
|
|
class ScheduleCreate(ScheduleBase):
|
|
pass
|
|
|
|
class ScheduleUpdate(BaseModel):
|
|
cron_expression: Optional[str] = None
|
|
enabled: Optional[bool] = None
|
|
|
|
class Schedule(ScheduleBase):
|
|
id: int
|
|
created_at: datetime
|
|
|
|
class Config:
|
|
from_attributes = True
|
|
|
|
# Execution schemas
|
|
class JobExecutionBase(BaseModel):
|
|
job_id: int
|
|
status: str = Field(default="pending", pattern="^(pending|running|success|failed|cancelled)$")
|
|
triggered_by: str = Field(..., pattern="^(manual|schedule)$")
|
|
|
|
class JobExecutionCreate(JobExecutionBase):
|
|
pass
|
|
|
|
class JobExecution(JobExecutionBase):
|
|
id: int
|
|
started_at: Optional[datetime] = None
|
|
completed_at: Optional[datetime] = None
|
|
bytes_processed: int = 0
|
|
bytes_backed_up: int = 0
|
|
error_message: Optional[str] = None
|
|
|
|
class Config:
|
|
from_attributes = True
|
|
|
|
# Backup schemas
|
|
class BackupBase(BaseModel):
|
|
execution_id: int
|
|
storage_path: str
|
|
size_bytes: int = 0
|
|
checksum: Optional[str] = None
|
|
type: str = Field(..., pattern="^(full|incremental)$")
|
|
parent_backup_id: Optional[int] = None
|
|
|
|
class BackupCreate(BackupBase):
|
|
pass
|
|
|
|
class Backup(BackupBase):
|
|
id: int
|
|
created_at: datetime
|
|
|
|
class Config:
|
|
from_attributes = True
|
|
|
|
# Settings schemas
|
|
class SettingBase(BaseModel):
|
|
key: str
|
|
value: Optional[str] = None
|
|
|
|
class SettingCreate(SettingBase):
|
|
pass
|
|
|
|
class SettingUpdate(BaseModel):
|
|
value: Optional[str] = None
|
|
|
|
class Setting(SettingBase):
|
|
updated_at: datetime
|
|
|
|
class Config:
|
|
from_attributes = True
|
|
|
|
# Dashboard schemas
|
|
class DashboardStats(BaseModel):
|
|
active_jobs: int
|
|
total_backups: int
|
|
storage_used_bytes: int
|
|
recent_failures: int
|
|
recent_executions: List[JobExecution]
|
|
```
|
|
|
|
- [ ] **Step 2: Commit**
|
|
|
|
```bash
|
|
git add backend/app/schemas.py
|
|
git commit -m "feat: add pydantic schemas for API validation"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Source Adapter Base and Local Adapter
|
|
|
|
**Files:**
|
|
- Create: `backend/backup/adapters/base.py`
|
|
- Create: `backend/backup/adapters/local.py`
|
|
- Create: `backend/backup/adapters/__init__.py`
|
|
|
|
- [ ] **Step 1: Create abstract base adapter**
|
|
|
|
`backend/backup/adapters/base.py`:
|
|
```python
|
|
from abc import ABC, abstractmethod
|
|
from typing import List, Dict, Any, AsyncIterator
|
|
from dataclasses import dataclass
|
|
|
|
@dataclass
|
|
class FileInfo:
|
|
path: str
|
|
size: int
|
|
modified_time: float
|
|
is_directory: bool
|
|
|
|
class SourceAdapter(ABC):
|
|
def __init__(self, config: Dict[str, Any]):
|
|
self.config = config
|
|
|
|
@abstractmethod
|
|
async def connect(self) -> None:
|
|
"""Establish connection to source."""
|
|
pass
|
|
|
|
@abstractmethod
|
|
async def disconnect(self) -> None:
|
|
"""Close connection to source."""
|
|
pass
|
|
|
|
@abstractmethod
|
|
async def list_files(self, path: str = "") -> List[FileInfo]:
|
|
"""List files at given path."""
|
|
pass
|
|
|
|
@abstractmethod
|
|
async def read_file(self, path: str) -> AsyncIterator[bytes]:
|
|
"""Read file in chunks."""
|
|
pass
|
|
|
|
@abstractmethod
|
|
async def get_database_dump(self, config: Dict[str, Any]) -> AsyncIterator[bytes]:
|
|
"""Get database dump. Only implemented for database adapters."""
|
|
pass
|
|
```
|
|
|
|
- [ ] **Step 2: Create local filesystem adapter**
|
|
|
|
`backend/backup/adapters/local.py`:
|
|
```python
|
|
import os
|
|
import aiofiles
|
|
from pathlib import Path
|
|
from typing import List, AsyncIterator
|
|
from .base import SourceAdapter, FileInfo
|
|
|
|
class LocalAdapter(SourceAdapter):
|
|
async def connect(self) -> None:
|
|
base_path = self.config.get("path", ".")
|
|
if not os.path.exists(base_path):
|
|
raise FileNotFoundError(f"Path not found: {base_path}")
|
|
|
|
async def disconnect(self) -> None:
|
|
pass
|
|
|
|
async def list_files(self, path: str = "") -> List[FileInfo]:
|
|
base_path = Path(self.config.get("path", "."))
|
|
target_path = base_path / path if path else base_path
|
|
|
|
files = []
|
|
exclude_patterns = self.config.get("exclude", [])
|
|
|
|
for item in target_path.iterdir():
|
|
# Check exclude patterns
|
|
if any(item.match(pattern) for pattern in exclude_patterns):
|
|
continue
|
|
|
|
stat = item.stat()
|
|
files.append(FileInfo(
|
|
path=str(item.relative_to(base_path)),
|
|
size=stat.st_size,
|
|
modified_time=stat.st_mtime,
|
|
is_directory=item.is_dir()
|
|
))
|
|
|
|
return files
|
|
|
|
async def read_file(self, path: str) -> AsyncIterator[bytes]:
|
|
base_path = Path(self.config.get("path", "."))
|
|
file_path = base_path / path
|
|
|
|
async with aiofiles.open(file_path, "rb") as f:
|
|
while chunk := await f.read(8192):
|
|
yield chunk
|
|
|
|
async def get_database_dump(self, config: Dict[str, Any]) -> AsyncIterator[bytes]:
|
|
raise NotImplementedError("Local adapter does not support database dumps")
|
|
```
|
|
|
|
- [ ] **Step 3: Create adapter factory**
|
|
|
|
`backend/backup/adapters/__init__.py`:
|
|
```python
|
|
from typing import Dict, Any
|
|
from .base import SourceAdapter
|
|
from .local import LocalAdapter
|
|
|
|
ADAPTER_MAP = {
|
|
"local": LocalAdapter,
|
|
}
|
|
|
|
def get_adapter(source_type: str, config: Dict[str, Any]) -> SourceAdapter:
|
|
adapter_class = ADAPTER_MAP.get(source_type)
|
|
if not adapter_class:
|
|
raise ValueError(f"Unknown source type: {source_type}")
|
|
return adapter_class(config)
|
|
```
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add backend/backup/adapters/
|
|
git commit -m "feat: add source adapter base class and local filesystem adapter
|
|
|
|
- Abstract SourceAdapter with FileInfo dataclass
|
|
- LocalAdapter for filesystem backups
|
|
- Adapter factory for extensibility"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Backup Engine Core
|
|
|
|
**Files:**
|
|
- Create: `backend/backup/engine.py`
|
|
- Create: `backend/tests/test_engine.py`
|
|
|
|
- [ ] **Step 1: Create backup engine**
|
|
|
|
`backend/backup/engine.py`:
|
|
```python
|
|
import os
|
|
import hashlib
|
|
import shutil
|
|
from datetime import datetime
|
|
from pathlib import Path
|
|
from typing import Optional
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from app.models import Job, JobExecution, Backup
|
|
from backup.adapters import get_adapter
|
|
|
|
class BackupEngine:
|
|
def __init__(self, db: AsyncSession):
|
|
self.db = db
|
|
|
|
async def execute_job(self, job_id: int, triggered_by: str = "manual") -> JobExecution:
|
|
# Create execution record
|
|
execution = JobExecution(
|
|
job_id=job_id,
|
|
status="pending",
|
|
triggered_by=triggered_by
|
|
)
|
|
self.db.add(execution)
|
|
await self.db.commit()
|
|
await self.db.refresh(execution)
|
|
|
|
try:
|
|
# Load job with source
|
|
from sqlalchemy import select
|
|
result = await self.db.execute(
|
|
select(Job).where(Job.id == job_id)
|
|
)
|
|
job = result.scalar_one()
|
|
|
|
# Update status to running
|
|
execution.status = "running"
|
|
execution.started_at = datetime.utcnow()
|
|
await self.db.commit()
|
|
|
|
# Determine strategy
|
|
strategy = job.strategy
|
|
parent_backup_id = None
|
|
|
|
if strategy == "incremental":
|
|
# Find last successful full backup
|
|
result = await self.db.execute(
|
|
select(Backup)
|
|
.join(JobExecution)
|
|
.where(
|
|
JobExecution.job_id == job_id,
|
|
JobExecution.status == "success",
|
|
Backup.type == "full"
|
|
)
|
|
.order_by(Backup.created_at.desc())
|
|
)
|
|
last_full = result.scalar_one_or_none()
|
|
|
|
if last_full:
|
|
parent_backup_id = last_full.id
|
|
else:
|
|
# No full backup exists, do full instead
|
|
strategy = "full"
|
|
|
|
# Create backup directory
|
|
timestamp = datetime.utcnow().strftime("%Y-%m-%d_%H%M%S")
|
|
backup_dir = Path(job.destination_path) / str(job_id) / f"{timestamp}_{strategy}"
|
|
backup_dir.mkdir(parents=True, exist_ok=True)
|
|
|
|
# Get adapter and connect
|
|
adapter = get_adapter(job.source.type, job.source.config)
|
|
await adapter.connect()
|
|
|
|
try:
|
|
# Copy files
|
|
total_processed = 0
|
|
total_backed_up = 0
|
|
|
|
source_path = Path(job.source.config.get("path", "."))
|
|
|
|
for item in source_path.rglob("*"):
|
|
if item.is_file():
|
|
rel_path = item.relative_to(source_path)
|
|
dest_path = backup_dir / "data" / rel_path
|
|
dest_path.parent.mkdir(parents=True, exist_ok=True)
|
|
|
|
# Copy file
|
|
shutil.copy2(item, dest_path)
|
|
|
|
size = item.stat().st_size
|
|
total_processed += size
|
|
total_backed_up += size
|
|
|
|
# Calculate checksum
|
|
checksum = await self._calculate_checksum(backup_dir)
|
|
|
|
# Create backup record
|
|
backup = Backup(
|
|
execution_id=execution.id,
|
|
storage_path=str(backup_dir),
|
|
size_bytes=total_backed_up,
|
|
checksum=checksum,
|
|
type=strategy,
|
|
parent_backup_id=parent_backup_id
|
|
)
|
|
self.db.add(backup)
|
|
|
|
# Update execution
|
|
execution.status = "success"
|
|
execution.completed_at = datetime.utcnow()
|
|
execution.bytes_processed = total_processed
|
|
execution.bytes_backed_up = total_backed_up
|
|
|
|
finally:
|
|
await adapter.disconnect()
|
|
|
|
except Exception as e:
|
|
execution.status = "failed"
|
|
execution.completed_at = datetime.utcnow()
|
|
execution.error_message = str(e)
|
|
|
|
await self.db.commit()
|
|
return execution
|
|
|
|
async def _calculate_checksum(self, path: Path) -> str:
|
|
hasher = hashlib.sha256()
|
|
for item in sorted(path.rglob("*")):
|
|
if item.is_file():
|
|
with open(item, "rb") as f:
|
|
while chunk := f.read(8192):
|
|
hasher.update(chunk)
|
|
return hasher.hexdigest()
|
|
```
|
|
|
|
- [ ] **Step 2: Write test for backup engine**
|
|
|
|
`backend/tests/test_engine.py`:
|
|
```python
|
|
import pytest
|
|
import tempfile
|
|
import os
|
|
from pathlib import Path
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from app.models import Source, Job, JobExecution
|
|
from backup.engine import BackupEngine
|
|
|
|
@pytest.fixture
|
|
async def test_source(db: AsyncSession):
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
# Create test files
|
|
(Path(tmpdir) / "test.txt").write_text("Hello, World!")
|
|
(Path(tmpdir) / "subdir").mkdir()
|
|
(Path(tmpdir) / "subdir" / "nested.txt").write_text("Nested content")
|
|
|
|
source = Source(
|
|
name="Test Source",
|
|
type="local",
|
|
config={"path": tmpdir}
|
|
)
|
|
db.add(source)
|
|
await db.commit()
|
|
await db.refresh(source)
|
|
yield source
|
|
|
|
@pytest.fixture
|
|
async def test_job(db: AsyncSession, test_source):
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
job = Job(
|
|
name="Test Job",
|
|
source_id=test_source.id,
|
|
strategy="full",
|
|
destination_path=tmpdir
|
|
)
|
|
db.add(job)
|
|
await db.commit()
|
|
await db.refresh(job)
|
|
yield job
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_execute_full_backup(db: AsyncSession, test_job):
|
|
engine = BackupEngine(db)
|
|
execution = await engine.execute_job(test_job.id, triggered_by="manual")
|
|
|
|
assert execution.status == "success"
|
|
assert execution.bytes_processed > 0
|
|
assert execution.bytes_backed_up > 0
|
|
assert execution.triggered_by == "manual"
|
|
|
|
# Verify backup was created
|
|
assert len(test_job.executions) == 1
|
|
backup = test_job.executions[0].backups[0]
|
|
assert backup.type == "full"
|
|
assert backup.checksum is not None
|
|
assert os.path.exists(backup.storage_path)
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_execute_incremental_without_full(db: AsyncSession, test_job):
|
|
# Set job to incremental but no full backup exists
|
|
test_job.strategy = "incremental"
|
|
await db.commit()
|
|
|
|
engine = BackupEngine(db)
|
|
execution = await engine.execute_job(test_job.id)
|
|
|
|
# Should fall back to full backup
|
|
assert execution.status == "success"
|
|
backup = execution.backups[0]
|
|
assert backup.type == "full"
|
|
assert backup.parent_backup_id is None
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add backend/backup/engine.py backend/tests/test_engine.py
|
|
git commit -m "feat: add backup engine with full/incremental support
|
|
|
|
- BackupEngine class executes jobs and creates backups
|
|
- Automatic fallback from incremental to full if no parent exists
|
|
- SHA-256 checksum calculation for integrity
|
|
- Tests for full backup and incremental fallback"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: API Routers - Sources
|
|
|
|
**Files:**
|
|
- Create: `backend/app/routers/sources.py`
|
|
- Create: `backend/tests/test_sources.py`
|
|
|
|
- [ ] **Step 1: Create sources router**
|
|
|
|
`backend/app/routers/sources.py`:
|
|
```python
|
|
from fastapi import APIRouter, Depends, HTTPException
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy import select
|
|
from typing import List
|
|
from app.database import get_db
|
|
from app.models import Source
|
|
from app.schemas import SourceCreate, SourceUpdate, Source as SourceSchema
|
|
|
|
router = APIRouter(prefix="/api/sources", tags=["sources"])
|
|
|
|
@router.get("/", response_model=List[SourceSchema])
|
|
async def list_sources(db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Source))
|
|
sources = result.scalars().all()
|
|
return sources
|
|
|
|
@router.post("/", response_model=SourceSchema)
|
|
async def create_source(source: SourceCreate, db: AsyncSession = Depends(get_db)):
|
|
db_source = Source(**source.model_dump())
|
|
db.add(db_source)
|
|
await db.commit()
|
|
await db.refresh(db_source)
|
|
return db_source
|
|
|
|
@router.get("/{source_id}", response_model=SourceSchema)
|
|
async def get_source(source_id: int, db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Source).where(Source.id == source_id))
|
|
source = result.scalar_one_or_none()
|
|
if not source:
|
|
raise HTTPException(status_code=404, detail="Source not found")
|
|
return source
|
|
|
|
@router.put("/{source_id}", response_model=SourceSchema)
|
|
async def update_source(
|
|
source_id: int,
|
|
source_update: SourceUpdate,
|
|
db: AsyncSession = Depends(get_db)
|
|
):
|
|
result = await db.execute(select(Source).where(Source.id == source_id))
|
|
source = result.scalar_one_or_none()
|
|
if not source:
|
|
raise HTTPException(status_code=404, detail="Source not found")
|
|
|
|
update_data = source_update.model_dump(exclude_unset=True)
|
|
for field, value in update_data.items():
|
|
setattr(source, field, value)
|
|
|
|
await db.commit()
|
|
await db.refresh(source)
|
|
return source
|
|
|
|
@router.delete("/{source_id}")
|
|
async def delete_source(source_id: int, db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Source).where(Source.id == source_id))
|
|
source = result.scalar_one_or_none()
|
|
if not source:
|
|
raise HTTPException(status_code=404, detail="Source not found")
|
|
|
|
await db.delete(source)
|
|
await db.commit()
|
|
return {"message": "Source deleted"}
|
|
```
|
|
|
|
- [ ] **Step 2: Create sources tests**
|
|
|
|
`backend/tests/test_sources.py`:
|
|
```python
|
|
import pytest
|
|
from httpx import AsyncClient
|
|
from app.main import app
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_create_source():
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
response = await ac.post("/api/sources/", json={
|
|
"name": "Test Source",
|
|
"type": "local",
|
|
"config": {"path": "/tmp/test"}
|
|
})
|
|
assert response.status_code == 200
|
|
data = response.json()
|
|
assert data["name"] == "Test Source"
|
|
assert data["type"] == "local"
|
|
assert "id" in data
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_list_sources():
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
# Create source first
|
|
await ac.post("/api/sources/", json={
|
|
"name": "Test Source",
|
|
"type": "local",
|
|
"config": {"path": "/tmp/test"}
|
|
})
|
|
|
|
response = await ac.get("/api/sources/")
|
|
assert response.status_code == 200
|
|
data = response.json()
|
|
assert len(data) >= 1
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_source():
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
create_resp = await ac.post("/api/sources/", json={
|
|
"name": "Test Source",
|
|
"type": "local",
|
|
"config": {"path": "/tmp/test"}
|
|
})
|
|
source_id = create_resp.json()["id"]
|
|
|
|
response = await ac.get(f"/api/sources/{source_id}")
|
|
assert response.status_code == 200
|
|
assert response.json()["id"] == source_id
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_delete_source():
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
create_resp = await ac.post("/api/sources/", json={
|
|
"name": "Delete Me",
|
|
"type": "local",
|
|
"config": {"path": "/tmp/test"}
|
|
})
|
|
source_id = create_resp.json()["id"]
|
|
|
|
response = await ac.delete(f"/api/sources/{source_id}")
|
|
assert response.status_code == 200
|
|
|
|
# Verify deletion
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
get_resp = await ac.get(f"/api/sources/{source_id}")
|
|
assert get_resp.status_code == 404
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add backend/app/routers/sources.py backend/tests/test_sources.py
|
|
git commit -m "feat: add sources CRUD API with tests
|
|
|
|
- Full CRUD endpoints for backup sources
|
|
- Pydantic validation for source types
|
|
- pytest tests for create, list, get, delete"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: API Routers - Jobs
|
|
|
|
**Files:**
|
|
- Create: `backend/app/routers/jobs.py`
|
|
- Create: `backend/tests/test_jobs.py`
|
|
|
|
- [ ] **Step 1: Create jobs router**
|
|
|
|
`backend/app/routers/jobs.py`:
|
|
```python
|
|
from fastapi import APIRouter, Depends, HTTPException, BackgroundTasks
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy import select
|
|
from typing import List
|
|
from app.database import get_db
|
|
from app.models import Job, Schedule
|
|
from app.schemas import JobCreate, JobUpdate, Job as JobSchema, ScheduleCreate
|
|
from backup.engine import BackupEngine
|
|
|
|
router = APIRouter(prefix="/api/jobs", tags=["jobs"])
|
|
|
|
@router.get("/", response_model=List[JobSchema])
|
|
async def list_jobs(db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Job))
|
|
jobs = result.scalars().all()
|
|
return jobs
|
|
|
|
@router.post("/", response_model=JobSchema)
|
|
async def create_job(job: JobCreate, db: AsyncSession = Depends(get_db)):
|
|
db_job = Job(**job.model_dump())
|
|
db.add(db_job)
|
|
await db.commit()
|
|
await db.refresh(db_job)
|
|
return db_job
|
|
|
|
@router.get("/{job_id}", response_model=JobSchema)
|
|
async def get_job(job_id: int, db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Job).where(Job.id == job_id))
|
|
job = result.scalar_one_or_none()
|
|
if not job:
|
|
raise HTTPException(status_code=404, detail="Job not found")
|
|
return job
|
|
|
|
@router.put("/{job_id}", response_model=JobSchema)
|
|
async def update_job(
|
|
job_id: int,
|
|
job_update: JobUpdate,
|
|
db: AsyncSession = Depends(get_db)
|
|
):
|
|
result = await db.execute(select(Job).where(Job.id == job_id))
|
|
job = result.scalar_one_or_none()
|
|
if not job:
|
|
raise HTTPException(status_code=404, detail="Job not found")
|
|
|
|
update_data = job_update.model_dump(exclude_unset=True)
|
|
for field, value in update_data.items():
|
|
setattr(job, field, value)
|
|
|
|
await db.commit()
|
|
await db.refresh(job)
|
|
return job
|
|
|
|
@router.delete("/{job_id}")
|
|
async def delete_job(job_id: int, db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Job).where(Job.id == job_id))
|
|
job = result.scalar_one_or_none()
|
|
if not job:
|
|
raise HTTPException(status_code=404, detail="Job not found")
|
|
|
|
await db.delete(job)
|
|
await db.commit()
|
|
return {"message": "Job deleted"}
|
|
|
|
@router.post("/{job_id}/run")
|
|
async def run_job(
|
|
job_id: int,
|
|
background_tasks: BackgroundTasks,
|
|
db: AsyncSession = Depends(get_db)
|
|
):
|
|
result = await db.execute(select(Job).where(Job.id == job_id))
|
|
job = result.scalar_one_or_none()
|
|
if not job:
|
|
raise HTTPException(status_code=404, detail="Job not found")
|
|
|
|
# Run in background
|
|
async def execute():
|
|
engine = BackupEngine(db)
|
|
await engine.execute_job(job_id, triggered_by="manual")
|
|
|
|
background_tasks.add_task(execute)
|
|
return {"message": "Job execution started"}
|
|
|
|
@router.post("/{job_id}/schedule", response_model=JobSchema)
|
|
async def create_schedule(
|
|
job_id: int,
|
|
schedule: ScheduleCreate,
|
|
db: AsyncSession = Depends(get_db)
|
|
):
|
|
result = await db.execute(select(Job).where(Job.id == job_id))
|
|
job = result.scalar_one_or_none()
|
|
if not job:
|
|
raise HTTPException(status_code=404, detail="Job not found")
|
|
|
|
db_schedule = Schedule(**schedule.model_dump())
|
|
db.add(db_schedule)
|
|
await db.commit()
|
|
await db.refresh(job)
|
|
return job
|
|
```
|
|
|
|
- [ ] **Step 2: Create jobs tests**
|
|
|
|
`backend/tests/test_jobs.py`:
|
|
```python
|
|
import pytest
|
|
from httpx import AsyncClient
|
|
from app.main import app
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_create_job():
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
# Create source first
|
|
source_resp = await ac.post("/api/sources/", json={
|
|
"name": "Test Source",
|
|
"type": "local",
|
|
"config": {"path": "/tmp/test"}
|
|
})
|
|
source_id = source_resp.json()["id"]
|
|
|
|
response = await ac.post("/api/jobs/", json={
|
|
"name": "Test Job",
|
|
"source_id": source_id,
|
|
"strategy": "full",
|
|
"destination_path": "/tmp/backups"
|
|
})
|
|
assert response.status_code == 200
|
|
data = response.json()
|
|
assert data["name"] == "Test Job"
|
|
assert data["strategy"] == "full"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_run_job():
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
# Create source and job
|
|
source_resp = await ac.post("/api/sources/", json={
|
|
"name": "Test Source",
|
|
"type": "local",
|
|
"config": {"path": "/tmp/test"}
|
|
})
|
|
source_id = source_resp.json()["id"]
|
|
|
|
job_resp = await ac.post("/api/jobs/", json={
|
|
"name": "Test Job",
|
|
"source_id": source_id,
|
|
"strategy": "full",
|
|
"destination_path": "/tmp/backups"
|
|
})
|
|
job_id = job_resp.json()["id"]
|
|
|
|
response = await ac.post(f"/api/jobs/{job_id}/run")
|
|
assert response.status_code == 200
|
|
assert "started" in response.json()["message"].lower()
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add backend/app/routers/jobs.py backend/tests/test_jobs.py
|
|
git commit -m "feat: add jobs CRUD API with manual execution
|
|
|
|
- Full CRUD for backup jobs
|
|
- Manual job trigger endpoint with background execution
|
|
- Schedule creation endpoint
|
|
- Tests for job creation and execution"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 8: Main FastAPI Application
|
|
|
|
**Files:**
|
|
- Create: `backend/app/main.py`
|
|
- Create: `backend/tests/conftest.py`
|
|
|
|
- [ ] **Step 1: Create main FastAPI app**
|
|
|
|
`backend/app/main.py`:
|
|
```python
|
|
from fastapi import FastAPI
|
|
from fastapi.middleware.cors import CORSMiddleware
|
|
from app.routers import sources, jobs, executions, backups, settings, dashboard
|
|
from app.database import engine, Base
|
|
|
|
app = FastAPI(title="Backup Tool API", version="0.1.0")
|
|
|
|
# CORS
|
|
app.add_middleware(
|
|
CORSMiddleware,
|
|
allow_origins=["http://localhost:3000"],
|
|
allow_credentials=True,
|
|
allow_methods=["*"],
|
|
allow_headers=["*"],
|
|
)
|
|
|
|
# Include routers
|
|
app.include_router(sources.router)
|
|
app.include_router(jobs.router)
|
|
app.include_router(executions.router)
|
|
app.include_router(backups.router)
|
|
app.include_router(settings.router)
|
|
app.include_router(dashboard.router)
|
|
|
|
@app.on_event("startup")
|
|
async def startup():
|
|
async with engine.begin() as conn:
|
|
await conn.run_sync(Base.metadata.create_all)
|
|
|
|
@app.get("/api/health")
|
|
async def health_check():
|
|
return {"status": "healthy"}
|
|
```
|
|
|
|
- [ ] **Step 2: Create test fixtures**
|
|
|
|
`backend/tests/conftest.py`:
|
|
```python
|
|
import pytest
|
|
import pytest_asyncio
|
|
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
|
|
from app.database import Base, get_db
|
|
from app.main import app
|
|
from httpx import AsyncClient
|
|
|
|
TEST_DATABASE_URL = "sqlite+aiosqlite:///./test.db"
|
|
|
|
@pytest_asyncio.fixture
|
|
async def db():
|
|
engine = create_async_engine(TEST_DATABASE_URL, echo=False)
|
|
async with engine.begin() as conn:
|
|
await conn.run_sync(Base.metadata.create_all)
|
|
|
|
async_session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
|
async with async_session() as session:
|
|
yield session
|
|
|
|
async with engine.begin() as conn:
|
|
await conn.run_sync(Base.metadata.drop_all)
|
|
await engine.dispose()
|
|
|
|
@pytest_asyncio.fixture
|
|
async def client(db):
|
|
async def override_get_db():
|
|
yield db
|
|
|
|
app.dependency_overrides[get_db] = override_get_db
|
|
async with AsyncClient(app=app, base_url="http://test") as ac:
|
|
yield ac
|
|
app.dependency_overrides.clear()
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add backend/app/main.py backend/tests/conftest.py
|
|
git commit -m "feat: setup FastAPI main app with routers and test fixtures
|
|
|
|
- Main FastAPI app with all routers included
|
|
- CORS middleware for frontend
|
|
- Test fixtures with in-memory SQLite database"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 9: Frontend - API Client and Layout
|
|
|
|
**Files:**
|
|
- Create: `frontend/src/api/client.ts`
|
|
- Create: `frontend/src/components/Layout.tsx`
|
|
- Create: `frontend/src/App.tsx`
|
|
|
|
- [ ] **Step 1: Create API client**
|
|
|
|
`frontend/src/api/client.ts`:
|
|
```typescript
|
|
import axios from 'axios';
|
|
|
|
export const api = axios.create({
|
|
baseURL: '/api',
|
|
headers: {
|
|
'Content-Type': 'application/json',
|
|
},
|
|
});
|
|
|
|
export const sourcesApi = {
|
|
list: () => api.get('/sources/'),
|
|
create: (data: any) => api.post('/sources/', data),
|
|
get: (id: number) => api.get(`/sources/${id}`),
|
|
update: (id: number, data: any) => api.put(`/sources/${id}`, data),
|
|
delete: (id: number) => api.delete(`/sources/${id}`),
|
|
};
|
|
|
|
export const jobsApi = {
|
|
list: () => api.get('/jobs/'),
|
|
create: (data: any) => api.post('/jobs/', data),
|
|
get: (id: number) => api.get(`/jobs/${id}`),
|
|
update: (id: number, data: any) => api.put(`/jobs/${id}`, data),
|
|
delete: (id: number) => api.delete(`/jobs/${id}`),
|
|
run: (id: number) => api.post(`/jobs/${id}/run`),
|
|
};
|
|
|
|
export const dashboardApi = {
|
|
getStats: () => api.get('/dashboard/'),
|
|
};
|
|
```
|
|
|
|
- [ ] **Step 2: Create layout component**
|
|
|
|
`frontend/src/components/Layout.tsx`:
|
|
```typescript
|
|
import { Link, useLocation } from 'react-router-dom';
|
|
import { LayoutDashboard, Database, Settings } from 'lucide-react';
|
|
|
|
export function Layout({ children }: { children: React.ReactNode }) {
|
|
const location = useLocation();
|
|
|
|
const navItems = [
|
|
{ path: '/', icon: LayoutDashboard, label: 'Dashboard' },
|
|
{ path: '/backups', icon: Database, label: 'Backups' },
|
|
{ path: '/settings', icon: Settings, label: 'Settings' },
|
|
];
|
|
|
|
return (
|
|
<div className="min-h-screen bg-gray-50">
|
|
<nav className="bg-white shadow-sm border-b">
|
|
<div className="max-w-7xl mx-auto px-4">
|
|
<div className="flex items-center h-16">
|
|
<div className="flex items-center gap-2 mr-8">
|
|
<Database className="w-6 h-6 text-blue-600" />
|
|
<span className="font-bold text-lg">Backup Tool</span>
|
|
</div>
|
|
|
|
<div className="flex gap-4">
|
|
{navItems.map((item) => (
|
|
<Link
|
|
key={item.path}
|
|
to={item.path}
|
|
className={`flex items-center gap-2 px-3 py-2 rounded-md text-sm font-medium ${
|
|
location.pathname === item.path
|
|
? 'bg-blue-50 text-blue-700'
|
|
: 'text-gray-600 hover:text-gray-900'
|
|
}`}
|
|
>
|
|
<item.icon className="w-4 h-4" />
|
|
{item.label}
|
|
</Link>
|
|
))}
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</nav>
|
|
|
|
<main className="max-w-7xl mx-auto px-4 py-8">
|
|
{children}
|
|
</main>
|
|
</div>
|
|
);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Create App component with routing**
|
|
|
|
`frontend/src/App.tsx`:
|
|
```typescript
|
|
import { BrowserRouter, Routes, Route } from 'react-router-dom';
|
|
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
|
|
import { Layout } from './components/Layout';
|
|
import { Dashboard } from './pages/Dashboard';
|
|
import { Backups } from './pages/Backups';
|
|
import { Settings } from './pages/Settings';
|
|
|
|
const queryClient = new QueryClient();
|
|
|
|
function App() {
|
|
return (
|
|
<QueryClientProvider client={queryClient}>
|
|
<BrowserRouter>
|
|
<Layout>
|
|
<Routes>
|
|
<Route path="/" element={<Dashboard />} />
|
|
<Route path="/backups" element={<Backups />} />
|
|
<Route path="/settings" element={<Settings />} />
|
|
</Routes>
|
|
</Layout>
|
|
</BrowserRouter>
|
|
</QueryClientProvider>
|
|
);
|
|
}
|
|
|
|
export default App;
|
|
```
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add frontend/src/api/client.ts frontend/src/components/Layout.tsx frontend/src/App.tsx
|
|
git commit -m "feat: add frontend API client and layout components
|
|
|
|
- Axios-based API client with endpoints for sources, jobs, dashboard
|
|
- Layout component with navigation
|
|
- React Router setup with three main views"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 10: Frontend - Dashboard Page
|
|
|
|
**Files:**
|
|
- Create: `frontend/src/pages/Dashboard.tsx`
|
|
|
|
- [ ] **Step 1: Create dashboard page**
|
|
|
|
`frontend/src/pages/Dashboard.tsx`:
|
|
```typescript
|
|
import { useQuery } from '@tanstack/react-query';
|
|
import { dashboardApi } from '../api/client';
|
|
import { Activity, Database, HardDrive, AlertCircle } from 'lucide-react';
|
|
|
|
export function Dashboard() {
|
|
const { data: stats, isLoading } = useQuery({
|
|
queryKey: ['dashboard'],
|
|
queryFn: () => dashboardApi.getStats().then((r) => r.data),
|
|
});
|
|
|
|
if (isLoading) {
|
|
return <div className="text-center py-8">Loading...</div>;
|
|
}
|
|
|
|
const statCards = [
|
|
{
|
|
title: 'Active Jobs',
|
|
value: stats?.active_jobs || 0,
|
|
icon: Activity,
|
|
color: 'text-green-600',
|
|
bgColor: 'bg-green-50',
|
|
},
|
|
{
|
|
title: 'Total Backups',
|
|
value: stats?.total_backups || 0,
|
|
icon: Database,
|
|
color: 'text-blue-600',
|
|
bgColor: 'bg-blue-50',
|
|
},
|
|
{
|
|
title: 'Storage Used',
|
|
value: formatBytes(stats?.storage_used_bytes || 0),
|
|
icon: HardDrive,
|
|
color: 'text-yellow-600',
|
|
bgColor: 'bg-yellow-50',
|
|
},
|
|
{
|
|
title: 'Recent Failures',
|
|
value: stats?.recent_failures || 0,
|
|
icon: AlertCircle,
|
|
color: 'text-red-600',
|
|
bgColor: 'bg-red-50',
|
|
},
|
|
];
|
|
|
|
return (
|
|
<div className="space-y-6">
|
|
<h1 className="text-2xl font-bold">Dashboard</h1>
|
|
|
|
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
|
|
{statCards.map((card) => (
|
|
<div key={card.title} className="bg-white rounded-lg shadow p-6">
|
|
<div className="flex items-center justify-between">
|
|
<div>
|
|
<p className="text-sm text-gray-600">{card.title}</p>
|
|
<p className="text-2xl font-bold mt-1">{card.value}</p>
|
|
</div>
|
|
<div className={`${card.bgColor} p-3 rounded-lg`}>
|
|
<card.icon className={`w-6 h-6 ${card.color}`} />
|
|
</div>
|
|
</div>
|
|
</div>
|
|
))}
|
|
</div>
|
|
|
|
<div className="bg-white rounded-lg shadow">
|
|
<div className="px-6 py-4 border-b">
|
|
<h2 className="font-semibold">Recent Activity</h2>
|
|
</div>
|
|
<div className="divide-y">
|
|
{(stats?.recent_executions || []).map((execution: any) => (
|
|
<div key={execution.id} className="px-6 py-4 flex items-center justify-between">
|
|
<div>
|
|
<p className="font-medium">Job #{execution.job_id}</p>
|
|
<p className="text-sm text-gray-600">
|
|
{execution.triggered_by} • {execution.status}
|
|
</p>
|
|
</div>
|
|
<StatusBadge status={execution.status} />
|
|
</div>
|
|
))}
|
|
</div>
|
|
</div>
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function StatusBadge({ status }: { status: string }) {
|
|
const colors: Record<string, string> = {
|
|
success: 'bg-green-100 text-green-800',
|
|
failed: 'bg-red-100 text-red-800',
|
|
running: 'bg-blue-100 text-blue-800',
|
|
pending: 'bg-gray-100 text-gray-800',
|
|
};
|
|
|
|
return (
|
|
<span className={`px-2 py-1 rounded-full text-xs font-medium ${colors[status] || colors.pending}`}>
|
|
{status}
|
|
</span>
|
|
);
|
|
}
|
|
|
|
function formatBytes(bytes: number): string {
|
|
if (bytes === 0) return '0 B';
|
|
const k = 1024;
|
|
const sizes = ['B', 'KB', 'MB', 'GB', 'TB'];
|
|
const i = Math.floor(Math.log(bytes) / Math.log(k));
|
|
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i];
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Commit**
|
|
|
|
```bash
|
|
git add frontend/src/pages/Dashboard.tsx
|
|
git commit -m "feat: add dashboard page with stats and activity feed
|
|
|
|
- Stats cards for active jobs, backups, storage, failures
|
|
- Recent activity list with status badges
|
|
- Byte formatting utility"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 11: Frontend - Backups Page
|
|
|
|
**Files:**
|
|
- Create: `frontend/src/pages/Backups.tsx`
|
|
- Create: `frontend/src/components/JobForm.tsx`
|
|
- Create: `frontend/src/components/SourceForm.tsx`
|
|
|
|
- [ ] **Step 1: Create backups page**
|
|
|
|
`frontend/src/pages/Backups.tsx`:
|
|
```typescript
|
|
import { useState } from 'react';
|
|
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
|
|
import { jobsApi, sourcesApi } from '../api/client';
|
|
import { Play, Plus, Trash2, Clock } from 'lucide-react';
|
|
|
|
export function Backups() {
|
|
const [showJobForm, setShowJobForm] = useState(false);
|
|
const queryClient = useQueryClient();
|
|
|
|
const { data: jobs } = useQuery({
|
|
queryKey: ['jobs'],
|
|
queryFn: () => jobsApi.list().then((r) => r.data),
|
|
});
|
|
|
|
const { data: sources } = useQuery({
|
|
queryKey: ['sources'],
|
|
queryFn: () => sourcesApi.list().then((r) => r.data),
|
|
});
|
|
|
|
const runMutation = useMutation({
|
|
mutationFn: (id: number) => jobsApi.run(id),
|
|
onSuccess: () => queryClient.invalidateQueries({ queryKey: ['jobs'] }),
|
|
});
|
|
|
|
const deleteMutation = useMutation({
|
|
mutationFn: (id: number) => jobsApi.delete(id),
|
|
onSuccess: () => queryClient.invalidateQueries({ queryKey: ['jobs'] }),
|
|
});
|
|
|
|
return (
|
|
<div className="space-y-6">
|
|
<div className="flex justify-between items-center">
|
|
<h1 className="text-2xl font-bold">Backups</h1>
|
|
<button
|
|
onClick={() => setShowJobForm(true)}
|
|
className="flex items-center gap-2 bg-blue-600 text-white px-4 py-2 rounded-lg hover:bg-blue-700"
|
|
>
|
|
<Plus className="w-4 h-4" />
|
|
New Job
|
|
</button>
|
|
</div>
|
|
|
|
<div className="bg-white rounded-lg shadow overflow-hidden">
|
|
<table className="w-full">
|
|
<thead className="bg-gray-50">
|
|
<tr>
|
|
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Name</th>
|
|
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Source</th>
|
|
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Strategy</th>
|
|
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Status</th>
|
|
<th className="px-6 py-3 text-right text-xs font-medium text-gray-500 uppercase">Actions</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody className="divide-y">
|
|
{(jobs || []).map((job: any) => (
|
|
<tr key={job.id}>
|
|
<td className="px-6 py-4 font-medium">{job.name}</td>
|
|
<td className="px-6 py-4 text-gray-600">
|
|
{sources?.find((s: any) => s.id === job.source_id)?.name || 'Unknown'}
|
|
</td>
|
|
<td className="px-6 py-4">
|
|
<span className={`px-2 py-1 rounded-full text-xs ${
|
|
job.strategy === 'full' ? 'bg-orange-100 text-orange-800' : 'bg-blue-100 text-blue-800'
|
|
}`}>
|
|
{job.strategy}
|
|
</span>
|
|
</td>
|
|
<td className="px-6 py-4">
|
|
<span className={`px-2 py-1 rounded-full text-xs ${
|
|
job.enabled ? 'bg-green-100 text-green-800' : 'bg-gray-100 text-gray-800'
|
|
}`}>
|
|
{job.enabled ? 'Active' : 'Disabled'}
|
|
</span>
|
|
</td>
|
|
<td className="px-6 py-4 text-right">
|
|
<button
|
|
onClick={() => runMutation.mutate(job.id)}
|
|
className="text-blue-600 hover:text-blue-800 mr-2"
|
|
title="Run now"
|
|
>
|
|
<Play className="w-4 h-4" />
|
|
</button>
|
|
<button
|
|
onClick={() => deleteMutation.mutate(job.id)}
|
|
className="text-red-600 hover:text-red-800"
|
|
title="Delete"
|
|
>
|
|
<Trash2 className="w-4 h-4" />
|
|
</button>
|
|
</td>
|
|
</tr>
|
|
))}
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
{showJobForm && (
|
|
<JobFormModal
|
|
sources={sources || []}
|
|
onClose={() => setShowJobForm(false)}
|
|
/>
|
|
)}
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function JobFormModal({ sources, onClose }: { sources: any[]; onClose: () => void }) {
|
|
const queryClient = useQueryClient();
|
|
const createMutation = useMutation({
|
|
mutationFn: (data: any) => jobsApi.create(data),
|
|
onSuccess: () => {
|
|
queryClient.invalidateQueries({ queryKey: ['jobs'] });
|
|
onClose();
|
|
},
|
|
});
|
|
|
|
const handleSubmit = (e: React.FormEvent<HTMLFormElement>) => {
|
|
e.preventDefault();
|
|
const formData = new FormData(e.currentTarget);
|
|
createMutation.mutate({
|
|
name: formData.get('name'),
|
|
source_id: Number(formData.get('source_id')),
|
|
strategy: formData.get('strategy'),
|
|
destination_path: formData.get('destination_path'),
|
|
});
|
|
};
|
|
|
|
return (
|
|
<div className="fixed inset-0 bg-black bg-opacity-50 flex items-center justify-center">
|
|
<div className="bg-white rounded-lg p-6 w-full max-w-md">
|
|
<h2 className="text-lg font-bold mb-4">Create New Job</h2>
|
|
<form onSubmit={handleSubmit} className="space-y-4">
|
|
<div>
|
|
<label className="block text-sm font-medium mb-1">Name</label>
|
|
<input name="name" required className="w-full border rounded-lg px-3 py-2" />
|
|
</div>
|
|
<div>
|
|
<label className="block text-sm font-medium mb-1">Source</label>
|
|
<select name="source_id" required className="w-full border rounded-lg px-3 py-2">
|
|
{sources.map((source) => (
|
|
<option key={source.id} value={source.id}>{source.name}</option>
|
|
))}
|
|
</select>
|
|
</div>
|
|
<div>
|
|
<label className="block text-sm font-medium mb-1">Strategy</label>
|
|
<select name="strategy" className="w-full border rounded-lg px-3 py-2">
|
|
<option value="full">Full</option>
|
|
<option value="incremental">Incremental</option>
|
|
</select>
|
|
</div>
|
|
<div>
|
|
<label className="block text-sm font-medium mb-1">Destination Path</label>
|
|
<input name="destination_path" required className="w-full border rounded-lg px-3 py-2" />
|
|
</div>
|
|
<div className="flex gap-2 pt-4">
|
|
<button type="submit" className="flex-1 bg-blue-600 text-white py-2 rounded-lg hover:bg-blue-700">
|
|
Create Job
|
|
</button>
|
|
<button type="button" onClick={onClose} className="flex-1 border py-2 rounded-lg hover:bg-gray-50">
|
|
Cancel
|
|
</button>
|
|
</div>
|
|
</form>
|
|
</div>
|
|
</div>
|
|
);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Commit**
|
|
|
|
```bash
|
|
git add frontend/src/pages/Backups.tsx
|
|
git commit -m "feat: add backups page with job management
|
|
|
|
- Job listing table with source, strategy, status
|
|
- Create job modal with form
|
|
- Run and delete job actions
|
|
- React Query integration for data fetching"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 12: Frontend - Settings Page
|
|
|
|
**Files:**
|
|
- Create: `frontend/src/pages/Settings.tsx`
|
|
|
|
- [ ] **Step 1: Create settings page**
|
|
|
|
`frontend/src/pages/Settings.tsx`:
|
|
```typescript
|
|
import { useState } from 'react';
|
|
import { HardDrive, Bell, Shield, FileText } from 'lucide-react';
|
|
|
|
export function Settings() {
|
|
const [activeTab, setActiveTab] = useState('general');
|
|
|
|
const tabs = [
|
|
{ id: 'general', label: 'General', icon: HardDrive },
|
|
{ id: 'notifications', label: 'Notifications', icon: Bell },
|
|
{ id: 'security', label: 'Security', icon: Shield },
|
|
{ id: 'logs', label: 'Logs', icon: FileText },
|
|
];
|
|
|
|
return (
|
|
<div className="space-y-6">
|
|
<h1 className="text-2xl font-bold">Settings</h1>
|
|
|
|
<div className="flex gap-6">
|
|
<div className="w-64">
|
|
<nav className="space-y-1">
|
|
{tabs.map((tab) => (
|
|
<button
|
|
key={tab.id}
|
|
onClick={() => setActiveTab(tab.id)}
|
|
className={`w-full flex items-center gap-3 px-3 py-2 rounded-lg text-left ${
|
|
activeTab === tab.id
|
|
? 'bg-blue-50 text-blue-700 font-medium'
|
|
: 'text-gray-600 hover:bg-gray-50'
|
|
}`}
|
|
>
|
|
<tab.icon className="w-5 h-5" />
|
|
{tab.label}
|
|
</button>
|
|
))}
|
|
</nav>
|
|
</div>
|
|
|
|
<div className="flex-1 bg-white rounded-lg shadow p-6">
|
|
{activeTab === 'general' && <GeneralSettings />}
|
|
{activeTab === 'notifications' && <NotificationSettings />}
|
|
{activeTab === 'security' && <SecuritySettings />}
|
|
{activeTab === 'logs' && <LogSettings />}
|
|
</div>
|
|
</div>
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function GeneralSettings() {
|
|
return (
|
|
<div className="space-y-6">
|
|
<h2 className="text-lg font-semibold">General Settings</h2>
|
|
|
|
<div>
|
|
<label className="block text-sm font-medium mb-2">Default Backup Location</label>
|
|
<input
|
|
type="text"
|
|
defaultValue="/var/backups/backup-tool"
|
|
className="w-full border rounded-lg px-3 py-2"
|
|
/>
|
|
</div>
|
|
|
|
<div>
|
|
<label className="block text-sm font-medium mb-2">Retention Policy</label>
|
|
<select className="w-full border rounded-lg px-3 py-2">
|
|
<option>Keep all backups</option>
|
|
<option>Keep last 30 days</option>
|
|
<option selected>Keep last 10 versions</option>
|
|
<option>Keep last 5 versions</option>
|
|
</select>
|
|
</div>
|
|
|
|
<div>
|
|
<label className="flex items-center gap-2">
|
|
<input type="checkbox" defaultChecked />
|
|
<span>Enable compression (gzip)</span>
|
|
</label>
|
|
</div>
|
|
|
|
<button className="bg-blue-600 text-white px-4 py-2 rounded-lg hover:bg-blue-700">
|
|
Save Changes
|
|
</button>
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function NotificationSettings() {
|
|
return (
|
|
<div className="space-y-6">
|
|
<h2 className="text-lg font-semibold">Notifications</h2>
|
|
|
|
<div>
|
|
<label className="block text-sm font-medium mb-2">Webhook URL</label>
|
|
<input
|
|
type="url"
|
|
placeholder="https://hooks.slack.com/..."
|
|
className="w-full border rounded-lg px-3 py-2"
|
|
/>
|
|
</div>
|
|
|
|
<div>
|
|
<label className="flex items-center gap-2">
|
|
<input type="checkbox" />
|
|
<span>Email notifications</span>
|
|
</label>
|
|
</div>
|
|
|
|
<div>
|
|
<label className="flex items-center gap-2">
|
|
<input type="checkbox" defaultChecked />
|
|
<span>Notify on failure</span>
|
|
</label>
|
|
</div>
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function SecuritySettings() {
|
|
return (
|
|
<div className="space-y-6">
|
|
<h2 className="text-lg font-semibold">Security</h2>
|
|
|
|
<div>
|
|
<label className="flex items-center gap-2">
|
|
<input type="checkbox" />
|
|
<span>Enable AES-256 encryption</span>
|
|
</label>
|
|
</div>
|
|
|
|
<div>
|
|
<label className="block text-sm font-medium mb-2">Encryption Key</label>
|
|
<input
|
|
type="password"
|
|
placeholder="Enter encryption key..."
|
|
className="w-full border rounded-lg px-3 py-2"
|
|
/>
|
|
</div>
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function LogSettings() {
|
|
return (
|
|
<div className="space-y-6">
|
|
<h2 className="text-lg font-semibold">Logs</h2>
|
|
|
|
<div>
|
|
<label className="block text-sm font-medium mb-2">Log Level</label>
|
|
<select className="w-full border rounded-lg px-3 py-2">
|
|
<option>DEBUG</option>
|
|
<option selected>INFO</option>
|
|
<option>WARNING</option>
|
|
<option>ERROR</option>
|
|
</select>
|
|
</div>
|
|
|
|
<div>
|
|
<label className="block text-sm font-medium mb-2">Log Retention (days)</label>
|
|
<input
|
|
type="number"
|
|
defaultValue={30}
|
|
className="w-full border rounded-lg px-3 py-2"
|
|
/>
|
|
</div>
|
|
|
|
<button className="border px-4 py-2 rounded-lg hover:bg-gray-50">
|
|
Download Logs
|
|
</button>
|
|
</div>
|
|
);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Commit**
|
|
|
|
```bash
|
|
git add frontend/src/pages/Settings.tsx
|
|
git commit -m "feat: add settings page with tabs
|
|
|
|
- General, Notifications, Security, Logs tabs
|
|
- Form inputs for retention, compression, encryption
|
|
- Save changes button"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 13: Frontend Entry Point and Styling
|
|
|
|
**Files:**
|
|
- Create: `frontend/src/main.tsx`
|
|
- Create: `frontend/src/index.css`
|
|
|
|
- [ ] **Step 1: Create entry point**
|
|
|
|
`frontend/src/main.tsx`:
|
|
```typescript
|
|
import React from 'react'
|
|
import ReactDOM from 'react-dom/client'
|
|
import App from './App'
|
|
import './index.css'
|
|
|
|
ReactDOM.createRoot(document.getElementById('root')!).render(
|
|
<React.StrictMode>
|
|
<App />
|
|
</React.StrictMode>,
|
|
)
|
|
```
|
|
|
|
`frontend/src/index.css`:
|
|
```css
|
|
@tailwind base;
|
|
@tailwind components;
|
|
@tailwind utilities;
|
|
|
|
body {
|
|
@apply antialiased;
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Create postcss config**
|
|
|
|
`frontend/postcss.config.js`:
|
|
```javascript
|
|
export default {
|
|
plugins: {
|
|
tailwindcss: {},
|
|
autoprefixer: {},
|
|
},
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add frontend/src/main.tsx frontend/src/index.css frontend/postcss.config.js
|
|
git commit -m "feat: add frontend entry point and tailwind styling
|
|
|
|
- React 18 root rendering
|
|
- Tailwind CSS directives
|
|
- PostCSS configuration"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 14: Dashboard API Endpoint
|
|
|
|
**Files:**
|
|
- Create: `backend/app/routers/dashboard.py`
|
|
|
|
- [ ] **Step 1: Create dashboard router**
|
|
|
|
`backend/app/routers/dashboard.py`:
|
|
```python
|
|
from fastapi import APIRouter, Depends
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy import select, func
|
|
from app.database import get_db
|
|
from app.models import Job, JobExecution, Backup
|
|
from app.schemas import DashboardStats
|
|
|
|
router = APIRouter(prefix="/api/dashboard", tags=["dashboard"])
|
|
|
|
@router.get("/", response_model=DashboardStats)
|
|
async def get_dashboard_stats(db: AsyncSession = Depends(get_db)):
|
|
# Active jobs count
|
|
result = await db.execute(select(func.count(Job.id)).where(Job.enabled == True))
|
|
active_jobs = result.scalar()
|
|
|
|
# Total backups count
|
|
result = await db.execute(select(func.count(Backup.id)))
|
|
total_backups = result.scalar()
|
|
|
|
# Storage used
|
|
result = await db.execute(select(func.sum(Backup.size_bytes)))
|
|
storage_used = result.scalar() or 0
|
|
|
|
# Recent failures (last 24 hours)
|
|
from datetime import datetime, timedelta
|
|
cutoff = datetime.utcnow() - timedelta(hours=24)
|
|
result = await db.execute(
|
|
select(func.count(JobExecution.id))
|
|
.where(
|
|
JobExecution.status == "failed",
|
|
JobExecution.started_at >= cutoff
|
|
)
|
|
)
|
|
recent_failures = result.scalar()
|
|
|
|
# Recent executions (last 10)
|
|
result = await db.execute(
|
|
select(JobExecution)
|
|
.order_by(JobExecution.started_at.desc())
|
|
.limit(10)
|
|
)
|
|
recent_executions = result.scalars().all()
|
|
|
|
return DashboardStats(
|
|
active_jobs=active_jobs,
|
|
total_backups=total_backups,
|
|
storage_used_bytes=storage_used,
|
|
recent_failures=recent_failures,
|
|
recent_executions=recent_executions
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 2: Commit**
|
|
|
|
```bash
|
|
git add backend/app/routers/dashboard.py
|
|
git commit -m "feat: add dashboard stats API endpoint
|
|
|
|
- Aggregates active jobs, total backups, storage used
|
|
- Recent failures count (24h window)
|
|
- Last 10 executions for activity feed"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 15: Remaining API Routers
|
|
|
|
**Files:**
|
|
- Create: `backend/app/routers/executions.py`
|
|
- Create: `backend/app/routers/backups.py`
|
|
- Create: `backend/app/routers/settings.py`
|
|
|
|
- [ ] **Step 1: Create executions router**
|
|
|
|
`backend/app/routers/executions.py`:
|
|
```python
|
|
from fastapi import APIRouter, Depends, HTTPException
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy import select
|
|
from typing import List
|
|
from app.database import get_db
|
|
from app.models import JobExecution
|
|
from app.schemas import JobExecution as JobExecutionSchema
|
|
|
|
router = APIRouter(prefix="/api/executions", tags=["executions"])
|
|
|
|
@router.get("/", response_model=List[JobExecutionSchema])
|
|
async def list_executions(db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(JobExecution).order_by(JobExecution.started_at.desc()))
|
|
return result.scalars().all()
|
|
|
|
@router.get("/{execution_id}", response_model=JobExecutionSchema)
|
|
async def get_execution(execution_id: int, db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(JobExecution).where(JobExecution.id == execution_id))
|
|
execution = result.scalar_one_or_none()
|
|
if not execution:
|
|
raise HTTPException(status_code=404, detail="Execution not found")
|
|
return execution
|
|
```
|
|
|
|
- [ ] **Step 2: Create backups router**
|
|
|
|
`backend/app/routers/backups.py`:
|
|
```python
|
|
from fastapi import APIRouter, Depends, HTTPException
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy import select
|
|
from typing import List
|
|
from app.database import get_db
|
|
from app.models import Backup
|
|
from app.schemas import Backup as BackupSchema
|
|
|
|
router = APIRouter(prefix="/api/backups", tags=["backups"])
|
|
|
|
@router.get("/", response_model=List[BackupSchema])
|
|
async def list_backups(db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Backup).order_by(Backup.created_at.desc()))
|
|
return result.scalars().all()
|
|
|
|
@router.get("/{backup_id}", response_model=BackupSchema)
|
|
async def get_backup(backup_id: int, db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Backup).where(Backup.id == backup_id))
|
|
backup = result.scalar_one_or_none()
|
|
if not backup:
|
|
raise HTTPException(status_code=404, detail="Backup not found")
|
|
return backup
|
|
|
|
@router.delete("/{backup_id}")
|
|
async def delete_backup(backup_id: int, db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Backup).where(Backup.id == backup_id))
|
|
backup = result.scalar_one_or_none()
|
|
if not backup:
|
|
raise HTTPException(status_code=404, detail="Backup not found")
|
|
|
|
await db.delete(backup)
|
|
await db.commit()
|
|
return {"message": "Backup deleted"}
|
|
```
|
|
|
|
- [ ] **Step 3: Create settings router**
|
|
|
|
`backend/app/routers/settings.py`:
|
|
```python
|
|
from fastapi import APIRouter, Depends, HTTPException
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy import select
|
|
from typing import List, Dict
|
|
from app.database import get_db
|
|
from app.models import Setting
|
|
from app.schemas import Setting as SettingSchema, SettingCreate, SettingUpdate
|
|
|
|
router = APIRouter(prefix="/api/settings", tags=["settings"])
|
|
|
|
@router.get("/", response_model=Dict[str, str])
|
|
async def get_settings(db: AsyncSession = Depends(get_db)):
|
|
result = await db.execute(select(Setting))
|
|
settings = result.scalars().all()
|
|
return {s.key: s.value for s in settings}
|
|
|
|
@router.put("/")
|
|
async def update_settings(
|
|
settings: Dict[str, str],
|
|
db: AsyncSession = Depends(get_db)
|
|
):
|
|
for key, value in settings.items():
|
|
result = await db.execute(select(Setting).where(Setting.key == key))
|
|
setting = result.scalar_one_or_none()
|
|
|
|
if setting:
|
|
setting.value = value
|
|
else:
|
|
setting = Setting(key=key, value=value)
|
|
db.add(setting)
|
|
|
|
await db.commit()
|
|
return {"message": "Settings updated"}
|
|
```
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add backend/app/routers/executions.py backend/app/routers/backups.py backend/app/routers/settings.py
|
|
git commit -m "feat: add executions, backups, and settings API routers
|
|
|
|
- Execution monitoring endpoints
|
|
- Backup listing and deletion
|
|
- Settings key-value store management"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 16: SSH and Database Adapters
|
|
|
|
**Files:**
|
|
- Create: `backend/backup/adapters/ssh.py`
|
|
- Create: `backend/backup/adapters/database.py`
|
|
|
|
- [ ] **Step 1: Create SSH adapter**
|
|
|
|
`backend/backup/adapters/ssh.py`:
|
|
```python
|
|
import paramiko
|
|
import os
|
|
from typing import List, AsyncIterator
|
|
from .base import SourceAdapter, FileInfo
|
|
|
|
class SSHAdapter(SourceAdapter):
|
|
def __init__(self, config):
|
|
super().__init__(config)
|
|
self.client = None
|
|
self.sftp = None
|
|
|
|
async def connect(self) -> None:
|
|
self.client = paramiko.SSHClient()
|
|
self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
|
|
|
connect_kwargs = {
|
|
"hostname": self.config["host"],
|
|
"port": self.config.get("port", 22),
|
|
"username": self.config["username"],
|
|
}
|
|
|
|
if "password" in self.config:
|
|
connect_kwargs["password"] = self.config["password"]
|
|
elif "key_path" in self.config:
|
|
connect_kwargs["key_filename"] = self.config["key_path"]
|
|
|
|
self.client.connect(**connect_kwargs)
|
|
self.sftp = self.client.open_sftp()
|
|
|
|
async def disconnect(self) -> None:
|
|
if self.sftp:
|
|
self.sftp.close()
|
|
if self.client:
|
|
self.client.close()
|
|
|
|
async def list_files(self, path: str = "") -> List[FileInfo]:
|
|
remote_path = os.path.join(self.config["path"], path)
|
|
files = []
|
|
|
|
for entry in self.sftp.listdir_attr(remote_path):
|
|
files.append(FileInfo(
|
|
path=os.path.join(path, entry.filename),
|
|
size=entry.st_size,
|
|
modified_time=entry.st_mtime,
|
|
is_directory=entry.st_mode & 0o40000 == 0o40000
|
|
))
|
|
|
|
return files
|
|
|
|
async def read_file(self, path: str) -> AsyncIterator[bytes]:
|
|
remote_path = os.path.join(self.config["path"], path)
|
|
|
|
with self.sftp.file(remote_path, "rb") as f:
|
|
while chunk := f.read(8192):
|
|
yield chunk
|
|
|
|
async def get_database_dump(self, config: dict) -> AsyncIterator[bytes]:
|
|
raise NotImplementedError("SSH adapter does not support database dumps")
|
|
```
|
|
|
|
- [ ] **Step 2: Create database adapter**
|
|
|
|
`backend/backup/adapters/database.py`:
|
|
```python
|
|
import subprocess
|
|
import tempfile
|
|
from typing import List, AsyncIterator
|
|
from .base import SourceAdapter, FileInfo
|
|
|
|
class DatabaseAdapter(SourceAdapter):
|
|
async def connect(self) -> None:
|
|
# Verify database connection by testing CLI tool
|
|
db_type = self.config.get("db_type", "postgresql")
|
|
if db_type == "postgresql":
|
|
result = subprocess.run(
|
|
["pg_dump", "--version"],
|
|
capture_output=True,
|
|
text=True
|
|
)
|
|
if result.returncode != 0:
|
|
raise RuntimeError("pg_dump not found")
|
|
elif db_type == "mysql":
|
|
result = subprocess.run(
|
|
["mysqldump", "--version"],
|
|
capture_output=True,
|
|
text=True
|
|
)
|
|
if result.returncode != 0:
|
|
raise RuntimeError("mysqldump not found")
|
|
|
|
async def disconnect(self) -> None:
|
|
pass
|
|
|
|
async def list_files(self, path: str = "") -> List[FileInfo]:
|
|
raise NotImplementedError("Database adapter does not support file listing")
|
|
|
|
async def read_file(self, path: str) -> AsyncIterator[bytes]:
|
|
raise NotImplementedError("Database adapter does not support file reading")
|
|
|
|
async def get_database_dump(self, config: dict) -> AsyncIterator[bytes]:
|
|
db_type = config.get("db_type", "postgresql")
|
|
|
|
with tempfile.NamedTemporaryFile(suffix=".sql", delete=False) as tmp:
|
|
tmp_path = tmp.name
|
|
|
|
try:
|
|
if db_type == "postgresql":
|
|
cmd = [
|
|
"pg_dump",
|
|
f"--host={config['host']}",
|
|
f"--port={config.get('port', 5432)}",
|
|
f"--username={config['username']}",
|
|
f"--dbname={config['database']}",
|
|
f"--file={tmp_path}"
|
|
]
|
|
env = {"PGPASSWORD": config.get("password", "")}
|
|
elif db_type == "mysql":
|
|
cmd = [
|
|
"mysqldump",
|
|
f"--host={config['host']}",
|
|
f"--port={config.get('port', 3306)}",
|
|
f"--user={config['username']}",
|
|
f"--password={config.get('password', '')}",
|
|
config["database"]
|
|
]
|
|
env = None
|
|
else:
|
|
raise ValueError(f"Unsupported database type: {db_type}")
|
|
|
|
result = subprocess.run(cmd, capture_output=True, text=True, env=env)
|
|
if result.returncode != 0:
|
|
raise RuntimeError(f"Database dump failed: {result.stderr}")
|
|
|
|
with open(tmp_path, "rb") as f:
|
|
while chunk := f.read(8192):
|
|
yield chunk
|
|
|
|
finally:
|
|
os.unlink(tmp_path)
|
|
```
|
|
|
|
- [ ] **Step 3: Update adapter factory**
|
|
|
|
Modify `backend/backup/adapters/__init__.py`:
|
|
```python
|
|
from typing import Dict, Any
|
|
from .base import SourceAdapter
|
|
from .local import LocalAdapter
|
|
from .ssh import SSHAdapter
|
|
from .database import DatabaseAdapter
|
|
|
|
ADAPTER_MAP = {
|
|
"local": LocalAdapter,
|
|
"ssh": SSHAdapter,
|
|
"database": DatabaseAdapter,
|
|
}
|
|
|
|
def get_adapter(source_type: str, config: Dict[str, Any]) -> SourceAdapter:
|
|
adapter_class = ADAPTER_MAP.get(source_type)
|
|
if not adapter_class:
|
|
raise ValueError(f"Unknown source type: {source_type}")
|
|
return adapter_class(config)
|
|
```
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add backend/backup/adapters/ssh.py backend/backup/adapters/database.py backend/backup/adapters/__init__.py
|
|
git commit -m "feat: add SSH and database source adapters
|
|
|
|
- SSHAdapter using Paramiko for remote file access
|
|
- DatabaseAdapter using native CLI tools (pg_dump, mysqldump)
|
|
- Updated adapter factory with all source types"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 17: Scheduler Integration
|
|
|
|
**Files:**
|
|
- Create: `backend/backup/scheduler.py`
|
|
- Modify: `backend/app/main.py`
|
|
|
|
- [ ] **Step 1: Create scheduler service**
|
|
|
|
`backend/backup/scheduler.py`:
|
|
```python
|
|
from apscheduler.schedulers.asyncio import AsyncIOScheduler
|
|
from apscheduler.triggers.cron import CronTrigger
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from app.database import AsyncSessionLocal
|
|
from app.models import Schedule, Job
|
|
from backup.engine import BackupEngine
|
|
|
|
class BackupScheduler:
|
|
def __init__(self):
|
|
self.scheduler = AsyncIOScheduler()
|
|
|
|
def start(self):
|
|
self.scheduler.start()
|
|
|
|
def shutdown(self):
|
|
self.scheduler.shutdown()
|
|
|
|
def add_job(self, schedule_id: int, cron_expression: str):
|
|
"""Add a scheduled job."""
|
|
trigger = CronTrigger.from_crontab(cron_expression)
|
|
|
|
self.scheduler.add_job(
|
|
func=self._execute_scheduled_job,
|
|
trigger=trigger,
|
|
id=f"schedule_{schedule_id}",
|
|
args=[schedule_id],
|
|
replace_existing=True,
|
|
misfire_grace_time=900 # 15 minutes
|
|
)
|
|
|
|
def remove_job(self, schedule_id: int):
|
|
"""Remove a scheduled job."""
|
|
job_id = f"schedule_{schedule_id}"
|
|
try:
|
|
self.scheduler.remove_job(job_id)
|
|
except Exception:
|
|
pass
|
|
|
|
async def _execute_scheduled_job(self, schedule_id: int):
|
|
"""Execute a scheduled backup job."""
|
|
async with AsyncSessionLocal() as db:
|
|
from sqlalchemy import select
|
|
|
|
result = await db.execute(
|
|
select(Schedule).where(Schedule.id == schedule_id)
|
|
)
|
|
schedule = result.scalar_one_or_none()
|
|
|
|
if not schedule or not schedule.enabled:
|
|
return
|
|
|
|
engine = BackupEngine(db)
|
|
await engine.execute_job(schedule.job_id, triggered_by="schedule")
|
|
|
|
# Global scheduler instance
|
|
scheduler = BackupScheduler()
|
|
```
|
|
|
|
- [ ] **Step 2: Integrate scheduler with FastAPI**
|
|
|
|
Modify `backend/app/main.py`:
|
|
```python
|
|
from fastapi import FastAPI
|
|
from fastapi.middleware.cors import CORSMiddleware
|
|
from app.routers import sources, jobs, executions, backups, settings, dashboard
|
|
from app.database import engine, Base
|
|
from backup.scheduler import scheduler
|
|
|
|
app = FastAPI(title="Backup Tool API", version="0.1.0")
|
|
|
|
# CORS
|
|
app.add_middleware(
|
|
CORSMiddleware,
|
|
allow_origins=["http://localhost:3000"],
|
|
allow_credentials=True,
|
|
allow_methods=["*"],
|
|
allow_headers=["*"],
|
|
)
|
|
|
|
# Include routers
|
|
app.include_router(sources.router)
|
|
app.include_router(jobs.router)
|
|
app.include_router(executions.router)
|
|
app.include_router(backups.router)
|
|
app.include_router(settings.router)
|
|
app.include_router(dashboard.router)
|
|
|
|
@app.on_event("startup")
|
|
async def startup():
|
|
async with engine.begin() as conn:
|
|
await conn.run_sync(Base.metadata.create_all)
|
|
|
|
# Start scheduler
|
|
scheduler.start()
|
|
|
|
# Load existing schedules
|
|
from sqlalchemy import select
|
|
from app.models import Schedule
|
|
|
|
async with AsyncSessionLocal() as db:
|
|
result = await db.execute(select(Schedule).where(Schedule.enabled == True))
|
|
schedules = result.scalars().all()
|
|
|
|
for schedule in schedules:
|
|
scheduler.add_job(schedule.id, schedule.cron_expression)
|
|
|
|
@app.on_event("shutdown")
|
|
async def shutdown():
|
|
scheduler.shutdown()
|
|
|
|
@app.get("/api/health")
|
|
async def health_check():
|
|
return {"status": "healthy"}
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add backend/backup/scheduler.py backend/app/main.py
|
|
git commit -m "feat: add APScheduler integration for cron-based job scheduling
|
|
|
|
- BackupScheduler class wrapping APScheduler
|
|
- Auto-load enabled schedules on startup
|
|
- Graceful shutdown on app stop"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 18: Retention Policy
|
|
|
|
**Files:**
|
|
- Create: `backend/backup/retention.py`
|
|
|
|
- [ ] **Step 1: Create retention policy logic**
|
|
|
|
`backend/backup/retention.py`:
|
|
```python
|
|
import os
|
|
import shutil
|
|
from datetime import datetime, timedelta
|
|
from typing import List
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy import select
|
|
from app.models import Job, Backup, JobExecution
|
|
|
|
class RetentionPolicy:
|
|
def __init__(self, db: AsyncSession):
|
|
self.db = db
|
|
|
|
async def apply_policy(
|
|
self,
|
|
job_id: int,
|
|
policy_type: str = "count",
|
|
keep_count: int = 10,
|
|
keep_days: int = 30
|
|
):
|
|
"""Apply retention policy to a job's backups."""
|
|
|
|
if policy_type == "count":
|
|
await self._apply_count_policy(job_id, keep_count)
|
|
elif policy_type == "days":
|
|
await self._apply_days_policy(job_id, keep_days)
|
|
elif policy_type == "all":
|
|
pass # Keep all backups
|
|
|
|
async def _apply_count_policy(self, job_id: int, keep_count: int):
|
|
"""Keep only the last N backups."""
|
|
result = await self.db.execute(
|
|
select(Backup)
|
|
.join(JobExecution)
|
|
.where(JobExecution.job_id == job_id)
|
|
.order_by(Backup.created_at.desc())
|
|
.offset(keep_count)
|
|
)
|
|
old_backups = result.scalars().all()
|
|
|
|
for backup in old_backups:
|
|
await self._delete_backup(backup)
|
|
|
|
async def _apply_days_policy(self, job_id: int, keep_days: int):
|
|
"""Delete backups older than N days."""
|
|
cutoff = datetime.utcnow() - timedelta(days=keep_days)
|
|
|
|
result = await self.db.execute(
|
|
select(Backup)
|
|
.join(JobExecution)
|
|
.where(
|
|
JobExecution.job_id == job_id,
|
|
Backup.created_at < cutoff
|
|
)
|
|
)
|
|
old_backups = result.scalars().all()
|
|
|
|
for backup in old_backups:
|
|
await self._delete_backup(backup)
|
|
|
|
async def _delete_backup(self, backup: Backup):
|
|
"""Delete backup files and database record."""
|
|
# Delete files
|
|
if os.path.exists(backup.storage_path):
|
|
if os.path.isdir(backup.storage_path):
|
|
shutil.rmtree(backup.storage_path)
|
|
else:
|
|
os.unlink(backup.storage_path)
|
|
|
|
# Delete database record
|
|
await self.db.delete(backup)
|
|
```
|
|
|
|
- [ ] **Step 2: Integrate retention in engine**
|
|
|
|
Modify `backend/backup/engine.py` to add retention application after successful backup:
|
|
|
|
Add at the end of the `execute_job` method, after storing backup metadata:
|
|
```python
|
|
# Apply retention policy
|
|
from backup.retention import RetentionPolicy
|
|
retention = RetentionPolicy(self.db)
|
|
await retention.apply_policy(
|
|
job_id=job_id,
|
|
policy_type="count", # TODO: Read from settings
|
|
keep_count=10
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add backend/backup/retention.py
|
|
git commit -m "feat: add retention policy management
|
|
|
|
- Count-based retention (keep last N backups)
|
|
- Days-based retention (delete older than N days)
|
|
- Automatic cleanup after successful backup"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 19: Frontend Build Integration
|
|
|
|
**Files:**
|
|
- Modify: `backend/app/main.py`
|
|
- Modify: `backend/requirements.txt`
|
|
|
|
- [ ] **Step 1: Add static file serving**
|
|
|
|
Modify `backend/app/main.py` to include static file serving:
|
|
|
|
```python
|
|
from fastapi import FastAPI
|
|
from fastapi.middleware.cors import CORSMiddleware
|
|
from fastapi.staticfiles import StaticFiles
|
|
from fastapi.responses import FileResponse
|
|
import os
|
|
from app.routers import sources, jobs, executions, backups, settings, dashboard
|
|
from app.database import engine, Base
|
|
from backup.scheduler import scheduler
|
|
|
|
app = FastAPI(title="Backup Tool API", version="0.1.0")
|
|
|
|
# CORS (only in development)
|
|
app.add_middleware(
|
|
CORSMiddleware,
|
|
allow_origins=["http://localhost:3000"],
|
|
allow_credentials=True,
|
|
allow_methods=["*"],
|
|
allow_headers=["*"],
|
|
)
|
|
|
|
# Include routers
|
|
app.include_router(sources.router)
|
|
app.include_router(jobs.router)
|
|
app.include_router(executions.router)
|
|
app.include_router(backups.router)
|
|
app.include_router(settings.router)
|
|
app.include_router(dashboard.router)
|
|
|
|
# Serve static files in production
|
|
static_dir = os.path.join(os.path.dirname(__file__), "../static")
|
|
if os.path.exists(static_dir):
|
|
app.mount("/", StaticFiles(directory=static_dir, html=True), name="static")
|
|
|
|
@app.on_event("startup")
|
|
async def startup():
|
|
async with engine.begin() as conn:
|
|
await conn.run_sync(Base.metadata.create_all)
|
|
|
|
# Start scheduler
|
|
scheduler.start()
|
|
|
|
# Load existing schedules
|
|
from sqlalchemy import select
|
|
from app.models import Schedule
|
|
from app.database import AsyncSessionLocal
|
|
|
|
async with AsyncSessionLocal() as db:
|
|
result = await db.execute(select(Schedule).where(Schedule.enabled == True))
|
|
schedules = result.scalars().all()
|
|
|
|
for schedule in schedules:
|
|
scheduler.add_job(schedule.id, schedule.cron_expression)
|
|
|
|
@app.on_event("shutdown")
|
|
async def shutdown():
|
|
scheduler.shutdown()
|
|
|
|
@app.get("/api/health")
|
|
async def health_check():
|
|
return {"status": "healthy"}
|
|
|
|
# Fallback for SPA routing
|
|
@app.get("/{path:path}")
|
|
async def catch_all(path: str):
|
|
index_file = os.path.join(static_dir, "index.html")
|
|
if os.path.exists(index_file):
|
|
return FileResponse(index_file)
|
|
return {"detail": "Not found"}
|
|
```
|
|
|
|
- [ ] **Step 2: Update requirements**
|
|
|
|
Add to `backend/requirements.txt`:
|
|
```
|
|
aiofiles==23.2.1
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add backend/app/main.py backend/requirements.txt
|
|
git commit -m "feat: add static file serving for production build
|
|
|
|
- Serve built frontend from /static directory
|
|
- SPA catch-all routing
|
|
- Added aiofiles for async file operations"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 20: Documentation and Final Setup
|
|
|
|
**Files:**
|
|
- Create: `README.md`
|
|
- Create: `.gitignore`
|
|
|
|
- [ ] **Step 1: Create comprehensive README**
|
|
|
|
`README.md`:
|
|
```markdown
|
|
# Backup Tool
|
|
|
|
A web-based backup management tool for small teams and SMBs.
|
|
|
|
## Features
|
|
|
|
- **Web Dashboard**: Monitor backup status, storage usage, and recent activity
|
|
- **Multiple Sources**: Backup from local filesystem, remote servers (SSH), and databases
|
|
- **Flexible Scheduling**: Cron-based scheduling for automated backups
|
|
- **Incremental Backups**: Save space with incremental backup strategies
|
|
- **Retention Policies**: Automatic cleanup of old backups
|
|
- **REST API**: Full API for integration with other tools
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- Python 3.11+
|
|
- Node.js 18+
|
|
- PostgreSQL client tools (pg_dump) or MySQL client tools (mysqldump) for database backups
|
|
|
|
### Backend Setup
|
|
|
|
```bash
|
|
cd backend
|
|
python -m venv venv
|
|
source venv/bin/activate # On Windows: venv\Scripts\activate
|
|
pip install -r requirements.txt
|
|
|
|
# Run migrations
|
|
alembic upgrade head
|
|
|
|
# Start server
|
|
uvicorn app.main:app --reload
|
|
```
|
|
|
|
### Frontend Setup
|
|
|
|
```bash
|
|
cd frontend
|
|
npm install
|
|
npm run dev
|
|
```
|
|
|
|
### Production Build
|
|
|
|
```bash
|
|
# Build frontend
|
|
cd frontend
|
|
npm run build
|
|
|
|
# Copy build to backend static directory
|
|
cp -r dist/* ../backend/static/
|
|
|
|
# Run backend with multiple workers
|
|
cd ../backend
|
|
uvicorn app.main:app --host 0.0.0.0 --port 8000
|
|
```
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Frontend (React + TypeScript)
|
|
↓ REST API
|
|
Backend (FastAPI + SQLAlchemy)
|
|
↓ Adapters
|
|
Sources: Local | SSH | Database
|
|
```
|
|
|
|
## API Documentation
|
|
|
|
When running locally, API docs are available at:
|
|
- Swagger UI: http://localhost:8000/docs
|
|
- ReDoc: http://localhost:8000/redoc
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
# Backend tests
|
|
cd backend
|
|
pytest
|
|
|
|
# Frontend tests
|
|
cd frontend
|
|
npm test
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Key settings managed via the Settings page or API:
|
|
|
|
- **Default Backup Location**: Where backups are stored
|
|
- **Retention Policy**: How many backups to keep
|
|
- **Compression**: Enable/disable gzip compression
|
|
- **Encryption**: AES-256 encryption for backups
|
|
- **Notifications**: Webhook URLs for alerts
|
|
|
|
## Development
|
|
|
|
### Adding a New Source Type
|
|
|
|
1. Create adapter class in `backend/backup/adapters/`
|
|
2. Inherit from `SourceAdapter`
|
|
3. Implement required methods
|
|
4. Register in `ADAPTER_MAP`
|
|
5. Add UI form components
|
|
|
|
### Database Migrations
|
|
|
|
```bash
|
|
cd backend
|
|
alembic revision --autogenerate -m "Description"
|
|
alembic upgrade head
|
|
```
|
|
|
|
## License
|
|
|
|
MIT
|
|
```
|
|
|
|
- [ ] **Step 2: Create .gitignore**
|
|
|
|
```gitignore
|
|
# Python
|
|
__pycache__/
|
|
*.py[cod]
|
|
*$py.class
|
|
*.so
|
|
.Python
|
|
build/
|
|
develop-eggs/
|
|
dist/
|
|
downloads/
|
|
eggs/
|
|
.eggs/
|
|
lib/
|
|
lib64/
|
|
parts/
|
|
sdist/
|
|
var/
|
|
wheels/
|
|
*.egg-info/
|
|
.installed.cfg
|
|
*.egg
|
|
|
|
# Virtual environments
|
|
venv/
|
|
env/
|
|
ENV/
|
|
|
|
# Database
|
|
*.db
|
|
*.sqlite
|
|
*.sqlite3
|
|
|
|
# Frontend
|
|
node_modules/
|
|
frontend/dist/
|
|
frontend/build/
|
|
|
|
# IDE
|
|
.vscode/
|
|
.idea/
|
|
*.swp
|
|
*.swo
|
|
*~
|
|
|
|
# OS
|
|
.DS_Store
|
|
Thumbs.db
|
|
|
|
# Logs
|
|
*.log
|
|
logs/
|
|
|
|
# Environment
|
|
.env
|
|
.env.local
|
|
|
|
# Backup tool specific
|
|
backups/
|
|
test.db
|
|
backup_tool.db
|
|
```
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add README.md .gitignore
|
|
git commit -m "docs: add comprehensive README and .gitignore
|
|
|
|
- Setup instructions for development and production
|
|
- Architecture overview
|
|
- API documentation links
|
|
- Development guide for extending source types"
|
|
```
|
|
|
|
---
|
|
|
|
## Self-Review
|
|
|
|
### Spec Coverage Check
|
|
|
|
| Spec Section | Implementation Task |
|
|
|--------------|-------------------|
|
|
| Database Models (4.1) | Task 2 |
|
|
| Pydantic Schemas (4.1) | Task 3 |
|
|
| Source Adapters (5.4) | Task 4, 16 |
|
|
| Backup Engine (5.2) | Task 5 |
|
|
| API Endpoints (5.1) | Tasks 6, 7, 14, 15 |
|
|
| Scheduler (5.3) | Task 17 |
|
|
| Dashboard View (6.1) | Tasks 10, 14 |
|
|
| Backups View (6.2) | Task 11 |
|
|
| Settings View (6.3) | Task 12 |
|
|
| Backup Flow (7.1) | Task 5 |
|
|
| Error Handling (8) | Task 5 |
|
|
| Retention Policy | Task 18 |
|
|
| Frontend Setup | Tasks 1, 9, 13 |
|
|
| Static File Serving | Task 19 |
|
|
|
|
**Coverage:** All spec requirements have corresponding implementation tasks.
|
|
|
|
### Placeholder Scan
|
|
|
|
No placeholders found. All steps contain complete code.
|
|
|
|
### Type Consistency
|
|
|
|
- Database models and Pydantic schemas are consistent
|
|
- API endpoints return correct schema types
|
|
- Adapter interface methods match across implementations
|
|
|
|
---
|
|
|
|
## Execution Handoff
|
|
|
|
**Plan complete and saved to `docs/superpowers/plans/2026-05-11-backup-tool-implementation.md`.**
|
|
|
|
**Two execution options:**
|
|
|
|
**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration
|
|
|
|
**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints for review
|
|
|
|
**Which approach?**
|