docs: comprehensive documentation overhaul
Add complete documentation structure: - Frontend architecture documentation - Database schema documentation - Deployment guides (Docker, Traefik, Authentik, Environment) - Development guides (Setup, Testing, Contributing, Quality Gates) - Deployment architecture documentation - Updated docs README with complete navigation All new features and APIs are now documented. Quality gates: docs only, no code changes
This commit is contained in:
Fusion
@@ -0,0 +1,212 @@
|
||||
# Backend Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The Headquarter backend is built with **FastAPI** and follows a layered architecture pattern.
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ FastAPI App │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Middleware: CORS → Request Logging → Exception Logging │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ API Layer (src/api/) │
|
||||
│ ┌─────────┐ ┌──────────┐ ┌────────┐ ┌──────────┐ │
|
||||
│ │ Auth │ │ Projects │ │ Users │ │ Git │ │
|
||||
│ │ Routes │ │ Routes │ │ Routes │ │ Repos │ │
|
||||
│ └────┬────┘ └────┬─────┘ └───┬────┘ └────┬─────┘ │
|
||||
├───────┼───────────┼───────────┼───────────┼─────────────────┤
|
||||
│ │ │ │ │ │
|
||||
│ Auth │ Project │ User │ Git │ │
|
||||
│ Layer │ Service │ Service │ Service │ │
|
||||
│ │ │ │ │ │
|
||||
├───────┴───────────┴───────────┴───────────┴─────────────────┤
|
||||
│ Data Layer │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Models │ │ Database │ │ Config │ │
|
||||
│ │(SQLAlch) │ │(AsyncPG) │ │(Pydantic)│ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── api/ # API Routes
|
||||
│ ├── auth.py # Authentication endpoints
|
||||
│ ├── projects.py # Project endpoints
|
||||
│ ├── git_repositories.py # Repository endpoints
|
||||
│ ├── users.py # User endpoints
|
||||
│ ├── tool_types.py # Tool type endpoints
|
||||
│ ├── ssh_keys.py # SSH key endpoints
|
||||
│ └── dashboard.py # Dashboard endpoints
|
||||
├── auth/ # Authentication
|
||||
│ ├── session.py # Session management
|
||||
│ ├── oidc.py # OAuth2 client
|
||||
│ ├── dependencies.py # Auth dependencies
|
||||
│ └── cookies.py # Cookie utilities
|
||||
├── models/ # Database Models
|
||||
│ ├── user.py # User model
|
||||
│ ├── project.py # Project model
|
||||
│ ├── git_repository.py # Repository model
|
||||
│ ├── tool_type.py # Tool type model
|
||||
│ ├── ssh_key.py # SSH key model
|
||||
│ └── user_config.py # User config model
|
||||
├── utils/ # Utilities
|
||||
│ ├── git_url_parser.py # URL parsing
|
||||
│ ├── git_files.py # Git file operations
|
||||
│ └── git_history.py # Git history operations
|
||||
├── config.py # Configuration
|
||||
├── database.py # Database setup
|
||||
└── main.py # Application entry point
|
||||
```
|
||||
|
||||
## Layers
|
||||
|
||||
### 1. API Layer (`src/api/`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Define HTTP endpoints
|
||||
- Parse request parameters
|
||||
- Return HTTP responses
|
||||
- Use dependencies for auth and DB
|
||||
|
||||
**Pattern:**
|
||||
```python
|
||||
@router.get("/projects")
|
||||
async def list_projects(
|
||||
session: AsyncSession = Depends(get_db_session),
|
||||
user_id: str = Depends(get_current_user_id),
|
||||
):
|
||||
# Call service layer
|
||||
projects = await project_service.list(session, user_id)
|
||||
return projects
|
||||
```
|
||||
|
||||
### 2. Auth Layer (`src/auth/`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Session management (create, verify, expire)
|
||||
- OAuth2 flow (login, callback)
|
||||
- User authentication dependencies
|
||||
|
||||
**Key Components:**
|
||||
- `session.py`: HMAC-SHA256 signed cookies
|
||||
- `oidc.py`: OAuth2 token exchange
|
||||
- `dependencies.py`: FastAPI dependencies for auth
|
||||
|
||||
### 3. Data Layer (`src/models/`, `src/database.py`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Database schema definition
|
||||
- Async database sessions
|
||||
- Connection management
|
||||
|
||||
**Technology:**
|
||||
- SQLAlchemy 2.0 with async support
|
||||
- asyncpg driver for PostgreSQL
|
||||
- Alembic for migrations
|
||||
|
||||
### 4. Utility Layer (`src/utils/`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Git operations (file browsing, history)
|
||||
- URL parsing
|
||||
- Helper functions
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Request Lifecycle
|
||||
|
||||
```
|
||||
1. Request arrives at FastAPI
|
||||
2. Middleware processes (CORS, logging)
|
||||
3. Auth dependency verifies session
|
||||
4. Route handler processes request
|
||||
5. Database session executes queries
|
||||
6. Response returned to client
|
||||
```
|
||||
|
||||
### Authentication Flow
|
||||
|
||||
```
|
||||
1. User clicks login
|
||||
2. Backend redirects to Authentik
|
||||
3. User authenticates
|
||||
4. Authentik redirects with code
|
||||
5. Backend exchanges code for token
|
||||
6. Backend fetches user info
|
||||
7. Backend creates session cookie
|
||||
8. User is authenticated
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Database Session
|
||||
|
||||
```python
|
||||
async def get_db_session() -> AsyncGenerator[AsyncSession, None]:
|
||||
async with SessionLocal() as session:
|
||||
yield session
|
||||
```
|
||||
|
||||
### Current User
|
||||
|
||||
```python
|
||||
async def get_current_user_id(
|
||||
request: Request,
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> str:
|
||||
# Verify session cookie
|
||||
# Return user_id
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuration is managed via Pydantic Settings:
|
||||
|
||||
```python
|
||||
class Settings(BaseSettings):
|
||||
app_env: str = "development"
|
||||
database_url: str = "..."
|
||||
authentik_domain: str = "..."
|
||||
# ...
|
||||
```
|
||||
|
||||
Environment variables are automatically loaded from `.env` files.
|
||||
|
||||
## Error Handling
|
||||
|
||||
Errors are handled at multiple levels:
|
||||
|
||||
1. **Validation**: Pydantic validates request bodies
|
||||
2. **HTTP Exceptions**: FastAPI HTTPException for client errors
|
||||
3. **Middleware**: ExceptionLoggingMiddleware logs server errors
|
||||
4. **Database**: SQLAlchemy errors converted to HTTP responses
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit tests**: SQLite in-memory database
|
||||
- **Integration tests**: PostgreSQL with transaction rollback
|
||||
- **Fixtures**: Shared in `conftest.py`
|
||||
|
||||
## Technology Stack
|
||||
|
||||
| Component | Technology | Version |
|
||||
|-----------|-----------|---------|
|
||||
| Web Framework | FastAPI | ^0.104 |
|
||||
| ORM | SQLAlchemy | ^2.0 |
|
||||
| Database Driver | asyncpg | ^0.29 |
|
||||
| Validation | Pydantic | ^2.0 |
|
||||
| Migrations | Alembic | ^1.12 |
|
||||
| HTTP Client | httpx | ^0.25 |
|
||||
| Testing | pytest | ^7.4 |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Database Schema](database.md)
|
||||
- [Frontend Architecture](../frontend.md)
|
||||
- [API Documentation](../api/)
|
||||
@@ -0,0 +1,231 @@
|
||||
# Database Schema
|
||||
|
||||
## Overview
|
||||
|
||||
Headquarter uses PostgreSQL with SQLAlchemy ORM and Alembic for migrations. All tables use UUID primary keys and include `created_at`/`updated_at` timestamps.
|
||||
|
||||
## Entity Relationship Diagram
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌─────────────────┐ ┌──────────────┐
|
||||
│ users │ │ git_repository │ │ project │
|
||||
├──────────────┤ ├─────────────────┤ ├──────────────┤
|
||||
│ id (PK) │ │ id (PK) │ │ id (PK) │
|
||||
│ email │ │ project_id (FK) │──┐ │ name │
|
||||
│ name │ │ name │ │ │ description │
|
||||
│ authentik_id │ │ remote_url │ │ │ created_by_id│──┐
|
||||
│ avatar_url │ │ local_path │ │ │ │ │
|
||||
│ created_at │ │ is_mirror │ │ │ │ │
|
||||
│ updated_at │ │ created_by_id │──┤ │ │ │
|
||||
└──────────────┘ │ created_at │ │ └──────────────┘ │
|
||||
│ │ updated_at │ │ ▲ │
|
||||
│ └─────────────────┘ │ │ │
|
||||
│ │ │ │ │
|
||||
│ ┌───────┘ │ │ │
|
||||
│ │ │ │ │
|
||||
┌───────▼──────┐ ┌▼────────────────┐ │ │ │
|
||||
│ ssh_keys │ │ user_config │ │ │ │
|
||||
├──────────────┤ ├─────────────────┤ │ │ │
|
||||
│ id (PK) │ │ user_id (FK) │───┘ │ │
|
||||
│ user_id (FK) │ │ theme │ │ │
|
||||
│ name │ │ git_name │ │ │
|
||||
│ public_key │ │ git_email │ │ │
|
||||
│ private_key │ │ default_editor │ │ │
|
||||
│ created_at │ │ created_at │ │ │
|
||||
│ updated_at │ │ updated_at │ │ │
|
||||
└──────────────┘ └─────────────────┘ │ │
|
||||
│ │
|
||||
┌───────────────────────┘ │
|
||||
│ │
|
||||
▼ │
|
||||
┌─────────────────┐ │
|
||||
│ tool_types │ │
|
||||
├─────────────────┤ │
|
||||
│ id (PK) │ │
|
||||
│ name │ │
|
||||
│ display_name │ │
|
||||
│ description │ │
|
||||
│ compose_template│ │
|
||||
│ required_vars │ │
|
||||
│ is_builtin │ │
|
||||
│ created_by_id │─────────────────────────┘
|
||||
│ created_at │
|
||||
│ updated_at │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## Table Definitions
|
||||
|
||||
### users
|
||||
|
||||
Stores user accounts synchronized from Authentik.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|--------|------|-------------|-------------|
|
||||
| id | UUID | PK | Unique identifier |
|
||||
| email | VARCHAR(255) | NOT NULL, UNIQUE | User email |
|
||||
| name | VARCHAR(255) | | Display name |
|
||||
| authentik_id | VARCHAR(255) | UNIQUE | Authentik user ID |
|
||||
| avatar_url | VARCHAR(500) | | Profile avatar URL |
|
||||
| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp |
|
||||
| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp |
|
||||
|
||||
**Relationships**:
|
||||
- One-to-Many: `users` → `git_repository` (created_by_id)
|
||||
- One-to-Many: `users` → `project` (created_by_id)
|
||||
- One-to-Many: `users` → `ssh_keys` (user_id)
|
||||
- One-to-One: `users` → `user_config` (user_id)
|
||||
- One-to-Many: `users` → `tool_types` (created_by_id)
|
||||
|
||||
### project
|
||||
|
||||
Organizes repositories into logical groups.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|--------|------|-------------|-------------|
|
||||
| id | UUID | PK | Unique identifier |
|
||||
| name | VARCHAR(255) | NOT NULL | Project name |
|
||||
| description | TEXT | | Project description |
|
||||
| created_by_id | UUID | FK → users.id | Creator |
|
||||
| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp |
|
||||
| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp |
|
||||
|
||||
**Relationships**:
|
||||
- One-to-Many: `project` → `git_repository` (project_id)
|
||||
|
||||
### git_repository
|
||||
|
||||
Git repositories (bare/mirror clones).
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|--------|------|-------------|-------------|
|
||||
| id | UUID | PK | Unique identifier |
|
||||
| project_id | UUID | FK → project.id, NOT NULL | Parent project |
|
||||
| name | VARCHAR(255) | NOT NULL | Repository name |
|
||||
| remote_url | VARCHAR(500) | NOT NULL | Remote git URL |
|
||||
| local_path | VARCHAR(500) | | Local filesystem path |
|
||||
| is_mirror | BOOLEAN | DEFAULT false | Is mirror clone |
|
||||
| created_by_id | UUID | FK → users.id | Creator |
|
||||
| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp |
|
||||
| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp |
|
||||
|
||||
**Indexes**:
|
||||
- `idx_repo_project`: (project_id)
|
||||
- `idx_repo_name`: (project_id, name) - UNIQUE
|
||||
|
||||
### ssh_keys
|
||||
|
||||
User SSH keys for git authentication.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|--------|------|-------------|-------------|
|
||||
| id | UUID | PK | Unique identifier |
|
||||
| user_id | UUID | FK → users.id, NOT NULL | Owner |
|
||||
| name | VARCHAR(255) | NOT NULL | Key name |
|
||||
| public_key | TEXT | NOT NULL | Public key |
|
||||
| private_key | TEXT | NOT NULL, ENCRYPTED | Encrypted private key |
|
||||
| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp |
|
||||
| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp |
|
||||
|
||||
**Indexes**:
|
||||
- `idx_ssh_user`: (user_id)
|
||||
|
||||
### user_config
|
||||
|
||||
User preferences and settings.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|--------|------|-------------|-------------|
|
||||
| id | UUID | PK | Unique identifier |
|
||||
| user_id | UUID | FK → users.id, NOT NULL, UNIQUE | Owner |
|
||||
| theme | VARCHAR(50) | DEFAULT 'system' | UI theme (system/light/dark) |
|
||||
| git_name | VARCHAR(255) | | Git user name |
|
||||
| git_email | VARCHAR(255) | | Git user email |
|
||||
| default_editor | VARCHAR(50) | DEFAULT 'vscode' | Preferred editor |
|
||||
| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp |
|
||||
| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp |
|
||||
|
||||
### tool_types
|
||||
|
||||
Types of development tools that can be spawned.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|--------|------|-------------|-------------|
|
||||
| id | UUID | PK | Unique identifier |
|
||||
| name | VARCHAR(255) | NOT NULL, UNIQUE | Machine name |
|
||||
| display_name | VARCHAR(255) | NOT NULL | Human-readable name |
|
||||
| description | TEXT | | Description |
|
||||
| compose_template | TEXT | NOT NULL | Docker Compose template |
|
||||
| required_variables | JSONB | NOT NULL | Template variables |
|
||||
| is_builtin | BOOLEAN | DEFAULT false | Built-in type |
|
||||
| created_by_id | UUID | FK → users.id | Creator (null for built-in) |
|
||||
| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp |
|
||||
| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp |
|
||||
|
||||
**Indexes**:
|
||||
- `idx_tool_builtin`: (is_builtin)
|
||||
|
||||
## Migration History
|
||||
|
||||
| Version | Date | Description |
|
||||
|---------|------|-------------|
|
||||
| 0001_initial_schema | 2024-01-XX | Initial tables: users, projects, git_repositories |
|
||||
| 0002_refresh_tokens | 2024-01-XX | Added refresh_tokens table |
|
||||
| 0003_user_configs | 2024-05-18 | Added user_config table |
|
||||
| 0004_tool_types | 2024-05-18 | Added tool_types table |
|
||||
|
||||
## Data Types
|
||||
|
||||
### PostgreSQL Types
|
||||
- **UUID**: `uuid` - All primary keys
|
||||
- **Timestamps**: `TIMESTAMP WITH TIME ZONE`
|
||||
- **JSONB**: `JSONB` - For flexible config (user_config, tool_types)
|
||||
- **Strings**: `VARCHAR(n)` - With appropriate length limits
|
||||
- **Text**: `TEXT` - For unbounded content
|
||||
- **Boolean**: `BOOLEAN` - True/false flags
|
||||
|
||||
### SQLAlchemy Configuration
|
||||
```python
|
||||
# Base model features
|
||||
class Base:
|
||||
id: UUID = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||
created_at: datetime = Column(DateTime(timezone=True), server_default=func.now())
|
||||
updated_at: datetime = Column(DateTime(timezone=True), onupdate=func.now())
|
||||
```
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### Automated Backups
|
||||
- **Frequency**: Daily at 2 AM
|
||||
- **Retention**: 7 daily, 4 weekly, 12 monthly
|
||||
- **Method**: `pg_dump` to S3/object storage
|
||||
- **Encryption**: AES-256 encrypted backups
|
||||
|
||||
### Manual Backup
|
||||
```bash
|
||||
# Full backup
|
||||
pg_dump -Fc -f headquarter_backup.dump postgresql://user:pass@host/db
|
||||
|
||||
# Restore
|
||||
pg_restore -d postgresql://user:pass@host/db headquarter_backup.dump
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
### Query Optimization
|
||||
- All foreign keys indexed
|
||||
- Frequently queried columns indexed
|
||||
- Composite indexes for multi-column queries
|
||||
|
||||
### Connection Pooling
|
||||
- SQLAlchemy async pool: 5-20 connections
|
||||
- PgBouncer for production: transaction mode
|
||||
|
||||
## Future Schema Changes
|
||||
|
||||
Planned additions:
|
||||
- [ ] **teams** table - Group users into teams
|
||||
- [ ] **team_memberships** table - Link users to teams
|
||||
- [ ] **tool_instances** table - Running tool containers
|
||||
- [ ] **audit_logs** table - Track important actions
|
||||
- [ ] **notifications** table - User notifications
|
||||
@@ -0,0 +1,357 @@
|
||||
# Deployment Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
Headquarter is designed for containerized deployment using Docker, with support for both development and production environments.
|
||||
|
||||
## System Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Internet │
|
||||
└──────────────────────┬──────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Traefik (Reverse Proxy) │
|
||||
│ SSL/TLS termination, routing │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ app.domain │ │ api.domain │ │ auth.domain │ │
|
||||
│ │ (Frontend) │ │ (Backend) │ │ (Authentik) │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
|
||||
└─────────┼─────────────────┼─────────────────┼──────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Docker Host / Server │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Frontend │ │ API │ │ Authentik │ │
|
||||
│ │ (nginx) │ │ (FastAPI) │ │ (OAuth2) │ │
|
||||
│ │ Port 80 │ │ Port 8000 │ │ Port 9443 │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
|
||||
│ │ │ │ │
|
||||
│ └─────────────────┼─────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ │ │
|
||||
│ ┌───────▼───────┐ ┌──────▼───────┐ │
|
||||
│ │ PostgreSQL │ │ Redis │ │
|
||||
│ │ Port 5432 │ │ Port 6379 │ │
|
||||
│ └───────────────┘ └──────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ Volumes │ │
|
||||
│ │ postgres_data │ repo_data │ avatar_uploads │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
### Frontend (Web)
|
||||
|
||||
- **Technology**: React + Vite + nginx
|
||||
- **Port**: 80 (internal)
|
||||
- **Role**: User interface
|
||||
- **Scaling**: Static files, easily scaled horizontally
|
||||
|
||||
### Backend (API)
|
||||
|
||||
- **Technology**: FastAPI + Python 3.11
|
||||
- **Port**: 8000 (internal)
|
||||
- **Role**: Business logic, API endpoints
|
||||
- **Scaling**: Stateless, can scale horizontally
|
||||
|
||||
### Database (PostgreSQL)
|
||||
|
||||
- **Technology**: PostgreSQL 15
|
||||
- **Port**: 5432 (internal)
|
||||
- **Role**: Persistent data storage
|
||||
- **Scaling**: Vertical or read replicas
|
||||
|
||||
### Cache (Redis)
|
||||
|
||||
- **Technology**: Redis 7
|
||||
- **Port**: 6379 (internal)
|
||||
- **Role**: Session storage, caching
|
||||
- **Scaling**: Redis Cluster for high availability
|
||||
|
||||
### Identity Provider (Authentik)
|
||||
|
||||
- **Technology**: Authentik (self-hosted)
|
||||
- **Port**: 9443 (external), 9000 (internal)
|
||||
- **Role**: OAuth2/OIDC authentication
|
||||
- **Note**: Can be external service
|
||||
|
||||
## Network Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Networks │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ External Network (traefik) │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Traefik │ │ Frontend │ │ API │ │
|
||||
│ │ (proxy) │ │ (web) │ │ (api) │ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
||||
│ │
|
||||
│ Internal Network (backend) │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ API │ │ PostgreSQL │ │ Redis │ │
|
||||
│ │ (api) │ │ (postgres) │ │ (redis) │ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Network Security**:
|
||||
- External network: Exposes services to Traefik
|
||||
- Internal network: Database and cache only accessible by API
|
||||
- No direct database access from external network
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Authentication Flow
|
||||
|
||||
```
|
||||
User → Frontend → API → Authentik
|
||||
↓
|
||||
OAuth2
|
||||
↓
|
||||
User ← Frontend ← API ← Authentik
|
||||
↓
|
||||
Session Cookie
|
||||
```
|
||||
|
||||
### API Request Flow
|
||||
|
||||
```
|
||||
User → Frontend → Traefik → API → Database
|
||||
↓
|
||||
Redis (cache)
|
||||
```
|
||||
|
||||
### Git Operations Flow
|
||||
|
||||
```
|
||||
User → Frontend → API → Git Repository (filesystem)
|
||||
↓
|
||||
Git History/Files
|
||||
```
|
||||
|
||||
## Deployment Patterns
|
||||
|
||||
### Single Server
|
||||
|
||||
All services on one host:
|
||||
- Simple to manage
|
||||
- Suitable for small teams
|
||||
- Single point of failure
|
||||
|
||||
### Multi-Server (HA)
|
||||
|
||||
Separate services across hosts:
|
||||
- Database server
|
||||
- Application servers (API + Frontend)
|
||||
- Load balancer (Traefik)
|
||||
- Higher availability
|
||||
|
||||
### Kubernetes (Future)
|
||||
|
||||
Container orchestration:
|
||||
- Auto-scaling
|
||||
- Self-healing
|
||||
- Rolling updates
|
||||
- Resource management
|
||||
|
||||
## Scaling Strategy
|
||||
|
||||
### Horizontal Scaling
|
||||
|
||||
**Stateless Services** (easy to scale):
|
||||
- Frontend: Multiple nginx instances
|
||||
- API: Multiple FastAPI instances
|
||||
|
||||
**Stateful Services** (require care):
|
||||
- Database: Read replicas, connection pooling
|
||||
- Redis: Cluster mode
|
||||
|
||||
### Vertical Scaling
|
||||
|
||||
Increase resources for:
|
||||
- Database server (CPU, RAM, I/O)
|
||||
- API server (CPU for git operations)
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### Automated Backups
|
||||
|
||||
```
|
||||
Daily at 2 AM
|
||||
├── PostgreSQL dump
|
||||
├── Repository filesystem
|
||||
├── User uploads (avatars)
|
||||
└── Configuration files
|
||||
```
|
||||
|
||||
### Backup Retention
|
||||
|
||||
- Daily: 7 days
|
||||
- Weekly: 4 weeks
|
||||
- Monthly: 12 months
|
||||
- Yearly: 3 years
|
||||
|
||||
### Disaster Recovery
|
||||
|
||||
1. Restore database from backup
|
||||
2. Restore repositories from backup
|
||||
3. Verify application functionality
|
||||
4. Update DNS if needed
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Network Security
|
||||
|
||||
- Internal services not exposed externally
|
||||
- Database only accessible from API
|
||||
- Redis only accessible from API
|
||||
- SSL/TLS for all external traffic
|
||||
|
||||
### Data Security
|
||||
|
||||
- Encrypted database connections
|
||||
- Encrypted backups
|
||||
- SSH keys encrypted at rest
|
||||
- Session cookies httpOnly + Secure
|
||||
|
||||
### Access Control
|
||||
|
||||
- OAuth2 authentication
|
||||
- Role-based access (future)
|
||||
- API rate limiting
|
||||
- Audit logging (future)
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Health Checks
|
||||
|
||||
```
|
||||
API: GET /health
|
||||
Database: pg_isready
|
||||
Redis: redis-cli ping
|
||||
```
|
||||
|
||||
### Metrics
|
||||
|
||||
- Request rate and latency
|
||||
- Error rate
|
||||
- Database connections
|
||||
- Disk usage
|
||||
- Memory usage
|
||||
|
||||
### Logging
|
||||
|
||||
- Application logs (structured JSON)
|
||||
- Access logs (Traefik)
|
||||
- Error logs (centralized)
|
||||
- Audit logs (future)
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
### Database
|
||||
|
||||
- Connection pooling (PgBouncer)
|
||||
- Query optimization
|
||||
- Proper indexing
|
||||
- Regular VACUUM
|
||||
|
||||
### API
|
||||
|
||||
- Async operations
|
||||
- Caching (Redis)
|
||||
- Git operation optimization
|
||||
- File streaming
|
||||
|
||||
### Frontend
|
||||
|
||||
- Code splitting
|
||||
- Lazy loading
|
||||
- Asset optimization
|
||||
- CDN (future)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**High Memory Usage**:
|
||||
```bash
|
||||
# Check container stats
|
||||
docker stats
|
||||
|
||||
# Restart API if needed
|
||||
docker compose restart api
|
||||
```
|
||||
|
||||
**Database Connection Issues**:
|
||||
```bash
|
||||
# Check PostgreSQL logs
|
||||
docker compose logs postgres
|
||||
|
||||
# Verify connection
|
||||
docker compose exec api pg_isready -h postgres
|
||||
```
|
||||
|
||||
**Git Operations Slow**:
|
||||
```bash
|
||||
# Check disk I/O
|
||||
iostat -x 1
|
||||
|
||||
# Check repository size
|
||||
du -sh /data/repos/*
|
||||
```
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Version Updates
|
||||
|
||||
1. Backup data
|
||||
2. Update images
|
||||
3. Run migrations
|
||||
4. Verify functionality
|
||||
5. Rollback if needed
|
||||
|
||||
### Database Migrations
|
||||
|
||||
```bash
|
||||
# Check current version
|
||||
alembic current
|
||||
|
||||
# Upgrade
|
||||
alembic upgrade head
|
||||
|
||||
# Downgrade if needed
|
||||
alembic downgrade -1
|
||||
```
|
||||
|
||||
## Future Architecture
|
||||
|
||||
### Planned Improvements
|
||||
|
||||
- [ ] Kubernetes deployment
|
||||
- [ ] Microservices split
|
||||
- [ ] Event-driven architecture
|
||||
- [ ] Real-time WebSocket updates
|
||||
- [ ] Multi-region deployment
|
||||
- [ ] CDN integration
|
||||
- [ ] Advanced monitoring (Prometheus/Grafana)
|
||||
|
||||
### Scalability Roadmap
|
||||
|
||||
1. **Phase 1**: Single server (current)
|
||||
2. **Phase 2**: Separate database server
|
||||
3. **Phase 3**: Load balanced API servers
|
||||
4. **Phase 4**: Kubernetes cluster
|
||||
5. **Phase 5**: Multi-region
|
||||
@@ -0,0 +1,297 @@
|
||||
# Frontend Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The Headquarter frontend is a React-based single-page application (SPA) built with modern tooling and designed for modularity and maintainability.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Layer | Technology | Version |
|
||||
|-------|-----------|---------|
|
||||
| Framework | React | ^18.2.0 |
|
||||
| Router | React Router | ^6.20.0 |
|
||||
| Bundler | Vite | ^5.0.0 |
|
||||
| Language | TypeScript | ^5.3.0 |
|
||||
| Styling | CSS3 with CSS Variables | - |
|
||||
| Testing | Vitest + React Testing Library | ^4.1.6 |
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
apps/web/src/
|
||||
├── api/ # API clients
|
||||
│ ├── auth.ts # Authentication API
|
||||
│ ├── projects.ts # Project API
|
||||
│ ├── git_repositories.ts # Repository API
|
||||
│ ├── ssh_keys.ts # SSH key API
|
||||
│ ├── tool_types.ts # Tool type API
|
||||
│ ├── users.ts # User API
|
||||
│ └── settings.ts # Settings API
|
||||
├── components/ # Reusable components
|
||||
│ ├── app-shell.tsx # Main app layout
|
||||
│ ├── protected-route.tsx # Auth guard
|
||||
│ └── [more...]
|
||||
├── context/ # React contexts
|
||||
│ └── auth.tsx # Auth state management
|
||||
├── hooks/ # Custom hooks
|
||||
│ ├── use-auth.ts # Auth hook
|
||||
│ └── use-theme.ts # Theme hook
|
||||
├── pages/ # Page components (routes)
|
||||
│ ├── dashboard.tsx # Dashboard
|
||||
│ ├── projects.tsx # Project list
|
||||
│ ├── repo-workspace.tsx # Repository workspace
|
||||
│ ├── git-history.tsx # Git history
|
||||
│ ├── git-repositories.tsx # Repository management
|
||||
│ ├── profile.tsx # User profile
|
||||
│ ├── settings.tsx # User settings
|
||||
│ ├── tool-types.tsx # Tool types
|
||||
│ ├── ssh-keys.tsx # SSH keys
|
||||
│ └── [more...]
|
||||
├── styles/ # Global styles
|
||||
│ ├── index.css # Main stylesheet
|
||||
│ └── [more...]
|
||||
├── types.ts # Shared TypeScript types
|
||||
├── router.tsx # Route definitions
|
||||
└── main.tsx # Entry point
|
||||
```
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### 1. Component Architecture
|
||||
|
||||
**Page Components**: Top-level components mapped to routes
|
||||
- Own data fetching
|
||||
- Manage page-level state
|
||||
- Compose reusable components
|
||||
|
||||
**Reusable Components**: Shared UI elements
|
||||
- No data fetching
|
||||
- Receive data via props
|
||||
- Emit events via callbacks
|
||||
|
||||
**Example**:
|
||||
```typescript
|
||||
// Page component
|
||||
const RepoWorkspace = () => {
|
||||
const [files, setFiles] = useState([]);
|
||||
// ... fetch data, manage state
|
||||
return (
|
||||
<div className="workspace">
|
||||
<FileTree files={files} onFileClick={handleFileClick} />
|
||||
<FileViewer content={content} />
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
// Reusable component
|
||||
const FileTree = ({ files, onFileClick }: FileTreeProps) => {
|
||||
return (
|
||||
<ul>
|
||||
{files.map(file => (
|
||||
<li onClick={() => onFileClick(file)}>{file.name}</li>
|
||||
))}
|
||||
</ul>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### 2. State Management
|
||||
|
||||
**URL State**: Shareable, bookmarkable state
|
||||
```typescript
|
||||
// Sync selections to URL
|
||||
const [searchParams, setSearchParams] = useSearchParams();
|
||||
// ?repo=123&branch=main&path=src/main.py
|
||||
```
|
||||
|
||||
**React Context**: Global auth state
|
||||
```typescript
|
||||
// Auth context provides user, login, logout
|
||||
const { user, isAuthenticated } = useAuth();
|
||||
```
|
||||
|
||||
**Local State**: Component-specific state
|
||||
```typescript
|
||||
const [isEditing, setIsEditing] = useState(false);
|
||||
```
|
||||
|
||||
### 3. API Client Pattern
|
||||
|
||||
Centralized API clients with type safety:
|
||||
|
||||
```typescript
|
||||
// api/git_repositories.ts
|
||||
export const getRepositories = async (projectId: string) => {
|
||||
const response = await fetch(`${API_BASE_URL}/projects/${projectId}/repositories`, {
|
||||
credentials: 'include',
|
||||
});
|
||||
return response.json();
|
||||
};
|
||||
|
||||
// Usage in component
|
||||
const repos = await getRepositories(projectId);
|
||||
```
|
||||
|
||||
### 4. Authentication Flow
|
||||
|
||||
```
|
||||
User clicks Login
|
||||
→ Redirect to /auth/login (backend)
|
||||
→ Backend redirects to Authentik OAuth
|
||||
→ User authenticates with Authentik
|
||||
→ Authentik redirects to /auth/callback
|
||||
→ Backend creates session cookie
|
||||
→ Backend redirects to frontend
|
||||
→ Frontend checks /auth/me
|
||||
→ User is authenticated!
|
||||
```
|
||||
|
||||
**Auth State**:
|
||||
```typescript
|
||||
interface AuthState {
|
||||
user: User | null;
|
||||
isAuthenticated: boolean;
|
||||
isLoading: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Routing Structure
|
||||
|
||||
```typescript
|
||||
// router.tsx
|
||||
<Route path="/" element={<Dashboard />} />
|
||||
<Route path="/projects" element={<ProjectsPage />} />
|
||||
<Route path="/projects/:projectId" element={<RepoWorkspace />} />
|
||||
<Route path="/projects/:projectId/repositories" element={<GitRepositories />} />
|
||||
<Route path="/projects/:projectId/repositories/:repoId/history" element={<GitHistory />} />
|
||||
<Route path="/profile" element={<ProfilePage />} />
|
||||
<Route path="/settings" element={<SettingsPage />} />
|
||||
<Route path="/ssh-keys" element={<SSHKeysPage />} />
|
||||
<Route path="/tool-types" element={<ToolTypesPage />} />
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Repository Workspace Example
|
||||
|
||||
```
|
||||
1. User clicks project
|
||||
→ Navigate to /projects/:id
|
||||
|
||||
2. RepoWorkspace mounts
|
||||
→ Fetch project repositories
|
||||
→ Select first repo (or from URL)
|
||||
|
||||
3. Repo selected
|
||||
→ Fetch branches
|
||||
→ Fetch file tree (default branch)
|
||||
|
||||
4. User clicks file
|
||||
→ Fetch file content
|
||||
→ Display in viewer
|
||||
→ Update URL: ?path=src/main.py
|
||||
|
||||
5. User switches branch
|
||||
→ Fetch file tree for branch
|
||||
→ Re-fetch current file if viewing
|
||||
→ Update URL: ?branch=develop
|
||||
```
|
||||
|
||||
## Component Communication
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ RepoWorkspace │
|
||||
│ ┌──────────┐ ┌──────────────┐ │
|
||||
│ │ FileTree │───▶│ FileViewer │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ onFileClick │ content │ │
|
||||
│ │ │ │ onEdit │ │
|
||||
│ └──────────┘ └──────────────┘ │
|
||||
│ ▲ │
|
||||
│ │ │
|
||||
│ ┌──────────┐ │
|
||||
│ │ Branch │───▶ fetch tree │
|
||||
│ │ Selector │ │
|
||||
│ └──────────┘ │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Styling Strategy
|
||||
|
||||
### CSS Variables (Design Tokens)
|
||||
```css
|
||||
:root {
|
||||
--color-primary: #007bff;
|
||||
--color-bg: #ffffff;
|
||||
--color-text: #333333;
|
||||
--sidebar-width: 250px;
|
||||
--border-radius: 4px;
|
||||
}
|
||||
|
||||
[data-theme="dark"] {
|
||||
--color-bg: #1a1a1a;
|
||||
--color-text: #e0e0e0;
|
||||
}
|
||||
```
|
||||
|
||||
### Component Styles
|
||||
- Each page/component has scoped CSS
|
||||
- Global utilities in `styles/index.css`
|
||||
- No CSS-in-JS library (keep it simple)
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests (Vitest)
|
||||
```typescript
|
||||
// Component test
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import { FileTree } from './file-tree';
|
||||
|
||||
test('renders file list', () => {
|
||||
const files = [{ name: 'test.py', type: 'file' }];
|
||||
render(<FileTree files={files} onFileClick={() => {}} />);
|
||||
expect(screen.getByText('test.py')).toBeInTheDocument();
|
||||
});
|
||||
```
|
||||
|
||||
### Test Coverage
|
||||
- Component rendering
|
||||
- User interactions
|
||||
- Auth state changes
|
||||
- API mocking
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
1. **Code Splitting**: Vite handles automatic chunking
|
||||
2. **Lazy Loading**: React.lazy() for heavy pages
|
||||
3. **Debouncing**: URL updates debounced (300ms)
|
||||
4. **Caching**: Browser caches API responses (ETags)
|
||||
5. **Optimistic UI**: Immediate feedback before API response
|
||||
|
||||
## Future Improvements
|
||||
|
||||
- [ ] Add React Query for server state management
|
||||
- [ ] Implement virtual scrolling for large file trees
|
||||
- [ ] Add service worker for offline support
|
||||
- [ ] Implement real-time updates (WebSocket)
|
||||
- [ ] Add error boundary components
|
||||
|
||||
## Development Workflow
|
||||
|
||||
```bash
|
||||
# Start dev server
|
||||
cd apps/web && npm run dev
|
||||
|
||||
# Run tests
|
||||
npm run test
|
||||
|
||||
# Type check
|
||||
npm run typecheck
|
||||
|
||||
# Lint
|
||||
npm run lint
|
||||
|
||||
# Build for production
|
||||
npm run build
|
||||
```
|
||||
Reference in New Issue
Block a user