headquarter/docs/architecture/deployment.md

Deployment Architecture

Overview

Headquarter is designed for containerized deployment using Docker, with support for both development and production environments.

System Architecture

┌─────────────────────────────────────────────────────────────┐
│                         Internet                             │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────┐
│                    Traefik (Reverse Proxy)                   │
│              SSL/TLS termination, routing                    │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │  app.domain  │  │ api.domain   │  │ auth.domain  │      │
│  │  (Frontend)  │  │ (Backend)    │  │ (Authentik)  │      │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘      │
└─────────┼─────────────────┼─────────────────┼──────────────┘
          │                 │                 │
          ▼                 ▼                 ▼
┌─────────────────────────────────────────────────────────────┐
│                  Docker Host / Server                        │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │   Frontend   │  │    API       │  │  Authentik   │      │
│  │   (nginx)    │  │  (FastAPI)   │  │   (OAuth2)   │      │
│  │   Port 80    │  │  Port 8000   │  │   Port 9443  │      │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘      │
│         │                 │                 │               │
│         └─────────────────┼─────────────────┘               │
│                           │                                  │
│                  ┌────────┴────────┐                        │
│                  │                 │                        │
│          ┌───────▼───────┐ ┌──────▼───────┐               │
│          │  PostgreSQL   │ │    Redis     │               │
│          │   Port 5432   │ │   Port 6379  │               │
│          └───────────────┘ └──────────────┘               │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │                    Volumes                            │  │
│  │  postgres_data  │  repo_data  │  avatar_uploads      │  │
│  └──────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Components

Frontend (Web)

• Technology: React + Vite + nginx
• Port: 80 (internal)
• Role: User interface
• Scaling: Static files, easily scaled horizontally

Backend (API)

• Technology: FastAPI + Python 3.11
• Port: 8000 (internal)
• Role: Business logic, API endpoints
• Scaling: Stateless, can scale horizontally

Database (PostgreSQL)

• Technology: PostgreSQL 15
• Port: 5432 (internal)
• Role: Persistent data storage
• Scaling: Vertical or read replicas

Cache (Redis)

• Technology: Redis 7
• Port: 6379 (internal)
• Role: Session storage, caching
• Scaling: Redis Cluster for high availability

Identity Provider (Authentik)

• Technology: Authentik (self-hosted)
• Port: 9443 (external), 9000 (internal)
• Role: OAuth2/OIDC authentication
• Note: Can be external service

Network Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Networks                                │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  External Network (traefik)                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │   Traefik    │  │   Frontend   │  │     API      │      │
│  │   (proxy)    │  │   (web)      │  │   (api)      │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
│                                                              │
│  Internal Network (backend)                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │     API      │  │  PostgreSQL  │  │    Redis     │      │
│  │   (api)      │  │   (postgres) │  │   (redis)    │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Network Security:

• External network: Exposes services to Traefik
• Internal network: Database and cache only accessible by API
• No direct database access from external network

Data Flow

Authentication Flow

User → Frontend → API → Authentik
                       ↓
                    OAuth2
                       ↓
User ← Frontend ← API ← Authentik
        ↓
    Session Cookie

API Request Flow

User → Frontend → Traefik → API → Database
                              ↓
                           Redis (cache)

Git Operations Flow

User → Frontend → API → Git Repository (filesystem)
                              ↓
                        Git History/Files

Deployment Patterns

Single Server

All services on one host:

• Simple to manage
• Suitable for small teams
• Single point of failure

Multi-Server (HA)

Separate services across hosts:

• Database server
• Application servers (API + Frontend)
• Load balancer (Traefik)
• Higher availability

Kubernetes (Future)

Container orchestration:

• Auto-scaling
• Self-healing
• Rolling updates
• Resource management

Scaling Strategy

Horizontal Scaling

Stateless Services (easy to scale):

• Frontend: Multiple nginx instances
• API: Multiple FastAPI instances

Stateful Services (require care):

• Database: Read replicas, connection pooling
• Redis: Cluster mode

Vertical Scaling

Increase resources for:

• Database server (CPU, RAM, I/O)
• API server (CPU for git operations)

Backup Strategy

Automated Backups

Daily at 2 AM
├── PostgreSQL dump
├── Repository filesystem
├── User uploads (avatars)
└── Configuration files

Backup Retention

• Daily: 7 days
• Weekly: 4 weeks
• Monthly: 12 months
• Yearly: 3 years

Disaster Recovery

1. Restore database from backup
2. Restore repositories from backup
3. Verify application functionality
4. Update DNS if needed

Security Considerations

Network Security

• Internal services not exposed externally
• Database only accessible from API
• Redis only accessible from API
• SSL/TLS for all external traffic

Data Security

• Encrypted database connections
• Encrypted backups
• SSH keys encrypted at rest
• Session cookies httpOnly + Secure

Access Control

• OAuth2 authentication
• Role-based access (future)
• API rate limiting
• Audit logging (future)

Monitoring

Health Checks

API: GET /health
Database: pg_isready
Redis: redis-cli ping

Metrics

• Request rate and latency
• Error rate
• Database connections
• Disk usage
• Memory usage

Logging

• Application logs (structured JSON)
• Access logs (Traefik)
• Error logs (centralized)
• Audit logs (future)

Performance Optimization

Database

• Connection pooling (PgBouncer)
• Query optimization
• Proper indexing
• Regular VACUUM

API

• Async operations
• Caching (Redis)
• Git operation optimization
• File streaming

Frontend

• Code splitting
• Lazy loading
• Asset optimization
• CDN (future)

Troubleshooting

Common Issues

High Memory Usage:

# Check container stats
docker stats

# Restart API if needed
docker compose restart api

Database Connection Issues:

# Check PostgreSQL logs
docker compose logs postgres

# Verify connection
docker compose exec api pg_isready -h postgres

Git Operations Slow:

# Check disk I/O
iostat -x 1

# Check repository size
du -sh /data/repos/*

Migration Strategy

Version Updates

1. Backup data
2. Update images
3. Run migrations
4. Verify functionality
5. Rollback if needed

Database Migrations

# Check current version
alembic current

# Upgrade
alembic upgrade head

# Downgrade if needed
alembic downgrade -1

Future Architecture

Planned Improvements

• Kubernetes deployment
• Microservices split
• Event-driven architecture
• Real-time WebSocket updates
• Multi-region deployment
• CDN integration
• Advanced monitoring (Prometheus/Grafana)

Scalability Roadmap

1. Phase 1: Single server (current)
2. Phase 2: Separate database server
3. Phase 3: Load balanced API servers
4. Phase 4: Kubernetes cluster
5. Phase 5: Multi-region