diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..d2c565e --- /dev/null +++ b/docs/README.md @@ -0,0 +1,72 @@ +# Headquarter Documentation + +Welcome to the Headquarter documentation. This is your central hub for understanding, using, and contributing to the platform. + +## Quick Links + +- [Features](features/) - Learn about available features +- [API Reference](api/) - API endpoint documentation +- [Architecture](architecture/) - System design and architecture +- [Deployment](deployment/) - Setup and deployment guides +- [Development](development/) - Development and contributing guides + +## Getting Started + +New to Headquarter? Start here: + +1. [Quick Start](../README.md#quick-start) - Get up and running +2. [Features](features/) - Discover what you can do +3. [Architecture](architecture/) - Understand the system + +## Documentation Structure + +### [Features](features/) +User guides for each feature: +- [Projects](features/projects.md) - Project management +- [Repositories](features/repositories.md) - Git repository management +- [Workspace](features/workspace.md) - Repository workspace +- [Git History](features/git-history.md) - History visualization +- [Authentication](features/auth.md) - Login and user management +- [Settings](features/settings.md) - User preferences +- [Tool Types](features/tool-types.md) - Development tool management +- [SSH Keys](features/ssh-keys.md) - SSH key management + +### [API Reference](api/) +Complete API documentation: +- [Auth](api/auth.md) - Authentication endpoints +- [Projects](api/projects.md) - Project endpoints +- [Repositories](api/repositories.md) - Repository endpoints +- [Users](api/users.md) - User endpoints +- [Tool Types](api/tool-types.md) - Tool type endpoints +- [SSH Keys](api/ssh-keys.md) - SSH key endpoints + +### [Architecture](architecture/) +System architecture documentation: +- [Backend](architecture/backend.md) - Backend architecture +- [Frontend](architecture/frontend.md) - Frontend architecture +- [Database](architecture/database.md) - Database schema +- [Deployment](architecture/deployment.md) - Deployment architecture + +### [Deployment](deployment/) +Guides for deploying Headquarter: +- [Docker](deployment/docker.md) - Docker setup +- [Traefik](deployment/traefik.md) - Traefik configuration +- [Authentik](deployment/authentik.md) - Authentik setup +- [Environment](deployment/environment.md) - Environment variables + +### [Development](development/) +Guides for developers: +- [Setup](development/setup.md) - Development environment +- [Testing](development/testing.md) - Testing strategy +- [Contributing](development/contributing.md) - How to contribute +- [Quality Gates](development/quality-gates.md) - Code quality + +## Contributing to Documentation + +When adding new features, please document them following our [templates](templates/): + +- [Feature Documentation Template](templates/feature-doc.md) +- [API Documentation Template](templates/api-endpoint.md) +- [Architecture Documentation Template](templates/architecture.md) + +See the [Contributing Guide](development/contributing.md) for more details. diff --git a/docs/api/README.md b/docs/api/README.md new file mode 100644 index 0000000..010785d --- /dev/null +++ b/docs/api/README.md @@ -0,0 +1,51 @@ +# API Documentation + +## Overview + +The Headquarter API is a RESTful API built with FastAPI. All endpoints (except auth) require authentication via session cookie. + +## Base URL + +- Development: `http://localhost:8000` +- Production: `https://api.yourdomain.com` + +## Authentication + +Most endpoints require authentication. Include the session cookie in requests: + +```bash +curl http://api.example.com/endpoint \ + -H "Cookie: session=your_session_cookie" +``` + +## Response Format + +All responses are JSON. Error responses follow this format: + +```json +{ + "detail": "Error message" +} +``` + +## API Sections + +- [Auth](auth.md) - Authentication endpoints +- [Projects](projects.md) - Project management +- [Repositories](repositories.md) - Git repositories and file operations +- [Users](users.md) - User management and settings +- [Tool Types](tool-types.md) - Tool type management +- [SSH Keys](ssh-keys.md) - SSH key management + +## Testing + +Interactive API documentation is available at `/docs` when running the backend: + +``` +http://localhost:8000/docs +``` + +This provides: +- Interactive endpoint testing +- Request/response schemas +- Authentication via "Authorize" button diff --git a/docs/api/auth.md b/docs/api/auth.md new file mode 100644 index 0000000..d6819e7 --- /dev/null +++ b/docs/api/auth.md @@ -0,0 +1,105 @@ +# Auth API + +Authentication endpoints for OAuth2 login via Authentik. + +## Authentication + +These endpoints handle the OAuth2 flow. No prior authentication is required for `/auth/login` and `/auth/callback`. + +--- + +## GET /auth/login + +**Description:** Initiate OAuth2 login flow. Redirects to Authentik. + +### Request + +#### Query Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `next` | `string` | No | URL to redirect to after login | + +### Response + +#### Success (307 Temporary Redirect) + +Redirects to Authentik OAuth2 authorization URL. + +--- + +## GET /auth/callback + +**Description:** Handle OAuth2 callback from Authentik. + +### Request + +#### Query Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `code` | `string` | Yes | Authorization code from Authentik | +| `state` | `string` | Yes | State parameter for CSRF protection | + +### Response + +#### Success (307 Temporary Redirect) + +Sets session cookie and redirects to frontend. + +#### Error (400 Bad Request) + +```json +{ + "detail": "Invalid state parameter" +} +``` + +--- + +## GET /auth/me + +**Description:** Get current authenticated user. + +**Auth:** Required (session cookie) + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "email": "user@example.com", + "name": "User Name", + "avatar_url": "https://..." +} +``` + +#### Error (401 Unauthorized) + +```json +{ + "detail": "Not authenticated" +} +``` + +--- + +## POST /auth/logout + +**Description:** Log out current user. + +**Auth:** Required (session cookie) + +### Response + +#### Success (200 OK) + +Clears session cookie. + +```json +{ + "message": "Logged out" +} +``` diff --git a/docs/api/projects.md b/docs/api/projects.md new file mode 100644 index 0000000..2477e08 --- /dev/null +++ b/docs/api/projects.md @@ -0,0 +1,163 @@ +# Projects API + +Project management endpoints. + +## Authentication + +All endpoints require authentication (session cookie). + +--- + +## GET /projects + +**Description:** List all projects for the current user. + +### Response + +#### Success (200 OK) + +```json +[ + { + "id": "uuid", + "name": "My Project", + "description": "Project description", + "created_at": "2024-01-01T00:00:00Z", + "updated_at": "2024-01-01T00:00:00Z" + } +] +``` + +--- + +## POST /projects + +**Description:** Create a new project. + +### Request + +#### Request Body + +```json +{ + "name": "My Project", + "description": "Optional description" +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `string` | Yes | Project name (max 255 chars) | +| `description` | `string` | No | Project description | + +### Response + +#### Success (201 Created) + +```json +{ + "id": "uuid", + "name": "My Project", + "description": "Optional description", + "created_at": "2024-01-01T00:00:00Z", + "updated_at": "2024-01-01T00:00:00Z" +} +``` + +#### Error (422 Validation Error) + +```json +{ + "detail": [ + { + "loc": ["body", "name"], + "msg": "field required", + "type": "value_error.missing" + } + ] +} +``` + +--- + +## GET /projects/{id} + +**Description:** Get project details. + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "name": "My Project", + "description": "Project description", + "created_at": "2024-01-01T00:00:00Z", + "updated_at": "2024-01-01T00:00:00Z" +} +``` + +#### Error (404 Not Found) + +```json +{ + "detail": "Project not found" +} +``` + +--- + +## PUT /projects/{id} + +**Description:** Update a project. + +### Request + +#### Request Body + +```json +{ + "name": "Updated Name", + "description": "Updated description" +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `string` | No | New project name | +| `description` | `string` | No | New description | + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "name": "Updated Name", + "description": "Updated description", + "created_at": "2024-01-01T00:00:00Z", + "updated_at": "2024-01-02T00:00:00Z" +} +``` + +--- + +## DELETE /projects/{id} + +**Description:** Delete a project and all associated repositories. + +### Response + +#### Success (204 No Content) + +#### Error (404 Not Found) + +```json +{ + "detail": "Project not found" +} +``` + +**Warning:** This also deletes all repositories and their data. Cannot be undone. diff --git a/docs/api/repositories.md b/docs/api/repositories.md new file mode 100644 index 0000000..c3d7925 --- /dev/null +++ b/docs/api/repositories.md @@ -0,0 +1,349 @@ +# Repositories API + +Git repository and file management endpoints. + +## Authentication + +All endpoints require authentication (session cookie). + +--- + +## GET /projects/{project_id}/repositories + +**Description:** List repositories in a project. + +### Response + +#### Success (200 OK) + +```json +[ + { + "id": "uuid", + "name": "my-repo", + "clone_url": "https://github.com/user/repo.git", + "is_mirror": true, + "project_id": "uuid", + "created_at": "2024-01-01T00:00:00Z" + } +] +``` + +--- + +## POST /projects/{project_id}/repositories + +**Description:** Create a new repository. + +### Request + +#### Request Body + +```json +{ + "name": "my-repo", + "remote_url": "https://github.com/user/repo.git", + "is_mirror": false +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `string` | Yes | Repository name | +| `remote_url` | `string` | No | Remote URL to clone from | +| `is_mirror` | `boolean` | No | Create mirror clone (default: false) | + +### Response + +#### Success (201 Created) + +```json +{ + "id": "uuid", + "name": "my-repo", + "clone_url": "https://github.com/user/repo.git", + "is_mirror": false, + "project_id": "uuid", + "created_at": "2024-01-01T00:00:00Z" +} +``` + +#### URL Parsing Suggestion (422 Unprocessable Entity) + +If the URL appears to be a browser URL: + +```json +{ + "detail": "URL appears to be a browser URL, not a git clone URL", + "suggested_url": "https://github.com/user/repo.git", + "original_url": "https://github.com/user/repo/tree/main", + "error_code": "URL_NEEDS_PARSING" +} +``` + +--- + +## DELETE /projects/{project_id}/repositories/{id} + +**Description:** Delete a repository. + +### Response + +#### Success (204 No Content) + +**Warning:** Permanently deletes the repository from disk. Cannot be undone. + +--- + +## POST /projects/{project_id}/repositories/parse-url + +**Description:** Parse and validate a git URL. + +### Request + +#### Request Body + +```json +{ + "url": "https://github.com/user/repo/tree/main" +} +``` + +### Response + +#### Success (200 OK) + +```json +{ + "original_url": "https://github.com/user/repo/tree/main", + "base_url": "https://github.com/user/repo.git", + "is_valid_clone_url": false, + "needs_parsing": true, + "host": "github.com", + "message": "This URL contains a branch path. The repository URL is: https://github.com/user/repo.git" +} +``` + +--- + +## GET /projects/{project_id}/repositories/{id}/files + +**Description:** List files in a directory. + +### Request + +#### Query Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `branch` | `string` | No | Branch name (default: repository default) | +| `path` | `string` | No | Directory path (default: root) | + +### Response + +#### Success (200 OK) + +```json +{ + "path": "src", + "branch": "main", + "entries": [ + { + "name": "components", + "type": "directory", + "path": "src/components" + }, + { + "name": "main.py", + "type": "file", + "path": "src/main.py", + "size": 1234, + "last_commit": { + "hash": "abc123", + "message": "Initial commit", + "author": "John Doe", + "date": "2024-01-01T00:00:00Z" + } + } + ] +} +``` + +--- + +## GET /projects/{project_id}/repositories/{id}/files/content + +**Description:** Get file content. + +### Request + +#### Query Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `branch` | `string` | Yes | Branch name | +| `path` | `string` | Yes | File path | + +### Response + +#### Success (200 OK) + +```json +{ + "path": "src/main.py", + "branch": "main", + "content": "function hello() {\n return 'world';\n}", + "size": 42, + "encoding": "utf-8", + "language": "python", + "is_binary": false +} +``` + +#### Binary File (200 OK) + +```json +{ + "path": "image.png", + "branch": "main", + "content": null, + "size": 12345, + "encoding": null, + "language": null, + "is_binary": true +} +``` + +--- + +## POST /projects/{project_id}/repositories/{id}/files/content + +**Description:** Update file content and commit. + +### Request + +#### Request Body + +```json +{ + "path": "src/main.py", + "branch": "main", + "content": "new content", + "commit_message": "Update file", + "author_name": "User", + "author_email": "user@example.com" +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | `string` | Yes | File path | +| `branch` | `string` | Yes | Branch name | +| `content` | `string` | Yes | New file content | +| `commit_message` | `string` | Yes | Commit message | +| `author_name` | `string` | Yes | Author name | +| `author_email` | `string` | Yes | Author email | + +### Response + +#### Success (200 OK) + +```json +{ + "commit_hash": "def789", + "message": "Update file", + "branch": "main" +} +``` + +--- + +## GET /projects/{project_id}/repositories/{id}/branches + +**Description:** List branches. + +### Response + +#### Success (200 OK) + +```json +{ + "branches": [ + { + "name": "main", + "is_default": true, + "last_commit": { + "hash": "abc123", + "message": "Initial commit", + "date": "2024-01-01T00:00:00Z" + } + } + ], + "default_branch": "main" +} +``` + +--- + +## GET /projects/{project_id}/repositories/{id}/history + +**Description:** Get commit history. + +### Request + +#### Query Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `branch` | `string` | No | Branch name (default: all branches) | +| `max_count` | `integer` | No | Maximum commits to return (default: 1000) | + +### Response + +#### Success (200 OK) + +```json +{ + "commits": [ + { + "hash": "abc123", + "author_name": "John Doe", + "author_email": "john@example.com", + "author_date": "2024-01-01T00:00:00Z", + "message": "Initial commit", + "refs": ["HEAD", "main"] + } + ], + "branch": "main", + "total_count": 50 +} +``` + +--- + +## GET /projects/{project_id}/repositories/{id}/commits/{hash} + +**Description:** Get commit details. + +### Response + +#### Success (200 OK) + +```json +{ + "hash": "abc123", + "author_name": "John Doe", + "author_email": "john@example.com", + "author_date": "2024-01-01T00:00:00Z", + "committer_name": "John Doe", + "committer_email": "john@example.com", + "commit_date": "2024-01-01T00:00:00Z", + "message": "Initial commit", + "stats": { + "files_changed": 2, + "insertions": 10, + "deletions": 0 + }, + "diff": "diff --git a/file.txt b/file.txt\n..." +} +``` diff --git a/docs/api/ssh-keys.md b/docs/api/ssh-keys.md new file mode 100644 index 0000000..1995e53 --- /dev/null +++ b/docs/api/ssh-keys.md @@ -0,0 +1,77 @@ +# SSH Keys API + +SSH key management endpoints. + +## Authentication + +All endpoints require authentication (session cookie). + +--- + +## GET /ssh-keys + +**Description:** List all SSH keys for the current user. + +### Response + +#### Success (200 OK) + +```json +[ + { + "id": "uuid", + "name": "GitHub Work", + "public_key": "ssh-ed25519 AAAAC3NzaC... user@example.com", + "created_at": "2024-01-01T00:00:00Z" + } +] +``` + +**Note:** Private keys are never returned. + +--- + +## POST /ssh-keys + +**Description:** Generate a new SSH key pair. + +### Request + +#### Request Body + +```json +{ + "name": "GitHub Personal" +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `string` | Yes | Key name/label | + +### Response + +#### Success (201 Created) + +```json +{ + "id": "uuid", + "name": "GitHub Personal", + "public_key": "ssh-ed25519 AAAAC3NzaC... user@example.com", + "created_at": "2024-01-01T00:00:00Z" +} +``` + +**Note:** The private key is generated and stored securely. It is not returned in the response. + +--- + +## DELETE /ssh-keys/{id} + +**Description:** Delete an SSH key. + +### Response + +#### Success (204 No Content) + +**Note:** This permanently deletes both public and private keys. diff --git a/docs/api/tool-types.md b/docs/api/tool-types.md new file mode 100644 index 0000000..4cf6e7b --- /dev/null +++ b/docs/api/tool-types.md @@ -0,0 +1,154 @@ +# Tool Types API + +Tool type management endpoints. + +## Authentication + +All endpoints require authentication (session cookie). + +--- + +## GET /tool-types + +**Description:** List all tool types. + +### Response + +#### Success (200 OK) + +```json +[ + { + "id": "uuid", + "name": "code-server", + "display_name": "VS Code Server", + "description": "VS Code in the browser", + "is_builtin": true, + "created_at": "2024-01-01T00:00:00Z" + } +] +``` + +--- + +## POST /tool-types + +**Description:** Create a new tool type. + +### Request + +#### Request Body + +```json +{ + "name": "my-tool", + "display_name": "My Custom Tool", + "description": "A custom development tool", + "compose_template": "version: \"3.8\"\nservices:\n tool:\n image: my-image:latest\n container_name: {{TOOL_NAME}}\n volumes:\n - {{REPO_PATH}}:/workspace", + "required_variables": ["TOOL_NAME", "REPO_PATH"] +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `string` | Yes | Unique identifier | +| `display_name` | `string` | Yes | Human-readable name | +| `description` | `string` | No | Description | +| `compose_template` | `string` | Yes | Docker Compose YAML | +| `required_variables` | `array` | Yes | Required template variables | + +### Response + +#### Success (201 Created) + +```json +{ + "id": "uuid", + "name": "my-tool", + "display_name": "My Custom Tool", + "description": "A custom development tool", + "compose_template": "...", + "required_variables": ["TOOL_NAME", "REPO_PATH"], + "is_builtin": false, + "created_at": "2024-01-01T00:00:00Z" +} +``` + +#### Error (400 Bad Request) + +```json +{ + "detail": "Invalid YAML in compose template" +} +``` + +--- + +## GET /tool-types/{id} + +**Description:** Get tool type details. + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "name": "code-server", + "display_name": "VS Code Server", + "description": "VS Code in the browser", + "compose_template": "...", + "required_variables": ["TOOL_NAME", "REPO_PATH"], + "is_builtin": true, + "created_at": "2024-01-01T00:00:00Z" +} +``` + +--- + +## PUT /tool-types/{id} + +**Description:** Update a tool type. + +**Note:** Built-in tool types cannot be modified. + +### Request + +#### Request Body + +Same as POST /tool-types. + +### Response + +#### Success (200 OK) + +Returns updated tool type. + +#### Error (403 Forbidden) + +```json +{ + "detail": "Cannot modify built-in tool types" +} +``` + +--- + +## DELETE /tool-types/{id} + +**Description:** Delete a tool type. + +**Note:** Built-in tool types cannot be deleted. + +### Response + +#### Success (204 No Content) + +#### Error (403 Forbidden) + +```json +{ + "detail": "Cannot delete built-in tool types" +} +``` diff --git a/docs/api/users.md b/docs/api/users.md new file mode 100644 index 0000000..9ed2bb3 --- /dev/null +++ b/docs/api/users.md @@ -0,0 +1,168 @@ +# Users API + +User profile and settings management. + +## Authentication + +All endpoints require authentication (session cookie). + +--- + +## GET /users/me + +**Description:** Get current user profile. + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "email": "user@example.com", + "name": "User Name", + "avatar_url": "https://...", + "authentik_id": "authentik-uuid", + "created_at": "2024-01-01T00:00:00Z", + "updated_at": "2024-01-01T00:00:00Z" +} +``` + +--- + +## PUT /users/me + +**Description:** Update user profile. + +### Request + +#### Request Body + +```json +{ + "name": "New Name", + "email": "newemail@example.com" +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `string` | No | New display name | +| `email` | `string` | No | New email address | + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "email": "newemail@example.com", + "name": "New Name", + "avatar_url": "https://...", + "updated_at": "2024-01-02T00:00:00Z" +} +``` + +--- + +## POST /users/me/avatar + +**Description:** Upload avatar image. + +### Request + +#### Request Body + +Multipart form data with `file` field. + +**Requirements:** +- Format: PNG or JPEG +- Max size: 2MB + +### Response + +#### Success (200 OK) + +```json +{ + "avatar_url": "/uploads/avatars/uuid.png" +} +``` + +#### Error (400 Bad Request) + +```json +{ + "detail": "Invalid file format. Only PNG and JPEG are allowed." +} +``` + +--- + +## GET /users/me/config + +**Description:** Get user settings. + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "user_id": "uuid", + "config": { + "theme": "dark", + "git_identity": { + "name": "User Name", + "email": "user@example.com" + }, + "default_editor": "code-server" + } +} +``` + +--- + +## PATCH /users/me/config + +**Description:** Update user settings. + +### Request + +#### Request Body + +```json +{ + "theme": "light", + "git_identity": { + "name": "New Name", + "email": "new@example.com" + } +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `theme` | `string` | No | "system", "light", or "dark" | +| `git_identity` | `object` | No | `{name, email}` | +| `default_editor` | `string` | No | Preferred editor | + +### Response + +#### Success (200 OK) + +```json +{ + "id": "uuid", + "user_id": "uuid", + "config": { + "theme": "light", + "git_identity": { + "name": "New Name", + "email": "new@example.com" + } + } +} +``` diff --git a/docs/architecture/backend.md b/docs/architecture/backend.md new file mode 100644 index 0000000..db82501 --- /dev/null +++ b/docs/architecture/backend.md @@ -0,0 +1,212 @@ +# Backend Architecture + +## Overview + +The Headquarter backend is built with **FastAPI** and follows a layered architecture pattern. + +## Architecture Diagram + +``` +┌─────────────────────────────────────────────────────────────┐ +│ FastAPI App │ +├─────────────────────────────────────────────────────────────┤ +│ Middleware: CORS → Request Logging → Exception Logging │ +├─────────────────────────────────────────────────────────────┤ +│ API Layer (src/api/) │ +│ ┌─────────┐ ┌──────────┐ ┌────────┐ ┌──────────┐ │ +│ │ Auth │ │ Projects │ │ Users │ │ Git │ │ +│ │ Routes │ │ Routes │ │ Routes │ │ Repos │ │ +│ └────┬────┘ └────┬─────┘ └───┬────┘ └────┬─────┘ │ +├───────┼───────────┼───────────┼───────────┼─────────────────┤ +│ │ │ │ │ │ +│ Auth │ Project │ User │ Git │ │ +│ Layer │ Service │ Service │ Service │ │ +│ │ │ │ │ │ +├───────┴───────────┴───────────┴───────────┴─────────────────┤ +│ Data Layer │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ +│ │ Models │ │ Database │ │ Config │ │ +│ │(SQLAlch) │ │(AsyncPG) │ │(Pydantic)│ │ +│ └──────────┘ └──────────┘ └──────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Directory Structure + +``` +src/ +├── api/ # API Routes +│ ├── auth.py # Authentication endpoints +│ ├── projects.py # Project endpoints +│ ├── git_repositories.py # Repository endpoints +│ ├── users.py # User endpoints +│ ├── tool_types.py # Tool type endpoints +│ ├── ssh_keys.py # SSH key endpoints +│ └── dashboard.py # Dashboard endpoints +├── auth/ # Authentication +│ ├── session.py # Session management +│ ├── oidc.py # OAuth2 client +│ ├── dependencies.py # Auth dependencies +│ └── cookies.py # Cookie utilities +├── models/ # Database Models +│ ├── user.py # User model +│ ├── project.py # Project model +│ ├── git_repository.py # Repository model +│ ├── tool_type.py # Tool type model +│ ├── ssh_key.py # SSH key model +│ └── user_config.py # User config model +├── utils/ # Utilities +│ ├── git_url_parser.py # URL parsing +│ ├── git_files.py # Git file operations +│ └── git_history.py # Git history operations +├── config.py # Configuration +├── database.py # Database setup +└── main.py # Application entry point +``` + +## Layers + +### 1. API Layer (`src/api/`) + +**Responsibilities:** +- Define HTTP endpoints +- Parse request parameters +- Return HTTP responses +- Use dependencies for auth and DB + +**Pattern:** +```python +@router.get("/projects") +async def list_projects( + session: AsyncSession = Depends(get_db_session), + user_id: str = Depends(get_current_user_id), +): + # Call service layer + projects = await project_service.list(session, user_id) + return projects +``` + +### 2. Auth Layer (`src/auth/`) + +**Responsibilities:** +- Session management (create, verify, expire) +- OAuth2 flow (login, callback) +- User authentication dependencies + +**Key Components:** +- `session.py`: HMAC-SHA256 signed cookies +- `oidc.py`: OAuth2 token exchange +- `dependencies.py`: FastAPI dependencies for auth + +### 3. Data Layer (`src/models/`, `src/database.py`) + +**Responsibilities:** +- Database schema definition +- Async database sessions +- Connection management + +**Technology:** +- SQLAlchemy 2.0 with async support +- asyncpg driver for PostgreSQL +- Alembic for migrations + +### 4. Utility Layer (`src/utils/`) + +**Responsibilities:** +- Git operations (file browsing, history) +- URL parsing +- Helper functions + +## Data Flow + +### Request Lifecycle + +``` +1. Request arrives at FastAPI +2. Middleware processes (CORS, logging) +3. Auth dependency verifies session +4. Route handler processes request +5. Database session executes queries +6. Response returned to client +``` + +### Authentication Flow + +``` +1. User clicks login +2. Backend redirects to Authentik +3. User authenticates +4. Authentik redirects with code +5. Backend exchanges code for token +6. Backend fetches user info +7. Backend creates session cookie +8. User is authenticated +``` + +## Dependencies + +### Database Session + +```python +async def get_db_session() -> AsyncGenerator[AsyncSession, None]: + async with SessionLocal() as session: + yield session +``` + +### Current User + +```python +async def get_current_user_id( + request: Request, + settings: Settings = Depends(get_settings), +) -> str: + # Verify session cookie + # Return user_id +``` + +## Configuration + +Configuration is managed via Pydantic Settings: + +```python +class Settings(BaseSettings): + app_env: str = "development" + database_url: str = "..." + authentik_domain: str = "..." + # ... +``` + +Environment variables are automatically loaded from `.env` files. + +## Error Handling + +Errors are handled at multiple levels: + +1. **Validation**: Pydantic validates request bodies +2. **HTTP Exceptions**: FastAPI HTTPException for client errors +3. **Middleware**: ExceptionLoggingMiddleware logs server errors +4. **Database**: SQLAlchemy errors converted to HTTP responses + +## Testing + +- **Unit tests**: SQLite in-memory database +- **Integration tests**: PostgreSQL with transaction rollback +- **Fixtures**: Shared in `conftest.py` + +## Technology Stack + +| Component | Technology | Version | +|-----------|-----------|---------| +| Web Framework | FastAPI | ^0.104 | +| ORM | SQLAlchemy | ^2.0 | +| Database Driver | asyncpg | ^0.29 | +| Validation | Pydantic | ^2.0 | +| Migrations | Alembic | ^1.12 | +| HTTP Client | httpx | ^0.25 | +| Testing | pytest | ^7.4 | + +## Related Documentation + +- [Database Schema](database.md) +- [Frontend Architecture](../frontend.md) +- [API Documentation](../api/) diff --git a/docs/architecture/database.md b/docs/architecture/database.md new file mode 100644 index 0000000..3e991dc --- /dev/null +++ b/docs/architecture/database.md @@ -0,0 +1,231 @@ +# Database Schema + +## Overview + +Headquarter uses PostgreSQL with SQLAlchemy ORM and Alembic for migrations. All tables use UUID primary keys and include `created_at`/`updated_at` timestamps. + +## Entity Relationship Diagram + +``` +┌──────────────┐ ┌─────────────────┐ ┌──────────────┐ +│ users │ │ git_repository │ │ project │ +├──────────────┤ ├─────────────────┤ ├──────────────┤ +│ id (PK) │ │ id (PK) │ │ id (PK) │ +│ email │ │ project_id (FK) │──┐ │ name │ +│ name │ │ name │ │ │ description │ +│ authentik_id │ │ remote_url │ │ │ created_by_id│──┐ +│ avatar_url │ │ local_path │ │ │ │ │ +│ created_at │ │ is_mirror │ │ │ │ │ +│ updated_at │ │ created_by_id │──┤ │ │ │ +└──────────────┘ │ created_at │ │ └──────────────┘ │ + │ │ updated_at │ │ ▲ │ + │ └─────────────────┘ │ │ │ + │ │ │ │ │ + │ ┌───────┘ │ │ │ + │ │ │ │ │ +┌───────▼──────┐ ┌▼────────────────┐ │ │ │ +│ ssh_keys │ │ user_config │ │ │ │ +├──────────────┤ ├─────────────────┤ │ │ │ +│ id (PK) │ │ user_id (FK) │───┘ │ │ +│ user_id (FK) │ │ theme │ │ │ +│ name │ │ git_name │ │ │ +│ public_key │ │ git_email │ │ │ +│ private_key │ │ default_editor │ │ │ +│ created_at │ │ created_at │ │ │ +│ updated_at │ │ updated_at │ │ │ +└──────────────┘ └─────────────────┘ │ │ + │ │ + ┌───────────────────────┘ │ + │ │ + ▼ │ + ┌─────────────────┐ │ + │ tool_types │ │ + ├─────────────────┤ │ + │ id (PK) │ │ + │ name │ │ + │ display_name │ │ + │ description │ │ + │ compose_template│ │ + │ required_vars │ │ + │ is_builtin │ │ + │ created_by_id │─────────────────────────┘ + │ created_at │ + │ updated_at │ + └─────────────────┘ +``` + +## Table Definitions + +### users + +Stores user accounts synchronized from Authentik. + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| id | UUID | PK | Unique identifier | +| email | VARCHAR(255) | NOT NULL, UNIQUE | User email | +| name | VARCHAR(255) | | Display name | +| authentik_id | VARCHAR(255) | UNIQUE | Authentik user ID | +| avatar_url | VARCHAR(500) | | Profile avatar URL | +| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp | +| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp | + +**Relationships**: +- One-to-Many: `users` → `git_repository` (created_by_id) +- One-to-Many: `users` → `project` (created_by_id) +- One-to-Many: `users` → `ssh_keys` (user_id) +- One-to-One: `users` → `user_config` (user_id) +- One-to-Many: `users` → `tool_types` (created_by_id) + +### project + +Organizes repositories into logical groups. + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| id | UUID | PK | Unique identifier | +| name | VARCHAR(255) | NOT NULL | Project name | +| description | TEXT | | Project description | +| created_by_id | UUID | FK → users.id | Creator | +| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp | +| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp | + +**Relationships**: +- One-to-Many: `project` → `git_repository` (project_id) + +### git_repository + +Git repositories (bare/mirror clones). + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| id | UUID | PK | Unique identifier | +| project_id | UUID | FK → project.id, NOT NULL | Parent project | +| name | VARCHAR(255) | NOT NULL | Repository name | +| remote_url | VARCHAR(500) | NOT NULL | Remote git URL | +| local_path | VARCHAR(500) | | Local filesystem path | +| is_mirror | BOOLEAN | DEFAULT false | Is mirror clone | +| created_by_id | UUID | FK → users.id | Creator | +| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp | +| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp | + +**Indexes**: +- `idx_repo_project`: (project_id) +- `idx_repo_name`: (project_id, name) - UNIQUE + +### ssh_keys + +User SSH keys for git authentication. + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| id | UUID | PK | Unique identifier | +| user_id | UUID | FK → users.id, NOT NULL | Owner | +| name | VARCHAR(255) | NOT NULL | Key name | +| public_key | TEXT | NOT NULL | Public key | +| private_key | TEXT | NOT NULL, ENCRYPTED | Encrypted private key | +| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp | +| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp | + +**Indexes**: +- `idx_ssh_user`: (user_id) + +### user_config + +User preferences and settings. + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| id | UUID | PK | Unique identifier | +| user_id | UUID | FK → users.id, NOT NULL, UNIQUE | Owner | +| theme | VARCHAR(50) | DEFAULT 'system' | UI theme (system/light/dark) | +| git_name | VARCHAR(255) | | Git user name | +| git_email | VARCHAR(255) | | Git user email | +| default_editor | VARCHAR(50) | DEFAULT 'vscode' | Preferred editor | +| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp | +| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp | + +### tool_types + +Types of development tools that can be spawned. + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| id | UUID | PK | Unique identifier | +| name | VARCHAR(255) | NOT NULL, UNIQUE | Machine name | +| display_name | VARCHAR(255) | NOT NULL | Human-readable name | +| description | TEXT | | Description | +| compose_template | TEXT | NOT NULL | Docker Compose template | +| required_variables | JSONB | NOT NULL | Template variables | +| is_builtin | BOOLEAN | DEFAULT false | Built-in type | +| created_by_id | UUID | FK → users.id | Creator (null for built-in) | +| created_at | TIMESTAMP | DEFAULT now() | Creation timestamp | +| updated_at | TIMESTAMP | DEFAULT now() | Last update timestamp | + +**Indexes**: +- `idx_tool_builtin`: (is_builtin) + +## Migration History + +| Version | Date | Description | +|---------|------|-------------| +| 0001_initial_schema | 2024-01-XX | Initial tables: users, projects, git_repositories | +| 0002_refresh_tokens | 2024-01-XX | Added refresh_tokens table | +| 0003_user_configs | 2024-05-18 | Added user_config table | +| 0004_tool_types | 2024-05-18 | Added tool_types table | + +## Data Types + +### PostgreSQL Types +- **UUID**: `uuid` - All primary keys +- **Timestamps**: `TIMESTAMP WITH TIME ZONE` +- **JSONB**: `JSONB` - For flexible config (user_config, tool_types) +- **Strings**: `VARCHAR(n)` - With appropriate length limits +- **Text**: `TEXT` - For unbounded content +- **Boolean**: `BOOLEAN` - True/false flags + +### SQLAlchemy Configuration +```python +# Base model features +class Base: + id: UUID = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) + created_at: datetime = Column(DateTime(timezone=True), server_default=func.now()) + updated_at: datetime = Column(DateTime(timezone=True), onupdate=func.now()) +``` + +## Backup Strategy + +### Automated Backups +- **Frequency**: Daily at 2 AM +- **Retention**: 7 daily, 4 weekly, 12 monthly +- **Method**: `pg_dump` to S3/object storage +- **Encryption**: AES-256 encrypted backups + +### Manual Backup +```bash +# Full backup +pg_dump -Fc -f headquarter_backup.dump postgresql://user:pass@host/db + +# Restore +pg_restore -d postgresql://user:pass@host/db headquarter_backup.dump +``` + +## Performance + +### Query Optimization +- All foreign keys indexed +- Frequently queried columns indexed +- Composite indexes for multi-column queries + +### Connection Pooling +- SQLAlchemy async pool: 5-20 connections +- PgBouncer for production: transaction mode + +## Future Schema Changes + +Planned additions: +- [ ] **teams** table - Group users into teams +- [ ] **team_memberships** table - Link users to teams +- [ ] **tool_instances** table - Running tool containers +- [ ] **audit_logs** table - Track important actions +- [ ] **notifications** table - User notifications diff --git a/docs/architecture/deployment.md b/docs/architecture/deployment.md new file mode 100644 index 0000000..6ca9ba2 --- /dev/null +++ b/docs/architecture/deployment.md @@ -0,0 +1,357 @@ +# Deployment Architecture + +## Overview + +Headquarter is designed for containerized deployment using Docker, with support for both development and production environments. + +## System Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Internet │ +└──────────────────────┬──────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Traefik (Reverse Proxy) │ +│ SSL/TLS termination, routing │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ app.domain │ │ api.domain │ │ auth.domain │ │ +│ │ (Frontend) │ │ (Backend) │ │ (Authentik) │ │ +│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ +└─────────┼─────────────────┼─────────────────┼──────────────┘ + │ │ │ + ▼ ▼ ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Docker Host / Server │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ Frontend │ │ API │ │ Authentik │ │ +│ │ (nginx) │ │ (FastAPI) │ │ (OAuth2) │ │ +│ │ Port 80 │ │ Port 8000 │ │ Port 9443 │ │ +│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ +│ │ │ │ │ +│ └─────────────────┼─────────────────┘ │ +│ │ │ +│ ┌────────┴────────┐ │ +│ │ │ │ +│ ┌───────▼───────┐ ┌──────▼───────┐ │ +│ │ PostgreSQL │ │ Redis │ │ +│ │ Port 5432 │ │ Port 6379 │ │ +│ └───────────────┘ └──────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────┐ │ +│ │ Volumes │ │ +│ │ postgres_data │ repo_data │ avatar_uploads │ │ +│ └──────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Components + +### Frontend (Web) + +- **Technology**: React + Vite + nginx +- **Port**: 80 (internal) +- **Role**: User interface +- **Scaling**: Static files, easily scaled horizontally + +### Backend (API) + +- **Technology**: FastAPI + Python 3.11 +- **Port**: 8000 (internal) +- **Role**: Business logic, API endpoints +- **Scaling**: Stateless, can scale horizontally + +### Database (PostgreSQL) + +- **Technology**: PostgreSQL 15 +- **Port**: 5432 (internal) +- **Role**: Persistent data storage +- **Scaling**: Vertical or read replicas + +### Cache (Redis) + +- **Technology**: Redis 7 +- **Port**: 6379 (internal) +- **Role**: Session storage, caching +- **Scaling**: Redis Cluster for high availability + +### Identity Provider (Authentik) + +- **Technology**: Authentik (self-hosted) +- **Port**: 9443 (external), 9000 (internal) +- **Role**: OAuth2/OIDC authentication +- **Note**: Can be external service + +## Network Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Networks │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ External Network (traefik) │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ Traefik │ │ Frontend │ │ API │ │ +│ │ (proxy) │ │ (web) │ │ (api) │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +│ │ +│ Internal Network (backend) │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ API │ │ PostgreSQL │ │ Redis │ │ +│ │ (api) │ │ (postgres) │ │ (redis) │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +**Network Security**: +- External network: Exposes services to Traefik +- Internal network: Database and cache only accessible by API +- No direct database access from external network + +## Data Flow + +### Authentication Flow + +``` +User → Frontend → API → Authentik + ↓ + OAuth2 + ↓ +User ← Frontend ← API ← Authentik + ↓ + Session Cookie +``` + +### API Request Flow + +``` +User → Frontend → Traefik → API → Database + ↓ + Redis (cache) +``` + +### Git Operations Flow + +``` +User → Frontend → API → Git Repository (filesystem) + ↓ + Git History/Files +``` + +## Deployment Patterns + +### Single Server + +All services on one host: +- Simple to manage +- Suitable for small teams +- Single point of failure + +### Multi-Server (HA) + +Separate services across hosts: +- Database server +- Application servers (API + Frontend) +- Load balancer (Traefik) +- Higher availability + +### Kubernetes (Future) + +Container orchestration: +- Auto-scaling +- Self-healing +- Rolling updates +- Resource management + +## Scaling Strategy + +### Horizontal Scaling + +**Stateless Services** (easy to scale): +- Frontend: Multiple nginx instances +- API: Multiple FastAPI instances + +**Stateful Services** (require care): +- Database: Read replicas, connection pooling +- Redis: Cluster mode + +### Vertical Scaling + +Increase resources for: +- Database server (CPU, RAM, I/O) +- API server (CPU for git operations) + +## Backup Strategy + +### Automated Backups + +``` +Daily at 2 AM +├── PostgreSQL dump +├── Repository filesystem +├── User uploads (avatars) +└── Configuration files +``` + +### Backup Retention + +- Daily: 7 days +- Weekly: 4 weeks +- Monthly: 12 months +- Yearly: 3 years + +### Disaster Recovery + +1. Restore database from backup +2. Restore repositories from backup +3. Verify application functionality +4. Update DNS if needed + +## Security Considerations + +### Network Security + +- Internal services not exposed externally +- Database only accessible from API +- Redis only accessible from API +- SSL/TLS for all external traffic + +### Data Security + +- Encrypted database connections +- Encrypted backups +- SSH keys encrypted at rest +- Session cookies httpOnly + Secure + +### Access Control + +- OAuth2 authentication +- Role-based access (future) +- API rate limiting +- Audit logging (future) + +## Monitoring + +### Health Checks + +``` +API: GET /health +Database: pg_isready +Redis: redis-cli ping +``` + +### Metrics + +- Request rate and latency +- Error rate +- Database connections +- Disk usage +- Memory usage + +### Logging + +- Application logs (structured JSON) +- Access logs (Traefik) +- Error logs (centralized) +- Audit logs (future) + +## Performance Optimization + +### Database + +- Connection pooling (PgBouncer) +- Query optimization +- Proper indexing +- Regular VACUUM + +### API + +- Async operations +- Caching (Redis) +- Git operation optimization +- File streaming + +### Frontend + +- Code splitting +- Lazy loading +- Asset optimization +- CDN (future) + +## Troubleshooting + +### Common Issues + +**High Memory Usage**: +```bash +# Check container stats +docker stats + +# Restart API if needed +docker compose restart api +``` + +**Database Connection Issues**: +```bash +# Check PostgreSQL logs +docker compose logs postgres + +# Verify connection +docker compose exec api pg_isready -h postgres +``` + +**Git Operations Slow**: +```bash +# Check disk I/O +iostat -x 1 + +# Check repository size +du -sh /data/repos/* +``` + +## Migration Strategy + +### Version Updates + +1. Backup data +2. Update images +3. Run migrations +4. Verify functionality +5. Rollback if needed + +### Database Migrations + +```bash +# Check current version +alembic current + +# Upgrade +alembic upgrade head + +# Downgrade if needed +alembic downgrade -1 +``` + +## Future Architecture + +### Planned Improvements + +- [ ] Kubernetes deployment +- [ ] Microservices split +- [ ] Event-driven architecture +- [ ] Real-time WebSocket updates +- [ ] Multi-region deployment +- [ ] CDN integration +- [ ] Advanced monitoring (Prometheus/Grafana) + +### Scalability Roadmap + +1. **Phase 1**: Single server (current) +2. **Phase 2**: Separate database server +3. **Phase 3**: Load balanced API servers +4. **Phase 4**: Kubernetes cluster +5. **Phase 5**: Multi-region diff --git a/docs/architecture/frontend.md b/docs/architecture/frontend.md new file mode 100644 index 0000000..fab9dd9 --- /dev/null +++ b/docs/architecture/frontend.md @@ -0,0 +1,297 @@ +# Frontend Architecture + +## Overview + +The Headquarter frontend is a React-based single-page application (SPA) built with modern tooling and designed for modularity and maintainability. + +## Tech Stack + +| Layer | Technology | Version | +|-------|-----------|---------| +| Framework | React | ^18.2.0 | +| Router | React Router | ^6.20.0 | +| Bundler | Vite | ^5.0.0 | +| Language | TypeScript | ^5.3.0 | +| Styling | CSS3 with CSS Variables | - | +| Testing | Vitest + React Testing Library | ^4.1.6 | + +## Directory Structure + +``` +apps/web/src/ +├── api/ # API clients +│ ├── auth.ts # Authentication API +│ ├── projects.ts # Project API +│ ├── git_repositories.ts # Repository API +│ ├── ssh_keys.ts # SSH key API +│ ├── tool_types.ts # Tool type API +│ ├── users.ts # User API +│ └── settings.ts # Settings API +├── components/ # Reusable components +│ ├── app-shell.tsx # Main app layout +│ ├── protected-route.tsx # Auth guard +│ └── [more...] +├── context/ # React contexts +│ └── auth.tsx # Auth state management +├── hooks/ # Custom hooks +│ ├── use-auth.ts # Auth hook +│ └── use-theme.ts # Theme hook +├── pages/ # Page components (routes) +│ ├── dashboard.tsx # Dashboard +│ ├── projects.tsx # Project list +│ ├── repo-workspace.tsx # Repository workspace +│ ├── git-history.tsx # Git history +│ ├── git-repositories.tsx # Repository management +│ ├── profile.tsx # User profile +│ ├── settings.tsx # User settings +│ ├── tool-types.tsx # Tool types +│ ├── ssh-keys.tsx # SSH keys +│ └── [more...] +├── styles/ # Global styles +│ ├── index.css # Main stylesheet +│ └── [more...] +├── types.ts # Shared TypeScript types +├── router.tsx # Route definitions +└── main.tsx # Entry point +``` + +## Architecture Patterns + +### 1. Component Architecture + +**Page Components**: Top-level components mapped to routes +- Own data fetching +- Manage page-level state +- Compose reusable components + +**Reusable Components**: Shared UI elements +- No data fetching +- Receive data via props +- Emit events via callbacks + +**Example**: +```typescript +// Page component +const RepoWorkspace = () => { + const [files, setFiles] = useState([]); + // ... fetch data, manage state + return ( +
+ + +
+ ); +}; + +// Reusable component +const FileTree = ({ files, onFileClick }: FileTreeProps) => { + return ( + + ); +}; +``` + +### 2. State Management + +**URL State**: Shareable, bookmarkable state +```typescript +// Sync selections to URL +const [searchParams, setSearchParams] = useSearchParams(); +// ?repo=123&branch=main&path=src/main.py +``` + +**React Context**: Global auth state +```typescript +// Auth context provides user, login, logout +const { user, isAuthenticated } = useAuth(); +``` + +**Local State**: Component-specific state +```typescript +const [isEditing, setIsEditing] = useState(false); +``` + +### 3. API Client Pattern + +Centralized API clients with type safety: + +```typescript +// api/git_repositories.ts +export const getRepositories = async (projectId: string) => { + const response = await fetch(`${API_BASE_URL}/projects/${projectId}/repositories`, { + credentials: 'include', + }); + return response.json(); +}; + +// Usage in component +const repos = await getRepositories(projectId); +``` + +### 4. Authentication Flow + +``` +User clicks Login + → Redirect to /auth/login (backend) + → Backend redirects to Authentik OAuth + → User authenticates with Authentik + → Authentik redirects to /auth/callback + → Backend creates session cookie + → Backend redirects to frontend + → Frontend checks /auth/me + → User is authenticated! +``` + +**Auth State**: +```typescript +interface AuthState { + user: User | null; + isAuthenticated: boolean; + isLoading: boolean; +} +``` + +### 5. Routing Structure + +```typescript +// router.tsx +} /> +} /> +} /> +} /> +} /> +} /> +} /> +} /> +} /> +``` + +## Data Flow + +### Repository Workspace Example + +``` +1. User clicks project + → Navigate to /projects/:id + +2. RepoWorkspace mounts + → Fetch project repositories + → Select first repo (or from URL) + +3. Repo selected + → Fetch branches + → Fetch file tree (default branch) + +4. User clicks file + → Fetch file content + → Display in viewer + → Update URL: ?path=src/main.py + +5. User switches branch + → Fetch file tree for branch + → Re-fetch current file if viewing + → Update URL: ?branch=develop +``` + +## Component Communication + +``` +┌─────────────────────────────────────┐ +│ RepoWorkspace │ +│ ┌──────────┐ ┌──────────────┐ │ +│ │ FileTree │───▶│ FileViewer │ │ +│ │ │ │ │ │ +│ │ onFileClick │ content │ │ +│ │ │ │ onEdit │ │ +│ └──────────┘ └──────────────┘ │ +│ ▲ │ +│ │ │ +│ ┌──────────┐ │ +│ │ Branch │───▶ fetch tree │ +│ │ Selector │ │ +│ └──────────┘ │ +└─────────────────────────────────────┘ +``` + +## Styling Strategy + +### CSS Variables (Design Tokens) +```css +:root { + --color-primary: #007bff; + --color-bg: #ffffff; + --color-text: #333333; + --sidebar-width: 250px; + --border-radius: 4px; +} + +[data-theme="dark"] { + --color-bg: #1a1a1a; + --color-text: #e0e0e0; +} +``` + +### Component Styles +- Each page/component has scoped CSS +- Global utilities in `styles/index.css` +- No CSS-in-JS library (keep it simple) + +## Testing Strategy + +### Unit Tests (Vitest) +```typescript +// Component test +import { render, screen } from '@testing-library/react'; +import { FileTree } from './file-tree'; + +test('renders file list', () => { + const files = [{ name: 'test.py', type: 'file' }]; + render( {}} />); + expect(screen.getByText('test.py')).toBeInTheDocument(); +}); +``` + +### Test Coverage +- Component rendering +- User interactions +- Auth state changes +- API mocking + +## Performance Considerations + +1. **Code Splitting**: Vite handles automatic chunking +2. **Lazy Loading**: React.lazy() for heavy pages +3. **Debouncing**: URL updates debounced (300ms) +4. **Caching**: Browser caches API responses (ETags) +5. **Optimistic UI**: Immediate feedback before API response + +## Future Improvements + +- [ ] Add React Query for server state management +- [ ] Implement virtual scrolling for large file trees +- [ ] Add service worker for offline support +- [ ] Implement real-time updates (WebSocket) +- [ ] Add error boundary components + +## Development Workflow + +```bash +# Start dev server +cd apps/web && npm run dev + +# Run tests +npm run test + +# Type check +npm run typecheck + +# Lint +npm run lint + +# Build for production +npm run build +``` diff --git a/docs/deployment/authentik.md b/docs/deployment/authentik.md new file mode 100644 index 0000000..659d3bc --- /dev/null +++ b/docs/deployment/authentik.md @@ -0,0 +1,201 @@ +# Authentik OAuth Configuration + +## Overview + +Headquarter uses Authentik as its OAuth2 provider for authentication. This guide covers setting up Authentik and configuring Headquarter to work with it. + +## Prerequisites + +- Running Authentik instance +- Admin access to Authentik +- Headquarter deployed and accessible + +## Authentik Setup + +### Step 1: Create Application + +1. Log in to Authentik Admin interface +2. Navigate to **Applications** → **Applications** +3. Click **Create** +4. Fill in: + - **Name**: Headquarter + - **Slug**: `headquarter-web` (or your preferred slug) + - **Provider**: Create new + +### Step 2: Create OAuth Provider + +1. In the provider creation form: + - **Name**: Headquarter OAuth + - **Authentication flow**: `default-authentication-flow` + - **Authorization flow**: `default-provider-authorization-explicit-consent` + - **Client type**: Confidential + - **Client ID**: Generate or use your own UUID + - **Client Secret**: Generate strong secret + - **Redirect URIs**: `https://api.yourdomain.com/auth/callback` + +2. **Advanced protocol settings**: + - **Signing Key**: Select a signing key (required) + - **Access Token validity**: Minutes (default: 5) + - **Refresh Token validity**: Days (default: 30) + +3. Save provider + +### Step 3: Configure Application + +1. Return to Application configuration +2. Select the created provider +3. Save application + +### Step 4: Verify URLs + +Note these URLs from your Authentik instance: +- **Authorize URL**: `https://auth.yourdomain.com/application/o/authorize/` +- **Token URL**: `https://auth.yourdomain.com/application/o/token/` +- **UserInfo URL**: `https://auth.yourdomain.com/application/o/userinfo/` +- **JWKS URL**: `https://auth.yourdomain.com/application/o/headquarter-web/jwks/` + +## Headquarter Configuration + +### Environment Variables + +Add to your `.env` file: + +```bash +# Authentik Configuration +AUTHENTIK_DOMAIN=auth.yourdomain.com +AUTHENTIK_CLIENT_ID=your-client-id-uuid +AUTHENTIK_CLIENT_SECRET=your-generated-secret +AUTHENTIK_APPLICATION_SLUG=headquarter-web +AUTHENTIK_AUDIENCE=your-client-id-uuid + +# Optional: Override default URLs if needed +# AUTHENTIK_AUTHORIZE_URL=https://auth.yourdomain.com/application/o/authorize/ +# AUTHENTIK_TOKEN_URL=https://auth.yourdomain.com/application/o/token/ +# AUTHENTIK_JWKS_URL=https://auth.yourdomain.com/application/o/headquarter-web/jwks/ +# AUTHENTIK_ISSUER=https://auth.yourdomain.com/application/o/headquarter-web/ +``` + +### Important Notes + +- **AUTHENTIK_CLIENT_ID**: The UUID from Authentik (used for OAuth) +- **AUTHENTIK_APPLICATION_SLUG**: The URL-friendly name (e.g., `headquarter-web`) +- **AUTHENTIK_AUDIENCE**: Usually same as Client ID + +## User Synchronization + +On first login, Headquarter creates a local user record: + +```python +user = User( + email="user@example.com", + name="User Name", + authentik_id="authentik-user-id", +) +``` + +### Synced Fields + +| Authentik Field | Headquarter Field | +|----------------|-------------------| +| email | email | +| name | name | +| sub (user ID) | authentik_id | +| groups | (future: team membership) | + +## Troubleshooting + +### Redirect URI Error + +**Problem**: "Redirect URI Error" from Authentik + +**Solution**: +1. Check redirect URI in Authentik matches exactly +2. Must include protocol: `https://api.yourdomain.com/auth/callback` +3. No trailing slash difference + +### Invalid Client + +**Problem**: "invalid_client" error + +**Solution**: +1. Verify Client ID matches +2. Verify Client Secret is correct +3. Check application slug in URLs + +### Missing Refresh Token + +**Problem**: Authentik doesn't return refresh token + +**Solution**: +This is normal. Headquarter creates its own session cookies and doesn't need refresh tokens from Authentik. + +### CORS Errors + +**Problem**: CORS errors in browser + +**Solution**: +1. Ensure API domain is in CORS origins +2. Check `WEB_BASE_URL` environment variable +3. Verify cookies have correct domain + +## Security Best Practices + +1. **Use HTTPS** - Never use HTTP in production +2. **Strong Client Secret** - Use generated secret, don't reuse +3. **Short Token Lifetime** - Keep access tokens short-lived +4. **Validate State** - Always verify state parameter +5. **Secure Cookies** - Use httpOnly, Secure, SameSite + +## Advanced Configuration + +### Custom Claims + +To add custom claims to the token: + +1. In Authentik, go to **Customization** → **Property Mappings** +2. Create new **Scope Mapping** +3. Add custom attributes +4. Assign to provider + +### Group Mapping + +For team/organization support: + +1. Configure group property mapping in Authentik +2. Headquarter will sync groups on login +3. Use groups for authorization + +### Multiple Applications + +If running multiple environments: + +1. Create separate applications in Authentik +2. Use different client IDs +3. Configure environment-specific redirect URIs + +## Testing + +### Manual Test + +1. Visit `https://app.yourdomain.com` +2. Click "Login" +3. Should redirect to Authentik +4. Login with Authentik credentials +5. Should redirect back to app, logged in + +### API Test + +```bash +# Check auth endpoint +curl https://api.yourdomain.com/auth/me +# Should return 401 (not authenticated) + +# After login, should return user data +curl https://api.yourdomain.com/auth/me --cookie "session=..." +``` + +## Resources + +- [Authentik Documentation](https://goauthentik.io/docs/) +- [OAuth2 Provider Setup](https://goauthentik.io/docs/providers/oauth2/) +- [Headquarter Auth Documentation](../features/auth.md) diff --git a/docs/deployment/docker.md b/docs/deployment/docker.md new file mode 100644 index 0000000..382fb48 --- /dev/null +++ b/docs/deployment/docker.md @@ -0,0 +1,266 @@ +# Docker Deployment Guide + +## Overview + +Headquarter is deployed as a multi-container Docker application using Docker Compose. The stack includes: + +- **API** (FastAPI/Python) +- **Web** (React/Vite) +- **PostgreSQL** (Database) +- **Redis** (Cache) + +## Quick Start + +### Prerequisites + +- Docker Engine 20.10+ +- Docker Compose 2.0+ +- 2GB RAM minimum +- 10GB disk space + +### Development Deployment + +```bash +# Clone repository +git clone https://github.com/your-org/headquarter.git +cd headquarter + +# Copy environment file +cp .env.example .env +# Edit .env with your settings + +# Start all services +docker compose up -d + +# Run database migrations +docker compose exec api alembic upgrade head + +# Access the app +# Frontend: http://localhost:5173 +# API: http://localhost:8000 +``` + +### Production Deployment + +```bash +# Use production compose file +cp .env.example .env +# Configure production values in .env + +docker compose -f docker-compose.traefik.yml up -d + +# Run migrations +docker compose -f docker-compose.traefik.yml exec api alembic upgrade head +``` + +## Configuration + +### Environment Variables + +Required variables: + +```bash +# Domains +API_DOMAIN=api.yourdomain.com +WEB_DOMAIN=app.yourdomain.com + +# Database +POSTGRES_USER=headquarter +POSTGRES_PASSWORD=secure-password +POSTGRES_DB=headquarter + +# Authentik OAuth +AUTHENTIK_DOMAIN=auth.yourdomain.com +AUTHENTIK_CLIENT_ID=your-client-id +AUTHENTIK_CLIENT_SECRET=your-client-secret +AUTHENTIK_APPLICATION_SLUG=headquarter + +# Session +SESSION_SECRET=your-session-secret-min-32-chars + +# Storage +REPO_BASE_PATH=/data/repos +``` + +### Volume Mounts + +| Volume | Container Path | Purpose | +|--------|---------------|---------| +| postgres_data | /var/lib/postgresql/data | Database persistence | +| repo_data | /data/repos | Git repositories | +| avatar_uploads | /app/uploads | User avatars | + +## Docker Compose Files + +### docker-compose.yml (Development) + +Standard development setup with: +- Hot reload for API +- Vite dev server for frontend +- Direct port access +- Local PostgreSQL + +### docker-compose.traefik.yml (Production) + +Production setup with: +- Traefik reverse proxy +- Let's Encrypt SSL +- External Traefik network +- Optimized builds + +## SSL/TLS + +### Development +Self-signed certificates or HTTP only. + +### Production +Automatic Let's Encrypt certificates via Traefik. + +```bash +# Traefik labels for SSL +traefik.http.routers.api.tls=true +traefik.http.routers.api.tls.certresolver=letsencrypt +``` + +## Health Checks + +All services include health checks: + +```yaml +healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8000/health"] + interval: 30s + timeout: 10s + retries: 3 +``` + +## Backup and Restore + +### Database Backup + +```bash +# Automated daily backup +docker compose exec postgres pg_dump -U headquarter headquarter > backup.sql + +# Restore +docker compose exec -T postgres psql -U headquarter < backup.sql +``` + +### Repository Backup + +```bash +# Backup repos volume +docker run --rm -v headquarter_repo_data:/data -v $(pwd):/backup alpine tar czf /backup/repos.tar.gz /data + +# Restore +docker run --rm -v headquarter_repo_data:/data -v $(pwd):/backup alpine tar xzf /backup/repos.tar.gz -C / +``` + +## Monitoring + +### Logs + +```bash +# All services +docker compose logs -f + +# Specific service +docker compose logs -f api + +# Last 100 lines +docker compose logs --tail=100 api +``` + +### Resource Usage + +```bash +# Container stats +docker stats + +# Disk usage +docker system df -v +``` + +## Troubleshooting + +### Common Issues + +**Database connection failed** +```bash +# Check PostgreSQL is running +docker compose ps + +# Check logs +docker compose logs postgres + +# Verify credentials in .env match +``` + +**Migrations failing** +```bash +# Check current migration version +docker compose exec api alembic current + +# Manual upgrade +docker compose exec api alembic upgrade head +``` + +**Permission denied on repos** +```bash +# Fix permissions +docker compose exec api chown -R appuser:appuser /data/repos +``` + +## Updates + +### Rolling Update + +```bash +# Pull latest images +docker compose pull + +# Restart with new images +docker compose up -d + +# Run migrations if needed +docker compose exec api alembic upgrade head +``` + +### Zero-Downtime Update + +```bash +# Scale API to 2 instances +docker compose up -d --scale api=2 + +# Update one instance at a time +# (Requires load balancer configuration) +``` + +## Security Best Practices + +1. **Use strong passwords** for database and session secret +2. **Enable HTTPS** in production +3. **Keep images updated** with security patches +4. **Use read-only volumes** where possible +5. **Limit container capabilities** +6. **Use secrets management** for sensitive data + +```yaml +# Example security hardening +api: + read_only: true + security_opt: + - no-new-privileges:true + cap_drop: + - ALL + cap_add: + - CHOWN + - SETGID + - SETUID +``` + +## Resources + +- [Docker Documentation](https://docs.docker.com/) +- [Docker Compose Reference](https://docs.docker.com/compose/) +- [Traefik Documentation](https://doc.traefik.io/traefik/) diff --git a/docs/deployment/environment.md b/docs/deployment/environment.md new file mode 100644 index 0000000..91ef0a0 --- /dev/null +++ b/docs/deployment/environment.md @@ -0,0 +1,268 @@ +# Environment Variables Reference + +## Overview + +This document describes all environment variables used by Headquarter. Variables are categorized by component and purpose. + +## Required Variables + +These variables must be set for the application to function: + +### Domains + +| Variable | Default | Description | +|----------|---------|-------------| +| `API_DOMAIN` | `localhost` | API server domain (e.g., `api.example.com`) | +| `WEB_DOMAIN` | `localhost` | Web frontend domain (e.g., `app.example.com`) | + +### Database + +| Variable | Default | Description | +|----------|---------|-------------| +| `POSTGRES_USER` | `headquarter` | PostgreSQL username | +| `POSTGRES_PASSWORD` | `headquarter` | PostgreSQL password (change in production!) | +| `POSTGRES_DB` | `headquarter` | PostgreSQL database name | + +### Authentication + +| Variable | Default | Description | +|----------|---------|-------------| +| `AUTHENTIK_DOMAIN` | `authentik.local` | Authentik server domain | +| `AUTHENTIK_CLIENT_ID` | `headquarter-web` | OAuth client ID from Authentik | +| `AUTHENTIK_CLIENT_SECRET` | `change-me` | OAuth client secret (change immediately!) | +| `AUTHENTIK_APPLICATION_SLUG` | `headquarter-web` | Authentik application slug | +| `SESSION_SECRET` | `change-me-session-secret` | Secret for signing session cookies (min 32 chars) | + +## Optional Variables + +### Application + +| Variable | Default | Description | +|----------|---------|-------------| +| `APP_ENV` | `development` | Environment: `development`, `staging`, `production` | +| `LOG_LEVEL` | `INFO` | Logging level: `DEBUG`, `INFO`, `WARNING`, `ERROR` | + +### Database + +| Variable | Default | Description | +|----------|---------|-------------| +| `DATABASE_URL` | (constructed) | Full PostgreSQL connection URL | +| `POSTGRES_HOST` | `postgres` | PostgreSQL hostname | +| `POSTGRES_PORT` | `5432` | PostgreSQL port | + +When `DATABASE_URL` is not set, it's constructed from: +``` +postgresql+asyncpg://POSTGRES_USER:POSTGRES_PASSWORD@POSTGRES_HOST:POSTGRES_PORT/POSTGRES_DB +``` + +### URLs + +| Variable | Default | Description | +|----------|---------|-------------| +| `API_PUBLIC_URL` | (constructed) | Public API URL (e.g., `https://api.example.com`) | +| `WEB_PUBLIC_URL` | (constructed) | Public web URL (e.g., `https://app.example.com`) | + +Constructed from domains when not set: +``` +API: https://API_DOMAIN (production) or http://API_DOMAIN:8000 (development) +WEB: https://WEB_DOMAIN (production) or http://WEB_DOMAIN:5173 (development) +``` + +### Authentik Overrides + +| Variable | Default | Description | +|----------|---------|-------------| +| `AUTHENTIK_AUTHORIZE_URL` | (constructed) | OAuth authorization endpoint | +| `AUTHENTIK_TOKEN_URL` | (constructed) | OAuth token endpoint | +| `AUTHENTIK_JWKS_URL` | (constructed) | JWKS endpoint | +| `AUTHENTIK_ISSUER` | (constructed) | OAuth issuer URL | +| `AUTHENTIK_AUDIENCE` | `headquarter-web` | Token audience | + +Constructed URLs: +``` +https://AUTHENTIK_DOMAIN/application/o/authorize/ +https://AUTHENTIK_DOMAIN/application/o/token/ +https://AUTHENTIK_DOMAIN/application/o/AUTHENTIK_APPLICATION_SLUG/jwks/ +https://AUTHENTIK_DOMAIN/application/o/AUTHENTIK_APPLICATION_SLUG/ +``` + +### Session + +| Variable | Default | Description | +|----------|---------|-------------| +| `SESSION_TTL_HOURS` | `24` | Session cookie lifetime in hours | +| `COOKIE_DOMAIN` | (none) | Cookie domain (set for cross-subdomain) | +| `COOKIE_SECURE` | `true` (prod) | Secure cookie flag | +| `COOKIE_SAMESITE` | `lax` | SameSite cookie attribute | + +### Storage + +| Variable | Default | Description | +|----------|---------|-------------| +| `REPO_BASE_PATH` | `/data/repos` | Base path for git repositories | +| `UPLOAD_DIR` | `uploads` | Directory for file uploads | + +### Development + +| Variable | Default | Description | +|----------|---------|-------------| +| `VITE_API_BASE_URL` | `http://localhost:8000` | Frontend API URL | + +## Docker Compose Variables + +### Traefik + +| Variable | Default | Description | +|----------|---------|-------------| +| `TRAEFIK_NETWORK` | `traefik` | Docker network name for Traefik | +| `TRAEFIK_CERT_RESOLVER` | `letsencrypt` | Certificate resolver name | + +### Docker Specific + +| Variable | Default | Description | +|----------|---------|-------------| +| `COMPOSE_PROJECT_NAME` | `headquarter` | Docker Compose project name | + +## Configuration Examples + +### Development + +```bash +APP_ENV=development +API_DOMAIN=localhost +WEB_DOMAIN=localhost +DATABASE_URL=postgresql+asyncpg://headquarter:headquarter@localhost:5432/headquarter +AUTHENTIK_DOMAIN=authentik.local +AUTHENTIK_CLIENT_ID=headquarter-web +AUTHENTIK_CLIENT_SECRET=dev-secret +SESSION_SECRET=dev-session-secret-change-in-production +VITE_API_BASE_URL=http://localhost:8000 +``` + +### Production + +```bash +APP_ENV=production +API_DOMAIN=api.headquarter.example.com +WEB_DOMAIN=app.headquarter.example.com +POSTGRES_PASSWORD=very-secure-password-here +AUTHENTIK_DOMAIN=auth.example.com +AUTHENTIK_CLIENT_ID=your-uuid-from-authentik +AUTHENTIK_CLIENT_SECRET=your-secret-from-authentik +AUTHENTIK_APPLICATION_SLUG=headquarter-web +SESSION_SECRET=minimum-32-characters-long-secret-key +SESSION_TTL_HOURS=24 +COOKIE_DOMAIN=.headquarter.example.com +TRAEFIK_NETWORK=web +TRAEFIK_CERT_RESOLVER=letsencrypt +REPO_BASE_PATH=/data/repos +``` + +### Testing + +```bash +APP_ENV=testing +DATABASE_URL=postgresql+asyncpg://headquarter:headquarter@localhost:5432/headquarter_test +SESSION_SECRET=test-secret +AUTHENTIK_CLIENT_ID=test-client +AUTHENTIK_CLIENT_SECRET=test-secret +``` + +## Security Checklist + +Before deploying to production, verify: + +- [ ] `POSTGRES_PASSWORD` is strong and unique +- [ ] `AUTHENTIK_CLIENT_SECRET` is kept secret +- [ ] `SESSION_SECRET` is at least 32 characters +- [ ] `APP_ENV` is set to `production` +- [ ] `COOKIE_SECURE` is enabled +- [ ] `COOKIE_DOMAIN` is set for your domain +- [ ] No default secrets in production +- [ ] `.env` file is not committed to git +- [ ] `.env` file has restricted permissions (600) + +## Troubleshooting + +### Variable Not Set + +```bash +# Error: "Environment variable not set" +# Solution: Export the variable or add to .env file +export SESSION_SECRET="your-secret-here" +``` + +### Invalid URL + +```bash +# Error: "Invalid URL" +# Solution: Check domain variables don't include protocol +# Bad: API_DOMAIN=https://api.example.com +# Good: API_DOMAIN=api.example.com +``` + +### Database Connection Failed + +```bash +# Check DATABASE_URL or individual components +# Verify PostgreSQL is running +# Check credentials match +``` + +## Migration from Old Config + +If upgrading from older versions: + +1. `JWT_SECRET` → Removed (not needed with session auth) +2. `ACCESS_TOKEN_TTL_MINUTES` → Removed +3. `REFRESH_TOKEN_TTL_DAYS` → Removed +4. `AUTHENTIK_AUDIENCE` → Now defaults to `AUTHENTIK_CLIENT_ID` +5. `AUTHENTIK_APPLICATION_SLUG` → New variable for URL construction + +## Environment Files + +### Files Structure + +``` +headquarter/ +├── .env # Main environment (not committed) +├── .env.example # Example/template +├── apps/ +│ ├── api/ +│ │ └── .env # API-specific overrides +│ └── web/ +│ └── .env # Frontend-specific overrides +└── docker-compose.traefik.yml # References .env +``` + +### Loading Order + +1. System environment variables +2. `.env` file in project root +3. Component-specific `.env` files +4. Default values in code + +Later values override earlier ones. + +## Validation + +The application validates required variables on startup: + +```python +# Missing critical variable +if not settings.session_secret or settings.session_secret == "change-me": + logger.warning("SESSION_SECRET not configured properly!") + +# Invalid configuration +if settings.app_env == "production" and "localhost" in settings.api_domain: + logger.warning("Using localhost in production!") +``` + +## Best Practices + +1. **Never commit `.env` files** +2. **Use strong passwords** for database and secrets +3. **Rotate secrets** regularly +4. **Use different secrets** per environment +5. **Document custom variables** in this file +6. **Validate configuration** before deployment diff --git a/docs/deployment/traefik.md b/docs/deployment/traefik.md new file mode 100644 index 0000000..106033b --- /dev/null +++ b/docs/deployment/traefik.md @@ -0,0 +1,305 @@ +# Traefik Reverse Proxy Setup + +## Overview + +Traefik serves as the reverse proxy and load balancer for Headquarter in production, handling: +- SSL/TLS termination +- Automatic HTTPS via Let's Encrypt +- Route-based traffic distribution +- WebSocket support + +## Architecture + +``` +Internet + ↓ +Traefik (443) + ├───▶ api.yourdomain.com → Headquarter API (8000) + └───▶ app.yourdomain.com → Headquarter Web (80) +``` + +## Prerequisites + +- Docker Compose installed +- DNS records pointing to your server: + - `api.yourdomain.com` → Server IP + - `app.yourdomain.com` → Server IP +- Ports 80 and 443 open in firewall + +## Configuration + +### 1. Environment Variables + +```bash +# .env file +API_DOMAIN=api.headquarter.example.com +WEB_DOMAIN=app.headquarter.example.com + +# Traefik network (shared with other Traefik instances) +TRAEFIK_NETWORK=web + +# SSL Certificate resolver +TRAEFIK_CERT_RESOLVER=letsencrypt +``` + +### 2. Traefik Labels + +Services are configured via Docker labels: + +```yaml +# API Service labels +labels: + - "traefik.enable=true" + - "traefik.http.routers.headquarter-api.rule=Host(`api.headquarter.example.com`)" + - "traefik.http.routers.headquarter-api.entrypoints=websecure" + - "traefik.http.routers.headquarter-api.tls=true" + - "traefik.http.routers.headquarter-api.tls.certresolver=${TRAEFIK_CERT_RESOLVER:-letsencrypt}" + - "traefik.http.services.headquarter-api.loadbalancer.server.port=8000" + +# Web Service labels +labels: + - "traefik.enable=true" + - "traefik.http.routers.headquarter-frontend.rule=Host(`app.headquarter.example.com`)" + - "traefik.http.routers.headquarter-frontend.entrypoints=websecure" + - "traefik.http.routers.headquarter-frontend.tls=true" + - "traefik.http.routers.headquarter-frontend.tls.certresolver=${TRAEFIK_CERT_RESOLVER:-letsencrypt}" + - "traefik.http.services.headquarter-frontend.loadbalancer.server.port=80" +``` + +### 3. External Network + +Connect to existing Traefik instance: + +```yaml +networks: + traefik: + external: true + name: ${TRAEFIK_NETWORK:-traefik} +``` + +## Deployment Steps + +### Step 1: Verify DNS + +Ensure DNS records resolve to your server: + +```bash +nslookup api.headquarter.example.com +nslookup app.headquarter.example.com +``` + +### Step 2: Start Services + +```bash +# Start with Traefik compose file +docker compose -f docker-compose.traefik.yml up -d + +# Verify containers are running +docker compose -f docker-compose.traefik.yml ps +``` + +### Step 3: Check SSL Certificates + +```bash +# View Traefik logs +docker compose -f docker-compose.traefik.yml logs -f + +# Check certificate status +curl -v https://api.headquarter.example.com/health +``` + +### Step 4: Run Migrations + +```bash +docker compose -f docker-compose.traefik.yml exec api alembic upgrade head +``` + +## SSL Configuration + +### Let's Encrypt (Default) + +Automatic certificate generation and renewal: + +```yaml +labels: + - "traefik.http.routers.headquarter-api.tls.certresolver=letsencrypt" +``` + +### Custom Certificates + +For custom or wildcard certificates: + +```yaml +labels: + - "traefik.http.routers.headquarter-api.tls=true" + - "traefik.http.routers.headquarter-api.tls.certresolver=myresolver" +``` + +### Self-Signed (Development) + +```yaml +labels: + - "traefik.http.routers.headquarter-api.tls=true" + - "traefik.http.routers.headquarter-api.tls.certresolver=selfsigned" +``` + +## Advanced Configuration + +### Rate Limiting + +```yaml +labels: + - "traefik.http.middlewares.ratelimit.ratelimit.average=100" + - "traefik.http.routers.headquarter-api.middlewares=ratelimit" +``` + +### Basic Auth (for staging) + +```yaml +labels: + - "traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/" + - "traefik.http.routers.headquarter-api.middlewares=auth" +``` + +### CORS Headers + +```yaml +labels: + - "traefik.http.middlewares.cors.headers.accesscontrolalloworiginlist=*" + - "traefik.http.routers.headquarter-api.middlewares=cors" +``` + +## Troubleshooting + +### Certificate Issues + +**Problem**: Certificate not generated +```bash +# Check Traefik logs +docker compose logs traefik + +# Verify DNS resolution +nslookup your-domain.com + +# Check port 80 is accessible (required for HTTP challenge) +curl -I http://your-domain.com +``` + +**Problem**: Certificate expired +```bash +# Force renewal +docker compose restart traefik + +# Or delete acme.json and restart +rm acme.json +docker compose restart traefik +``` + +### Routing Issues + +**Problem**: 404 errors +```bash +# Check Traefik dashboard (if enabled) +# http://traefik.yourdomain.com + +# Verify labels are correct +docker compose -f docker-compose.traefik.yml config + +# Check container is on correct network +docker network inspect ${TRAEFIK_NETWORK:-traefik} +``` + +**Problem**: Services not detected +```bash +# Verify traefik.enable label +docker compose -f docker-compose.traefik.yml exec api labels + +# Check Docker provider in Traefik +docker compose logs traefik | grep "Provider connection established" +``` + +## Maintenance + +### Update Traefik + +```bash +# Pull latest Traefik image +docker compose -f docker-compose.traefik.yml pull traefik + +# Restart +docker compose -f docker-compose.traefik.yml up -d traefik +``` + +### View Dashboard + +Enable Traefik dashboard (secure it in production): + +```yaml +# traefik.yml +dashboard: + enabled: true + +# Add to docker-compose.traefik.yml +labels: + - "traefik.http.routers.dashboard.rule=Host(`traefik.yourdomain.com`)" + - "traefik.http.routers.dashboard.tls=true" + - "traefik.http.routers.dashboard.tls.certresolver=letsencrypt" +``` + +## Security Considerations + +1. **Always use HTTPS** in production (redirect HTTP to HTTPS) +2. **Secure Traefik dashboard** with authentication +3. **Use strong certificate resolver** (Let's Encrypt production) +4. **Keep Traefik updated** for security patches +5. **Restrict Docker socket access** if using Docker provider + +## Example Complete Configuration + +```yaml +# docker-compose.traefik.yml +services: + api: + build: + context: ./apps/api + environment: + API_DOMAIN: ${API_DOMAIN} + WEB_DOMAIN: ${WEB_DOMAIN} + # ... other env vars + labels: + - "traefik.enable=true" + - "traefik.http.routers.headquarter-api.rule=Host(`${API_DOMAIN}`)" + - "traefik.http.routers.headquarter-api.entrypoints=websecure" + - "traefik.http.routers.headquarter-api.tls=true" + - "traefik.http.routers.headquarter-api.tls.certresolver=${TRAEFIK_CERT_RESOLVER:-letsencrypt}" + - "traefik.http.services.headquarter-api.loadbalancer.server.port=8000" + networks: + - backend + - traefik + + web: + build: + context: ./apps/web + labels: + - "traefik.enable=true" + - "traefik.http.routers.headquarter-frontend.rule=Host(`${WEB_DOMAIN}`)" + - "traefik.http.routers.headquarter-frontend.entrypoints=websecure" + - "traefik.http.routers.headquarter-frontend.tls=true" + - "traefik.http.routers.headquarter-frontend.tls.certresolver=${TRAEFIK_CERT_RESOLVER:-letsencrypt}" + - "traefik.http.services.headquarter-frontend.loadbalancer.server.port=80" + networks: + - traefik + +networks: + traefik: + external: true + name: ${TRAEFIK_NETWORK:-traefik} + backend: + internal: true +``` + +## Resources + +- [Traefik Documentation](https://doc.traefik.io/traefik/) +- [Docker Compose Integration](https://doc.traefik.io/traefik/providers/docker/) +- [Let's Encrypt Configuration](https://doc.traefik.io/traefik/https/acme/) diff --git a/docs/development/contributing.md b/docs/development/contributing.md new file mode 100644 index 0000000..4e79c41 --- /dev/null +++ b/docs/development/contributing.md @@ -0,0 +1,261 @@ +# Contributing Guide + +## Welcome + +Thank you for your interest in contributing to Headquarter! This document provides guidelines and workflows for contributing. + +## Getting Started + +1. Fork the repository +2. Clone your fork: `git clone https://github.com/your-username/headquarter.git` +3. Set up development environment (see [Setup Guide](./setup.md)) +4. Create a branch: `git checkout -b feature/your-feature` + +## Development Workflow + +### 1. Find or Create an Issue + +- Check existing issues for something to work on +- Create an issue to discuss new features before implementing +- Comment on issues to claim them + +### 2. Create a Branch + +```bash +# Feature branch +git checkout -b feature/description + +# Bug fix branch +git checkout -b fix/description + +# Documentation branch +git checkout -b docs/description +``` + +### 3. Make Changes + +- Write clear, concise code +- Follow existing patterns and conventions +- Add tests for new functionality +- Update documentation as needed + +### 4. Run Quality Gates + +```bash +# Backend +cd apps/api +ruff check src/ tests/ +mypy src/ +pytest + +# Frontend +cd apps/web +npm run lint +npm run typecheck +npm run test +npm run build +``` + +### 5. Commit Changes + +We use conventional commits: + +```bash +# Format: type(scope): description + +# Examples: +git commit -m "feat(auth): add OAuth2 login" +git commit -m "fix(api): handle missing user gracefully" +git commit -m "docs(readme): update installation instructions" +git commit -m "test(git): add URL parsing tests" +git commit -m "refactor(models): extract base repository" +``` + +**Types**: +- `feat`: New feature +- `fix`: Bug fix +- `docs`: Documentation +- `style`: Formatting (no code change) +- `refactor`: Code restructuring +- `test`: Adding tests +- `chore`: Maintenance tasks + +### 6. Push and Create Pull Request + +```bash +git push origin feature/description +``` + +**PR Description should include**: +- What changed and why +- How to test +- Screenshots (for UI changes) +- Link to related issue + +## Code Standards + +### Python (Backend) + +**Style**: Follow PEP 8 and project conventions + +```python +# Function naming: snake_case +def get_user_by_id(user_id: str) -> User | None: + pass + +# Class naming: PascalCase +class GitRepositoryService: + pass + +# Constants: UPPER_SNAKE_CASE +MAX_FILE_SIZE = 1024 * 1024 # 1MB + +# Type hints required +def process_data(data: dict[str, Any]) -> ProcessedResult: + pass +``` + +**Docstrings**: Google style +```python +def extract_base_repo_url(url: str) -> str | None: + """Extract base repository URL from a browser URL. + + Args: + url: The URL to parse, may be a browser URL or git URL. + + Returns: + The base repository URL with .git suffix, or None if parsing fails. + + Examples: + >>> extract_base_repo_url("https://github.com/user/repo/tree/main") + 'https://github.com/user/repo.git' + + >>> extract_base_repo_url("https://github.com/user/repo.git") + 'https://github.com/user/repo.git' + """ + pass +``` + +### TypeScript (Frontend) + +**Style**: Follow existing patterns + +```typescript +// Interface naming: PascalCase +interface User { + id: string; + email: string; + name: string; +} + +// Function naming: camelCase +function getUserById(userId: string): Promise { + return api.get(`/users/${userId}`); +} + +// Component naming: PascalCase +const UserProfile: React.FC = ({ user }) => { + return
{user.name}
; +}; +``` + +## Testing Requirements + +### New Features + +- Unit tests for business logic +- Integration tests for API endpoints +- Component tests for UI components + +### Bug Fixes + +- Regression test that would catch the bug +- Verify fix with the test + +### Example + +```python +# Backend test +def test_extract_github_browser_url(): + url = "https://github.com/user/repo/tree/main" + result = extract_base_repo_url(url) + assert result == "https://github.com/user/repo.git" + +# Frontend test +test('shows file tree', () => { + render( {}} />); + expect(screen.getByText('src')).toBeInTheDocument(); +}); +``` + +## Documentation + +Update documentation when: +- Adding new features +- Changing API endpoints +- Modifying configuration +- Adding environment variables + +**Documentation locations**: +- `README.md` - Project overview +- `docs/features/` - Feature documentation +- `docs/api/` - API documentation +- `docs/deployment/` - Deployment guides + +## Review Process + +1. **Automated checks** must pass (CI/CD) +2. **Code review** by at least one maintainer +3. **Approval** required before merge +4. **Squash merge** to keep history clean + +### Review Checklist + +**For Authors**: +- [ ] Tests pass locally +- [ ] Quality gates pass +- [ ] Documentation updated +- [ ] PR description is clear + +**For Reviewers**: +- [ ] Code makes sense +- [ ] Tests cover changes +- [ ] No security issues +- [ ] Follows conventions + +## Release Process + +1. Update version in `pyproject.toml` and `package.json` +2. Update `CHANGELOG.md` +3. Create git tag: `git tag v1.2.3` +4. Push tag: `git push origin v1.2.3` +5. Create GitHub release with notes + +## Community + +### Communication Channels + +- GitHub Issues: Bug reports and feature requests +- GitHub Discussions: Questions and ideas +- Pull Requests: Code contributions + +### Code of Conduct + +- Be respectful and inclusive +- Welcome newcomers +- Focus on constructive feedback +- Respect different viewpoints + +## Questions? + +- Check existing documentation +- Search closed issues +- Ask in GitHub Discussions +- Join community chat (if available) + +## Resources + +- [Development Setup](./setup.md) +- [Testing Guide](./testing.md) +- [Quality Gates](./quality-gates.md) +- [Project README](../../README.md) diff --git a/docs/development/quality-gates.md b/docs/development/quality-gates.md new file mode 100644 index 0000000..3815ac9 --- /dev/null +++ b/docs/development/quality-gates.md @@ -0,0 +1,365 @@ +# Quality Gates + +## Overview + +All code changes must pass quality gates before being merged. These gates ensure code consistency, type safety, and prevent common issues. + +## Backend Quality Gates + +### 1. Code Formatting (ruff) + +**Purpose**: Enforce consistent code style + +```bash +cd apps/api +ruff check src/ tests/ +ruff format src/ tests/ +``` + +**Configuration** (`pyproject.toml`): +```toml +[tool.ruff] +line-length = 100 +target-version = "py311" +select = ["E", "F", "I", "W", "UP"] +ignore = ["E501"] + +[tool.ruff.lint.pydocstyle] +convention = "google" +``` + +**Pre-commit hook**: +```bash +# Install pre-commit +pip install pre-commit +pre-commit install + +# Run manually +pre-commit run --all-files +``` + +### 2. Type Checking (mypy) + +**Purpose**: Catch type errors before runtime + +```bash +cd apps/api +mypy src/ +``` + +**Configuration** (`pyproject.toml`): +```toml +[tool.mypy] +python_version = "3.11" +warn_return_any = true +warn_unused_ignores = true +disallow_untyped_defs = true +disallow_incomplete_defs = true +check_untyped_defs = true +``` + +**Common issues**: +- Missing type hints on function parameters +- Returning wrong type +- None checks needed + +### 3. Unit Tests (pytest) + +**Purpose**: Verify functionality works as expected + +```bash +cd apps/api +pytest -v +``` + +**Requirements**: +- All tests must pass +- New code should have tests +- Coverage should not decrease + +### 4. Security Checks + +**Bandit** (security linter): +```bash +bandit -r src/ +``` + +**Safety** (dependency vulnerabilities): +```bash +safety check +``` + +## Frontend Quality Gates + +### 1. Type Checking (TypeScript) + +**Purpose**: Catch type errors at build time + +```bash +cd apps/web +npm run typecheck +``` + +**Configuration** (`tsconfig.json`): +```json +{ + "compilerOptions": { + "target": "ES2020", + "lib": ["ES2020", "DOM", "DOM.Iterable"], + "module": "ESNext", + "skipLibCheck": true, + "strict": true, + "noUnusedLocals": true, + "noUnusedParameters": true, + "noFallthroughCasesInSwitch": true + } +} +``` + +### 2. Linting (ESLint) + +**Purpose**: Enforce code style and catch issues + +```bash +cd apps/web +npm run lint +npm run lint:fix +``` + +**Configuration** (`.eslintrc.cjs`): +```javascript +module.exports = { + root: true, + env: { browser: true, es2020: true }, + extends: [ + 'eslint:recommended', + 'plugin:@typescript-eslint/recommended', + 'plugin:react-hooks/recommended', + ], + parser: '@typescript-eslint/parser', + plugins: ['react-refresh'], + rules: { + 'react-refresh/only-export-components': [ + 'warn', + { allowConstantExport: true }, + ], + }, +} +``` + +### 3. Build Check + +**Purpose**: Ensure production build succeeds + +```bash +cd apps/web +npm run build +``` + +**Requirements**: +- No TypeScript errors +- No build warnings +- Bundle size reasonable + +### 4. Unit Tests (Vitest) + +```bash +cd apps/web +npm run test +``` + +**Requirements**: +- All tests pass +- No test failures +- Coverage maintained + +## Running All Gates + +### Backend + +```bash +cd apps/api + +# Run all gates +ruff check src/ tests/ && \ +mypy src/ && \ +pytest + +# Or use Makefile +make lint # ruff + mypy +make test # pytest +make check # All backend gates +``` + +### Frontend + +```bash +cd apps/web + +# Run all gates +npm run lint && \ +npm run typecheck && \ +npm run test && \ +npm run build + +# Or use package.json scripts +npm run check # All frontend gates +``` + +### Full Project + +```bash +# From root +make check-all # Run all backend and frontend gates +``` + +## Continuous Integration + +All quality gates run automatically on: +- Every Pull Request +- Every push to main branch + +**CI Pipeline**: +```yaml +stages: + - lint + - test + - build + +backend-lint: + stage: lint + script: + - cd apps/api && ruff check src/ tests/ + - cd apps/api && mypy src/ + +backend-test: + stage: test + script: + - cd apps/api && pytest -v + +frontend-lint: + stage: lint + script: + - cd apps/web && npm run lint + - cd apps/web && npm run typecheck + +frontend-test: + stage: test + script: + - cd apps/web && npm run test + +frontend-build: + stage: build + script: + - cd apps/web && npm run build +``` + +## Fixing Common Issues + +### Backend + +**ruff: Line too long** +```python +# Bad +result = some_very_long_function_name(with_many_parameters, that_make_the_line_too_long) + +# Good +result = some_very_long_function_name( + with_many_parameters, + that_make_the_line_too_long, +) +``` + +**mypy: Missing return type** +```python +# Bad +def get_user(user_id): + return User.query.get(user_id) + +# Good +def get_user(user_id: str) -> User | None: + return User.query.get(user_id) +``` + +### Frontend + +**TypeScript: Implicit any** +```typescript +// Bad +function processData(data) { + return data.map(item => item.name); +} + +// Good +function processData(data: DataItem[]) { + return data.map(item => item.name); +} +``` + +**ESLint: Unused variable** +```typescript +// Bad +const [count, setCount] = useState(0); +// count is never used + +// Good +const [count] = useState(0); +// Or remove if not needed +``` + +## IDE Integration + +### VS Code + +**Settings** (`.vscode/settings.json`): +```json +{ + "python.linting.enabled": true, + "python.linting.mypyEnabled": true, + "python.formatting.provider": "ruff", + "editor.formatOnSave": true, + "editor.codeActionsOnSave": { + "source.fixAll.eslint": true + } +} +``` + +### PyCharm + +1. **File** → **Settings** → **Tools** → **External Tools** +2. Add ruff and mypy as external tools +3. Set up pre-commit hooks + +## Pre-commit Hooks + +`.pre-commit-config.yaml`: +```yaml +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.1.0 + hooks: + - id: ruff + args: [--fix, --exit-non-zero-on-fix] + - id: ruff-format + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.7.0 + hooks: + - id: mypy + additional_dependencies: [types-all] + + - repo: local + hooks: + - id: frontend-lint + name: Frontend Lint + entry: bash -c 'cd apps/web && npm run lint' + language: system + files: ^apps/web/ +``` + +## Resources + +- [ruff Documentation](https://docs.astral.sh/ruff/) +- [mypy Documentation](https://mypy.readthedocs.io/) +- [ESLint Documentation](https://eslint.org/) +- [TypeScript Handbook](https://www.typescriptlang.org/docs/) diff --git a/docs/development/setup.md b/docs/development/setup.md new file mode 100644 index 0000000..44e8649 --- /dev/null +++ b/docs/development/setup.md @@ -0,0 +1,299 @@ +# Development Setup Guide + +## Prerequisites + +- Python 3.11+ +- Node.js 18+ +- PostgreSQL 15+ +- Redis 7+ +- Git + +## Quick Setup + +### 1. Clone Repository + +```bash +git clone https://github.com/your-org/headquarter.git +cd headquarter +``` + +### 2. Backend Setup + +```bash +cd apps/api + +# Create virtual environment +python -m venv .venv +source .venv/bin/activate # Linux/Mac +# .venv\Scripts\activate # Windows + +# Install dependencies +pip install -e ".[dev]" + +# Copy environment file +cp .env.example .env +# Edit .env with your settings +``` + +### 3. Frontend Setup + +```bash +cd apps/web + +# Install dependencies +npm install + +# Copy environment file +cp .env.example .env +# Edit .env with your settings +``` + +### 4. Database Setup + +```bash +# Start PostgreSQL and Redis +# (Using Docker or local installation) +docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=headquarter postgres:15 +docker run -d -p 6379:6379 redis:7 + +# Create database +createdb headquarter + +# Run migrations +cd apps/api +alembic upgrade head + +# (Optional) Seed data +python -m src.scripts.seed +``` + +### 5. Start Development Servers + +Terminal 1 - Backend: +```bash +cd apps/api +source .venv/bin/activate +python -m uvicorn src.main:app --reload --port 8000 +``` + +Terminal 2 - Frontend: +```bash +cd apps/web +npm run dev +``` + +Terminal 3 - (Optional) Authentik: +```bash +# If using local Authentik +docker compose -f docker-compose.authentik.yml up -d +``` + +## Development Environment Variables + +Create `apps/api/.env`: + +```bash +APP_ENV=development +DATABASE_URL=postgresql+asyncpg://headquarter:headquarter@localhost:5432/headquarter +REDIS_URL=redis://localhost:6379/0 +SESSION_SECRET=dev-session-secret-change-me +API_DOMAIN=localhost +WEB_DOMAIN=localhost + +# Authentik (optional for local dev) +AUTHENTIK_DOMAIN=authentik.local +AUTHENTIK_CLIENT_ID=headquarter-web +AUTHENTIK_CLIENT_SECRET=change-me +AUTHENTIK_APPLICATION_SLUG=headquarter-web +``` + +Create `apps/web/.env`: + +```bash +VITE_API_BASE_URL=http://localhost:8000 +``` + +## IDE Setup + +### VS Code Extensions + +Recommended extensions: +- Python (ms-python.python) +- Pylance (ms-python.vscode-pylance) +- ESLint (dbaeumer.vscode-eslint) +- Prettier (esbenp.prettier-vscode) +- TypeScript Importer (pmneo.tsimporter) + +### PyCharm/IntelliJ + +1. Open `apps/api` as project +2. Set Python interpreter to `.venv` +3. Enable Django/Flask support for FastAPI + +## Debugging + +### Backend Debugging + +**VS Code launch.json**: +```json +{ + "version": "0.2.0", + "configurations": [ + { + "name": "Python: FastAPI", + "type": "python", + "request": "launch", + "module": "uvicorn", + "args": ["src.main:app", "--reload", "--port", "8000"], + "jinja": true, + "justMyCode": true + } + ] +} +``` + +**PyCharm**: +1. Run → Edit Configurations +2. Add Python configuration +3. Module name: `uvicorn` +4. Parameters: `src.main:app --reload --port 8000` + +### Frontend Debugging + +**VS Code launch.json**: +```json +{ + "type": "chrome", + "request": "launch", + "name": "Launch Chrome against localhost", + "url": "http://localhost:5173", + "webRoot": "${workspaceFolder}/apps/web/src" +} +``` + +## Common Tasks + +### Database Migrations + +```bash +cd apps/api + +# Create new migration +alembic revision --autogenerate -m "description" + +# Run migrations +alembic upgrade head + +# Downgrade +alembic downgrade -1 + +# Current version +alembic current + +# History +alembic history +``` + +### Adding Dependencies + +**Backend**: +```bash +cd apps/api +# Add production dependency +pip install package-name +# Add to pyproject.toml [project.dependencies] + +# Add dev dependency +pip install -e ".[dev]" +# Add to pyproject.toml [project.optional-dependencies.dev] +``` + +**Frontend**: +```bash +cd apps/web +npm install package-name +npm install -D package-name # dev dependency +``` + +### Git Workflow + +1. Create feature branch: `git checkout -b feature/name` +2. Make changes +3. Run quality gates (see below) +4. Commit: `git commit -m "feat: description"` +5. Push: `git push origin feature/name` +6. Create Pull Request + +## Project Structure + +``` +headquarter/ +├── apps/ +│ ├── api/ # Backend (FastAPI) +│ │ ├── src/ +│ │ │ ├── api/ # API routes +│ │ │ ├── auth/ # Authentication +│ │ │ ├── models/ # Database models +│ │ │ ├── utils/ # Utilities +│ │ │ └── main.py # Entry point +│ │ ├── tests/ # Test suites +│ │ ├── alembic/ # Migrations +│ │ └── pyproject.toml # Dependencies +│ └── web/ # Frontend (React) +│ ├── src/ +│ │ ├── api/ # API clients +│ │ ├── components/ # React components +│ │ ├── pages/ # Page components +│ │ └── styles/ # CSS +│ └── package.json # Dependencies +├── docs/ # Documentation +├── docker-compose.yml # Dev setup +└── Makefile # Common commands +``` + +## Troubleshooting + +### Database Connection Errors + +```bash +# Check PostgreSQL is running +pg_isready -h localhost -p 5432 + +# Check credentials +psql postgresql://headquarter:headquarter@localhost:5432/headquarter -c "SELECT 1" +``` + +### Port Conflicts + +```bash +# Find process using port 8000 +lsof -i :8000 + +# Kill process +kill -9 +``` + +### Node Modules Issues + +```bash +cd apps/web +rm -rf node_modules package-lock.json +npm install +``` + +### Python Environment Issues + +```bash +cd apps/api +rm -rf .venv +python -m venv .venv +source .venv/bin/activate +pip install -e ".[dev]" +``` + +## Resources + +- [FastAPI Documentation](https://fastapi.tiangolo.com/) +- [React Documentation](https://react.dev/) +- [SQLAlchemy Documentation](https://docs.sqlalchemy.org/) +- [Alembic Documentation](https://alembic.sqlalchemy.org/) diff --git a/docs/development/testing.md b/docs/development/testing.md new file mode 100644 index 0000000..c2b3b99 --- /dev/null +++ b/docs/development/testing.md @@ -0,0 +1,365 @@ +# Testing Guide + +## Overview + +Headquarter has comprehensive testing for both backend and frontend to ensure reliability and catch regressions. + +## Backend Testing + +### Test Framework + +- **pytest**: Test runner +- **pytest-asyncio**: Async test support +- **httpx**: HTTP client for API tests +- **factory-boy**: Test data generation (recommended) + +### Test Structure + +``` +apps/api/tests/ +├── conftest.py # Shared fixtures +├── test_auth_api.py # Auth endpoint tests +├── test_auth_services.py # Auth service tests +├── test_models.py # Database model tests +├── test_projects_api.py # Project endpoint tests +├── test_git_repositories.py # Repository tests +└── ... +``` + +### Running Tests + +```bash +cd apps/api + +# Run all tests +pytest + +# Run with coverage +pytest --cov=src --cov-report=html + +# Run specific test file +pytest tests/test_auth_api.py + +# Run specific test +pytest tests/test_auth_api.py::test_login_redirect + +# Run with verbose output +pytest -v + +# Run async tests +pytest --asyncio-mode=auto +``` + +### Writing Tests + +**Unit Test Example**: +```python +import pytest +from src.utils.git_url_parser import extract_base_repo_url + +def test_extract_github_url(): + url = "https://github.com/user/repo/tree/main" + result = extract_base_repo_url(url) + assert result == "https://github.com/user/repo.git" + +def test_valid_git_url_unchanged(): + url = "https://github.com/user/repo.git" + result = extract_base_repo_url(url) + assert result == url +``` + +**Async Test Example**: +```python +import pytest +from httpx import AsyncClient + +@pytest.mark.asyncio +async def test_get_projects(client: AsyncClient): + response = await client.get("/projects") + assert response.status_code == 200 + assert isinstance(response.json(), list) +``` + +**API Integration Test**: +```python +@pytest.mark.asyncio +async def test_create_project(client: AsyncClient, auth_headers): + response = await client.post( + "/projects", + json={"name": "Test Project"}, + headers=auth_headers + ) + assert response.status_code == 201 + data = response.json() + assert data["name"] == "Test Project" + assert "id" in data +``` + +### Test Fixtures + +**conftest.py** provides: + +```python +import pytest +from httpx import AsyncClient + +@pytest.fixture +async def client(): + from src.main import app + async with AsyncClient(app=app, base_url="http://test") as client: + yield client + +@pytest.fixture +async def auth_headers(client): + # Login and return auth headers + response = await client.post("/auth/login") + # ... setup session + return {"Cookie": "session=..."} +``` + +### Test Database + +Tests use a separate database: + +```bash +# Test database URL (from .env) +TEST_DATABASE_URL=postgresql+asyncpg://headquarter:headquarter@localhost:5432/headquarter_test + +# Run tests with test DB +TEST_DATABASE_URL=... pytest +``` + +## Frontend Testing + +### Test Framework + +- **Vitest**: Test runner +- **React Testing Library**: Component testing +- **jsdom**: DOM environment + +### Test Structure + +``` +apps/web/src/ +├── components/ +│ └── protected-route.test.tsx +├── pages/ +│ ├── dashboard.test.tsx +│ └── projects.test.tsx +└── test/ + └── setup.ts # Test setup +``` + +### Running Tests + +```bash +cd apps/web + +# Run all tests +npm run test + +# Run in watch mode +npm run test -- --watch + +# Run with coverage +npm run test -- --coverage + +# Run specific file +npm run test -- protected-route +``` + +### Writing Tests + +**Component Test Example**: +```typescript +import { render, screen, fireEvent } from '@testing-library/react'; +import { FileTree } from '../components/file-tree'; + +describe('FileTree', () => { + const mockFiles = [ + { name: 'src', type: 'directory', path: 'src' }, + { name: 'main.py', type: 'file', path: 'src/main.py' }, + ]; + + it('renders file list', () => { + render( {}} />); + + expect(screen.getByText('src')).toBeInTheDocument(); + expect(screen.getByText('main.py')).toBeInTheDocument(); + }); + + it('calls onFileClick when file clicked', () => { + const handleClick = vi.fn(); + render(); + + fireEvent.click(screen.getByText('main.py')); + expect(handleClick).toHaveBeenCalledWith(mockFiles[1]); + }); +}); +``` + +**Async Test Example**: +```typescript +import { render, screen, waitFor } from '@testing-library/react'; +import { ProjectsPage } from '../pages/projects'; + +describe('ProjectsPage', () => { + it('loads and displays projects', async () => { + render(); + + await waitFor(() => { + expect(screen.getByText('My Projects')).toBeInTheDocument(); + }); + }); +}); +``` + +## E2E Testing (Future) + +### Playwright Setup + +```bash +cd apps/web +npm install -D @playwright/test +npx playwright install +``` + +**Example E2E Test**: +```typescript +import { test, expect } from '@playwright/test'; + +test('user can login', async ({ page }) => { + await page.goto('http://localhost:5173'); + await page.click('text=Login'); + + // Authentik login + await page.fill('[name="username"]', 'test@example.com'); + await page.fill('[name="password"]', 'password'); + await page.click('text=Sign In'); + + // Should redirect back to app + await expect(page).toHaveURL('http://localhost:5173/dashboard'); +}); +``` + +## Test Data + +### Factories (Recommended) + +Use factory-boy for test data: + +```python +# tests/factories.py +import factory +from src.models.user import User + +class UserFactory(factory.Factory): + class Meta: + model = User + + email = factory.Faker('email') + name = factory.Faker('name') + authentik_id = factory.Faker('uuid4') +``` + +### Fixtures + +```python +@pytest.fixture +async def test_user(db_session): + user = UserFactory() + db_session.add(user) + await db_session.commit() + return user +``` + +## Coverage Goals + +| Component | Target Coverage | +|-----------|----------------| +| Backend API | 80%+ | +| Backend Services | 90%+ | +| Backend Models | 90%+ | +| Frontend Components | 70%+ | +| Frontend Pages | 60%+ | + +## Continuous Integration + +### GitHub Actions (Example) + +```yaml +name: Tests +on: [push, pull_request] + +jobs: + backend: + runs-on: ubuntu-latest + services: + postgres: + image: postgres:15 + env: + POSTGRES_PASSWORD: headquarter + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-python@v4 + with: + python-version: '3.11' + - run: pip install -e ".[dev]" + - run: pytest --cov=src --cov-report=xml + + frontend: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-node@v3 + with: + node-version: '18' + - run: cd apps/web && npm ci + - run: cd apps/web && npm run test +``` + +## Best Practices + +### Backend +1. **Test isolation**: Each test should be independent +2. **Use fixtures**: Don't repeat setup code +3. **Mock external services**: Authentik, git operations +4. **Test edge cases**: Empty lists, invalid inputs, errors +5. **Async properly**: Use `pytest.mark.asyncio` and async fixtures + +### Frontend +1. **Test behavior, not implementation**: Check what user sees +2. **Use data-testid**: For stable selectors +3. **Mock API calls**: Don't hit real backend +4. **Test accessibility**: Use `screen.getByRole` +5. **Snapshot sparingly**: Only for complex UIs + +## Debugging Tests + +### Backend +```bash +# Run with debugger +pytest --pdb + +# Run specific test with verbose +pytest -v -s test_file.py::test_name + +# Show print statements +pytest -s +``` + +### Frontend +```bash +# Debug mode +npm run test -- --reporter=verbose + +# Show browser (for E2E) +npx playwright test --headed +``` + +## Resources + +- [pytest Documentation](https://docs.pytest.org/) +- [React Testing Library](https://testing-library.com/docs/react-testing-library/intro/) +- [Vitest Documentation](https://vitest.dev/) +- [Playwright Documentation](https://playwright.dev/) diff --git a/docs/features/auth.md b/docs/features/auth.md new file mode 100644 index 0000000..7135c31 --- /dev/null +++ b/docs/features/auth.md @@ -0,0 +1,100 @@ +# Authentication + +## Overview + +Headquarter uses OAuth2 authentication via Authentik. Users log in through their Authentik identity provider and receive a session cookie for authenticated access. + +## How to Use + +### Logging In + +1. Navigate to the application +2. Click the **"Login"** button in the header +3. You will be redirected to **Authentik** +4. Enter your Authentik credentials +5. You will be redirected back to Headquarter, now logged in + +### User Profile + +After logging in, you can view your profile: + +1. Click your **name** in the header +2. Select **"Profile"** from the dropdown +3. View and edit: + - Display name + - Email + - Avatar (upload or change) + +### Logging Out + +1. Click your **name** in the header +2. Select **"Logout"** +3. Your session will be cleared +4. You will be redirected to the login page + +## Authentication Flow + +``` +User → Click Login → Authentik Login → OAuth2 Callback → Session Cookie → Authenticated +``` + +### Technical Details + +**Session Management:** +- Uses signed session cookies +- Cookie is `HttpOnly` and `Secure` (in production) +- Session expires after configurable TTL (default: 24 hours) + +**OAuth2 Flow:** +1. User clicks login +2. Backend redirects to Authentik authorize URL +3. User authenticates with Authentik +4. Authentik redirects back with authorization code +5. Backend exchanges code for access token +6. Backend fetches user info from Authentik +7. Backend creates/updates local user record +8. Backend sets session cookie +9. User is authenticated + +## API Reference + +### Endpoints + +- `GET /auth/login` - Initiate login (redirects to Authentik) +- `GET /auth/callback` - OAuth2 callback +- `GET /auth/me` - Get current user +- `POST /auth/logout` - Logout (clears session) + +See [Auth API](../api/auth.md) for detailed endpoint documentation. + +## Configuration + +| Variable | Description | Default | +|----------|-------------|---------| +| `AUTHENTIK_DOMAIN` | Authentik server domain | - | +| `AUTHENTIK_CLIENT_ID` | OAuth client ID | - | +| `AUTHENTIK_CLIENT_SECRET` | OAuth client secret | - | +| `AUTHENTIK_APPLICATION_SLUG` | Application slug for URLs | `headquarter-web` | +| `SESSION_SECRET` | Session cookie signing secret | `change-me` | +| `SESSION_TTL_HOURS` | Session duration | `24` | + +## Troubleshooting + +### Login Loop + +**Issue:** After logging in, you're redirected back to login +**Solution:** Check that cookie domain matches your domain configuration + +### "Invalid State" Error + +**Issue:** Error about invalid state parameter +**Solution:** Clear cookies and try again. If persistent, check Authentik configuration. + +### Session Expired + +**Issue:** "Session expired" message +**Solution:** Log in again. Session duration is configurable via `SESSION_TTL_HOURS`. + +## Related Features + +- [User Settings](settings.md) - Configure user preferences diff --git a/docs/features/git-history.md b/docs/features/git-history.md new file mode 100644 index 0000000..c8f32cb --- /dev/null +++ b/docs/features/git-history.md @@ -0,0 +1,106 @@ +# Git History Visualization + +## Overview + +The Git History page provides a visual representation of a repository's commit history, including a branch graph and detailed commit information. + +## How to Use + +### Accessing History + +1. Navigate to a **project workspace** +2. Select a **repository** from the dropdown +3. Click the **"History"** button on the repository card + +Or: + +1. Go to the **Repositories** page +2. Click **"History"** on any repository card + +### History View Layout + +``` +┌─────────────────────────────────────────────┐ +│ [Branch Selector ▼] [Repository Name] │ +├─────────────────┬───────────────────────────┤ +│ Commit List │ Commit Details │ +│ │ │ +│ ●─●─○─● │ Commit: abc1234 │ +│ │ └─○ │ Author: John Doe │ +│ │ │ Date: 2024-01-01 │ +│ ● │ │ +│ │ │ Message: │ +│ ○ │ Fix bug in parser │ +│ │ │ +│ │ Stats: │ +│ │ +15 -3 lines │ +│ │ │ +│ │ Diff: │ +│ │ ```diff │ +│ │ + new line │ +│ │ - old line │ +│ │ ``` │ +└─────────────────┴───────────────────────────┘ +``` + +### Commit Graph + +The left panel shows: +- **Commit hashes** (abbreviated) +- **Branch/merge indicators** (lines connecting commits) +- **Commit messages** (first line) +- **Author** and **date** +- **Branch tags** (colored labels) + +**Graph symbols:** +- `●` - Regular commit +- Branch lines show merge history +- Different colors indicate different branches + +### Commit Details + +Click any commit to see details: + +**Metadata:** +- Full commit hash +- Author name and email +- Commit date and time +- Complete commit message + +**Statistics:** +- Files changed +- Lines added (+) +- Lines removed (-) + +**Diff:** +- Syntax-highlighted diff +- Added lines in green +- Removed lines in red +- Context lines for reference + +### Branch Filtering + +Use the **Branch Selector** to filter commits: +- Select **"All"** to see all branches +- Select a specific branch to see only that branch's history +- The graph updates to show only relevant commits + +### Navigation + +- **Click** a commit to view details +- **Scroll** the commit list to see older commits +- The view shows up to 10,000 commits (loads all at once) + +## API Reference + +### Endpoints + +- `GET /projects/{id}/repositories/{id}/history` - Get commit history +- `GET /projects/{id}/repositories/{id}/commits/{hash}` - Get commit details + +See [Repositories API](../api/repositories.md) for detailed endpoint documentation. + +## Related Features + +- [Git Repositories](repositories.md) - Manage repositories +- [Repository Workspace](workspace.md) - Browse files diff --git a/docs/features/projects.md b/docs/features/projects.md new file mode 100644 index 0000000..d9be780 --- /dev/null +++ b/docs/features/projects.md @@ -0,0 +1,73 @@ +# Project Management + +## Overview + +Projects are the top-level organizational unit in Headquarter. Each project can contain multiple git repositories and serves as a workspace for related development work. + +## How to Use + +### Creating a Project + +1. Navigate to the **Projects** page from the sidebar +2. Click the **"New Project"** button +3. Enter a **name** for your project (required) +4. Optionally add a **description** +5. Click **"Create Project"** + +### Viewing Projects + +The Projects page displays all your projects in a card layout showing: +- Project name +- Description (if set) +- Creation date +- Associated repositories count + +### Opening a Project Workspace + +Click on any project card to open its **workspace**. The workspace is the default view for a project and shows: +- Repository file browser +- Branch selector +- File viewer + +### Editing a Project + +1. From the Projects page, click the **menu icon** (⋮) on a project card +2. Select **"Edit"** +3. Update the name or description +4. Click **"Save"** + +### Deleting a Project + +1. From the Projects page, click the **menu icon** (⋮) on a project card +2. Select **"Delete"** +3. Confirm the deletion + +**Note:** Deleting a project also deletes all associated repositories and their data. This action cannot be undone. + +## Project Workspace + +The project workspace is the default view when you click on a project. It provides: + +- **Repository Browser**: Navigate files and directories +- **File Viewer**: View file contents with syntax highlighting +- **Branch Management**: Switch between branches +- **Repository Switcher**: Switch between project repositories + +See [Repository Workspace](workspace.md) for detailed documentation. + +## API Reference + +### Endpoints + +- `GET /projects` - List all projects +- `POST /projects` - Create a new project +- `GET /projects/{id}` - Get project details +- `PUT /projects/{id}` - Update a project +- `DELETE /projects/{id}` - Delete a project + +See [Projects API](../api/projects.md) for detailed endpoint documentation. + +## Related Features + +- [Repository Workspace](workspace.md) - Browse and edit repository files +- [Git Repositories](repositories.md) - Manage project repositories diff --git a/docs/features/repositories.md b/docs/features/repositories.md new file mode 100644 index 0000000..216bcea --- /dev/null +++ b/docs/features/repositories.md @@ -0,0 +1,91 @@ +# Git Repositories + +## Overview + +Git repositories are managed within projects. You can create bare repositories for new projects or clone existing repositories from remote sources. + +## How to Use + +### Creating a Repository + +1. Navigate to a **project workspace** or the **Repositories** page +2. Click the **"New Repository"** button +3. Fill in the form: + - **Name**: Repository name (required) + - **Remote URL**: For cloning (optional) + - **Mirror Clone**: Toggle for mirror clones +4. Click **"Create Repository"** + +#### Bare Repository (No Remote URL) + +Creates a new bare git repository. Use this for: +- New projects +- Local-only repositories +- Repositories that will be pushed to later + +#### Clone from Remote + +Enter a git URL to clone from: +- `https://github.com/user/repo.git` +- `git@github.com:user/repo.git` +- `https://gitlab.com/user/repo.git` + +**Smart URL Parsing:** If you paste a browser URL (like `https://github.com/user/repo/tree/main`), the system will automatically suggest the correct git URL. + +#### Mirror Clone + +Enable **"Mirror Clone"** to create a full mirror of a remote repository: +- Clones all branches and tags +- Sets up remote tracking +- Updates can be fetched later + +### Smart URL Parsing + +When pasting URLs, the system automatically detects browser URLs and suggests the correct git clone URL: + +**Examples:** +- `https://github.com/user/repo/tree/main` → `https://github.com/user/repo.git` +- `https://github.com/user/repo/blob/main/README.md` → `https://github.com/user/repo.git` +- `https://gitlab.com/user/repo/-/tree/develop` → `https://gitlab.com/user/repo.git` + +You can accept the suggestion or proceed with the original URL. + +### Viewing Repositories + +The Repositories page shows all repositories in a project: +- Repository name +- Clone URL +- Mirror status +- Creation date + +### Repository Actions + +Each repository card provides: +- **History**: View commit history and branch graph +- **Browse**: Open in workspace file browser +- **Delete**: Remove the repository + +### Deleting a Repository + +1. Click the **menu icon** (⋮) on a repository card +2. Select **"Delete"** +3. Confirm the deletion + +**Note:** This permanently deletes the repository from disk. This action cannot be undone. + +## API Reference + +### Endpoints + +- `GET /projects/{id}/repositories` - List repositories +- `POST /projects/{id}/repositories` - Create repository +- `DELETE /projects/{id}/repositories/{id}` - Delete repository +- `POST /projects/{id}/repositories/parse-url` - Parse and validate URL + +See [Repositories API](../api/repositories.md) for detailed endpoint documentation. + +## Related Features + +- [Repository Workspace](workspace.md) - Browse repository files +- [Git History](git-history.md) - View commit history +- [Smart Git URL Parsing](repositories.md#smart-url-parsing) - Automatic URL correction diff --git a/docs/features/settings.md b/docs/features/settings.md new file mode 100644 index 0000000..5f10aa0 --- /dev/null +++ b/docs/features/settings.md @@ -0,0 +1,56 @@ +# User Settings + +## Overview + +User settings allow you to customize your Headquarter experience, including theme preferences and git identity. + +## How to Use + +### Accessing Settings + +1. Click your **name** in the header +2. Select **"Settings"** from the dropdown + +### Theme Selection + +Choose your preferred theme: + +- **System** - Follows your operating system preference +- **Light** - Light color scheme +- **Dark** - Dark color scheme + +Changes are applied immediately and persist across sessions. + +### Git Identity + +Configure your git identity for commits made through the workspace: + +- **Name**: Your display name for git commits +- **Email**: Your email for git commits + +This information is used when you make quick edits in the repository workspace. + +### Default Editor + +Choose your preferred editor for quick edits: + +- Options depend on available tool types +- Used when opening files for editing + +## Configuration + +Settings are stored per-user in the database and persist across sessions. + +## API Reference + +### Endpoints + +- `GET /users/me/config` - Get user settings +- `PATCH /users/me/config` - Update user settings + +See [Users API](../api/users.md) for detailed endpoint documentation. + +## Related Features + +- [Authentication](auth.md) - User authentication +- [Repository Workspace](workspace.md) - Edit files with git identity diff --git a/docs/features/ssh-keys.md b/docs/features/ssh-keys.md new file mode 100644 index 0000000..370161b --- /dev/null +++ b/docs/features/ssh-keys.md @@ -0,0 +1,70 @@ +# SSH Key Management + +## Overview + +Manage SSH key pairs for authenticating with git remotes. Keys are generated and stored securely, with the private key encrypted. + +## How to Use + +### Generating a Key Pair + +1. Navigate to **Settings** → **SSH Keys** +2. Click **"Generate New Key"** +3. Enter a **name** for the key (e.g., "GitHub Work Account") +4. Click **"Generate"** + +The system will: +- Generate an Ed25519 key pair +- Encrypt the private key +- Store both keys securely +- Display the public key + +### Copying Public Key + +1. Find the key in the list +2. Click the **"Copy"** button next to the public key +3. Paste into your git host (GitHub, GitLab, etc.) + +### Adding to Git Hosts + +#### GitHub +1. Go to Settings → SSH and GPG keys +2. Click "New SSH key" +3. Paste the public key +4. Give it a title +5. Click "Add SSH key" + +#### GitLab +1. Go to Preferences → SSH Keys +2. Paste the public key +3. Set expiration (optional) +4. Click "Add key" + +### Deleting Keys + +1. Find the key in the list +2. Click the **"Delete"** button +3. Confirm deletion + +**Note:** Deleting a key removes both public and private keys. This cannot be undone. + +## Security + +- **Ed25519 algorithm**: Modern, secure key type +- **Encrypted storage**: Private keys are encrypted at rest +- **No export**: Private keys cannot be exported +- **One-way generation**: Keys are generated server-side, never transmitted + +## API Reference + +### Endpoints + +- `GET /ssh-keys` - List SSH keys +- `POST /ssh-keys` - Generate new key +- `DELETE /ssh-keys/{id}` - Delete key + +See [SSH Keys API](../api/ssh-keys.md) for detailed endpoint documentation. + +## Related Features + +- [Git Repositories](repositories.md) - Use SSH keys for repository access diff --git a/docs/features/tool-types.md b/docs/features/tool-types.md new file mode 100644 index 0000000..27b238e --- /dev/null +++ b/docs/features/tool-types.md @@ -0,0 +1,104 @@ +# Tool Types + +## Overview + +Tool types define development tools that can be spawned for projects. Headquarter includes built-in types and supports creating custom tool types with Docker Compose templates. + +## How to Use + +### Built-in Tool Types + +Headquarter includes these built-in tool types: + +#### VS Code Server +- **Image**: `lscr.io/linuxserver/code-server:latest` +- **Purpose**: Full VS Code in the browser +- **Features**: Extensions, terminal, debugging +- **Access**: Port 8443 + +#### Jupyter Notebook +- **Image**: `jupyter/scipy-notebook:latest` +- **Purpose**: Interactive Python development +- **Features**: Notebooks, data visualization +- **Access**: Port 8888 + +### Managing Tool Types + +#### Viewing Tool Types + +1. Navigate to **Settings** → **Tool Types** +2. See a list of all tool types +3. Built-in types are marked with a badge + +#### Creating Custom Tool Types + +1. Click **"New Tool Type"** +2. Fill in the form: + - **Name**: Unique identifier (e.g., `my-custom-tool`) + - **Display Name**: Human-readable name + - **Description**: What this tool does + - **Compose Template**: Docker Compose YAML +3. Click **"Create"** + +#### Compose Template Format + +The compose template uses Docker Compose syntax with template variables: + +```yaml +version: "3.8" +services: + my-tool: + image: my-image:latest + container_name: {{TOOL_NAME}} + environment: + - VARIABLE=value + volumes: + - {{REPO_PATH}}:/workspace + ports: + - "8080:8080" +``` + +**Required Variables:** +- `{{TOOL_NAME}}` - Unique name for the container +- `{{REPO_PATH}}` - Path to the repository + +#### Validating Templates + +The system validates templates: +- Must be valid YAML +- Must contain a `services` section +- Must use all required variables +- Invalid templates will be rejected + +#### Editing Tool Types + +1. Find the tool type in the list +2. Click **"Edit"** +3. Update fields +4. Click **"Save"** + +**Note:** Built-in tool types cannot be modified or deleted. + +#### Deleting Tool Types + +1. Find the tool type in the list +2. Click **"Delete"** +3. Confirm deletion + +**Note:** Built-in tool types cannot be deleted. + +## API Reference + +### Endpoints + +- `GET /tool-types` - List tool types +- `POST /tool-types` - Create tool type +- `GET /tool-types/{id}` - Get tool type details +- `PUT /tool-types/{id}` - Update tool type +- `DELETE /tool-types/{id}` - Delete tool type + +See [Tool Types API](../api/tool-types.md) for detailed endpoint documentation. + +## Related Features + +- [Tool Instances](tool-instances.md) - Spawn and manage tool instances diff --git a/docs/features/workspace.md b/docs/features/workspace.md new file mode 100644 index 0000000..0e8d3a8 --- /dev/null +++ b/docs/features/workspace.md @@ -0,0 +1,101 @@ +# Repository Workspace + +## Overview + +The Repository Workspace is the default view when you open a project. It provides a file browser and viewer for exploring repository contents, similar to GitHub or GitLab's file browser. + +## How to Use + +### Workspace Layout + +``` +┌─────────────────────────────────────────────┐ +│ [Repo Selector ▼] [Branch Selector ▼] │ +├─────────────────┬───────────────────────────┤ +│ File Tree │ Main Content │ +│ │ │ +│ 📁 src/ │ Breadcrumbs: src > main │ +│ 📁 tests/ │ │ +│ 📄 README.md │ [Edit] [History] │ +│ 📄 .gitignore │ │ +│ │ File content here... │ +│ │ │ +└─────────────────┴───────────────────────────┘ +``` + +### Repository Selection + +If a project has multiple repositories: +1. Use the **Repository Selector** dropdown at the top +2. Choose the repository you want to browse +3. The file tree updates automatically + +### Branch Selection + +1. Use the **Branch Selector** dropdown +2. Select a branch from the list +3. The file tree refreshes to show that branch's contents + +### Browsing Files + +**Navigate directories:** +- Click on a folder (📁) to expand it +- Click again to collapse +- The file tree shows the full directory structure + +**View file contents:** +- Click on a file (📄) to open it +- The file viewer shows: + - File path breadcrumbs + - File content with syntax highlighting + - File metadata (size, last commit) + +### File Viewer + +The file viewer supports: +- **Syntax highlighting** for common languages +- **Line numbers** +- **Breadcrumb navigation** (click any path segment) + +### Quick Editing + +For small changes: + +1. Open a file in the viewer +2. Click the **"Edit"** button +3. Make your changes in the text area +4. Enter a **commit message** +5. Click **"Save"** + +The system will: +- Commit the changes to the current branch +- Show the new commit hash +- Refresh the file view + +**Note:** This creates a real git commit. Make sure your commit message describes the change. + +### Binary Files + +Binary files (images, compiled code, etc.) cannot be viewed or edited in the workspace. The viewer will show "Binary file - cannot display." + +## Keyboard Navigation + +- **Click** folder to expand/collapse +- **Click** file to view +- **Click** breadcrumb to navigate up + +## API Reference + +### Endpoints + +- `GET /projects/{id}/repositories/{id}/files` - List files in directory +- `GET /projects/{id}/repositories/{id}/files/content` - Get file content +- `POST /projects/{id}/repositories/{id}/files/content` - Update file +- `GET /projects/{id}/repositories/{id}/branches` - List branches + +See [Repositories API](../api/repositories.md) for detailed endpoint documentation. + +## Related Features + +- [Git Repositories](repositories.md) - Manage repositories +- [Git History](git-history.md) - View commit history diff --git a/docs/templates/api-endpoint.md b/docs/templates/api-endpoint.md new file mode 100644 index 0000000..c738077 --- /dev/null +++ b/docs/templates/api-endpoint.md @@ -0,0 +1,97 @@ +# API Endpoint Documentation Template + +Use this template when documenting API endpoints. + +## Endpoint Group Name + +Brief description of what this group of endpoints does. + +### Authentication + +Describe authentication requirements (e.g., "Requires valid session cookie"). + +--- + +## METHOD /path/to/endpoint + +**Description:** What this endpoint does. + +### Request + +#### Headers + +| Header | Required | Description | +|--------|----------|-------------| +| `Authorization` | Yes/No | Description | +| `Content-Type` | Yes/No | Description | + +#### Query Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `param_name` | `string` | Yes/No | Description | + +#### Request Body + +```json +{ + "field_name": "string", + "field_name": "number" +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `field_name` | `string` | Yes/No | Description | + +### Response + +#### Success (200 OK) + +```json +{ + "field_name": "value", + "field_name": "value" +} +``` + +#### Error Responses + +**400 Bad Request** +```json +{ + "detail": "Error message" +} +``` + +**401 Unauthorized** +```json +{ + "detail": "Authentication required" +} +``` + +### Example + +#### Request +```bash +curl -X METHOD http://api.example.com/path/to/endpoint \ + -H "Content-Type: application/json" \ + -H "Cookie: session=your_session_cookie" \ + -d '{ + "field_name": "value" + }' +``` + +#### Response +```json +{ + "field_name": "value" +} +``` + +--- + +## METHOD /path/to/another/endpoint + +[Repeat the same structure for each endpoint] diff --git a/docs/templates/architecture.md b/docs/templates/architecture.md new file mode 100644 index 0000000..4c82313 --- /dev/null +++ b/docs/templates/architecture.md @@ -0,0 +1,69 @@ +# Architecture Documentation Template + +Use this template when documenting system architecture. + +## System/Component Name + +## Overview + +Provide a high-level description of this system or component. + +## Architecture Diagram + +``` +[ASCII or Mermaid diagram showing components and relationships] +``` + +## Components + +### Component 1 + +**Purpose:** What this component does + +**Responsibilities:** +- Responsibility 1 +- Responsibility 2 + +**Dependencies:** +- Dependency 1 +- Dependency 2 + +**Key Files:** +- `path/to/file1.py` +- `path/to/file2.py` + +### Component 2 + +[Repeat for each component] + +## Data Flow + +Describe how data flows through the system. + +``` +[Step 1] → [Step 2] → [Step 3] +``` + +## Technology Stack + +| Component | Technology | Purpose | +|-----------|-----------|---------| +| Component 1 | Technology 1 | Purpose | +| Component 2 | Technology 2 | Purpose | + +## Configuration + +Describe relevant configuration options. + +## Security Considerations + +List security considerations for this architecture. + +## Scaling Considerations + +Describe how this scales (or doesn't). + +## Related Documentation + +- [Link to related doc 1](link.md) +- [Link to related doc 2](link.md) diff --git a/docs/templates/feature-doc.md b/docs/templates/feature-doc.md new file mode 100644 index 0000000..a92c663 --- /dev/null +++ b/docs/templates/feature-doc.md @@ -0,0 +1,65 @@ +# Feature Documentation Template + +Use this template when documenting a new feature. + +## Feature Name + +## Overview + +Provide a 1-2 sentence description of what this feature does and why it exists. + +## How to Use + +### Prerequisites + +List any prerequisites (e.g., must be logged in, must have project created). + +### Step-by-Step Guide + +1. **Step 1**: Description of first step + - Details + - Screenshots if applicable + +2. **Step 2**: Description of second step + - Details + +3. **Step 3**: Description of third step + - Details + +## Screenshots + +Include screenshots or diagrams if they help understanding. + +## API Reference + +List related API endpoints with links to API docs. + +### Endpoints + +- `METHOD /endpoint/path` - Brief description +- `METHOD /endpoint/path` - Brief description + +## Configuration + +List any relevant configuration options (environment variables, settings). + +| Option | Description | Default | +|--------|-------------|---------| +| `VAR_NAME` | What this controls | `default_value` | + +## Troubleshooting + +### Common Issues + +**Issue**: Description of problem +**Solution**: How to fix it + +## Related Features + +- [Feature Name](link.md) - How this relates +- [Feature Name](link.md) - How this relates + +## See Also + +- [API Documentation](../api/feature.md) +- [Architecture Documentation](../architecture/feature.md)