merge: align dev branch with main
This commit is contained in:
Developer
@@ -0,0 +1,142 @@
|
||||
## Context
|
||||
|
||||
Currently, tool instances are exposed via an API proxy endpoint that forwards requests from `/instances/{id}/proxy/` to the internal Docker container. This creates latency, adds load to the API service, and doesn't support WebSocket features well. Cloudflare Tunnel offers a better architecture where each instance gets its own HTTPS subdomain.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Each running tool instance gets a unique public HTTPS subdomain
|
||||
- No manual DNS or reverse proxy configuration per instance
|
||||
- Automatic cleanup when instances are stopped or deleted
|
||||
- Support for WebSocket and real-time features (code-server terminal, jupyter kernels)
|
||||
- Minimal latency compared to API proxy approach
|
||||
|
||||
**Non-Goals:**
|
||||
- Custom domains per instance (use Cloudflare zone's wildcard)
|
||||
- Advanced tunnel features (load balancing, failover, ingress rules)
|
||||
- Replacing Traefik for the main app (API + frontend)
|
||||
- Supporting non-HTTP protocols (TCP/UDP raw tunneling)
|
||||
|
||||
## Decisions
|
||||
|
||||
### Cloudflare API vs cloudflared CLI
|
||||
|
||||
**Decision:** Use the Cloudflare REST API to create/manage tunnels, not the `cloudflared` CLI.
|
||||
|
||||
**Rationale:**
|
||||
- The API gives us programmatic control without parsing CLI output
|
||||
- We can use `httpx` (already a dependency) instead of subprocess calls
|
||||
- Easier to test and mock
|
||||
|
||||
**Alternative considered:** Running `cloudflared tunnel create` via subprocess
|
||||
- Rejected: Fragile, harder to test, requires cloudflared binary in API container
|
||||
|
||||
### Architecture: cloudflared as a separate container
|
||||
|
||||
**Decision:** Run `cloudflared` as a standalone Docker service that connects to Cloudflare and routes traffic.
|
||||
|
||||
**Rationale:**
|
||||
- Separation of concerns: API manages tunnels, cloudflared handles connectivity
|
||||
- The cloudflared container can access the Docker internal network where instances run
|
||||
- Easier to scale/restart independently
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Cloudflare Edge │
|
||||
└──────────────────────┬──────────────────────────────────────┘
|
||||
│ HTTPS
|
||||
┌──────────────────────▼──────────────────────────────────────┐
|
||||
│ cloudflared container │
|
||||
│ (connects to Cloudflare, receives traffic for *.zone) │
|
||||
└──────────┬──────────────────────────────────────────────────┘
|
||||
│ Docker network
|
||||
┌──────────▼──────────────────────────────────────────────────┐
|
||||
│ code-server container:8443 jupyter container:8888 │
|
||||
│ (tool instances on Docker network with DNS names) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Subdomain naming
|
||||
|
||||
**Decision:** Use `instance-{short-uuid}.{zone}` format (e.g., `instance-a1b2c3d4.headquarter.commumedia.org`)
|
||||
|
||||
**Rationale:**
|
||||
- Predictable and URL-safe
|
||||
- Short enough to be readable
|
||||
- UUID ensures uniqueness without exposing internal IDs
|
||||
|
||||
### Tunnel lifecycle
|
||||
|
||||
**Decision:** Create tunnel on instance start, delete on instance stop/delete.
|
||||
|
||||
**Flow:**
|
||||
1. User clicks "Start"
|
||||
2. Backend creates Cloudflare tunnel via API
|
||||
3. Backend creates DNS CNAME record: `instance-abc123` → `{tunnel-id}.cfargotunnel.com`
|
||||
4. Backend stores `tunnel_id` and `public_url` in ToolInstance
|
||||
5. cloudflared container routes traffic to container:port
|
||||
6. On stop: delete DNS record, delete tunnel
|
||||
|
||||
### cloudflared configuration
|
||||
|
||||
**Decision:** Use a single cloudflared container with dynamic config file updates.
|
||||
|
||||
**Approach:**
|
||||
- The cloudflared container reads an `config.yml` file mounted as a volume
|
||||
- The API writes ingress rules to this file when instances start/stop
|
||||
- cloudflared automatically reloads the config (or we restart the container)
|
||||
|
||||
```yaml
|
||||
# /etc/cloudflared/config.yml
|
||||
tunnel: {tunnel-token}
|
||||
credentials-file: /etc/cloudflared/credentials.json
|
||||
ingress:
|
||||
- hostname: instance-abc123.headquarter.commumedia.org
|
||||
service: http://code-server-repo-abc123:8443
|
||||
- hostname: instance-xyz789.headquarter.commumedia.org
|
||||
service: http://jupyter-repo-def:8888
|
||||
- service: http_status:404
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
**Decision:** Cloudflare tunnels provide HTTPS but do NOT handle app-level auth. Tool instances without built-in auth (like code-server) will be publicly accessible.
|
||||
|
||||
**Rationale:**
|
||||
- Cloudflare Access could add auth, but adds complexity
|
||||
- Many tools (code-server) have their own password/auth mechanisms
|
||||
- Users should configure tool-level auth via ToolConfig
|
||||
|
||||
**Mitigation:** Document that users must configure tool passwords via ToolConfig (e.g., `PASSWORD` env for code-server).
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[Risk]** Cloudflare API rate limits (1200 requests/5 min)
|
||||
→ **Mitigation:** Tunnel creation is infrequent (user-initiated), unlikely to hit limits
|
||||
|
||||
**[Risk]** cloudflared container becomes a single point of failure
|
||||
→ **Mitigation:** It's stateless; can be restarted quickly. All instances share one cloudflared.
|
||||
|
||||
**[Risk]** Subdomain enumeration exposes running instances
|
||||
→ **Mitigation:** UUID-based names are hard to guess. Consider adding Cloudflare Access in future.
|
||||
|
||||
**[Risk]** cloudflared config file updates require container restart
|
||||
→ **Mitigation:** Investigate `cloudflared --no-autoupdate` with config watch, or accept brief restart
|
||||
|
||||
**[Risk]** Tool instances publicly accessible without auth
|
||||
→ **Mitigation:** Document security best practices, recommend setting tool passwords
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Deploy cloudflared container with base config
|
||||
2. Add Cloudflare env vars to API container
|
||||
3. Deploy backend changes (tunnel service, updated lifecycle)
|
||||
4. Deploy frontend changes (use public_url instead of proxy)
|
||||
5. Test with code-server instance
|
||||
6. Remove old proxy endpoint code
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should we add Cloudflare Access (Zero Trust) to protect instances?
|
||||
- Do we need to support custom subdomains (e.g., `myproject.headquarter.commumedia.org`)?
|
||||
- Should we keep the proxy endpoint as a fallback?
|
||||
Reference in New Issue
Block a user