Fusion
d5f9df33b7
- Add new status badges: starting, probing, unhealthy - Show tunnel error only when tunnel_status is unreachable - Show app error badge with status code for error_response - Add collapsible probe output section for diagnostics - Update health polling to check all active instances - Only show Recreate Tunnel button for unreachable tunnels
2.5 KiB
2.5 KiB
Why
The current instance management has significant gaps in health monitoring. When starting instances, there's no verification that containers actually boot successfully - failures only surface when users try to access broken tunnels. The existing health check only validates tunnel URLs, not container health, leading to false positives where a "healthy" tunnel serves 502 errors from a crashed tool. Additionally, readiness probes exist as unused infrastructure, and auto-recovery blindly recreates tunnels on any HTTP error including legitimate 502s from the application itself.
What Changes
- Startup health checks: Verify containers reach a running state after
docker compose up, with clear failure messages when containers crash or fail to start - Container health checks: Check container status via Docker API (
docker ps,docker inspect) in addition to tunnel URL checks - Readiness probe integration: Wire the existing
execute_probe()service into the instance startup flow, using tool type configured probes - Smart auto-recovery: Only recreate tunnels when the tunnel endpoint itself is unreachable (connection refused, timeout, DNS failure), NOT when the tool returns 502/503/504 errors
- Instance status granularity: Distinguish between "starting" (container booting), "running" (healthy), "unhealthy" (container up but probe failing), and "error" (failed to start)
Capabilities
New Capabilities
instance-startup-health: Container startup verification and failure detectioninstance-runtime-health: Continuous health monitoring combining container and tunnel checksreadiness-probe-integration: Tool-type configured readiness probes during instance startupsmart-tunnel-recovery: Context-aware tunnel recreation that distinguishes tunnel failures from application errors
Modified Capabilities
session-management-fixes: Update health check endpoint to include container status, modify tunnel health logic to be smarter about error codes
Impact
- Backend:
api/tool_instances.py(start_instance, health check, recreate tunnel),services/docker.py(container status checks),services/readiness_probe.py(integration into startup flow) - Frontend:
pages/sessions.tsx(display new status states, show startup errors, smarter health badges) - Database: No schema changes - uses existing
statusfield with new state values - API: New response fields in health check endpoint (container_status, probe_result, last_probe_at)