## ADDED Requirements ### Requirement: Container startup verification The system SHALL verify that containers reach a running state before marking instances as "running". #### Scenario: Container starts successfully - **WHEN** `docker compose up` completes - **THEN** the system polls `docker ps` every 2 seconds for up to 30 seconds - **AND** when the container state is "running", the instance status becomes "starting" - **AND** the readiness probe begins execution #### Scenario: Container fails to start - **WHEN** `docker compose up` completes - **AND** the container exits within 30 seconds - **THEN** the instance status becomes "error" - **AND** the container exit code is stored in the error message #### Scenario: Container stays in restarting loop - **WHEN** `docker compose up` completes - **AND** the container remains in "restarting" state after 30 seconds - **THEN** the instance status becomes "error" - **AND** the error message indicates the container is stuck restarting ### Requirement: Readiness probe execution The system SHALL execute readiness probes for web-enabled tool instances before marking them as "running". #### Scenario: Probe succeeds - **GIVEN** a tool instance with status "starting" - **AND** the tool type has a readiness probe configured - **WHEN** the probe command returns exit code 0 within the timeout - **THEN** the instance status becomes "running" - **AND** the tunnel is created (for web tools) #### Scenario: Probe times out - **GIVEN** a tool instance with status "starting" - **AND** the tool type has a readiness probe configured - **WHEN** the probe does not succeed within the configured timeout (default 30s) - **THEN** the instance status becomes "unhealthy" - **AND** the tunnel is still created (the container is running) - **AND** the last probe output is stored for diagnostics #### Scenario: Terminal tool skips probe - **GIVEN** a tool instance for a terminal-only tool type - **WHEN** the container reaches "running" state - **THEN** the instance status immediately becomes "running" - **AND** no readiness probe is executed ### Requirement: Container health monitoring The system SHALL check container health in addition to tunnel health. #### Scenario: Container is healthy - **GIVEN** a running instance - **WHEN** the health endpoint is queried - **THEN** the response includes `container_status: "running"` - **AND** the response includes `container_health: "healthy"` if Docker healthcheck exists #### Scenario: Container has crashed - **GIVEN** a running instance - **WHEN** the container exits or is stopped externally - **AND** the health endpoint is queried - **THEN** the response includes `container_status: "exited"` - **AND** the response includes `healthy: false` - **AND** the instance status in the database is updated to "error" ## MODIFIED Requirements ### Requirement: Status Monitoring The system SHALL track tool status with startup and health states. #### Scenario: Status check with health details - **GIVEN** a tool instance - **WHEN** status is queried - **THEN** the real-time container status is returned: - `pending`: Instance created, container not yet started - `starting`: Container is running, readiness probe in progress - `running`: Container is running and probe passed (or terminal tool) - `unhealthy`: Container is running but probe failed/timed out - `stopped`: Container was stopped by user - `error`: Container failed to start or crashed ## REMOVED Requirements None.