# Capability: Readiness Probes

## Overview

Readiness probes ensure tool instances are fully initialized before being marked as "running". They execute a configurable command inside the container and wait for it to succeed, with configurable timeout and retry interval.

## Functional Requirements

### FR-1: Probe Definition
- Tool types can define an optional readiness probe
- Probe configuration: command, timeout, interval, retries
- If no probe is defined, instance is marked running immediately after container start
- Probe can be any shell command that returns exit code 0 for success

### FR-2: Probe Execution
- Probe runs inside the container via `docker exec`
- Probe starts after container is in "running" state
- Probe executes periodically (interval) until success or timeout
- Each execution has a separate timeout (not the total timeout)
- Probe output is captured and stored

### FR-3: Status Management
- While probing: instance status is "starting"
- On success: instance status changes to "running"
- On timeout: instance status changes to "failed"
- Failed instances include probe logs in error details
- Users can view probe execution history

### FR-4: Probe Types
Support common probe patterns:
- **HTTP probe**: `curl -f http://localhost:8080/health`
- **Command probe**: `opencode --version`
- **File probe**: `[ -f /app/ready ]`
- **Port probe**: `nc -z localhost 8080`

## Configuration

```python
class ReadinessProbe(BaseModel):
    command: str           # Command to execute
    timeout: int = 30      # Total timeout in seconds
    interval: int = 2      # Seconds between checks
    
    @property
    def max_retries(self) -> int:
        return self.timeout // self.interval
```

## Execution Flow

```
Container Start
       │
       ▼
Container Running?
       │
       ├── No ──▶ Wait 1s ──▶ Retry (max 30s)
       │
       ▼ Yes
Execute Probe Command
       │
       ├── Exit 0 ──▶ Status: "running" ✓
       │
       ├── Exit !=0 ──▶ Wait interval ──▶ Retry
       │                              │
       │                              └── Max retries reached?
       │                                     ├── No ──▶ Execute again
       │                                     │
       │                                     ▼ Yes
       │                              Status: "failed" ✗
       │                              Store logs
       │
       └── Timeout ──▶ Status: "failed" ✗
                        Store logs
```

## Probe Examples

### Web Tool (VS Code Server)
```json
{
  "command": "curl -sf http://localhost:8080/health || curl -sf http://localhost:8080",
  "timeout": 60,
  "interval": 3
}
```

### Terminal Tool (OpenCode)
```json
{
  "command": "which opencode && opencode --version",
  "timeout": 30,
  "interval": 2
}
```

### Database Tool
```json
{
  "command": "pg_isready -U postgres",
  "timeout": 30,
  "interval": 2
}
```

## Error Handling

### Probe Command Not Found
- Exit code: 127
- Behavior: Retry (command might not be in PATH yet)
- Log: "Command not found, retrying..."

### Probe Times Out
- Mark instance as "failed"
- Store last probe output
- Include timeout details in error message
- Allow user to view full probe logs

### Container Exits During Probe
- Stop probing immediately
- Mark instance as "failed"
- Include container exit code and logs

## API Integration

### Tool Type Response
```json
{
  "id": "...",
  "name": "code-server",
  "readiness_probe": {
    "command": "curl -sf http://localhost:8080",
    "timeout": 60,
    "interval": 3
  }
}
```

### Instance Response (Failed Probe)
```json
{
  "id": "...",
  "status": "failed",
  "error": "Readiness probe failed after 60s",
  "probe_logs": [
    "Attempt 1/20: Connection refused",
    "Attempt 2/20: Connection refused",
    "...",
    "Attempt 20/20: Timeout"
  ]
}
```

## Acceptance Criteria

- [ ] Probe executes inside container and waits for success
- [ ] Successful probe marks instance as "running"
- [ ] Failed probe (timeout) marks instance as "failed"
- [ ] Probe logs are stored and retrievable
- [ ] Probe respects timeout and interval settings
- [ ] No probe defined = immediate "running" status
- [ ] Container exit during probe is handled gracefully
- [ ] Common probe patterns work (HTTP, command, file, port)