fix: prevent failed containers from showing as running on dashboard
- Add final get_container_status check in start_tool_instance before writing status=running; mark as error and return logs if container stopped - Treat restarting as error in HealthMonitor when DB status was already running, so crash loops are surfaced instead of preserved - Disable auto-restart (restart: unless-stopped -> restart: no) for tool instances in manifest compiler, legacy dockerfile path, and built-in seeds Quality gates: - pytest tests/unit: 210 passed - ruff: clean on changed files - mypy: clean on changed files
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
dir: openspec
|
||||
|
||||
## role
|
||||
Defines the OpenSpec methodology and configuration for managing software requirements, specifications, and task tracking as living documentation within a project repository.
|
||||
Defines the OpenSpec methodology and configuration for managing software requirements, specifications, and task tracking as living documentation within a Docker-based coding agent management platform.
|
||||
## parent
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
dir: openspec/changes
|
||||
|
||||
## role
|
||||
Manages and tracks specification changes, diffs, and versioning history for OpenAPI documents.
|
||||
Provides change-tracking and diffing utilities for OpenAPI specification evolution and version comparison.
|
||||
## parent
|
||||
index: openspec/.pi-map.index.md
|
||||
map: openspec/.pi-map.md
|
||||
@@ -10,6 +10,9 @@ map: openspec/.pi-map.md
|
||||
- openspec/changes/archive
|
||||
index: openspec/changes/archive/.pi-map.index.md
|
||||
map: openspec/changes/archive/.pi-map.md
|
||||
- openspec/changes/fix-container-status-false-positive
|
||||
index: openspec/changes/fix-container-status-false-positive/.pi-map.index.md
|
||||
map: openspec/changes/fix-container-status-false-positive/.pi-map.md
|
||||
- openspec/changes/fix-pi-container-mount-permissions
|
||||
index: openspec/changes/fix-pi-container-mount-permissions/.pi-map.index.md
|
||||
map: openspec/changes/fix-pi-container-mount-permissions/.pi-map.md
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# openspec/changes/fix-container-status-false-positive (index)
|
||||
dir: openspec/changes/fix-container-status-false-positive
|
||||
|
||||
## role
|
||||
Documents a bug fix for correcting container status false positives where failed containers incorrectly appeared as running in a dashboard/monitoring system.
|
||||
## parent
|
||||
index: openspec/changes/.pi-map.index.md
|
||||
map: openspec/changes/.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- change.md
|
||||
- tasks.md
|
||||
## links
|
||||
index: openspec/changes/fix-container-status-false-positive/.pi-map.index.md
|
||||
map: openspec/changes/fix-container-status-false-positive/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,20 @@
|
||||
# openspec/changes/fix-container-status-false-positive
|
||||
dir: openspec/changes/fix-container-status-false-positive
|
||||
|
||||
index: openspec/changes/fix-container-status-false-positive/.pi-map.index.md
|
||||
|
||||
## role
|
||||
Documents a bug fix for correcting container status false positives where failed containers incorrectly appeared as running in a dashboard/monitoring system.
|
||||
## files
|
||||
- change.md | Documents a bug fix for failed containers incorrectly displaying as running on a dashboard, detailing root cause analysis and the fix implementation across multiple files.
|
||||
- tasks.md | Track progress of fixing a false-positive running status bug in a container/system management system
|
||||
## arch
|
||||
Issue-tracking documentation pattern using markdown files (change.md for technical specification, tasks.md for progress tracking) to manage a bug fix across multiple system components.
|
||||
## tags
|
||||
bug, fix, running, change, system, tasks, documents, failed
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
@@ -0,0 +1,32 @@
|
||||
# Fix: failed containers shown as running on dashboard
|
||||
|
||||
## Problem
|
||||
|
||||
Containers that fail during startup (e.g. `ln: failed to create symbolic link '/workspace': Permission denied`) are still displayed as "Running" on the web dashboard and session list.
|
||||
|
||||
## Root cause
|
||||
|
||||
1. `start_tool_instance` in `apps/api/src/services/tool/instance_service.py` sets `instance.status = "running"` after post-start setup without verifying the container is still up.
|
||||
2. Generated compose files use `restart: unless-stopped`, so Docker immediately restarts a crashed container, putting it into the `restarting` state.
|
||||
3. The background `HealthMonitor` treats `restarting` as a transient state and preserves the current DB status (`running`).
|
||||
|
||||
## Fix
|
||||
|
||||
1. Add a final `get_container_status` check in `start_tool_instance` immediately before writing `status = "running"`. If the container has stopped/exited, mark it as `error` and return the logs.
|
||||
2. Update `HealthMonitor._derive_status` so that when the DB status is `running` and Docker reports `restarting`, the instance is marked as `error`.
|
||||
3. Disable auto-restart for tool instances by changing `restart: unless-stopped` to `restart: "no"` in:
|
||||
- `apps/api/src/services/build/manifest_compiler.py`
|
||||
- `apps/api/src/services/tool/instance_service.py` (legacy dockerfile path)
|
||||
- `apps/api/src/seeds/builtin_tool_types.py`
|
||||
|
||||
## Affected files
|
||||
|
||||
- `apps/api/src/services/tool/instance_service.py`
|
||||
- `apps/api/src/services/instance/health_monitor.py`
|
||||
- `apps/api/src/services/build/manifest_compiler.py`
|
||||
- `apps/api/src/seeds/builtin_tool_types.py`
|
||||
|
||||
## Verification
|
||||
|
||||
- `pytest apps/api/tests/unit`
|
||||
- `ruff`, `mypy` on changed files
|
||||
@@ -0,0 +1,8 @@
|
||||
# Tasks: fix false-positive running status
|
||||
|
||||
- [x] Investigate status flow and identify root cause
|
||||
- [x] Add final container status check before writing "running"
|
||||
- [x] Treat restarting as error when DB status is already running
|
||||
- [x] Disable auto-restart in manifest compiler, legacy path, and seed data
|
||||
- [ ] Run quality gates
|
||||
- [ ] Commit and push
|
||||
Reference in New Issue
Block a user