headquarter/openspec/changes/container-monitoring-notifications/apply-progress.md

PR-1 Apply Progress: Backend Core for Container Monitoring & Notifications

TDD Cycle Evidence

EventBus (MON-PR1-005 + MON-PR1-010)

Cycle	Action	Evidence
RED	Wrote test_event_bus.py with imports to non-existent module	pytest collection error: ModuleNotFoundError: No module named 'src.services.event_bus'
GREEN	Implemented event_bus.py with singleton, subscribe, publish, unsubscribe, exception isolation	pytest tests/unit/test_event_bus.py -v → 6 passed
REFACTOR	Replaced asyncio.iscoroutinefunction with inspect.iscoroutinefunction; moved Awaitable/Callable to collections.abc	Tests still pass, ruff clean
TRIANGULATE	Added wildcard ("*") support in publish() for SSE endpoint	Verified by SSE endpoint test logic

HealthMonitor (MON-PR1-006 + MON-PR1-011)

Cycle	Action	Evidence
RED	Wrote test_health_monitor.py with imports to non-existent module	pytest collection error: ModuleNotFoundError: No module named 'src.services.health_monitor'
GREEN	Implemented health_monitor.py with poll loop, state comparison, DB writes, event publication	pytest tests/unit/test_health_monitor.py -v → 6 passed
REFACTOR	Added per-instance exception handling in _check_instance; removed redundant try/except in _run_check_cycle	Tests still pass

Models (MON-PR1-003 + MON-PR1-004)

Cycle	Action	Evidence
RED	Wrote test_monitoring_models.py with imports to non-existent models	pytest collection error: ModuleNotFoundError for models
GREEN	Implemented instance_event.py and health_check.py; exported in models/__init__.py	pytest tests/unit/test_monitoring_models.py -v → 3 passed

Completed Tasks

• MON-PR1-001: Alembic migration 2026_05_28_add_monitoring_tables.py (creates both instance_events and health_checks with all indexes)
• MON-PR1-002: SQLAlchemy models InstanceEvent and HealthCheck
• MON-PR1-003: InstanceEvent model (apps/api/src/models/instance_event.py)
• MON-PR1-004: HealthCheck model (apps/api/src/models/health_check.py)
• MON-PR1-005: InstanceEventBus service (apps/api/src/services/event_bus.py)
• MON-PR1-006: HealthMonitor background task (apps/api/src/services/health_monitor.py)
• MON-PR1-007: SSE endpoint GET /events/stream (apps/api/src/api/events.py)
• MON-PR1-008: Lifecycle hooks in tool_instances.py + lifecycle_hooks.py service
• MON-PR1-009: Structured JSON logging in logging_config.py + CorrelationIdMiddleware
• MON-PR1-010: Unit tests for EventBus (apps/api/tests/unit/test_event_bus.py)
• MON-PR1-011: Unit tests for HealthMonitor (apps/api/tests/unit/test_health_monitor.py)

Files Changed

New Files

• apps/api/alembic/versions/2026_05_28_add_monitoring_tables.py
• apps/api/src/models/instance_event.py
• apps/api/src/models/health_check.py
• apps/api/src/services/event_bus.py
• apps/api/src/services/health_monitor.py
• apps/api/src/services/correlation.py
• apps/api/src/services/lifecycle_hooks.py
• apps/api/src/api/events.py
• apps/api/tests/unit/test_event_bus.py
• apps/api/tests/unit/test_health_monitor.py
• apps/api/tests/unit/test_monitoring_models.py

Modified Files

• apps/api/src/models/__init__.py — export new models
• apps/api/src/api/__init__.py — export events router
• apps/api/src/api/tool_instances.py — lifecycle hook instrumentation
• apps/api/src/logging_config.py — JSON formatter + CorrelationIdFilter
• apps/api/src/main.py — register events router, CorrelationIdMiddleware, HealthMonitor lifespan

Test Evidence

# New unit tests — all pass
$ cd apps/api && python -m pytest tests/unit/test_event_bus.py tests/unit/test_health_monitor.py tests/unit/test_monitoring_models.py -v
15 passed, 4 warnings in 0.98s

# Full unit suite — no regressions (4 pre-existing failures unrelated to this change)
$ cd apps/api && python -m pytest tests/unit/ -v
172 passed, 4 failed, 2 warnings in 6.57s

# Ruff linting on new/modified files
$ cd apps/api && python -m ruff check <new files>
All checks passed!

Deviation from Design

1. Single migration vs. two migrations: Prompt listed MON-PR1-001 and MON-PR1-002 as separate migrations, but design.md specifies a single revision. Implemented as one migration 2026_05_28_add_monitoring_tables.py creating both tables.
2. metadata column name: SQLAlchemy DeclarativeBase reserves metadata as a class attribute. Used event_metadata as the Python attribute name with "metadata" as the DB column name via mapped_column("metadata", ...). The event payload still uses metadata key.
3. Delete audit row: The FK ON DELETE CASCADE on instance_events.instance_id means the audit row for instance.deleted cannot survive deletion. The row is inserted before session.delete(instance) and is cascade-deleted on commit. The event bus publication still occurs.
4. SQLite test compatibility: Used JSON instead of JSONB in the SQLAlchemy model to maintain SQLite test compatibility. The migration uses sa.JSON() which maps appropriately.

Remaining Tasks (for PR-2 / PR-3)

• Frontend useEvents() hook, ToastProvider, toast-rules.ts
• Frontend badge real-time updates + polling removal
• Integration tests for SSE endpoint (MON-PR1-012)
• Integration tests for lifecycle hooks
• E2E tests
• Performance tuning and documentation