headquarter/openspec/changes/archive/2025-05-18-test-infrastructure-improvements/design.md

Context

Current test infrastructure has ~50 backend tests and 12 frontend tests. Backend tests are mixed in a single directory with inconsistent patterns. Some tests require a running PostgreSQL instance even when testing pure logic. There's no E2E test coverage.

Goals / Non-Goals

Goals:

• Separate tests into unit/integration/system categories with clear boundaries.
• Create shared fixtures to eliminate duplication.
• Enable fast unit tests without PostgreSQL (SQLite in-memory).
• Implement proper transaction isolation for integration tests.
• Add E2E tests for critical user journeys (login, project creation).
• Make tests runnable both locally and in Docker.

Non-Goals:

• Rewriting all existing tests (restructure and migrate gradually).
• Adding tests for features that don't exist yet.
• Complex test parallelization setup.
• Performance benchmarking.

Decisions

1. Use SQLite for unit tests, PostgreSQL for integration/system tests

   • Rationale: Unit tests should be fast and not require external services.
   • SQLite runs in-memory, no Docker needed.

2. Use pytest-asyncio with asyncio_mode=auto

   • Rationale: Simplifies test writing, no need for @pytest.mark.asyncio on every test.

3. Shared conftest.py with session-scoped engine

   • Rationale: Eliminates duplicate engine creation across files.
   • Integration tests use begin_nested() for transaction rollback.

4. Directory structure: tests/unit/, tests/integration/, tests/system/

   • Rationale: Clear separation, easy to run selectively.
   • pytest -m unit runs all unit tests regardless of location.

5. Playwright for E2E tests

   • Rationale: Modern, well-supported, works with React/Vite.
   • Tests run against the actual deployed application.

Risks / Trade-offs

• [SQLite vs PostgreSQL behavior differences] -> Document known differences (e.g., JSON operators, asyncpg-specific features).
• [Migration effort for existing tests] -> Migrate gradually, prioritize new tests over rewriting old ones.
• [E2E tests are slower] -> Run them selectively (not on every commit), maybe only in CI or nightly.

Migration Plan

1. Create new directory structure.
2. Implement shared fixtures in conftest.py.
3. Add SQLite support for unit tests.
4. Migrate 2-3 existing tests as examples.
5. Add Playwright setup and 1-2 E2E tests.
6. Update Makefile targets.
7. Document testing strategy.

Open Questions

• Should we use pytest-xdist for parallel test execution?
• Should E2E tests run against the Docker compose setup or a staging environment?