Implement layered maps and context retrieval

2026-06-11 12:56:18 +02:00
parent 010e4b83eb
commit c6064f8d94
35 changed files with 4410 additions and 383 deletions
@@ -0,0 +1,96 @@
+# Proposal: Layered Map Protocol
+
+## Status
+
+| Field | Value |
+|---|---|
+| Phase | **Proposal** |
+| Based on | User interview + current design doc |
+| Next | Spec |
+
+## Problem
+
+The current design centers consumption on discovering all `.pi-map.md` files and loading them at session start, with pruning for larger repositories. That scales poorly and frames maps as bulk context instead of as a navigation system.
+
+The repo currently lacks:
+
+1. a clear root routing artifact,
+2. an explicit trust/use protocol,
+3. a layered loading model,
+4. fast per-directory navigation artifacts,
+5. a clean split between routing metadata and richer orientation metadata.
+
+## Proposed change
+
+Adopt a layered navigation model built around a **paired artifact** for every non-ignored directory:
+
+- `.pi-map.index.md` → fast routing
+- `.pi-map.md` → richer orientation
+
+### Tier model
+- **Tier 0 — always injected:** Project Map Protocol + root `.pi-map.index.md`
+- **Tier 1 — task-start load:** likely relevant directory indexes first, then rich maps for strongest matches
+- **Tier 2 — exact verification:** source files, tests, configs, and docs
+
+### Paired artifacts per directory
+Every non-ignored directory should generate both:
+- a rich `.pi-map.md`
+- a lightweight `.pi-map.index.md`
+
+The index is for movement. The map is for understanding. Source remains the final authority.
+
+This change stays self-contained: it does not add a vector store, Engram dependency, or other external memory/retrieval backend.
+
+### Operating model
+- Root `.pi-map.md` remains rich, but points to root `.pi-map.index.md` for traversal.
+- Each index points to its sibling rich map.
+- Each rich map points back to its sibling index.
+- If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
+
+## In scope
+
+- [ ] Generate `.pi-map.index.md` for every non-ignored directory
+- [ ] Keep `.pi-map.md` as the richer sibling artifact for every non-ignored directory
+- [ ] Emit a Project Map Protocol in docs/prompt guidance and in generated root artifacts
+- [ ] Replace eager "read all maps" guidance with layered retrieval guidance
+- [ ] Define fixed index/map responsibilities and section order
+- [ ] Cascade freshness and repair logic across changed directory pairs and affected ancestors
+- [ ] Add strict validation for missing/stale/broken index-map pairs
+- [ ] Update docs and skill guidance to reflect the new model
+
+## Out of scope
+
+- [ ] Add a query-driven retrieval command such as `project-map context <query>`
+- [ ] Preserve backward compatibility for agents that only understand the old preload-only model
+- [ ] Replace source verification with map-based authority
+- [ ] Redesign the core AST/LLM extraction pipeline beyond what is needed to feed the new artifacts
+
+## Decisions from grilling
+
+| Topic | Decision |
+|---|---|
+| Change split | Two specs |
+| This change slug | `layered-map-protocol` |
+| Follow-up change slug | `map-context-retrieval` |
+| Backward compatibility required | No |
+| Artifact model | Every non-ignored directory gets both `.pi-map.md` and `.pi-map.index.md` |
+| Root Tier 0 | Project Map Protocol + root `.pi-map.index.md` |
+| Root rich map | Auto-read for architecture/system questions and ambiguous tasks |
+| Root relationship | Root map points to root index |
+| Retrieval mode | Hybrid auto-load, indexes first |
+| Patch strategy | Changed dir updates both; ancestors refresh by size/structure rules |
+| Validation | Hard-fail on missing/stale indexes, repair via `validate --fix` |
+| Prompt trust boundary | index routes, map orients, source decides |
+| Extra retrieval backends | None in this change |
+| SDD mode | Interactive |
+| Artifact store | OpenSpec |
+| PR strategy | Auto-forecast |
+| Review budget | 400 lines |
+
+## Success criteria
+
+- [ ] Every non-ignored directory has both a fast index and a richer map
+- [ ] The root artifacts clearly tell an agent how to navigate and when to read source
+- [ ] The documented consumption model no longer requires reading every `.pi-map.md` up front
+- [ ] Validation fails on stale or missing paired artifacts and offers a repair path
+- [ ] Patch/refresh behavior keeps routing trustworthy without requiring full reinit for small changes