Implement layered maps and context retrieval

2026-06-11 12:56:18 +02:00
parent 010e4b83eb
commit c6064f8d94
35 changed files with 4410 additions and 383 deletions
@@ -0,0 +1,127 @@
+# Spec: Map Context Retrieval
+
+## Status
+
+| Field | Value |
+|---|---|
+| Phase | **Spec** |
+| Based on | [Proposal](proposal.md) |
+| Next | Design |
+
+## Overview
+
+Add a retrieval-oriented `context` command that turns a user task into a compact project-map routing bundle. This change depends on the paired root/per-directory index+map metadata introduced by `layered-map-protocol`.
+
+## Dependency
+
+This change should not begin implementation before `layered-map-protocol` has landed enough metadata to support routing and ranking.
+
+## Functional requirements
+
+### 1. Tool-first surface
+Pi should expose `project-map context` as a first-class tool action so agents can request a context bundle directly.
+
+For v1, the tool input should be minimal:
+- `query: string`
+
+### 2. CLI follow-up surface
+The CLI should support the same conceptual shape:
+
+```bash
+project-map context "<query>"
+```
+
+CLI support may follow after the tool contract is stable.
+The CLI should mirror the same main input name: `query`.
+### 3. Bundle contents
+A context bundle must contain, at minimum:
+- relevant indexes
+- strongest-match relevant maps
+- likely files
+- relevant symbols when useful
+- short next-step instructions
+### 4. Ranking behavior
+The command must rank candidates using generated metadata such as:
+- root workflow hints
+- directory index routing hints
+- parent/child/sibling link structure
+- package tags
+- symbol references
+- task/workflow entries
+- `files:` / `index:` / `map:` targets
+
+Indexes should be the first routing layer. Rich maps and symbols should expand from the strongest routed candidates.
+
+By default, retrieval should usually surface the top 3 candidate directories, with adaptive omission of weaker rich maps when confidence drops.
+### 5. Trust boundary
+The bundle must instruct the agent to:
+- read indexes first for orientation,
+- read rich maps next when deeper context is needed,
+- read source before editing or asserting exact behavior.
+
+## Output shape
+
+The first version should be markdown-first, with stable, schema-like sections optimized for LLM consumption rather than human prose.
+
+Default section order:
+1. `query`
+2. `relevant indexes`
+3. `relevant maps`
+4. `likely files`
+5. `relevant symbols`
+6. `instructions`
+
+The retrieval title format should remain retrieval-specific:
+- `# Context bundle: <query>`
+
+A representative response shape:
+
+```markdown
+# Context bundle: validation stale signatures
+
+## query
+validation stale signatures
+
+## relevant indexes
+- .pi-map.index.md
+- src/.pi-map.index.md
+- tests/.pi-map.index.md
+
+## relevant maps
+- src/.pi-map.md
+- tests/.pi-map.md
+
+## likely files
+- src/validate.ts
+- src/ast/ast-extract.ts
+- tests/validate.test.ts
+
+## relevant symbols
+- validateMaps
+- generateDirectoryMap
+- extractFileAST
+
+## instructions
+Read the indexes first, then the strongest-match rich maps, then verify behavior from source before editing.
+```
+
+The tool should omit weak/unhelpful symbol output rather than forcing a noisy symbol section.
+## Non-functional requirements
+
+- Output must be compact enough for prompt injection.
+- Ranking should be deterministic enough to test.
+- The implementation should prefer lightweight heuristics over heavyweight indexing.
+- Retrieval should reuse the paired-artifact parsing layer instead of ad hoc string slicing.
+- Retrieval should not require a vector store, Engram, or another external memory backend.
+- The first version should accept natural language queries first; structured filters can come later.
+- Candidate explanations are not required by default.
+- Result-count override and root-rich-map inclusion can come later if needed, but are not part of v1.
+## Acceptance criteria
+
+- [ ] Pi exposes `project-map context <query>` as a tool-first surface
+- [ ] CLI support follows the same shape after the tool contract is stable
+- [ ] Retrieval output includes indexes, strongest-match maps, files, and instructions
+- [ ] Symbol hints appear only when paired metadata makes them useful
+- [ ] Ranking respects the index-first routing model and usually returns top 3 candidates
+- [ ] Docs show when to use the retrieval command
+- [ ] Trust-boundary guidance remains explicit