alex/pi-map
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement layered maps and context retrieval

Browse Source
This commit is contained in:
Alex
2026-06-11 12:56:18 +02:00
parent 010e4b83eb
commit c6064f8d94
35 changed files with 4410 additions and 383 deletions
Download Patch File Download Diff File Expand all files Collapse all files
openspec/changes/layered-map-protocol/design.md
+200
View File
@@ -0,0 +1,200 @@
# Design: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Design** |
| Based on | [Spec](spec.md) |
| Next | Tasks |
## Design summary
This change introduces a paired, navigation-first artifact model without changing the core mission of `pi-project-map`. The implementation should keep the existing discover → analyze → merge → render pipeline, but route both outputs through a shared intermediate directory model:
- `.pi-map.index.md` = routing-first view
- `.pi-map.md` = orientation-first view
Source remains the final authority.
## Affected areas
### Source files likely to change
- `src/format.ts`
- `src/init.ts`
- `src/patch.ts`
- `src/validate.ts`
- `pi-extension.ts`
- `README.md`
- `SKILL.md`
- `design-doc.md`
- CLI argument parsing for `validate --fix` if not already supported
### New modules likely to appear
- `src/root-index.ts` or equivalent shared index generation helper
- `src/directory-model.ts` or equivalent shared intermediate model helper
- optional config support for workflow/tag caps if not already present in config handling
## Architecture changes
### 1. Shared intermediate model
Build one structured directory model per mapped directory, then render two views from it.
The model should carry at least:
- directory identity (`dir`)
- sibling/parent/child relationships
- likely files
- role/arch summaries
- tags
- prioritized symbols
- workflow candidates in normalized schema
- stale/freshness markers
This shared model is the main guard against map/index drift.
### 2. Universal paired artifacts
During init/reinit, generate both `.pi-map.md` and `.pi-map.index.md` for every non-ignored directory.
#### Index rendering target
Indexes should be small and role-first, using this fixed order:
- `# <relative-path> (index)`
- `## role`
- `## parent`
- `## children`
- `## files`
- `## links`
- `## workflows`
- `## dirty`
Index content should include:
- `dir`
- short role summary
- parent link
- child directory links
- likely files
- explicit sibling/child `index:` / `map:` links
- up to configured workflow hints
- dirty marker
Indexes must omit dependency edges and keep architectural prose minimal.
#### Rich-map rendering target
Maps should remain dense but richer, using this fixed order:
- `# <relative-path>`
- `## role`
- `## files`
- `## arch`
- `## tags`
- `## symbols`
- `## workflows`
- `## dirty`
Map content should include:
- `dir`
- sibling `index:` link near the top
- slightly richer file lines than today when useful
- prioritized symbol lists in large directories
### 3. Root artifact behavior
Root `.pi-map.index.md` is the Tier 0 routing artifact.
Root `.pi-map.md` remains rich and must:
- point to root index,
- restate the trust boundary,
- be suitable for automatic loading on architecture/system or ambiguous tasks.
Both root artifacts should explicitly encode the Tier 0 behavior.
### 4. Workflow synthesis
Workflow hints may be LLM-heavy, but they must be normalized into a deterministic shape before rendering.
Suggested normalized fields:
- `task`
- `read`
- `index`
- `map`
- optional `files`
Indexes should render workflow hints in compact task→route form.
Because the user wants cross-repo workflow routing allowed, workflow synthesis may target other directories outside the local subtree when the hints are strong.
### 5. Configuration
Paired mode is the only mode for now, but these knobs should be configurable:
- workflow-hint cap (default 5)
- rich-map tag cap (default 8)
Configuration belongs in shared config handling rather than ad hoc generator constants.
Suggested config shape for v1:
```yaml
workflowHintCap: 5
tagCap: 8
```
### 6. Patch behavior
Patch should always regenerate, never hand-edit, generated artifacts.
#### Changed directory
Always regenerate both the local `.pi-map.md` and `.pi-map.index.md`.
#### Ancestor strategy
Use auto-detected patch sizing with explicit override available from both CLI and tool surfaces.
- **Small default:** up to 3 related files, no structure/routing shift
- refresh ancestor indexes only
- **Large/structural:** export changes, directory shape changes, routing metadata impact, or wider cross-area effect
- refresh both ancestor indexes and ancestor maps
This suggests a helper that computes the affected chain and the required artifact depth for each ancestor.
### 7. Validation behavior
Validation should become pair-aware, not just file-list-aware.
Checks should include:
- missing sibling artifact
- sibling `dir` mismatch
- broken parent/child/sibling references
- stale `dirty` markers
- stale or missing likely-file references
- disagreement between map/index routing references
Validation failures for missing/stale indexes are hard failures.
When validation classifies a patch as structural, the output should explain the main reason concisely (for example export change, directory-shape change, or routing impact).
Repair should be exposed through `validate --fix`, which regenerates the affected chain by default.
Validation and repair messaging should stay concise and inline, with the main structural reason plus the chosen `patchMode` when repair runs.
### 8. Prompt and doc integration
`pi-extension.ts`, `SKILL.md`, and docs should explicitly teach:
- Tier 0 = protocol + root index
- indexes first, maps next
- local rich map + source before edits
- validate before freshness-sensitive handoff
- `index routes, map orients, source decides`
### 9. External dependency boundary
This design should stay self-contained.
- No vector store is required.
- No Engram dependency is required.
- Retrieval remains local, deterministic, and based on generated paired artifacts.
## Risks
| Risk | Mitigation |
|---|---|
| Pair generation increases implementation size | Keep a shared intermediate model and split delivery into slices |
| Workflow hints become noisy | Omit uncertain hints and normalize schema before rendering |
| Ancestor refresh logic becomes brittle | Centralize affected-chain computation and test small vs structural cases |
| Pair drift creates false confidence | Use common `dir` identity and pair-aware validation |
| Config knobs spread inconsistently | Keep workflow/tag caps in shared config loading |
## Settled v1 defaults
- Root `.pi-map.md` points to root `.pi-map.index.md` rather than repeating route summaries.
- The shared intermediate directory model remains internal; behavior is tested through rendered outputs.
- Validation surfaces structural reasons as concise inline messages.
openspec/changes/layered-map-protocol/proposal.md
+96
View File
@@ -0,0 +1,96 @@
# Proposal: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Proposal** |
| Based on | User interview + current design doc |
| Next | Spec |
## Problem
The current design centers consumption on discovering all `.pi-map.md` files and loading them at session start, with pruning for larger repositories. That scales poorly and frames maps as bulk context instead of as a navigation system.
The repo currently lacks:
1. a clear root routing artifact,
2. an explicit trust/use protocol,
3. a layered loading model,
4. fast per-directory navigation artifacts,
5. a clean split between routing metadata and richer orientation metadata.
## Proposed change
Adopt a layered navigation model built around a **paired artifact** for every non-ignored directory:
- `.pi-map.index.md` → fast routing
- `.pi-map.md` → richer orientation
### Tier model
- **Tier 0 — always injected:** Project Map Protocol + root `.pi-map.index.md`
- **Tier 1 — task-start load:** likely relevant directory indexes first, then rich maps for strongest matches
- **Tier 2 — exact verification:** source files, tests, configs, and docs
### Paired artifacts per directory
Every non-ignored directory should generate both:
- a rich `.pi-map.md`
- a lightweight `.pi-map.index.md`
The index is for movement. The map is for understanding. Source remains the final authority.
This change stays self-contained: it does not add a vector store, Engram dependency, or other external memory/retrieval backend.
### Operating model
- Root `.pi-map.md` remains rich, but points to root `.pi-map.index.md` for traversal.
- Each index points to its sibling rich map.
- Each rich map points back to its sibling index.
- If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
## In scope
- [ ] Generate `.pi-map.index.md` for every non-ignored directory
- [ ] Keep `.pi-map.md` as the richer sibling artifact for every non-ignored directory
- [ ] Emit a Project Map Protocol in docs/prompt guidance and in generated root artifacts
- [ ] Replace eager "read all maps" guidance with layered retrieval guidance
- [ ] Define fixed index/map responsibilities and section order
- [ ] Cascade freshness and repair logic across changed directory pairs and affected ancestors
- [ ] Add strict validation for missing/stale/broken index-map pairs
- [ ] Update docs and skill guidance to reflect the new model
## Out of scope
- [ ] Add a query-driven retrieval command such as `project-map context <query>`
- [ ] Preserve backward compatibility for agents that only understand the old preload-only model
- [ ] Replace source verification with map-based authority
- [ ] Redesign the core AST/LLM extraction pipeline beyond what is needed to feed the new artifacts
## Decisions from grilling
| Topic | Decision |
|---|---|
| Change split | Two specs |
| This change slug | `layered-map-protocol` |
| Follow-up change slug | `map-context-retrieval` |
| Backward compatibility required | No |
| Artifact model | Every non-ignored directory gets both `.pi-map.md` and `.pi-map.index.md` |
| Root Tier 0 | Project Map Protocol + root `.pi-map.index.md` |
| Root rich map | Auto-read for architecture/system questions and ambiguous tasks |
| Root relationship | Root map points to root index |
| Retrieval mode | Hybrid auto-load, indexes first |
| Patch strategy | Changed dir updates both; ancestors refresh by size/structure rules |
| Validation | Hard-fail on missing/stale indexes, repair via `validate --fix` |
| Prompt trust boundary | index routes, map orients, source decides |
| Extra retrieval backends | None in this change |
| SDD mode | Interactive |
| Artifact store | OpenSpec |
| PR strategy | Auto-forecast |
| Review budget | 400 lines |
## Success criteria
- [ ] Every non-ignored directory has both a fast index and a richer map
- [ ] The root artifacts clearly tell an agent how to navigate and when to read source
- [ ] The documented consumption model no longer requires reading every `.pi-map.md` up front
- [ ] Validation fails on stale or missing paired artifacts and offers a repair path
- [ ] Patch/refresh behavior keeps routing trustworthy without requiring full reinit for small changes
openspec/changes/layered-map-protocol/spec.md
+229
View File
@@ -0,0 +1,229 @@
# Spec: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Spec** |
| Based on | [Proposal](proposal.md) |
| Next | Design |
## Overview
`pi-project-map` must move from a bulk-preload model to a layered navigation model. The system should always provide a small root protocol plus root index, selectively load likely relevant directory indexes first, load rich maps next for the strongest matches, and require source verification before edits or precise runtime claims.
## Decisions
| # | Question | Answer |
|---|---|---|
| 1 | Per-directory artifact model | Every non-ignored directory gets both `.pi-map.md` and `.pi-map.index.md` |
| 2 | Tier 0 default | Project Map Protocol + root `.pi-map.index.md` |
| 3 | Root rich map auto-read | Yes for architecture/system questions and ambiguous tasks |
| 4 | Retrieval style before query tool exists | Hybrid auto-load, indexes first |
| 5 | Backward compatibility with old preload-only model | Not required |
| 6 | Query-driven context bundling | Deferred to follow-up spec `map-context-retrieval` |
| 7 | Mode shape | Paired artifact mode is the only mode for now |
| 8 | Index section naming | Simple fixed names: `role`, `parent`, `children`, `files`, `links`, `workflows`, `dirty` |
| 9 | Rich map section naming | Current-plus fixed names: `role`, `files`, `arch`, `tags`, `symbols`, `workflows`, `dirty` |
| 10 | Patch-size override surface | Available from both CLI and tool surfaces |
| 11 | Structural classification output | Validation explains the main structural reason concisely |
| 12 | Extra retrieval backends | No vector store or Engram dependency in this change |
## Functional requirements
### 1. Paired artifacts per directory
Every non-ignored directory must generate two artifacts:
- `.pi-map.index.md` — quick-routing index
- `.pi-map.md` — rich orientation document
Both artifacts must share a common identity marker such as `dir: <relative-path>`.
### 2. Root entrypoint
The tool must generate a root navigation layer in both root artifacts.
#### Root Tier 0 behavior
Tier 0 must always include:
- Project Map Protocol
- root `.pi-map.index.md`
The generated root artifacts must explicitly state this behavior.
#### Root rich map behavior
Root `.pi-map.md` must:
- point to root `.pi-map.index.md` for traversal,
- restate the trust boundary,
- be suitable for automatic loading on architecture/system questions and ambiguous tasks.
### 3. Project Map Protocol
The generated system must define an explicit protocol with at least these rules:
1. Read the Project Map Protocol and root `.pi-map.index.md` first.
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
3. Load indexes before rich maps during task-start navigation.
4. Read the local rich map and actual source before editing.
5. Treat non-empty `## dirty` sections in either artifact as stale.
6. If source and generated artifacts disagree, trust source.
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
8. After editing source, run `project_map_patch` for each changed file.
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
### 4. Index contract
Every `.pi-map.index.md` must use a fixed section order and remain optimized for fast routing.
#### Required behavior
- Keep architectural prose minimal.
- Omit uncertain workflow/file hints rather than labeling confidence.
- Allow up to 5 workflow hints by default, with configuration support.
- Do not include dependency edges in indexes.
- Include explicit parent link when a parent mapped directory exists.
- For child directory entries, point to both child `index:` and child `map:`.
- Leaf-directory indexes remain tiny but still follow the same contract.
#### Fixed section order
Each index must use this fixed order:
1. `## role`
2. `## parent`
3. `## children`
4. `## files`
5. `## links`
6. `## workflows`
7. `## dirty`
#### Minimum contents
Each index must minimally support:
- title header in the form `# <relative-path> (index)`
- `dir` identity marker
- short role summary
- parent link where applicable
- child directories where applicable
- likely files
- explicit sibling/child `index:` / `map:` links
- short workflow hints in hybrid task→route form
- `## dirty`
### 5. Rich map contract
Every `.pi-map.md` must use a fixed section order and remain optimized for dense orientation.
#### Required behavior
- Rich maps must explicitly link to their sibling `.pi-map.index.md`.
- Rich maps may keep some routing overlap, but indexes remain primary for navigation.
- `files` entries should be slightly richer than the current dense line format when useful.
- Tags should default to up to 8, with configuration support.
- Symbols should be prioritized for density in large directories using structural importance plus LLM refinement.
- Cross-repo workflow routing should live mostly in indexes, not maps.
#### Fixed section order
Each rich map must use this fixed order:
1. `## role`
2. `## files`
3. `## arch`
4. `## tags`
5. `## symbols`
6. `## workflows`
7. `## dirty`
#### Minimum contents
Each rich map must minimally support:
- title header in the form `# <relative-path>`
- `dir` identity marker
- sibling `index:` link near the top
- richer but still compact workflow guidance
- `## dirty`
### 6. Workflow generation
Workflow hints may be LLM-heavy, but they must be normalized into a deterministic schema before rendering.
#### Index workflow rules
- Primary form is hybrid task→route, for example: `change CLI behavior -> read: src/.pi-map.index.md, src/cli.ts`
- Cross-repo routing is allowed.
- Uncertain hints should be omitted.
- Indexes should optimize for fast routing, not standalone completeness.
### 7. Shared generation model
Map and index generation must come from a shared intermediate directory model rather than unrelated passes.
The renderer may present different views, but the underlying directory identity and source facts must stay aligned.
### 8. Patch behavior
`project_map_patch` must never manually edit generated artifacts; it must regenerate them.
#### Changed directory
For every changed source file, patch must update both artifacts for the changed directory.
#### Ancestor refresh behavior
Patch sizing must be auto-detected with an explicit override path available from both CLI and tool surfaces.
- **Small change default:** up to 3 closely related files and no structural/routing shift
- changed directory: refresh map + index
- ancestors: refresh indexes only
- **Large/structural change:** exports, directory shape, routing metadata, or broader cross-area impact
- changed directory: refresh map + index
- ancestors: refresh both map + index
### 9. Validation behavior
`project_map_validate` must hard-fail when paired artifacts are missing or structurally stale.
Validation output should prefer concise inline wording, for example:
- `[structural] src/foo.ts: export change`
- `[stale-index] src/.pi-map.index.md: likely files out of date`
Validation must check at least:
- missing `.pi-map.index.md` or `.pi-map.md`
- stale `## dirty` sections in either artifact
- broken `index:` / `map:` references
- stale `files:` pointers
- map/index pair disagreement on routing hints or referenced areas
- sibling pair identity mismatches
When validation classifies a patch as structural, it should report the main reason concisely, such as export change, directory-shape change, or routing impact.
#### Repair
Repair must be exposed through `project-map validate --fix`.
Default repair scope is the affected directory pair plus the necessary ancestor chain.
Repair summaries should also report the chosen `patchMode` concisely, for example: `repaired affected chain (patchMode: structural)`.
### 10. Visibility and policy
`.pi-map.index.md` must follow the same hidden/ignored policy as `.pi-map.md`.
Agents should never manually edit either artifact directly.
## Non-functional requirements
- Prefer density over completeness.
- Keep indexes routing-first and maps understanding-first.
- Keep the protocol simple enough to emit in root artifacts and prompt guidance.
- Support configuration for top-level `workflowHintCap` and `tagCap` keys.
- Do not require a vector store, Engram, or other extra retrieval backend for this design.
## User flows
### Flow 1: Agent starts work on a repo
1. Agent reads the Project Map Protocol and root `.pi-map.index.md`
2. For architecture/system or ambiguous tasks, agent also reads root `.pi-map.md`
3. Agent opens likely relevant directory indexes first
4. Agent opens the strongest-match rich maps
5. Agent reads source only when exact behavior or editing is involved
### Flow 2: Agent edits a file
1. Agent routes with index data
2. Agent reads the local rich map and actual source
3. Agent edits source
4. Agent runs `project_map_patch` for changed files
5. Patch refreshes the changed directory pair and the required ancestor chain
### Flow 3: Validation and repair
1. Agent runs `project_map_validate`
2. Missing/stale pair failures are reported as hard errors
3. Agent runs `project-map validate --fix` when repair is desired
4. The affected chain is regenerated
## Acceptance criteria
- [ ] Every non-ignored directory produces both `.pi-map.md` and `.pi-map.index.md`
- [ ] Tier 0 behavior is both documented and emitted in generated root artifacts
- [ ] Indexes have a fixed routing-first contract with parent/child/sibling links
- [ ] Rich maps have a fixed orientation-first contract with sibling index links
- [ ] `project_map_patch` refreshes both changed-directory artifacts and size-appropriate ancestor artifacts
- [ ] `project_map_validate` hard-fails on missing/stale paired artifacts and supports `--fix`
- [ ] Workflow-hint count and tag cap are configurable
- [ ] Prompt guidance expresses: index routes, map orients, source decides
openspec/changes/layered-map-protocol/tasks.md
+75
View File
@@ -0,0 +1,75 @@
# Tasks: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Tasks** |
| Based on | [Design](design.md) |
| Next | Apply |
## Delivery slices
### Slice 1: Shared model and paired rendering
**Scope**: shared intermediate directory model, universal paired artifacts, fixed section names/order, root Tier 0 entrypoint
**Review goal**: establish the core paired architecture without retrieval-command work
**Tasks**:
1. [ ] Add a shared intermediate directory model for paired map/index generation
2. [ ] Extend render/parse support for paired `.pi-map.md` and `.pi-map.index.md` outputs with the agreed fixed section names/order
3. [ ] Add generation support for `.pi-map.index.md` in every non-ignored directory
4. [ ] Enrich root artifacts with explicit Tier 0 behavior and sibling links
5. [ ] Add/update tests for map/index rendering, fixed order, and sibling identity
### Slice 2: Routing metadata and workflow normalization
**Scope**: likely files, parent/child/sibling links, workflow schema, rich-map metadata density rules
**Review goal**: make the navigation model operational on real output
**Tasks**:
1. [ ] Add index-first routing metadata generation from AST/structure + LLM judgment
2. [ ] Normalize workflow hints into a deterministic schema before rendering
3. [ ] Add parent/child/sibling `index:` / `map:` references
4. [ ] Implement rich-map density rules for slightly richer file lines, tag cap, and prioritized symbols
5. [ ] Add/update tests for routing hints, leaf indexes, and workflow omission on low confidence
### Slice 3: Patch sizing, cascade refresh, and repair
**Scope**: changed-directory pair refresh, ancestor-chain refresh rules, validation hard failures, `validate --fix`
**Review goal**: make freshness and repair trustworthy
**Tasks**:
1. [ ] Make `project_map_patch` always regenerate both artifacts for the changed directory
2. [ ] Add auto-detected small vs structural patch sizing with explicit override in both CLI and tool surfaces
3. [ ] Refresh ancestor indexes for small changes and ancestor map+index pairs for structural changes
4. [ ] Expand validation for pair-aware checks, hard failures on missing/stale indexes, and concise structural-reason output
5. [ ] Add/confirm `project-map validate --fix` repair flow for affected-chain regeneration
6. [ ] Add/update tests for patch classification, ancestor refresh, and repair behavior
### Slice 4: Prompt guidance, config knobs, and docs
**Scope**: Project Map Protocol, Tier 0 runtime guidance, workflow/tag config, docs rewrite
**Review goal**: align runtime behavior and documentation with the final operating model
**Tasks**:
1. [ ] Update `pi-extension.ts` prompt guidance to teach the layered paired protocol
2. [ ] Update `SKILL.md` to describe indexes as routing and maps as orientation
3. [ ] Update `README.md` and `design-doc.md` consumption guidance
4. [ ] Add config support for workflow-hint cap and rich-map tag cap
5. [ ] Keep the paired-artifact model self-contained with no vector-store or Engram dependency
6. [ ] Add/update tests for config-driven caps and root Tier 0 guidance
## Acceptance checklist
- [ ] Every non-ignored directory has both `.pi-map.md` and `.pi-map.index.md`
- [ ] Root Tier 0 behavior is emitted in generated root artifacts
- [ ] Indexes are routing-first, role-first, and include parent/child/sibling links
- [ ] Rich maps are orientation-first and include sibling index links
- [ ] Changed-directory patch always regenerates both artifacts
- [ ] Ancestor refresh follows small vs structural rules
- [ ] Validation hard-fails on missing/stale paired artifacts and supports `validate --fix`
- [ ] Workflow-hint count and tag cap are configurable
- [ ] No vector-store or Engram dependency is introduced
- [ ] `npm run typecheck` passes
- [ ] `npm test` passes
- [ ] `npm run lint` passes
## Review workload note
This change crosses format, generation, patching, validation, config, docs, and prompt guidance. Keep implementation in narrow, reviewable slices and avoid one oversized PR.
openspec/changes/map-context-retrieval/design.md
+65
View File
@@ -0,0 +1,65 @@
# Design: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Design** |
| Based on | [Spec](spec.md) |
| Next | Tasks |
## Design summary
This change adds a lightweight retrieval layer on top of paired project-map metadata. It should not require a new external storage system, vector store, or Engram dependency. Instead, it should scan root and directory indexes first, expand to rich maps and files from the strongest candidates, and emit an agent-friendly bundle.
## Likely implementation areas
- `pi-extension.ts` for the first-class tool surface
- `src/index.ts` for exports
- new retrieval module, e.g. `src/context.ts` or `src/retrieve.ts`
- shared parsing/model code from the layered protocol
- `src/cli/*` for follow-up CLI wiring
- docs and skill guidance
## Retrieval pipeline
1. Accept a natural-language `query`
2. Read root `.pi-map.index.md` and, when needed, root `.pi-map.md`
3. Parse paired index/map metadata through the shared format/model layer
4. Score candidate directories primarily from indexes
5. Keep the top 3 candidates by default
6. Expand strongest candidates to rich maps, likely files, and symbols
7. Emit a compact markdown bundle with stable section order and retrieval-specific title `# Context bundle: <query>`
## Candidate scoring inputs
- direct term matches in workflow hints
- direct term matches in likely files
- parent/child/sibling link context
- direct term matches in tags
- direct term matches in symbols
- path/name similarity
- root workflow routing hits
A first implementation can use deterministic weighted lexical scoring.
Candidate-selection reasoning does not need to be exposed by default.
## Parsing strategy
Reuse the paired-artifact parser/model from the layered protocol. Avoid retrieval-specific ad hoc parsing.
## Risks
| Risk | Mitigation |
|---|---|
| Retrieval becomes too fuzzy to trust | Keep output advisory and always direct agent back to source |
| Pair parser complexity grows | Extend shared format/model logic instead of command-local parsing |
| Command output becomes too large | Limit result count and keep instructions terse |
## Open choices
1. Exact scoring weights for workflow hints vs files vs tags vs symbols
2. Whether the tool should support optional structured output later
3. Whether broad architecture queries should be allowed to exceed the usual top-3 default in a later version
4. Whether later versions should support optional root-rich-map inclusion as a flag
openspec/changes/map-context-retrieval/proposal.md
+53
View File
@@ -0,0 +1,53 @@
# Proposal: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Proposal** |
| Based on | Follow-up to `layered-map-protocol` |
| Next | Spec |
## Problem
After the layered map protocol lands, the agent will know it should load root protocol + root index first, then route through per-directory indexes and rich maps. But the repo will still lack an ergonomic retrieval primitive that can turn a natural-language task into a compact context bundle.
Without that helper, the agent still has to manually inspect directory indexes, rank candidate areas, and expand to rich maps/files. That weakens the value of the new routing layer.
## Proposed change
Add a retrieval tool and command shape:
```bash
project-map context "<user task>"
```
The first-class surface should be a Pi tool. CLI support can follow the same shape.
The retrieval result should return a compact context bundle containing:
- relevant indexes,
- strongest-match rich maps,
- likely source files,
- relevant symbols when useful,
- short instructions on what to read next.
This remains a local metadata-driven retrieval feature, not a vector-store or Engram-backed memory system.
## In scope
- [ ] Add Pi tool support for `project-map context <query>`
- [ ] Add CLI support using the same shape after the tool contract is stable
- [ ] Rank results using paired index/map metadata from the layered protocol
- [ ] Return a compact, markdown-first, agent-friendly context bundle
- [ ] Document recommended usage from prompts and docs
## Out of scope
- [ ] Full semantic search across arbitrary source contents
- [ ] Replacing source verification with map-based answers
- [ ] Building a heavyweight external indexer or vector database
## Success criteria
- [ ] A natural-language task can resolve to likely indexes/maps/files/symbols without manual repo scanning
- [ ] Output is compact enough to inject directly into the next step
- [ ] Retrieval builds on generated paired metadata rather than bypassing it
- [ ] The first version is tool-first and markdown-first
openspec/changes/map-context-retrieval/spec.md
+127
View File
@@ -0,0 +1,127 @@
# Spec: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Spec** |
| Based on | [Proposal](proposal.md) |
| Next | Design |
## Overview
Add a retrieval-oriented `context` command that turns a user task into a compact project-map routing bundle. This change depends on the paired root/per-directory index+map metadata introduced by `layered-map-protocol`.
## Dependency
This change should not begin implementation before `layered-map-protocol` has landed enough metadata to support routing and ranking.
## Functional requirements
### 1. Tool-first surface
Pi should expose `project-map context` as a first-class tool action so agents can request a context bundle directly.
For v1, the tool input should be minimal:
- `query: string`
### 2. CLI follow-up surface
The CLI should support the same conceptual shape:
```bash
project-map context "<query>"
```
CLI support may follow after the tool contract is stable.
The CLI should mirror the same main input name: `query`.
### 3. Bundle contents
A context bundle must contain, at minimum:
- relevant indexes
- strongest-match relevant maps
- likely files
- relevant symbols when useful
- short next-step instructions
### 4. Ranking behavior
The command must rank candidates using generated metadata such as:
- root workflow hints
- directory index routing hints
- parent/child/sibling link structure
- package tags
- symbol references
- task/workflow entries
- `files:` / `index:` / `map:` targets
Indexes should be the first routing layer. Rich maps and symbols should expand from the strongest routed candidates.
By default, retrieval should usually surface the top 3 candidate directories, with adaptive omission of weaker rich maps when confidence drops.
### 5. Trust boundary
The bundle must instruct the agent to:
- read indexes first for orientation,
- read rich maps next when deeper context is needed,
- read source before editing or asserting exact behavior.
## Output shape
The first version should be markdown-first, with stable, schema-like sections optimized for LLM consumption rather than human prose.
Default section order:
1. `query`
2. `relevant indexes`
3. `relevant maps`
4. `likely files`
5. `relevant symbols`
6. `instructions`
The retrieval title format should remain retrieval-specific:
- `# Context bundle: <query>`
A representative response shape:
```markdown
# Context bundle: validation stale signatures
## query
validation stale signatures
## relevant indexes
- .pi-map.index.md
- src/.pi-map.index.md
- tests/.pi-map.index.md
## relevant maps
- src/.pi-map.md
- tests/.pi-map.md
## likely files
- src/validate.ts
- src/ast/ast-extract.ts
- tests/validate.test.ts
## relevant symbols
- validateMaps
- generateDirectoryMap
- extractFileAST
## instructions
Read the indexes first, then the strongest-match rich maps, then verify behavior from source before editing.
```
The tool should omit weak/unhelpful symbol output rather than forcing a noisy symbol section.
## Non-functional requirements
- Output must be compact enough for prompt injection.
- Ranking should be deterministic enough to test.
- The implementation should prefer lightweight heuristics over heavyweight indexing.
- Retrieval should reuse the paired-artifact parsing layer instead of ad hoc string slicing.
- Retrieval should not require a vector store, Engram, or another external memory backend.
- The first version should accept natural language queries first; structured filters can come later.
- Candidate explanations are not required by default.
- Result-count override and root-rich-map inclusion can come later if needed, but are not part of v1.
## Acceptance criteria
- [ ] Pi exposes `project-map context <query>` as a tool-first surface
- [ ] CLI support follows the same shape after the tool contract is stable
- [ ] Retrieval output includes indexes, strongest-match maps, files, and instructions
- [ ] Symbol hints appear only when paired metadata makes them useful
- [ ] Ranking respects the index-first routing model and usually returns top 3 candidates
- [ ] Docs show when to use the retrieval command
- [ ] Trust-boundary guidance remains explicit
openspec/changes/map-context-retrieval/tasks.md
+34
View File
@@ -0,0 +1,34 @@
# Tasks: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Tasks** |
| Based on | [Design](design.md) |
| Next | Apply |
## Tasks
1. [ ] Add Pi tool support for `project-map context` with `query` as the main input
2. [ ] Add retrieval module for scoring paired index/map metadata
3. [ ] Reuse or extend shared paired-artifact parsing/model code
4. [ ] Rank candidate directories from indexes first, then expand to maps/files/symbols
5. [ ] Emit compact markdown-first context bundles with stable section order and retrieval-specific `Context bundle` title
6. [ ] Add tests for retrieval ranking, top-3 default behavior, and output shape
7. [ ] Add CLI support after the tool contract is stable
8. [ ] Update docs and skill guidance for retrieval usage
## Acceptance checklist
- [ ] Tool works for natural-language queries via `query`
- [ ] Output includes relevant indexes, strongest-match maps, likely files, symbols when useful, and instructions
- [ ] Retrieval depends on paired metadata rather than raw source scanning
- [ ] Ranking follows the index-first routing model with a top-3 default
- [ ] `npm run typecheck` passes
- [ ] `npm test` passes
- [ ] `npm run lint` passes
## Sequencing note
This change is intentionally follow-up work. It should start after the layered map protocol produces reliable paired routing metadata.
The first delivery surface is the Pi tool. CLI support follows the stabilized tool contract.