334 lines
9.4 KiB
Markdown
334 lines
9.4 KiB
Markdown
# Design Reference: pi-project-map
|
|
|
|
> Audience: maintainers and contributors.
|
|
> Purpose: explain how `pi-project-map` works internally, not how to install or use it.
|
|
|
|
## 1. Overview
|
|
|
|
`pi-project-map` is a TypeScript/Node.js skill package that generates and maintains hierarchical, paired project-map artifacts for AI coding agents:
|
|
|
|
- `.pi-map.index.md` — routing-first, sparse directory metadata
|
|
- `.pi-map.md` — orientation-first, richer directory metadata
|
|
|
|
It runs as both:
|
|
- a standalone CLI (`project-map`)
|
|
- a Pi extension (`pi-extension.ts`)
|
|
|
|
The extension registers tools and event hooks that keep the artifacts fresh and can inject them into agent context at runtime.
|
|
|
|
### Core design principle
|
|
|
|
The artifacts are **navigation aids, not source-of-truth**. Source code is always the final authority.
|
|
|
|
> **index routes, map orients, source decides.**
|
|
|
|
## 2. Artifact model
|
|
|
|
Each non-ignored directory receives a matched pair.
|
|
|
|
### 2.1 Shared model
|
|
|
|
Both files are generated from the same in-memory `DirectoryArtifactModel`:
|
|
|
|
```ts
|
|
interface DirectoryArtifactModel {
|
|
dir: string;
|
|
role: string;
|
|
files: FileEntry[];
|
|
arch: string;
|
|
dirty?: string;
|
|
isRoot: boolean;
|
|
parent?: string;
|
|
children: string[];
|
|
tags: string[];
|
|
symbols: string[];
|
|
workflows: WorkflowHint[];
|
|
}
|
|
|
|
interface FileEntry {
|
|
name: string;
|
|
purpose: string;
|
|
exports: string[];
|
|
deps: string[];
|
|
}
|
|
```
|
|
|
|
### 2.2 `.pi-map.md` (rich map)
|
|
|
|
Rendered by `src/format.ts` → `renderDirectoryMap()`.
|
|
|
|
Contains:
|
|
- `dir:` line and sibling `index:` link
|
|
- `Project Map Protocol` (root only)
|
|
- `## role`
|
|
- `## files`
|
|
- `## arch`
|
|
- `## tags`
|
|
- `## symbols`
|
|
- `## workflows`
|
|
- `## dirty`
|
|
|
|
### 2.3 `.pi-map.index.md` (index)
|
|
|
|
Rendered by `src/format.ts` → `renderDirectoryIndex()`.
|
|
|
|
Contains:
|
|
- same protocol (root only)
|
|
- `## role`
|
|
- `## parent`
|
|
- `## children`
|
|
- `## files`
|
|
- `## links`
|
|
- `## workflows`
|
|
- `## dirty`
|
|
|
|
### 2.4 Why a paired format?
|
|
|
|
- **indexes are cheap** — many can be loaded without consuming much context
|
|
- **maps are dense** — loaded only after an index suggests relevance
|
|
- **paired generation** guarantees structural consistency
|
|
|
|
## 3. High-level architecture
|
|
|
|
### 3.1 Main modules
|
|
|
|
```text
|
|
discover → directory tree, .gitignore-aware
|
|
init → full generation
|
|
llm-extract → LLM-based file/package analysis
|
|
ast-extract → tree-sitter parsing
|
|
merge → combine LLM + AST into FileEntry
|
|
routing-metadata → tags, symbols, workflow hints
|
|
format → render / parse the markdown pair
|
|
patch → incremental update after edits
|
|
validate → consistency checking with optional repair
|
|
retrieve → deterministic query scoring
|
|
prompt-injection → runtime context policy
|
|
pi-extension → Pi tool/event registration
|
|
cli → standalone command dispatcher
|
|
config → defaults and .pi-project-map.json loader
|
|
```
|
|
|
|
### 3.2 Runtime modes
|
|
|
|
| Mode | Entry point | LLM client |
|
|
|------|-------------|------------|
|
|
| Pi extension | `pi-extension.ts` | `PiLLMClient` via Pi runtime |
|
|
| Standalone CLI | `src/cli/cli.ts` | `ExternalLLMClient` or `KimiLLMClient` |
|
|
|
|
## 4. Extraction pipeline
|
|
|
|
### 4.1 Discovery
|
|
|
|
`src/discover.ts` walks the filesystem with `ignore`, merging built-in exclusions and `.gitignore`.
|
|
|
|
### 4.2 Per-directory generation
|
|
|
|
`src/init.ts` → `generateDirectoryArtifacts()`:
|
|
|
|
1. `processFiles()` runs in parallel over directory files
|
|
2. for each file:
|
|
- `extractFileLLM()` gets `purpose`, `deps`, `concepts`
|
|
- `extractFileAST()` gets exports/imports/calls where possible
|
|
- `mergeFileData()` combines both into a `FileEntry`
|
|
3. `extractPackageLLM()` produces directory `role` and `arch`
|
|
4. `createDirectoryModel()` builds the shared model
|
|
5. `populateRoutingMetadata()` derives `tags`, `symbols`, `workflows`
|
|
6. `writeDirectoryArtifacts()` writes both `.pi-map.md` and `.pi-map.index.md`
|
|
|
|
Directories are processed sequentially; files within a directory are processed concurrently.
|
|
|
|
### 4.3 LLM extraction
|
|
|
|
`src/llm/llm-extract.ts`:
|
|
- prompts are minimal and line-oriented
|
|
- binary files are skipped
|
|
- files over 500KB are labeled large and skipped
|
|
- source is truncated before prompting
|
|
- results are cached by SHA-256 of file content
|
|
- missing client throws `LLMError`
|
|
|
|
### 4.4 AST extraction
|
|
|
|
`src/ast/ast-extract.ts` uses `tree-sitter` for supported languages to extract:
|
|
- imports / requires
|
|
- exported classes, functions, constants
|
|
- methods, parameters, return types
|
|
- direct calls and raised exceptions
|
|
|
|
Unsupported languages fall back to LLM-only extraction.
|
|
|
|
### 4.5 Merging
|
|
|
|
`src/merge.ts`:
|
|
- purpose/concepts come from the LLM
|
|
- exports come from AST when available
|
|
- rich AST symbols are encoded as compact DSL:
|
|
- `class:Foo`
|
|
- `method:bar(a: string) → number`
|
|
- `call:baz`
|
|
- `raise:Error`
|
|
- deps are deduplicated union of AST + LLM deps
|
|
|
|
### 4.6 Routing metadata
|
|
|
|
`src/routing-metadata.ts` generates deterministic metadata used by retrieval and injection:
|
|
- **tags**
|
|
- **symbols**
|
|
- **workflow hints**
|
|
|
|
Caps are configurable via `tagCap` and `workflowHintCap`.
|
|
|
|
## 5. Patch / validate / reinit behavior
|
|
|
|
### 5.1 Patch
|
|
|
|
`src/patch.ts`:
|
|
1. resolve directory containing changed file
|
|
2. rediscover project tree
|
|
3. regenerate changed directory pair
|
|
4. refresh ancestors according to patch mode
|
|
|
|
Patch mode:
|
|
- **small** — refresh ancestor indexes only
|
|
- **structural** — refresh ancestor map/index pairs
|
|
|
|
### 5.2 Validate
|
|
|
|
`src/validate.ts` compares artifacts against filesystem and AST.
|
|
|
|
Important discrepancy types:
|
|
- `missing`
|
|
- `orphaned`
|
|
- `stale-signature`
|
|
- `dirty`
|
|
- `stale-map`
|
|
- `stale-index`
|
|
- `broken-link`
|
|
- `structural`
|
|
|
|
With `--fix`, validate builds a repair plan and regenerates directories deepest-first.
|
|
|
|
### 5.3 Reinit
|
|
|
|
`reinitPath()` is the blunt instrument for widespread staleness.
|
|
|
|
## 6. Retrieval architecture
|
|
|
|
`src/retrieve.ts` implements deterministic, index-first context retrieval.
|
|
|
|
1. walk the project for paired artifacts
|
|
2. parse indexes/maps into `DirectoryArtifactModel`
|
|
3. normalize the query
|
|
4. score every directory
|
|
5. return top-K (default: 3) as a markdown bundle with:
|
|
- relevant indexes
|
|
- relevant maps
|
|
- likely files
|
|
- relevant symbols
|
|
- instructions to verify from source
|
|
|
|
No LLM is used during retrieval. It is intentionally separate from automatic prompt injection.
|
|
|
|
## 7. Prompt injection architecture
|
|
|
|
`src/prompt-injection.ts` and `pi-extension.ts` implement runtime guidance injection.
|
|
|
|
### 7.1 Mode ladder
|
|
|
|
| Mode | Behavior |
|
|
|------|----------|
|
|
| `off` | No automatic injection |
|
|
| `advisory` | Visible startup/init hints; no artifact preload |
|
|
| `strong` (default) | Root pair preloaded, budgeted expansion, reinjection on relevant turns |
|
|
| `strict` | Same as strong, plus bypass guard for sensitive edits/architecture reasoning without protocol path |
|
|
|
|
### 7.2 Event hooks
|
|
|
|
The extension currently registers:
|
|
- `session_start`
|
|
- `before_agent_start`
|
|
- `context`
|
|
|
|
Payload fallback scanning is handled inside `context`-level decision logic; there is no separately registered `before_provider_request` hook in the current implementation.
|
|
|
|
### 7.3 Reinjection policy
|
|
|
|
`shouldReinjectForEvent()` decides whether to inject:
|
|
- only active in `strong` or `strict`
|
|
- skips if the canonical marker is already present in outgoing messages or payload
|
|
- triggers on:
|
|
- `agent_start`
|
|
- `edit_intent`
|
|
- `architecture_sensitive`
|
|
- `compaction`
|
|
- `artifact_change`
|
|
- `artifact_change` always forces reinjection
|
|
|
|
`detectEditIntent()` and `detectArchitectureSensitiveReasoning()` provide heuristic fallback for generic turns.
|
|
|
|
### 7.4 Protocol path and strict bypass
|
|
|
|
The **protocol path** is present when outgoing context contains:
|
|
1. the canonical root-pair marker/block
|
|
2. the trust-boundary text
|
|
|
|
In `strict` mode, a sensitive turn without the protocol path is blocked with a visible guard. The agent can override with:
|
|
|
|
```text
|
|
[PI_MAP_BYPASS: brief justification]
|
|
```
|
|
|
|
Empty or whitespace reasons are rejected.
|
|
|
|
### 7.5 Budgeted expansion
|
|
|
|
`buildInjectionPayload()`:
|
|
- computes budget as `min(relative, absolute)`
|
|
- default is 15% of context window, capped at 100k tokens
|
|
- always includes the root pair
|
|
- adds additional pairs shallow-first until budget is exhausted
|
|
- prepends a maintenance reminder
|
|
|
|
Token estimation is best-effort: `ceil(char_count / 4)`.
|
|
|
|
### 7.6 Context-window discovery
|
|
|
|
`discoverContextWindow()` inspects the Pi runtime model for context metadata and falls back to the absolute cap when unavailable.
|
|
|
|
## 8. Known limits and tradeoffs
|
|
|
|
### Correctness vs cost
|
|
- init/patch/repair make LLM calls
|
|
- large repositories can be expensive
|
|
- caching reduces duplicate work
|
|
|
|
### AST coverage
|
|
- TypeScript/TSX, Python, and Go have the richest support
|
|
- other languages may be partial or LLM-only
|
|
|
|
### Token estimation
|
|
- 4 chars/token is only a heuristic
|
|
- oversized files may be truncated or skipped
|
|
|
|
### Staleness
|
|
- there is no filesystem watcher
|
|
- maps go stale when edits happen outside the patch flow
|
|
- validate detects but does not prevent staleness
|
|
|
|
### Patch mode inference
|
|
- auto-mode heuristics are good but imperfect
|
|
- contributors can force structural mode when needed
|
|
|
|
### Strict mode ergonomics
|
|
- strict guards can be surprising on casual phrasing
|
|
- bypass markers are intentionally explicit and user-visible
|
|
|
|
### Retrieval scoring
|
|
- deterministic scoring is reproducible but not semantic-search-smart
|
|
- broader queries may still need manual browsing
|
|
|
|
### Cache
|
|
- cache grows unless manually cleaned
|
|
- corrupted cache files are recovered by starting fresh
|