pi-map/design-doc.md

# Design Reference: pi-project-map

> Audience: maintainers and contributors.  
> Purpose: explain how `pi-project-map` works internally, not how to install or use it.

## 1. Overview

`pi-project-map` is a TypeScript/Node.js skill package that generates and maintains hierarchical, paired project-map artifacts for AI coding agents:

- `.pi-map.index.md` — routing-first, sparse directory metadata
- `.pi-map.md` — orientation-first, richer directory metadata

It runs as both:
- a standalone CLI (`project-map`)
- a Pi extension (`pi-extension.ts`)

The extension registers tools and event hooks that keep the artifacts fresh and can inject them into agent context at runtime.

### Core design principle

The artifacts are **navigation aids, not source-of-truth**. Source code is always the final authority.

> **index routes, map orients, source decides.**

## 2. Artifact model

Each non-ignored directory receives a matched pair.

### 2.1 Shared model

Both files are generated from the same in-memory `DirectoryArtifactModel`:

```ts
interface DirectoryArtifactModel {
  dir: string;
  role: string;
  files: FileEntry[];
  arch: string;
  dirty?: string;
  isRoot: boolean;
  parent?: string;
  children: string[];
  tags: string[];
  symbols: string[];
  workflows: WorkflowHint[];
}

interface FileEntry {
  name: string;
  purpose: string;
  exports: string[];
  deps: string[];
}
```

### 2.2 `.pi-map.md` (rich map)

Rendered by `src/format.ts` → `renderDirectoryMap()`.

Contains:
- `dir:` line and sibling `index:` link
- `Project Map Protocol` (root only)
- `## role`
- `## files`
- `## arch`
- `## tags`
- `## symbols`
- `## workflows`
- `## dirty`

### 2.3 `.pi-map.index.md` (index)

Rendered by `src/format.ts` → `renderDirectoryIndex()`.

Contains:
- same protocol (root only)
- `## role`
- `## parent`
- `## children`
- `## files`
- `## links`
- `## workflows`
- `## dirty`

### 2.4 Why a paired format?

- **indexes are cheap** — many can be loaded without consuming much context
- **maps are dense** — loaded only after an index suggests relevance
- **paired generation** guarantees structural consistency

## 3. High-level architecture

### 3.1 Main modules

```text
discover          → directory tree, .gitignore-aware
init              → full generation
llm-extract       → LLM-based file/package analysis
ast-extract       → tree-sitter parsing
merge             → combine LLM + AST into FileEntry
routing-metadata  → tags, symbols, workflow hints
format            → render / parse the markdown pair
patch             → incremental update after edits
validate          → consistency checking with optional repair
retrieve          → deterministic query scoring
prompt-injection  → runtime context policy
pi-extension      → Pi tool/event registration
cli               → standalone command dispatcher
config            → defaults and .pi-project-map.json loader
```

### 3.2 Runtime modes

| Mode | Entry point | LLM client |
|------|-------------|------------|
| Pi extension | `pi-extension.ts` | `PiLLMClient` via Pi runtime |
| Standalone CLI | `src/cli/cli.ts` | `ExternalLLMClient` or `KimiLLMClient` |

## 4. Extraction pipeline

### 4.1 Discovery

`src/discover.ts` walks the filesystem with `ignore`, merging built-in exclusions and `.gitignore`.

### 4.2 Per-directory generation

`src/init.ts` → `generateDirectoryArtifacts()`:

1. `processFiles()` runs in parallel over directory files
2. for each file:
   - `extractFileLLM()` gets `purpose`, `deps`, `concepts`
   - `extractFileAST()` gets exports/imports/calls where possible
   - `mergeFileData()` combines both into a `FileEntry`
3. `extractPackageLLM()` produces directory `role` and `arch`
4. `createDirectoryModel()` builds the shared model
5. `populateRoutingMetadata()` derives `tags`, `symbols`, `workflows`
6. `writeDirectoryArtifacts()` writes both `.pi-map.md` and `.pi-map.index.md`

Directories are processed sequentially; files within a directory are processed concurrently.

### 4.3 LLM extraction

`src/llm/llm-extract.ts`:
- prompts are minimal and line-oriented
- binary files are skipped
- files over 500KB are labeled large and skipped
- source is truncated before prompting
- results are cached by SHA-256 of file content
- missing client throws `LLMError`

### 4.4 AST extraction

`src/ast/ast-extract.ts` uses `tree-sitter` for supported languages to extract:
- imports / requires
- exported classes, functions, constants
- methods, parameters, return types
- direct calls and raised exceptions

Unsupported languages fall back to LLM-only extraction.

### 4.5 Merging

`src/merge.ts`:
- purpose/concepts come from the LLM
- exports come from AST when available
- rich AST symbols are encoded as compact DSL:
  - `class:Foo`
  - `method:bar(a: string) → number`
  - `call:baz`
  - `raise:Error`
- deps are deduplicated union of AST + LLM deps

### 4.6 Routing metadata

`src/routing-metadata.ts` generates deterministic metadata used by retrieval and injection:
- **tags**
- **symbols**
- **workflow hints**

Caps are configurable via `tagCap` and `workflowHintCap`.

## 5. Patch / validate / reinit behavior

### 5.1 Patch

`src/patch.ts`:
1. resolve directory containing changed file
2. rediscover project tree
3. regenerate changed directory pair
4. refresh ancestors according to patch mode

Patch mode:
- **small** — refresh ancestor indexes only
- **structural** — refresh ancestor map/index pairs

### 5.2 Validate

`src/validate.ts` compares artifacts against filesystem and AST.

Important discrepancy types:
- `missing`
- `orphaned`
- `stale-signature`
- `dirty`
- `stale-map`
- `stale-index`
- `broken-link`
- `structural`

With `--fix`, validate builds a repair plan and regenerates directories deepest-first.

### 5.3 Reinit

`reinitPath()` is the blunt instrument for widespread staleness.

## 6. Retrieval architecture

`src/retrieve.ts` implements deterministic, index-first context retrieval.

1. walk the project for paired artifacts
2. parse indexes/maps into `DirectoryArtifactModel`
3. normalize the query
4. score every directory
5. return top-K (default: 3) as a markdown bundle with:
   - relevant indexes
   - relevant maps
   - likely files
   - relevant symbols
   - instructions to verify from source

No LLM is used during retrieval. It is intentionally separate from automatic prompt injection.

## 7. Prompt injection architecture

`src/prompt-injection.ts` and `pi-extension.ts` implement runtime guidance injection.

### 7.1 Mode ladder

| Mode | Behavior |
|------|----------|
| `off` | No automatic injection |
| `advisory` | Visible startup/init hints; no artifact preload |
| `strong` (default) | Root pair preloaded, budgeted expansion, reinjection on relevant turns |
| `strict` | Same as strong, plus bypass guard for sensitive edits/architecture reasoning without protocol path |

### 7.2 Event hooks

The extension currently registers:
- `session_start`
- `before_agent_start`
- `context`

Payload fallback scanning is handled inside `context`-level decision logic; there is no separately registered `before_provider_request` hook in the current implementation.

### 7.3 Reinjection policy

`shouldReinjectForEvent()` decides whether to inject:
- only active in `strong` or `strict`
- skips if the canonical marker is already present in outgoing messages or payload
- triggers on:
  - `agent_start`
  - `edit_intent`
  - `architecture_sensitive`
  - `compaction`
  - `artifact_change`
- `artifact_change` always forces reinjection

`detectEditIntent()` and `detectArchitectureSensitiveReasoning()` provide heuristic fallback for generic turns.

In addition to marker-based deduplication, `before_agent_start` scans the active session context via `ctx.sessionManager.buildSessionContext()` for an existing `pi-project-map-hint` custom message. If one is already present in the current branch, the handler skips injection entirely. This prevents duplicate visible hints in advisory/pre-init modes and duplicate hidden hints in strong/strict modes when the session context already contains the guidance. The hint is automatically re-injected after compaction or `/tree` navigation removes it from the active path.

### 7.4 Protocol path and strict bypass

The **protocol path** is present when outgoing context contains:
1. the canonical root-pair marker/block
2. the trust-boundary text

In `strict` mode, a sensitive turn without the protocol path is blocked with a visible guard. The agent can override with:

```text
[PI_MAP_BYPASS: brief justification]
```

Empty or whitespace reasons are rejected.

### 7.5 Budgeted expansion

`buildInjectionPayload()`:
- computes budget as `min(relative, absolute)`
- default is 15% of context window, capped at 100k tokens
- always includes the root pair
- adds additional pairs shallow-first until budget is exhausted
- prepends a maintenance reminder

Token estimation is best-effort: `ceil(char_count / 4)`.

### 7.6 Context-window discovery

`discoverContextWindow()` inspects the Pi runtime model for context metadata and falls back to the absolute cap when unavailable.

## 8. Known limits and tradeoffs

### Correctness vs cost
- init/patch/repair make LLM calls
- large repositories can be expensive
- caching reduces duplicate work

### AST coverage
- TypeScript/TSX, Python, and Go have the richest support
- other languages may be partial or LLM-only

### Token estimation
- 4 chars/token is only a heuristic
- oversized files may be truncated or skipped

### Staleness
- there is no filesystem watcher
- maps go stale when edits happen outside the patch flow
- validate detects but does not prevent staleness

### Patch mode inference
- auto-mode heuristics are good but imperfect
- contributors can force structural mode when needed

### Strict mode ergonomics
- strict guards can be surprising on casual phrasing
- bypass markers are intentionally explicit and user-visible

### Retrieval scoring
- deterministic scoring is reproducible but not semantic-search-smart
- broader queries may still need manual browsing

### Cache
- cache grows unless manually cleaned
- corrupted cache files are recovered by starting fresh