pi-map/design-doc.md

Design Reference: pi-project-map

Audience: maintainers and contributors.
Purpose: explain how pi-project-map works internally, not how to install or use it.

1. Overview

pi-project-map is a TypeScript/Node.js skill package that generates and maintains hierarchical, paired project-map artifacts for AI coding agents:

• .pi-map.index.md — routing-first, sparse directory metadata
• .pi-map.md — orientation-first, richer directory metadata

It runs as both:

• a standalone CLI (project-map)
• a Pi extension (pi-extension.ts)

The extension registers tools and event hooks that keep the artifacts fresh and can inject them into agent context at runtime.

Core design principle

The artifacts are navigation aids, not source-of-truth. Source code is always the final authority.

index routes, map orients, source decides.

2. Artifact model

Each non-ignored directory receives a matched pair.

2.1 Shared model

Both files are generated from the same in-memory DirectoryArtifactModel:

interface DirectoryArtifactModel {
  dir: string;
  role: string;
  files: FileEntry[];
  arch: string;
  dirty?: string;
  isRoot: boolean;
  parent?: string;
  children: string[];
  tags: string[];
  symbols: string[];
  workflows: WorkflowHint[];
}

interface FileEntry {
  name: string;
  purpose: string;
  exports: string[];
  deps: string[];
}

2.2 .pi-map.md (rich map)

Rendered by src/format.ts → renderDirectoryMap().

Contains:

• dir: line and sibling index: link
• Project Map Protocol (root only)
• ## role
• ## files
• ## arch
• ## tags
• ## symbols
• ## workflows
• ## dirty

2.3 .pi-map.index.md (index)

Rendered by src/format.ts → renderDirectoryIndex().

Contains:

• same protocol (root only)
• ## role
• ## parent
• ## children
• ## files
• ## links
• ## workflows
• ## dirty

2.4 Why a paired format?

• indexes are cheap — many can be loaded without consuming much context
• maps are dense — loaded only after an index suggests relevance
• paired generation guarantees structural consistency

3. High-level architecture

3.1 Main modules

discover          → directory tree, .gitignore-aware
init              → full generation
llm-extract       → LLM-based file/package analysis
ast-extract       → tree-sitter parsing
merge             → combine LLM + AST into FileEntry
routing-metadata  → tags, symbols, workflow hints
format            → render / parse the markdown pair
patch             → incremental update after edits
validate          → consistency checking with optional repair
retrieve          → deterministic query scoring
prompt-injection  → runtime context policy
pi-extension      → Pi tool/event registration
cli               → standalone command dispatcher
config            → defaults and .pi-project-map.json loader

3.2 Runtime modes

Mode	Entry point	LLM client
Pi extension	pi-extension.ts	PiLLMClient via Pi runtime
Standalone CLI	src/cli/cli.ts	ExternalLLMClient or KimiLLMClient

4. Extraction pipeline

4.1 Discovery

src/discover.ts walks the filesystem with ignore, merging built-in exclusions and .gitignore.

4.2 Per-directory generation

src/init.ts → generateDirectoryArtifacts():

1. processFiles() runs in parallel over directory files
2. for each file:
   • extractFileLLM() gets purpose, deps, concepts
   • extractFileAST() gets exports/imports/calls where possible
   • mergeFileData() combines both into a FileEntry
3. extractPackageLLM() produces directory role and arch
4. createDirectoryModel() builds the shared model
5. populateRoutingMetadata() derives tags, symbols, workflows
6. writeDirectoryArtifacts() writes both .pi-map.md and .pi-map.index.md

Directories are processed sequentially; files within a directory are processed concurrently.

4.3 LLM extraction

src/llm/llm-extract.ts:

• prompts are minimal and line-oriented
• binary files are skipped
• files over 500KB are labeled large and skipped
• source is truncated before prompting
• results are cached by SHA-256 of file content
• missing client throws LLMError

4.4 AST extraction

src/ast/ast-extract.ts uses tree-sitter for supported languages to extract:

• imports / requires
• exported classes, functions, constants
• methods, parameters, return types
• direct calls and raised exceptions

Unsupported languages fall back to LLM-only extraction.

4.5 Merging

src/merge.ts:

• purpose/concepts come from the LLM
• exports come from AST when available
• rich AST symbols are encoded as compact DSL:
  • class:Foo
  • method:bar(a: string) → number
  • call:baz
  • raise:Error
• deps are deduplicated union of AST + LLM deps

4.6 Routing metadata

src/routing-metadata.ts generates deterministic metadata used by retrieval and injection:

• tags
• symbols
• workflow hints

Caps are configurable via tagCap and workflowHintCap.

5. Patch / validate / reinit behavior

5.1 Patch

src/patch.ts:

1. resolve directory containing changed file
2. rediscover project tree
3. regenerate changed directory pair
4. refresh ancestors according to patch mode

Patch mode:

• small — refresh ancestor indexes only
• structural — refresh ancestor map/index pairs

5.2 Validate

src/validate.ts compares artifacts against filesystem and AST.

Important discrepancy types:

• missing
• orphaned
• stale-signature
• dirty
• stale-map
• stale-index
• broken-link
• structural

With --fix, validate builds a repair plan and regenerates directories deepest-first.

5.3 Reinit

reinitPath() is the blunt instrument for widespread staleness.

6. Retrieval architecture

src/retrieve.ts implements deterministic, index-first context retrieval.

1. walk the project for paired artifacts
2. parse indexes/maps into DirectoryArtifactModel
3. normalize the query
4. score every directory
5. return top-K (default: 3) as a markdown bundle with:
   • relevant indexes
   • relevant maps
   • likely files
   • relevant symbols
   • instructions to verify from source

No LLM is used during retrieval. It is intentionally separate from automatic prompt injection.

7. Prompt injection architecture

src/prompt-injection.ts and pi-extension.ts implement runtime guidance injection.

7.1 Mode ladder

Mode	Behavior
off	No automatic injection
advisory	Visible startup/init hints; no artifact preload
strong (default)	Root pair preloaded, budgeted expansion, reinjection on relevant turns
strict	Same as strong, plus bypass guard for sensitive edits/architecture reasoning without protocol path

7.2 Event hooks

The extension currently registers:

• session_start
• before_agent_start
• context

Payload fallback scanning is handled inside context-level decision logic; there is no separately registered before_provider_request hook in the current implementation.

7.3 Reinjection policy

shouldReinjectForEvent() decides whether to inject:

• only active in strong or strict
• skips if the canonical marker is already present in outgoing messages or payload
• triggers on:
  • agent_start
  • edit_intent
  • architecture_sensitive
  • compaction
  • artifact_change
• artifact_change always forces reinjection

detectEditIntent() and detectArchitectureSensitiveReasoning() provide heuristic fallback for generic turns.

7.4 Protocol path and strict bypass

The protocol path is present when outgoing context contains:

1. the canonical root-pair marker/block
2. the trust-boundary text

In strict mode, a sensitive turn without the protocol path is blocked with a visible guard. The agent can override with:

[PI_MAP_BYPASS: brief justification]

Empty or whitespace reasons are rejected.

7.5 Budgeted expansion

buildInjectionPayload():

• computes budget as min(relative, absolute)
• default is 15% of context window, capped at 100k tokens
• always includes the root pair
• adds additional pairs shallow-first until budget is exhausted
• prepends a maintenance reminder

Token estimation is best-effort: ceil(char_count / 4).

7.6 Context-window discovery

discoverContextWindow() inspects the Pi runtime model for context metadata and falls back to the absolute cap when unavailable.

8. Known limits and tradeoffs

Correctness vs cost

• init/patch/repair make LLM calls
• large repositories can be expensive
• caching reduces duplicate work

AST coverage

• TypeScript/TSX, Python, and Go have the richest support
• other languages may be partial or LLM-only

Token estimation

• 4 chars/token is only a heuristic
• oversized files may be truncated or skipped

Staleness

• there is no filesystem watcher
• maps go stale when edits happen outside the patch flow
• validate detects but does not prevent staleness

Patch mode inference

• auto-mode heuristics are good but imperfect
• contributors can force structural mode when needed

Strict mode ergonomics

• strict guards can be surprising on casual phrasing
• bypass markers are intentionally explicit and user-visible

Retrieval scoring

• deterministic scoring is reproducible but not semantic-search-smart
• broader queries may still need manual browsing

Cache

• cache grows unless manually cleaned
• corrupted cache files are recovered by starting fresh