Compare commits

...

6 Commits

Author SHA1 Message Date
alex 97a4dd22f5 docs: refresh README 2026-07-27 15:24:01 +02:00
Developer 3e7410b6bd chore: track .pi-map.md and .pi-map.index.md artifacts
Remove the map files from .gitignore so they are committed as project
navigation artifacts, and also unignore them in the sample fixture.
Regenerate all maps so the committed versions reflect the current source.
2026-06-16 14:38:42 +00:00
Developer cb581f44b9 feat: smart subtree-aware reinit and validate --fix
- project_map_reinit now regenerates only the target subtree + ancestors
  by default, falling back to full reinit when subtree file count exceeds
  reinitFullThresholdPercent (default 10%).
- Add reinitFullThresholdPercent config option.
- Expose fix=true on project_map_validate Pi tool for localized repair.
- Update docs and runtime guidance to prefer patch / validate --fix
  before full reinit.
- Add integration tests for smart reinit and update typebox mock.
2026-06-16 11:46:48 +00:00
Developer 5f1c107667 feat: avoid duplicate project-map hint injection by checking context
The before_agent_start handler now scans the active session context via
ctx.sessionManager.buildSessionContext() for an existing pi-project-map-hint
custom message and skips injection when one is already present in the
current branch. This prevents duplicate visible hints in advisory/pre-init
modes and duplicate hidden hints in strong/strict modes. The hint is
automatically re-injected after compaction or /tree navigation removes it
from the active path.

Also removes the unused hooks/on-prompt.ts prompt-text injector.
2026-06-14 08:57:23 +00:00
Developer 6bc7be4c21 feat: avoid duplicate project-map hint injection by checking context
The before_agent_start handler now inspects the current session context
and skips injection if a pi-project-map-hint custom message is already
present in the active branch. This prevents duplicate hints on every
prompt while still re-injecting after compaction or tree navigation.

Also removes the unused hooks/on-prompt.ts prompt-text injector.
2026-06-14 08:53:26 +00:00
alex fb302a033e docs: rewrite documentation system 2026-06-12 12:04:28 +02:00
61 changed files with 2420 additions and 872 deletions
+18
View File
@@ -0,0 +1,18 @@
# .atl (index)
dir: .atl
## role
Empty directory placeholder, likely intended for Atlassian tool configuration or automation artifacts that have not yet been populated.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
-
## files
## links
index: .atl/.pi-map.index.md
map: .atl/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# .atl
dir: .atl
index: .atl/.pi-map.index.md
## role
Empty directory placeholder, likely intended for Atlassian tool configuration or automation artifacts that have not yet been populated.
## files
## arch
N/A - no files or architectural patterns present in this directory.
## tags
-
## symbols
-
## workflows
-
## dirty
-
-2
View File
@@ -4,8 +4,6 @@ coverage/
*.log *.log
.DS_Store .DS_Store
.env .env
.pi-map.md
.pi-map.index.md
# Local Pi runtime state # Local Pi runtime state
.atl/ .atl/
.pi .pi
+61
View File
@@ -0,0 +1,61 @@
# . (index)
dir: .
## Project Map Protocol
1. Read this protocol and the root `.pi-map.index.md` first.
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
3. Load indexes before rich maps during task-start navigation.
4. Read the local rich map and actual source before editing.
5. Treat non-empty `## dirty` sections in either artifact as stale.
6. If source and generated artifacts disagree, trust source.
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
8. After editing source, run `project_map_patch` for each changed file.
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
Trust boundary: index routes, map orients, source decides.
## role
A TypeScript/Node.js CLI tool and Pi extension that generates paired markdown analysis artifacts (.pi-map.index.md and .pi-map.md) to provide hierarchical codebase navigation and contextual orientation for AI coding agents.
## parent
-
## children
- .atl
index: .atl/.pi-map.index.md
map: .atl/.pi-map.md
- fixtures
index: fixtures/.pi-map.index.md
map: fixtures/.pi-map.md
- openspec
index: openspec/.pi-map.index.md
map: openspec/.pi-map.md
- src
index: src/.pi-map.index.md
map: src/.pi-map.md
- tests
index: tests/.pi-map.index.md
map: tests/.pi-map.md
## files
- .gitignore
- .npmrc
- README.md
- SKILL.md
- design-doc.md
- package-lock.json
- package.json
- pi-extension.ts
- troubleshooting.md
- tsconfig.json
- usage-guide.md
## links
index: ./.pi-map.index.md
map: ./.pi-map.md
## workflows
- change project behavior
read: .gitignore, .npmrc, pi-extension.ts
- change project config
read: package-lock.json, package.json, tsconfig.json
- explore project subdirectories
index: .atl/.pi-map.index.md, fixtures/.pi-map.index.md, openspec/.pi-map.index.md
## dirty
-
+48
View File
@@ -0,0 +1,48 @@
# .
dir: .
index: ./.pi-map.index.md
## Project Map Protocol
1. Read this protocol and the root `.pi-map.index.md` first.
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
3. Load indexes before rich maps during task-start navigation.
4. Read the local rich map and actual source before editing.
5. Treat non-empty `## dirty` sections in either artifact as stale.
6. If source and generated artifacts disagree, trust source.
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
8. After editing source, run `project_map_patch` for each changed file.
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
Trust boundary: index routes, map orients, source decides.
## role
A TypeScript/Node.js CLI tool and Pi extension that generates paired markdown analysis artifacts (.pi-map.index.md and .pi-map.md) to provide hierarchical codebase navigation and contextual orientation for AI coding agents.
## files
- .gitignore | Specifies files and directories for Git to ignore in a Node.js/TypeScript project with Pi tooling integration | dep: git
- .npmrc | Configures npm to use legacy peer dependency resolution behavior | dep: npm
- README.md | Documents a CLI tool and Pi extension that generates paired machine-readable analysis artifacts (.pi-map.index.md and .pi-map.md) for hierarchical codebase navigation and agent orientation. | dep: npm, Node.js, Pi runtime environment, LLM provider (OpenAI, etc.), file system
- SKILL.md | Defines a Pi skill that generates and maintains hierarchical paired project-analysis artifacts (`.pi-map.index.md` + `.pi-map.md`) to enable AI agents to navigate codebases without reading every source file. | dep: markdown, AST parsing, LLM API, JSON configuration, CLI/tool interface
- design-doc.md | A design document explaining the internal architecture of `pi-project-map`, a TypeScript/Node.js tool that generates and maintains hierarchical paired markdown artifacts (`.pi-map.index.md` and `.pi-map.md`) to serve as navigation aids for AI coding agents, with both CLI and Pi extension runtime modes. | dep: TypeScript, Node.js, tree-sitter, ignore, LLM client (PiLLMClient/ExternalLLMClient/KimiLLMClient)
- package-lock.json | Auto-generated npm lock file that records exact dependency versions and tree structure for reproducible installs of the "pi-project-map" Node.js CLI tool. | dep: npm, esbuild, eslint, typescript, vitest, openai, tree-sitter, tree-sitter-python, tree-sitter-typescript, p-limit, picocolors, ignore
- package.json | Pi skill for hierarchical project analysis that generates and maintains .pi-map.md files | dep: ignore, openai, p-limit, picocolors, tree-sitter, tree-sitter-python, tree-sitter-typescript, typescript, vitest, eslint, @types/node, @typescript-eslint
- pi-extension.ts | Pi extension that registers tools for managing project map artifacts (.pi-map.md/.pi-map.index.md) and injects contextual hints into agent sessions based on configuration modes. | dep: @mariozechner/pi-coding-agent, typebox, fs, path, ./src/index.js, ./src/config.js, ./src/llm/llm-client.js, ./src/llm/llm-error.js
- troubleshooting.md | Troubleshooting guide for diagnosing and resolving issues with the pi-project-map tool across validation, prompt injection, strict mode, LLM provider, and testing scenarios. | dep: project_map_validate, project_map_patch, project_map_reinit, LLM client, Pi runtime, tree-sitter, npm
- tsconfig.json | Configures TypeScript compiler options for a Node.js project targeting ES2022 with strict type checking and declaration output
- usage-guide.md | User documentation explaining how to use the pi-project-map tool for navigating and maintaining project-map artifacts in codebases.
## arch
Dual-runtime architecture supporting both standalone CLI and Pi extension modes, using a hierarchical paired-file pattern (index + detail) with TypeScript/Node.js, strict type checking, and configuration-driven behavior injection for agent session integration.
## tags
map, project, tree, typescript, sitter, node, npm, js
## symbols
-
## workflows
- change project behavior
read: .gitignore, .npmrc, pi-extension.ts
- change project config
read: package-lock.json, package.json, tsconfig.json
- explore project subdirectories
index: .atl/.pi-map.index.md, fixtures/.pi-map.index.md, openspec/.pi-map.index.md
## dirty
-
+66 -116
View File
@@ -1,148 +1,98 @@
# pi-project-map # pi-project-map
Pi skill for hierarchical project analysis. A Pi extension and command-line tool that generates paired project-navigation artifacts: `.pi-map.index.md` files for routing and `.pi-map.md` files for directory orientation. The artifacts help agents navigate a codebase; source remains the authority.
## What it does ## Status and limitations
Generates **paired** project-analysis artifacts throughout your project: - Generation and repair commands call an LLM and write map artifacts into the target project.
- `validate` without `--fix` checks artifacts without creating an LLM client; `validate --fix` can modify them and requires an LLM.
- Token budgeting and relevant-turn detection are best-effort. Retrieved maps route and orient work but do not replace source verification.
- No Node.js version requirement or deployment configuration is declared in this repository.
- `.pi-map.index.md` — routing-first index for deciding what to open next ## Install and develop
- `.pi-map.md` — orientation-first rich map for understanding a directory
This gives Pi agents fast navigation plus deeper architectural context without reading every source file up front. Install the published CLI globally:
## Quick Start
```bash ```bash
npm install -g pi-project-map npm install -g pi-project-map
project-map init
``` ```
## Agent operating model For local development, install dependencies and build from this repository:
### Tier 0
Always start with:
- `Project Map Protocol`
- root `.pi-map.index.md`
### Tier 1
Load likely relevant directory indexes first, then open the strongest-match rich maps.
### Tier 2
Read actual source, tests, config, and docs before editing or making exact runtime claims.
**Trust boundary:** index routes, map orients, source decides.
## Prompt Injection Policy
The Pi extension can automatically inject lightweight project-map guidance into the agent context according to the configured `promptInjectionMode`.
### Before init
When no `.pi-map.md` / `.pi-map.index.md` artifacts exist, only a visible startup hint is injected. It tells the agent that the project-map extension is active and to run `project_map_init`. No synthetic or fake map content is ever injected before real artifacts exist.
### After init
Once real artifacts exist, the runtime guarantees that the root pair is loaded before any budgeted expansion:
- root `.pi-map.index.md`
- root `.pi-map.md`
Additional directory pairs are expanded only while the configured context budget allows, in shallow-first order.
### Trust boundary
Injected maps and indexes are navigation and orientation aids, not final authority:
> **index routes, map orients, source decides**
If an injected artifact and the source disagree, source wins. Always verify critical behavior from source before editing or making exact runtime claims.
### Mode ladder
- `off`: no automatic injection beyond existing tool/docs discovery.
- `advisory`: inject startup/init hints and allow optional root-pair preload; use light reminders.
- `strong` (default): inject the root pair, expand under the configured budget, run reinjection checks on relevant turns, and remind before edits or architecture-sensitive reasoning.
- `strict`: same as `strong`, plus require an explicit bypass justification before sensitive edits or architectural claims when the protocol path is missing.
The **protocol path** is present when the outgoing context contains the canonical injected root-pair block and the trust-boundary instruction. In `strict` mode, sensitive actions without it are blocked unless the agent includes `[PI_MAP_BYPASS: <brief justification>]`.
### Context budget
The default budget is **15% of the active model context window**, capped at **100k tokens**. The smaller of the relative and absolute values wins. If the runtime cannot discover the active model's context window, it falls back to the absolute cap.
Configure it in `.pi-project-map.json`:
```json
{
"promptInjectionMode": "strong",
"contextBudgetPercent": 15,
"contextBudgetMaxTokens": 100000
}
```
## Retrieval
When you have a specific query (e.g. "authentication logic" or "routing metadata"):
1. Run `project-map context <query>` or use the Pi tool `project_map_context`
2. Read the returned **Context bundle** — it contains relevant indexes, maps, likely files, and symbols
3. Always verify critical behavior from source before editing
Retrieval via `project_map_context` (tool) or `project-map context` (CLI) remains a separate, on-demand path. Use it for targeted navigation when you have a specific query; automatic injection does not replace it.
### Integration-test expectations and known limitations
The implementation is validated by integration tests covering:
- pre-init hint behavior,
- post-init root-pair preload,
- budgeted expansion behavior,
- canonical-marker dedupe,
- reinjection after compaction,
- reinjection after root-pair artifact changes,
- mixed visibility behavior,
- guidance-mode differences,
- strict-mode bypass behavior.
Known limitations:
- Token estimation is best-effort (≈ 4 chars per token); actual provider token counts may differ.
- Relevant-turn detection uses explicit event types when available and falls back to heuristics on generic turns.
- Message-layer scanning is preferred; provider-payload serialization quirks require a fallback scan path.
## Commands
```bash ```bash
project-map init npm install
npm run build
```
Available development checks:
```bash
npm run test
npm run lint
npm run typecheck
npm run dev
```
`npm run dev` runs TypeScript in watch mode. The project declares no deployment command.
## Use the CLI
Run the installed `project-map` command from the project to map. Use `--help` or `--version` for CLI metadata.
```bash
project-map init [path]
project-map patch <file> project-map patch <file>
project-map validate [--fix] project-map validate [--fix] [path]
project-map reinit project-map reinit [path]
project-map context <query> project-map context <query>
``` ```
### Context retrieval Typical workflow:
`project-map context <query>` searches the paired map/index artifacts and returns a compact markdown bundle with the most relevant directories, files, and symbols. No LLM call is needed — it uses deterministic metadata scoring. 1. Run `project-map init` to generate maps for a project.
2. After editing a file, run `project-map patch <file>`.
3. Run `project-map validate` before relying on maps; use `--fix` only when you intend to repair artifacts.
4. Use `project-map context "<query>"` to retrieve a ranked context bundle.
## Configuration `init`, `patch`, `reinit`, and `validate --fix` mutate `.pi-map.md` and/or `.pi-map.index.md` artifacts. `context` retrieves from existing artifacts.
Create `.pi-project-map.json` in the project root: ### Pi extension
The repository exposes `pi-extension.ts` as a Pi extension. Inside Pi it uses Pi's configured model and registers equivalent `project_map_*` tools. The extension can also inject project-map guidance according to `promptInjectionMode`.
## LLM configuration
Standalone CLI use needs credentials for the selected provider. Do not commit keys.
- `openai` (the default) reads `OPENAI_API_KEY`; its model can be set with `OPENAI_MODEL` or `LLM_MODEL`.
- `kimi` reads `KIMI_API_KEY` (or `KIMI_COM_API_KEY`); its model can be set with `KIMI_MODEL` or `LLM_MODEL`.
- CLI options `--llm-provider=openai|kimi`, `--llm-model=<model>`, and `--llm-base-url=<url>` override corresponding settings for a command.
Create an optional `.pi-project-map.json` in the project being mapped. It is merged with the defaults:
```json ```json
{ {
"ignorePatterns": ["node_modules", ".git"],
"smallPackageThreshold": 10,
"contextBudget": 4000,
"autoInjectPrompt": true,
"tagCap": 8,
"workflowHintCap": 5,
"promptInjectionMode": "strong", "promptInjectionMode": "strong",
"contextBudgetPercent": 15, "contextBudgetPercent": 15,
"contextBudgetMaxTokens": 100000 "contextBudgetMaxTokens": 100000,
"llmProvider": "openai",
"llmModel": "gpt-4o-mini",
"ignorePatterns": ["node_modules", ".git"],
"tagCap": 8,
"workflowHintCap": 5
} }
``` ```
- `promptInjectionMode`: `off`, `advisory`, `strong` (default), or `strict`. `promptInjectionMode` accepts `off`, `advisory`, `strong`, or `strict`. Supplying `ignorePatterns` **replaces** the built-in ignore list rather than extending it. Additional supported settings include `llmBaseUrl` and `reinitFullThresholdPercent`.
- `contextBudgetPercent`: relative share of the active model context window used for automatic map/index injection.
- `contextBudgetMaxTokens`: hard absolute cap on the injection budget.
## Design ## Repository layout
See [design-doc.md](design-doc.md) for the full specification. - `src/` — CLI, map generation, validation, retrieval, configuration, and LLM clients
- `pi-extension.ts` — Pi extension entry point
- `SKILL.md` — Pi skill definition and operator guidance
- `usage-guide.md`, `design-doc.md`, `troubleshooting.md` — additional usage and design documentation
- `package.json` — package metadata, dependencies, and npm scripts
## Implementation Plan ## Operations
See [implementation-plan.md](implementation-plan.md) for the engineering roadmap. There is no server or deployment manifest. Operate the tool from the project being mapped and keep generated maps under the target project's normal review and version-control practices.
+84 -66
View File
@@ -1,71 +1,95 @@
--- ---
name: pi-map name: pi-map
description: Generates and maintains hierarchical, machine-readable paired project analysis artifacts (.pi-map.index.md and .pi-map.md) for fast codebase navigation and orientation. description: Generates and maintains hierarchical paired project-analysis artifacts (`.pi-map.index.md` + `.pi-map.md`) so Pi agents can navigate and orient in a codebase without reading every source file.
--- ---
# pi-project-map # pi-project-map
A Pi skill that generates and maintains a paired analysis for each non-ignored directory: A Pi skill that creates and maintains one paired analysis artifact set per non-ignored directory:
- `.pi-map.index.md` for routing - `.pi-map.index.md` routing-first index
- `.pi-map.md` for orientation - `.pi-map.md` orientation-first rich map
## What It Does ## What it does
- **Scans** your project and creates one paired map/index artifact set per directory - scans the project and emits one paired map/index set per directory
- **Extracts** exports, imports, and dependencies via AST parsing and LLM heuristics - extracts per-file purpose, dependencies, and concepts via an LLM
- **Updates** generated artifacts after source edits - extracts exact exports/imports via AST parsing where supported
- **Validates** stale, missing, broken, or inconsistent paired artifacts - patches artifacts incrementally after source edits
- **Retrieves** relevant context on demand via deterministic metadata scoring over the paired artifacts - validates stale, missing, broken, or inconsistent paired artifacts
- retrieves relevant context on demand via deterministic metadata scoring
- injects lightweight project-map guidance into agent context according to a configurable mode ladder
## Quick Start ## Operating model
```bash ### Tier 0 — protocol
project-map init Always read the root `.pi-map.index.md` and the `Project Map Protocol` first.
project-map patch src/components/Button.tsx
project-map validate
project-map reinit
project-map context "authentication logic"
```
## Operating Model ### Tier 1 — routing
Use indexes first to decide where to go next. Open the strongest-match `.pi-map.md` files for orientation.
### Tier 0 ### Tier 2 — source
Read the root `.pi-map.index.md` and the `Project Map Protocol` first. Read actual source, tests, config, and docs before editing or asserting exact runtime behavior.
### Tier 1 **Trust boundary:**
Use indexes first for routing. Open the strongest-match `.pi-map.md` files next.
### Tier 2 > **index routes, map orients, source decides**
Read actual source before editing or asserting exact behavior.
**Trust boundary:** index routes, map orients, source decides. ## Agent instructions
## Prompt Injection Policy When project-map artifacts exist in the repo:
This skill can automatically inject lightweight project-map guidance into your context. The behavior is controlled by `promptInjectionMode` in `.pi-project-map.json`. 1. start with the root `.pi-map.index.md` and the `Project Map Protocol`
2. use indexes first to route into the right directory
3. read the local `.pi-map.md` plus relevant source before editing
4. run `project_map_patch <file>` (tool) or `project-map patch <file>` (CLI) after each source edit
5. run `project_map_validate` (tool) or `project-map validate` (CLI) before freshness-sensitive architectural decisions or final handoff
6. if validation shows localized discrepancies, run `project_map_validate` with `fix=true` (tool) or `project-map validate --fix` (CLI)
7. only if discrepancies are widespread or structural, run `project_map_reinit` (tool) or `project-map reinit` (CLI)
8. for targeted exploration, use `project_map_context <query>` (tool) or `project-map context <query>` (CLI)
9. in `strict` mode, only bypass the protocol-path guard with an explicit marker: `[PI_MAP_BYPASS: <brief justification>]`
## Prompt injection modes
The Pi extension can inject project-map guidance automatically. Behavior is controlled by `promptInjectionMode` in `.pi-project-map.json`.
### Before init ### Before init
When no `.pi-map.md` / `.pi-map.index.md` artifacts exist, you see a visible startup hint telling you to run `project_map_init`. No synthetic map content is injected before real artifacts exist. If no `.pi-map.md` / `.pi-map.index.md` artifacts exist, the extension emits a lightweight visible hint to run `project_map_init`. No synthetic or fake map content is injected.
### After init ### After init
Once artifacts exist, the runtime guarantees that the root pair is loaded first: Once real artifacts exist, the runtime guarantees that the **root pair** is loaded first:
- root `.pi-map.index.md` - root `.pi-map.index.md`
- root `.pi-map.md` - root `.pi-map.md`
Additional directory pairs may be added while the configured context budget allows. Additional directory pairs may be expanded while the configured context budget allows, in shallow-first order.
### Mode ladder ### Mode ladder
- `off`: no automatic injection beyond existing tool/docs discovery.
- `advisory`: startup/init hints are shown; you may read the root pair manually when you want routing/orientation context.
- `strong` (default): the root pair is injected automatically, expansion stays within the context budget, and reinjection checks run on relevant turns (agent start, before edits, before architecture-sensitive reasoning, after compaction, after root-pair artifact changes).
- `strict`: same as `strong`, but before sensitive edits or architectural claims you must either have the protocol path in context or include an explicit bypass marker: `[PI_MAP_BYPASS: <brief justification>]`.
The **protocol path** means the outgoing context contains the canonical injected root-pair block and the trust-boundary instruction (`index routes, map orients, source decides`). | Mode | Behavior |
|------|----------|
| `off` | No automatic injection. Use tools/CLI manually. |
| `advisory` | Startup/init hints are shown. Root pair is not auto-loaded; read it manually when needed. |
| `strong` (default) | Root pair is auto-loaded, expansion stays within budget, and reinjection runs on relevant turns. |
| `strict` | Same as `strong`, but sensitive edits or architectural claims are guarded unless the protocol path is present or a bypass marker is provided. |
The **protocol path** means the outgoing context contains the canonical injected root-pair block and the trust-boundary text.
### Context budget ### Context budget
Default budget: **15% of the active model context window**, capped at **100k tokens**. The smaller of the relative and absolute values wins. If the runtime cannot discover the active model's context window, it uses the absolute cap. Default budget: **15% of the active model context window**, capped at **100k tokens**. The smaller of the relative and absolute values wins. If the runtime cannot discover the context window, it uses the absolute cap.
## Retrieval usage
`project_map_context` (tool) and `project-map context` (CLI) are **on-demand retrieval**, separate from automatic injection.
They do not call the LLM. They score paired metadata against the query and return a compact markdown bundle with:
- relevant indexes
- relevant maps
- likely files
- relevant symbols when useful
Always read the suggested indexes first, then maps, then verify critical behavior from source.
## Configuration ## Configuration
@@ -73,41 +97,35 @@ Create `.pi-project-map.json` in the project root:
```json ```json
{ {
"tagCap": 8,
"workflowHintCap": 5,
"promptInjectionMode": "strong", "promptInjectionMode": "strong",
"contextBudgetPercent": 15, "contextBudgetPercent": 15,
"contextBudgetMaxTokens": 100000 "contextBudgetMaxTokens": 100000,
"tagCap": 8,
"workflowHintCap": 5,
"llmProvider": "openai",
"llmModel": "gpt-4o-mini",
"reinitFullThresholdPercent": 10,
"ignorePatterns": ["node_modules", ".git", "dist", "build"]
} }
``` ```
- `promptInjectionMode`: `off`, `advisory`, `strong` (default), or `strict`. Providing `ignorePatterns` replaces the built-in default list, so include any defaults you want to keep.
- `contextBudgetPercent`: relative share of the active model context window to use for automatic map/index injection.
- `contextBudgetMaxTokens`: hard absolute cap on the injection budget.
## Agent Instructions - `reinitFullThresholdPercent` — when `project_map_reinit` targets a subtree that covers more than this percentage of project files, it falls back to full regeneration
When project map artifacts exist in the repo: Knobs that matter in practice:
- `promptInjectionMode``off` | `advisory` | `strong` | `strict`
- `contextBudgetPercent` / `contextBudgetMaxTokens` — caps automatic map/index injection
- `tagCap` / `workflowHintCap` — caps routing metadata per directory
- `llmProvider` / `llmModel` / `llmBaseUrl` — standalone CLI only
- `ignorePatterns` — discovery exclusions
1. Start with the root `.pi-map.index.md` ## Tools and commands
2. Use indexes first to route into the right directory
3. Read the local `.pi-map.md` plus source before editing
4. Run `project-map patch <path>` after editing source
5. Run `project-map validate` before freshness-sensitive architectural decisions
6. In `strict` mode, include `[PI_MAP_BYPASS: <brief justification>]` only when you deliberately need to proceed without the protocol path
7. For **targeted navigation**, use `project_map_context` (Pi tool) or `project-map context` (CLI) with a natural-language query. It returns a compact markdown bundle with the strongest-match indexes, maps, likely files, and symbols.
## Retrieval Model | Tool (Pi) | CLI command | Purpose |
|-----------|-------------|---------|
`project_map_context` and `project-map context` implement **index-first retrieval**: | `project_map_init` | `project-map init [path]` | Generate all paired artifacts. |
| `project_map_patch` | `project-map patch <file>` | Regenerate the pair for the changed file's directory and refresh ancestors. |
1. Score every directory's paired map/index metadata against the query | `project_map_validate` | `project-map validate [--fix]` | Check paired artifacts for staleness and discrepancies; optionally repair. |
2. Keep the top 3 strongest matches | `project_map_reinit` | `project-map reinit [path]` | Smart regeneration: recomputes the target subtree plus ancestors, and only falls back to full regeneration when the subtree covers more than `reinitFullThresholdPercent` of project files (default 10%). |
3. Expand those matches into: | `project_map_context` | `project-map context <query>` | Retrieve a ranked context bundle for a natural-language query. |
- Relevant indexes (routing-first)
- Relevant maps (orientation-first)
- Likely files
- Relevant symbols (only when useful)
4. Return a stable markdown bundle titled `# Context bundle: <query>`
**Trust boundary still applies:** the bundle routes and orients, but source decides. Always read actual source before editing.
+290 -396
View File
@@ -1,441 +1,335 @@
# Design Doc: Hierarchical Project Analysis Skill for Pi # Design Reference: pi-project-map
## 1. Goals and Success Criteria > Audience: maintainers and contributors.
> Purpose: explain how `pi-project-map` works internally, not how to install or use it.
### Primary Goal ## 1. Overview
Enable a Pi coding agent to understand a software project's architecture and code relationships without scanning the entire repository. The agent should have a compact, hierarchical "internal representation" of the project that it can consume in-context.
### Success Criteria `pi-project-map` is a TypeScript/Node.js skill package that generates and maintains hierarchical, paired project-map artifacts for AI coding agents:
- The agent can orient itself in a new or familiar project without reading dozens of source files.
- The agent understands cross-package dependencies, data flows, and architectural patterns from the analysis files alone.
- Analysis files stay sufficiently fresh that the agent does not make decisions based on stale information.
- The representation is token-dense: maximum information per token, optimized for LLM consumption, not human readability.
## 2. Format Specification: Dense Markdown with Conventions - `.pi-map.index.md` — routing-first, sparse directory metadata
- `.pi-map.md` — orientation-first, richer directory metadata
### Design Rationale It runs as both:
- **Not JSON/YAML**: Brackets, quotes, and indentation add token overhead with no benefit to LLM comprehension. - a standalone CLI (`project-map`)
- **Not a custom DSL**: Fragile, requires a parser, and LLMs may hallucinate syntax. - a Pi extension (`pi-extension.ts`)
- **Dense markdown**: Hierarchical headings, bullet points, and abbreviations are natively understood by LLMs and extremely token-efficient.
### Structure The extension registers tools and event hooks that keep the artifacts fresh and can inject them into agent context at runtime.
Each non-ignored directory in the project gets **two** hidden analysis files:
- `.pi-map.index.md` — routing-first index ### Core design principle
- `.pi-map.md` — orientation-first rich map
```markdown The artifacts are **navigation aids, not source-of-truth**. Source code is always the final authority.
# <relative-path> (index)
dir: <relative-path>
## role
<short routing summary>
## parent
<parent links or ->
## children
<child links or ->
## files
<likely files>
## links
index: <self-index>
map: <self-map>
## workflows
<compact task -> route hints>
## dirty
<timestamp or ->
```
```markdown > **index routes, map orients, source decides.**
# <relative-path>
dir: <relative-path>
index: <sibling-index>
## role
<one-line package role>
## files
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
## arch
<free-form architectural notes>
## tags
<compact tags>
## symbols
<prioritized symbols>
## workflows
<compact workflow hints>
## dirty
<timestamp or ->
```
### Abbreviation Conventions ## 2. Artifact model
| Abbreviation | Meaning |
|-------------|---------|
| `exp:` | exported symbols (functions, classes, types, constants) |
| `dep:` | dependencies (other packages, files, or external libs) |
| `pkg/` | project-internal package reference |
| `ext/` | external dependency reference |
| `->` | data flow direction |
| `|` | field delimiter within a line |
### Example Each non-ignored directory receives a matched pair.
```markdown ### 2.1 Shared model
# pkg/auth
## role
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
## files
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
- types.ts | shared auth types | exp: AuthToken, UserClaims, Role
## arch
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
Session state stored in Redis via db/sessions. No server-side JWT storage.
## dirty
-
```
### Rules Both files are generated from the same in-memory `DirectoryArtifactModel`:
- One file per directory, placed inside that directory.
- Every non-excluded file in the directory gets one bullet under `## files`.
- Subdirectories are referenced in `## role` via `Dep:` or in `## arch` as structural notes, not duplicated.
- The `## dirty` section is empty (`-`) when clean, or contains a timestamp/flag when stale.
## 3. Pipeline Architecture ```ts
interface DirectoryArtifactModel {
dir: string;
role: string;
files: FileEntry[];
arch: string;
dirty?: string;
isRoot: boolean;
parent?: string;
children: string[];
tags: string[];
symbols: string[];
workflows: WorkflowHint[];
}
### Hybrid Extraction: LLM + AST interface FileEntry {
name: string;
Two independent extraction layers contribute to the same output file. purpose: string;
exports: string[];
#### Layer 1: LLM-Based Extraction (All Files) deps: string[];
- **Input**: Raw file contents of every non-excluded file in the directory.
- **Output**: Purpose description, architectural role, and cross-file relationships.
- **Applies to**: Code files, config files, Dockerfiles, READMEs, YAML, JSON, shell scripts — everything.
- **When it runs**: Once per file during init; again on changed files during patching.
- **Implementation**: Calls an actual LLM (not regex heuristics). Inside Pi, it uses Pi's built-in LLM via the ExtensionAPI. Standalone CLI falls back to an external LLM API (OpenAI-compatible).
#### Layer 2: AST-Based Extraction (Code Files Only)
- **Input**: Source code of files where a tree-sitter or LSP parser is available.
- **Output**: Precise symbol lists (functions, classes, types), signatures, import/export graphs, class hierarchies.
- **Applies to**: Supported languages only (TypeScript, Python, Go, Rust, etc.).
- **When it runs**: Once per file during init; again on changed files during patching.
#### Merging
The two layers merge into a single line per file under `## files`:
```
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
^ LLM ^ LLM ^ AST ^ AST + LLM
```
- File name and purpose: LLM.
- Exported symbols and signatures: AST (augmented by LLM if AST unavailable).
- Dependency list: AST for imports; LLM for inferred architectural dependencies.
### LLM Client Architecture
The LLM client is abstracted behind a unified interface:
```typescript
interface LLMClient {
complete(prompt: string): Promise<string>;
} }
``` ```
Two implementations: ### 2.2 `.pi-map.md` (rich map)
1. **PiLLMClient** (Pi extension): Uses `ctx.model` or `ctx.modelRegistry` to invoke Pi's configured LLM. Called from `pi-extension.ts` when the skill runs inside Pi. Rendered by `src/format.ts` `renderDirectoryMap()`.
2. **ExternalLLMClient** (standalone CLI): Calls an external OpenAI-compatible API. Configured via environment variable (e.g., `OPENAI_API_KEY`) or config file.
### Caching Contains:
- `dir:` line and sibling `index:` link
- `Project Map Protocol` (root only)
- `## role`
- `## files`
- `## arch`
- `## tags`
- `## symbols`
- `## workflows`
- `## dirty`
LLM results are cached to avoid re-querying unchanged files. ### 2.3 `.pi-map.index.md` (index)
- **Key**: SHA-256 hash of file contents. Rendered by `src/format.ts``renderDirectoryIndex()`.
- **Storage**: JSON file at `~/.cache/pi-project-map/llm-cache.json`.
- **Behavior**: Before calling the LLM, compute the file hash and check the cache. If hit, reuse the cached result. If miss, call the LLM and store the result.
- **Invalidation**: Cache entries are implicitly invalidated when the file content changes (because the hash changes). There is no TTL; the cache is append-only.
### Parallelization and Rate Limiting Contains:
- same protocol (root only)
- `## role`
- `## parent`
- `## children`
- `## files`
- `## links`
- `## workflows`
- `## dirty`
- **Concurrency**: 4-8 LLM calls in parallel, controlled by `p-limit`. ### 2.4 Why a paired format?
- **Batch delays**: A small delay (e.g., 100ms) is inserted between batches to avoid triggering rate limits.
- **Retry policy**: Each LLM call retries up to 3 times with exponential backoff (1s, 2s, 4s). If all retries fail, the entire operation stops with a hard error.
### Error Handling - **indexes are cheap** — many can be loaded without consuming much context
- **maps are dense** — loaded only after an index suggests relevance
- **paired generation** guarantees structural consistency
- **Hard error on failure**: If an LLM call fails after all retries, `init` or `patch` stops immediately and prints a clear error. There is no heuristic fallback. The user must resolve the issue (set API key, wait for rate limit, check network). ## 3. High-level architecture
- **Context limit protection**: Files larger than the LLM's context window are truncated from the end (with a note in the prompt) before being sent.
### Init Pipeline ### 3.1 Main modules
```
For each directory (depth-first): ```text
1. List all non-excluded files. discover → directory tree, .gitignore-aware
2. For each file (parallel, 4-8 concurrent): init → full generation
a. Compute SHA-256 of file contents. llm-extract → LLM-based file/package analysis
b. Check disk cache. If hit, use cached result. ast-extract → tree-sitter parsing
c. If miss: call LLM (with retries/backoff) to extract purpose and role. merge → combine LLM + AST into FileEntry
d. Store result in cache. routing-metadata → tags, symbols, workflow hints
e. If code file + parser available: run AST extraction (symbols, imports). format → render / parse the markdown pair
3. Merge per-file outputs into lines. patch → incremental update after edits
4. Run LLM on merged lines + directory context to generate: validate → consistency checking with optional repair
- `## role` (package-level summary) retrieve → deterministic query scoring
- `## arch` (architectural notes) prompt-injection → runtime context policy
5. Write `.pi-map.md` to directory. pi-extension → Pi tool/event registration
cli → standalone command dispatcher
config → defaults and .pi-project-map.json loader
``` ```
### Patch Pipeline ### 3.2 Runtime modes
```
When agent edits file(s): | Mode | Entry point | LLM client |
1. Classify patch mode: auto, small, or structural. |------|-------------|------------|
2. Always regenerate the changed directory pair: | Pi extension | `pi-extension.ts` | `PiLLMClient` via Pi runtime |
- `.pi-map.md` | Standalone CLI | `src/cli/cli.ts` | `ExternalLLMClient` or `KimiLLMClient` |
- `.pi-map.index.md`
3. For small changes: refresh ancestor indexes. ## 4. Extraction pipeline
4. For structural changes: refresh ancestor map/index pairs.
5. Use `validate --fix` to repair affected chains when paired artifacts are stale or missing. ### 4.1 Discovery
`src/discover.ts` walks the filesystem with `ignore`, merging built-in exclusions and `.gitignore`.
### 4.2 Per-directory generation
`src/init.ts``generateDirectoryArtifacts()`:
1. `processFiles()` runs in parallel over directory files
2. for each file:
- `extractFileLLM()` gets `purpose`, `deps`, `concepts`
- `extractFileAST()` gets exports/imports/calls where possible
- `mergeFileData()` combines both into a `FileEntry`
3. `extractPackageLLM()` produces directory `role` and `arch`
4. `createDirectoryModel()` builds the shared model
5. `populateRoutingMetadata()` derives `tags`, `symbols`, `workflows`
6. `writeDirectoryArtifacts()` writes both `.pi-map.md` and `.pi-map.index.md`
Directories are processed sequentially; files within a directory are processed concurrently.
### 4.3 LLM extraction
`src/llm/llm-extract.ts`:
- prompts are minimal and line-oriented
- binary files are skipped
- files over 500KB are labeled large and skipped
- source is truncated before prompting
- results are cached by SHA-256 of file content
- missing client throws `LLMError`
### 4.4 AST extraction
`src/ast/ast-extract.ts` uses `tree-sitter` for supported languages to extract:
- imports / requires
- exported classes, functions, constants
- methods, parameters, return types
- direct calls and raised exceptions
Unsupported languages fall back to LLM-only extraction.
### 4.5 Merging
`src/merge.ts`:
- purpose/concepts come from the LLM
- exports come from AST when available
- rich AST symbols are encoded as compact DSL:
- `class:Foo`
- `method:bar(a: string) → number`
- `call:baz`
- `raise:Error`
- deps are deduplicated union of AST + LLM deps
### 4.6 Routing metadata
`src/routing-metadata.ts` generates deterministic metadata used by retrieval and injection:
- **tags**
- **symbols**
- **workflow hints**
Caps are configurable via `tagCap` and `workflowHintCap`.
## 5. Patch / validate / reinit behavior
### 5.1 Patch
`src/patch.ts`:
1. resolve directory containing changed file
2. rediscover project tree
3. regenerate changed directory pair
4. refresh ancestors according to patch mode
Patch mode:
- **small** — refresh ancestor indexes only
- **structural** — refresh ancestor map/index pairs
### 5.2 Validate
`src/validate.ts` compares artifacts against filesystem and AST.
Important discrepancy types:
- `missing`
- `orphaned`
- `stale-signature`
- `dirty`
- `stale-map`
- `stale-index`
- `broken-link`
- `structural`
With `--fix`, validate builds a repair plan and regenerates directories deepest-first.
### 5.3 Reinit
`reinitPath()` is the blunt instrument for widespread staleness.
## 6. Retrieval architecture
`src/retrieve.ts` implements deterministic, index-first context retrieval.
1. walk the project for paired artifacts
2. parse indexes/maps into `DirectoryArtifactModel`
3. normalize the query
4. score every directory
5. return top-K (default: 3) as a markdown bundle with:
- relevant indexes
- relevant maps
- likely files
- relevant symbols
- instructions to verify from source
No LLM is used during retrieval. It is intentionally separate from automatic prompt injection.
## 7. Prompt injection architecture
`src/prompt-injection.ts` and `pi-extension.ts` implement runtime guidance injection.
### 7.1 Mode ladder
| Mode | Behavior |
|------|----------|
| `off` | No automatic injection |
| `advisory` | Visible startup/init hints; no artifact preload |
| `strong` (default) | Root pair preloaded, budgeted expansion, reinjection on relevant turns |
| `strict` | Same as strong, plus bypass guard for sensitive edits/architecture reasoning without protocol path |
### 7.2 Event hooks
The extension currently registers:
- `session_start`
- `before_agent_start`
- `context`
Payload fallback scanning is handled inside `context`-level decision logic; there is no separately registered `before_provider_request` hook in the current implementation.
### 7.3 Reinjection policy
`shouldReinjectForEvent()` decides whether to inject:
- only active in `strong` or `strict`
- skips if the canonical marker is already present in outgoing messages or payload
- triggers on:
- `agent_start`
- `edit_intent`
- `architecture_sensitive`
- `compaction`
- `artifact_change`
- `artifact_change` always forces reinjection
`detectEditIntent()` and `detectArchitectureSensitiveReasoning()` provide heuristic fallback for generic turns.
In addition to marker-based deduplication, `before_agent_start` scans the active session context via `ctx.sessionManager.buildSessionContext()` for an existing `pi-project-map-hint` custom message. If one is already present in the current branch, the handler skips injection entirely. This prevents duplicate visible hints in advisory/pre-init modes and duplicate hidden hints in strong/strict modes when the session context already contains the guidance. The hint is automatically re-injected after compaction or `/tree` navigation removes it from the active path.
### 7.4 Protocol path and strict bypass
The **protocol path** is present when outgoing context contains:
1. the canonical root-pair marker/block
2. the trust-boundary text
In `strict` mode, a sensitive turn without the protocol path is blocked with a visible guard. The agent can override with:
```text
[PI_MAP_BYPASS: brief justification]
``` ```
## 4. LLM Prompt Design Empty or whitespace reasons are rejected.
### File-Level Prompt ### 7.5 Budgeted expansion
The LLM prompt for a single file is designed to produce a structured, concise analysis. `buildInjectionPayload()`:
- computes budget as `min(relative, absolute)`
- default is 15% of context window, capped at 100k tokens
- always includes the root pair
- adds additional pairs shallow-first until budget is exhausted
- prepends a maintenance reminder
``` Token estimation is best-effort: `ceil(char_count / 4)`.
You are analyzing a source file for a project map. Read the file below and summarize:
1. PURPOSE: What does this file do? Describe its role in the project (2-3 sentences max). ### 7.6 Context-window discovery
2. DEPENDENCIES: What does this file depend on? List internal modules/packages and external libraries.
3. KEY CONCEPTS: Mention any important patterns, algorithms, or domain concepts.
File path: <file-path> `discoverContextWindow()` inspects the Pi runtime model for context metadata and falls back to the absolute cap when unavailable.
``` ## 8. Known limits and tradeoffs
<file-contents-truncated>
```
Respond in this exact format: ### Correctness vs cost
PURPOSE: <concise description> - init/patch/repair make LLM calls
DEPS: <comma-separated list, or "none"> - large repositories can be expensive
CONCEPTS: <comma-separated list, or "none"> - caching reduces duplicate work
```
### Package-Level Prompt ### AST coverage
- TypeScript/TSX, Python, and Go have the richest support
- other languages may be partial or LLM-only
After all file summaries are collected for a directory, a second LLM call synthesizes the package role and architecture. ### Token estimation
- 4 chars/token is only a heuristic
- oversized files may be truncated or skipped
``` ### Staleness
You are analyzing a directory in a software project. Below is a list of files in this directory with their purposes. - there is no filesystem watcher
- maps go stale when edits happen outside the patch flow
- validate detects but does not prevent staleness
Directory: <dir-path> ### Patch mode inference
Files: - auto-mode heuristics are good but imperfect
- <file1>: <purpose1> - contributors can force structural mode when needed
- <file2>: <purpose2>
...
Respond in this exact format: ### Strict mode ergonomics
ROLE: <one-line description of this directory's role in the project> - strict guards can be surprising on casual phrasing
ARCH: <2-4 sentences describing architecture, data flow, patterns, and design decisions> - bypass markers are intentionally explicit and user-visible
```
### Output Parsing ### Retrieval scoring
- deterministic scoring is reproducible but not semantic-search-smart
- broader queries may still need manual browsing
The LLM client's response is parsed to extract `PURPOSE`, `DEPS`, `CONCEPTS`, `ROLE`, and `ARCH` fields. These are merged with AST data into the final `.pi-map.md` format. ### Cache
- cache grows unless manually cleaned
### Context Limit Protection - corrupted cache files are recovered by starting fresh
- Files are truncated from the end if they exceed a configurable max token budget (default: 4000 tokens of source).
- A marker `[...truncated]` is appended to the truncated content so the LLM knows it is not seeing the full file.
- Very large binary or generated files are skipped entirely for LLM analysis (they still appear in `.pi-map.md` with a note like "Large/generated file").
## 5. Consumption Model
### Session Start
1. Agent reads the root `Project Map Protocol` and root `.pi-map.index.md`.
2. For architecture/system or ambiguous tasks, agent also reads the root `.pi-map.md`.
3. Agent does **not** preload every directory map by default.
In the Pi extension, this session-start consumption is assisted by automatic prompt injection:
- **Before init**: only a lightweight visible startup hint is injected, telling the agent to run `project_map_init`. No synthetic map content is injected.
- **After init**: the root pair (`.pi-map.index.md` + `.pi-map.md`) is guaranteed to be preloaded automatically. Additional directory pairs are expanded only while the configured context budget allows.
### Automatic Prompt Injection
The Pi extension uses event hooks (`before_agent_start`, `context`, etc.) to maintain guidance context.
- The **mode ladder** controls how much is injected:
- `off`: no automatic injection.
- `advisory`: visible startup/init hints and optional root-pair preload.
- `strong` (default): root pair + budgeted expansion + relevant-turn reinjection checks.
- `strict`: same as `strong`, plus explicit bypass justification for sensitive edits/architectural claims when the protocol path is missing.
- The **protocol path** requires both the canonical injected root-pair block and the trust-boundary instruction (`index routes, map orients, source decides`) to be present in outgoing context.
- **Reinjection avoidance** scans actual outgoing messages (and falls back to provider payload) for a stable canonical marker before adding the root pair again.
- **Relevant-turn triggers** are: agent start, before edits, before architecture-sensitive reasoning, after compaction, and after root-pair artifact changes.
- **Visibility** is mixed: startup/init hints are user-visible; raw injected artifact blocks are agent-visible by default.
### Context Budget
Default automatic-injection budget: **15% of the active model context window**, capped at **100k tokens**. The smaller of the relative and absolute values wins. If the runtime cannot discover the active model's context window, it falls back to the absolute cap.
Configurable via `.pi-project-map.json`:
```json
{
"promptInjectionMode": "strong",
"contextBudgetPercent": 15,
"contextBudgetMaxTokens": 100000
}
```
### During Session
- Use directory indexes first to decide what to open next.
- For **targeted queries**, run `project_map_context` (tool) or `project-map context` (CLI). The retrieval engine scores all paired metadata and returns a compact markdown bundle with the top-3 strongest matches: indexes, maps, likely files, and symbols.
- Open the strongest-match `.pi-map.md` files for richer orientation.
- Read actual source before editing or making exact runtime claims.
### Context Management
- Tier 0 stays tiny and stable.
- Tier 1 loads only likely relevant indexes/maps.
- Tier 2 is real source, tests, config, and docs.
- Automatic injection stays within the configured budget and avoids redundant reinjection by scanning outgoing context.
## 6. Stale Data Mitigation
### Combined Strategy
#### 5.1 Dirty Markers
- Whenever the agent edits a file, it appends a dirty flag to the directory's `.pi-map.md`:
```markdown
## dirty
2024-06-09T14:32:00Z: tokens.ts modified
```
- A background or post-session reconciliation step regenerates dirty files.
- The agent can also be instructed to reconcile before making architectural decisions.
#### 5.2 Periodic Full Re-init
- On every new session start, or on a configurable schedule (e.g., daily), the skill offers to run a full re-scan.
- This catches any changes made outside the agent's awareness (e.g., by other developers).
#### 5.3 Validation Command
- A `validate` tool/command that the agent can invoke:
- Checks for missing files (new files not in `.pi-map.md`).
- Checks for orphaned entries (files listed but deleted).
- Checks for changed signatures (AST mismatch between listed symbols and actual code).
- Reports discrepancies and suggests corrections.
### Recovery
- If validation finds staleness beyond a threshold (e.g., > 3 dirty packages), the skill recommends a full re-init.
- The agent can also trigger re-init for a specific subtree.
## 7. Scope Boundaries and Non-Goals
### In Scope
- Every directory in the project gets a `.pi-map.md` file.
- Every non-excluded file gets analyzed by the LLM layer.
- Code files get augmented by the AST layer where parsers exist.
- Respect `.gitignore` and known junk patterns (node_modules, .git, dist, build, coverage, .next, .venv, __pycache__, .DS_Store).
### Out of Scope (Non-Goals)
- **Human-readable documentation**: These files are machine-only. Human docs live elsewhere.
- **Line-by-line code explanation**: The format captures symbols and architecture, not implementation details.
- **Auto-regeneration on filesystem events**: The skill relies on agent-initiated updates and periodic re-init, not filesystem watchers.
- **Cross-project analysis**: Each project is independent. No global index across repos.
- **IDE integration**: This is a Pi agent skill, not a VS Code extension or LSP server.
## 8. Pi Skill Package Structure
```
pi-project-map/
├── SKILL.md # Skill definition for Pi
├── package.json # npm package metadata
├── src/
│ ├── init.ts # Full project scan + generation
│ ├── patch.ts # Incremental patch logic
│ ├── validate.ts # Consistency checker
│ ├── ast-extract.ts # Tree-sitter / LSP wrappers
│ ├── llm-extract.ts # LLM prompt templates for extraction
│ ├── merge.ts # Merge AST + LLM outputs
│ ├── format.ts # Dense markdown formatter
│ ├── config.ts # Skill configuration (thresholds, ignore patterns)
│ └── prompt-injection.ts # Runtime guidance injection policy and helpers
├── hooks/
│ └── on-prompt.ts # Injects maintenance command into prompts (legacy; Pi extension uses event hooks)
└── README.md # Setup and usage for humans
```
### Custom Tools
- `project-map:init` — Run full project scan. Creates all paired map/index artifacts.
- `project-map:patch <file-path>` — Update analysis for a specific file/directory.
- `project-map:validate` — Run consistency check across all paired artifacts.
- `project-map:context <query>` — Retrieve a compact markdown bundle of the most relevant directories, files, and symbols for a natural-language query.
- `project-map:reinit [path]` — Force re-initialization of entire project or subtree.
### Prompt Injection Hooks
The Pi extension registers event hooks instead of a single per-prompt append:
- `before_agent_start`: emits the pre-init hint when no artifacts exist, or preloads the root pair (plus budgeted expansion) after init.
- `context`: performs relevant-turn reinjection checks, detects compaction/artifact-change invalidation, and enforces `strict`-mode bypass guards.
- `before_provider_request`: optional fallback for marker scanning when message-layer detection is insufficient.
The injected maintenance reminder is:
> Start with the root `.pi-map.index.md`, use indexes first for routing, read the local `.pi-map.md` plus source before edits, run `project_map_patch` after source edits, and run `project_map_validate` before freshness-sensitive architectural handoff.
This is layered on top of the canonical root-pair block, which includes the trust boundary (`index routes, map orients, source decides`).
## 9. Risks and Tradeoffs
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Token bloat (1000+ dirs) | Medium | High | Summary mode, lazy loading, context budget |
| Stale analysis files | High | High | Dirty markers + periodic re-init + validation |
| Agent trusts stale data | Medium | High | Clear instructions to validate before architectural decisions |
| Expensive init on large repos | Medium | Medium | Parallelization, caching, optional incremental init |
| Overlap with LSP/typedoc | Low | Low | This is agent-context, not IDE tooling. Different use case. |
| AST parser unavailable | Medium | Low | Graceful fallback to LLM-only extraction |
| Message-level marker scanning misses provider serialization quirks | Medium | Medium | Add payload fallback scanning |
| Root-pair marker becomes brittle | Low | Medium | Stable deterministic boundaries and normalized artifact identity lines |
| 15% / 100k default budget too aggressive for some fleets | Low | Medium | Both knobs are configurable |
| Relevant-turn detection fuzzy | Medium | Medium | Centralized heuristics + extensive integration tests |
| `strict` mode friction | Low | Medium | Keep `strong` as default; isolate strict-only bypass behavior |
### Prompt Injection Known Limitations
- Token estimation is best-effort (≈ 4 chars per token); actual provider token counts may differ.
- Relevant-turn detection relies on explicit event types when available, with heuristic fallback for generic turns.
- Provider payload serialization may require the fallback scan path.
- The mode ladder is config-driven in v1; future UX may expose runtime controls.
- Retrieval (`project_map_context`) remains separate from automatic injection.
## 10. Concrete Example: Full Project Snapshot
```
project-root/
├── .pi-map.md
├── src/
│ ├── .pi-map.md
│ ├── auth/
│ │ ├── .pi-map.md
│ │ ├── tokens.ts
│ │ ├── middleware.ts
│ │ └── types.ts
│ └── db/
│ ├── .pi-map.md
│ ├── connection.ts
│ └── migrations/
│ ├── .pi-map.md
│ └── 001_init.sql
├── docker/
│ ├── .pi-map.md
│ ├── Dockerfile
│ └── docker-compose.yml
└── README.md
```
Each `.pi-map.md` follows the format in Section 2, creating a navigable hierarchy.
## 11. Future Extensions
- **Cross-reference graph**: A top-level `project-graph.md` linking all packages with dependency arrows.
- **Search index**: A lightweight FTS5 index over all `.pi-map.md` files for fast symbol lookup.
- **Diff-aware patching**: Only re-run LLM on changed functions, not entire files.
- **Multi-repo workspaces**: Support monorepos with independent package boundaries.
+20
View File
@@ -0,0 +1,20 @@
# fixtures (index)
dir: fixtures
## role
Provides test data and setup utilities for automated testing across the project.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
- fixtures/sample-project
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## files
## links
index: fixtures/.pi-map.index.md
map: fixtures/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# fixtures
dir: fixtures
index: fixtures/.pi-map.index.md
## role
Provides test data and setup utilities for automated testing across the project.
## files
## arch
Simple static data fixtures with possible factory/helper patterns for consistent test state generation.
## tags
-
## symbols
-
## workflows
-
## dirty
-
-1
View File
@@ -1,4 +1,3 @@
dist/ dist/
node_modules/ node_modules/
.pi-map.md
.env .env
+30
View File
@@ -0,0 +1,30 @@
# fixtures/sample-project (index)
dir: fixtures/sample-project
## role
Provides a minimal sample Node.js/TypeScript project fixture for testing and demonstrating the pi-project-map functionality.
## parent
index: fixtures/.pi-map.index.md
map: fixtures/.pi-map.md
## children
- fixtures/sample-project/docs
index: fixtures/sample-project/docs/.pi-map.index.md
map: fixtures/sample-project/docs/.pi-map.md
- fixtures/sample-project/src
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
- fixtures/sample-project/tests
index: fixtures/sample-project/tests/.pi-map.index.md
map: fixtures/sample-project/tests/.pi-map.md
## files
- .gitignore
- README.md
- package.json
- tsconfig.json
## links
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## workflows
-
## dirty
-
+22
View File
@@ -0,0 +1,22 @@
# fixtures/sample-project
dir: fixtures/sample-project
index: fixtures/sample-project/.pi-map.index.md
## role
Provides a minimal sample Node.js/TypeScript project fixture for testing and demonstrating the pi-project-map functionality.
## files
- .gitignore | Specifies files and directories for Git to ignore in version control | dep: git
- README.md | Provides a brief overview and directory structure for a small test project related to pi-project-map functionality.
- package.json | Defines a sample Node.js project configuration with TypeScript build and Vitest testing scripts. | dep: typescript, vitest
- tsconfig.json | Configures TypeScript compiler options for a Node.js project targeting ES2022 with strict type checking.
## arch
Standard Node.js project structure using TypeScript with strict compilation, Vitest for testing, and ES2022 module output.
## tags
project, typescript, git, readme, node, vitest, package, tsconfig
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,19 @@
# fixtures/sample-project/docs (index)
dir: fixtures/sample-project/docs
## role
Provides API documentation and usage examples for the sample project's user management functionality.
## parent
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## children
-
## files
- API.md
## links
index: fixtures/sample-project/docs/.pi-map.index.md
map: fixtures/sample-project/docs/.pi-map.md
## workflows
-
## dirty
-
+19
View File
@@ -0,0 +1,19 @@
# fixtures/sample-project/docs
dir: fixtures/sample-project/docs
index: fixtures/sample-project/docs/.pi-map.index.md
## role
Provides API documentation and usage examples for the sample project's user management functionality.
## files
- API.md | Documents a user management API with user creation/serialization functions and validation utilities
## arch
Documentation-as-code pattern with markdown-based reference material for external API consumers.
## tags
api, user, documents, management, creation, serialization, validation, utilities
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,30 @@
# fixtures/sample-project/src (index)
dir: fixtures/sample-project/src
## role
Entry point module that demonstrates user creation, email validation, and logging for a sample project.
## parent
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## children
- fixtures/sample-project/src/components
index: fixtures/sample-project/src/components/.pi-map.index.md
map: fixtures/sample-project/src/components/.pi-map.md
- fixtures/sample-project/src/models
index: fixtures/sample-project/src/models/.pi-map.index.md
map: fixtures/sample-project/src/models/.pi-map.md
- fixtures/sample-project/src/utils
index: fixtures/sample-project/src/utils/.pi-map.index.md
map: fixtures/sample-project/src/utils/.pi-map.md
## files
- index.ts
## links
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## workflows
- change src behavior
read: index.ts
- explore src subdirectories
index: fixtures/sample-project/src/components/.pi-map.index.md, fixtures/sample-project/src/models/.pi-map.index.md, fixtures/sample-project/src/utils/.pi-map.index.md
## dirty
-
+26
View File
@@ -0,0 +1,26 @@
# fixtures/sample-project/src
dir: fixtures/sample-project/src
index: fixtures/sample-project/src/.pi-map.index.md
## role
Entry point module that demonstrates user creation, email validation, and logging for a sample project.
## files
- index.ts | Entry point that creates a user, validates their email, and logs the result | exp: func:main(), call:createUser, call:validateEmail, call:logger.error, call:logger.info | dep: ./models/user.js, ./utils/validation.js, ./utils/logger.js
## arch
Simple procedural script with direct function calls and sequential execution pattern.
## tags
user, email, js, main, call:create, call:validate, call:logger.error, call:logger.info
## symbols
- main
- call:createUser
- call:validateEmail
- call:logger.error
- call:logger.info
## workflows
- change src behavior
read: index.ts
- explore src subdirectories
index: fixtures/sample-project/src/components/.pi-map.index.md, fixtures/sample-project/src/models/.pi-map.index.md, fixtures/sample-project/src/utils/.pi-map.index.md
## dirty
-
@@ -0,0 +1,21 @@
# fixtures/sample-project/src/components (index)
dir: fixtures/sample-project/src/components
## role
Provides reusable UI components for building the application's interface.
## parent
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## children
-
## files
- Button.tsx
- UserCard.tsx
## links
index: fixtures/sample-project/src/components/.pi-map.index.md
map: fixtures/sample-project/src/components/.pi-map.md
## workflows
- change components behavior
read: Button.tsx, UserCard.tsx
## dirty
-
@@ -0,0 +1,26 @@
# fixtures/sample-project/src/components
dir: fixtures/sample-project/src/components
index: fixtures/sample-project/src/components/.pi-map.index.md
## role
Provides reusable UI components for building the application's interface.
## files
- Button.tsx | A reusable React button component that renders a styled button with configurable label, visual variant, click handler, and disabled state. | exp: ButtonProps, func:Button({ label, variant = "primary", onClick, disabled = false, }: ButtonProps) → JSX.Element | dep: react, React
- UserCard.tsx | Renders a user card component with optional edit and delete action buttons. | exp: UserCardProps, func:UserCard({ user, onEdit, onDelete }: UserCardProps) → JSX.Element, call:onEdit, call:onDelete | dep: react, ../models/user.js, React
## arch
Simple functional React components with props-based configuration, following a basic presentational component pattern.
## tags
button, user, card, react, props, call:on, edit, delete
## symbols
- Button
- UserCard
- ButtonProps
- UserCardProps
- call:onEdit
- call:onDelete
## workflows
- change components behavior
read: Button.tsx, UserCard.tsx
## dirty
-
@@ -0,0 +1,20 @@
# fixtures/sample-project/src/models (index)
dir: fixtures/sample-project/src/models
## role
Defines the User data model with creation and serialization capabilities including email validation.
## parent
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## children
-
## files
- user.ts
## links
index: fixtures/sample-project/src/models/.pi-map.index.md
map: fixtures/sample-project/src/models/.pi-map.md
## workflows
- change models behavior
read: user.ts
## dirty
-
@@ -0,0 +1,26 @@
# fixtures/sample-project/src/models
dir: fixtures/sample-project/src/models
index: fixtures/sample-project/src/models/.pi-map.index.md
## role
Defines the User data model with creation and serialization capabilities including email validation.
## files
- user.ts | Defines a User interface and provides functions to create and serialize users with email validation. | exp: User, func:createUser(data: Omit<User, "id" | "createdAt">) → User, call:validateEmail, call:crypto.randomUUID, raise:Error, func:serializeUser(user: User) → string, call:JSON.stringify | dep: ../utils/validation.js
## arch
Domain model pattern with interface-based typing, pure functions for data transformation, and embedded validation logic.
## tags
user, create, serialize, email, validation, call:validate, call:crypto.random, uuid
## symbols
- createUser
- serializeUser
- User
- call:validateEmail
- call:crypto.randomUUID
- raise:Error
- call:JSON.stringify
## workflows
- change models behavior
read: user.ts
## dirty
-
@@ -0,0 +1,21 @@
# fixtures/sample-project/src/utils (index)
dir: fixtures/sample-project/src/utils
## role
Provides foundational cross-cutting utility functions for logging and input validation used throughout the application.
## parent
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## children
-
## files
- logger.ts
- validation.ts
## links
index: fixtures/sample-project/src/utils/.pi-map.index.md
map: fixtures/sample-project/src/utils/.pi-map.md
## workflows
- change utils behavior
read: logger.ts, validation.ts
## dirty
-
@@ -0,0 +1,28 @@
# fixtures/sample-project/src/utils
dir: fixtures/sample-project/src/utils
index: fixtures/sample-project/src/utils/.pi-map.index.md
## role
Provides foundational cross-cutting utility functions for logging and input validation used throughout the application.
## files
- logger.ts | Provides a simple typed console logger with timestamps and convenience wrappers for different log levels. | exp: LogLevel, func:log(level: LogLevel, message: string) → void, call:new Date().toISOString, call:console.log, call:level.toUpperCase, func:debug(message: string) → void, call:log, func:info(message: string) → void, call:log, func:warn(message: string) → void, call:log, func:error(message: string) → void, call:log
- validation.ts | Provides basic string validation utilities for emails, non-empty checks, and minimum length requirements. | exp: func:validateEmail(email: string) → boolean, call:EMAIL_REGEX.test, func:validateNotEmpty(value: string) → boolean, call:value.trim, func:validateMinLength(value: string, min: number) → boolean
## arch
Flat utility module pattern with pure functions, no dependencies between modules, each exporting independent typed helper functions.
## tags
call:log, validate, log, logger, validation, empty, length, provides
## symbols
- log
- debug
- info
- warn
- error
- validateEmail
- validateNotEmpty
- validateMinLength
## workflows
- change utils behavior
read: logger.ts, validation.ts
## dirty
-
@@ -0,0 +1,21 @@
# fixtures/sample-project/tests (index)
dir: fixtures/sample-project/tests
## role
Provides unit test coverage for user model behavior and string validation utilities in the sample project.
## parent
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## children
-
## files
- user.test.ts
- validation.test.ts
## links
index: fixtures/sample-project/tests/.pi-map.index.md
map: fixtures/sample-project/tests/.pi-map.md
## workflows
- update tests tests
read: user.test.ts, validation.test.ts
## dirty
-
+21
View File
@@ -0,0 +1,21 @@
# fixtures/sample-project/tests
dir: fixtures/sample-project/tests
index: fixtures/sample-project/tests/.pi-map.index.md
## role
Provides unit test coverage for user model behavior and string validation utilities in the sample project.
## files
- user.test.ts | Unit tests for user model creation, validation, and serialization | dep: vitest, ../src/models/user.js
- validation.test.ts | Unit tests for string validation utility functions | dep: vitest, ../src/utils/validation.js
## arch
Standard test suite using isolated unit tests with file-based grouping by domain concern (model vs. utility).
## tags
validation, unit, tests, user, vitest, src, js, user.test
## symbols
-
## workflows
- update tests tests
read: user.test.ts, validation.test.ts
## dirty
-
-11
View File
@@ -1,11 +0,0 @@
// Pi skill prompt hook
// Injected into every prompt to remind the agent to maintain .pi-map.md files
export const MAINTENANCE_INSTRUCTION = `
If you modify any source file, run \`project-map:patch <file-path>\` to update the analysis.
If you suspect staleness, run \`project-map:validate\`.
`;
export function injectPrompt(originalPrompt: string): string {
return `${originalPrompt}\n\n---\n${MAINTENANCE_INSTRUCTION}`;
}
-241
View File
@@ -1,241 +0,0 @@
# Implementation Plan: Hierarchical Project Analysis Skill for Pi
## Overview
Build a Pi skill package (`pi-project-map`) that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file. The skill provides custom tools for init, patch, validate, and re-init, plus a prompt hook that ensures the agent keeps analysis files in sync.
---
## Current Status
- **M1: Foundation and Format** — ✅ Complete
- **M2: Heuristic Extraction** — ✅ Complete (placeholder only; to be replaced by real LLM)
- **M3: AST Extraction** — ✅ Complete
- **M4: Patch and Update** — ✅ Complete
- **M5: Validation and `--fix`** — ✅ Complete
- **M6: Pi Skill Integration** — ✅ Complete (basic)
- **M7: Proper LLM Integration** — 🔄 In Progress / Next up
The remaining major work is to replace the heuristic "LLM" extraction with **actual LLM API calls**.
---
## Remaining Milestone: M7 — Proper LLM Integration
**Goal**: Replace the regex heuristic `llm-extract.ts` with real LLM calls, including dual-provider support (Pi native + external fallback), disk caching, parallelization, and retries.
**Estimated time**: 1 week of focused work.
---
### Task 1: LLM Client Abstraction (`src/llm-client.ts`)
Create a unified interface for LLM calls.
```typescript
interface LLMClient {
complete(prompt: string): Promise<string>;
}
```
Sub-tasks:
- Define the `LLMClient` interface.
- Add a factory function `createLLMClient(mode: 'pi' | 'external', options)`.
- Handle mode selection:
- `'pi'`: used inside `pi-extension.ts` with Pi's model registry.
- `'external'`: used in standalone CLI with OpenAI-compatible API.
**Acceptance Criteria**
- `LLMClient` interface exists and compiles.
- Factory correctly selects implementation based on mode.
---
### Task 2: External LLM Client (`src/external-llm-client.ts`)
Implement the standalone CLI LLM client using an OpenAI-compatible API.
Sub-tasks:
- Add `openai` (or a lightweight fetch-based client) as a dependency.
- Read configuration from:
- Environment variable `OPENAI_API_KEY` (or `ANTHROPIC_API_KEY`, etc.)
- Config file `.pi-project-map.json` fields: `llmProvider`, `llmModel`, `llmBaseUrl`
- Implement `complete(prompt)` using chat completions API.
- Default to `gpt-4o-mini` or similar cheap model.
**Acceptance Criteria**
- `project-map init` works standalone with `OPENAI_API_KEY` set.
- Missing API key produces a clear, actionable error message.
- Failed API call throws a descriptive `LLMError`.
---
### Task 3: Pi LLM Client (`src/pi-llm-client.ts`)
Implement the Pi-native LLM client for use inside the extension.
Sub-tasks:
- Accept Pi's `ExtensionContext` or model registry as a constructor argument.
- Use `ctx.model` / `ctx.modelRegistry` to get the configured model and API key.
- Call the provider directly (likely using the same OpenAI-compatible endpoint Pi uses).
- If Pi does not expose direct LLM calls, fall back to emitting a tool call / follow-up message pattern.
**Acceptance Criteria**
- Pi extension can successfully call an LLM when running inside Pi.
- Errors surface clearly to the user.
---
### Task 4: Disk Cache (`src/llm-cache.ts`)
Implement persistent SHA-256 → LLM result caching.
Sub-tasks:
- Cache directory: `~/.cache/pi-project-map/` (create if missing).
- Cache file: `llm-cache.json` (simple JSON object).
- Functions:
- `getCached(hash: string): string | null`
- `setCached(hash: string, result: string): void`
- Ensure atomic writes (write to temp file, rename).
- Add cache size limit (e.g., 10,000 entries, LRU eviction).
**Acceptance Criteria**
- Second `init` run on unchanged files does not call LLM.
- Cache persists across process restarts.
- Corrupted cache file does not crash the tool.
---
### Task 5: Parallelization and Retries (`src/llm-batch.ts`)
Run LLM requests in parallel with retries and backoff.
Sub-tasks:
- Add `p-limit` dependency for concurrency control.
- Default concurrency: 4 (configurable via `.pi-project-map.json` `llmConcurrency`).
- Add small delay (100ms) between batches.
- Implement retry logic: 3 retries, delays 1s → 2s → 4s.
- On final failure, throw a hard error and stop the entire process.
**Acceptance Criteria**
- 100-file project completes significantly faster than sequential.
- Simulated transient failures are retried and recovered.
- Persistent failure stops the tool with a clear error.
---
### Task 6: Rewrite `llm-extract.ts` to Use Real LLM
Replace regex heuristics with LLM calls.
Sub-tasks:
- Accept an `LLMClient` in `extractFileLLM` and `extractPackageLLM`.
- Check disk cache before calling LLM.
- Construct file-level prompt (see design doc Section 4).
- Parse response into `purpose`, `deps`, `concepts`.
- Construct package-level prompt.
- Parse response into `role`, `arch`.
- Remove heuristic code from production path; keep only for test mocks if useful.
**Acceptance Criteria**
- `llm-extract.ts` calls the LLM client for every file.
- Output includes rich, non-trivial descriptions for typical files.
- Unit tests mock the LLM client to verify prompt structure and parsing.
---
### Task 7: Context Limit Handling
Protect against oversized files.
Sub-tasks:
- Measure prompt + file content tokens (approximate: 1 token ≈ 4 chars for ASCII).
- If file exceeds max context budget (configurable, default 4000 tokens), truncate from the end.
- Append `[...truncated]` marker in the prompt.
- Skip LLM for binary/generated files over a hard limit (e.g., 50KB) and mark them as "Large/generated file".
**Acceptance Criteria**
- A 1MB minified JS file does not crash or consume excessive tokens.
- Truncated files still produce useful output.
---
### Task 8: Update CLI and Extension
Wire the new LLM client into all entry points.
Sub-tasks:
- `src/cli.ts`: create external LLM client, pass into `initProject` / `patchFile`.
- `pi-extension.ts`: create Pi LLM client, pass into tools.
- Update `init.ts` and `patch.ts` signatures to accept an optional `LLMClient`.
- Add CLI flag `--llm-provider=openai` for explicit selection.
- Update error handling to catch `LLMError` and print helpful messages.
**Acceptance Criteria**
- CLI works with external API key.
- Extension works inside Pi (if Pi exposes LLM access).
- Clear errors on misconfiguration.
---
### Task 9: Update Tests
Sub-tasks:
- Replace heuristic tests with mocked LLM client tests.
- Add integration test: create a fake LLM client, run `initProject`, verify output contains LLM-provided text.
- Add cache test: verify cache hit skips LLM call.
- Add retry test: verify transient failures retry, persistent failures hard-stop.
**Acceptance Criteria**
- All tests pass.
- Test coverage includes: LLM client, cache, batching, prompt parsing, error handling.
---
## Sequencing and Dependencies
```
Task 1 (LLMClient interface)
├── Task 2 (External client)
├── Task 3 (Pi client)
├── Task 4 (Cache)
├── Task 5 (Batch + retries)
├── Task 6 (Rewrite llm-extract.ts)
├── Task 7 (Context limits)
├── Task 8 (Wire CLI + extension)
└── Task 9 (Tests)
```
---
## Validation Criteria (for LLM Integration)
1. **Real LLM calls**: `llm-extract.ts` invokes the configured LLM client for every file.
2. **Cache hit**: Second `init` on unchanged repo completes with zero LLM calls.
3. **Parallel speed**: 100-file project init completes in under 30 seconds (assuming average LLM latency 500ms).
4. **Retry works**: Transient 429/5xx errors are retried; permanent failures stop with a clear error.
5. **Context limit safety**: Files > max token budget are truncated, not rejected.
6. **Dual provider**: CLI uses external API; Pi extension uses Pi's LLM.
7. **Quality**: LLM output is visibly richer than the old heuristic output (verified by manual inspection).
---
## Rollout Plan
1. **Test on real projects** (Day 1-2): Run `init` on 2-3 real codebases with the new LLM integration.
2. **Cost audit** (Day 3): Measure token usage per project; adjust defaults if too expensive.
3. **Prompt tuning** (Day 4-5): Iterate prompt design based on output quality.
4. **Release** (Day 6-7): Publish updated npm package, update Pi extension docs.
---
## Completed Milestones (for reference)
- **M1**: Foundation and Format
- **M2**: Heuristic Extraction (placeholder, to be replaced by M7)
- **M3**: AST Extraction
- **M4**: Patch and Update
- **M5**: Validation and `--fix`
- **M6**: Pi Skill Integration
+22
View File
@@ -0,0 +1,22 @@
# openspec (index)
dir: openspec
## role
Defines configuration and project conventions for a TypeScript CLI tool that generates AI-oriented codebase orientation maps for coding agents.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
- openspec/changes
index: openspec/changes/.pi-map.index.md
map: openspec/changes/.pi-map.md
## files
- config.yaml
- project.md
## links
index: openspec/.pi-map.index.md
map: openspec/.pi-map.md
## workflows
-
## dirty
-
+20
View File
@@ -0,0 +1,20 @@
# openspec
dir: openspec
index: openspec/.pi-map.index.md
## role
Defines configuration and project conventions for a TypeScript CLI tool that generates AI-oriented codebase orientation maps for coding agents.
## files
- config.yaml | Defines project configuration, stack metadata, and software-driven development (SDD) workflow rules for a TypeScript-based CLI tool that generates AI-oriented codebase map files. | dep: TypeScript, Node.js, Vitest, npm, openspec
- project.md | Defines project context and conventions for pi-project-map, a Pi skill and CLI tool that generates hierarchical `.pi-map.md` orientation files for coding agents.
## arch
YAML-driven configuration with software-driven development (SDD) workflow rules, hierarchical markdown output generation, and Pi skill integration for AI agent context provision.
## tags
project, map, defines, typescript, cli, tool, generates, config
## symbols
-
## workflows
-
## dirty
-
+20
View File
@@ -0,0 +1,20 @@
# openspec/changes (index)
dir: openspec/changes
## role
Manages change tracking, versioning, and audit history for OpenAPI specification modifications
## parent
index: openspec/.pi-map.index.md
map: openspec/.pi-map.md
## children
- openspec/changes/archive
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## files
## links
index: openspec/changes/.pi-map.index.md
map: openspec/changes/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# openspec/changes
dir: openspec/changes
index: openspec/changes/.pi-map.index.md
## role
Manages change tracking, versioning, and audit history for OpenAPI specification modifications
## files
## arch
Event-sourced or changelog-based pattern with immutable change records and versioned snapshots
## tags
-
## symbols
-
## workflows
-
## dirty
-
+26
View File
@@ -0,0 +1,26 @@
# openspec/changes/archive (index)
dir: openspec/changes/archive
## role
Provides persistent storage and retrieval of historical change records in an archived format for audit trails and long-term data retention.
## parent
index: openspec/changes/.pi-map.index.md
map: openspec/changes/.pi-map.md
## children
- openspec/changes/archive/2026-06-11-layered-map-protocol
index: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.md
- openspec/changes/archive/2026-06-11-map-context-retrieval
index: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.md
- openspec/changes/archive/2026-06-11-project-map-prompt-injection
index: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.md
## files
## links
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# openspec/changes/archive
dir: openspec/changes/archive
index: openspec/changes/archive/.pi-map.index.md
## role
Provides persistent storage and retrieval of historical change records in an archived format for audit trails and long-term data retention.
## files
## arch
Simple archive storage pattern using file-based serialization with read/write operations for immutable change log records, likely with date-based or sequential naming conventions.
## tags
-
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-layered-map-protocol (index)
dir: openspec/changes/archive/2026-06-11-layered-map-protocol
## role
Contains archived specification documents for a deprecated layered map protocol that introduced paired navigation artifacts to replace bulk-loading of map files with a tiered directory-level routing system.
## parent
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## children
-
## files
- apply-progress.md
- archive-report.md
- design.md
- proposal.md
- spec.md
- sync-report.md
- tasks.md
- verify-report.md
## links
index: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.md
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-layered-map-protocol
dir: openspec/changes/archive/2026-06-11-layered-map-protocol
index: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.index.md
## role
Contains archived specification documents for a deprecated layered map protocol that introduced paired navigation artifacts to replace bulk-loading of map files with a tiered directory-level routing system.
## files
- apply-progress.md | Documents the completion status and summary of implemented features for an "Apply Progress" project or milestone.
- archive-report.md | Documents the archival status and metadata for a deprecated layered map protocol specification directory.
- design.md | Design document for a paired navigation-first artifact model that generates both routing indexes and orientation rich-maps from a shared intermediate directory model while preserving existing pipeline behavior. | dep: spec.md, src/format.ts, src/init.ts, src/patch.ts, src/validate.ts, pi-extension.ts, README.md, SKILL.md, design-doc.md, CLI argument parsing, YAML config handling
- proposal.md | Proposes a layered navigation protocol using paired index/map artifacts to replace bulk-loading of map files with a tiered, directory-level routing system.
- spec.md | Specifies a layered navigation protocol for project maps using paired index and rich map artifacts per directory with defined generation, patching, and validation behaviors.
- sync-report.md | Documents that a canonical spec synchronization was not performed due to legacy flat change artifact structure, with user-approved archival fallback
- tasks.md | Defines a phased task plan for implementing a layered paired-map protocol with directory-level `.pi-map.md` and `.pi-map.index.md` artifacts, routing metadata, patch sizing, validation, and documentation. | dep: design.md, pi-extension.ts, SKILL.md, README.md, design-doc.md, npm/node toolchain
- verify-report.md | Documents verification results for a code change implementing a layered map protocol with paired artifacts. | dep: npm, vitest, node, typescript
## arch
Document-driven specification archive using a layered architecture with paired index/map artifacts per directory, phased implementation tasks, and formal verification/synchronization reporting, preserved for historical reference despite legacy flat structure preventing canonical sync.
## tags
map, md, report, design, ts, layered, protocol, directory
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-map-context-retrieval (index)
dir: openspec/changes/archive/2026-06-11-map-context-retrieval
## role
Archives a completed specification change package for implementing metadata-driven context retrieval capabilities in a project-map tool.
## parent
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## children
-
## files
- apply-progress.md
- archive-report.md
- design.md
- proposal.md
- spec.md
- sync-report.md
- tasks.md
- verify-report.md
## links
index: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.md
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-map-context-retrieval
dir: openspec/changes/archive/2026-06-11-map-context-retrieval
index: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.index.md
## role
Archives a completed specification change package for implementing metadata-driven context retrieval capabilities in a project-map tool.
## files
- apply-progress.md | Documents the completion status of a project implementing deterministic index-first context retrieval via tool and CLI, along with retrieval documentation and skill guidance.
- archive-report.md | Documents the archival of a set of specification change documents including metadata about archive location, mode, and preserved inputs.
- design.md | Design document for a lightweight context retrieval system that scans paired project-map metadata to find and return relevant code context as a markdown bundle for AI agents. | dep: pi-extension.ts, src/index.ts, src/cli/*, paired-artifact parser/model from layered protocol, .pi-map.index.md, .pi-map.md
- proposal.md | Proposes a "Map Context Retrieval" tool that enables natural-language queries to return compact, metadata-driven context bundles from a layered map protocol. | dep: layered-map-protocol, Pi tool, CLI
- spec.md | Defines a specification for adding a retrieval-oriented `context` command to a project-map tool that converts user tasks into compact routing bundles for LLM agents | dep: layered-map-protocol, project-map (Pi tool/CLI), markdown output formatting
- sync-report.md | Documents that a canonical spec synchronization was not performed due to legacy flat change artifact structure, with user-approved archival fallback
- tasks.md | Defines implementation tasks for adding map context retrieval functionality to a Pi tool, using paired index/map metadata for ranked, query-based context retrieval. | dep: design.md, Pi tool, project-map context, npm (typecheck, test, lint)
- verify-report.md | Documents verification results for a code change implementing deterministic index-first context retrieval. | dep: npm, vitest, node, CLI tooling
## arch
Flat archival directory structure preserving legacy change artifacts (specification, design, proposal, tasks, verification, sync, apply-progress, archive-report) without hierarchical organization.
## tags
map, context, retrieval, report, documents, project, index, tool
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-project-map-prompt-injection (index)
dir: openspec/changes/archive/2026-06-11-project-map-prompt-injection
## role
Archives a completed prompt-injection feature specification that enables runtime injection of project-map artifacts into LLM context with configurable guidance modes and budget controls.
## parent
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## children
-
## files
- apply-progress.md
- archive-report.md
- design.md
- proposal.md
- spec.md
- sync-report.md
- tasks.md
- verify-report.md
## links
index: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.md
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-project-map-prompt-injection
dir: openspec/changes/archive/2026-06-11-project-map-prompt-injection
index: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.index.md
## role
Archives a completed prompt-injection feature specification that enables runtime injection of project-map artifacts into LLM context with configurable guidance modes and budget controls.
## files
- apply-progress.md | Documents the completion status and implementation details of a prompt-injection feature delivered across five incremental slices.
- archive-report.md | Documents the archival status and metadata of a deprecated project change specification directory.
- design.md | Design document for adding a runtime prompt injection layer that guides LLM behavior using paired map/index artifacts through configurable modes, canonical markers, and budgeted context expansion. | dep: pi-extension.ts, src/config.ts, spec.md, event.messages, before_agent_start, before_provider_request, context hooks, .pi-project-map.json, .pi-map.index.md, .pi-map.md
- proposal.md | Proposes a runtime prompt-injection policy for project-map artifacts with configurable guidance modes, hybrid context budgets, and reinjection avoidance
- spec.md | Defines a specification for automatic runtime prompt injection of project map/index artifacts with configurable guidance modes, budgeted expansion, and reinjection avoidance based on actual outgoing context scanning.
- sync-report.md | Documents that a canonical spec synchronization was not performed due to legacy flat change artifact structure, with user-approved archival fallback
- tasks.md | Defines phased implementation tasks for a prompt injection policy system that controls how project map artifacts are injected into LLM context with budget constraints, mode semantics, and reinjection avoidance.
- verify-report.md | Documents verification results for a prompt-injection security feature implementation in a software project.
## arch
Document-driven specification architecture using phased slice-based delivery (proposal → design → spec → tasks → verification), with flat artifact structure and canonical marker-based context expansion patterns.
## tags
map, prompt, injection, project, report, documents, artifacts, context
## symbols
-
## workflows
-
## dirty
-
+49 -11
View File
@@ -81,6 +81,25 @@ function renderProgressBar(
return `[${bar}] ${completed}/${total}${file}`; return `[${bar}] ${completed}/${total}${file}`;
} }
const HINT_CUSTOM_TYPE = "pi-project-map-hint";
function hintAlreadyInContext(ctx: any): boolean {
const manager = ctx?.sessionManager;
if (!manager || typeof manager.buildSessionContext !== "function") {
return false;
}
const { messages } = manager.buildSessionContext();
if (!Array.isArray(messages)) return false;
return messages.some(
(m: any) =>
m &&
m.role === "custom" &&
m.customType === HINT_CUSTOM_TYPE,
);
}
export default function (pi: ExtensionAPI) { export default function (pi: ExtensionAPI) {
let lastRootPairMtimes: import("./src/index.js").RootPairMtimes = {}; let lastRootPairMtimes: import("./src/index.js").RootPairMtimes = {};
pi.registerTool({ pi.registerTool({
@@ -190,11 +209,12 @@ export default function (pi: ExtensionAPI) {
name: "project_map_validate", name: "project_map_validate",
label: "Project Map Validate", label: "Project Map Validate",
description: description:
"Check all .pi-map.md / .pi-map.index.md files for staleness and discrepancies", "Check all .pi-map.md / .pi-map.index.md files for staleness and discrepancies. Optionally repair them.",
promptSnippet: "Validate paired project map/index artifacts for accuracy", promptSnippet: "Validate paired project map/index artifacts for accuracy",
promptGuidelines: [ promptGuidelines: [
"Use project_map_validate before making architectural decisions if you suspect stale data", "Use project_map_validate before making architectural decisions if you suspect stale data",
"Use project_map_validate to detect files that were deleted or added outside the agent", "Use project_map_validate to detect files that were deleted or added outside the agent",
"Set fix=true to repair localized discrepancies without running a full project_map_reinit",
], ],
parameters: Type.Object({ parameters: Type.Object({
path: Type.Optional( path: Type.Optional(
@@ -202,13 +222,23 @@ export default function (pi: ExtensionAPI) {
description: "Project root path (default: current directory)", description: "Project root path (default: current directory)",
}), }),
), ),
fix: Type.Optional(
Type.Boolean({
description:
"Repair discrepancies automatically (default: false). Requires an LLM client.",
default: false,
}),
),
}), }),
async execute(_toolCallId, params, _signal, _onUpdate, ctx) { async execute(_toolCallId, params, _signal, _onUpdate, ctx) {
try { try {
const targetPath = params.path || ctx.cwd; const targetPath = params.path || ctx.cwd;
const client = params.fix ? getPiLLMClient(ctx) : undefined;
const result = await validateMaps(targetPath, { const result = await validateMaps(targetPath, {
fix: false, fix: params.fix ?? false,
verbose: false, verbose: false,
llmClient: client,
cacheDir: ctx.cwd,
}); });
const text = result.clean const text = result.clean
? "All .pi-map.md files are clean." ? "All .pi-map.md files are clean."
@@ -234,12 +264,14 @@ export default function (pi: ExtensionAPI) {
name: "project_map_reinit", name: "project_map_reinit",
label: "Project Map Reinit", label: "Project Map Reinit",
description: description:
"Force full regeneration of all .pi-map.md / .pi-map.index.md artifacts", "Regenerate .pi-map.md / .pi-map.index.md artifacts for a subtree plus its ancestors, falling back to full regeneration only when the subtree covers more than the configured percentage of project files (default: 10%)",
promptSnippet: promptSnippet:
"Force full regeneration of paired project map/index artifacts", "Regenerate paired project map/index artifacts for a subtree or the whole project",
promptGuidelines: [ promptGuidelines: [
"Use project_map_reinit when validation shows widespread staleness", "Use project_map_reinit only after project_map_patch and project_map_validate --fix cannot resolve the staleness",
"Use project_map_reinit after pulling major changes from version control", "For localized changes, prefer project_map_patch <changed-file> or project_map_validate with fix=true",
"Use project_map_reinit for widespread structural damage (e.g. broken links across many directories) or after large merges",
"When reinit falls back to full regeneration, it is because the target subtree covers more than the configured reinitFullThresholdPercent of project files",
], ],
parameters: Type.Object({ parameters: Type.Object({
path: Type.Optional( path: Type.Optional(
@@ -345,13 +377,15 @@ export default function (pi: ExtensionAPI) {
if (dirtyFiles.length > 0) { if (dirtyFiles.length > 0) {
ctx.ui.notify( ctx.ui.notify(
`pi-project-map: ${dirtyFiles.length} dirty packages detected. Run project_map_validate or project_map_reinit.`, `pi-project-map: ${dirtyFiles.length} dirty package(s) detected. Run project_map_validate first; use project_map_reinit only if staleness is widespread.`,
"warning", "warning",
); );
} }
}); });
// Inject maintenance instructions before agent starts // Inject maintenance instructions before agent starts, but only once
// within the current branch of context. Re-inject after compaction or
// tree navigation removes the hint from the active path.
pi.on("before_agent_start", async (_event, _ctx) => { pi.on("before_agent_start", async (_event, _ctx) => {
const config = loadConfig(_ctx.cwd); const config = loadConfig(_ctx.cwd);
const mapFiles = findPiMapFiles(_ctx.cwd); const mapFiles = findPiMapFiles(_ctx.cwd);
@@ -366,9 +400,10 @@ export default function (pi: ExtensionAPI) {
if (!modeAllowsPreInitHint(config.promptInjectionMode)) { if (!modeAllowsPreInitHint(config.promptInjectionMode)) {
return {}; return {};
} }
if (hintAlreadyInContext(_ctx)) return {};
return { return {
message: { message: {
customType: "pi-project-map-hint", customType: HINT_CUSTOM_TYPE,
content: buildPreInitHint(), content: buildPreInitHint(),
display: true, display: true,
}, },
@@ -378,9 +413,10 @@ export default function (pi: ExtensionAPI) {
// Slice 4: advisory mode shows a visible lightweight reminder after init. // Slice 4: advisory mode shows a visible lightweight reminder after init.
// No root-pair preload, no per-turn reinjection. // No root-pair preload, no per-turn reinjection.
if (config.promptInjectionMode === "advisory") { if (config.promptInjectionMode === "advisory") {
if (hintAlreadyInContext(_ctx)) return {};
return { return {
message: { message: {
customType: "pi-project-map-hint", customType: HINT_CUSTOM_TYPE,
content: buildAdvisoryReminder(), content: buildAdvisoryReminder(),
display: true, display: true,
}, },
@@ -414,12 +450,14 @@ export default function (pi: ExtensionAPI) {
return {}; return {};
} }
if (hintAlreadyInContext(_ctx)) return {};
// Slice 2: post-init root-pair preload + budgeted expansion // Slice 2: post-init root-pair preload + budgeted expansion
const contextWindow = discoverContextWindow(_ctx); const contextWindow = discoverContextWindow(_ctx);
const payload = buildInjectionPayload(_ctx.cwd, config, contextWindow); const payload = buildInjectionPayload(_ctx.cwd, config, contextWindow);
return { return {
message: { message: {
customType: "pi-project-map-hint", customType: HINT_CUSTOM_TYPE,
content: payload.content, content: payload.content,
display: payload.display, display: payload.display,
}, },
+49
View File
@@ -0,0 +1,49 @@
# src (index)
dir: src
## role
A project mapping and codebase navigation system that generates, maintains, and queries AI-readable documentation artifacts for software projects.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
- src/ast
index: src/ast/.pi-map.index.md
map: src/ast/.pi-map.md
- src/cli
index: src/cli/.pi-map.index.md
map: src/cli/.pi-map.md
- src/llm
index: src/llm/.pi-map.index.md
map: src/llm/.pi-map.md
- src/types
index: src/types/.pi-map.index.md
map: src/types/.pi-map.md
## files
- cli.ts
- config.ts
- directory-model.ts
- discover.ts
- format.ts
- index.ts
- init.ts
- merge.ts
- patch.ts
- prompt-injection.ts
- retrieve.ts
- routing-metadata.ts
- validate.ts
## links
index: src/.pi-map.index.md
map: src/.pi-map.md
## workflows
- change src behavior
read: cli.ts, config.ts, directory-model.ts
- change src CLI
read: cli.ts
- change src config
read: config.ts
- explore src subdirectories
index: src/ast/.pi-map.index.md, src/cli/.pi-map.index.md, src/llm/.pi-map.index.md
## dirty
-
+45
View File
File diff suppressed because one or more lines are too long
+20
View File
@@ -0,0 +1,20 @@
# src/ast (index)
dir: src/ast
## role
Extracts structured metadata from source code ASTs across multiple languages to enable code analysis and dependency understanding.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- ast-extract.ts
## links
index: src/ast/.pi-map.index.md
map: src/ast/.pi-map.md
## workflows
- change ast behavior
read: ast-extract.ts
## dirty
-
+27
View File
@@ -0,0 +1,27 @@
# src/ast
dir: src/ast
index: src/ast/.pi-map.index.md
## role
Extracts structured metadata from source code ASTs across multiple languages to enable code analysis and dependency understanding.
## files
- ast-extract.ts | Extracts AST-based metadata (exports, dependencies, classes, functions, method calls, and exceptions) from source code files across multiple languages using Tree-sitter parsers. | exp: ASTFileData, func:extractFileAST(filePath: string) → Promise<ASTFileData | null>, call:extname(filePath).toLowerCase, call:require, call:parser.setLanguage, call:readFileSync, call:parser.parse, call:extractPythonData, call:extractTypeScriptData, call:extractGoData, call:extractExportsFromTree, call:extractDepsFromTree | dep: fs, path, tree-sitter, tree-sitter-typescript, tree-sitter-python, tree-sitter-go, tree-sitter-rust, tree-sitter-java, tree-sitter-c, tree-sitter-cpp, tree-sitter-ruby
## arch
Language-agnostic parser abstraction using Tree-sitter grammars with unified extraction pipeline for cross-language code analysis.
## tags
tree, sitter, call:extract, data, ast, extract, python, go
## symbols
- extractFileAST
- ASTFileData
- call:extname(filePath).toLowerCase
- call:require
- call:parser.setLanguage
- call:readFileSync
- call:parser.parse
- call:extractPythonData
## workflows
- change ast behavior
read: ast-extract.ts
## dirty
-
+22
View File
@@ -0,0 +1,22 @@
# src/cli (index)
dir: src/cli
## role
Command-line interface entry point for a project mapping tool that manages hierarchical `.pi-map.md` files through generation, patching, validation, and LLM-powered context retrieval.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- cli.ts
## links
index: src/cli/.pi-map.index.md
map: src/cli/.pi-map.md
## workflows
- change cli behavior
read: cli.ts
- change cli CLI
read: cli.ts
## dirty
-
+22
View File
@@ -0,0 +1,22 @@
# src/cli
dir: src/cli
index: src/cli/.pi-map.index.md
## role
Command-line interface entry point for a project mapping tool that manages hierarchical `.pi-map.md` files through generation, patching, validation, and LLM-powered context retrieval.
## files
- cli.ts | CLI entry point for a project mapping tool that generates, patches, validates, and retrieves context from hierarchical `.pi-map.md` files using LLM-powered analysis. | dep: ../init.js, ../patch.js, ../validate.js, ../discover.js, ../retrieve.js, ../llm/llm-client.js, ../config.js, picocolors, process, fs (implied via require)
## arch
Single-file CLI facade with command routing to core engine services, likely using a command pattern or direct service delegation for map lifecycle operations.
## tags
js, cli, llm, entry, point, project, mapping, tool
## symbols
-
## workflows
- change cli behavior
read: cli.ts
- change cli CLI
read: cli.ts
## dirty
-
+2 -2
View File
@@ -28,7 +28,7 @@ function printUsage() {
` project-map ${pc.cyan("validate")} [--fix] [path] Check for stale/missing/orphaned entries`, ` project-map ${pc.cyan("validate")} [--fix] [path] Check for stale/missing/orphaned entries`,
); );
console.log( console.log(
` project-map ${pc.cyan("reinit")} [path] Force full regeneration`, ` project-map ${pc.cyan("reinit")} [path] Regenerate a subtree or the whole project`,
); );
console.log( console.log(
` project-map ${pc.cyan("--help")} Show this help message`, ` project-map ${pc.cyan("--help")} Show this help message`,
@@ -252,7 +252,7 @@ async function main() {
await reinitPath(targetPath, { await reinitPath(targetPath, {
verbose: false, verbose: false,
llmClient: client, llmClient: client,
cacheDir: targetPath, cacheDir: process.cwd(),
onProgress: (info) => { onProgress: (info) => {
const line = renderProgressBar( const line = renderProgressBar(
info.completed, info.completed,
+3
View File
@@ -16,6 +16,8 @@ export interface SkillConfig {
promptInjectionMode: PromptInjectionMode; promptInjectionMode: PromptInjectionMode;
contextBudgetPercent: number; contextBudgetPercent: number;
contextBudgetMaxTokens: number; contextBudgetMaxTokens: number;
/** Percentage of project files a target subtree must cover before reinit falls back to full regeneration (default: 10). */
reinitFullThresholdPercent: number;
} }
export const DEFAULT_CONFIG: SkillConfig = { export const DEFAULT_CONFIG: SkillConfig = {
@@ -51,6 +53,7 @@ export const DEFAULT_CONFIG: SkillConfig = {
promptInjectionMode: "strong", promptInjectionMode: "strong",
contextBudgetPercent: 15, contextBudgetPercent: 15,
contextBudgetMaxTokens: 100_000, contextBudgetMaxTokens: 100_000,
reinitFullThresholdPercent: 10,
}; };
export function loadConfig(cwd: string = process.cwd()): SkillConfig { export function loadConfig(cwd: string = process.cwd()): SkillConfig {
+131 -4
View File
@@ -7,8 +7,8 @@ import { extractFileLLM, extractPackageLLM } from "./llm/llm-extract.js";
import { extractFileAST } from "./ast/ast-extract.js"; import { extractFileAST } from "./ast/ast-extract.js";
import { mergeFileData } from "./merge.js"; import { mergeFileData } from "./merge.js";
import { processFiles } from "./llm/llm-batch.js"; import { processFiles } from "./llm/llm-batch.js";
import { writeFileSync } from "fs"; import { existsSync, writeFileSync } from "fs";
import { join } from "path"; import { join, relative, resolve } from "path";
import type { LLMClient } from "./llm/llm-client.js"; import type { LLMClient } from "./llm/llm-client.js";
import { import {
createDirectoryModel, createDirectoryModel,
@@ -105,6 +105,60 @@ export interface DirectoryContext {
isRoot: boolean; isRoot: boolean;
} }
export function getAncestorEntries(
entries: DirectoryEntry[],
ctx: DirectoryContext,
entry: DirectoryEntry,
): DirectoryEntry[] {
const result: DirectoryEntry[] = [];
let parent = ctx.parentMap.get(entry.relativePath);
while (parent) {
const ancestor = entries.find(
(candidate) => candidate.relativePath === parent,
);
if (ancestor) {
result.push(ancestor);
}
parent = ctx.parentMap.get(parent);
}
return result;
}
function countFiles(entries: DirectoryEntry[]): number {
return entries.reduce((sum, e) => sum + e.files.length, 0);
}
function isSubdirectory(parent: string, child: string): boolean {
if (parent === ".") return true;
if (child === parent) return true;
return child.startsWith(`${parent}/`);
}
function findProjectRoot(targetPath: string): string {
let current = resolve(targetPath);
while (true) {
if (existsSync(join(current, ".pi-project-map.json"))) {
return current;
}
if (existsSync(join(current, ".git"))) {
return current;
}
if (existsSync(join(current, "package.json"))) {
return current;
}
const parent = resolve(current, "..");
if (parent === current) {
return resolve(targetPath);
}
current = parent;
}
}
function normalizeRelativePath(relPath: string): string {
if (!relPath || relPath === ".") return ".";
return relPath.replace(/\\/g, "/");
}
export type ArtifactWriteMode = "both" | "map" | "index"; export type ArtifactWriteMode = "both" | "map" | "index";
export function buildDirectoryContext( export function buildDirectoryContext(
@@ -244,8 +298,81 @@ export async function reinitPath(
path: string, path: string,
options: InitOptions = {}, options: InitOptions = {},
): Promise<void> { ): Promise<void> {
// Full regeneration clears all dirty markers by overwriting every map/index pair const targetPath = resolve(path);
await initProject(path, options); const rootPath = findProjectRoot(targetPath);
const targetRelPath =
rootPath === targetPath
? "."
: normalizeRelativePath(relative(rootPath, targetPath));
const entries = discoverProject(rootPath);
const config = loadConfig(rootPath);
const threshold = config.reinitFullThresholdPercent;
const totalFiles = countFiles(entries);
if (totalFiles === 0 || targetRelPath === ".") {
await initProject(rootPath, options);
return;
}
const subtreeEntries = entries.filter((entry) =>
isSubdirectory(targetRelPath, entry.relativePath),
);
const subtreeFiles = countFiles(subtreeEntries);
const percentage = (subtreeFiles / totalFiles) * 100;
if (percentage > threshold) {
await initProject(rootPath, options);
return;
}
const routingOpts: RoutingMetadataOptions = {
tagCap: options.tagCap ?? config.tagCap,
workflowHintCap: options.workflowHintCap ?? config.workflowHintCap,
};
const targetEntry = entries.find((e) => e.relativePath === targetRelPath);
if (!targetEntry) {
await initProject(rootPath, options);
return;
}
const changedCtx = buildDirectoryContext(entries, targetEntry);
const ancestors = getAncestorEntries(entries, changedCtx, targetEntry);
const dirsToRegenerate = new Set<DirectoryEntry>([
...subtreeEntries,
...ancestors,
]);
const dirs = Array.from(dirsToRegenerate);
const filesToRegenerate = countFiles(dirs);
let completedFiles = 0;
for (const entry of dirs) {
const ctx = buildDirectoryContext(entries, entry);
await generateDirectoryArtifacts(
entry,
ctx,
options.llmClient,
options.cacheDir,
(info) => {
options.onProgress?.({
...info,
completed: completedFiles + info.completed,
total: filesToRegenerate,
dir: entry.relativePath,
});
},
routingOpts,
"both",
);
completedFiles += entry.files.length;
}
if (options.verbose !== false) {
console.log(
`Smart reinit: regenerated ${dirsToRegenerate.size} directories under ${targetRelPath} (${subtreeFiles}/${totalFiles} files, ${percentage.toFixed(1)}%).`,
);
}
} }
// Backward-compatible wrapper for patch/validate compatibility // Backward-compatible wrapper for patch/validate compatibility
+29
View File
@@ -0,0 +1,29 @@
# src/llm (index)
dir: src/llm
## role
Provides a unified abstraction layer for interacting with multiple LLM providers (OpenAI, Kimi, Pi) with caching, batching, and structured response extraction capabilities.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- external-llm-client.ts
- kimi-llm-client.ts
- llm-batch.ts
- llm-cache.ts
- llm-client.ts
- llm-error.ts
- llm-extract.ts
- pi-llm-client.ts
## links
index: src/llm/.pi-map.index.md
map: src/llm/.pi-map.md
## workflows
- change llm behavior
read: external-llm-client.ts, kimi-llm-client.ts, llm-batch.ts
- change llm CLI
read: external-llm-client.ts, kimi-llm-client.ts, llm-client.ts
## dirty
-
+36
View File
@@ -0,0 +1,36 @@
# src/llm
dir: src/llm
index: src/llm/.pi-map.index.md
## role
Provides a unified abstraction layer for interacting with multiple LLM providers (OpenAI, Kimi, Pi) with caching, batching, and structured response extraction capabilities.
## files
- external-llm-client.ts | Implements an LLM client adapter for OpenAI's API to send code analysis prompts and return structured responses. | exp: class:ExternalLLMClient, method:constructor(options: LLMClientOptions), raise:LLMError, method:complete(prompt: string) → Promise<string>, call:this.client.chat.completions.create, call:response.choices[0]?.message?.content?.trim, raise:LLMError | dep: openai, ./llm-error.js, ./llm-client.js
- kimi-llm-client.ts | Implements an LLM client for the Kimi.com API using an Anthropic-compatible HTTP interface. | exp: class:KimiLLMClient, method:constructor(options: LLMClientOptions), raise:LLMError, method:complete(prompt: string) → Promise<string>, call:fetch, call:JSON.stringify, call:response.text, call:response.json, call:data.content?.[0]?.text?.trim, raise:LLMError, raise:err | dep: ./llm-error.js, ./llm-client.js, llm-error.js, llm-client.js
- llm-batch.ts | Provides batched, concurrent file processing with retry logic and progress callbacks for LLM operations. | exp: BatchOptions, func:withRetry(fn: () => Promise<T>, options: Pick<BatchOptions, "maxRetries" | "retryDelaysMs">) → Promise<T>, call:fn, call:sleep, raise:lastError, func:processFiles(files: T[], processor: (file: T) => Promise<R>, options: BatchOptions, onProgress: (completed: number, total: number, currentFile: T) => void) → Promise<R[]>, call:pLimit, call:files.map, call:limit, call:sleep, call:withRetry, call:processor, call:onProgress, call:Promise.all | dep: p-limit, ./llm-error.js
- llm-cache.ts | Provides a persistent file-based caching system for LLM responses keyed by hash, storing results in JSON with atomic writes and automatic directory creation. | exp: func:getCached(hash: string, cacheDir: string) → string | undefined, call:getCachePath, call:loadCache, func:setCached(hash: string, result: string, cacheDir: string) → void, call:getCachePath, call:loadCache, call:Date.now, call:saveCache | dep: fs, path
- llm-client.ts | Factory for creating LLM client instances based on different provider modes (pi, openai, kimi). | exp: LLMClient, LLMClientOptions, func:createLLMClient(mode: "pi" | "openai" | "kimi", options: LLMClientOptions) → LLMClient | dep: ./llm-error.js, ./external-llm-client.js, ./kimi-llm-client.js, ./pi-llm-client.js, LLMError, ExternalLLMClient, KimiLLMClient, PiLLMClient
- llm-error.ts | Defines a custom error class for LLM-related errors with optional cause chaining | exp: class:LLMError, method:constructor(message: string, cause: unknown)
- llm-extract.ts | Extracts structured metadata (purpose, dependencies, concepts) from source files and packages using an LLM client, with binary detection, caching, and context window management. | exp: func:extractFileLLM(filePath: string, client: LLMClient, cacheDir: string) → Promise<LLMFileData>, call:isBinaryFile, call:readFileSync, call:createHash("sha256").update(content).digest, call:getCached, call:parseFileResponse, call:statSync, call:buildFilePrompt, call:truncateForContext, call:client.complete, call:setCached, raise:LLMError, func:extractPackageLLM(relativePath: string, fileData: { name: string; purpose: string }[], client: LLMClient, _cacheDir: string) → Promise<LLMPackageData>, call:buildPackagePrompt, call:client.complete, call:parsePackageResponse, call:basename, raise:LLMError | dep: fs, crypto, path, ./llm-client.js, ./llm-cache.js, ./llm-error.js
- pi-llm-client.ts | Implements an LLM client adapter that bridges to Pi's internal AI runtime using its built-in `complete()` function | exp: class:PiLLMClient, method:constructor(extensionContext: unknown), method:complete(prompt: string) → Promise<string>, call:ctx.modelRegistry?.get, call:ctx.modelRegistry?.getApiKeyAndHeaders, call:complete, call:Date.now, call:response.content .filter((c: any) => c.type === "text") .map((c: any) => c.text) .join("") .trim, raise:LLMError, raise:err | dep: ./llm-error.js, ./llm-client.js, @mariozechner/pi-ai
## arch
Adapter pattern for provider-specific LLM clients with a factory; decorator/wrapper pattern for cross-cutting concerns (caching, batching, retries, error handling); functional pipeline for file extraction with binary detection and context window management.
## tags
llm, client, js, raise:llmerror, cache, llmclient, error, constructor
## symbols
- ExternalLLMClient
- KimiLLMClient
- LLMError
- PiLLMClient
- constructor
- complete
- withRetry
- processFiles
## workflows
- change llm behavior
read: external-llm-client.ts, kimi-llm-client.ts, llm-batch.ts
- change llm CLI
read: external-llm-client.ts, kimi-llm-client.ts, llm-client.ts
## dirty
-
+1 -19
View File
@@ -5,6 +5,7 @@ import {
discoverProject, discoverProject,
generateDirectoryArtifacts, generateDirectoryArtifacts,
buildDirectoryContext, buildDirectoryContext,
getAncestorEntries,
} from "./init.js"; } from "./init.js";
import type { DirectoryEntry } from "./discover.js"; import type { DirectoryEntry } from "./discover.js";
import type { LLMClient } from "./llm/llm-client.js"; import type { LLMClient } from "./llm/llm-client.js";
@@ -133,25 +134,6 @@ function countDirectChildren(rootPath: string, relDir: string): number {
}).length; }).length;
} }
function getAncestorEntries(
entries: DirectoryEntry[],
ctx: ReturnType<typeof buildDirectoryContext>,
entry: DirectoryEntry,
): DirectoryEntry[] {
const result: DirectoryEntry[] = [];
let parent = ctx.parentMap.get(entry.relativePath);
while (parent) {
const ancestor = entries.find(
(candidate) => candidate.relativePath === parent,
);
if (ancestor) {
result.push(ancestor);
}
parent = ctx.parentMap.get(parent);
}
return result;
}
function normalizeRelativePath(relPath: string): string { function normalizeRelativePath(relPath: string): string {
if (!relPath || relPath === ".") return "."; if (!relPath || relPath === ".") return ".";
return relPath.replace(/\\/g, "/"); return relPath.replace(/\\/g, "/");
+1 -1
View File
@@ -291,6 +291,6 @@ function renderNoResultsBundle(query: string): string {
"-", "-",
"", "",
"## instructions", "## instructions",
"No relevant directories found. Try rephrasing the query or run `project_map_reinit` if artifacts are stale.", "No relevant directories found. Try rephrasing the query, or run `project_map_validate` to check whether artifacts are stale.",
].join("\n"); ].join("\n");
} }
+20
View File
@@ -0,0 +1,20 @@
# src/types (index)
dir: src/types
## role
Provides TypeScript type declarations for the Pi AI module's LLM chat completion functionality within the Pi runtime.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- pi-ai.d.ts
## links
index: src/types/.pi-map.index.md
map: src/types/.pi-map.md
## workflows
- change types behavior
read: pi-ai.d.ts
## dirty
-
+20
View File
@@ -0,0 +1,20 @@
# src/types
dir: src/types
index: src/types/.pi-map.index.md
## role
Provides TypeScript type declarations for the Pi AI module's LLM chat completion functionality within the Pi runtime.
## files
- pi-ai.d.ts | TypeScript declaration file for the Pi AI module's `complete` function that provides LLM chat completions within the Pi runtime | exp: complete
## arch
Minimal declaration-only types package using ambient module declarations (.d.ts) to define external API interfaces without implementation.
## tags
complete, pi, ai.d, typescript, declaration, provides, llm, chat
## symbols
- complete
## workflows
- change types behavior
read: pi-ai.d.ts
## dirty
-
+37
View File
@@ -0,0 +1,37 @@
# tests (index)
dir: tests
## role
Comprehensive test suite for a project mapping tool that generates AI-readable codebase documentation with LLM integration, caching, and CLI query capabilities.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
-
## files
- ast-extract.test.ts
- cli.test.ts
- format.test.ts
- integration.test.ts
- llm-batch.test.ts
- llm-cache.test.ts
- llm-extract.test.ts
- llm-integration.test.ts
- merge.test.ts
- mock-llm.ts
- pi-extension.test.ts
- prompt-injection.test.ts
- retrieve.test.ts
- routing-metadata.test.ts
## links
index: tests/.pi-map.index.md
map: tests/.pi-map.md
## workflows
- change tests behavior
read: mock-llm.ts
- update tests tests
read: ast-extract.test.ts, cli.test.ts, format.test.ts
- change tests CLI
read: cli.test.ts
## dirty
-
+38
View File
@@ -0,0 +1,38 @@
# tests
dir: tests
index: tests/.pi-map.index.md
## role
Comprehensive test suite for a project mapping tool that generates AI-readable codebase documentation with LLM integration, caching, and CLI query capabilities.
## files
- ast-extract.test.ts | Tests AST extraction of TypeScript exports, imports, and dependency resolution with fallback for unsupported file types | dep: vitest, ../src/ast/ast-extract.js, fs, path, os, ast-extract.js
- cli.test.ts | Integration tests for a CLI tool that queries project context bundles from `.pi-map.md` and `.pi-map.index.md` files | dep: vitest, fs, path, os, child_process, url
- format.test.ts | Tests markdown rendering and parsing functions for package maps, directory maps, and directory indexes in a project mapping tool. | dep: vitest, ../src/format.js, ../src/directory-model.js
- integration.test.ts | Integration tests for a project mapping tool that generates and maintains .pi-map.md and .pi-map.index.md files across a codebase. | dep: vitest, fs, path, os, ../src/init.js, ../src/patch.js, ../src/validate.js, ./mock-llm.js, ../src/format.js
- llm-batch.test.ts | Unit tests for retry and batch processing utilities in an LLM module | dep: vitest, ../src/llm/llm-batch.js, ../src/llm-error.js
- llm-cache.test.ts | Tests a file-based caching system for LLM responses with get/set operations and cleanup. | dep: vitest, ../src/llm/llm-cache.js, fs, path, os
- llm-extract.test.ts | Unit tests for LLM-based file extraction with mock client, testing file size limits, binary detection, and response parsing | dep: vitest, ../src/llm/llm-extract.js, fs, path, os, ../src/llm/llm-client.js
- llm-integration.test.ts | Integration tests for LLM client functionality including Kimi API calls, file/package extraction, caching, parallel processing, and error handling | dep: vitest, fs, path, os, ../src/llm/llm-client.js, ../src/llm/llm-extract.js, ../src/llm/llm-batch.js, llm-client, llm-extract, llm-batch
- merge.test.ts | Tests that mergeFileData normalizes multi-line function/method parameters and return types into single-line export signatures | dep: vitest, ../src/merge.js
- mock-llm.ts | Provides mock LLM client implementations for testing purposes | exp: func:createMockFileClient(purpose) → LLMClient, func:createMockPackageClient() → LLMClient | dep: ../src/llm/llm-client.js, llm-client.js
- pi-extension.test.ts | Tests a Pi coding agent extension that manages project map initialization, patching, validation, reinitialization, and context retrieval with configurable prompt injection modes. | dep: vitest, fs, path, os, ../src/prompt-injection.js, ../pi-extension.js, @mariozechner/pi-coding-agent, @mariozechner/pi-ai, typebox
- prompt-injection.test.ts | Tests a prompt injection mitigation system that manages root-pair markers, context budgets, mode-based injection policies, and bypass detection for LLM interactions. | dep: vitest, fs, path, os, ../src/prompt-injection.js
- retrieve.test.ts | Tests the `retrieveContext` function that searches and ranks project map files to build context bundles for AI queries. | dep: vitest, fs, path, os, ../src/retrieve.js
- routing-metadata.test.ts | Tests the `populateRoutingMetadata` function which generates tags, symbols, and workflow hints from directory models for code navigation/routing purposes | dep: vitest, ../src/directory-model.js, ../src/routing-metadata.js
## arch
Layered testing architecture with unit, integration, and mock layers; uses file-based fixtures, mock LLM clients, and tests across AST extraction, LLM batching/caching, markdown rendering, context retrieval, and prompt injection security.
## tags
llm, js, src, tests, vitest, client, fs, path
## symbols
- createMockFileClient
- createMockPackageClient
## workflows
- change tests behavior
read: mock-llm.ts
- update tests tests
read: ast-extract.test.ts, cli.test.ts, format.test.ts
- change tests CLI
read: cli.test.ts
## dirty
-
+186 -1
View File
@@ -8,7 +8,7 @@ import {
} from "fs"; } from "fs";
import { join } from "path"; import { join } from "path";
import { tmpdir } from "os"; import { tmpdir } from "os";
import { initProject } from "../src/init.js"; import { initProject, reinitPath } from "../src/init.js";
import { patchFile } from "../src/patch.js"; import { patchFile } from "../src/patch.js";
import { validateMaps } from "../src/validate.js"; import { validateMaps } from "../src/validate.js";
import { createMockFileClient } from "./mock-llm.js"; import { createMockFileClient } from "./mock-llm.js";
@@ -356,4 +356,189 @@ describe("integration", () => {
readFileSync(join(dir, "src", ".pi-map.index.md"), "utf8"), readFileSync(join(dir, "src", ".pi-map.index.md"), "utf8"),
).toContain("# src (index)"); ).toContain("# src (index)");
}); });
it("reinit on root regenerates all artifacts", async () => {
mkdirSync(join(dir, "src"));
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
const client = createMockFileClient();
await initProject(dir, { llmClient: client, verbose: false });
writeFileSync(
join(dir, ".pi-map.md"),
readFileSync(join(dir, ".pi-map.md"), "utf8") + "\nCORRUPTED",
);
await reinitPath(dir, { llmClient: client, verbose: false });
expect(readFileSync(join(dir, ".pi-map.md"), "utf8")).not.toContain(
"CORRUPTED",
);
});
it("reinit on a small subtree regenerates subtree and ancestors but not siblings", async () => {
mkdirSync(join(dir, "src"));
mkdirSync(join(dir, "lib"));
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
for (let i = 0; i < 10; i++) {
writeFileSync(
join(dir, "lib", `b${i}.ts`),
`export const b${i} = ${i};\n`,
);
}
writeFileSync(
join(dir, ".pi-project-map.json"),
JSON.stringify({ reinitFullThresholdPercent: 10 }),
);
const client = createMockFileClient();
await initProject(dir, { llmClient: client, verbose: false });
writeFileSync(
join(dir, "src", ".pi-map.md"),
readFileSync(join(dir, "src", ".pi-map.md"), "utf8") + "\nSUBTREE_MARKER",
);
writeFileSync(
join(dir, "lib", ".pi-map.md"),
readFileSync(join(dir, "lib", ".pi-map.md"), "utf8") + "\nSIBLING_MARKER",
);
await reinitPath(join(dir, "src"), {
llmClient: client,
verbose: false,
});
expect(readFileSync(join(dir, "src", ".pi-map.md"), "utf8")).not.toContain(
"SUBTREE_MARKER",
);
expect(readFileSync(join(dir, "lib", ".pi-map.md"), "utf8")).toContain(
"SIBLING_MARKER",
);
});
it("reinit falls back to full regeneration when subtree exceeds threshold", async () => {
mkdirSync(join(dir, "src"));
mkdirSync(join(dir, "lib"));
for (let i = 0; i < 11; i++) {
writeFileSync(
join(dir, "src", `a${i}.ts`),
`export const a${i} = ${i};\n`,
);
}
writeFileSync(join(dir, "lib", "b.ts"), `export const b = 1;\n`);
writeFileSync(
join(dir, ".pi-project-map.json"),
JSON.stringify({ reinitFullThresholdPercent: 10 }),
);
const client = createMockFileClient();
await initProject(dir, { llmClient: client, verbose: false });
writeFileSync(
join(dir, "lib", ".pi-map.md"),
readFileSync(join(dir, "lib", ".pi-map.md"), "utf8") + "\nSIBLING_MARKER",
);
await reinitPath(join(dir, "src"), {
llmClient: client,
verbose: false,
});
expect(readFileSync(join(dir, "lib", ".pi-map.md"), "utf8")).not.toContain(
"SIBLING_MARKER",
);
});
it("reinit respects a custom reinitFullThresholdPercent", async () => {
mkdirSync(join(dir, "src"));
mkdirSync(join(dir, "lib"));
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
writeFileSync(join(dir, "lib", "b.ts"), `export const b = 1;\n`);
writeFileSync(
join(dir, ".pi-project-map.json"),
JSON.stringify({ reinitFullThresholdPercent: 50 }),
);
const client = createMockFileClient();
await initProject(dir, { llmClient: client, verbose: false });
writeFileSync(
join(dir, "lib", ".pi-map.md"),
readFileSync(join(dir, "lib", ".pi-map.md"), "utf8") + "\nSIBLING_MARKER",
);
await reinitPath(join(dir, "src"), {
llmClient: client,
verbose: false,
});
expect(readFileSync(join(dir, "lib", ".pi-map.md"), "utf8")).toContain(
"SIBLING_MARKER",
);
});
it("reinit on a small subtree regenerates ancestors up to the root", async () => {
mkdirSync(join(dir, "src"));
mkdirSync(join(dir, "lib"));
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
for (let i = 0; i < 10; i++) {
writeFileSync(
join(dir, "lib", `b${i}.ts`),
`export const b${i} = ${i};\n`,
);
}
writeFileSync(
join(dir, ".pi-project-map.json"),
JSON.stringify({ reinitFullThresholdPercent: 10 }),
);
const client = createMockFileClient();
await initProject(dir, { llmClient: client, verbose: false });
writeFileSync(
join(dir, ".pi-map.md"),
readFileSync(join(dir, ".pi-map.md"), "utf8") + "\nROOT_MARKER",
);
writeFileSync(
join(dir, "lib", ".pi-map.md"),
readFileSync(join(dir, "lib", ".pi-map.md"), "utf8") + "\nSIBLING_MARKER",
);
await reinitPath(join(dir, "src"), {
llmClient: client,
verbose: false,
});
expect(readFileSync(join(dir, ".pi-map.md"), "utf8")).not.toContain(
"ROOT_MARKER",
);
expect(readFileSync(join(dir, "lib", ".pi-map.md"), "utf8")).toContain(
"SIBLING_MARKER",
);
});
it("reinit on a non-existent path falls back to full regeneration", async () => {
mkdirSync(join(dir, "src"));
mkdirSync(join(dir, "lib"));
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
writeFileSync(join(dir, "lib", "b.ts"), `export const b = 1;\n`);
writeFileSync(
join(dir, ".pi-project-map.json"),
JSON.stringify({ reinitFullThresholdPercent: 10 }),
);
const client = createMockFileClient();
await initProject(dir, { llmClient: client, verbose: false });
writeFileSync(
join(dir, "lib", ".pi-map.md"),
readFileSync(join(dir, "lib", ".pi-map.md"), "utf8") + "\nSIBLING_MARKER",
);
await reinitPath(join(dir, "does-not-exist"), {
llmClient: client,
verbose: false,
});
expect(readFileSync(join(dir, "lib", ".pi-map.md"), "utf8")).not.toContain(
"SIBLING_MARKER",
);
});
}); });
+52 -1
View File
@@ -16,6 +16,7 @@ vi.mock("typebox", () => ({
Object: (props: unknown) => props, Object: (props: unknown) => props,
Optional: (prop: unknown) => prop, Optional: (prop: unknown) => prop,
String: (opts: unknown) => ({ type: "string", ...(opts as object) }), String: (opts: unknown) => ({ type: "string", ...(opts as object) }),
Boolean: (opts: unknown) => ({ type: "boolean", ...(opts as object) }),
}, },
})); }));
@@ -70,6 +71,9 @@ describe("pi-extension", () => {
})), })),
}, },
ui: { notify: mockNotify }, ui: { notify: mockNotify },
sessionManager: {
buildSessionContext: vi.fn(() => ({ messages: [] })),
},
}; };
const mockPi = { const mockPi = {
@@ -248,7 +252,7 @@ describe("pi-extension", () => {
await handler(null, mockCtx); await handler(null, mockCtx);
expect(mockNotify).toHaveBeenCalledWith( expect(mockNotify).toHaveBeenCalledWith(
expect.stringContaining("1 dirty packages detected"), expect.stringContaining("1 dirty package(s) detected"),
"warning", "warning",
); );
}); });
@@ -505,6 +509,53 @@ describe("pi-extension", () => {
expect(result.message.content).toContain("Updated"); expect(result.message.content).toContain("Updated");
expect(result.message.content).toContain("TestIndex"); expect(result.message.content).toContain("TestIndex");
}); });
it("skips injection when hint is already present in session context", async () => {
const dir = mkdtempSync(join(tmpdir(), "pi-ext-test-"));
writeFileSync(join(dir, ".pi-map.md"), "# .\n## role\nTest\n");
writeFileSync(
join(dir, ".pi-project-map.json"),
JSON.stringify({ promptInjectionMode: "advisory" }),
);
mockCtx.cwd = dir;
mockCtx.sessionManager.buildSessionContext = vi.fn(() => ({
messages: [
{
role: "custom",
customType: "pi-project-map-hint",
content: "existing hint",
display: false,
},
],
}));
const handler = registeredEvents.before_agent_start;
const result = await handler(null, mockCtx);
expect(result).toEqual({});
});
it("injects hint again when session context does not contain it", async () => {
const dir = mkdtempSync(join(tmpdir(), "pi-ext-test-"));
writeFileSync(join(dir, ".pi-map.md"), "# .\n## role\nTest\n");
writeFileSync(
join(dir, ".pi-project-map.json"),
JSON.stringify({ promptInjectionMode: "advisory" }),
);
mockCtx.cwd = dir;
mockCtx.sessionManager.buildSessionContext = vi.fn(() => ({
messages: [
{ role: "user", content: "hello" },
{ role: "assistant", content: [{ type: "text", text: "hi" }] },
],
}));
const handler = registeredEvents.before_agent_start;
const result = await handler(null, mockCtx);
expect(result).toHaveProperty("message");
expect(result.message.customType).toBe("pi-project-map-hint");
});
}); });
describe("context event", () => { describe("context event", () => {
+186
View File
@@ -0,0 +1,186 @@
# Troubleshooting pi-project-map
## Stale map issues
### Symptom
`project_map_validate` reports `missing`, `orphaned`, or `stale-signature` discrepancies, or a directory map describes files that no longer exist.
### Likely cause
A source file was edited, added, or deleted without running `project_map_patch` or `project_map_reinit`.
### What to do
1. run `project_map_validate` to inspect discrepancies
2. if the list is small and localized, run `project_map_patch <changed-file>` or `project_map_validate` with `fix=true`
3. if the list is large or structural, run `project_map_reinit`
4. re-run `project_map_validate` to confirm clean state
### Prevention
- patch immediately after editing source
- in Pi, use the `project_map_patch` tool with the changed file path
- watch for the `session_start` dirty warning
## Validate discrepancies
### Symptom
`validate --fix` fails with `validate --fix requires an LLM client`.
### Cause
Repair needs an LLM client to regenerate stale artifacts.
### Fix
- CLI: provide `--llm-provider=...` and the required API key
- Pi: use the tool surface so the runtime provides the LLM client
### Symptom
Many `stale-signature` discrepancies appear.
### Cause
AST exports no longer match listed exports, or generated signatures are stale.
### Fix
- patch the changed file/directory, or run `project-map validate --fix`
- if widespread, reinit and validate again
### Symptom
`broken-link` appears after moving directories.
### Fix
Run `project-map validate --fix` first. If the damage spans many directories, run `project_map_reinit`.
## Prompt injection mode surprises
### Symptom
No project-map context appears in a Pi session.
### Checks before init
1. confirm `.pi-project-map.json` does not set `promptInjectionMode` to `off`
2. confirm root `.pi-map.md` / `.pi-map.index.md` do not exist yet
3. confirm the session is at agent start; the pre-init hint is emitted from `before_agent_start`
### Checks after init
1. confirm root `.pi-map.md` and `.pi-map.index.md` exist
2. confirm mode is `strong` or `strict` if you expect automatic artifact injection
3. confirm the outgoing context does not already contain the root-pair marker
4. confirm the turn is actually relevant (`agent_start`, `edit_intent`, `architecture_sensitive`, `compaction`, `artifact_change`)
### Symptom
Root pair is injected repeatedly.
### Likely cause
Marker scanning failed or artifact invalidation forced reinjection, often because `project_map_reinit` is being used instead of `project_map_patch` or `project_map_validate --fix`.
### Fix
- inspect whether `<!-- PI_MAP_ROOT_PAIR_START -->` is present in outgoing context
- check whether root artifacts changed on disk
- confirm payload/message serialization still exposes marker text
- prefer `project_map_patch <file>` for edits and `project_map_validate` with `fix=true` for localized discrepancies
### Symptom
`advisory` mode shows a reminder but no maps are loaded.
### Expected behavior
That is intentional. Advisory mode is reminder-only; read the root pair manually.
## Strict bypass confusion
### Symptom
A sensitive edit is blocked with the strict guard.
### What is happening
In `strict` mode, the turn was classified as sensitive and the protocol path is missing from outgoing context.
### Resolution options
1. restore the protocol path by ensuring the root pair and trust boundary are present in context
2. include an explicit bypass marker:
```text
[PI_MAP_BYPASS: emergency fix needed without full context]
```
3. switch to `strong` mode if strict is too noisy for the task
### Symptom
Bypass marker is ignored.
### Cause
The marker must contain a non-empty reason. Empty or whitespace-only reasons are rejected.
### Symptom
Strict guard triggers on obviously non-sensitive turns.
### Cause
Heuristic sensitive-turn detection can false-positive on words like `refactor`, `redesign`, or `dependency graph`.
## LLM / provider / cache issues
### Symptom
`project-map init` fails with `No LLM client configured`.
### CLI fix
- set `OPENAI_API_KEY` for OpenAI provider
- set `KIMI_API_KEY` for Kimi provider
- or configure provider/model in `.pi-project-map.json`
### Pi fix
- select a model in Pi
- log in if the provider requires credentials
- ensure Pi runtime LLM access is working
### Symptom
External/provider request fails.
### Checks
1. API key validity
2. network connectivity
3. `llmBaseUrl` correctness if using a proxy
4. provider rate limits
### Symptom
Second init still makes many LLM calls.
### Cause
Cache miss.
### Checks
- consistent cache directory between runs
- cache file exists and is readable
- file contents did not change
### Symptom
Cache file is huge.
### Cause
There is no eviction policy yet.
### Fix
Remove the local cache directory if needed.
## Test / lint expectations
### Main checks
```bash
npm test
npm run typecheck
npm run build
node dist/cli.js validate .
```
### Lint note
`npm run lint` is currently not a reliable gate if the repo has no ESLint config. Treat missing-lint-config failures as repository hygiene issues, not feature regressions.
### Common failures
| Failure | Likely cause |
|---------|--------------|
| Pi LLM client tests fail | Pi runtime or mocks drifted |
| Prompt-injection sequence tests fail | Reinjection / strict logic changed |
| Validate tests fail | Format parsing, merge encoding, or discovery rules changed |
| AST tests fail | tree-sitter grammar/runtime drift |
## Known limitations
- token budgeting is heuristic, not tokenizer-exact
- retrieval is deterministic, not semantic-search-driven
- strict mode is a prompt-level guard, not a hard tool sandbox
- payload scanning assumes string or JSON-serializable structures
+211
View File
@@ -0,0 +1,211 @@
# pi-project-map Usage Guide
For Pi agents and advanced users who want predictable, low-friction navigation and maintenance of project-map artifacts.
## When to use each command
| Situation | Command |
|-----------|---------|
| New project, no `.pi-map.md` files yet, or artifacts are severely outdated | `project_map_init` / `project-map init` |
| You just edited one or more source files | `project_map_patch <file>` / `project-map patch <file>` |
| You suspect stale data, or you are about to make an architectural decision | `project_map_validate` / `project-map validate` |
| Localized discrepancies detected by validate | `project_map_validate` with `fix=true` / `project-map validate --fix` |
| Widespread structural staleness, or you pulled major changes from version control | `project_map_reinit` / `project-map reinit` |
| You have a specific question like "where is auth handled?" | `project_map_context <query>` / `project-map context <query>` |
## Command behavior
### init
Run once when you start work on a repo, or after large restructuring. It discovers every non-ignored directory, analyzes files with LLM + AST, and writes both `.pi-map.md` and `.pi-map.index.md` for every directory.
### patch
Run **immediately after editing a source file**. The command regenerates the pair for that file's directory and refreshes ancestor artifacts.
Patch mode is chosen automatically:
- **small** - refresh ancestor indexes only
- **structural** - refresh ancestor map/index pairs
You can force a mode with `project-map patch <file> --patch-mode=small|structural`.
`auto` remains the default.
### validate
Run before architectural decisions, broad refactors, or final handoff. It checks for:
- missing/orphaned files
- stale signatures
- dirty markers
- broken parent/children/sibling links
- map/index disagreements
Use `project-map validate --fix` to repair affected chains. `--fix` requires an LLM client.
### reinit
Use as a last resort. By default, `project_map_reinit` (or `project-map reinit [path]`) regenerates only the target subtree plus its ancestor directories up to the root. It falls back to full regeneration only when the target subtree covers more than `reinitFullThresholdPercent` of the project's files (default: 10%).
Reach for reinit when `project_map_patch` and `project_map_validate --fix` cannot repair widespread structural damage (e.g. broken parent/child links across many directories) or after a large merge.
### context
Use when you know what you are looking for:
```bash
project-map context "authentication logic"
project-map context "routing metadata generation"
project-map context "LLM client error handling"
```
The returned bundle is deterministic and ranked. Read indexes first, then strongest-match maps, then the actual source files.
## Practical workflows
### Starting a new task in a mapped project
1. read the root `.pi-map.index.md` and the `Project Map Protocol`
2. use the root index to find the relevant child directory
3. read that directory's `.pi-map.index.md`, then its `.pi-map.md`
4. read the relevant source files
5. edit source
6. run `project_map_patch <changed-file>`
7. run tests/build
8. run `project_map_validate` before architectural summary or handoff
9. if validation reports localized discrepancies, run `project_map_validate` with `fix=true` (or `project-map validate --fix`)
10. only if discrepancies are widespread or structural, run `project_map_reinit`
### Exploring an unfamiliar area
```bash
project-map context "how is validation implemented"
```
Treat the returned bundle as a ranked entry point, not as truth.
### After pulling changes from version control
```bash
project-map validate
# if localized discrepancies:
project-map validate --fix
# if widespread structural damage:
project-map reinit
```
### Before a big refactor
```bash
project-map validate
# if localized discrepancies:
project-map validate --fix
# if widespread structural damage:
project-map reinit
```
## Prompt injection behavior before and after init
### Before init
If no paired artifacts exist:
- `off`: nothing happens
- `advisory`, `strong`, `strict`: a visible hint appears telling you to run `project_map_init`
No map content is fabricated.
### After init
Once the root pair exists:
- `off`: no automatic injection
- `advisory`: a visible reminder that maps are available, but the root pair is **not** auto-loaded
- `strong`: root pair is auto-loaded; additional pairs are added within the budget; reinjection happens on relevant turns
- `strict`: same as `strong`, plus enforcement of the protocol path for sensitive actions
Relevant turns that trigger reinjection in `strong`/`strict`:
- agent start
- edit intent
- architecture-sensitive reasoning
- compaction
- root-pair artifact changes
## Advisory vs strong vs strict in practice
| Concern | Use |
|---------|-----|
| You want maps available but do not want automatic context expansion | `advisory` |
| Normal daily work; you want routing/orientation preloaded without friction | `strong` (default) |
| High-stakes codebase or you want explicit justification before bypassing context discipline | `strict` |
| You prefer fully manual control | `off` |
In `strict`, if you attempt a sensitive edit or architectural claim without the protocol path in context, a guard appears. To proceed, either restore the project-map context or include:
```text
[PI_MAP_BYPASS: editing a one-line comment, map context not needed]
```
Use bypass markers sparingly.
## Example task flows
### Fix a bug in `src/utils/validation.ts`
```text
1. project-map context "validation utilities"
2. Read src/utils/.pi-map.index.md and .pi-map.md
3. Read src/utils/validation.ts
4. Edit the file
5. project_map_patch src/utils/validation.ts
6. Run tests
7. project_map_validate
```
### Add a new file to `src/llm/`
```text
1. Create src/llm/new-client.ts
2. Implement the file
3. project_map_patch src/llm/new-client.ts
4. project_map_validate
```
### Review architecture before approving a PR
```text
1. project-map validate
2. If localized discrepancies: project-map validate --fix
3. If clean, read root .pi-map.md and key directory maps
4. Cross-check claims against source
5. Only if widespread: run project-map reinit
```
## Retrieval vs automatic injection
| | Automatic injection | `project_map_context` / `project-map context` |
|---|---|---|
| Trigger | Configured mode + relevant turn | Explicit request |
| Content | Root pair + budgeted expansion | Top-ranked directories for a query |
| Cost | No LLM call; reads artifacts | No LLM call; deterministic scoring |
| Best use | Maintain baseline orientation | Targeted navigation for a specific task |
Use both together: injection for baseline orientation, retrieval for focused entry points.
## Keeping artifacts fresh
- patch after every edit
- validate before architectural claims
- use `validate --fix` for localized discrepancies
- use reinit only for widespread structural staleness or after large merges
- watch for the Pi extension warning about dirty packages on session start
## Configuration quick reference
```json
{
"promptInjectionMode": "strong",
"contextBudgetPercent": 15,
"contextBudgetMaxTokens": 100000,
"tagCap": 8,
"workflowHintCap": 5,
"reinitFullThresholdPercent": 10,
"ignorePatterns": ["node_modules", ".git", "dist", "build"]
}
```
Providing `ignorePatterns` replaces the built-in default list, so include any defaults you want to keep.
- lower `contextBudgetPercent` / `contextBudgetMaxTokens` to reduce token use
- raise them if you want deeper auto-loaded context in large projects
- `reinitFullThresholdPercent` controls when `project-map reinit [path]` falls back to full regeneration
- `strict` is the safest enforcement mode; `strong` is the best default for everyday work