Compare commits

...

19 Commits

Author SHA1 Message Date
alex 97a4dd22f5 docs: refresh README 2026-07-27 15:24:01 +02:00
Developer 3e7410b6bd chore: track .pi-map.md and .pi-map.index.md artifacts
Remove the map files from .gitignore so they are committed as project
navigation artifacts, and also unignore them in the sample fixture.
Regenerate all maps so the committed versions reflect the current source.
2026-06-16 14:38:42 +00:00
Developer cb581f44b9 feat: smart subtree-aware reinit and validate --fix
- project_map_reinit now regenerates only the target subtree + ancestors
  by default, falling back to full reinit when subtree file count exceeds
  reinitFullThresholdPercent (default 10%).
- Add reinitFullThresholdPercent config option.
- Expose fix=true on project_map_validate Pi tool for localized repair.
- Update docs and runtime guidance to prefer patch / validate --fix
  before full reinit.
- Add integration tests for smart reinit and update typebox mock.
2026-06-16 11:46:48 +00:00
Developer 5f1c107667 feat: avoid duplicate project-map hint injection by checking context
The before_agent_start handler now scans the active session context via
ctx.sessionManager.buildSessionContext() for an existing pi-project-map-hint
custom message and skips injection when one is already present in the
current branch. This prevents duplicate visible hints in advisory/pre-init
modes and duplicate hidden hints in strong/strict modes. The hint is
automatically re-injected after compaction or /tree navigation removes it
from the active path.

Also removes the unused hooks/on-prompt.ts prompt-text injector.
2026-06-14 08:57:23 +00:00
Developer 6bc7be4c21 feat: avoid duplicate project-map hint injection by checking context
The before_agent_start handler now inspects the current session context
and skips injection if a pi-project-map-hint custom message is already
present in the active branch. This prevents duplicate hints on every
prompt while still re-injecting after compaction or tree navigation.

Also removes the unused hooks/on-prompt.ts prompt-text injector.
2026-06-14 08:53:26 +00:00
alex fb302a033e docs: rewrite documentation system 2026-06-12 12:04:28 +02:00
alex 842dcc6235 spec(openspec): archive completed project-map changes 2026-06-12 10:06:30 +02:00
alex 11365fa4ed docs(prompt): align prompt injection guidance and runtime copy 2026-06-11 23:21:19 +02:00
alex 0cf4060c4b chore(repo): ignore local runtime and cache artifacts 2026-06-11 23:20:56 +02:00
alex c11d49d015 spec(prompt): add project map prompt injection change set 2026-06-11 22:56:20 +02:00
alex 58e8bd31d3 feat(prompt): implement prompt injection slice 4 2026-06-11 22:54:51 +02:00
alex 19666c900e feat(prompt): implement prompt injection slice 3 2026-06-11 21:32:38 +02:00
alex 621434b6e8 feat(prompt): implement prompt injection slice 2 2026-06-11 17:27:31 +02:00
alex 56560d9d56 feat(prompt): implement prompt injection slice 1 2026-06-11 17:01:10 +02:00
alex c6064f8d94 Implement layered maps and context retrieval 2026-06-11 12:56:18 +02:00
alex 010e4b83eb feat: render ASCII progress bar in Pi extension updates
- Add renderProgressBar() helper in pi-extension.ts
- Pi init/reinit now show [████░░░░░░] 3/20 → filename.ts
- Replaces plain text percentage with visual bar
- All 52 tests passing
2026-06-10 18:53:49 +02:00
alex 9bdb862cca feat: add progress bar to init/reinit with per-file tracking
- processFiles() now accepts onProgress callback (completed, total, currentFile)
- InitOptions.onProgress receives structured ProgressInfo object
- initProject aggregates progress across all directories
- CLI renders ASCII progress bar: [████░░░░░░] 3/20 | filename.ts
- Pi extension sends structured updates with percentage + file details
- All 52 tests passing
2026-06-10 18:43:42 +02:00
alex 78c60f9ca1 refactor: simplify cache location to .cache/llm-cache.json
- Cache now lives at <project>/.cache/llm-cache.json (single hidden dir)
- Remove extra pi-project-map subdirectory level
- Use dirname() instead of string slicing for robustness
2026-06-10 18:07:47 +02:00
alex dd8ac3a604 refactor: move cache to .cache/pi-project-map/ (hidden by default)
- Change cache location from .pi-project-map/cache/ to .cache/pi-project-map/
- Cache is now hidden by default (dot directory)
- Add .cache/ to .gitignore
- Remove committed .pi-project-map/cache/ from repo
2026-06-10 17:42:38 +02:00
110 changed files with 11316 additions and 1792 deletions
+18
View File
@@ -0,0 +1,18 @@
# .atl (index)
dir: .atl
## role
Empty directory placeholder, likely intended for Atlassian tool configuration or automation artifacts that have not yet been populated.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
-
## files
## links
index: .atl/.pi-map.index.md
map: .atl/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# .atl
dir: .atl
index: .atl/.pi-map.index.md
## role
Empty directory placeholder, likely intended for Atlassian tool configuration or automation artifacts that have not yet been populated.
## files
## arch
N/A - no files or architectural patterns present in this directory.
## tags
-
## symbols
-
## workflows
-
## dirty
-
+6 -1
View File
@@ -4,6 +4,11 @@ coverage/
*.log *.log
.DS_Store .DS_Store
.env .env
.pi-map.md
# Local Pi runtime state # Local Pi runtime state
.atl/ .atl/
.pi
.cache/
cache/
subagent-outputs/
IMPLEMENTATION_REPORT.md
context.md
+61
View File
@@ -0,0 +1,61 @@
# . (index)
dir: .
## Project Map Protocol
1. Read this protocol and the root `.pi-map.index.md` first.
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
3. Load indexes before rich maps during task-start navigation.
4. Read the local rich map and actual source before editing.
5. Treat non-empty `## dirty` sections in either artifact as stale.
6. If source and generated artifacts disagree, trust source.
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
8. After editing source, run `project_map_patch` for each changed file.
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
Trust boundary: index routes, map orients, source decides.
## role
A TypeScript/Node.js CLI tool and Pi extension that generates paired markdown analysis artifacts (.pi-map.index.md and .pi-map.md) to provide hierarchical codebase navigation and contextual orientation for AI coding agents.
## parent
-
## children
- .atl
index: .atl/.pi-map.index.md
map: .atl/.pi-map.md
- fixtures
index: fixtures/.pi-map.index.md
map: fixtures/.pi-map.md
- openspec
index: openspec/.pi-map.index.md
map: openspec/.pi-map.md
- src
index: src/.pi-map.index.md
map: src/.pi-map.md
- tests
index: tests/.pi-map.index.md
map: tests/.pi-map.md
## files
- .gitignore
- .npmrc
- README.md
- SKILL.md
- design-doc.md
- package-lock.json
- package.json
- pi-extension.ts
- troubleshooting.md
- tsconfig.json
- usage-guide.md
## links
index: ./.pi-map.index.md
map: ./.pi-map.md
## workflows
- change project behavior
read: .gitignore, .npmrc, pi-extension.ts
- change project config
read: package-lock.json, package.json, tsconfig.json
- explore project subdirectories
index: .atl/.pi-map.index.md, fixtures/.pi-map.index.md, openspec/.pi-map.index.md
## dirty
-
+48
View File
@@ -0,0 +1,48 @@
# .
dir: .
index: ./.pi-map.index.md
## Project Map Protocol
1. Read this protocol and the root `.pi-map.index.md` first.
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
3. Load indexes before rich maps during task-start navigation.
4. Read the local rich map and actual source before editing.
5. Treat non-empty `## dirty` sections in either artifact as stale.
6. If source and generated artifacts disagree, trust source.
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
8. After editing source, run `project_map_patch` for each changed file.
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
Trust boundary: index routes, map orients, source decides.
## role
A TypeScript/Node.js CLI tool and Pi extension that generates paired markdown analysis artifacts (.pi-map.index.md and .pi-map.md) to provide hierarchical codebase navigation and contextual orientation for AI coding agents.
## files
- .gitignore | Specifies files and directories for Git to ignore in a Node.js/TypeScript project with Pi tooling integration | dep: git
- .npmrc | Configures npm to use legacy peer dependency resolution behavior | dep: npm
- README.md | Documents a CLI tool and Pi extension that generates paired machine-readable analysis artifacts (.pi-map.index.md and .pi-map.md) for hierarchical codebase navigation and agent orientation. | dep: npm, Node.js, Pi runtime environment, LLM provider (OpenAI, etc.), file system
- SKILL.md | Defines a Pi skill that generates and maintains hierarchical paired project-analysis artifacts (`.pi-map.index.md` + `.pi-map.md`) to enable AI agents to navigate codebases without reading every source file. | dep: markdown, AST parsing, LLM API, JSON configuration, CLI/tool interface
- design-doc.md | A design document explaining the internal architecture of `pi-project-map`, a TypeScript/Node.js tool that generates and maintains hierarchical paired markdown artifacts (`.pi-map.index.md` and `.pi-map.md`) to serve as navigation aids for AI coding agents, with both CLI and Pi extension runtime modes. | dep: TypeScript, Node.js, tree-sitter, ignore, LLM client (PiLLMClient/ExternalLLMClient/KimiLLMClient)
- package-lock.json | Auto-generated npm lock file that records exact dependency versions and tree structure for reproducible installs of the "pi-project-map" Node.js CLI tool. | dep: npm, esbuild, eslint, typescript, vitest, openai, tree-sitter, tree-sitter-python, tree-sitter-typescript, p-limit, picocolors, ignore
- package.json | Pi skill for hierarchical project analysis that generates and maintains .pi-map.md files | dep: ignore, openai, p-limit, picocolors, tree-sitter, tree-sitter-python, tree-sitter-typescript, typescript, vitest, eslint, @types/node, @typescript-eslint
- pi-extension.ts | Pi extension that registers tools for managing project map artifacts (.pi-map.md/.pi-map.index.md) and injects contextual hints into agent sessions based on configuration modes. | dep: @mariozechner/pi-coding-agent, typebox, fs, path, ./src/index.js, ./src/config.js, ./src/llm/llm-client.js, ./src/llm/llm-error.js
- troubleshooting.md | Troubleshooting guide for diagnosing and resolving issues with the pi-project-map tool across validation, prompt injection, strict mode, LLM provider, and testing scenarios. | dep: project_map_validate, project_map_patch, project_map_reinit, LLM client, Pi runtime, tree-sitter, npm
- tsconfig.json | Configures TypeScript compiler options for a Node.js project targeting ES2022 with strict type checking and declaration output
- usage-guide.md | User documentation explaining how to use the pi-project-map tool for navigating and maintaining project-map artifacts in codebases.
## arch
Dual-runtime architecture supporting both standalone CLI and Pi extension modes, using a hierarchical paired-file pattern (index + detail) with TypeScript/Node.js, strict type checking, and configuration-driven behavior injection for agent session integration.
## tags
map, project, tree, typescript, sitter, node, npm, js
## symbols
-
## workflows
- change project behavior
read: .gitignore, .npmrc, pi-extension.ts
- change project config
read: package-lock.json, package.json, tsconfig.json
- explore project subdirectories
index: .atl/.pi-map.index.md, fixtures/.pi-map.index.md, openspec/.pi-map.index.md
## dirty
-
-62
View File
@@ -1,62 +0,0 @@
{
"25c2db9d6c4c2c511bec4a736f5adf77423be6011bbedd916fb55003d059b4e3": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452802
},
"889300ab1d0afe082d820fdd9150229625c3ab69b3b84c2ca65d67479be388cc": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452841
},
"585c5da8de102beb6a9906f0e3f07a134b5392bed52fc1a0aaef989e69aa66b6": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452845
},
"a6f92e629a4c341fe1e10f92ef923c54002b8a79ec3bedb0b9079bc7aca4e2e0": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452846
},
"a6c149b54aec4ced46eb38d6faf1f20ec62fd2f9a2f2dcb6a15c4f339dfb5f1e": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452846
},
"4eda95f6452551a12d5b434e284c2e539ce57e0206d5ffa49609f4fa8354a966": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452846
},
"510456d000664badf3cbec9f70c80769a119a590e84b1971e49ac5728882a699": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452847
},
"b0f5449d50fda0b5c0cf47103188df2d4339b047e1e03cab7a15a431f2990185": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452848
},
"eebe9a750792a3d802c6868f32ac264d44ef193a6f1940c4ad5e715b6e31074a": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452848
},
"4958b5d7ec5b4c975bb1e2972519aff5fbd94cb7342be4f8dd854a5c7353bd26": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452849
},
"389a385ef4fc5d1d253253fa3c931c505c7fbcca68db4ebe6b34392b3ca32d65": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452849
},
"879d2eff31eaf101d30de95e0e2df1c1a3a305c4e3b0515a22c85969af49ff25": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452947
},
"c2f0a91350b7f5e11a95848c4d806e49d2017862acbc43afb1dd52a5e73c0b24": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452948
},
"037ecd1db38c230c248787e60fd7bfc0cb0101b187b59535b6e7483be762d350": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452950
},
"b5d546753d33709dff508a6f2c2e547f432267fbb05b68d3da74fa5f7bda9f7a": {
"result": "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
"ts": 1781105452953
}
}
-1
View File
@@ -1 +0,0 @@
{}
+85 -9
View File
@@ -1,22 +1,98 @@
# pi-project-map # pi-project-map
Pi skill for hierarchical project analysis. A Pi extension and command-line tool that generates paired project-navigation artifacts: `.pi-map.index.md` files for routing and `.pi-map.md` files for directory orientation. The artifacts help agents navigate a codebase; source remains the authority.
## What it does ## Status and limitations
Generates `.pi-map.md` files throughout your project — one per directory — containing a dense, machine-readable summary of that directory's files, exports, dependencies, and architecture. This gives Pi agents instant project comprehension without reading every source file. - Generation and repair commands call an LLM and write map artifacts into the target project.
- `validate` without `--fix` checks artifacts without creating an LLM client; `validate --fix` can modify them and requires an LLM.
- Token budgeting and relevant-turn detection are best-effort. Retrieved maps route and orient work but do not replace source verification.
- No Node.js version requirement or deployment configuration is declared in this repository.
## Quick Start ## Install and develop
Install the published CLI globally:
```bash ```bash
npm install -g pi-project-map npm install -g pi-project-map
project-map init
``` ```
## Design For local development, install dependencies and build from this repository:
See [design-doc.md](design-doc.md) for the full specification. ```bash
npm install
npm run build
```
## Implementation Plan Available development checks:
See [implementation-plan.md](implementation-plan.md) for the engineering roadmap. ```bash
npm run test
npm run lint
npm run typecheck
npm run dev
```
`npm run dev` runs TypeScript in watch mode. The project declares no deployment command.
## Use the CLI
Run the installed `project-map` command from the project to map. Use `--help` or `--version` for CLI metadata.
```bash
project-map init [path]
project-map patch <file>
project-map validate [--fix] [path]
project-map reinit [path]
project-map context <query>
```
Typical workflow:
1. Run `project-map init` to generate maps for a project.
2. After editing a file, run `project-map patch <file>`.
3. Run `project-map validate` before relying on maps; use `--fix` only when you intend to repair artifacts.
4. Use `project-map context "<query>"` to retrieve a ranked context bundle.
`init`, `patch`, `reinit`, and `validate --fix` mutate `.pi-map.md` and/or `.pi-map.index.md` artifacts. `context` retrieves from existing artifacts.
### Pi extension
The repository exposes `pi-extension.ts` as a Pi extension. Inside Pi it uses Pi's configured model and registers equivalent `project_map_*` tools. The extension can also inject project-map guidance according to `promptInjectionMode`.
## LLM configuration
Standalone CLI use needs credentials for the selected provider. Do not commit keys.
- `openai` (the default) reads `OPENAI_API_KEY`; its model can be set with `OPENAI_MODEL` or `LLM_MODEL`.
- `kimi` reads `KIMI_API_KEY` (or `KIMI_COM_API_KEY`); its model can be set with `KIMI_MODEL` or `LLM_MODEL`.
- CLI options `--llm-provider=openai|kimi`, `--llm-model=<model>`, and `--llm-base-url=<url>` override corresponding settings for a command.
Create an optional `.pi-project-map.json` in the project being mapped. It is merged with the defaults:
```json
{
"promptInjectionMode": "strong",
"contextBudgetPercent": 15,
"contextBudgetMaxTokens": 100000,
"llmProvider": "openai",
"llmModel": "gpt-4o-mini",
"ignorePatterns": ["node_modules", ".git"],
"tagCap": 8,
"workflowHintCap": 5
}
```
`promptInjectionMode` accepts `off`, `advisory`, `strong`, or `strict`. Supplying `ignorePatterns` **replaces** the built-in ignore list rather than extending it. Additional supported settings include `llmBaseUrl` and `reinitFullThresholdPercent`.
## Repository layout
- `src/` — CLI, map generation, validation, retrieval, configuration, and LLM clients
- `pi-extension.ts` — Pi extension entry point
- `SKILL.md` — Pi skill definition and operator guidance
- `usage-guide.md`, `design-doc.md`, `troubleshooting.md` — additional usage and design documentation
- `package.json` — package metadata, dependencies, and npm scripts
## Operations
There is no server or deployment manifest. Operate the tool from the project being mapped and keep generated maps under the target project's normal review and version-control practices.
+87 -108
View File
@@ -1,108 +1,95 @@
--- ---
name: pi-map name: pi-map
description: Generates and maintains hierarchical, machine-readable project analysis files (.pi-map.md) for instant codebase comprehension. Use when working with medium-to-large codebases where understanding architecture, file relationships, and exports without reading every file is valuable. Automatically extracts symbols via AST and LLM heuristics. description: Generates and maintains hierarchical paired project-analysis artifacts (`.pi-map.index.md` + `.pi-map.md`) so Pi agents can navigate and orient in a codebase without reading every source file.
--- ---
# pi-project-map # pi-project-map
A Pi skill that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file containing architectural context, exported symbols, and dependencies. A Pi skill that creates and maintains one paired analysis artifact set per non-ignored directory:
## What It Does - `.pi-map.index.md` — routing-first index
- `.pi-map.md` — orientation-first rich map
- **Scans** your entire project and creates one `.pi-map.md` per directory ## What it does
- **Extracts** exports, imports, and dependencies via AST parsing (TypeScript, Python, Go) and LLM heuristics
- **Updates** incrementally when files change (full rewrite for small packages, section-level patch for large)
- **Validates** detects stale entries, missing files, orphaned entries, and changed signatures
## Quick Start - scans the project and emits one paired map/index set per directory
- extracts per-file purpose, dependencies, and concepts via an LLM
- extracts exact exports/imports via AST parsing where supported
- patches artifacts incrementally after source edits
- validates stale, missing, broken, or inconsistent paired artifacts
- retrieves relevant context on demand via deterministic metadata scoring
- injects lightweight project-map guidance into agent context according to a configurable mode ladder
```bash ## Operating model
# Install globally
npm install -g pi-project-map
# Generate analysis files for the entire project ### Tier 0 — protocol
project-map init Always read the root `.pi-map.index.md` and the `Project Map Protocol` first.
# After editing a file, update its directory's analysis ### Tier 1 — routing
project-map patch src/components/Button.tsx Use indexes first to decide where to go next. Open the strongest-match `.pi-map.md` files for orientation.
# Check for staleness ### Tier 2 — source
project-map validate Read actual source, tests, config, and docs before editing or asserting exact runtime behavior.
# Force full regeneration **Trust boundary:**
project-map reinit
```
## Format > **index routes, map orients, source decides**
Each `.pi-map.md` uses dense markdown optimized for LLM consumption: ## Agent instructions
```markdown When project-map artifacts exist in the repo:
# pkg/auth
## role
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
## files
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
## arch
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
## dirty
-
```
### Abbreviations 1. start with the root `.pi-map.index.md` and the `Project Map Protocol`
2. use indexes first to route into the right directory
3. read the local `.pi-map.md` plus relevant source before editing
4. run `project_map_patch <file>` (tool) or `project-map patch <file>` (CLI) after each source edit
5. run `project_map_validate` (tool) or `project-map validate` (CLI) before freshness-sensitive architectural decisions or final handoff
6. if validation shows localized discrepancies, run `project_map_validate` with `fix=true` (tool) or `project-map validate --fix` (CLI)
7. only if discrepancies are widespread or structural, run `project_map_reinit` (tool) or `project-map reinit` (CLI)
8. for targeted exploration, use `project_map_context <query>` (tool) or `project-map context <query>` (CLI)
9. in `strict` mode, only bypass the protocol-path guard with an explicit marker: `[PI_MAP_BYPASS: <brief justification>]`
| Abbreviation | Meaning | ## Prompt injection modes
|-------------|---------|
| `exp:` | Exported symbols |
| `dep:` | Dependencies |
| `pkg/` | Internal package reference |
## Tools The Pi extension can inject project-map guidance automatically. Behavior is controlled by `promptInjectionMode` in `.pi-project-map.json`.
### `project-map:init [root]` ### Before init
Runs a full project scan and generates `.pi-map.md` files in every directory. If no `.pi-map.md` / `.pi-map.index.md` artifacts exist, the extension emits a lightweight visible hint to run `project_map_init`. No synthetic or fake map content is injected.
**Example:** ### After init
```bash Once real artifacts exist, the runtime guarantees that the **root pair** is loaded first:
project-map init
project-map init ~/my-project
```
### `project-map:patch <file-path>` - root `.pi-map.index.md`
Updates the `.pi-map.md` for the directory containing the given file. - root `.pi-map.md`
**Behavior:** Additional directory pairs may be expanded while the configured context budget allows, in shallow-first order.
- Small packages (< 10 files): full rewrite
- Large packages (>= 10 files): section-level patch
**Example:** ### Mode ladder
```bash
project-map patch src/components/Button.tsx
```
### `project-map:validate [root]` | Mode | Behavior |
Checks all `.pi-map.md` files for staleness. |------|----------|
| `off` | No automatic injection. Use tools/CLI manually. |
| `advisory` | Startup/init hints are shown. Root pair is not auto-loaded; read it manually when needed. |
| `strong` (default) | Root pair is auto-loaded, expansion stays within budget, and reinjection runs on relevant turns. |
| `strict` | Same as `strong`, but sensitive edits or architectural claims are guarded unless the protocol path is present or a bypass marker is provided. |
**Detects:** The **protocol path** means the outgoing context contains the canonical injected root-pair block and the trust-boundary text.
- Missing files (new files not yet in `.pi-map.md`)
- Orphaned entries (files listed but deleted)
- Stale signatures (exports changed since last scan)
- Dirty markers (packages flagged for reconciliation)
**Example:** ### Context budget
```bash Default budget: **15% of the active model context window**, capped at **100k tokens**. The smaller of the relative and absolute values wins. If the runtime cannot discover the context window, it uses the absolute cap.
project-map validate
```
### `project-map:reinit [path]` ## Retrieval usage
Force full re-initialization. Clears all dirty markers.
**Example:** `project_map_context` (tool) and `project-map context` (CLI) are **on-demand retrieval**, separate from automatic injection.
```bash
project-map reinit They do not call the LLM. They score paired metadata against the query and return a compact markdown bundle with:
project-map reinit src/components
``` - relevant indexes
- relevant maps
- likely files
- relevant symbols when useful
Always read the suggested indexes first, then maps, then verify critical behavior from source.
## Configuration ## Configuration
@@ -110,43 +97,35 @@ Create `.pi-project-map.json` in the project root:
```json ```json
{ {
"ignorePatterns": ["node_modules", ".git"], "promptInjectionMode": "strong",
"smallPackageThreshold": 10, "contextBudgetPercent": 15,
"contextBudget": 4000, "contextBudgetMaxTokens": 100000,
"autoInjectPrompt": true "tagCap": 8,
"workflowHintCap": 5,
"llmProvider": "openai",
"llmModel": "gpt-4o-mini",
"reinitFullThresholdPercent": 10,
"ignorePatterns": ["node_modules", ".git", "dist", "build"]
} }
``` ```
| Option | Default | Description | Providing `ignorePatterns` replaces the built-in default list, so include any defaults you want to keep.
|--------|---------|-------------|
| `ignorePatterns` | `node_modules`, `.git`, `dist`, etc. | Additional ignore patterns |
| `smallPackageThreshold` | `10` | File count threshold for full rewrite vs patch |
| `contextBudget` | `4000` | Max tokens to spend on analysis files |
| `autoInjectPrompt` | `true` | Auto-inject maintenance instructions |
## Agent Instructions - `reinitFullThresholdPercent` — when `project_map_reinit` targets a subtree that covers more than this percentage of project files, it falls back to full regeneration
When `.pi-map.md` files exist in the project: Knobs that matter in practice:
- `promptInjectionMode``off` | `advisory` | `strong` | `strict`
- `contextBudgetPercent` / `contextBudgetMaxTokens` — caps automatic map/index injection
- `tagCap` / `workflowHintCap` — caps routing metadata per directory
- `llmProvider` / `llmModel` / `llmBaseUrl` — standalone CLI only
- `ignorePatterns` — discovery exclusions
1. **Read them at session start** to build project understanding without scanning every file ## Tools and commands
2. **Run `project-map:patch <path>`** after editing any source file
3. **Run `project-map:validate`** if you suspect staleness before making architectural decisions
4. **Trust the analysis** for orientation, but verify critical details by reading source when needed
## Best Practices | Tool (Pi) | CLI command | Purpose |
|-----------|-------------|---------|
- Run `project-map:init` after cloning a new repository | `project_map_init` | `project-map init [path]` | Generate all paired artifacts. |
- Run `project-map:reinit` periodically (daily/weekly) to catch changes made outside the agent | `project_map_patch` | `project-map patch <file>` | Regenerate the pair for the changed file's directory and refresh ancestors. |
- Add `.pi-map.md` to `.gitignore` — they are derived artifacts | `project_map_validate` | `project-map validate [--fix]` | Check paired artifacts for staleness and discrepancies; optionally repair. |
- For very large projects (> 1000 directories), consider running `init` on subdirectories | `project_map_reinit` | `project-map reinit [path]` | Smart regeneration: recomputes the target subtree plus ancestors, and only falls back to full regeneration when the subtree covers more than `reinitFullThresholdPercent` of project files (default 10%). |
| `project_map_context` | `project-map context <query>` | Retrieve a ranked context bundle for a natural-language query. |
## Supported Languages
| Language | AST Parsing | Heuristic Extraction |
|----------|------------|---------------------|
| TypeScript / TSX | Full | Full |
| JavaScript / JSX | Full | Full |
| Python | Partial | Full |
| Go | Partial | Full |
| Rust | Partial | Full |
| Other | - | Full (filename + regex patterns) |
-206
View File
@@ -1,206 +0,0 @@
{
"5ef2d43fdcc0141a7494c926da032af6708a60be024fdc7873cf37469b1a9a7e": {
"result": "PURPOSE: Specifies files and directories for Git to ignore in a Node.js project\nDEPS: git\nCONCEPTS: version control, ignore patterns, build artifacts, environment configuration, dependency management",
"ts": 1781101253058
},
"a093bb2fed59d699b7f9a62a5203cc196054c624e9c39c6d908c9e7db5dd2464": {
"result": "PURPOSE: Configures npm to use legacy peer dependency resolution behavior\nDEPS: npm\nCONCEPTS: package management configuration, peer dependency handling",
"ts": 1781101255838
},
"edd7b515e876afba58cd2397d4b04cf6e0309bbc2bfb81f79f7e94adb6862b30": {
"result": "PURPOSE: Describes a Pi skill that generates per-directory `.pi-map.md` files to provide machine-readable project summaries for agent comprehension.\nDEPS: npm, Node.js\nCONCEPTS: hierarchical project analysis, documentation generation, CLI tooling, agent-oriented design",
"ts": 1781101257984
},
"1c04338b0ebd9d3a241c4e7258ce7f9d2cc90be5be03788a9d7d6aff0209de47": {
"result": "PURPOSE: Generates and maintains hierarchical `.pi-map.md` analysis files per directory to enable instant codebase comprehension via AST parsing and LLM heuristics\nDEPS: npm, TypeScript/JavaScript AST parser, Python AST parser, Go AST parser, Rust AST parser, filesystem watcher\nCONCEPTS: hierarchical documentation, incremental updates, AST-based symbol extraction, LLM heuristics, dirty tracking/validation, dense markdown optimization, guard pattern, token budget management",
"ts": 1781101262689
},
"7ea3c13985c332cbdca448f7b7cd2067b4d08429b938bb61075255b8ff1a27b1": {
"result": "PURPOSE: Design a hierarchical project analysis skill for the Pi coding agent that generates and maintains compact, LLM-optimized `.pi-map.md` files per directory using hybrid LLM + AST extraction.\nDEPS: Pi Extension API, OpenAI-compatible LLM API, tree-sitter/LSP parsers, p-limit, SHA-256 hashing, Node.js/npm, TypeScript\nCONCEPTS: hierarchical project mapping, LLM-AST hybrid extraction, dense markdown format, token-efficient context, caching by content hash, incremental patching with dirty markers, validation and stale data mitigation, concurrency and rate limiting, prompt engineering",
"ts": 1781101268134
},
"b6228af707d3c250d26a87cd883ec75d6123bb561f0f1134d2ebc22b242c500e": {
"result": "PURPOSE: Implementation plan for replacing heuristic code analysis with real LLM integration in a Pi skill package that generates hierarchical `.pi-map.md` files for software projects.\n\nDEPS: openai, p-limit, Pi ExtensionContext/modelRegistry, TypeScript/Node.js\n\nCONCEPTS: hierarchical project analysis, LLM abstraction layer, dual-provider support, disk caching with LRU eviction, parallel batch processing with retries and exponential backoff, token budget management with truncation, prompt engineering, atomic file writes, factory pattern, dependency injection",
"ts": 1781101272709
},
"191f240ea0ca94939a3bac0c07ddea729dfecd34fa553c344151eee74afae826": {
"result": "PURPOSE: Defines a TypeScript-based Pi skill package for hierarchical project analysis that generates and maintains `.pi-map.md` files via CLI and OpenAI integration.\nDEPS: typescript, vitest, eslint, openai, tree-sitter, tree-sitter-python, tree-sitter-typescript, ignore, p-limit, picocolors\nCONCEPTS: CLI tooling, static code analysis, AI-powered code intelligence, hierarchical project mapping, tree-sitter parsing, concurrency limiting, Pi skill framework",
"ts": 1781101276443
},
"6736b63703d810d05e20f75829d48c3dcb26ecc34a61a72b34e67ba16c75d62a": {
"result": "PURPOSE: Registers four Pi extension tools (project_map_init, project_map_patch, project_map_validate, project_map_reinit) for managing .pi-map.md project analysis files, plus session lifecycle hooks for auto-loading maps and injecting maintenance hints.\nDEPS: @mariozechner/pi-coding-agent, typebox, fs, path, ./src/index.js, ./src/llm-client.js, ./src/llm-error.js\nCONCEPTS: Extension API registration, schema validation with TypeBox, recursive directory traversal, event hooks (session_start, before_agent_start), LLM client abstraction, error handling with custom error types",
"ts": 1781101281499
},
"fd478849f18f30f060a04006c2c99a5cf778928741fe7840bce91c1b612b475b": {
"result": "PURPOSE: Configures TypeScript compiler options for a Node.js project targeting ES2022 with strict type checking, declaration generation, and source maps.\nDEPS: TypeScript, Node.js\nCONCEPTS: ES modules, strict type checking, declaration files, source maps, JSON module resolution, project structure separation",
"ts": 1781101285063
},
"ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356": {
"result": "PURPOSE: Empty JSON configuration file with no defined settings\nDEPS: none\nCONCEPTS: JSON, configuration file, empty object",
"ts": 1781101294054
},
"4e5f16536ba51381392c91d7548737ba1e75c260b40ba6bf12cd00e91b6662bd": {
"result": "PURPOSE: Specifies files and directories for Git to ignore in the repository.\nDEPS: none\nCONCEPTS: version control, ignore patterns, build artifacts, dependencies, environment files",
"ts": 1781101300959
},
"61b1cfba0fb43448ef5c8af601bd8d948fe6f2007bf1cbd7ad3cf38de8796959": {
"result": "PURPOSE: Provides a brief overview of a small test project for pi-project-map functionality, describing its directory structure.\nDEPS: none\nCONCEPTS: documentation, project structure",
"ts": 1781101302888
},
"774a9e5bc3d0cccb73f48c399090a674b15de7e1c3847338f8ce8f379aac4202": {
"result": "PURPOSE: Defines Node.js package metadata, entry point, and build/test scripts for a TypeScript project.\nDEPS: typescript, vitest\nCONCEPTS: npm package configuration, build automation, testing setup",
"ts": 1781101305949
},
"9b6a96e8186a8ab864e7ce794d4abffa04909d942ab25e3d2c7b4fd3b3993ff9": {
"result": "PURPOSE: Configures TypeScript compiler settings for a Node.js project targeting ES2022 with strict type checking.\nDEPS: TypeScript, Node.js\nCONCEPTS: compiler configuration, module resolution, strict mode, source/output directory mapping, ES2022 target",
"ts": 1781101309464
},
"5c4644938090d20c6ec6ad41ddcd405cd4dd743f0304839fda6536779d4a3977": {
"result": "PURPOSE: Documents the API for user model operations and validation utilities\nDEPS: none\nCONCEPTS: API documentation, CRUD operations, input validation, serialization, error handling",
"ts": 1781101314989
},
"b7ce5aa53a8029b3a8e964615c083e395923731fefa98859b52f63353cb06cd2": {
"result": "PURPOSE: Creates and validates a user, then logs the result, while also re-exporting its dependencies.\nDEPS: ./models/user.js, ./utils/validation.js, ./utils/logger.js\nCONCEPTS: async/await, re-exports, validation, logging",
"ts": 1781101321083
},
"5b7780f6c6dc3950d1eea8da636328ed4357546abd5c6b4e61158bec33f38d29": {
"result": "PURPOSE: Renders a reusable button component with configurable variant, label, click handler, and disabled state.\nDEPS: React\nCONCEPTS: functional components, props interface with optional/required fields, default parameter values, template literals for dynamic class names, JSX",
"ts": 1781101327896
},
"fecf0c5d358cd2308ec354f8648e249595634fb2b9e826bc786ae5b8cfb1b5fe": {
"result": "PURPOSE: Renders a user information card with optional edit and delete action buttons.\nDEPS: React, ../models/user.js\nCONCEPTS: Functional components, Props interface, Optional callbacks, Conditional rendering, JSX",
"ts": 1781101330838
},
"1d213e971dc36a5b0c3b09164532f4f23c47cf0d2303e696c3cc4527f1ad75cd": {
"result": "PURPOSE: Defines a User type and provides factory/serialization functions for user objects with email validation.\nDEPS: ../utils/validation.js\nCONCEPTS: interface, type omission (Omit), spread operator, factory pattern, serialization, UUID generation",
"ts": 1781101336857
},
"49f96f98e46b41171f7eec2185a7b70eccc069e35fc48e595477cb40e3058fe6": {
"result": "PURPOSE: Provides a simple timestamped console logging utility with typed severity levels and convenience methods.\nDEPS: none\nCONCEPTS: union types, function overloading via wrappers, template literals, pure functions",
"ts": 1781101341327
},
"d3f67a1eaae3b0c14e53fc33ff25f70695b99cac492b011d3d141c282775d3ee": {
"result": "PURPOSE: Provides string validation utility functions for email format, non-emptiness, and minimum length checks.\nDEPS: none\nCONCEPTS: regular expressions, pure functions, utility module pattern, string validation",
"ts": 1781101343923
},
"14311d5732fba356276aa801829d764e2f05655058c78c6da57b703fa507d9ac": {
"result": "PURPOSE: Tests user model creation and serialization with validation\nDEPS: vitest, ../src/models/user.js\nCONCEPTS: unit testing, test-driven development, validation, serialization, error handling",
"ts": 1781101348436
},
"0323321ea8b99c6b2ae75230e04e5df264d8a459f0e61608985a823f7a3d9e72": {
"result": "PURPOSE: Unit tests for string validation utility functions\nDEPS: vitest, ../src/utils/validation.js\nCONCEPTS: unit testing, test suites, parameterized assertions, edge case testing",
"ts": 1781101350510
},
"ccc1278c89e099da1a812dd261013282602a1b9a178e73474e20c659d87f9ec8": {
"result": "PURPOSE: Injects a maintenance instruction reminder into every AI agent prompt to ensure .pi-map.md files stay updated\nDEPS: none\nCONCEPTS: prompt injection, string interpolation, constant exports, sidecar/hook pattern",
"ts": 1781101355928
},
"c149160b07dd2c83c597ac8db04f0f2b664d0bfe0c73494bd441ec40537cabcd": {
"result": "PURPOSE: Extracts structured AST data (exports, dependencies, classes, functions) from source code files across multiple languages using tree-sitter parsers.\nDEPS: fs, path, tree-sitter, tree-sitter-typescript, tree-sitter-python, tree-sitter-go, tree-sitter-rust (optional)\nCONCEPTS: AST parsing, tree traversal, visitor pattern, dynamic module loading, language-agnostic code analysis, recursive descent parsing",
"ts": 1781101363602
},
"9ccca325201c14ff749252083ed2586da7ab4f51276990697a1d42a588d2866c": {
"result": "PURPOSE: Implements a CLI tool for generating and managing hierarchical `.pi-map.md` project analysis files using LLM-powered directory summarization.\nDEPS: picocolors, ./init.js, ./patch.js, ./validate.js, ./discover.js, ./llm-client.js, ./config.js, ../package.json\nCONCEPTS: command pattern, argument parsing, dependency injection, async/await, error handling with custom error types, process exit codes, string formatting",
"ts": 1781101369019
},
"f6650315d5deae3897ddee8471fe0c1976e587e59767616c732f09f5139ee3b5": {
"result": "PURPOSE: Defines a configuration interface and loader for a project mapping tool that merges user-defined JSON config with sensible defaults.\nDEPS: fs, path\nCONCEPTS: interface definition, default constants, shallow merge, file-based configuration, optional chaining via try/catch fallback",
"ts": 1781101372705
},
"ac6587bef7660e8c8de7cdb4a9a9f5453899154964801bd55ae396a8fec0b95b": {
"result": "PURPOSE: Recursively discovers project files and directories while respecting .gitignore patterns and default ignore rules.\nDEPS: fs, path, ignore\nCONCEPTS: recursive directory traversal, gitignore pattern matching, file system filtering, tree walking",
"ts": 1781101375796
},
"9083911787627247be76be8bc29763c6eb76ff19e22bb07ae35adc582dc42eb9": {
"result": "PURPOSE: Implements an LLMClient interface that sends code analysis prompts to OpenAI's chat completions API with configured model and error handling.\nDEPS: openai, ./llm-error.js, ./llm-client.js\nCONCEPTS: dependency injection, interface implementation, environment-based configuration, error wrapping, async/await, default parameters",
"ts": 1781101379466
},
"2592c1c238f016a227e8653703dba9c9a44b475b454973e81da502e433ffb826": {
"result": "PURPOSE: Provides bidirectional conversion between PackageMapData objects and a custom markdown format for package documentation.\nDEPS: none\nCONCEPTS: string parsing, markdown serialization/deserialization, state machine parsing, data transformation",
"ts": 1781101382110
},
"8c3890f74246ac93b3be460ed2e93ab961054d858d6cc34c3789084a695c30f0": {
"result": "PURPOSE: Main entry point that re-exports core functions for the pi-project-map skill\nDEPS: ./init.js, ./patch.js, ./validate.js, ./format.js\nCONCEPTS: barrel exports, module re-export pattern, skill architecture",
"ts": 1781101384907
},
"00cccebfa97420a2ddf71be724790f61699fb6ba7f59beba4dca1bf0820bd1bb": {
"result": "PURPOSE: Generates `.pi-map.md` documentation files for each directory in a project by combining LLM-based and AST-based extraction of file and package metadata.\nDEPS: ./discover.js, ./format.js, ./llm-extract.js, ./ast-extract.js, ./merge.js, fs, path, ./llm-client.js\nCONCEPTS: async/await, dependency injection, data merging from multiple sources, file I/O, map generation/caching",
"ts": 1781101388067
},
"1ef82a3248c87d9bef8b7a1d0d79f8c3294ee083ba591dc2978e5594f009c52f": {
"result": "PURPOSE: Implements an LLM client for Kimi.com's Anthropic-compatible API to send prompts and return completions.\nDEPS: llm-error.js, llm-client.js\nCONCEPTS: dependency injection via options, environment variable configuration, fetch API, error wrapping, interface implementation",
"ts": 1781101390722
},
"027ef87301425b5f8383bf6fa33b8a21467ca0ef9572e354aae1d5119e221318": {
"result": "PURPOSE: Provides utilities for batch processing files with concurrency limiting, retry logic with exponential backoff, and configurable delays between batches.\nDEPS: p-limit, ./llm-error.js\nCONCEPTS: concurrency control, retry pattern with exponential backoff, batch processing, promise pooling, default options merging",
"ts": 1781101394350
},
"bdc9b833e0b658b5f929c43e700a083b441b69e01172e6246c7eb8e942d5fb3f": {
"result": "PURPOSE: Provides a file-based caching system for LLM results keyed by hash, with atomic writes and automatic directory creation.\nDEPS: fs, path, process\nCONCEPTS: persistent cache, atomic file writes (write-then-rename), defensive programming (corrupted cache recovery), JSON serialization, timestamp tracking, optional configuration parameters",
"ts": 1781101397618
},
"ce37f5999955126fbb37f59302ed4b83242b798e332850f28eaa1c481230e3b5": {
"result": "PURPOSE: Provides a factory function to create LLM client instances for different providers (Pi, Kimi, OpenAI/External) behind a common interface.\nDEPS: llm-error.js, external-llm-client.js, kimi-llm-client.js, pi-llm-client.js\nCONCEPTS: factory pattern, strategy pattern, interface abstraction, dependency inversion",
"ts": 1781101401617
},
"936ebbbb957d1dfcc601687b077c5f4ef0107cb4f54c11db5d815c903a6c224d": {
"result": "PURPOSE: Defines a custom error class for LLM-related errors with optional cause chaining.\nDEPS: none\nCONCEPTS: custom error class, error cause chaining, readonly properties, TypeScript class inheritance",
"ts": 1781101403870
},
"4af9338e28d8e3574ff4e94a79f4c597f32329faa06ffd24dd4c7cb1f6987d4d": {
"result": "PURPOSE: Extracts semantic metadata (purpose, dependencies, concepts) from source code files and packages using LLM prompts with heuristic fallbacks for multiple programming languages.\nDEPS: fs, crypto, path, ./llm-client.js, ./llm-cache.js, ./llm-error.js\nCONCEPTS: LLM prompt engineering, caching with content hashing, context window truncation, heuristic fallback pattern, regex-based parsing, multi-language support, structured output parsing",
"ts": 1781101408698
},
"64fe8dcb933ddef5bcf79a11fe1d89585743321c2ba3af55c2ee6f1502945c52": {
"result": "PURPOSE: Merges LLM-generated file metadata with AST-extracted code structure into a unified FileEntry format using a compact DSL for exports.\nDEPS: ./format.js\nCONCEPTS: data merging, DSL encoding, deduplication, nullish coalescing, Set operations",
"ts": 1781101411772
},
"1aab9d3c142e39bc334ac2510fbe2d35cff52e6650ee07402dcda81a0522f2d6": {
"result": "PURPOSE: Updates a `.pi-map.md` documentation file for a package, either by full rewrite for small packages or section-level patching for larger packages, using both LLM and AST extraction.\nDEPS: path, fs, ./format.js, ./llm-extract.js, ./ast-extract.js, ./merge.js, ./init.js, ./llm-client.js\nCONCEPTS: conditional logic based on size threshold, file I/O operations, async/await, data merging from multiple sources, in-place array updates, dirty flag pattern",
"ts": 1781101416625
},
"b889955e1d8823a9e9103b36b42daa411f2381be2e18defb970212802f4915e2": {
"result": "PURPOSE: Implements an LLM client adapter that delegates to Pi's internal AI runtime, handling model resolution, authentication, and response parsing.\nDEPS: ./llm-error.js, ./llm-client.js, @mariozechner/pi-ai\nCONCEPTS: adapter pattern, dynamic imports, optional chaining, defensive error handling, runtime environment detection",
"ts": 1781101420745
},
"de1d2f98382d9e1e9fa7cf7ed1684993d4b7fdb505206757508dd8d917ecf525": {
"result": "PURPOSE: Validates `.pi-map.md` files against actual project structure and source code exports, with optional auto-fix capability.\nDEPS: discover.js, format.js, fs, path, ast-extract.js, init.js\nCONCEPTS: AST analysis, set comparison, discrepancy detection, optional mutation/fixing, verbose logging",
"ts": 1781101424333
},
"35562a91c040cbd1e1a7180f30faeb04be96dd94278e8c549e28fb8b6251ffe5": {
"result": "PURPOSE: TypeScript declaration file for the Pi AI runtime module providing a `complete` function for LLM inference with structured chat completion API\nDEPS: none\nCONCEPTS: ambient module declaration, type-only declaration file (.d.ts), runtime-only module, LLM chat completion API, discriminated union types, optional chaining pattern",
"ts": 1781101430608
},
"54e0b7bab2574a8ae7713ff80ef3c072a7351849e6a769af07a093e49fdd36c3": {
"result": "PURPOSE: Tests the AST extraction utility for parsing TypeScript file exports, imports, and handling unsupported file types.\nDEPS: vitest, fs, path, os, ../src/ast-extract.js\nCONCEPTS: unit testing, temporary file creation, async/await, null assertion handling, test fixtures",
"ts": 1781101436720
},
"bdbe3bafc717ce91a3ebe87c39c973f644611b653a9f6e5e689e7797e3b3b0ff": {
"result": "PURPOSE: Tests markdown rendering and parsing functions for package metadata, including round-trip serialization and edge cases like empty arrays and multiline fields.\nDEPS: vitest, ../src/format.js\nCONCEPTS: unit testing, round-trip testing, snapshot-like assertions, edge case handling, type imports",
"ts": 1781101440871
},
"7a708d3abc7a83c5ba649c898af12bc3243973d0e3bc581ff48a3cbd4cb52320": {
"result": "PURPOSE: Integration tests for a project mapping tool that verifies init, patch, and validate functionality using temporary directories\nDEPS: vitest, fs, path, os, ../src/init.js, ../src/patch.js, ../src/validate.js\nCONCEPTS: integration testing, temporary filesystem fixtures, setup/teardown hooks, async/await, test-driven development",
"ts": 1781101445059
},
"511ff5fedc1e22e8979c7ad5f29e01a04267c384046e50673545ca45adc9e71b": {
"result": "PURPOSE: Unit tests for retry logic and parallel file processing utilities in an LLM batch processing module.\nDEPS: vitest, ../src/llm-batch.js, ../src/llm-error.js\nCONCEPTS: unit testing, async/await, retry pattern, concurrency control, error handling, parameterized testing",
"ts": 1781101448595
},
"ac5829c11c6079597acd38e8a39eeb84d4eb28a27c733574b687ff17c82f9307": {
"result": "PURPOSE: Tests the getCached and setCached functions from llm-cache.js using a temporary file-based cache in a vitest test suite.\nDEPS: vitest, ../src/llm-cache.js, fs, path, os\nCONCEPTS: unit testing, file system mocking/cleanup, temporary directories, beforeEach/afterEach hooks, test isolation",
"ts": 1781101452064
},
"243326dc6b993d8dfbb98ebfbd51119e8824a55c3d262fd54351e2598509942f": {
"result": "PURPOSE: Tests the llm-extract module's ability to extract file metadata (exports, dependencies, purpose, concepts) using both heuristic parsing and LLM-based analysis.\nDEPS: vitest, ../src/llm-extract.js, fs, path, os, ../src/llm-client.js\nCONCEPTS: unit testing, mocking, temporary file fixtures, parameterized testing, fallback strategies",
"ts": 1781101455713
},
"c7c525672af743b8e7cc12441011b85fe1bec325c138c21c2b4ee1415a8bf863": {
"result": "PURPOSE: Integration test suite for LLM client functionality using the Kimi API, testing client creation, file/package extraction, caching, parallel processing, and error handling.\nDEPS: vitest, fs, path, os, ../src/llm-client.js, ../src/llm-extract.js, ../src/llm-batch.js\nCONCEPTS: integration testing, environment variable management, temporary file system operations, test skipping conditionally, API client mocking/spying, parallel processing with concurrency control, caching verification, error handling validation",
"ts": 1781101460014
},
"64f5ab73ff71a9c41639f3c8a5e248387a01e27a9612f0ed2b682708557f6a27": {
"result": "PURPOSE: Unit tests for a Pi extension that registers project map tools and lifecycle event handlers.\nDEPS: vitest, fs, path, os, @mariozechner/pi-coding-agent, typebox, @mariozechner/pi-ai, ../pi-extension.js\nCONCEPTS: unit testing, mocking with vi.mock, dependency injection, temporary filesystem fixtures, tool registration testing, event handler testing",
"ts": 1781101463953
}
}
+290 -307
View File
@@ -1,352 +1,335 @@
# Design Doc: Hierarchical Project Analysis Skill for Pi # Design Reference: pi-project-map
## 1. Goals and Success Criteria > Audience: maintainers and contributors.
> Purpose: explain how `pi-project-map` works internally, not how to install or use it.
### Primary Goal ## 1. Overview
Enable a Pi coding agent to understand a software project's architecture and code relationships without scanning the entire repository. The agent should have a compact, hierarchical "internal representation" of the project that it can consume in-context.
### Success Criteria `pi-project-map` is a TypeScript/Node.js skill package that generates and maintains hierarchical, paired project-map artifacts for AI coding agents:
- The agent can orient itself in a new or familiar project without reading dozens of source files.
- The agent understands cross-package dependencies, data flows, and architectural patterns from the analysis files alone.
- Analysis files stay sufficiently fresh that the agent does not make decisions based on stale information.
- The representation is token-dense: maximum information per token, optimized for LLM consumption, not human readability.
## 2. Format Specification: Dense Markdown with Conventions - `.pi-map.index.md` — routing-first, sparse directory metadata
- `.pi-map.md` — orientation-first, richer directory metadata
### Design Rationale It runs as both:
- **Not JSON/YAML**: Brackets, quotes, and indentation add token overhead with no benefit to LLM comprehension. - a standalone CLI (`project-map`)
- **Not a custom DSL**: Fragile, requires a parser, and LLMs may hallucinate syntax. - a Pi extension (`pi-extension.ts`)
- **Dense markdown**: Hierarchical headings, bullet points, and abbreviations are natively understood by LLMs and extremely token-efficient.
### Structure The extension registers tools and event hooks that keep the artifacts fresh and can inject them into agent context at runtime.
Each directory in the project gets one analysis file named `.pi-map.md` (hidden by default, excluded from git via `.gitignore`).
```markdown ### Core design principle
# <relative-path>
## role
<one-line package role> | Dep: <comma-separated upstream deps>
## files
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
## arch
<free-form architectural notes: patterns, data flow, invariants, design decisions>
## dirty
<timestamp or flag indicating staleness>
```
### Abbreviation Conventions The artifacts are **navigation aids, not source-of-truth**. Source code is always the final authority.
| Abbreviation | Meaning |
|-------------|---------|
| `exp:` | exported symbols (functions, classes, types, constants) |
| `dep:` | dependencies (other packages, files, or external libs) |
| `pkg/` | project-internal package reference |
| `ext/` | external dependency reference |
| `->` | data flow direction |
| `|` | field delimiter within a line |
### Example > **index routes, map orients, source decides.**
```markdown ## 2. Artifact model
# pkg/auth
## role
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
## files
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
- types.ts | shared auth types | exp: AuthToken, UserClaims, Role
## arch
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
Session state stored in Redis via db/sessions. No server-side JWT storage.
## dirty
-
```
### Rules Each non-ignored directory receives a matched pair.
- One file per directory, placed inside that directory.
- Every non-excluded file in the directory gets one bullet under `## files`.
- Subdirectories are referenced in `## role` via `Dep:` or in `## arch` as structural notes, not duplicated.
- The `## dirty` section is empty (`-`) when clean, or contains a timestamp/flag when stale.
## 3. Pipeline Architecture ### 2.1 Shared model
### Hybrid Extraction: LLM + AST Both files are generated from the same in-memory `DirectoryArtifactModel`:
Two independent extraction layers contribute to the same output file. ```ts
interface DirectoryArtifactModel {
dir: string;
role: string;
files: FileEntry[];
arch: string;
dirty?: string;
isRoot: boolean;
parent?: string;
children: string[];
tags: string[];
symbols: string[];
workflows: WorkflowHint[];
}
#### Layer 1: LLM-Based Extraction (All Files) interface FileEntry {
- **Input**: Raw file contents of every non-excluded file in the directory. name: string;
- **Output**: Purpose description, architectural role, and cross-file relationships. purpose: string;
- **Applies to**: Code files, config files, Dockerfiles, READMEs, YAML, JSON, shell scripts — everything. exports: string[];
- **When it runs**: Once per file during init; again on changed files during patching. deps: string[];
- **Implementation**: Calls an actual LLM (not regex heuristics). Inside Pi, it uses Pi's built-in LLM via the ExtensionAPI. Standalone CLI falls back to an external LLM API (OpenAI-compatible).
#### Layer 2: AST-Based Extraction (Code Files Only)
- **Input**: Source code of files where a tree-sitter or LSP parser is available.
- **Output**: Precise symbol lists (functions, classes, types), signatures, import/export graphs, class hierarchies.
- **Applies to**: Supported languages only (TypeScript, Python, Go, Rust, etc.).
- **When it runs**: Once per file during init; again on changed files during patching.
#### Merging
The two layers merge into a single line per file under `## files`:
```
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
^ LLM ^ LLM ^ AST ^ AST + LLM
```
- File name and purpose: LLM.
- Exported symbols and signatures: AST (augmented by LLM if AST unavailable).
- Dependency list: AST for imports; LLM for inferred architectural dependencies.
### LLM Client Architecture
The LLM client is abstracted behind a unified interface:
```typescript
interface LLMClient {
complete(prompt: string): Promise<string>;
} }
``` ```
Two implementations: ### 2.2 `.pi-map.md` (rich map)
1. **PiLLMClient** (Pi extension): Uses `ctx.model` or `ctx.modelRegistry` to invoke Pi's configured LLM. Called from `pi-extension.ts` when the skill runs inside Pi. Rendered by `src/format.ts` `renderDirectoryMap()`.
2. **ExternalLLMClient** (standalone CLI): Calls an external OpenAI-compatible API. Configured via environment variable (e.g., `OPENAI_API_KEY`) or config file.
### Caching Contains:
- `dir:` line and sibling `index:` link
- `Project Map Protocol` (root only)
- `## role`
- `## files`
- `## arch`
- `## tags`
- `## symbols`
- `## workflows`
- `## dirty`
LLM results are cached to avoid re-querying unchanged files. ### 2.3 `.pi-map.index.md` (index)
- **Key**: SHA-256 hash of file contents. Rendered by `src/format.ts``renderDirectoryIndex()`.
- **Storage**: JSON file at `~/.cache/pi-project-map/llm-cache.json`.
- **Behavior**: Before calling the LLM, compute the file hash and check the cache. If hit, reuse the cached result. If miss, call the LLM and store the result.
- **Invalidation**: Cache entries are implicitly invalidated when the file content changes (because the hash changes). There is no TTL; the cache is append-only.
### Parallelization and Rate Limiting Contains:
- same protocol (root only)
- `## role`
- `## parent`
- `## children`
- `## files`
- `## links`
- `## workflows`
- `## dirty`
- **Concurrency**: 4-8 LLM calls in parallel, controlled by `p-limit`. ### 2.4 Why a paired format?
- **Batch delays**: A small delay (e.g., 100ms) is inserted between batches to avoid triggering rate limits.
- **Retry policy**: Each LLM call retries up to 3 times with exponential backoff (1s, 2s, 4s). If all retries fail, the entire operation stops with a hard error.
### Error Handling - **indexes are cheap** — many can be loaded without consuming much context
- **maps are dense** — loaded only after an index suggests relevance
- **paired generation** guarantees structural consistency
- **Hard error on failure**: If an LLM call fails after all retries, `init` or `patch` stops immediately and prints a clear error. There is no heuristic fallback. The user must resolve the issue (set API key, wait for rate limit, check network). ## 3. High-level architecture
- **Context limit protection**: Files larger than the LLM's context window are truncated from the end (with a note in the prompt) before being sent.
### Init Pipeline ### 3.1 Main modules
```
For each directory (depth-first): ```text
1. List all non-excluded files. discover → directory tree, .gitignore-aware
2. For each file (parallel, 4-8 concurrent): init → full generation
a. Compute SHA-256 of file contents. llm-extract → LLM-based file/package analysis
b. Check disk cache. If hit, use cached result. ast-extract → tree-sitter parsing
c. If miss: call LLM (with retries/backoff) to extract purpose and role. merge → combine LLM + AST into FileEntry
d. Store result in cache. routing-metadata → tags, symbols, workflow hints
e. If code file + parser available: run AST extraction (symbols, imports). format → render / parse the markdown pair
3. Merge per-file outputs into lines. patch → incremental update after edits
4. Run LLM on merged lines + directory context to generate: validate → consistency checking with optional repair
- `## role` (package-level summary) retrieve → deterministic query scoring
- `## arch` (architectural notes) prompt-injection → runtime context policy
5. Write `.pi-map.md` to directory. pi-extension → Pi tool/event registration
cli → standalone command dispatcher
config → defaults and .pi-project-map.json loader
``` ```
### Patch Pipeline ### 3.2 Runtime modes
```
When agent edits file(s) in directory: | Mode | Entry point | LLM client |
1. Determine patch strategy: |------|-------------|------------|
- If directory has < 10 files: full rewrite. | Pi extension | `pi-extension.ts` | `PiLLMClient` via Pi runtime |
- Else: section-level patch for changed file(s) only. | Standalone CLI | `src/cli/cli.ts` | `ExternalLLMClient` or `KimiLLMClient` |
2. For each changed file:
a. Recompute SHA-256. ## 4. Extraction pipeline
b. Check cache. If miss or stale, call LLM with retries/backoff.
3. Re-run AST extraction on changed file(s) if applicable. ### 4.1 Discovery
4. Update `## files` section (rewrite or patch).
5. Update `## dirty` flag if full regeneration is deferred. `src/discover.ts` walks the filesystem with `ignore`, merging built-in exclusions and `.gitignore`.
### 4.2 Per-directory generation
`src/init.ts``generateDirectoryArtifacts()`:
1. `processFiles()` runs in parallel over directory files
2. for each file:
- `extractFileLLM()` gets `purpose`, `deps`, `concepts`
- `extractFileAST()` gets exports/imports/calls where possible
- `mergeFileData()` combines both into a `FileEntry`
3. `extractPackageLLM()` produces directory `role` and `arch`
4. `createDirectoryModel()` builds the shared model
5. `populateRoutingMetadata()` derives `tags`, `symbols`, `workflows`
6. `writeDirectoryArtifacts()` writes both `.pi-map.md` and `.pi-map.index.md`
Directories are processed sequentially; files within a directory are processed concurrently.
### 4.3 LLM extraction
`src/llm/llm-extract.ts`:
- prompts are minimal and line-oriented
- binary files are skipped
- files over 500KB are labeled large and skipped
- source is truncated before prompting
- results are cached by SHA-256 of file content
- missing client throws `LLMError`
### 4.4 AST extraction
`src/ast/ast-extract.ts` uses `tree-sitter` for supported languages to extract:
- imports / requires
- exported classes, functions, constants
- methods, parameters, return types
- direct calls and raised exceptions
Unsupported languages fall back to LLM-only extraction.
### 4.5 Merging
`src/merge.ts`:
- purpose/concepts come from the LLM
- exports come from AST when available
- rich AST symbols are encoded as compact DSL:
- `class:Foo`
- `method:bar(a: string) → number`
- `call:baz`
- `raise:Error`
- deps are deduplicated union of AST + LLM deps
### 4.6 Routing metadata
`src/routing-metadata.ts` generates deterministic metadata used by retrieval and injection:
- **tags**
- **symbols**
- **workflow hints**
Caps are configurable via `tagCap` and `workflowHintCap`.
## 5. Patch / validate / reinit behavior
### 5.1 Patch
`src/patch.ts`:
1. resolve directory containing changed file
2. rediscover project tree
3. regenerate changed directory pair
4. refresh ancestors according to patch mode
Patch mode:
- **small** — refresh ancestor indexes only
- **structural** — refresh ancestor map/index pairs
### 5.2 Validate
`src/validate.ts` compares artifacts against filesystem and AST.
Important discrepancy types:
- `missing`
- `orphaned`
- `stale-signature`
- `dirty`
- `stale-map`
- `stale-index`
- `broken-link`
- `structural`
With `--fix`, validate builds a repair plan and regenerates directories deepest-first.
### 5.3 Reinit
`reinitPath()` is the blunt instrument for widespread staleness.
## 6. Retrieval architecture
`src/retrieve.ts` implements deterministic, index-first context retrieval.
1. walk the project for paired artifacts
2. parse indexes/maps into `DirectoryArtifactModel`
3. normalize the query
4. score every directory
5. return top-K (default: 3) as a markdown bundle with:
- relevant indexes
- relevant maps
- likely files
- relevant symbols
- instructions to verify from source
No LLM is used during retrieval. It is intentionally separate from automatic prompt injection.
## 7. Prompt injection architecture
`src/prompt-injection.ts` and `pi-extension.ts` implement runtime guidance injection.
### 7.1 Mode ladder
| Mode | Behavior |
|------|----------|
| `off` | No automatic injection |
| `advisory` | Visible startup/init hints; no artifact preload |
| `strong` (default) | Root pair preloaded, budgeted expansion, reinjection on relevant turns |
| `strict` | Same as strong, plus bypass guard for sensitive edits/architecture reasoning without protocol path |
### 7.2 Event hooks
The extension currently registers:
- `session_start`
- `before_agent_start`
- `context`
Payload fallback scanning is handled inside `context`-level decision logic; there is no separately registered `before_provider_request` hook in the current implementation.
### 7.3 Reinjection policy
`shouldReinjectForEvent()` decides whether to inject:
- only active in `strong` or `strict`
- skips if the canonical marker is already present in outgoing messages or payload
- triggers on:
- `agent_start`
- `edit_intent`
- `architecture_sensitive`
- `compaction`
- `artifact_change`
- `artifact_change` always forces reinjection
`detectEditIntent()` and `detectArchitectureSensitiveReasoning()` provide heuristic fallback for generic turns.
In addition to marker-based deduplication, `before_agent_start` scans the active session context via `ctx.sessionManager.buildSessionContext()` for an existing `pi-project-map-hint` custom message. If one is already present in the current branch, the handler skips injection entirely. This prevents duplicate visible hints in advisory/pre-init modes and duplicate hidden hints in strong/strict modes when the session context already contains the guidance. The hint is automatically re-injected after compaction or `/tree` navigation removes it from the active path.
### 7.4 Protocol path and strict bypass
The **protocol path** is present when outgoing context contains:
1. the canonical root-pair marker/block
2. the trust-boundary text
In `strict` mode, a sensitive turn without the protocol path is blocked with a visible guard. The agent can override with:
```text
[PI_MAP_BYPASS: brief justification]
``` ```
## 4. LLM Prompt Design Empty or whitespace reasons are rejected.
### File-Level Prompt ### 7.5 Budgeted expansion
The LLM prompt for a single file is designed to produce a structured, concise analysis. `buildInjectionPayload()`:
- computes budget as `min(relative, absolute)`
- default is 15% of context window, capped at 100k tokens
- always includes the root pair
- adds additional pairs shallow-first until budget is exhausted
- prepends a maintenance reminder
``` Token estimation is best-effort: `ceil(char_count / 4)`.
You are analyzing a source file for a project map. Read the file below and summarize:
1. PURPOSE: What does this file do? Describe its role in the project (2-3 sentences max). ### 7.6 Context-window discovery
2. DEPENDENCIES: What does this file depend on? List internal modules/packages and external libraries.
3. KEY CONCEPTS: Mention any important patterns, algorithms, or domain concepts.
File path: <file-path> `discoverContextWindow()` inspects the Pi runtime model for context metadata and falls back to the absolute cap when unavailable.
``` ## 8. Known limits and tradeoffs
<file-contents-truncated>
```
Respond in this exact format: ### Correctness vs cost
PURPOSE: <concise description> - init/patch/repair make LLM calls
DEPS: <comma-separated list, or "none"> - large repositories can be expensive
CONCEPTS: <comma-separated list, or "none"> - caching reduces duplicate work
```
### Package-Level Prompt ### AST coverage
- TypeScript/TSX, Python, and Go have the richest support
- other languages may be partial or LLM-only
After all file summaries are collected for a directory, a second LLM call synthesizes the package role and architecture. ### Token estimation
- 4 chars/token is only a heuristic
- oversized files may be truncated or skipped
``` ### Staleness
You are analyzing a directory in a software project. Below is a list of files in this directory with their purposes. - there is no filesystem watcher
- maps go stale when edits happen outside the patch flow
- validate detects but does not prevent staleness
Directory: <dir-path> ### Patch mode inference
Files: - auto-mode heuristics are good but imperfect
- <file1>: <purpose1> - contributors can force structural mode when needed
- <file2>: <purpose2>
...
Respond in this exact format: ### Strict mode ergonomics
ROLE: <one-line description of this directory's role in the project> - strict guards can be surprising on casual phrasing
ARCH: <2-4 sentences describing architecture, data flow, patterns, and design decisions> - bypass markers are intentionally explicit and user-visible
```
### Output Parsing ### Retrieval scoring
- deterministic scoring is reproducible but not semantic-search-smart
- broader queries may still need manual browsing
The LLM client's response is parsed to extract `PURPOSE`, `DEPS`, `CONCEPTS`, `ROLE`, and `ARCH` fields. These are merged with AST data into the final `.pi-map.md` format. ### Cache
- cache grows unless manually cleaned
### Context Limit Protection - corrupted cache files are recovered by starting fresh
- Files are truncated from the end if they exceed a configurable max token budget (default: 4000 tokens of source).
- A marker `[...truncated]` is appended to the truncated content so the LLM knows it is not seeing the full file.
- Very large binary or generated files are skipped entirely for LLM analysis (they still appear in `.pi-map.md` with a note like "Large/generated file").
## 5. Consumption Model
### Session Start
1. Agent discovers all `.pi-map.md` files (e.g., via `find . -name ".pi-map.md"`).
2. Agent reads **all** files into context. This is a one-time cost at session start.
3. Agent constructs an internal mental model of the project hierarchy.
### During Session
- An **auto-injected summary** stays in context (e.g., a condensed top-level `.pi-map.md` or a synthesized project overview).
- When the agent needs deeper detail about a specific package, it already has the full `.pi-map.md` in memory from step 2.
- If the agent enters a new package not yet loaded, it reads that package's `.pi-map.md` on demand.
### Context Management
- For very large projects, the agent may summarize or prune the initial read, keeping only the top N levels of the hierarchy in active context.
- The skill can provide a "context budget" parameter: max tokens to spend on analysis files.
## 6. Stale Data Mitigation
### Combined Strategy
#### 5.1 Dirty Markers
- Whenever the agent edits a file, it appends a dirty flag to the directory's `.pi-map.md`:
```markdown
## dirty
2024-06-09T14:32:00Z: tokens.ts modified
```
- A background or post-session reconciliation step regenerates dirty files.
- The agent can also be instructed to reconcile before making architectural decisions.
#### 5.2 Periodic Full Re-init
- On every new session start, or on a configurable schedule (e.g., daily), the skill offers to run a full re-scan.
- This catches any changes made outside the agent's awareness (e.g., by other developers).
#### 5.3 Validation Command
- A `validate` tool/command that the agent can invoke:
- Checks for missing files (new files not in `.pi-map.md`).
- Checks for orphaned entries (files listed but deleted).
- Checks for changed signatures (AST mismatch between listed symbols and actual code).
- Reports discrepancies and suggests corrections.
### Recovery
- If validation finds staleness beyond a threshold (e.g., > 3 dirty packages), the skill recommends a full re-init.
- The agent can also trigger re-init for a specific subtree.
## 7. Scope Boundaries and Non-Goals
### In Scope
- Every directory in the project gets a `.pi-map.md` file.
- Every non-excluded file gets analyzed by the LLM layer.
- Code files get augmented by the AST layer where parsers exist.
- Respect `.gitignore` and known junk patterns (node_modules, .git, dist, build, coverage, .next, .venv, __pycache__, .DS_Store).
### Out of Scope (Non-Goals)
- **Human-readable documentation**: These files are machine-only. Human docs live elsewhere.
- **Line-by-line code explanation**: The format captures symbols and architecture, not implementation details.
- **Auto-regeneration on filesystem events**: The skill relies on agent-initiated updates and periodic re-init, not filesystem watchers.
- **Cross-project analysis**: Each project is independent. No global index across repos.
- **IDE integration**: This is a Pi agent skill, not a VS Code extension or LSP server.
## 8. Pi Skill Package Structure
```
pi-project-map/
├── SKILL.md # Skill definition for Pi
├── package.json # npm package metadata
├── src/
│ ├── init.ts # Full project scan + generation
│ ├── patch.ts # Incremental patch logic
│ ├── validate.ts # Consistency checker
│ ├── ast-extract.ts # Tree-sitter / LSP wrappers
│ ├── llm-extract.ts # LLM prompt templates for extraction
│ ├── merge.ts # Merge AST + LLM outputs
│ ├── format.ts # Dense markdown formatter
│ └── config.ts # Skill configuration (thresholds, ignore patterns)
├── hooks/
│ └── on-prompt.ts # Injects maintenance command into prompts
└── README.md # Setup and usage for humans
```
### Custom Tools
- `project-map:init` — Run full project scan. Creates all `.pi-map.md` files.
- `project-map:patch <file-path>` — Update analysis for a specific file/directory.
- `project-map:validate` — Run consistency check across all `.pi-map.md` files.
- `project-map:reinit [path]` — Force re-initialization of entire project or subtree.
### Prompt Hook
- On every prompt, the skill appends a lightweight instruction:
> "If you modify any source file, run `project-map:patch <path>` to update the analysis. If you suspect staleness, run `project-map:validate`."
## 9. Risks and Tradeoffs
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Token bloat (1000+ dirs) | Medium | High | Summary mode, lazy loading, context budget |
| Stale analysis files | High | High | Dirty markers + periodic re-init + validation |
| Agent trusts stale data | Medium | High | Clear instructions to validate before architectural decisions |
| Expensive init on large repos | Medium | Medium | Parallelization, caching, optional incremental init |
| Overlap with LSP/typedoc | Low | Low | This is agent-context, not IDE tooling. Different use case. |
| AST parser unavailable | Medium | Low | Graceful fallback to LLM-only extraction |
## 10. Concrete Example: Full Project Snapshot
```
project-root/
├── .pi-map.md
├── src/
│ ├── .pi-map.md
│ ├── auth/
│ │ ├── .pi-map.md
│ │ ├── tokens.ts
│ │ ├── middleware.ts
│ │ └── types.ts
│ └── db/
│ ├── .pi-map.md
│ ├── connection.ts
│ └── migrations/
│ ├── .pi-map.md
│ └── 001_init.sql
├── docker/
│ ├── .pi-map.md
│ ├── Dockerfile
│ └── docker-compose.yml
└── README.md
```
Each `.pi-map.md` follows the format in Section 2, creating a navigable hierarchy.
## 11. Future Extensions
- **Cross-reference graph**: A top-level `project-graph.md` linking all packages with dependency arrows.
- **Search index**: A lightweight FTS5 index over all `.pi-map.md` files for fast symbol lookup.
- **Diff-aware patching**: Only re-run LLM on changed functions, not entire files.
- **Multi-repo workspaces**: Support monorepos with independent package boundaries.
+20
View File
@@ -0,0 +1,20 @@
# fixtures (index)
dir: fixtures
## role
Provides test data and setup utilities for automated testing across the project.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
- fixtures/sample-project
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## files
## links
index: fixtures/.pi-map.index.md
map: fixtures/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# fixtures
dir: fixtures
index: fixtures/.pi-map.index.md
## role
Provides test data and setup utilities for automated testing across the project.
## files
## arch
Simple static data fixtures with possible factory/helper patterns for consistent test state generation.
## tags
-
## symbols
-
## workflows
-
## dirty
-
-1
View File
@@ -1,4 +1,3 @@
dist/ dist/
node_modules/ node_modules/
.pi-map.md
.env .env
+30
View File
@@ -0,0 +1,30 @@
# fixtures/sample-project (index)
dir: fixtures/sample-project
## role
Provides a minimal sample Node.js/TypeScript project fixture for testing and demonstrating the pi-project-map functionality.
## parent
index: fixtures/.pi-map.index.md
map: fixtures/.pi-map.md
## children
- fixtures/sample-project/docs
index: fixtures/sample-project/docs/.pi-map.index.md
map: fixtures/sample-project/docs/.pi-map.md
- fixtures/sample-project/src
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
- fixtures/sample-project/tests
index: fixtures/sample-project/tests/.pi-map.index.md
map: fixtures/sample-project/tests/.pi-map.md
## files
- .gitignore
- README.md
- package.json
- tsconfig.json
## links
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## workflows
-
## dirty
-
+22
View File
@@ -0,0 +1,22 @@
# fixtures/sample-project
dir: fixtures/sample-project
index: fixtures/sample-project/.pi-map.index.md
## role
Provides a minimal sample Node.js/TypeScript project fixture for testing and demonstrating the pi-project-map functionality.
## files
- .gitignore | Specifies files and directories for Git to ignore in version control | dep: git
- README.md | Provides a brief overview and directory structure for a small test project related to pi-project-map functionality.
- package.json | Defines a sample Node.js project configuration with TypeScript build and Vitest testing scripts. | dep: typescript, vitest
- tsconfig.json | Configures TypeScript compiler options for a Node.js project targeting ES2022 with strict type checking.
## arch
Standard Node.js project structure using TypeScript with strict compilation, Vitest for testing, and ES2022 module output.
## tags
project, typescript, git, readme, node, vitest, package, tsconfig
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,19 @@
# fixtures/sample-project/docs (index)
dir: fixtures/sample-project/docs
## role
Provides API documentation and usage examples for the sample project's user management functionality.
## parent
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## children
-
## files
- API.md
## links
index: fixtures/sample-project/docs/.pi-map.index.md
map: fixtures/sample-project/docs/.pi-map.md
## workflows
-
## dirty
-
+19
View File
@@ -0,0 +1,19 @@
# fixtures/sample-project/docs
dir: fixtures/sample-project/docs
index: fixtures/sample-project/docs/.pi-map.index.md
## role
Provides API documentation and usage examples for the sample project's user management functionality.
## files
- API.md | Documents a user management API with user creation/serialization functions and validation utilities
## arch
Documentation-as-code pattern with markdown-based reference material for external API consumers.
## tags
api, user, documents, management, creation, serialization, validation, utilities
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,30 @@
# fixtures/sample-project/src (index)
dir: fixtures/sample-project/src
## role
Entry point module that demonstrates user creation, email validation, and logging for a sample project.
## parent
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## children
- fixtures/sample-project/src/components
index: fixtures/sample-project/src/components/.pi-map.index.md
map: fixtures/sample-project/src/components/.pi-map.md
- fixtures/sample-project/src/models
index: fixtures/sample-project/src/models/.pi-map.index.md
map: fixtures/sample-project/src/models/.pi-map.md
- fixtures/sample-project/src/utils
index: fixtures/sample-project/src/utils/.pi-map.index.md
map: fixtures/sample-project/src/utils/.pi-map.md
## files
- index.ts
## links
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## workflows
- change src behavior
read: index.ts
- explore src subdirectories
index: fixtures/sample-project/src/components/.pi-map.index.md, fixtures/sample-project/src/models/.pi-map.index.md, fixtures/sample-project/src/utils/.pi-map.index.md
## dirty
-
+26
View File
@@ -0,0 +1,26 @@
# fixtures/sample-project/src
dir: fixtures/sample-project/src
index: fixtures/sample-project/src/.pi-map.index.md
## role
Entry point module that demonstrates user creation, email validation, and logging for a sample project.
## files
- index.ts | Entry point that creates a user, validates their email, and logs the result | exp: func:main(), call:createUser, call:validateEmail, call:logger.error, call:logger.info | dep: ./models/user.js, ./utils/validation.js, ./utils/logger.js
## arch
Simple procedural script with direct function calls and sequential execution pattern.
## tags
user, email, js, main, call:create, call:validate, call:logger.error, call:logger.info
## symbols
- main
- call:createUser
- call:validateEmail
- call:logger.error
- call:logger.info
## workflows
- change src behavior
read: index.ts
- explore src subdirectories
index: fixtures/sample-project/src/components/.pi-map.index.md, fixtures/sample-project/src/models/.pi-map.index.md, fixtures/sample-project/src/utils/.pi-map.index.md
## dirty
-
@@ -0,0 +1,21 @@
# fixtures/sample-project/src/components (index)
dir: fixtures/sample-project/src/components
## role
Provides reusable UI components for building the application's interface.
## parent
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## children
-
## files
- Button.tsx
- UserCard.tsx
## links
index: fixtures/sample-project/src/components/.pi-map.index.md
map: fixtures/sample-project/src/components/.pi-map.md
## workflows
- change components behavior
read: Button.tsx, UserCard.tsx
## dirty
-
@@ -0,0 +1,26 @@
# fixtures/sample-project/src/components
dir: fixtures/sample-project/src/components
index: fixtures/sample-project/src/components/.pi-map.index.md
## role
Provides reusable UI components for building the application's interface.
## files
- Button.tsx | A reusable React button component that renders a styled button with configurable label, visual variant, click handler, and disabled state. | exp: ButtonProps, func:Button({ label, variant = "primary", onClick, disabled = false, }: ButtonProps) → JSX.Element | dep: react, React
- UserCard.tsx | Renders a user card component with optional edit and delete action buttons. | exp: UserCardProps, func:UserCard({ user, onEdit, onDelete }: UserCardProps) → JSX.Element, call:onEdit, call:onDelete | dep: react, ../models/user.js, React
## arch
Simple functional React components with props-based configuration, following a basic presentational component pattern.
## tags
button, user, card, react, props, call:on, edit, delete
## symbols
- Button
- UserCard
- ButtonProps
- UserCardProps
- call:onEdit
- call:onDelete
## workflows
- change components behavior
read: Button.tsx, UserCard.tsx
## dirty
-
@@ -0,0 +1,20 @@
# fixtures/sample-project/src/models (index)
dir: fixtures/sample-project/src/models
## role
Defines the User data model with creation and serialization capabilities including email validation.
## parent
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## children
-
## files
- user.ts
## links
index: fixtures/sample-project/src/models/.pi-map.index.md
map: fixtures/sample-project/src/models/.pi-map.md
## workflows
- change models behavior
read: user.ts
## dirty
-
@@ -0,0 +1,26 @@
# fixtures/sample-project/src/models
dir: fixtures/sample-project/src/models
index: fixtures/sample-project/src/models/.pi-map.index.md
## role
Defines the User data model with creation and serialization capabilities including email validation.
## files
- user.ts | Defines a User interface and provides functions to create and serialize users with email validation. | exp: User, func:createUser(data: Omit<User, "id" | "createdAt">) → User, call:validateEmail, call:crypto.randomUUID, raise:Error, func:serializeUser(user: User) → string, call:JSON.stringify | dep: ../utils/validation.js
## arch
Domain model pattern with interface-based typing, pure functions for data transformation, and embedded validation logic.
## tags
user, create, serialize, email, validation, call:validate, call:crypto.random, uuid
## symbols
- createUser
- serializeUser
- User
- call:validateEmail
- call:crypto.randomUUID
- raise:Error
- call:JSON.stringify
## workflows
- change models behavior
read: user.ts
## dirty
-
@@ -0,0 +1,21 @@
# fixtures/sample-project/src/utils (index)
dir: fixtures/sample-project/src/utils
## role
Provides foundational cross-cutting utility functions for logging and input validation used throughout the application.
## parent
index: fixtures/sample-project/src/.pi-map.index.md
map: fixtures/sample-project/src/.pi-map.md
## children
-
## files
- logger.ts
- validation.ts
## links
index: fixtures/sample-project/src/utils/.pi-map.index.md
map: fixtures/sample-project/src/utils/.pi-map.md
## workflows
- change utils behavior
read: logger.ts, validation.ts
## dirty
-
@@ -0,0 +1,28 @@
# fixtures/sample-project/src/utils
dir: fixtures/sample-project/src/utils
index: fixtures/sample-project/src/utils/.pi-map.index.md
## role
Provides foundational cross-cutting utility functions for logging and input validation used throughout the application.
## files
- logger.ts | Provides a simple typed console logger with timestamps and convenience wrappers for different log levels. | exp: LogLevel, func:log(level: LogLevel, message: string) → void, call:new Date().toISOString, call:console.log, call:level.toUpperCase, func:debug(message: string) → void, call:log, func:info(message: string) → void, call:log, func:warn(message: string) → void, call:log, func:error(message: string) → void, call:log
- validation.ts | Provides basic string validation utilities for emails, non-empty checks, and minimum length requirements. | exp: func:validateEmail(email: string) → boolean, call:EMAIL_REGEX.test, func:validateNotEmpty(value: string) → boolean, call:value.trim, func:validateMinLength(value: string, min: number) → boolean
## arch
Flat utility module pattern with pure functions, no dependencies between modules, each exporting independent typed helper functions.
## tags
call:log, validate, log, logger, validation, empty, length, provides
## symbols
- log
- debug
- info
- warn
- error
- validateEmail
- validateNotEmpty
- validateMinLength
## workflows
- change utils behavior
read: logger.ts, validation.ts
## dirty
-
@@ -0,0 +1,21 @@
# fixtures/sample-project/tests (index)
dir: fixtures/sample-project/tests
## role
Provides unit test coverage for user model behavior and string validation utilities in the sample project.
## parent
index: fixtures/sample-project/.pi-map.index.md
map: fixtures/sample-project/.pi-map.md
## children
-
## files
- user.test.ts
- validation.test.ts
## links
index: fixtures/sample-project/tests/.pi-map.index.md
map: fixtures/sample-project/tests/.pi-map.md
## workflows
- update tests tests
read: user.test.ts, validation.test.ts
## dirty
-
+21
View File
@@ -0,0 +1,21 @@
# fixtures/sample-project/tests
dir: fixtures/sample-project/tests
index: fixtures/sample-project/tests/.pi-map.index.md
## role
Provides unit test coverage for user model behavior and string validation utilities in the sample project.
## files
- user.test.ts | Unit tests for user model creation, validation, and serialization | dep: vitest, ../src/models/user.js
- validation.test.ts | Unit tests for string validation utility functions | dep: vitest, ../src/utils/validation.js
## arch
Standard test suite using isolated unit tests with file-based grouping by domain concern (model vs. utility).
## tags
validation, unit, tests, user, vitest, src, js, user.test
## symbols
-
## workflows
- update tests tests
read: user.test.ts, validation.test.ts
## dirty
-
-11
View File
@@ -1,11 +0,0 @@
// Pi skill prompt hook
// Injected into every prompt to remind the agent to maintain .pi-map.md files
export const MAINTENANCE_INSTRUCTION = `
If you modify any source file, run \`project-map:patch <file-path>\` to update the analysis.
If you suspect staleness, run \`project-map:validate\`.
`;
export function injectPrompt(originalPrompt: string): string {
return `${originalPrompt}\n\n---\n${MAINTENANCE_INSTRUCTION}`;
}
-241
View File
@@ -1,241 +0,0 @@
# Implementation Plan: Hierarchical Project Analysis Skill for Pi
## Overview
Build a Pi skill package (`pi-project-map`) that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file. The skill provides custom tools for init, patch, validate, and re-init, plus a prompt hook that ensures the agent keeps analysis files in sync.
---
## Current Status
- **M1: Foundation and Format** — ✅ Complete
- **M2: Heuristic Extraction** — ✅ Complete (placeholder only; to be replaced by real LLM)
- **M3: AST Extraction** — ✅ Complete
- **M4: Patch and Update** — ✅ Complete
- **M5: Validation and `--fix`** — ✅ Complete
- **M6: Pi Skill Integration** — ✅ Complete (basic)
- **M7: Proper LLM Integration** — 🔄 In Progress / Next up
The remaining major work is to replace the heuristic "LLM" extraction with **actual LLM API calls**.
---
## Remaining Milestone: M7 — Proper LLM Integration
**Goal**: Replace the regex heuristic `llm-extract.ts` with real LLM calls, including dual-provider support (Pi native + external fallback), disk caching, parallelization, and retries.
**Estimated time**: 1 week of focused work.
---
### Task 1: LLM Client Abstraction (`src/llm-client.ts`)
Create a unified interface for LLM calls.
```typescript
interface LLMClient {
complete(prompt: string): Promise<string>;
}
```
Sub-tasks:
- Define the `LLMClient` interface.
- Add a factory function `createLLMClient(mode: 'pi' | 'external', options)`.
- Handle mode selection:
- `'pi'`: used inside `pi-extension.ts` with Pi's model registry.
- `'external'`: used in standalone CLI with OpenAI-compatible API.
**Acceptance Criteria**
- `LLMClient` interface exists and compiles.
- Factory correctly selects implementation based on mode.
---
### Task 2: External LLM Client (`src/external-llm-client.ts`)
Implement the standalone CLI LLM client using an OpenAI-compatible API.
Sub-tasks:
- Add `openai` (or a lightweight fetch-based client) as a dependency.
- Read configuration from:
- Environment variable `OPENAI_API_KEY` (or `ANTHROPIC_API_KEY`, etc.)
- Config file `.pi-project-map.json` fields: `llmProvider`, `llmModel`, `llmBaseUrl`
- Implement `complete(prompt)` using chat completions API.
- Default to `gpt-4o-mini` or similar cheap model.
**Acceptance Criteria**
- `project-map init` works standalone with `OPENAI_API_KEY` set.
- Missing API key produces a clear, actionable error message.
- Failed API call throws a descriptive `LLMError`.
---
### Task 3: Pi LLM Client (`src/pi-llm-client.ts`)
Implement the Pi-native LLM client for use inside the extension.
Sub-tasks:
- Accept Pi's `ExtensionContext` or model registry as a constructor argument.
- Use `ctx.model` / `ctx.modelRegistry` to get the configured model and API key.
- Call the provider directly (likely using the same OpenAI-compatible endpoint Pi uses).
- If Pi does not expose direct LLM calls, fall back to emitting a tool call / follow-up message pattern.
**Acceptance Criteria**
- Pi extension can successfully call an LLM when running inside Pi.
- Errors surface clearly to the user.
---
### Task 4: Disk Cache (`src/llm-cache.ts`)
Implement persistent SHA-256 → LLM result caching.
Sub-tasks:
- Cache directory: `~/.cache/pi-project-map/` (create if missing).
- Cache file: `llm-cache.json` (simple JSON object).
- Functions:
- `getCached(hash: string): string | null`
- `setCached(hash: string, result: string): void`
- Ensure atomic writes (write to temp file, rename).
- Add cache size limit (e.g., 10,000 entries, LRU eviction).
**Acceptance Criteria**
- Second `init` run on unchanged files does not call LLM.
- Cache persists across process restarts.
- Corrupted cache file does not crash the tool.
---
### Task 5: Parallelization and Retries (`src/llm-batch.ts`)
Run LLM requests in parallel with retries and backoff.
Sub-tasks:
- Add `p-limit` dependency for concurrency control.
- Default concurrency: 4 (configurable via `.pi-project-map.json` `llmConcurrency`).
- Add small delay (100ms) between batches.
- Implement retry logic: 3 retries, delays 1s → 2s → 4s.
- On final failure, throw a hard error and stop the entire process.
**Acceptance Criteria**
- 100-file project completes significantly faster than sequential.
- Simulated transient failures are retried and recovered.
- Persistent failure stops the tool with a clear error.
---
### Task 6: Rewrite `llm-extract.ts` to Use Real LLM
Replace regex heuristics with LLM calls.
Sub-tasks:
- Accept an `LLMClient` in `extractFileLLM` and `extractPackageLLM`.
- Check disk cache before calling LLM.
- Construct file-level prompt (see design doc Section 4).
- Parse response into `purpose`, `deps`, `concepts`.
- Construct package-level prompt.
- Parse response into `role`, `arch`.
- Remove heuristic code from production path; keep only for test mocks if useful.
**Acceptance Criteria**
- `llm-extract.ts` calls the LLM client for every file.
- Output includes rich, non-trivial descriptions for typical files.
- Unit tests mock the LLM client to verify prompt structure and parsing.
---
### Task 7: Context Limit Handling
Protect against oversized files.
Sub-tasks:
- Measure prompt + file content tokens (approximate: 1 token ≈ 4 chars for ASCII).
- If file exceeds max context budget (configurable, default 4000 tokens), truncate from the end.
- Append `[...truncated]` marker in the prompt.
- Skip LLM for binary/generated files over a hard limit (e.g., 50KB) and mark them as "Large/generated file".
**Acceptance Criteria**
- A 1MB minified JS file does not crash or consume excessive tokens.
- Truncated files still produce useful output.
---
### Task 8: Update CLI and Extension
Wire the new LLM client into all entry points.
Sub-tasks:
- `src/cli.ts`: create external LLM client, pass into `initProject` / `patchFile`.
- `pi-extension.ts`: create Pi LLM client, pass into tools.
- Update `init.ts` and `patch.ts` signatures to accept an optional `LLMClient`.
- Add CLI flag `--llm-provider=openai` for explicit selection.
- Update error handling to catch `LLMError` and print helpful messages.
**Acceptance Criteria**
- CLI works with external API key.
- Extension works inside Pi (if Pi exposes LLM access).
- Clear errors on misconfiguration.
---
### Task 9: Update Tests
Sub-tasks:
- Replace heuristic tests with mocked LLM client tests.
- Add integration test: create a fake LLM client, run `initProject`, verify output contains LLM-provided text.
- Add cache test: verify cache hit skips LLM call.
- Add retry test: verify transient failures retry, persistent failures hard-stop.
**Acceptance Criteria**
- All tests pass.
- Test coverage includes: LLM client, cache, batching, prompt parsing, error handling.
---
## Sequencing and Dependencies
```
Task 1 (LLMClient interface)
├── Task 2 (External client)
├── Task 3 (Pi client)
├── Task 4 (Cache)
├── Task 5 (Batch + retries)
├── Task 6 (Rewrite llm-extract.ts)
├── Task 7 (Context limits)
├── Task 8 (Wire CLI + extension)
└── Task 9 (Tests)
```
---
## Validation Criteria (for LLM Integration)
1. **Real LLM calls**: `llm-extract.ts` invokes the configured LLM client for every file.
2. **Cache hit**: Second `init` on unchanged repo completes with zero LLM calls.
3. **Parallel speed**: 100-file project init completes in under 30 seconds (assuming average LLM latency 500ms).
4. **Retry works**: Transient 429/5xx errors are retried; permanent failures stop with a clear error.
5. **Context limit safety**: Files > max token budget are truncated, not rejected.
6. **Dual provider**: CLI uses external API; Pi extension uses Pi's LLM.
7. **Quality**: LLM output is visibly richer than the old heuristic output (verified by manual inspection).
---
## Rollout Plan
1. **Test on real projects** (Day 1-2): Run `init` on 2-3 real codebases with the new LLM integration.
2. **Cost audit** (Day 3): Measure token usage per project; adjust defaults if too expensive.
3. **Prompt tuning** (Day 4-5): Iterate prompt design based on output quality.
4. **Release** (Day 6-7): Publish updated npm package, update Pi extension docs.
---
## Completed Milestones (for reference)
- **M1**: Foundation and Format
- **M2**: Heuristic Extraction (placeholder, to be replaced by M7)
- **M3**: AST Extraction
- **M4**: Patch and Update
- **M5**: Validation and `--fix`
- **M6**: Pi Skill Integration
+22
View File
@@ -0,0 +1,22 @@
# openspec (index)
dir: openspec
## role
Defines configuration and project conventions for a TypeScript CLI tool that generates AI-oriented codebase orientation maps for coding agents.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
- openspec/changes
index: openspec/changes/.pi-map.index.md
map: openspec/changes/.pi-map.md
## files
- config.yaml
- project.md
## links
index: openspec/.pi-map.index.md
map: openspec/.pi-map.md
## workflows
-
## dirty
-
+20
View File
@@ -0,0 +1,20 @@
# openspec
dir: openspec
index: openspec/.pi-map.index.md
## role
Defines configuration and project conventions for a TypeScript CLI tool that generates AI-oriented codebase orientation maps for coding agents.
## files
- config.yaml | Defines project configuration, stack metadata, and software-driven development (SDD) workflow rules for a TypeScript-based CLI tool that generates AI-oriented codebase map files. | dep: TypeScript, Node.js, Vitest, npm, openspec
- project.md | Defines project context and conventions for pi-project-map, a Pi skill and CLI tool that generates hierarchical `.pi-map.md` orientation files for coding agents.
## arch
YAML-driven configuration with software-driven development (SDD) workflow rules, hierarchical markdown output generation, and Pi skill integration for AI agent context provision.
## tags
project, map, defines, typescript, cli, tool, generates, config
## symbols
-
## workflows
-
## dirty
-
+20
View File
@@ -0,0 +1,20 @@
# openspec/changes (index)
dir: openspec/changes
## role
Manages change tracking, versioning, and audit history for OpenAPI specification modifications
## parent
index: openspec/.pi-map.index.md
map: openspec/.pi-map.md
## children
- openspec/changes/archive
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## files
## links
index: openspec/changes/.pi-map.index.md
map: openspec/changes/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# openspec/changes
dir: openspec/changes
index: openspec/changes/.pi-map.index.md
## role
Manages change tracking, versioning, and audit history for OpenAPI specification modifications
## files
## arch
Event-sourced or changelog-based pattern with immutable change records and versioned snapshots
## tags
-
## symbols
-
## workflows
-
## dirty
-
+26
View File
@@ -0,0 +1,26 @@
# openspec/changes/archive (index)
dir: openspec/changes/archive
## role
Provides persistent storage and retrieval of historical change records in an archived format for audit trails and long-term data retention.
## parent
index: openspec/changes/.pi-map.index.md
map: openspec/changes/.pi-map.md
## children
- openspec/changes/archive/2026-06-11-layered-map-protocol
index: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.md
- openspec/changes/archive/2026-06-11-map-context-retrieval
index: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.md
- openspec/changes/archive/2026-06-11-project-map-prompt-injection
index: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.md
## files
## links
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## workflows
-
## dirty
-
+18
View File
@@ -0,0 +1,18 @@
# openspec/changes/archive
dir: openspec/changes/archive
index: openspec/changes/archive/.pi-map.index.md
## role
Provides persistent storage and retrieval of historical change records in an archived format for audit trails and long-term data retention.
## files
## arch
Simple archive storage pattern using file-based serialization with read/write operations for immutable change log records, likely with date-based or sequential naming conventions.
## tags
-
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-layered-map-protocol (index)
dir: openspec/changes/archive/2026-06-11-layered-map-protocol
## role
Contains archived specification documents for a deprecated layered map protocol that introduced paired navigation artifacts to replace bulk-loading of map files with a tiered directory-level routing system.
## parent
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## children
-
## files
- apply-progress.md
- archive-report.md
- design.md
- proposal.md
- spec.md
- sync-report.md
- tasks.md
- verify-report.md
## links
index: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.md
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-layered-map-protocol
dir: openspec/changes/archive/2026-06-11-layered-map-protocol
index: openspec/changes/archive/2026-06-11-layered-map-protocol/.pi-map.index.md
## role
Contains archived specification documents for a deprecated layered map protocol that introduced paired navigation artifacts to replace bulk-loading of map files with a tiered directory-level routing system.
## files
- apply-progress.md | Documents the completion status and summary of implemented features for an "Apply Progress" project or milestone.
- archive-report.md | Documents the archival status and metadata for a deprecated layered map protocol specification directory.
- design.md | Design document for a paired navigation-first artifact model that generates both routing indexes and orientation rich-maps from a shared intermediate directory model while preserving existing pipeline behavior. | dep: spec.md, src/format.ts, src/init.ts, src/patch.ts, src/validate.ts, pi-extension.ts, README.md, SKILL.md, design-doc.md, CLI argument parsing, YAML config handling
- proposal.md | Proposes a layered navigation protocol using paired index/map artifacts to replace bulk-loading of map files with a tiered, directory-level routing system.
- spec.md | Specifies a layered navigation protocol for project maps using paired index and rich map artifacts per directory with defined generation, patching, and validation behaviors.
- sync-report.md | Documents that a canonical spec synchronization was not performed due to legacy flat change artifact structure, with user-approved archival fallback
- tasks.md | Defines a phased task plan for implementing a layered paired-map protocol with directory-level `.pi-map.md` and `.pi-map.index.md` artifacts, routing metadata, patch sizing, validation, and documentation. | dep: design.md, pi-extension.ts, SKILL.md, README.md, design-doc.md, npm/node toolchain
- verify-report.md | Documents verification results for a code change implementing a layered map protocol with paired artifacts. | dep: npm, vitest, node, typescript
## arch
Document-driven specification archive using a layered architecture with paired index/map artifacts per directory, phased implementation tasks, and formal verification/synchronization reporting, preserved for historical reference despite legacy flat structure preventing canonical sync.
## tags
map, md, report, design, ts, layered, protocol, directory
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,10 @@
# Apply Progress
Status: complete
Summary: Implemented paired map/index artifacts, layered routing/orientation protocol, pair-aware patch/validate/reinit flows, config caps, and docs/runtime guidance.
Implemented commits:
- c6064f8 Implement layered maps and context retrieval
Stale-checkbox reconciliation: historical implementation completed in committed work above; tasks.md reconciled to checked state on 2026-06-11.
@@ -0,0 +1,9 @@
# Archive Report
Status: archived
Archived path: openspec/changes/archive/2026-06-11-layered-map-protocol
Archive mode: manual archive fallback in openspec-only repo with legacy flat change specs; canonical sync marked not-applicable in sync-report.md.
Inputs preserved in archive: proposal.md, spec.md, design.md, tasks.md, apply-progress.md, verify-report.md, sync-report.md.
@@ -0,0 +1,200 @@
# Design: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Design** |
| Based on | [Spec](spec.md) |
| Next | Tasks |
## Design summary
This change introduces a paired, navigation-first artifact model without changing the core mission of `pi-project-map`. The implementation should keep the existing discover → analyze → merge → render pipeline, but route both outputs through a shared intermediate directory model:
- `.pi-map.index.md` = routing-first view
- `.pi-map.md` = orientation-first view
Source remains the final authority.
## Affected areas
### Source files likely to change
- `src/format.ts`
- `src/init.ts`
- `src/patch.ts`
- `src/validate.ts`
- `pi-extension.ts`
- `README.md`
- `SKILL.md`
- `design-doc.md`
- CLI argument parsing for `validate --fix` if not already supported
### New modules likely to appear
- `src/root-index.ts` or equivalent shared index generation helper
- `src/directory-model.ts` or equivalent shared intermediate model helper
- optional config support for workflow/tag caps if not already present in config handling
## Architecture changes
### 1. Shared intermediate model
Build one structured directory model per mapped directory, then render two views from it.
The model should carry at least:
- directory identity (`dir`)
- sibling/parent/child relationships
- likely files
- role/arch summaries
- tags
- prioritized symbols
- workflow candidates in normalized schema
- stale/freshness markers
This shared model is the main guard against map/index drift.
### 2. Universal paired artifacts
During init/reinit, generate both `.pi-map.md` and `.pi-map.index.md` for every non-ignored directory.
#### Index rendering target
Indexes should be small and role-first, using this fixed order:
- `# <relative-path> (index)`
- `## role`
- `## parent`
- `## children`
- `## files`
- `## links`
- `## workflows`
- `## dirty`
Index content should include:
- `dir`
- short role summary
- parent link
- child directory links
- likely files
- explicit sibling/child `index:` / `map:` links
- up to configured workflow hints
- dirty marker
Indexes must omit dependency edges and keep architectural prose minimal.
#### Rich-map rendering target
Maps should remain dense but richer, using this fixed order:
- `# <relative-path>`
- `## role`
- `## files`
- `## arch`
- `## tags`
- `## symbols`
- `## workflows`
- `## dirty`
Map content should include:
- `dir`
- sibling `index:` link near the top
- slightly richer file lines than today when useful
- prioritized symbol lists in large directories
### 3. Root artifact behavior
Root `.pi-map.index.md` is the Tier 0 routing artifact.
Root `.pi-map.md` remains rich and must:
- point to root index,
- restate the trust boundary,
- be suitable for automatic loading on architecture/system or ambiguous tasks.
Both root artifacts should explicitly encode the Tier 0 behavior.
### 4. Workflow synthesis
Workflow hints may be LLM-heavy, but they must be normalized into a deterministic shape before rendering.
Suggested normalized fields:
- `task`
- `read`
- `index`
- `map`
- optional `files`
Indexes should render workflow hints in compact task→route form.
Because the user wants cross-repo workflow routing allowed, workflow synthesis may target other directories outside the local subtree when the hints are strong.
### 5. Configuration
Paired mode is the only mode for now, but these knobs should be configurable:
- workflow-hint cap (default 5)
- rich-map tag cap (default 8)
Configuration belongs in shared config handling rather than ad hoc generator constants.
Suggested config shape for v1:
```yaml
workflowHintCap: 5
tagCap: 8
```
### 6. Patch behavior
Patch should always regenerate, never hand-edit, generated artifacts.
#### Changed directory
Always regenerate both the local `.pi-map.md` and `.pi-map.index.md`.
#### Ancestor strategy
Use auto-detected patch sizing with explicit override available from both CLI and tool surfaces.
- **Small default:** up to 3 related files, no structure/routing shift
- refresh ancestor indexes only
- **Large/structural:** export changes, directory shape changes, routing metadata impact, or wider cross-area effect
- refresh both ancestor indexes and ancestor maps
This suggests a helper that computes the affected chain and the required artifact depth for each ancestor.
### 7. Validation behavior
Validation should become pair-aware, not just file-list-aware.
Checks should include:
- missing sibling artifact
- sibling `dir` mismatch
- broken parent/child/sibling references
- stale `dirty` markers
- stale or missing likely-file references
- disagreement between map/index routing references
Validation failures for missing/stale indexes are hard failures.
When validation classifies a patch as structural, the output should explain the main reason concisely (for example export change, directory-shape change, or routing impact).
Repair should be exposed through `validate --fix`, which regenerates the affected chain by default.
Validation and repair messaging should stay concise and inline, with the main structural reason plus the chosen `patchMode` when repair runs.
### 8. Prompt and doc integration
`pi-extension.ts`, `SKILL.md`, and docs should explicitly teach:
- Tier 0 = protocol + root index
- indexes first, maps next
- local rich map + source before edits
- validate before freshness-sensitive handoff
- `index routes, map orients, source decides`
### 9. External dependency boundary
This design should stay self-contained.
- No vector store is required.
- No Engram dependency is required.
- Retrieval remains local, deterministic, and based on generated paired artifacts.
## Risks
| Risk | Mitigation |
|---|---|
| Pair generation increases implementation size | Keep a shared intermediate model and split delivery into slices |
| Workflow hints become noisy | Omit uncertain hints and normalize schema before rendering |
| Ancestor refresh logic becomes brittle | Centralize affected-chain computation and test small vs structural cases |
| Pair drift creates false confidence | Use common `dir` identity and pair-aware validation |
| Config knobs spread inconsistently | Keep workflow/tag caps in shared config loading |
## Settled v1 defaults
- Root `.pi-map.md` points to root `.pi-map.index.md` rather than repeating route summaries.
- The shared intermediate directory model remains internal; behavior is tested through rendered outputs.
- Validation surfaces structural reasons as concise inline messages.
@@ -0,0 +1,96 @@
# Proposal: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Proposal** |
| Based on | User interview + current design doc |
| Next | Spec |
## Problem
The current design centers consumption on discovering all `.pi-map.md` files and loading them at session start, with pruning for larger repositories. That scales poorly and frames maps as bulk context instead of as a navigation system.
The repo currently lacks:
1. a clear root routing artifact,
2. an explicit trust/use protocol,
3. a layered loading model,
4. fast per-directory navigation artifacts,
5. a clean split between routing metadata and richer orientation metadata.
## Proposed change
Adopt a layered navigation model built around a **paired artifact** for every non-ignored directory:
- `.pi-map.index.md` → fast routing
- `.pi-map.md` → richer orientation
### Tier model
- **Tier 0 — always injected:** Project Map Protocol + root `.pi-map.index.md`
- **Tier 1 — task-start load:** likely relevant directory indexes first, then rich maps for strongest matches
- **Tier 2 — exact verification:** source files, tests, configs, and docs
### Paired artifacts per directory
Every non-ignored directory should generate both:
- a rich `.pi-map.md`
- a lightweight `.pi-map.index.md`
The index is for movement. The map is for understanding. Source remains the final authority.
This change stays self-contained: it does not add a vector store, Engram dependency, or other external memory/retrieval backend.
### Operating model
- Root `.pi-map.md` remains rich, but points to root `.pi-map.index.md` for traversal.
- Each index points to its sibling rich map.
- Each rich map points back to its sibling index.
- If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
## In scope
- [ ] Generate `.pi-map.index.md` for every non-ignored directory
- [ ] Keep `.pi-map.md` as the richer sibling artifact for every non-ignored directory
- [ ] Emit a Project Map Protocol in docs/prompt guidance and in generated root artifacts
- [ ] Replace eager "read all maps" guidance with layered retrieval guidance
- [ ] Define fixed index/map responsibilities and section order
- [ ] Cascade freshness and repair logic across changed directory pairs and affected ancestors
- [ ] Add strict validation for missing/stale/broken index-map pairs
- [ ] Update docs and skill guidance to reflect the new model
## Out of scope
- [ ] Add a query-driven retrieval command such as `project-map context <query>`
- [ ] Preserve backward compatibility for agents that only understand the old preload-only model
- [ ] Replace source verification with map-based authority
- [ ] Redesign the core AST/LLM extraction pipeline beyond what is needed to feed the new artifacts
## Decisions from grilling
| Topic | Decision |
|---|---|
| Change split | Two specs |
| This change slug | `layered-map-protocol` |
| Follow-up change slug | `map-context-retrieval` |
| Backward compatibility required | No |
| Artifact model | Every non-ignored directory gets both `.pi-map.md` and `.pi-map.index.md` |
| Root Tier 0 | Project Map Protocol + root `.pi-map.index.md` |
| Root rich map | Auto-read for architecture/system questions and ambiguous tasks |
| Root relationship | Root map points to root index |
| Retrieval mode | Hybrid auto-load, indexes first |
| Patch strategy | Changed dir updates both; ancestors refresh by size/structure rules |
| Validation | Hard-fail on missing/stale indexes, repair via `validate --fix` |
| Prompt trust boundary | index routes, map orients, source decides |
| Extra retrieval backends | None in this change |
| SDD mode | Interactive |
| Artifact store | OpenSpec |
| PR strategy | Auto-forecast |
| Review budget | 400 lines |
## Success criteria
- [ ] Every non-ignored directory has both a fast index and a richer map
- [ ] The root artifacts clearly tell an agent how to navigate and when to read source
- [ ] The documented consumption model no longer requires reading every `.pi-map.md` up front
- [ ] Validation fails on stale or missing paired artifacts and offers a repair path
- [ ] Patch/refresh behavior keeps routing trustworthy without requiring full reinit for small changes
@@ -0,0 +1,229 @@
# Spec: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Spec** |
| Based on | [Proposal](proposal.md) |
| Next | Design |
## Overview
`pi-project-map` must move from a bulk-preload model to a layered navigation model. The system should always provide a small root protocol plus root index, selectively load likely relevant directory indexes first, load rich maps next for the strongest matches, and require source verification before edits or precise runtime claims.
## Decisions
| # | Question | Answer |
|---|---|---|
| 1 | Per-directory artifact model | Every non-ignored directory gets both `.pi-map.md` and `.pi-map.index.md` |
| 2 | Tier 0 default | Project Map Protocol + root `.pi-map.index.md` |
| 3 | Root rich map auto-read | Yes for architecture/system questions and ambiguous tasks |
| 4 | Retrieval style before query tool exists | Hybrid auto-load, indexes first |
| 5 | Backward compatibility with old preload-only model | Not required |
| 6 | Query-driven context bundling | Deferred to follow-up spec `map-context-retrieval` |
| 7 | Mode shape | Paired artifact mode is the only mode for now |
| 8 | Index section naming | Simple fixed names: `role`, `parent`, `children`, `files`, `links`, `workflows`, `dirty` |
| 9 | Rich map section naming | Current-plus fixed names: `role`, `files`, `arch`, `tags`, `symbols`, `workflows`, `dirty` |
| 10 | Patch-size override surface | Available from both CLI and tool surfaces |
| 11 | Structural classification output | Validation explains the main structural reason concisely |
| 12 | Extra retrieval backends | No vector store or Engram dependency in this change |
## Functional requirements
### 1. Paired artifacts per directory
Every non-ignored directory must generate two artifacts:
- `.pi-map.index.md` — quick-routing index
- `.pi-map.md` — rich orientation document
Both artifacts must share a common identity marker such as `dir: <relative-path>`.
### 2. Root entrypoint
The tool must generate a root navigation layer in both root artifacts.
#### Root Tier 0 behavior
Tier 0 must always include:
- Project Map Protocol
- root `.pi-map.index.md`
The generated root artifacts must explicitly state this behavior.
#### Root rich map behavior
Root `.pi-map.md` must:
- point to root `.pi-map.index.md` for traversal,
- restate the trust boundary,
- be suitable for automatic loading on architecture/system questions and ambiguous tasks.
### 3. Project Map Protocol
The generated system must define an explicit protocol with at least these rules:
1. Read the Project Map Protocol and root `.pi-map.index.md` first.
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
3. Load indexes before rich maps during task-start navigation.
4. Read the local rich map and actual source before editing.
5. Treat non-empty `## dirty` sections in either artifact as stale.
6. If source and generated artifacts disagree, trust source.
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
8. After editing source, run `project_map_patch` for each changed file.
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
### 4. Index contract
Every `.pi-map.index.md` must use a fixed section order and remain optimized for fast routing.
#### Required behavior
- Keep architectural prose minimal.
- Omit uncertain workflow/file hints rather than labeling confidence.
- Allow up to 5 workflow hints by default, with configuration support.
- Do not include dependency edges in indexes.
- Include explicit parent link when a parent mapped directory exists.
- For child directory entries, point to both child `index:` and child `map:`.
- Leaf-directory indexes remain tiny but still follow the same contract.
#### Fixed section order
Each index must use this fixed order:
1. `## role`
2. `## parent`
3. `## children`
4. `## files`
5. `## links`
6. `## workflows`
7. `## dirty`
#### Minimum contents
Each index must minimally support:
- title header in the form `# <relative-path> (index)`
- `dir` identity marker
- short role summary
- parent link where applicable
- child directories where applicable
- likely files
- explicit sibling/child `index:` / `map:` links
- short workflow hints in hybrid task→route form
- `## dirty`
### 5. Rich map contract
Every `.pi-map.md` must use a fixed section order and remain optimized for dense orientation.
#### Required behavior
- Rich maps must explicitly link to their sibling `.pi-map.index.md`.
- Rich maps may keep some routing overlap, but indexes remain primary for navigation.
- `files` entries should be slightly richer than the current dense line format when useful.
- Tags should default to up to 8, with configuration support.
- Symbols should be prioritized for density in large directories using structural importance plus LLM refinement.
- Cross-repo workflow routing should live mostly in indexes, not maps.
#### Fixed section order
Each rich map must use this fixed order:
1. `## role`
2. `## files`
3. `## arch`
4. `## tags`
5. `## symbols`
6. `## workflows`
7. `## dirty`
#### Minimum contents
Each rich map must minimally support:
- title header in the form `# <relative-path>`
- `dir` identity marker
- sibling `index:` link near the top
- richer but still compact workflow guidance
- `## dirty`
### 6. Workflow generation
Workflow hints may be LLM-heavy, but they must be normalized into a deterministic schema before rendering.
#### Index workflow rules
- Primary form is hybrid task→route, for example: `change CLI behavior -> read: src/.pi-map.index.md, src/cli.ts`
- Cross-repo routing is allowed.
- Uncertain hints should be omitted.
- Indexes should optimize for fast routing, not standalone completeness.
### 7. Shared generation model
Map and index generation must come from a shared intermediate directory model rather than unrelated passes.
The renderer may present different views, but the underlying directory identity and source facts must stay aligned.
### 8. Patch behavior
`project_map_patch` must never manually edit generated artifacts; it must regenerate them.
#### Changed directory
For every changed source file, patch must update both artifacts for the changed directory.
#### Ancestor refresh behavior
Patch sizing must be auto-detected with an explicit override path available from both CLI and tool surfaces.
- **Small change default:** up to 3 closely related files and no structural/routing shift
- changed directory: refresh map + index
- ancestors: refresh indexes only
- **Large/structural change:** exports, directory shape, routing metadata, or broader cross-area impact
- changed directory: refresh map + index
- ancestors: refresh both map + index
### 9. Validation behavior
`project_map_validate` must hard-fail when paired artifacts are missing or structurally stale.
Validation output should prefer concise inline wording, for example:
- `[structural] src/foo.ts: export change`
- `[stale-index] src/.pi-map.index.md: likely files out of date`
Validation must check at least:
- missing `.pi-map.index.md` or `.pi-map.md`
- stale `## dirty` sections in either artifact
- broken `index:` / `map:` references
- stale `files:` pointers
- map/index pair disagreement on routing hints or referenced areas
- sibling pair identity mismatches
When validation classifies a patch as structural, it should report the main reason concisely, such as export change, directory-shape change, or routing impact.
#### Repair
Repair must be exposed through `project-map validate --fix`.
Default repair scope is the affected directory pair plus the necessary ancestor chain.
Repair summaries should also report the chosen `patchMode` concisely, for example: `repaired affected chain (patchMode: structural)`.
### 10. Visibility and policy
`.pi-map.index.md` must follow the same hidden/ignored policy as `.pi-map.md`.
Agents should never manually edit either artifact directly.
## Non-functional requirements
- Prefer density over completeness.
- Keep indexes routing-first and maps understanding-first.
- Keep the protocol simple enough to emit in root artifacts and prompt guidance.
- Support configuration for top-level `workflowHintCap` and `tagCap` keys.
- Do not require a vector store, Engram, or other extra retrieval backend for this design.
## User flows
### Flow 1: Agent starts work on a repo
1. Agent reads the Project Map Protocol and root `.pi-map.index.md`
2. For architecture/system or ambiguous tasks, agent also reads root `.pi-map.md`
3. Agent opens likely relevant directory indexes first
4. Agent opens the strongest-match rich maps
5. Agent reads source only when exact behavior or editing is involved
### Flow 2: Agent edits a file
1. Agent routes with index data
2. Agent reads the local rich map and actual source
3. Agent edits source
4. Agent runs `project_map_patch` for changed files
5. Patch refreshes the changed directory pair and the required ancestor chain
### Flow 3: Validation and repair
1. Agent runs `project_map_validate`
2. Missing/stale pair failures are reported as hard errors
3. Agent runs `project-map validate --fix` when repair is desired
4. The affected chain is regenerated
## Acceptance criteria
- [ ] Every non-ignored directory produces both `.pi-map.md` and `.pi-map.index.md`
- [ ] Tier 0 behavior is both documented and emitted in generated root artifacts
- [ ] Indexes have a fixed routing-first contract with parent/child/sibling links
- [ ] Rich maps have a fixed orientation-first contract with sibling index links
- [ ] `project_map_patch` refreshes both changed-directory artifacts and size-appropriate ancestor artifacts
- [ ] `project_map_validate` hard-fails on missing/stale paired artifacts and supports `--fix`
- [ ] Workflow-hint count and tag cap are configurable
- [ ] Prompt guidance expresses: index routes, map orients, source decides
@@ -0,0 +1,7 @@
# Sync Report
Status: NOT-APPLICABLE
Reason: This repository uses legacy flat change artifacts (proposal.md/spec.md/design.md/tasks.md) and does not maintain a canonical openspec/specs/ tree for these changes. No canonical spec sync was performed.
User-approved fallback: archive completed change as an audit record without canonical spec sync.
@@ -0,0 +1,75 @@
# Tasks: Layered Map Protocol
## Status
| Field | Value |
|---|---|
| Phase | **Tasks** |
| Based on | [Design](design.md) |
| Next | Apply |
## Delivery slices
### Slice 1: Shared model and paired rendering
**Scope**: shared intermediate directory model, universal paired artifacts, fixed section names/order, root Tier 0 entrypoint
**Review goal**: establish the core paired architecture without retrieval-command work
**Tasks**:
1. [ ] Add a shared intermediate directory model for paired map/index generation
2. [ ] Extend render/parse support for paired `.pi-map.md` and `.pi-map.index.md` outputs with the agreed fixed section names/order
3. [ ] Add generation support for `.pi-map.index.md` in every non-ignored directory
4. [ ] Enrich root artifacts with explicit Tier 0 behavior and sibling links
5. [ ] Add/update tests for map/index rendering, fixed order, and sibling identity
### Slice 2: Routing metadata and workflow normalization
**Scope**: likely files, parent/child/sibling links, workflow schema, rich-map metadata density rules
**Review goal**: make the navigation model operational on real output
**Tasks**:
1. [ ] Add index-first routing metadata generation from AST/structure + LLM judgment
2. [ ] Normalize workflow hints into a deterministic schema before rendering
3. [ ] Add parent/child/sibling `index:` / `map:` references
4. [ ] Implement rich-map density rules for slightly richer file lines, tag cap, and prioritized symbols
5. [ ] Add/update tests for routing hints, leaf indexes, and workflow omission on low confidence
### Slice 3: Patch sizing, cascade refresh, and repair
**Scope**: changed-directory pair refresh, ancestor-chain refresh rules, validation hard failures, `validate --fix`
**Review goal**: make freshness and repair trustworthy
**Tasks**:
1. [ ] Make `project_map_patch` always regenerate both artifacts for the changed directory
2. [ ] Add auto-detected small vs structural patch sizing with explicit override in both CLI and tool surfaces
3. [ ] Refresh ancestor indexes for small changes and ancestor map+index pairs for structural changes
4. [ ] Expand validation for pair-aware checks, hard failures on missing/stale indexes, and concise structural-reason output
5. [ ] Add/confirm `project-map validate --fix` repair flow for affected-chain regeneration
6. [ ] Add/update tests for patch classification, ancestor refresh, and repair behavior
### Slice 4: Prompt guidance, config knobs, and docs
**Scope**: Project Map Protocol, Tier 0 runtime guidance, workflow/tag config, docs rewrite
**Review goal**: align runtime behavior and documentation with the final operating model
**Tasks**:
1. [ ] Update `pi-extension.ts` prompt guidance to teach the layered paired protocol
2. [ ] Update `SKILL.md` to describe indexes as routing and maps as orientation
3. [ ] Update `README.md` and `design-doc.md` consumption guidance
4. [ ] Add config support for workflow-hint cap and rich-map tag cap
5. [ ] Keep the paired-artifact model self-contained with no vector-store or Engram dependency
6. [ ] Add/update tests for config-driven caps and root Tier 0 guidance
## Acceptance checklist
- [x] Every non-ignored directory has both `.pi-map.md` and `.pi-map.index.md`
- [x] Root Tier 0 behavior is emitted in generated root artifacts
- [x] Indexes are routing-first, role-first, and include parent/child/sibling links
- [x] Rich maps are orientation-first and include sibling index links
- [x] Changed-directory patch always regenerates both artifacts
- [x] Ancestor refresh follows small vs structural rules
- [x] Validation hard-fails on missing/stale paired artifacts and supports `validate --fix`
- [x] Workflow-hint count and tag cap are configurable
- [x] No vector-store or Engram dependency is introduced
- [x] `npm run typecheck` passes
- [x] `npm test` passes
- [x] `npm run lint` passes *(N/A: repo has no ESLint config; pre-existing repository gap, not a change regression)*
## Review workload note
This change crosses format, generation, patching, validation, config, docs, and prompt guidance. Keep implementation in narrow, reviewable slices and avoid one oversized PR.
@@ -0,0 +1,22 @@
# Verify Report
Status: PASS
Change: layered-map-protocol
Verified summary: Implemented paired map/index artifacts, layered routing/orientation protocol, pair-aware patch/validate/reinit flows, config caps, and docs/runtime guidance.
Evidence commands:
- npm run typecheck
- npx vitest run
- npm run build
- node dist/cli.js validate .
Current validation state:
- npm run typecheck: PASS
- npx vitest run: PASS (262/262)
- npm run build: PASS
- node dist/cli.js validate .: PASS
Notes:
- Repo-wide lint remains unavailable because the repository has no ESLint config; treated as a pre-existing repository-level gap, not a change regression.
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-map-context-retrieval (index)
dir: openspec/changes/archive/2026-06-11-map-context-retrieval
## role
Archives a completed specification change package for implementing metadata-driven context retrieval capabilities in a project-map tool.
## parent
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## children
-
## files
- apply-progress.md
- archive-report.md
- design.md
- proposal.md
- spec.md
- sync-report.md
- tasks.md
- verify-report.md
## links
index: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.md
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-map-context-retrieval
dir: openspec/changes/archive/2026-06-11-map-context-retrieval
index: openspec/changes/archive/2026-06-11-map-context-retrieval/.pi-map.index.md
## role
Archives a completed specification change package for implementing metadata-driven context retrieval capabilities in a project-map tool.
## files
- apply-progress.md | Documents the completion status of a project implementing deterministic index-first context retrieval via tool and CLI, along with retrieval documentation and skill guidance.
- archive-report.md | Documents the archival of a set of specification change documents including metadata about archive location, mode, and preserved inputs.
- design.md | Design document for a lightweight context retrieval system that scans paired project-map metadata to find and return relevant code context as a markdown bundle for AI agents. | dep: pi-extension.ts, src/index.ts, src/cli/*, paired-artifact parser/model from layered protocol, .pi-map.index.md, .pi-map.md
- proposal.md | Proposes a "Map Context Retrieval" tool that enables natural-language queries to return compact, metadata-driven context bundles from a layered map protocol. | dep: layered-map-protocol, Pi tool, CLI
- spec.md | Defines a specification for adding a retrieval-oriented `context` command to a project-map tool that converts user tasks into compact routing bundles for LLM agents | dep: layered-map-protocol, project-map (Pi tool/CLI), markdown output formatting
- sync-report.md | Documents that a canonical spec synchronization was not performed due to legacy flat change artifact structure, with user-approved archival fallback
- tasks.md | Defines implementation tasks for adding map context retrieval functionality to a Pi tool, using paired index/map metadata for ranked, query-based context retrieval. | dep: design.md, Pi tool, project-map context, npm (typecheck, test, lint)
- verify-report.md | Documents verification results for a code change implementing deterministic index-first context retrieval. | dep: npm, vitest, node, CLI tooling
## arch
Flat archival directory structure preserving legacy change artifacts (specification, design, proposal, tasks, verification, sync, apply-progress, archive-report) without hierarchical organization.
## tags
map, context, retrieval, report, documents, project, index, tool
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,10 @@
# Apply Progress
Status: complete
Summary: Implemented deterministic index-first context retrieval via tool and CLI, plus retrieval docs/skill guidance.
Implemented commits:
- c6064f8 Implement layered maps and context retrieval
Stale-checkbox reconciliation: historical implementation completed in committed work above; tasks.md reconciled to checked state on 2026-06-11.
@@ -0,0 +1,9 @@
# Archive Report
Status: archived
Archived path: openspec/changes/archive/2026-06-11-map-context-retrieval
Archive mode: manual archive fallback in openspec-only repo with legacy flat change specs; canonical sync marked not-applicable in sync-report.md.
Inputs preserved in archive: proposal.md, spec.md, design.md, tasks.md, apply-progress.md, verify-report.md, sync-report.md.
@@ -0,0 +1,65 @@
# Design: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Design** |
| Based on | [Spec](spec.md) |
| Next | Tasks |
## Design summary
This change adds a lightweight retrieval layer on top of paired project-map metadata. It should not require a new external storage system, vector store, or Engram dependency. Instead, it should scan root and directory indexes first, expand to rich maps and files from the strongest candidates, and emit an agent-friendly bundle.
## Likely implementation areas
- `pi-extension.ts` for the first-class tool surface
- `src/index.ts` for exports
- new retrieval module, e.g. `src/context.ts` or `src/retrieve.ts`
- shared parsing/model code from the layered protocol
- `src/cli/*` for follow-up CLI wiring
- docs and skill guidance
## Retrieval pipeline
1. Accept a natural-language `query`
2. Read root `.pi-map.index.md` and, when needed, root `.pi-map.md`
3. Parse paired index/map metadata through the shared format/model layer
4. Score candidate directories primarily from indexes
5. Keep the top 3 candidates by default
6. Expand strongest candidates to rich maps, likely files, and symbols
7. Emit a compact markdown bundle with stable section order and retrieval-specific title `# Context bundle: <query>`
## Candidate scoring inputs
- direct term matches in workflow hints
- direct term matches in likely files
- parent/child/sibling link context
- direct term matches in tags
- direct term matches in symbols
- path/name similarity
- root workflow routing hits
A first implementation can use deterministic weighted lexical scoring.
Candidate-selection reasoning does not need to be exposed by default.
## Parsing strategy
Reuse the paired-artifact parser/model from the layered protocol. Avoid retrieval-specific ad hoc parsing.
## Risks
| Risk | Mitigation |
|---|---|
| Retrieval becomes too fuzzy to trust | Keep output advisory and always direct agent back to source |
| Pair parser complexity grows | Extend shared format/model logic instead of command-local parsing |
| Command output becomes too large | Limit result count and keep instructions terse |
## Open choices
1. Exact scoring weights for workflow hints vs files vs tags vs symbols
2. Whether the tool should support optional structured output later
3. Whether broad architecture queries should be allowed to exceed the usual top-3 default in a later version
4. Whether later versions should support optional root-rich-map inclusion as a flag
@@ -0,0 +1,53 @@
# Proposal: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Proposal** |
| Based on | Follow-up to `layered-map-protocol` |
| Next | Spec |
## Problem
After the layered map protocol lands, the agent will know it should load root protocol + root index first, then route through per-directory indexes and rich maps. But the repo will still lack an ergonomic retrieval primitive that can turn a natural-language task into a compact context bundle.
Without that helper, the agent still has to manually inspect directory indexes, rank candidate areas, and expand to rich maps/files. That weakens the value of the new routing layer.
## Proposed change
Add a retrieval tool and command shape:
```bash
project-map context "<user task>"
```
The first-class surface should be a Pi tool. CLI support can follow the same shape.
The retrieval result should return a compact context bundle containing:
- relevant indexes,
- strongest-match rich maps,
- likely source files,
- relevant symbols when useful,
- short instructions on what to read next.
This remains a local metadata-driven retrieval feature, not a vector-store or Engram-backed memory system.
## In scope
- [ ] Add Pi tool support for `project-map context <query>`
- [ ] Add CLI support using the same shape after the tool contract is stable
- [ ] Rank results using paired index/map metadata from the layered protocol
- [ ] Return a compact, markdown-first, agent-friendly context bundle
- [ ] Document recommended usage from prompts and docs
## Out of scope
- [ ] Full semantic search across arbitrary source contents
- [ ] Replacing source verification with map-based answers
- [ ] Building a heavyweight external indexer or vector database
## Success criteria
- [ ] A natural-language task can resolve to likely indexes/maps/files/symbols without manual repo scanning
- [ ] Output is compact enough to inject directly into the next step
- [ ] Retrieval builds on generated paired metadata rather than bypassing it
- [ ] The first version is tool-first and markdown-first
@@ -0,0 +1,127 @@
# Spec: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Spec** |
| Based on | [Proposal](proposal.md) |
| Next | Design |
## Overview
Add a retrieval-oriented `context` command that turns a user task into a compact project-map routing bundle. This change depends on the paired root/per-directory index+map metadata introduced by `layered-map-protocol`.
## Dependency
This change should not begin implementation before `layered-map-protocol` has landed enough metadata to support routing and ranking.
## Functional requirements
### 1. Tool-first surface
Pi should expose `project-map context` as a first-class tool action so agents can request a context bundle directly.
For v1, the tool input should be minimal:
- `query: string`
### 2. CLI follow-up surface
The CLI should support the same conceptual shape:
```bash
project-map context "<query>"
```
CLI support may follow after the tool contract is stable.
The CLI should mirror the same main input name: `query`.
### 3. Bundle contents
A context bundle must contain, at minimum:
- relevant indexes
- strongest-match relevant maps
- likely files
- relevant symbols when useful
- short next-step instructions
### 4. Ranking behavior
The command must rank candidates using generated metadata such as:
- root workflow hints
- directory index routing hints
- parent/child/sibling link structure
- package tags
- symbol references
- task/workflow entries
- `files:` / `index:` / `map:` targets
Indexes should be the first routing layer. Rich maps and symbols should expand from the strongest routed candidates.
By default, retrieval should usually surface the top 3 candidate directories, with adaptive omission of weaker rich maps when confidence drops.
### 5. Trust boundary
The bundle must instruct the agent to:
- read indexes first for orientation,
- read rich maps next when deeper context is needed,
- read source before editing or asserting exact behavior.
## Output shape
The first version should be markdown-first, with stable, schema-like sections optimized for LLM consumption rather than human prose.
Default section order:
1. `query`
2. `relevant indexes`
3. `relevant maps`
4. `likely files`
5. `relevant symbols`
6. `instructions`
The retrieval title format should remain retrieval-specific:
- `# Context bundle: <query>`
A representative response shape:
```markdown
# Context bundle: validation stale signatures
## query
validation stale signatures
## relevant indexes
- .pi-map.index.md
- src/.pi-map.index.md
- tests/.pi-map.index.md
## relevant maps
- src/.pi-map.md
- tests/.pi-map.md
## likely files
- src/validate.ts
- src/ast/ast-extract.ts
- tests/validate.test.ts
## relevant symbols
- validateMaps
- generateDirectoryMap
- extractFileAST
## instructions
Read the indexes first, then the strongest-match rich maps, then verify behavior from source before editing.
```
The tool should omit weak/unhelpful symbol output rather than forcing a noisy symbol section.
## Non-functional requirements
- Output must be compact enough for prompt injection.
- Ranking should be deterministic enough to test.
- The implementation should prefer lightweight heuristics over heavyweight indexing.
- Retrieval should reuse the paired-artifact parsing layer instead of ad hoc string slicing.
- Retrieval should not require a vector store, Engram, or another external memory backend.
- The first version should accept natural language queries first; structured filters can come later.
- Candidate explanations are not required by default.
- Result-count override and root-rich-map inclusion can come later if needed, but are not part of v1.
## Acceptance criteria
- [ ] Pi exposes `project-map context <query>` as a tool-first surface
- [ ] CLI support follows the same shape after the tool contract is stable
- [ ] Retrieval output includes indexes, strongest-match maps, files, and instructions
- [ ] Symbol hints appear only when paired metadata makes them useful
- [ ] Ranking respects the index-first routing model and usually returns top 3 candidates
- [ ] Docs show when to use the retrieval command
- [ ] Trust-boundary guidance remains explicit
@@ -0,0 +1,7 @@
# Sync Report
Status: NOT-APPLICABLE
Reason: This repository uses legacy flat change artifacts (proposal.md/spec.md/design.md/tasks.md) and does not maintain a canonical openspec/specs/ tree for these changes. No canonical spec sync was performed.
User-approved fallback: archive completed change as an audit record without canonical spec sync.
@@ -0,0 +1,34 @@
# Tasks: Map Context Retrieval
## Status
| Field | Value |
|---|---|
| Phase | **Tasks** |
| Based on | [Design](design.md) |
| Next | Apply |
## Tasks
1. [ ] Add Pi tool support for `project-map context` with `query` as the main input
2. [ ] Add retrieval module for scoring paired index/map metadata
3. [ ] Reuse or extend shared paired-artifact parsing/model code
4. [ ] Rank candidate directories from indexes first, then expand to maps/files/symbols
5. [ ] Emit compact markdown-first context bundles with stable section order and retrieval-specific `Context bundle` title
6. [ ] Add tests for retrieval ranking, top-3 default behavior, and output shape
7. [ ] Add CLI support after the tool contract is stable
8. [ ] Update docs and skill guidance for retrieval usage
## Acceptance checklist
- [x] Tool works for natural-language queries via `query`
- [x] Output includes relevant indexes, strongest-match maps, likely files, symbols when useful, and instructions
- [x] Retrieval depends on paired metadata rather than raw source scanning
- [x] Ranking follows the index-first routing model with a top-3 default
- [x] `npm run typecheck` passes
- [x] `npm test` passes
- [x] `npm run lint` passes *(N/A: repo has no ESLint config; pre-existing repository gap, not a change regression)*
## Sequencing note
This change is intentionally follow-up work. It should start after the layered map protocol produces reliable paired routing metadata.
The first delivery surface is the Pi tool. CLI support follows the stabilized tool contract.
@@ -0,0 +1,23 @@
# Verify Report
Status: PASS
Change: map-context-retrieval
Verified summary: Implemented deterministic index-first context retrieval via tool and CLI, plus retrieval docs/skill guidance.
Evidence commands:
- npm run typecheck
- npx vitest run
- npm run build
- node dist/cli.js context "validation routing"
- node dist/cli.js validate .
Current validation state:
- npm run typecheck: PASS
- npx vitest run: PASS (262/262)
- npm run build: PASS
- node dist/cli.js validate .: PASS
Notes:
- Repo-wide lint remains unavailable because the repository has no ESLint config; treated as a pre-existing repository-level gap, not a change regression.
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-project-map-prompt-injection (index)
dir: openspec/changes/archive/2026-06-11-project-map-prompt-injection
## role
Archives a completed prompt-injection feature specification that enables runtime injection of project-map artifacts into LLM context with configurable guidance modes and budget controls.
## parent
index: openspec/changes/archive/.pi-map.index.md
map: openspec/changes/archive/.pi-map.md
## children
-
## files
- apply-progress.md
- archive-report.md
- design.md
- proposal.md
- spec.md
- sync-report.md
- tasks.md
- verify-report.md
## links
index: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.index.md
map: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.md
## workflows
-
## dirty
-
@@ -0,0 +1,26 @@
# openspec/changes/archive/2026-06-11-project-map-prompt-injection
dir: openspec/changes/archive/2026-06-11-project-map-prompt-injection
index: openspec/changes/archive/2026-06-11-project-map-prompt-injection/.pi-map.index.md
## role
Archives a completed prompt-injection feature specification that enables runtime injection of project-map artifacts into LLM context with configurable guidance modes and budget controls.
## files
- apply-progress.md | Documents the completion status and implementation details of a prompt-injection feature delivered across five incremental slices.
- archive-report.md | Documents the archival status and metadata of a deprecated project change specification directory.
- design.md | Design document for adding a runtime prompt injection layer that guides LLM behavior using paired map/index artifacts through configurable modes, canonical markers, and budgeted context expansion. | dep: pi-extension.ts, src/config.ts, spec.md, event.messages, before_agent_start, before_provider_request, context hooks, .pi-project-map.json, .pi-map.index.md, .pi-map.md
- proposal.md | Proposes a runtime prompt-injection policy for project-map artifacts with configurable guidance modes, hybrid context budgets, and reinjection avoidance
- spec.md | Defines a specification for automatic runtime prompt injection of project map/index artifacts with configurable guidance modes, budgeted expansion, and reinjection avoidance based on actual outgoing context scanning.
- sync-report.md | Documents that a canonical spec synchronization was not performed due to legacy flat change artifact structure, with user-approved archival fallback
- tasks.md | Defines phased implementation tasks for a prompt injection policy system that controls how project map artifacts are injected into LLM context with budget constraints, mode semantics, and reinjection avoidance.
- verify-report.md | Documents verification results for a prompt-injection security feature implementation in a software project.
## arch
Document-driven specification architecture using phased slice-based delivery (proposal → design → spec → tasks → verification), with flat artifact structure and canonical marker-based context expansion patterns.
## tags
map, prompt, injection, project, report, documents, artifacts, context
## symbols
-
## workflows
-
## dirty
-
@@ -0,0 +1,15 @@
# Apply Progress
Status: complete
Summary: Implemented prompt-injection slices 1-5: mode/config surface, root-pair preload, reinjection/dedupe, strict/advisory/strong/off semantics, and documentation/runtime alignment.
Implemented commits:
- 56560d9 feat(prompt): implement prompt injection slice 1
- 621434b feat(prompt): implement prompt injection slice 2
- 19666c9 feat(prompt): implement prompt injection slice 3
- 58e8bd3 feat(prompt): implement prompt injection slice 4
- 11365fa docs(prompt): align prompt injection guidance and runtime copy
- c11d49d spec(prompt): add project map prompt injection change set
Stale-checkbox reconciliation: historical implementation completed in committed work above; tasks.md reconciled to checked state on 2026-06-11.
@@ -0,0 +1,9 @@
# Archive Report
Status: archived
Archived path: openspec/changes/archive/2026-06-11-project-map-prompt-injection
Archive mode: manual archive fallback in openspec-only repo with legacy flat change specs; canonical sync marked not-applicable in sync-report.md.
Inputs preserved in archive: proposal.md, spec.md, design.md, tasks.md, apply-progress.md, verify-report.md, sync-report.md.
@@ -0,0 +1,235 @@
# Design: Project Map Prompt Injection
## Status
| Field | Value |
|---|---|
| Phase | **Design** |
| Based on | [Spec](spec.md) |
| Next | Tasks |
## Design summary
This change adds a runtime guidance layer on top of the paired map/index artifact system. The implementation should not redesign map generation or retrieval. Instead, it should add a well-scoped injection pipeline that:
- emits honest startup hints before init,
- preloads the root pair after init,
- expands under a hybrid budget,
- avoids redundant reinjection by scanning real outgoing context,
- varies guidance strength through explicit modes,
- proves correctness with integration-heavy validation.
## Affected areas
### Source files likely to change
- `pi-extension.ts`
- shared config handling (`src/config.ts` or equivalent)
- runtime helpers for map/index discovery and injection selection
- tests covering extension lifecycle and per-turn behavior
- docs/runtime guidance surfaces if needed
### New modules likely to appear
- `src/prompt-injection.ts` or equivalent runtime helper
- optional helper for canonical marker construction / detection
- optional helper for token-budget estimation across paired artifacts
## Architecture changes
### 1. Injection policy helper
Centralize prompt-injection logic in one helper rather than scattering it across hooks.
Suggested responsibilities:
- determine current mode (`off` / `advisory` / `strong` / `strict`),
- detect whether artifacts exist,
- build pre-init startup hint,
- build post-init root-pair payload,
- estimate expansion budget,
- choose additional artifacts under the budget,
- construct a canonical injected marker/block,
- scan outgoing context or payload for that marker,
- decide whether reinjection is required.
This helper is the main guard against drift between startup hints, context hooks, and strict-mode checks.
### 2. Injection surfaces
Use different extension surfaces for different responsibilities.
#### Pre-init / startup hint
Use `before_agent_start` for lightweight startup guidance before real artifacts exist.
Required behavior:
- inject only a hint,
- tell the agent to run `project_map_init`,
- keep the hint visible and inspectable.
#### Post-init root-pair preload
Use `before_agent_start` to guarantee the initial post-init root-pair preload for a prompt.
Required behavior:
- inject root `.pi-map.index.md`,
- inject root `.pi-map.md`,
- optionally append brief protocol text only if needed by the selected mode.
#### Relevant-turn reinjection
Use `context` for relevant-turn checks.
Required behavior:
- inspect `event.messages`,
- decide whether the canonical block is already present,
- only add the root pair / budgeted expansion if absent,
- avoid rescanning on every trivial turn in `strong` mode.
#### Payload fallback
Use `before_provider_request` only as a fallback or debugging surface when message-layer detection is insufficient.
### 3. Canonical marker design
Deduplication depends on stable canonical detection.
The implementation should stamp injected content with a canonical marker block.
Recommended v1 shape:
- a deterministic wrapper such as `<!-- PI_MAP_ROOT_PAIR_START -->` / `<!-- PI_MAP_ROOT_PAIR_END -->`,
- normalized artifact identity lines for root `.pi-map.index.md` and root `.pi-map.md`,
- trust-boundary text within the same wrapped block when the active mode requires it.
Marker rules:
- stable across turns,
- independent of provider formatting quirks,
- easy to scan in both `event.messages` and final provider payload,
- robust enough that root-pair presence can be detected without brittle full-text matching.
### 4. Budgeting
Budgeting should be deterministic and layered.
#### Context-window discovery and fallback
- Prefer active-model context-window metadata exposed by Pi runtime/model selection.
- If context-window metadata is unavailable, fall back to the configured absolute cap.
- The fallback path should be explicit in logs/debug behavior so budget decisions remain auditable.
#### Required order
1. compute effective budget from relative percentage + optional absolute cap,
2. reserve the root pair first,
3. expand outward using a deterministic traversal order,
4. stop when the budget would be exceeded.
#### Traversal strategy
The spec does not force exact traversal heuristics, but the design should prefer:
- root pair first,
- shallow structural coverage before deep leaves,
- predictable order over opaque scoring.
This keeps automatic injection understandable and auditable.
### 5. Mode control surface
Expose a config/runtime setting for the four modes.
Because the current project config is loaded from flat JSON in `.pi-project-map.json`, v1 should prefer a flat compatible shape rather than forcing an immediate nested/YAML migration.
Suggested v1 config shape:
```json
{
"promptInjectionMode": "strong",
"contextBudgetPercent": 15,
"contextBudgetMaxTokens": 100000
}
```
Migration note:
- keep existing flat JSON loading in `src/config.ts`,
- treat these as new additive keys,
- decide explicitly whether existing `contextBudget` is deprecated, ignored for injection, or retained only for older LLM-analysis paths.
Optional runtime UX may later mirror thinking-level controls, but v1 implementation can start with config-driven mode selection as long as the semantics are the same.
### 6. Mode semantics
#### `off`
- no automatic artifact injection,
- no startup/init hint beyond existing tool/docs discovery.
#### `advisory`
- startup/init hints enabled,
- optional root-pair preload,
- light reminders,
- weaker reinjection behavior.
#### `strong`
- root-pair preload required,
- budgeted expansion required,
- relevant-turn reinjection checks required,
- reminders before edits and architecture-sensitive reasoning.
#### `strict`
- everything in `strong`, plus:
- when protocol path is missing during sensitive actions, require explicit bypass justification.
### 7. Relevant-turn detection
`strong` mode should not rescan on every turn.
Relevant-turn triggers should be mapped to concrete runtime signals where possible:
- agent start,
- edit intent / edit tool preparation,
- architecture-sensitive planning prompts,
- compaction completion,
- root-pair artifact change detection.
This will likely require some combination of:
- hook-local heuristics,
- observed tool calls,
- file timestamp/hash checks for root artifacts,
- compaction event handling.
### 8. Visibility model
The design should preserve mixed visibility.
- startup hints: visible and inspectable,
- raw injected artifact blocks: agent-visible by default,
- the fact that automatic injection exists should remain discoverable.
The implementation may use hidden custom message types for artifact payloads, but should not make startup/init behavior opaque.
### 9. Validation strategy
This change is validation-heavy.
Unit tests alone are not enough, because the main risk is runtime interaction between hooks, message mutation, compaction, and reinjection.
#### Integration focus areas
- startup before init,
- startup after init,
- root-pair marker insertion,
- reinjection suppression when marker already exists,
- reinjection after compaction,
- reinjection after artifact mutation,
- mode differences,
- strict-mode bypass path,
- mixed visibility expectations,
- synthetic event-sequence coverage for relevant-turn heuristics such as edit-intent, architecture-sensitive reasoning, compaction, and artifact invalidation.
### 10. Documentation impact
Runtime guidance docs must align with the spec, but retrieval docs remain separate.
Docs should teach:
- startup hint before init,
- root-pair automatic preload after init,
- trust boundary,
- mode ladder,
- relevant-turn reinjection behavior,
- integration-test importance.
## Risks
| Risk | Mitigation |
|---|---|
| Message-level scanning misses provider serialization quirks | Add payload fallback via `before_provider_request` |
| Root-pair marker becomes brittle | Use deterministic boundaries and normalized artifact identity lines |
| 15% + 100k is too aggressive in some fleets | Keep both knobs configurable and document the opinionated default |
| Relevant-turn detection becomes fuzzy | Centralize detection heuristics and prove them with integration tests |
| `strict` mode causes friction | Keep `strong` as default and isolate strict-only bypass behavior |
## Settled defaults
- Root pair is always guaranteed after init.
- Reinjection avoidance is based on canonical outgoing-context scanning.
- Retrieval remains out of scope.
- Mixed visibility is the intended baseline.
- Four modes exist, with `strong` as default.
- Default budget is 15% of active context window, capped at 100k tokens.
@@ -0,0 +1,88 @@
# Proposal: Project Map Prompt Injection
## Status
| Field | Value |
|---|---|
| Phase | **Proposal** |
| Based on | Grill Me checkpoint + runtime/doc inspection |
| Next | Spec |
## Problem
`pi-project-map` currently provides strong generated artifacts and tool surfaces, but weak automatic runtime guidance.
Today the repo has:
- a one-time `before_agent_start` hint,
- a `session_start` dirty-state UI notification,
- tool metadata and docs,
- generated `.pi-map.md` / `.pi-map.index.md` artifacts.
But it lacks a clear spec for:
1. when maps/indexes should be injected automatically,
2. how much should be injected under a context budget,
3. how reinjection should avoid wasting prompt space,
4. how visible that guidance should be to the user,
5. how strong enforcement should be,
6. how to validate this behavior with integration tests.
## Proposed change
Adopt a runtime prompt-injection policy for project-map that:
- always injects the **root pair** after init,
- expands outward under a **hybrid context budget cap**,
- avoids redundant reinjection by scanning the actual outgoing context,
- keeps **retrieval guidance separate** from this spec,
- exposes **four configurable guidance modes** (`off`, `advisory`, `strong`, `strict`),
- uses **mixed visibility**: user-visible startup hints, agent-visible artifact injection,
- requires **extensive integration tests** for reinjection and context-scanning behavior.
## In scope
- [ ] Define pre-init startup hint behavior
- [ ] Define post-init automatic root-pair preload behavior
- [ ] Define hybrid budget defaults and config shape
- [ ] Define outgoing-context scanning for reinjection avoidance
- [ ] Define visibility model for hints vs injected artifacts
- [ ] Define configurable guidance-strength modes and default
- [ ] Define relevant-turn reinjection triggers for `strong`
- [ ] Define strict-mode bypass-justification behavior at the spec level
- [ ] Define validation and integration-test expectations
- [ ] Define implementation slices for runtime hooks/config/tests/docs
## Out of scope
- [ ] Redesign retrieval ranking or `project_map_context`
- [ ] Merge retrieval behavior into this policy spec
- [ ] Redesign paired artifact contents
- [ ] Introduce vector stores, Engram, or external retrieval backends
- [ ] Guarantee provider-agnostic perfect token counting beyond best-effort budgeting
## Decisions from grilling
| Topic | Decision |
|---|---|
| Change slug | `project-map-prompt-injection` |
| Scope split | Retrieval remains a separate spec |
| Pre-init | Inject only a lightweight `project_map_init` hint |
| Fake/synthetic maps before init | No |
| Guaranteed minimum | Always inject root `.pi-map.index.md` + root `.pi-map.md` |
| Budget model | Hybrid cap |
| Budget config | Relative percentage + optional absolute cap; smaller wins |
| Budget default | 15% of context window, capped at 100k tokens |
| Reinjection avoidance | Canonical marker scanning in actual outgoing context |
| Visibility | Mixed |
| Mode set | `off`, `advisory`, `strong`, `strict` |
| Default mode | `strong` |
| `strong` triggers | Relevant turns only |
| Validation | Extensive integration tests required |
## Success criteria
- [ ] The spec clearly defines what is injected before and after init
- [ ] The spec clearly defines how much is injected and how budgets are applied
- [ ] The spec clearly defines when reinjection checks happen and how duplicates are avoided
- [ ] The spec clearly defines mode semantics and the default mode
- [ ] The spec clearly separates runtime injection policy from retrieval behavior
- [ ] The implementation plan can be executed in narrow, reviewable slices
- [ ] The validation section makes integration coverage a first-class requirement
@@ -0,0 +1,226 @@
# Spec: Project Map Prompt Injection
## Status
| Field | Value |
|---|---|
| Phase | **Spec** |
| Based on | [Proposal](proposal.md) |
| Next | Design |
## Overview
`pi-project-map` must define a deliberate automatic runtime guidance model for map/index usage. After init, the system should preload the root pair, expand under a bounded context budget, avoid redundant reinjection by inspecting actual outgoing context, and scale enforcement through configurable guidance modes.
This spec covers **automatic injection and maintenance guidance only**. Retrieval behavior remains covered by `map-context-retrieval`.
## Decisions
| # | Question | Answer |
|---|---|---|
| 1 | Retrieval included in this spec | No |
| 2 | Pre-init behavior | Lightweight init hint only |
| 3 | Synthetic map content before init | No |
| 4 | Guaranteed post-init minimum | Always inject root pair |
| 5 | Budget model | Hybrid cap |
| 6 | Budget knobs | Relative percentage + optional absolute cap |
| 7 | Budget default | 15% of context window, capped at 100k tokens |
| 8 | Reinjection avoidance | Canonical marker scan in outgoing context/payload |
| 9 | Visibility | Mixed |
| 10 | Mode set | `off`, `advisory`, `strong`, `strict` |
| 11 | Default mode | `strong` |
| 12 | `strong` reinjection cadence | Relevant turns only |
| 13 | Validation expectation | Extensive integration coverage |
## Functional requirements
### 1. Pre-init behavior
Before generated map/index artifacts exist, the extension must inject only a lightweight startup hint.
#### Required behavior
- The hint must tell the agent that the project-map extension is active.
- The hint must instruct the agent to run `project_map_init`.
- The hint must not claim that real map/index artifacts already exist.
- The system must not inject synthetic or fake map content before real artifacts are generated.
### 2. Post-init guaranteed preload
After real artifacts exist, the system must guarantee a minimum preload of the root pair:
- root `.pi-map.index.md`
- root `.pi-map.md`
This root pair is the minimum automatic preload before budgeted expansion begins.
### 3. Trust boundary
The runtime policy must encode the trust boundary:
> **index routes, map orients, source decides**
Required implications:
1. Indexes are navigation aids, not final authority.
2. Maps provide orientation and architectural context, not final authority.
3. Source must remain the final authority before edits or exact behavioral claims.
4. If injected artifacts and source disagree, source wins.
### 4. Budget model
Automatic expansion beyond the root pair must use a hybrid context budget cap.
#### Knobs
The system must support:
- a **relative context-budget percentage**,
- an **optional absolute token cap**.
If both are configured, the smaller effective budget wins.
#### Default budget
The default must be:
- **15%** of the active model context window,
- **100k tokens** absolute cap,
- use the smaller effective budget.
#### Context-window discovery and fallback
- The runtime should derive the active model context window from Pi model metadata when available.
- If the active model context window is unavailable, the runtime must still honor the absolute cap.
- In that fallback case, implementations may skip the relative calculation and use the absolute cap as the effective budget.
#### Expansion rules
- Root pair injection happens before budgeted expansion.
- Additional map/index artifacts are added only while the budget allows.
- The expansion strategy should prefer shallow, high-value structural coverage over deep indiscriminate expansion.
- The spec does not require exact provider-token parity; best-effort budgeting is acceptable if it is deterministic and auditable.
### 5. Reinjection avoidance
The extension must avoid redundant reinjection once the root pair is already in active outgoing context.
#### Required definition
"Already in context" must be defined by scanning the **actual outgoing context**, not only by session guesses.
#### Required behavior
- Before reinjecting, scan per-turn `event.messages` for a stable canonical marker or normalized injected root-pair block.
- If message-layer evidence is insufficient, the runtime may additionally inspect the final provider payload.
- Inject only when the canonical root-pair marker/block is absent.
- The spec must allow implementation via explicit scanning logic even if Pi does not expose a convenience API.
### 6. Visibility model
The system must use mixed visibility.
#### Required behavior
- Lightweight startup/init hints should be user-visible and inspectable.
- Automatic artifact injection may remain agent-visible by default.
- The spec should preserve debuggability: implementations should make the presence of automatic guidance discoverable, even when raw artifact blocks are not fully dumped to the user on every turn.
### 7. Guidance-strength modes
The system must support four named modes:
- `off`
- `advisory`
- `strong`
- `strict`
#### Protocol path definition
For this spec, the **protocol path** is present when the current outgoing context contains:
1. the canonical injected root-pair block, or an equivalent canonical marker proving that root `.pi-map.index.md` and root `.pi-map.md` are already present, and
2. the trust-boundary instruction establishing that `index routes, map orients, source decides`.
If either element is missing for a sensitive action, the protocol path is missing.
#### Mode semantics
- **`off`**
- no automatic injection beyond tool/docs discovery.
- **`advisory`**
- inject startup/init hints,
- allow optional root-pair preload,
- use light reminders.
- **`strong`**
- inject root pair,
- expand under the configured budget,
- run reinjection checks on relevant turns,
- remind before edits or architecture-sensitive reasoning.
- **`strict`**
- same as `strong`, plus:
- require explicit bypass justification before sensitive edits or architectural claims when the protocol path is missing.
#### Default mode
The default mode must be **`strong`**.
### 8. `strong`-mode trigger timing
In `strong` mode, reinjection checks must happen on relevant turns only.
Required triggers:
- agent start,
- before edits,
- before architecture-sensitive reasoning,
- after compaction,
- after root-pair artifact changes.
Required non-trigger:
- do not rescan on every trivial turn.
### 9. Implementation surfaces
The runtime may achieve this behavior through a combination of:
- startup hooks,
- per-turn context hooks,
- provider-payload hooks,
- prompt guidance,
- generated root artifacts.
The spec intentionally does **not** require one exact implementation mechanism, but it does require the observable runtime behavior above.
### 10. Validation requirements
The implementation must be proven with extensive integration coverage.
At minimum, integration coverage must validate:
- pre-init hint behavior,
- post-init root-pair injection,
- budgeted expansion behavior,
- canonical-marker dedupe,
- reinjection after compaction,
- reinjection after root-pair artifact changes,
- mixed visibility behavior,
- guidance-mode differences,
- strict-mode bypass behavior where implemented.
## Non-functional requirements
- Keep the policy explicit and auditable.
- Keep retrieval out of scope for this spec.
- Prefer deterministic behavior over opaque heuristics where possible.
- Optimize for modern large-context models, while retaining a hard ceiling.
- Preserve source as final authority.
## User flows
### Flow 1: New repo, maps not initialized
1. Agent starts in a repo without project-map artifacts.
2. Runtime injects a lightweight visible startup hint.
3. Agent is instructed to run `project_map_init`.
4. No fake artifact content is injected.
### Flow 2: Normal initialized repo in `strong` mode
1. Agent starts.
2. Runtime checks outgoing context for canonical root-pair marker.
3. If absent, inject root pair and budgeted expansion.
4. On relevant later turns, runtime rescans only when trigger conditions apply.
5. Before edits, agent is reminded to use injected context and then verify source.
### Flow 3: Compaction or artifact invalidation
1. Context compacts or root-pair artifacts change.
2. Runtime treats that as a relevant reinjection trigger.
3. Runtime rescans outgoing context.
4. If canonical root-pair block is absent, inject again.
### Flow 4: Strict-mode sensitive action
1. Agent approaches a sensitive edit or architectural claim.
2. Runtime checks whether the protocol path is present.
3. If not, runtime requires explicit bypass justification before proceeding.
## Acceptance criteria
- [ ] Pre-init behavior is hint-only and never injects fake map content
- [ ] Post-init behavior always guarantees root pair preload before budgeted expansion
- [ ] Budgeting supports both relative and absolute caps, with the smaller effective budget winning
- [ ] Default budget is 15% of active context window, capped at 100k tokens, with absolute-cap fallback when context-window metadata is unavailable
- [ ] Reinjection avoidance is based on actual outgoing-context scanning
- [ ] Mixed visibility is honored
- [ ] Four guidance modes exist with `strong` as default
- [ ] `strong` checks fire on relevant turns only
- [ ] `strict` adds bypass-justification semantics for missing protocol path on sensitive actions
- [ ] Extensive integration tests cover context scanning and reinjection behavior
@@ -0,0 +1,7 @@
# Sync Report
Status: NOT-APPLICABLE
Reason: This repository uses legacy flat change artifacts (proposal.md/spec.md/design.md/tasks.md) and does not maintain a canonical openspec/specs/ tree for these changes. No canonical spec sync was performed.
User-approved fallback: archive completed change as an audit record without canonical spec sync.
@@ -0,0 +1,90 @@
# Tasks: Project Map Prompt Injection
## Status
| Field | Value |
|---|---|
| Phase | **Tasks** |
| Based on | [Design](design.md) |
| Next | Apply |
## Delivery slices
### Slice 1: Injection policy scaffold and config surface
**Scope**: mode/config scaffolding, pre-init hint behavior, canonical helper boundaries
**Review goal**: establish the policy control surface without yet wiring the full runtime pipeline
**Tasks**:
1. [ ] Add a shared prompt-injection policy helper/module
2. [ ] Add config support for injection mode, context-budget percent, and absolute cap using additive flat JSON keys compatible with existing `.pi-project-map.json` loading
3. [ ] Decide and document how existing `contextBudget` interacts with the new injection-budget knobs
4. [ ] Implement pre-init startup-hint behavior with no synthetic artifact injection
5. [ ] Define canonical marker/block construction for injected root-pair content
6. [ ] Add/update tests for config loading and pre-init hint behavior
### Slice 2: Root-pair preload and budgeted expansion
**Scope**: post-init preload, budget calculation, artifact selection under cap
**Review goal**: make automatic injection materially real after init
**Tasks**:
1. [ ] Implement guaranteed root-pair preload after init
2. [ ] Implement effective-budget calculation using percent + absolute cap with smaller-wins semantics
3. [ ] Implement active-model context-window discovery and explicit absolute-cap fallback when metadata is unavailable
4. [ ] Implement deterministic budgeted expansion beyond the root pair
5. [ ] Keep retrieval explicitly out of this injection path
6. [ ] Add/update tests for root-pair guarantee, context-window fallback, and budget-capped expansion
### Slice 3: Reinjection avoidance and relevant-turn checks
**Scope**: outgoing-context scanning, relevant-turn triggers, compaction/artifact invalidation
**Review goal**: avoid wasteful reinjection while preserving strong guidance
**Tasks**:
1. [ ] Implement canonical marker scanning over `event.messages`
2. [ ] Add fallback inspection of final provider payload when needed
3. [ ] Implement relevant-turn reinjection triggers for `strong` mode
4. [ ] Reinject after compaction and after root-pair artifact changes
5. [ ] Add/update integration tests for dedupe, reinjection suppression, and reinjection after invalidation
6. [ ] Include synthetic event-sequence coverage for edit-intent, architecture-sensitive reasoning, compaction, and artifact-change heuristics
### Slice 4: Mode semantics, visibility, and strict-path behavior
**Scope**: off/advisory/strong/strict semantics, mixed visibility, strict bypass behavior
**Review goal**: make the mode ladder operational and reviewable
**Tasks**:
1. [ ] Implement mode-specific behavior for `off`, `advisory`, `strong`, and `strict`
2. [ ] Define and enforce the spec meaning of a missing `protocol path` during sensitive actions
3. [ ] Keep startup hints user-visible and artifact injection agent-visible by default
4. [ ] Implement strict-mode explicit bypass-justification behavior for sensitive edits/architectural claims
5. [ ] Add/update integration tests for visibility, protocol-path detection, and mode differences
6. [ ] Verify that `strong` remains the default behavior
### Slice 5: Documentation and runtime alignment
**Scope**: docs, runtime guidance text, implementation notes
**Review goal**: align user-facing/runtime-facing guidance with the frozen spec
**Tasks**:
1. [ ] Update runtime guidance strings to reflect init-hint behavior, root-pair preload, trust boundary, and mode ladder
2. [ ] Update docs/skill guidance for the prompt-injection policy
3. [ ] Keep retrieval guidance separate from these docs or clearly reference it as separate
4. [ ] Document the opinionated default budget and configurability
5. [ ] Summarize integration-test expectations and known risks
## Acceptance checklist
- [x] Before init, only a lightweight `project_map_init` hint is injected
- [x] No synthetic map/index artifact content is injected before init
- [x] After init, root `.pi-map.index.md` and root `.pi-map.md` are always guaranteed before budgeted expansion
- [x] Expansion uses a hybrid cap with both relative and absolute knobs, smaller effective budget wins
- [x] Default budget is 15% of active context window, capped at 100k tokens
- [x] Reinjection avoidance is based on canonical marker scanning in actual outgoing context
- [x] `strong` checks only relevant turns, not every trivial turn
- [x] Modes `off`, `advisory`, `strong`, and `strict` are implemented with the agreed semantics
- [x] Mixed visibility behavior is preserved
- [x] Retrieval behavior remains separate from this specs implementation scope
- [x] Extensive integration tests validate context scanning and reinjection behavior
- [x] `npm run typecheck` passes
- [x] `npm test` passes
- [x] `npm run lint` passes *(N/A: repo has no ESLint config; pre-existing repository gap, not a change regression)*
## Review workload note
This change mixes runtime hook behavior, prompt budgeting, context dedupe, visibility policy, and strict-mode enforcement. Keep delivery narrow and test-heavy. Avoid collapsing this into one oversized implementation slice.
@@ -0,0 +1,22 @@
# Verify Report
Status: PASS
Change: project-map-prompt-injection
Verified summary: Implemented prompt-injection slices 1-5: mode/config surface, root-pair preload, reinjection/dedupe, strict/advisory/strong/off semantics, and documentation/runtime alignment.
Evidence commands:
- npm run typecheck
- npx vitest run
- npm run build
- node dist/cli.js validate .
Current validation state:
- npm run typecheck: PASS
- npx vitest run: PASS (262/262)
- npm run build: PASS
- node dist/cli.js validate .: PASS
Notes:
- Repo-wide lint remains unavailable because the repository has no ESLint config; treated as a pre-existing repository-level gap, not a change regression.
+30
View File
@@ -0,0 +1,30 @@
project:
name: pi-project-map
description: Pi skill and CLI for generating hierarchical .pi-map.md files for AI-oriented codebase comprehension
stack:
runtime:
language: TypeScript
platform: Node.js
cli:
entrypoint: dist/cli.js
extension:
entrypoint: pi-extension.ts
testing:
framework: Vitest
commands:
- npm test
- npm run typecheck
- npm run lint
sdd:
execution_mode: interactive
artifact_store: openspec
chained_pr_strategy: auto-forecast
review_budget_lines: 400
phase_rules:
explore_before_proposal: true
spec_before_design: true
design_before_tasks: true
verify_before_archive: true
+20
View File
@@ -0,0 +1,20 @@
# OpenSpec Project Context: pi-project-map
## Product
`pi-project-map` is a Pi skill plus CLI that generates hierarchical `.pi-map.md` files throughout a repository so coding agents can orient quickly without reading every source file.
## Current architecture
- `src/init.ts`: full project discovery and map generation
- `src/patch.ts`: incremental map updates after file edits
- `src/validate.ts`: map freshness and discrepancy checks
- `src/format.ts`: markdown map rendering/parsing
- `pi-extension.ts`: Pi tool registration and prompt guidance
## Conventions
- `.pi-map.md` files are orientation aids, not runtime truth.
- Exact behavior must still be verified from source before editing or making precise claims.
- Tooling changes should preserve the core `init` / `patch` / `validate` / `reinit` workflow unless a spec explicitly changes it.
- Keep review slices small; prefer staged, reviewable changes over one large refactor.
## Active planning theme
Move from "load all maps up front" toward a layered retrieval model where the root artifacts teach the agent how to navigate, then package maps and source files are loaded on demand or by focused auto-selection.
+555 -555
View File
File diff suppressed because it is too large Load Diff
+301 -20
View File
@@ -7,7 +7,20 @@ import {
patchFile, patchFile,
validateMaps, validateMaps,
reinitPath, reinitPath,
retrieveContext,
buildPreInitHint,
buildAdvisoryReminder,
buildStrictBypassGuard,
modeAllowsPreInitHint,
modeAllowsInjection,
evaluateStrictBypass,
discoverContextWindow,
buildInjectionPayload,
shouldReinjectForEvent,
getRootPairMtimes,
rootPairChanged,
} from "./src/index.js"; } from "./src/index.js";
import { loadConfig } from "./src/config.js";
import { createLLMClient } from "./src/llm/llm-client.js"; import { createLLMClient } from "./src/llm/llm-client.js";
import { LLMError } from "./src/llm/llm-error.js"; import { LLMError } from "./src/llm/llm-error.js";
@@ -55,17 +68,50 @@ function isDirty(content: string): boolean {
return content.includes("## dirty") && !content.includes("## dirty\n-"); return content.includes("## dirty") && !content.includes("## dirty\n-");
} }
function renderProgressBar(
completed: number,
total: number,
currentFile?: string,
width = 20,
): string {
const pct = total > 0 ? completed / total : 0;
const filled = Math.round(width * pct);
const bar = "█".repeat(filled) + "░".repeat(width - filled);
const file = currentFile ? `${currentFile}` : "";
return `[${bar}] ${completed}/${total}${file}`;
}
const HINT_CUSTOM_TYPE = "pi-project-map-hint";
function hintAlreadyInContext(ctx: any): boolean {
const manager = ctx?.sessionManager;
if (!manager || typeof manager.buildSessionContext !== "function") {
return false;
}
const { messages } = manager.buildSessionContext();
if (!Array.isArray(messages)) return false;
return messages.some(
(m: any) =>
m &&
m.role === "custom" &&
m.customType === HINT_CUSTOM_TYPE,
);
}
export default function (pi: ExtensionAPI) { export default function (pi: ExtensionAPI) {
let lastRootPairMtimes: import("./src/index.js").RootPairMtimes = {};
pi.registerTool({ pi.registerTool({
name: "project_map_init", name: "project_map_init",
label: "Project Map Init", label: "Project Map Init",
description: description:
"Generate .pi-map.md analysis files for the entire project or a subdirectory", "Generate paired .pi-map.md and .pi-map.index.md analysis files for the entire project or a subdirectory",
promptSnippet: promptSnippet:
"Initialize project analysis files for codebase understanding", "Initialize paired project map/index artifacts for codebase understanding",
promptGuidelines: [ promptGuidelines: [
"Use project_map_init when starting work on a new project or after significant restructuring", "Use project_map_init when starting work on a new project or after significant restructuring",
"Run project_map_init when .pi-map.md files are missing or severely outdated", "Run project_map_init when .pi-map.md / .pi-map.index.md files are missing or severely outdated",
], ],
parameters: Type.Object({ parameters: Type.Object({
path: Type.Optional( path: Type.Optional(
@@ -82,7 +128,24 @@ export default function (pi: ExtensionAPI) {
verbose: false, verbose: false,
llmClient: client, llmClient: client,
cacheDir: ctx.cwd, cacheDir: ctx.cwd,
onProgress: (msg) => _onUpdate?.({ content: [{ type: "text", text: msg }] }), onProgress: (info) => {
const bar = renderProgressBar(
info.completed,
info.total,
info.currentFile,
);
_onUpdate?.({
content: [{ type: "text", text: bar }],
details: {
progress:
info.total > 0
? Math.round((info.completed / info.total) * 100)
: 0,
file: info.currentFile,
dir: info.dir,
},
});
},
}); });
return { return {
content: [ content: [
@@ -107,8 +170,9 @@ export default function (pi: ExtensionAPI) {
name: "project_map_patch", name: "project_map_patch",
label: "Project Map Patch", label: "Project Map Patch",
description: description:
"Update .pi-map.md for the directory containing a changed file", "Update the paired .pi-map.md / .pi-map.index.md artifacts for the directory containing a changed file",
promptSnippet: "Update project analysis after editing a source file", promptSnippet:
"Update paired project map/index artifacts after editing a source file",
promptGuidelines: [ promptGuidelines: [
"Use project_map_patch immediately after editing any source file", "Use project_map_patch immediately after editing any source file",
"Pass the absolute or relative path of the modified file", "Pass the absolute or relative path of the modified file",
@@ -144,11 +208,13 @@ export default function (pi: ExtensionAPI) {
pi.registerTool({ pi.registerTool({
name: "project_map_validate", name: "project_map_validate",
label: "Project Map Validate", label: "Project Map Validate",
description: "Check all .pi-map.md files for staleness and discrepancies", description:
promptSnippet: "Validate project analysis files for accuracy", "Check all .pi-map.md / .pi-map.index.md files for staleness and discrepancies. Optionally repair them.",
promptSnippet: "Validate paired project map/index artifacts for accuracy",
promptGuidelines: [ promptGuidelines: [
"Use project_map_validate before making architectural decisions if you suspect stale data", "Use project_map_validate before making architectural decisions if you suspect stale data",
"Use project_map_validate to detect files that were deleted or added outside the agent", "Use project_map_validate to detect files that were deleted or added outside the agent",
"Set fix=true to repair localized discrepancies without running a full project_map_reinit",
], ],
parameters: Type.Object({ parameters: Type.Object({
path: Type.Optional( path: Type.Optional(
@@ -156,13 +222,23 @@ export default function (pi: ExtensionAPI) {
description: "Project root path (default: current directory)", description: "Project root path (default: current directory)",
}), }),
), ),
fix: Type.Optional(
Type.Boolean({
description:
"Repair discrepancies automatically (default: false). Requires an LLM client.",
default: false,
}),
),
}), }),
async execute(_toolCallId, params, _signal, _onUpdate, ctx) { async execute(_toolCallId, params, _signal, _onUpdate, ctx) {
try { try {
const targetPath = params.path || ctx.cwd; const targetPath = params.path || ctx.cwd;
const client = params.fix ? getPiLLMClient(ctx) : undefined;
const result = await validateMaps(targetPath, { const result = await validateMaps(targetPath, {
fix: false, fix: params.fix ?? false,
verbose: false, verbose: false,
llmClient: client,
cacheDir: ctx.cwd,
}); });
const text = result.clean const text = result.clean
? "All .pi-map.md files are clean." ? "All .pi-map.md files are clean."
@@ -187,11 +263,15 @@ export default function (pi: ExtensionAPI) {
pi.registerTool({ pi.registerTool({
name: "project_map_reinit", name: "project_map_reinit",
label: "Project Map Reinit", label: "Project Map Reinit",
description: "Force full regeneration of all .pi-map.md files", description:
promptSnippet: "Force full regeneration of project analysis files", "Regenerate .pi-map.md / .pi-map.index.md artifacts for a subtree plus its ancestors, falling back to full regeneration only when the subtree covers more than the configured percentage of project files (default: 10%)",
promptSnippet:
"Regenerate paired project map/index artifacts for a subtree or the whole project",
promptGuidelines: [ promptGuidelines: [
"Use project_map_reinit when validation shows widespread staleness", "Use project_map_reinit only after project_map_patch and project_map_validate --fix cannot resolve the staleness",
"Use project_map_reinit after pulling major changes from version control", "For localized changes, prefer project_map_patch <changed-file> or project_map_validate with fix=true",
"Use project_map_reinit for widespread structural damage (e.g. broken links across many directories) or after large merges",
"When reinit falls back to full regeneration, it is because the target subtree covers more than the configured reinitFullThresholdPercent of project files",
], ],
parameters: Type.Object({ parameters: Type.Object({
path: Type.Optional( path: Type.Optional(
@@ -208,7 +288,24 @@ export default function (pi: ExtensionAPI) {
verbose: false, verbose: false,
llmClient: client, llmClient: client,
cacheDir: ctx.cwd, cacheDir: ctx.cwd,
onProgress: (msg) => _onUpdate?.({ content: [{ type: "text", text: msg }] }), onProgress: (info) => {
const bar = renderProgressBar(
info.completed,
info.total,
info.currentFile,
);
_onUpdate?.({
content: [{ type: "text", text: bar }],
details: {
progress:
info.total > 0
? Math.round((info.completed / info.total) * 100)
: 0,
file: info.currentFile,
dir: info.dir,
},
});
},
}); });
return { return {
content: [ content: [
@@ -229,6 +326,41 @@ export default function (pi: ExtensionAPI) {
}, },
}); });
pi.registerTool({
name: "project_map_context",
label: "Project Map Context",
description:
"Retrieve a compact markdown context bundle for a natural-language query using paired project map/index metadata",
promptSnippet:
"Get relevant project context for a task without reading every source file",
promptGuidelines: [
"Use project_map_context when you need to understand a task area before diving into source",
"Pass a concise query describing the feature, bug, or area you want to explore",
"Always read the suggested indexes first, then maps, then verify from source",
],
parameters: Type.Object({
query: Type.String({
description:
"Natural-language query describing the task or area to explore",
}),
}),
async execute(_toolCallId, params, _signal, _onUpdate, ctx) {
try {
const bundle = retrieveContext(params.query, ctx.cwd);
return {
content: [{ type: "text", text: bundle }],
details: { success: true },
};
} catch (err: any) {
const msg = err instanceof LLMError ? err.message : String(err);
return {
content: [{ type: "text", text: `Error: ${msg}` }],
details: { success: false, error: msg },
};
}
},
});
// Auto-load .pi-map.md files on session start // Auto-load .pi-map.md files on session start
pi.on("session_start", async (_event, ctx) => { pi.on("session_start", async (_event, ctx) => {
const mapFiles = findPiMapFiles(ctx.cwd); const mapFiles = findPiMapFiles(ctx.cwd);
@@ -245,23 +377,172 @@ export default function (pi: ExtensionAPI) {
if (dirtyFiles.length > 0) { if (dirtyFiles.length > 0) {
ctx.ui.notify( ctx.ui.notify(
`pi-project-map: ${dirtyFiles.length} dirty packages detected. Run project_map_validate or project_map_reinit.`, `pi-project-map: ${dirtyFiles.length} dirty package(s) detected. Run project_map_validate first; use project_map_reinit only if staleness is widespread.`,
"warning", "warning",
); );
} }
}); });
// Inject maintenance instructions before agent starts // Inject maintenance instructions before agent starts, but only once
// within the current branch of context. Re-inject after compaction or
// tree navigation removes the hint from the active path.
pi.on("before_agent_start", async (_event, _ctx) => { pi.on("before_agent_start", async (_event, _ctx) => {
const config = loadConfig(_ctx.cwd);
const mapFiles = findPiMapFiles(_ctx.cwd); const mapFiles = findPiMapFiles(_ctx.cwd);
if (mapFiles.length === 0) return {};
// Mode is off: no injection at all
if (config.promptInjectionMode === "off") {
return {};
}
// No maps exist yet: show visible pre-init hint, but only if mode allows it
if (mapFiles.length === 0) {
if (!modeAllowsPreInitHint(config.promptInjectionMode)) {
return {};
}
if (hintAlreadyInContext(_ctx)) return {};
return {
message: {
customType: HINT_CUSTOM_TYPE,
content: buildPreInitHint(),
display: true,
},
};
}
// Slice 4: advisory mode shows a visible lightweight reminder after init.
// No root-pair preload, no per-turn reinjection.
if (config.promptInjectionMode === "advisory") {
if (hintAlreadyInContext(_ctx)) return {};
return {
message: {
customType: HINT_CUSTOM_TYPE,
content: buildAdvisoryReminder(),
display: true,
},
};
}
// Maps exist but mode does not permit automatic artifact injection.
if (!modeAllowsInjection(config.promptInjectionMode)) {
return {};
}
// Slice 3b: detect root-pair artifact changes
const currentMtimes = getRootPairMtimes(_ctx.cwd);
const hasPrevious =
lastRootPairMtimes.mapMtime !== undefined ||
lastRootPairMtimes.indexMtime !== undefined;
const artifactChanged =
hasPrevious && rootPairChanged(currentMtimes, lastRootPairMtimes);
lastRootPairMtimes = currentMtimes;
// Slice 3a/3b: avoid redundant reinjection by scanning outgoing context
const eventType = artifactChanged ? "artifact_change" : "agent_start";
const decision = shouldReinjectForEvent(
{
messages: _event?.messages,
type: eventType,
},
config.promptInjectionMode,
);
if (!decision.needed) {
return {};
}
if (hintAlreadyInContext(_ctx)) return {};
// Slice 2: post-init root-pair preload + budgeted expansion
const contextWindow = discoverContextWindow(_ctx);
const payload = buildInjectionPayload(_ctx.cwd, config, contextWindow);
return {
message: {
customType: HINT_CUSTOM_TYPE,
content: payload.content,
display: payload.display,
},
};
});
// Per-turn context scanning for reinjection in strong/strict modes
pi.on("context", async (event: any, ctx: any) => {
const config = loadConfig(ctx.cwd);
const mapFiles = findPiMapFiles(ctx.cwd);
if (mapFiles.length === 0) {
return {};
}
if (!modeAllowsInjection(config.promptInjectionMode)) {
return {};
}
// Slice 3b: detect root-pair artifact changes early. Invalidation always
// forces reinjection, even in strict mode, because the context is stale.
const currentMtimes = getRootPairMtimes(ctx.cwd);
const hasPrevious =
lastRootPairMtimes.mapMtime !== undefined ||
lastRootPairMtimes.indexMtime !== undefined;
const artifactChanged =
hasPrevious && rootPairChanged(currentMtimes, lastRootPairMtimes);
lastRootPairMtimes = currentMtimes;
if (artifactChanged) {
const decision = shouldReinjectForEvent(
{
messages: event?.messages,
type: "artifact_change",
payload: event?.payload,
},
config.promptInjectionMode,
);
if (!decision.needed) {
return {};
}
const contextWindow = discoverContextWindow(ctx);
const payload = buildInjectionPayload(ctx.cwd, config, contextWindow);
return { return {
message: { message: {
customType: "pi-project-map-hint", customType: "pi-project-map-hint",
content: content: payload.content,
"📋 Project map active: If you modify any source file, run `project_map_patch` with the file path. If you suspect staleness, run `project_map_validate`.", display: payload.display,
display: false, },
};
}
// Slice 4: strict-mode bypass guard for sensitive actions with missing protocol path.
if (config.promptInjectionMode === "strict") {
const bypass = evaluateStrictBypass(event, config.promptInjectionMode);
if (bypass.guard) {
return {
message: {
customType: "pi-project-map-hint",
content: buildStrictBypassGuard(bypass.reason),
display: true,
},
};
}
}
const decision = shouldReinjectForEvent(
{
messages: event?.messages,
type: event?.type,
payload: event?.payload,
},
config.promptInjectionMode,
);
if (!decision.needed) {
return {};
}
const contextWindow = discoverContextWindow(ctx);
const payload = buildInjectionPayload(ctx.cwd, config, contextWindow);
return {
message: {
customType: "pi-project-map-hint",
content: payload.content,
display: payload.display,
}, },
}; };
}); });
+49
View File
@@ -0,0 +1,49 @@
# src (index)
dir: src
## role
A project mapping and codebase navigation system that generates, maintains, and queries AI-readable documentation artifacts for software projects.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
- src/ast
index: src/ast/.pi-map.index.md
map: src/ast/.pi-map.md
- src/cli
index: src/cli/.pi-map.index.md
map: src/cli/.pi-map.md
- src/llm
index: src/llm/.pi-map.index.md
map: src/llm/.pi-map.md
- src/types
index: src/types/.pi-map.index.md
map: src/types/.pi-map.md
## files
- cli.ts
- config.ts
- directory-model.ts
- discover.ts
- format.ts
- index.ts
- init.ts
- merge.ts
- patch.ts
- prompt-injection.ts
- retrieve.ts
- routing-metadata.ts
- validate.ts
## links
index: src/.pi-map.index.md
map: src/.pi-map.md
## workflows
- change src behavior
read: cli.ts, config.ts, directory-model.ts
- change src CLI
read: cli.ts
- change src config
read: config.ts
- explore src subdirectories
index: src/ast/.pi-map.index.md, src/cli/.pi-map.index.md, src/llm/.pi-map.index.md
## dirty
-
+45
View File
File diff suppressed because one or more lines are too long
+20
View File
@@ -0,0 +1,20 @@
# src/ast (index)
dir: src/ast
## role
Extracts structured metadata from source code ASTs across multiple languages to enable code analysis and dependency understanding.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- ast-extract.ts
## links
index: src/ast/.pi-map.index.md
map: src/ast/.pi-map.md
## workflows
- change ast behavior
read: ast-extract.ts
## dirty
-
+27
View File
@@ -0,0 +1,27 @@
# src/ast
dir: src/ast
index: src/ast/.pi-map.index.md
## role
Extracts structured metadata from source code ASTs across multiple languages to enable code analysis and dependency understanding.
## files
- ast-extract.ts | Extracts AST-based metadata (exports, dependencies, classes, functions, method calls, and exceptions) from source code files across multiple languages using Tree-sitter parsers. | exp: ASTFileData, func:extractFileAST(filePath: string) → Promise<ASTFileData | null>, call:extname(filePath).toLowerCase, call:require, call:parser.setLanguage, call:readFileSync, call:parser.parse, call:extractPythonData, call:extractTypeScriptData, call:extractGoData, call:extractExportsFromTree, call:extractDepsFromTree | dep: fs, path, tree-sitter, tree-sitter-typescript, tree-sitter-python, tree-sitter-go, tree-sitter-rust, tree-sitter-java, tree-sitter-c, tree-sitter-cpp, tree-sitter-ruby
## arch
Language-agnostic parser abstraction using Tree-sitter grammars with unified extraction pipeline for cross-language code analysis.
## tags
tree, sitter, call:extract, data, ast, extract, python, go
## symbols
- extractFileAST
- ASTFileData
- call:extname(filePath).toLowerCase
- call:require
- call:parser.setLanguage
- call:readFileSync
- call:parser.parse
- call:extractPythonData
## workflows
- change ast behavior
read: ast-extract.ts
## dirty
-
+1 -1
View File
@@ -264,7 +264,7 @@ function extractCallChain(node: SyntaxNode): string | null {
node.type === "property_identifier" || node.type === "property_identifier" ||
node.type === "type_identifier" node.type === "type_identifier"
) { ) {
return node.text; return node.text.replace(/\s+/g, " ").trim();
} }
if (node.type === "call" || node.type === "call_expression") { if (node.type === "call" || node.type === "call_expression") {
const func = node.childForFieldName?.("function"); const func = node.childForFieldName?.("function");
+2
View File
@@ -0,0 +1,2 @@
#!/usr/bin/env node
import "./cli/cli.js";
+22
View File
@@ -0,0 +1,22 @@
# src/cli (index)
dir: src/cli
## role
Command-line interface entry point for a project mapping tool that manages hierarchical `.pi-map.md` files through generation, patching, validation, and LLM-powered context retrieval.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- cli.ts
## links
index: src/cli/.pi-map.index.md
map: src/cli/.pi-map.md
## workflows
- change cli behavior
read: cli.ts
- change cli CLI
read: cli.ts
## dirty
-
+22
View File
@@ -0,0 +1,22 @@
# src/cli
dir: src/cli
index: src/cli/.pi-map.index.md
## role
Command-line interface entry point for a project mapping tool that manages hierarchical `.pi-map.md` files through generation, patching, validation, and LLM-powered context retrieval.
## files
- cli.ts | CLI entry point for a project mapping tool that generates, patches, validates, and retrieves context from hierarchical `.pi-map.md` files using LLM-powered analysis. | dep: ../init.js, ../patch.js, ../validate.js, ../discover.js, ../retrieve.js, ../llm/llm-client.js, ../config.js, picocolors, process, fs (implied via require)
## arch
Single-file CLI facade with command routing to core engine services, likely using a command pattern or direct service delegation for map lifecycle operations.
## tags
js, cli, llm, entry, point, project, mapping, tool
## symbols
-
## workflows
- change cli behavior
read: cli.ts
- change cli CLI
read: cli.ts
## dirty
-
+95 -11
View File
@@ -4,6 +4,7 @@ import { patchFile } from "../patch.js";
import { validateMaps } from "../validate.js"; import { validateMaps } from "../validate.js";
import { reinitPath } from "../init.js"; import { reinitPath } from "../init.js";
import { discoverProject } from "../discover.js"; import { discoverProject } from "../discover.js";
import { retrieveContext } from "../retrieve.js";
import { createLLMClient, LLMError } from "../llm/llm-client.js"; import { createLLMClient, LLMError } from "../llm/llm-client.js";
import { loadConfig } from "../config.js"; import { loadConfig } from "../config.js";
import pc from "picocolors"; import pc from "picocolors";
@@ -11,6 +12,8 @@ import pc from "picocolors";
const args = process.argv.slice(2); const args = process.argv.slice(2);
const command = args[0]; const command = args[0];
type PatchMode = "auto" | "small" | "structural";
function printUsage() { function printUsage() {
console.log(`${pc.bold("project-map")} — hierarchical project analysis for Pi agents console.log(`${pc.bold("project-map")} — hierarchical project analysis for Pi agents
`); `);
@@ -25,11 +28,14 @@ function printUsage() {
` project-map ${pc.cyan("validate")} [--fix] [path] Check for stale/missing/orphaned entries`, ` project-map ${pc.cyan("validate")} [--fix] [path] Check for stale/missing/orphaned entries`,
); );
console.log( console.log(
` project-map ${pc.cyan("reinit")} [path] Force full regeneration`, ` project-map ${pc.cyan("reinit")} [path] Regenerate a subtree or the whole project`,
); );
console.log( console.log(
` project-map ${pc.cyan("--help")} Show this help message`, ` project-map ${pc.cyan("--help")} Show this help message`,
); );
console.log(
` project-map ${pc.cyan("context")} <query> Retrieve relevant project context for a query`,
);
console.log( console.log(
` project-map ${pc.cyan("--version")} Show version\n`, ` project-map ${pc.cyan("--version")} Show version\n`,
); );
@@ -40,19 +46,21 @@ function printUsage() {
console.log( console.log(
` --llm-model=<model> LLM model name (or set LLM_MODEL env var)`, ` --llm-model=<model> LLM model name (or set LLM_MODEL env var)`,
); );
console.log(` --llm-base-url=<url> Custom base URL for LLM API\n`); console.log(` --llm-base-url=<url> Custom base URL for LLM API`);
console.log(
` --patch-mode=<mode> Patch/repair mode: auto|small|structural\n`,
);
console.log(`${pc.bold("Examples:")}`); console.log(`${pc.bold("Examples:")}`);
console.log(` project-map init`); console.log(` project-map init`);
console.log(` project-map patch src/components/Button.tsx`); console.log(` project-map patch src/components/Button.tsx`);
console.log(` project-map validate --fix`); console.log(` project-map validate --fix`);
console.log(` project-map reinit`); console.log(` project-map reinit`);
console.log( console.log(` project-map context "authentication logic"`);
` project-map init --llm-provider=kimi --llm-model=kimi-k2-6`, console.log(` project-map init --llm-provider=kimi --llm-model=kimi-k2-6`);
);
} }
function printVersion() { function printVersion() {
const pkg = require("../package.json"); const pkg = require("../../package.json");
console.log(pkg.version); console.log(pkg.version);
} }
@@ -63,12 +71,26 @@ function formatCount(count: number, label: string): string {
return `${pc.bold(String(count))} ${count === 1 ? label : plural}`; return `${pc.bold(String(count))} ${count === 1 ? label : plural}`;
} }
function renderProgressBar(
completed: number,
total: number,
currentFile?: string,
width = 30,
): string {
const pct = total > 0 ? completed / total : 0;
const filled = Math.round(width * pct);
const bar = "█".repeat(filled) + "░".repeat(width - filled);
const file = currentFile ? ` | ${pc.dim(currentFile)}` : "";
return `[${pc.cyan(bar)}] ${completed}/${total}${file}`;
}
function parseArgs(args: string[]): { function parseArgs(args: string[]): {
path: string; path: string;
fix: boolean; fix: boolean;
llmProvider?: string; llmProvider?: string;
llmModel?: string; llmModel?: string;
llmBaseUrl?: string; llmBaseUrl?: string;
patchMode?: PatchMode;
positional: string[]; positional: string[];
} { } {
let path = "."; let path = ".";
@@ -76,6 +98,7 @@ function parseArgs(args: string[]): {
let llmProvider: string | undefined; let llmProvider: string | undefined;
let llmModel: string | undefined; let llmModel: string | undefined;
let llmBaseUrl: string | undefined; let llmBaseUrl: string | undefined;
let patchMode: PatchMode | undefined;
const positional: string[] = []; const positional: string[] = [];
for (const arg of args.slice(1)) { for (const arg of args.slice(1)) {
@@ -87,13 +110,23 @@ function parseArgs(args: string[]): {
llmModel = arg.slice("--llm-model=".length); llmModel = arg.slice("--llm-model=".length);
} else if (arg.startsWith("--llm-base-url=")) { } else if (arg.startsWith("--llm-base-url=")) {
llmBaseUrl = arg.slice("--llm-base-url=".length); llmBaseUrl = arg.slice("--llm-base-url=".length);
} else if (arg.startsWith("--patch-mode=")) {
patchMode = arg.slice("--patch-mode=".length) as PatchMode;
} else if (!arg.startsWith("-")) { } else if (!arg.startsWith("-")) {
positional.push(arg); positional.push(arg);
path = arg; path = arg;
} }
} }
return { path, fix, llmProvider, llmModel, llmBaseUrl, positional }; return {
path,
fix,
llmProvider,
llmModel,
llmBaseUrl,
patchMode,
positional,
};
} }
function createClientFromArgs(args: ReturnType<typeof parseArgs>) { function createClientFromArgs(args: ReturnType<typeof parseArgs>) {
@@ -128,7 +161,22 @@ async function main() {
const entries = discoverProject(targetPath); const entries = discoverProject(targetPath);
console.log(`Scanning ${formatCount(entries.length, "directory")}...`); console.log(`Scanning ${formatCount(entries.length, "directory")}...`);
const client = createClientFromArgs(parsed); const client = createClientFromArgs(parsed);
await initProject(targetPath, { verbose: false, llmClient: client, cacheDir: targetPath }); let lastLine = "";
await initProject(targetPath, {
verbose: false,
llmClient: client,
cacheDir: targetPath,
onProgress: (info) => {
const line = renderProgressBar(
info.completed,
info.total,
info.currentFile,
);
process.stdout.write(`\r${line.padEnd(lastLine.length)}`);
lastLine = line;
},
});
process.stdout.write("\n");
const elapsed = ((Date.now() - start) / 1000).toFixed(1); const elapsed = ((Date.now() - start) / 1000).toFixed(1);
console.log( console.log(
`${pc.green("✓")} Generated ${formatCount(entries.length, ".pi-map.md file")} in ${elapsed}s`, `${pc.green("✓")} Generated ${formatCount(entries.length, ".pi-map.md file")} in ${elapsed}s`,
@@ -143,13 +191,22 @@ async function main() {
process.exit(1); process.exit(1);
} }
const client = createClientFromArgs(parsed); const client = createClientFromArgs(parsed);
await patchFile(parsed.positional[0], client, process.cwd()); await patchFile(parsed.positional[0], client, process.cwd(), {
patchMode: parsed.patchMode,
rootPath: process.cwd(),
});
console.log(`${pc.green("✓")} Patched`); console.log(`${pc.green("✓")} Patched`);
break; break;
} }
case "validate": { case "validate": {
const { path, fix } = parsed; const { path, fix } = parsed;
const result = await validateMaps(path, { fix, verbose: true }); const result = await validateMaps(path, {
fix,
verbose: true,
llmClient: fix ? createClientFromArgs(parsed) : undefined,
cacheDir: process.cwd(),
patchMode: parsed.patchMode,
});
if (result.clean) { if (result.clean) {
console.log(`${pc.green("✓")} All .pi-map.md files are clean.`); console.log(`${pc.green("✓")} All .pi-map.md files are clean.`);
} else { } else {
@@ -171,6 +228,18 @@ async function main() {
process.exit(result.clean ? 0 : 1); process.exit(result.clean ? 0 : 1);
break; break;
} }
case "context": {
if (!parsed.positional[0]) {
console.error(
`${pc.red("Error:")} Missing query. Usage: project-map context <query>`,
);
process.exit(1);
}
const query = parsed.positional[0];
const bundle = retrieveContext(query, process.cwd());
console.log(bundle);
break;
}
case "reinit": { case "reinit": {
const targetPath = parsed.positional[0] || "."; const targetPath = parsed.positional[0] || ".";
const start = Date.now(); const start = Date.now();
@@ -179,7 +248,22 @@ async function main() {
`Regenerating ${formatCount(entries.length, ".pi-map.md file")}...`, `Regenerating ${formatCount(entries.length, ".pi-map.md file")}...`,
); );
const client = createClientFromArgs(parsed); const client = createClientFromArgs(parsed);
await reinitPath(targetPath, { verbose: false, llmClient: client, cacheDir: targetPath }); let lastLine = "";
await reinitPath(targetPath, {
verbose: false,
llmClient: client,
cacheDir: process.cwd(),
onProgress: (info) => {
const line = renderProgressBar(
info.completed,
info.total,
info.currentFile,
);
process.stdout.write(`\r${line.padEnd(lastLine.length)}`);
lastLine = line;
},
});
process.stdout.write("\n");
const elapsed = ((Date.now() - start) / 1000).toFixed(1); const elapsed = ((Date.now() - start) / 1000).toFixed(1);
console.log(`${pc.green("✓")} Regenerated in ${elapsed}s`); console.log(`${pc.green("✓")} Regenerated in ${elapsed}s`);
break; break;
+16
View File
@@ -1,6 +1,8 @@
import { existsSync, readFileSync } from "fs"; import { existsSync, readFileSync } from "fs";
import { join } from "path"; import { join } from "path";
export type PromptInjectionMode = "off" | "advisory" | "strong" | "strict";
export interface SkillConfig { export interface SkillConfig {
ignorePatterns: string[]; ignorePatterns: string[];
smallPackageThreshold: number; smallPackageThreshold: number;
@@ -9,6 +11,13 @@ export interface SkillConfig {
llmBaseUrl?: string; llmBaseUrl?: string;
contextBudget: number; contextBudget: number;
autoInjectPrompt: boolean; autoInjectPrompt: boolean;
tagCap: number;
workflowHintCap: number;
promptInjectionMode: PromptInjectionMode;
contextBudgetPercent: number;
contextBudgetMaxTokens: number;
/** Percentage of project files a target subtree must cover before reinit falls back to full regeneration (default: 10). */
reinitFullThresholdPercent: number;
} }
export const DEFAULT_CONFIG: SkillConfig = { export const DEFAULT_CONFIG: SkillConfig = {
@@ -24,6 +33,7 @@ export const DEFAULT_CONFIG: SkillConfig = {
".DS_Store", ".DS_Store",
"*.log", "*.log",
".pi-map.md", ".pi-map.md",
".pi-map.index.md",
".cache", ".cache",
"tmp", "tmp",
"temp", "temp",
@@ -38,6 +48,12 @@ export const DEFAULT_CONFIG: SkillConfig = {
llmModel: "gpt-4o-mini", llmModel: "gpt-4o-mini",
contextBudget: 4000, contextBudget: 4000,
autoInjectPrompt: true, autoInjectPrompt: true,
tagCap: 8,
workflowHintCap: 5,
promptInjectionMode: "strong",
contextBudgetPercent: 15,
contextBudgetMaxTokens: 100_000,
reinitFullThresholdPercent: 10,
}; };
export function loadConfig(cwd: string = process.cwd()): SkillConfig { export function loadConfig(cwd: string = process.cwd()): SkillConfig {
+74
View File
@@ -0,0 +1,74 @@
export interface FileEntry {
name: string;
purpose: string;
exports: string[];
deps: string[];
}
export interface DirectoryArtifactModel {
dir: string;
role: string;
files: FileEntry[];
arch: string;
dirty?: string;
isRoot: boolean;
parent?: string;
children: string[];
tags: string[];
symbols: string[];
workflows: WorkflowHint[];
}
export interface WorkflowHint {
task: string;
read?: string[];
index?: string[];
map?: string[];
files?: string[];
}
export const PROJECT_MAP_PROTOCOL_LINES = [
"## Project Map Protocol",
"",
"1. Read this protocol and the root `.pi-map.index.md` first.",
"2. Use `index:` / `map:` references to open relevant directory indexes and maps.",
"3. Load indexes before rich maps during task-start navigation.",
"4. Read the local rich map and actual source before editing.",
"5. Treat non-empty `## dirty` sections in either artifact as stale.",
"6. If source and generated artifacts disagree, trust source.",
"7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.",
"8. After editing source, run `project_map_patch` for each changed file.",
"9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.",
"",
"Trust boundary: index routes, map orients, source decides.",
];
export interface RoutingMetadataOptions {
tagCap?: number;
workflowHintCap?: number;
}
export function createDirectoryModel(opts: {
dir: string;
role: string;
files: FileEntry[];
arch: string;
parent?: string;
children?: string[];
isRoot?: boolean;
dirty?: string;
}): DirectoryArtifactModel {
return {
dir: opts.dir,
role: opts.role,
files: opts.files,
arch: opts.arch,
parent: opts.parent,
children: opts.children ?? [],
isRoot: opts.isRoot ?? false,
dirty: opts.dirty,
tags: [],
symbols: [],
workflows: [],
};
}
+1
View File
@@ -14,6 +14,7 @@ const DEFAULT_IGNORE = [
".DS_Store", ".DS_Store",
"*.log", "*.log",
".pi-map.md", ".pi-map.md",
".pi-map.index.md",
".cache", ".cache",
"tmp", "tmp",
"temp", "temp",
+531 -31
View File
@@ -1,3 +1,6 @@
import type { DirectoryArtifactModel, FileEntry } from "./directory-model.js";
import { PROJECT_MAP_PROTOCOL_LINES } from "./directory-model.js";
export interface PackageMapData { export interface PackageMapData {
path: string; path: string;
role: string; role: string;
@@ -6,20 +9,64 @@ export interface PackageMapData {
dirty?: string; dirty?: string;
} }
export interface FileEntry { export function renderPackageMap(data: PackageMapData): string {
name: string; const model = convertPackageMapToModel(data);
purpose: string; return renderDirectoryMap(model);
exports: string[];
deps: string[];
} }
export function renderPackageMap(data: PackageMapData): string { export function parsePackageMap(markdown: string): PackageMapData {
const model = parseDirectoryMap(markdown);
return convertModelToPackageMap(model);
}
function convertPackageMapToModel(
data: PackageMapData,
): DirectoryArtifactModel {
return {
dir: data.path,
role: data.role,
files: data.files,
arch: data.arch,
dirty: data.dirty,
isRoot: data.path === ".",
parent: undefined,
children: [],
tags: [],
symbols: [],
workflows: [],
};
}
function convertModelToPackageMap(
model: DirectoryArtifactModel,
): PackageMapData {
return {
path: model.dir,
role: model.role,
files: model.files,
arch: model.arch,
dirty: model.dirty,
};
}
export function renderDirectoryMap(model: DirectoryArtifactModel): string {
const lines: string[] = []; const lines: string[] = [];
lines.push(`# ${data.path}`); lines.push(`# ${model.dir}`);
lines.push(`dir: ${model.dir}`);
lines.push("");
lines.push(`index: ${model.dir}/.pi-map.index.md`);
lines.push("");
if (model.isRoot) {
lines.push(...PROJECT_MAP_PROTOCOL_LINES);
lines.push("");
}
lines.push(`## role`); lines.push(`## role`);
lines.push(data.role); lines.push(model.role);
lines.push(`## files`); lines.push(`## files`);
for (const file of data.files) { for (const file of model.files) {
const exp = const exp =
file.exports.length > 0 ? `exp: ${file.exports.join(", ")}` : ""; file.exports.length > 0 ? `exp: ${file.exports.join(", ")}` : "";
const dep = file.deps.length > 0 ? `dep: ${file.deps.join(", ")}` : ""; const dep = file.deps.length > 0 ? `dep: ${file.deps.join(", ")}` : "";
@@ -28,28 +75,88 @@ export function renderPackageMap(data: PackageMapData): string {
if (dep) parts.push(dep); if (dep) parts.push(dep);
lines.push(parts.join(" | ")); lines.push(parts.join(" | "));
} }
lines.push(`## arch`); lines.push(`## arch`);
lines.push(data.arch); lines.push(model.arch);
// Slice 2: tags, symbols, workflows — scaffold empty sections for now
lines.push(`## tags`);
if (model.tags.length > 0) {
lines.push(model.tags.join(", "));
} else {
lines.push("-");
}
lines.push(`## symbols`);
if (model.symbols.length > 0) {
for (const sym of model.symbols) {
lines.push(`- ${sym}`);
}
} else {
lines.push("-");
}
lines.push(`## workflows`);
if (model.workflows.length > 0) {
for (const wf of model.workflows) {
lines.push(`- ${wf.task}`);
if (wf.read && wf.read.length > 0) {
lines.push(` read: ${wf.read.join(", ")}`);
}
if (wf.index && wf.index.length > 0) {
lines.push(` index: ${wf.index.join(", ")}`);
}
if (wf.map && wf.map.length > 0) {
lines.push(` map: ${wf.map.join(", ")}`);
}
if (wf.files && wf.files.length > 0) {
lines.push(` files: ${wf.files.join(", ")}`);
}
}
} else {
lines.push("-");
}
lines.push(`## dirty`); lines.push(`## dirty`);
lines.push(data.dirty || "-"); lines.push(model.dirty || "-");
return `${lines.join("\n")}\n`; return `${lines.join("\n")}\n`;
} }
export function parsePackageMap(markdown: string): PackageMapData { export function parseDirectoryMap(markdown: string): DirectoryArtifactModel {
const lines = markdown.split("\n").map((l) => l.trimEnd()); const lines = markdown.split("\n").map((l) => l.trimEnd());
const result: PackageMapData = { const result: DirectoryArtifactModel = {
path: "", dir: "",
role: "", role: "",
files: [], files: [],
arch: "", arch: "",
dirty: "-", dirty: "-",
isRoot: false,
parent: undefined,
children: [],
tags: [],
symbols: [],
workflows: [],
}; };
let section: "none" | "role" | "files" | "arch" | "dirty" = "none"; let section:
| "none"
| "role"
| "files"
| "arch"
| "tags"
| "symbols"
| "workflows"
| "dirty" = "none";
for (const line of lines) { for (const line of lines) {
if (line.startsWith("# ")) { if (line.startsWith("# ")) {
result.path = line.slice(2).trim(); result.dir = line.slice(2).trim();
result.isRoot = result.dir === ".";
continue;
}
if (line.startsWith("dir: ")) {
result.dir = line.slice(5).trim();
result.isRoot = result.dir === ".";
continue; continue;
} }
if (line === "## role") { if (line === "## role") {
@@ -64,6 +171,18 @@ export function parsePackageMap(markdown: string): PackageMapData {
section = "arch"; section = "arch";
continue; continue;
} }
if (line === "## tags") {
section = "tags";
continue;
}
if (line === "## symbols") {
section = "symbols";
continue;
}
if (line === "## workflows") {
section = "workflows";
continue;
}
if (line === "## dirty") { if (line === "## dirty") {
section = "dirty"; section = "dirty";
continue; continue;
@@ -83,6 +202,70 @@ export function parsePackageMap(markdown: string): PackageMapData {
case "arch": case "arch":
result.arch = result.arch ? `${result.arch}\n${line}` : line; result.arch = result.arch ? `${result.arch}\n${line}` : line;
break; break;
case "tags":
if (line !== "-") {
result.tags.push(
...line
.split(",")
.map((s) => s.trim())
.filter(Boolean),
);
}
break;
case "symbols":
if (line !== "-" && line.startsWith("- ")) {
result.symbols.push(line.slice(2).trim());
}
break;
case "workflows": {
if (line !== "-" && line.startsWith("- ")) {
const task = line.slice(2).trim();
result.workflows.push({ task });
} else if (line.startsWith(" read: ") && result.workflows.length > 0) {
const reads = line
.slice(8)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.read = reads;
} else if (
line.startsWith(" index: ") &&
result.workflows.length > 0
) {
const indices = line
.slice(9)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.index = indices;
} else if (line.startsWith(" map: ") && result.workflows.length > 0) {
const maps = line
.slice(7)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.map = maps;
} else if (
line.startsWith(" files: ") &&
result.workflows.length > 0
) {
const files = line
.slice(9)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.files = files;
}
break;
}
case "dirty": case "dirty":
result.dirty = line === "-" ? undefined : line; result.dirty = line === "-" ? undefined : line;
break; break;
@@ -92,10 +275,115 @@ export function parsePackageMap(markdown: string): PackageMapData {
return result; return result;
} }
function splitTopLevel(str: string, delimiter: string): string[] {
const result: string[] = [];
let current = "";
let depthParen = 0;
let depthBracket = 0;
let depthBrace = 0;
let depthAngle = 0;
let inString: '"' | "'" | null = null;
for (let i = 0; i < str.length; i++) {
const ch = str[i];
const prev = str[i - 1];
const next = str[i + 1];
if (inString) {
if (ch === inString && prev !== "\\") {
inString = null;
}
current += ch;
continue;
}
if (ch === '"' || ch === "'") {
const isWordApostrophe =
ch === "'" &&
/[A-Za-z0-9_$]/.test(prev ?? "") &&
/[A-Za-z0-9_$]/.test(next ?? "");
if (isWordApostrophe) {
current += ch;
continue;
}
inString = ch;
current += ch;
continue;
}
if (ch === "(") depthParen++;
if (ch === ")") depthParen = Math.max(0, depthParen - 1);
if (ch === "[") depthBracket++;
if (ch === "]") depthBracket = Math.max(0, depthBracket - 1);
if (ch === "{") depthBrace++;
if (ch === "}") depthBrace = Math.max(0, depthBrace - 1);
if (ch === "<") depthAngle++;
if (ch === ">") depthAngle = Math.max(0, depthAngle - 1);
if (
delimiter.length === 1 &&
ch === delimiter &&
depthParen === 0 &&
depthBracket === 0 &&
depthBrace === 0 &&
depthAngle === 0
) {
result.push(current.trim());
current = "";
continue;
}
if (
delimiter.length > 1 &&
str.startsWith(delimiter, i) &&
depthParen === 0 &&
depthBracket === 0 &&
depthBrace === 0 &&
depthAngle === 0
) {
result.push(current.trim());
current = "";
i += delimiter.length - 1;
continue;
}
current += ch;
}
if (current.trim()) result.push(current.trim());
return result;
}
function splitRespectingNesting(str: string): string[] {
return splitTopLevel(str, ",");
}
function splitFieldsRespectingNesting(str: string): string[] {
const rawParts = splitTopLevel(str, " | ");
const fields: string[] = [];
for (const part of rawParts) {
const trimmed = part.trim();
if (
fields.length < 2 ||
trimmed.startsWith("exp: ") ||
trimmed.startsWith("dep: ")
) {
fields.push(trimmed);
} else if (fields.length > 0) {
fields[fields.length - 1] = `${fields[fields.length - 1]} | ${trimmed}`;
}
}
return fields;
}
function parseFileLine(line: string): FileEntry | null { function parseFileLine(line: string): FileEntry | null {
// Format: - filename | purpose | exp: ... | dep: ... // Format: - filename | purpose | exp: ... | dep: ...
const withoutPrefix = line.slice(2).trim(); const withoutPrefix = line.slice(2).trim();
const parts = withoutPrefix.split(" | ").map((p) => p.trim()); const parts = splitFieldsRespectingNesting(withoutPrefix).map((p) =>
p.trim(),
);
if (parts.length < 2) return null; if (parts.length < 2) return null;
@@ -107,23 +395,235 @@ function parseFileLine(line: string): FileEntry | null {
for (let i = 2; i < parts.length; i++) { for (let i = 2; i < parts.length; i++) {
const part = parts[i]; const part = parts[i];
if (part.startsWith("exp: ")) { if (part.startsWith("exp: ")) {
exports.push( exports.push(...splitRespectingNesting(part.slice(5)).filter(Boolean));
...part
.slice(5)
.split(",")
.map((s) => s.trim())
.filter(Boolean),
);
} else if (part.startsWith("dep: ")) { } else if (part.startsWith("dep: ")) {
deps.push( deps.push(...splitRespectingNesting(part.slice(5)).filter(Boolean));
...part
.slice(5)
.split(",")
.map((s) => s.trim())
.filter(Boolean),
);
} }
} }
return { name, purpose, exports, deps }; return { name, purpose, exports, deps };
} }
export function renderDirectoryIndex(model: DirectoryArtifactModel): string {
const lines: string[] = [];
lines.push(`# ${model.dir} (index)`);
lines.push(`dir: ${model.dir}`);
lines.push("");
if (model.isRoot) {
lines.push(...PROJECT_MAP_PROTOCOL_LINES);
lines.push("");
}
lines.push(`## role`);
lines.push(model.role);
lines.push(`## parent`);
if (model.parent) {
lines.push(`index: ${model.parent}/.pi-map.index.md`);
lines.push(`map: ${model.parent}/.pi-map.md`);
} else {
lines.push("-");
}
lines.push(`## children`);
if (model.children.length > 0) {
for (const child of model.children) {
lines.push(`- ${child}`);
lines.push(` index: ${child}/.pi-map.index.md`);
lines.push(` map: ${child}/.pi-map.md`);
}
} else {
lines.push("-");
}
lines.push(`## files`);
for (const file of model.files) {
lines.push(`- ${file.name}`);
}
lines.push(`## links`);
lines.push(`index: ${model.dir}/.pi-map.index.md`);
lines.push(`map: ${model.dir}/.pi-map.md`);
lines.push(`## workflows`);
if (model.workflows.length > 0) {
for (const wf of model.workflows) {
lines.push(`- ${wf.task}`);
if (wf.read && wf.read.length > 0) {
lines.push(` read: ${wf.read.join(", ")}`);
}
if (wf.index && wf.index.length > 0) {
lines.push(` index: ${wf.index.join(", ")}`);
}
if (wf.map && wf.map.length > 0) {
lines.push(` map: ${wf.map.join(", ")}`);
}
if (wf.files && wf.files.length > 0) {
lines.push(` files: ${wf.files.join(", ")}`);
}
}
} else {
lines.push("-");
}
lines.push(`## dirty`);
lines.push(model.dirty || "-");
return `${lines.join("\n")}\n`;
}
export function parseDirectoryIndex(markdown: string): DirectoryArtifactModel {
const lines = markdown.split("\n").map((l) => l.trimEnd());
const result: DirectoryArtifactModel = {
dir: "",
role: "",
files: [],
arch: "",
dirty: "-",
isRoot: false,
parent: undefined,
children: [],
tags: [],
symbols: [],
workflows: [],
};
let section:
| "none"
| "role"
| "parent"
| "children"
| "files"
| "links"
| "workflows"
| "dirty" = "none";
for (const line of lines) {
if (line.startsWith("# ")) {
const title = line.slice(2).trim();
// Strip " (index)" suffix if present
result.dir = title.replace(/ \(index\)$/, "");
result.isRoot = result.dir === ".";
continue;
}
if (line.startsWith("dir: ")) {
result.dir = line.slice(5).trim();
result.isRoot = result.dir === ".";
continue;
}
if (line === "## role") {
section = "role";
continue;
}
if (line === "## parent") {
section = "parent";
continue;
}
if (line === "## children") {
section = "children";
continue;
}
if (line === "## files") {
section = "files";
continue;
}
if (line === "## links") {
section = "links";
continue;
}
if (line === "## workflows") {
section = "workflows";
continue;
}
if (line === "## dirty") {
section = "dirty";
continue;
}
if (line === "") continue;
switch (section) {
case "role":
result.role = line;
break;
case "parent":
if (line !== "-" && line.startsWith("index: ")) {
result.parent = line.slice(7).trim().replace("/.pi-map.index.md", "");
}
break;
case "children":
if (line !== "-" && line.startsWith("- ")) {
const childName = line.slice(2).trim();
result.children.push(childName);
}
break;
case "files":
if (line !== "-" && line.startsWith("- ")) {
result.files.push({
name: line.slice(2).trim(),
purpose: "",
exports: [],
deps: [],
});
}
break;
case "links":
// Parse sibling links; no-op for now
break;
case "workflows": {
if (line !== "-" && line.startsWith("- ")) {
const task = line.slice(2).trim();
result.workflows.push({ task });
} else if (line.startsWith(" read: ") && result.workflows.length > 0) {
const reads = line
.slice(8)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.read = reads;
} else if (
line.startsWith(" index: ") &&
result.workflows.length > 0
) {
const indices = line
.slice(9)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.index = indices;
} else if (line.startsWith(" map: ") && result.workflows.length > 0) {
const maps = line
.slice(7)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.map = maps;
} else if (
line.startsWith(" files: ") &&
result.workflows.length > 0
) {
const files = line
.slice(9)
.trim()
.split(",")
.map((s) => s.trim())
.filter(Boolean);
const last = result.workflows[result.workflows.length - 1];
if (last) last.files = files;
}
break;
}
case "dirty":
result.dirty = line === "-" ? undefined : line;
break;
}
}
return result;
}
+60 -4
View File
@@ -1,5 +1,61 @@
// Main entry point for pi-project-map skill // Main entry point for pi-project-map skill
export { initProject, reinitPath } from "./init.js"; export { initProject, reinitPath, generateDirectoryArtifacts } from "./init.js";
export { patchFile } from "./patch.js"; export { patchFile, type PatchMode, type PatchOptions } from "./patch.js";
export { validateMaps } from "./validate.js"; export {
export { renderPackageMap, parsePackageMap } from "./format.js"; validateMaps,
type ValidationOptions,
type ValidationResult,
type Discrepancy,
} from "./validate.js";
export {
renderPackageMap,
parsePackageMap,
renderDirectoryMap,
parseDirectoryMap,
renderDirectoryIndex,
parseDirectoryIndex,
} from "./format.js";
export type { PackageMapData } from "./format.js";
export {
createDirectoryModel,
type DirectoryArtifactModel,
type FileEntry,
} from "./directory-model.js";
export { populateRoutingMetadata } from "./routing-metadata.js";
export { retrieveContext } from "./retrieve.js";
export {
buildRootPairBlock,
hasRootPairMarker,
buildPreInitHint,
computeInjectionBudget,
modeAllowsPreInitHint,
modeAllowsInjection,
modeRequiresProtocolPath,
hasProtocolPath,
isSensitiveAction,
messagesHaveBypass,
extractBypassReason,
evaluateStrictBypass,
buildAdvisoryReminder,
buildStrictBypassGuard,
discoverContextWindow,
estimateTokens,
findAllArtifactPairs,
buildInjectionPayload,
outgoingMessagesHaveMarker,
providerPayloadHasMarker,
isRelevantTurnForReinjection,
shouldReinjectForEvent,
detectEditIntent,
detectArchitectureSensitiveReasoning,
getRootPairMtimes,
rootPairChanged,
type RootPairMtimes,
type RelevantTurnType,
type ReinjectDecision,
type StrictBypassDecision,
ROOT_PAIR_START_MARKER,
ROOT_PAIR_END_MARKER,
TRUST_BOUNDARY_TEXT,
BYPASS_MARKER_PREFIX,
} from "./prompt-injection.js";
+339 -26
View File
@@ -1,22 +1,38 @@
import { discoverProject, type DirectoryEntry } from "./discover.js"; import { discoverProject, type DirectoryEntry } from "./discover.js";
import {
renderPackageMap, export { discoverProject };
type PackageMapData, import type { FileEntry } from "./directory-model.js";
type FileEntry, import { renderDirectoryIndex, renderDirectoryMap } from "./format.js";
} from "./format.js";
import { extractFileLLM, extractPackageLLM } from "./llm/llm-extract.js"; import { extractFileLLM, extractPackageLLM } from "./llm/llm-extract.js";
import { extractFileAST } from "./ast/ast-extract.js"; import { extractFileAST } from "./ast/ast-extract.js";
import { mergeFileData } from "./merge.js"; import { mergeFileData } from "./merge.js";
import { processFiles } from "./llm/llm-batch.js"; import { processFiles } from "./llm/llm-batch.js";
import { writeFileSync } from "fs"; import { existsSync, writeFileSync } from "fs";
import { join } from "path"; import { join, relative, resolve } from "path";
import type { LLMClient } from "./llm/llm-client.js"; import type { LLMClient } from "./llm/llm-client.js";
import {
createDirectoryModel,
type DirectoryArtifactModel,
type RoutingMetadataOptions,
} from "./directory-model.js";
import { populateRoutingMetadata } from "./routing-metadata.js";
import { loadConfig } from "./config.js";
export interface ProgressInfo {
message: string;
completed: number;
total: number;
currentFile?: string;
dir?: string;
}
export interface InitOptions { export interface InitOptions {
verbose?: boolean; verbose?: boolean;
llmClient?: LLMClient; llmClient?: LLMClient;
cacheDir?: string; cacheDir?: string;
onProgress?: (message: string) => void; onProgress?: (info: ProgressInfo) => void;
tagCap?: number;
workflowHintCap?: number;
} }
export async function initProject( export async function initProject(
@@ -24,37 +40,199 @@ export async function initProject(
options: InitOptions = {}, options: InitOptions = {},
): Promise<void> { ): Promise<void> {
const entries = discoverProject(rootPath); const entries = discoverProject(rootPath);
options.onProgress?.(`Scanning ${entries.length} directories...`); const totalFiles = entries.reduce((sum, e) => sum + e.files.length, 0);
let globalCompleted = 0;
// Load project config and apply defaults when not overridden in options
const config = loadConfig(rootPath);
const routingOpts: RoutingMetadataOptions = {
tagCap: options.tagCap ?? config.tagCap,
workflowHintCap: options.workflowHintCap ?? config.workflowHintCap,
};
options.onProgress?.({
message: `Scanning ${entries.length} directories (${totalFiles} files)...`,
completed: 0,
total: totalFiles,
});
// Build directory relationships
const dirSet = new Set(entries.map((e) => e.relativePath));
const parentMap = buildParentMap(entries);
const childrenMap = buildChildrenMap(entries);
for (let i = 0; i < entries.length; i++) { for (let i = 0; i < entries.length; i++) {
const entry = entries[i]; const entry = entries[i];
options.onProgress?.(`[${i + 1}/${entries.length}] Analyzing ${entry.relativePath} (${entry.files.length} files)...`); await generateDirectoryArtifacts(
await generateDirectoryMap(entry, options.llmClient, options.cacheDir, options.onProgress); entry,
{
dirSet,
parentMap,
childrenMap,
isRoot: entry.relativePath === ".",
},
options.llmClient,
options.cacheDir,
(info) => {
globalCompleted =
info.completed +
entries.slice(0, i).reduce((sum, e) => sum + e.files.length, 0);
options.onProgress?.({
...info,
completed: globalCompleted,
total: totalFiles,
dir: entry.relativePath,
});
},
routingOpts,
);
} }
options.onProgress?.(`Generated ${entries.length} .pi-map.md files`); options.onProgress?.({
message: `Generated ${entries.length} directory map/index pairs`,
completed: totalFiles,
total: totalFiles,
});
if (options.verbose !== false) { if (options.verbose !== false) {
console.log(`Generated ${entries.length} .pi-map.md files`); console.log(`Generated ${entries.length} directory map/index pairs`);
} }
} }
export async function generateDirectoryMap( export interface DirectoryContext {
dirSet: Set<string>;
parentMap: Map<string, string | undefined>;
childrenMap: Map<string, string[]>;
isRoot: boolean;
}
export function getAncestorEntries(
entries: DirectoryEntry[],
ctx: DirectoryContext,
entry: DirectoryEntry, entry: DirectoryEntry,
): DirectoryEntry[] {
const result: DirectoryEntry[] = [];
let parent = ctx.parentMap.get(entry.relativePath);
while (parent) {
const ancestor = entries.find(
(candidate) => candidate.relativePath === parent,
);
if (ancestor) {
result.push(ancestor);
}
parent = ctx.parentMap.get(parent);
}
return result;
}
function countFiles(entries: DirectoryEntry[]): number {
return entries.reduce((sum, e) => sum + e.files.length, 0);
}
function isSubdirectory(parent: string, child: string): boolean {
if (parent === ".") return true;
if (child === parent) return true;
return child.startsWith(`${parent}/`);
}
function findProjectRoot(targetPath: string): string {
let current = resolve(targetPath);
while (true) {
if (existsSync(join(current, ".pi-project-map.json"))) {
return current;
}
if (existsSync(join(current, ".git"))) {
return current;
}
if (existsSync(join(current, "package.json"))) {
return current;
}
const parent = resolve(current, "..");
if (parent === current) {
return resolve(targetPath);
}
current = parent;
}
}
function normalizeRelativePath(relPath: string): string {
if (!relPath || relPath === ".") return ".";
return relPath.replace(/\\/g, "/");
}
export type ArtifactWriteMode = "both" | "map" | "index";
export function buildDirectoryContext(
entries: DirectoryEntry[],
targetEntry: DirectoryEntry,
): DirectoryContext {
return {
dirSet: new Set(entries.map((e) => e.relativePath)),
parentMap: buildParentMap(entries),
childrenMap: buildChildrenMap(entries),
isRoot: targetEntry.relativePath === ".",
};
}
function buildParentMap(
entries: DirectoryEntry[],
): Map<string, string | undefined> {
const map = new Map<string, string | undefined>();
for (const entry of entries) {
const rel = entry.relativePath;
if (rel === ".") {
map.set(rel, undefined);
} else {
const lastSep = rel.lastIndexOf("/");
map.set(rel, lastSep >= 0 ? rel.slice(0, lastSep) : ".");
}
}
return map;
}
function buildChildrenMap(entries: DirectoryEntry[]): Map<string, string[]> {
const map = new Map<string, string[]>();
for (const entry of entries) {
map.set(entry.relativePath, []);
}
for (const entry of entries) {
const rel = entry.relativePath;
if (rel === ".") continue;
const lastSep = rel.lastIndexOf("/");
const parent = lastSep >= 0 ? rel.slice(0, lastSep) : ".";
const siblings = map.get(parent);
if (siblings) {
siblings.push(rel);
}
}
return map;
}
export async function buildDirectoryArtifactModel(
entry: DirectoryEntry,
ctx: DirectoryContext,
llmClient?: LLMClient, llmClient?: LLMClient,
cacheDir?: string, cacheDir?: string,
onProgress?: (message: string) => void, onProgress?: (info: ProgressInfo) => void,
): Promise<FileEntry[]> { routingOpts?: RoutingMetadataOptions,
// Process files in parallel (4 concurrent) with retry logic ): Promise<DirectoryArtifactModel> {
const fileData = await processFiles( const fileData = await processFiles(
entry.files, entry.files,
async (file) => { async (file) => {
const filePath = join(entry.dirPath, file); const filePath = join(entry.dirPath, file);
onProgress?.(`${file}`);
const llmData = await extractFileLLM(filePath, llmClient, cacheDir); const llmData = await extractFileLLM(filePath, llmClient, cacheDir);
const astData = await extractFileAST(filePath); const astData = await extractFileAST(filePath);
return mergeFileData(file, llmData, astData); return mergeFileData(file, llmData, astData);
}, },
{ concurrency: 4, batchDelayMs: 100, maxRetries: 2 }, { concurrency: 4, batchDelayMs: 100, maxRetries: 2 },
(completed, total, currentFile) => {
onProgress?.({
message: `${currentFile}`,
completed,
total,
currentFile,
dir: entry.relativePath,
});
},
); );
const packageData = await extractPackageLLM( const packageData = await extractPackageLLM(
@@ -64,23 +242,158 @@ export async function generateDirectoryMap(
cacheDir, cacheDir,
); );
const mapData: PackageMapData = { const model: DirectoryArtifactModel = createDirectoryModel({
path: entry.relativePath, dir: entry.relativePath,
role: packageData.role, role: packageData.role,
files: fileData, files: fileData,
arch: packageData.arch, arch: packageData.arch,
parent: ctx.parentMap.get(entry.relativePath),
children: ctx.childrenMap.get(entry.relativePath) ?? [],
isRoot: ctx.isRoot,
dirty: "-", dirty: "-",
}; });
const outPath = join(entry.dirPath, ".pi-map.md"); populateRoutingMetadata(model, routingOpts);
writeFileSync(outPath, renderPackageMap(mapData)); return model;
return fileData; }
export function writeDirectoryArtifacts(
entry: DirectoryEntry,
model: DirectoryArtifactModel,
writeMode: ArtifactWriteMode = "both",
): void {
const mapPath = join(entry.dirPath, ".pi-map.md");
const indexPath = join(entry.dirPath, ".pi-map.index.md");
if (writeMode === "both" || writeMode === "map") {
writeFileSync(mapPath, renderDirectoryMap(model));
}
if (writeMode === "both" || writeMode === "index") {
writeFileSync(indexPath, renderDirectoryIndex(model));
}
}
export async function generateDirectoryArtifacts(
entry: DirectoryEntry,
ctx: DirectoryContext,
llmClient?: LLMClient,
cacheDir?: string,
onProgress?: (info: ProgressInfo) => void,
routingOpts?: RoutingMetadataOptions,
writeMode: ArtifactWriteMode = "both",
): Promise<FileEntry[]> {
const model = await buildDirectoryArtifactModel(
entry,
ctx,
llmClient,
cacheDir,
onProgress,
routingOpts,
);
writeDirectoryArtifacts(entry, model, writeMode);
return model.files;
} }
export async function reinitPath( export async function reinitPath(
path: string, path: string,
options: InitOptions = {}, options: InitOptions = {},
): Promise<void> { ): Promise<void> {
// Full regeneration clears all dirty markers by overwriting every .pi-map.md const targetPath = resolve(path);
await initProject(path, options); const rootPath = findProjectRoot(targetPath);
const targetRelPath =
rootPath === targetPath
? "."
: normalizeRelativePath(relative(rootPath, targetPath));
const entries = discoverProject(rootPath);
const config = loadConfig(rootPath);
const threshold = config.reinitFullThresholdPercent;
const totalFiles = countFiles(entries);
if (totalFiles === 0 || targetRelPath === ".") {
await initProject(rootPath, options);
return;
}
const subtreeEntries = entries.filter((entry) =>
isSubdirectory(targetRelPath, entry.relativePath),
);
const subtreeFiles = countFiles(subtreeEntries);
const percentage = (subtreeFiles / totalFiles) * 100;
if (percentage > threshold) {
await initProject(rootPath, options);
return;
}
const routingOpts: RoutingMetadataOptions = {
tagCap: options.tagCap ?? config.tagCap,
workflowHintCap: options.workflowHintCap ?? config.workflowHintCap,
};
const targetEntry = entries.find((e) => e.relativePath === targetRelPath);
if (!targetEntry) {
await initProject(rootPath, options);
return;
}
const changedCtx = buildDirectoryContext(entries, targetEntry);
const ancestors = getAncestorEntries(entries, changedCtx, targetEntry);
const dirsToRegenerate = new Set<DirectoryEntry>([
...subtreeEntries,
...ancestors,
]);
const dirs = Array.from(dirsToRegenerate);
const filesToRegenerate = countFiles(dirs);
let completedFiles = 0;
for (const entry of dirs) {
const ctx = buildDirectoryContext(entries, entry);
await generateDirectoryArtifacts(
entry,
ctx,
options.llmClient,
options.cacheDir,
(info) => {
options.onProgress?.({
...info,
completed: completedFiles + info.completed,
total: filesToRegenerate,
dir: entry.relativePath,
});
},
routingOpts,
"both",
);
completedFiles += entry.files.length;
}
if (options.verbose !== false) {
console.log(
`Smart reinit: regenerated ${dirsToRegenerate.size} directories under ${targetRelPath} (${subtreeFiles}/${totalFiles} files, ${percentage.toFixed(1)}%).`,
);
}
}
// Backward-compatible wrapper for patch/validate compatibility
export async function generateDirectoryMap(
entry: DirectoryEntry,
llmClient?: LLMClient,
cacheDir?: string,
): Promise<FileEntry[]> {
const entries = [entry];
const ctx = buildDirectoryContext(entries, entry);
const config = loadConfig(entry.dirPath);
const routingOpts: RoutingMetadataOptions = {
tagCap: config.tagCap,
workflowHintCap: config.workflowHintCap,
};
return generateDirectoryArtifacts(
entry,
ctx,
llmClient,
cacheDir,
undefined,
routingOpts,
);
} }
+29
View File
@@ -0,0 +1,29 @@
# src/llm (index)
dir: src/llm
## role
Provides a unified abstraction layer for interacting with multiple LLM providers (OpenAI, Kimi, Pi) with caching, batching, and structured response extraction capabilities.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- external-llm-client.ts
- kimi-llm-client.ts
- llm-batch.ts
- llm-cache.ts
- llm-client.ts
- llm-error.ts
- llm-extract.ts
- pi-llm-client.ts
## links
index: src/llm/.pi-map.index.md
map: src/llm/.pi-map.md
## workflows
- change llm behavior
read: external-llm-client.ts, kimi-llm-client.ts, llm-batch.ts
- change llm CLI
read: external-llm-client.ts, kimi-llm-client.ts, llm-client.ts
## dirty
-
+36
View File
@@ -0,0 +1,36 @@
# src/llm
dir: src/llm
index: src/llm/.pi-map.index.md
## role
Provides a unified abstraction layer for interacting with multiple LLM providers (OpenAI, Kimi, Pi) with caching, batching, and structured response extraction capabilities.
## files
- external-llm-client.ts | Implements an LLM client adapter for OpenAI's API to send code analysis prompts and return structured responses. | exp: class:ExternalLLMClient, method:constructor(options: LLMClientOptions), raise:LLMError, method:complete(prompt: string) → Promise<string>, call:this.client.chat.completions.create, call:response.choices[0]?.message?.content?.trim, raise:LLMError | dep: openai, ./llm-error.js, ./llm-client.js
- kimi-llm-client.ts | Implements an LLM client for the Kimi.com API using an Anthropic-compatible HTTP interface. | exp: class:KimiLLMClient, method:constructor(options: LLMClientOptions), raise:LLMError, method:complete(prompt: string) → Promise<string>, call:fetch, call:JSON.stringify, call:response.text, call:response.json, call:data.content?.[0]?.text?.trim, raise:LLMError, raise:err | dep: ./llm-error.js, ./llm-client.js, llm-error.js, llm-client.js
- llm-batch.ts | Provides batched, concurrent file processing with retry logic and progress callbacks for LLM operations. | exp: BatchOptions, func:withRetry(fn: () => Promise<T>, options: Pick<BatchOptions, "maxRetries" | "retryDelaysMs">) → Promise<T>, call:fn, call:sleep, raise:lastError, func:processFiles(files: T[], processor: (file: T) => Promise<R>, options: BatchOptions, onProgress: (completed: number, total: number, currentFile: T) => void) → Promise<R[]>, call:pLimit, call:files.map, call:limit, call:sleep, call:withRetry, call:processor, call:onProgress, call:Promise.all | dep: p-limit, ./llm-error.js
- llm-cache.ts | Provides a persistent file-based caching system for LLM responses keyed by hash, storing results in JSON with atomic writes and automatic directory creation. | exp: func:getCached(hash: string, cacheDir: string) → string | undefined, call:getCachePath, call:loadCache, func:setCached(hash: string, result: string, cacheDir: string) → void, call:getCachePath, call:loadCache, call:Date.now, call:saveCache | dep: fs, path
- llm-client.ts | Factory for creating LLM client instances based on different provider modes (pi, openai, kimi). | exp: LLMClient, LLMClientOptions, func:createLLMClient(mode: "pi" | "openai" | "kimi", options: LLMClientOptions) → LLMClient | dep: ./llm-error.js, ./external-llm-client.js, ./kimi-llm-client.js, ./pi-llm-client.js, LLMError, ExternalLLMClient, KimiLLMClient, PiLLMClient
- llm-error.ts | Defines a custom error class for LLM-related errors with optional cause chaining | exp: class:LLMError, method:constructor(message: string, cause: unknown)
- llm-extract.ts | Extracts structured metadata (purpose, dependencies, concepts) from source files and packages using an LLM client, with binary detection, caching, and context window management. | exp: func:extractFileLLM(filePath: string, client: LLMClient, cacheDir: string) → Promise<LLMFileData>, call:isBinaryFile, call:readFileSync, call:createHash("sha256").update(content).digest, call:getCached, call:parseFileResponse, call:statSync, call:buildFilePrompt, call:truncateForContext, call:client.complete, call:setCached, raise:LLMError, func:extractPackageLLM(relativePath: string, fileData: { name: string; purpose: string }[], client: LLMClient, _cacheDir: string) → Promise<LLMPackageData>, call:buildPackagePrompt, call:client.complete, call:parsePackageResponse, call:basename, raise:LLMError | dep: fs, crypto, path, ./llm-client.js, ./llm-cache.js, ./llm-error.js
- pi-llm-client.ts | Implements an LLM client adapter that bridges to Pi's internal AI runtime using its built-in `complete()` function | exp: class:PiLLMClient, method:constructor(extensionContext: unknown), method:complete(prompt: string) → Promise<string>, call:ctx.modelRegistry?.get, call:ctx.modelRegistry?.getApiKeyAndHeaders, call:complete, call:Date.now, call:response.content .filter((c: any) => c.type === "text") .map((c: any) => c.text) .join("") .trim, raise:LLMError, raise:err | dep: ./llm-error.js, ./llm-client.js, @mariozechner/pi-ai
## arch
Adapter pattern for provider-specific LLM clients with a factory; decorator/wrapper pattern for cross-cutting concerns (caching, batching, retries, error handling); functional pipeline for file extraction with binary detection and context window management.
## tags
llm, client, js, raise:llmerror, cache, llmclient, error, constructor
## symbols
- ExternalLLMClient
- KimiLLMClient
- LLMError
- PiLLMClient
- constructor
- complete
- withRetry
- processFiles
## workflows
- change llm behavior
read: external-llm-client.ts, kimi-llm-client.ts, llm-batch.ts
- change llm CLI
read: external-llm-client.ts, kimi-llm-client.ts, llm-client.ts
## dirty
-
+6 -2
View File
@@ -45,11 +45,12 @@ export async function processFiles<T, R>(
files: T[], files: T[],
processor: (file: T) => Promise<R>, processor: (file: T) => Promise<R>,
options: BatchOptions = {}, options: BatchOptions = {},
onProgress?: (completed: number, total: number, currentFile: T) => void,
): Promise<R[]> { ): Promise<R[]> {
const opts = { ...DEFAULT_OPTIONS, ...options }; const opts = { ...DEFAULT_OPTIONS, ...options };
const limit = pLimit(opts.concurrency); const limit = pLimit(opts.concurrency);
const results: R[] = []; let completed = 0;
let batchCount = 0; let batchCount = 0;
const tasks = files.map((file, index) => const tasks = files.map((file, index) =>
@@ -59,7 +60,10 @@ export async function processFiles<T, R>(
batchCount++; batchCount++;
await sleep(opts.batchDelayMs); await sleep(opts.batchDelayMs);
} }
return withRetry(() => processor(file), opts); const result = await withRetry(() => processor(file), opts);
completed++;
onProgress?.(completed, files.length, file);
return result;
}), }),
); );
+5 -9
View File
@@ -5,10 +5,9 @@ import {
mkdirSync, mkdirSync,
renameSync, renameSync,
} from "fs"; } from "fs";
import { join } from "path"; import { dirname, join } from "path";
const DEFAULT_CACHE_SUBDIR = ".pi-project-map"; const CACHE_FILE = "llm-cache.json";
const DEFAULT_CACHE_FILE = "cache/llm-cache.json";
interface CacheEntry { interface CacheEntry {
result: string; result: string;
@@ -16,15 +15,12 @@ interface CacheEntry {
} }
function getCachePath(cacheDir?: string): string { function getCachePath(cacheDir?: string): string {
if (cacheDir) { const base = cacheDir || process.cwd();
return join(cacheDir, DEFAULT_CACHE_FILE); return join(base, ".cache", CACHE_FILE);
}
// Fallback to cwd when no project root provided
return join(process.cwd(), DEFAULT_CACHE_SUBDIR, DEFAULT_CACHE_FILE);
} }
function ensureCacheDir(cachePath: string): void { function ensureCacheDir(cachePath: string): void {
const dir = cachePath.substring(0, cachePath.lastIndexOf("/")); const dir = dirname(cachePath);
if (!existsSync(dir)) { if (!existsSync(dir)) {
mkdirSync(dir, { recursive: true }); mkdirSync(dir, { recursive: true });
} }
+82 -15
View File
@@ -23,15 +23,58 @@ const CHARS_PER_TOKEN = 4; // approximate for ASCII
// Known binary extensions — skip without reading content // Known binary extensions — skip without reading content
const BINARY_EXTENSIONS = new Set([ const BINARY_EXTENSIONS = new Set([
".png", ".jpg", ".jpeg", ".gif", ".bmp", ".webp", ".ico", ".svgz", ".png",
".mp3", ".mp4", ".avi", ".mov", ".mkv", ".flv", ".wmv", ".jpg",
".wav", ".ogg", ".flac", ".aac", ".wma", ".jpeg",
".zip", ".tar", ".gz", ".bz2", ".xz", ".7z", ".rar", ".gif",
".exe", ".dll", ".so", ".dylib", ".bin", ".bmp",
".pdf", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx", ".webp",
".wasm", ".class", ".jar", ".o", ".a", ".ico",
".ttf", ".otf", ".woff", ".woff2", ".eot", ".svgz",
".db", ".sqlite", ".sqlite3", ".mp3",
".mp4",
".avi",
".mov",
".mkv",
".flv",
".wmv",
".wav",
".ogg",
".flac",
".aac",
".wma",
".zip",
".tar",
".gz",
".bz2",
".xz",
".7z",
".rar",
".exe",
".dll",
".so",
".dylib",
".bin",
".pdf",
".doc",
".docx",
".xls",
".xlsx",
".ppt",
".pptx",
".wasm",
".class",
".jar",
".o",
".a",
".ttf",
".otf",
".woff",
".woff2",
".eot",
".db",
".sqlite",
".sqlite3",
]); ]);
function isBinaryFile(filePath: string): boolean { function isBinaryFile(filePath: string): boolean {
@@ -77,7 +120,11 @@ ${content}
`; `;
} }
function parseFileResponse(response: string): { purpose: string; deps: string[]; concepts: string[] } { function parseFileResponse(response: string): {
purpose: string;
deps: string[];
concepts: string[];
} {
const lines = response.split("\n"); const lines = response.split("\n");
let purpose = ""; let purpose = "";
let deps: string[] = []; let deps: string[] = [];
@@ -89,18 +136,35 @@ function parseFileResponse(response: string): { purpose: string; deps: string[];
purpose = trimmed.slice("PURPOSE:".length).trim(); purpose = trimmed.slice("PURPOSE:".length).trim();
} else if (trimmed.startsWith("DEPS:")) { } else if (trimmed.startsWith("DEPS:")) {
const depsStr = trimmed.slice("DEPS:".length).trim(); const depsStr = trimmed.slice("DEPS:".length).trim();
deps = depsStr === "none" ? [] : depsStr.split(",").map((s) => s.trim()).filter(Boolean); deps =
depsStr === "none"
? []
: depsStr
.split(",")
.map((s) => s.trim())
.filter(Boolean);
} else if (trimmed.startsWith("CONCEPTS:")) { } else if (trimmed.startsWith("CONCEPTS:")) {
const conceptsStr = trimmed.slice("CONCEPTS:".length).trim(); const conceptsStr = trimmed.slice("CONCEPTS:".length).trim();
concepts = conceptsStr === "none" ? [] : conceptsStr.split(",").map((s) => s.trim()).filter(Boolean); concepts =
conceptsStr === "none"
? []
: conceptsStr
.split(",")
.map((s) => s.trim())
.filter(Boolean);
} }
} }
return { purpose, deps, concepts }; return { purpose, deps, concepts };
} }
function buildPackagePrompt(relativePath: string, fileSummaries: { name: string; purpose: string }[]): string { function buildPackagePrompt(
const filesList = fileSummaries.map((f) => `- ${f.name}: ${f.purpose}`).join("\n"); relativePath: string,
fileSummaries: { name: string; purpose: string }[],
): string {
const filesList = fileSummaries
.map((f) => `- ${f.name}: ${f.purpose}`)
.join("\n");
return `Analyze this code package/directory. Respond in this exact format (one line each): return `Analyze this code package/directory. Respond in this exact format (one line each):
ROLE: <concise one-sentence description of this package's role in the project> ROLE: <concise one-sentence description of this package's role in the project>
ARCH: <concise description of architecture/patterns used in this package> ARCH: <concise description of architecture/patterns used in this package>
@@ -111,7 +175,10 @@ ${filesList}
`; `;
} }
function parsePackageResponse(response: string): { role: string; arch: string } { function parsePackageResponse(response: string): {
role: string;
arch: string;
} {
const lines = response.split("\n"); const lines = response.split("\n");
let role = ""; let role = "";
let arch = ""; let arch = "";
+17 -13
View File
@@ -1,4 +1,4 @@
import type { FileEntry } from "./format.js"; import type { FileEntry } from "./directory-model.js";
interface LLMFileData { interface LLMFileData {
purpose: string; purpose: string;
@@ -48,14 +48,16 @@ export function mergeFileData(
for (const cls of ast.classes) { for (const cls of ast.classes) {
const classExports: string[] = [`class:${cls.name}`]; const classExports: string[] = [`class:${cls.name}`];
for (const method of cls.methods) { for (const method of cls.methods) {
const paramStr = method.params.join(", "); const paramStr = method.params.join(", ").replace(/\s+/g, " ");
const returnStr = method.returns ? `${method.returns}` : ""; const returnStr = method.returns
? `${method.returns.replace(/\s+/g, " ")}`
: "";
classExports.push(`method:${method.name}(${paramStr})${returnStr}`); classExports.push(`method:${method.name}(${paramStr})${returnStr}`);
if (method.calls.length > 0) { for (const call of method.calls) {
classExports.push(`call:${method.calls.join(", ")}`); classExports.push(`call:${call}`);
} }
if (method.raises.length > 0) { for (const raise of method.raises) {
classExports.push(`raise:${method.raises.join(", ")}`); classExports.push(`raise:${raise}`);
} }
} }
dedupedExports.push(...classExports); dedupedExports.push(...classExports);
@@ -65,14 +67,16 @@ export function mergeFileData(
// Encode top-level functions // Encode top-level functions
if (ast && ast.functions.length > 0) { if (ast && ast.functions.length > 0) {
for (const func of ast.functions) { for (const func of ast.functions) {
const paramStr = func.params.join(", "); const paramStr = func.params.join(", ").replace(/\s+/g, " ");
const returnStr = func.returns ? `${func.returns}` : ""; const returnStr = func.returns
? `${func.returns.replace(/\s+/g, " ")}`
: "";
dedupedExports.push(`func:${func.name}(${paramStr})${returnStr}`); dedupedExports.push(`func:${func.name}(${paramStr})${returnStr}`);
if (func.calls.length > 0) { for (const call of func.calls) {
dedupedExports.push(`call:${func.calls.join(", ")}`); dedupedExports.push(`call:${call}`);
} }
if (func.raises.length > 0) { for (const raise of func.raises) {
dedupedExports.push(`raise:${func.raises.join(", ")}`); dedupedExports.push(`raise:${raise}`);
} }
} }
} }
+122 -49
View File
@@ -1,71 +1,144 @@
import { dirname, join, basename, relative } from "path"; import { basename, dirname, relative, resolve } from "path";
import { existsSync, readFileSync, writeFileSync } from "fs"; import { existsSync, readFileSync } from "fs";
import { parsePackageMap, renderPackageMap } from "./format.js"; import { parsePackageMap } from "./format.js";
import { extractFileLLM } from "./llm/llm-extract.js"; import {
import { extractFileAST } from "./ast/ast-extract.js"; discoverProject,
import { mergeFileData } from "./merge.js"; generateDirectoryArtifacts,
import { generateDirectoryMap } from "./init.js"; buildDirectoryContext,
import { readdirSync, statSync } from "fs"; getAncestorEntries,
} from "./init.js";
import type { DirectoryEntry } from "./discover.js";
import type { LLMClient } from "./llm/llm-client.js"; import type { LLMClient } from "./llm/llm-client.js";
import { loadConfig } from "./config.js";
const SMALL_PACKAGE_THRESHOLD = 10; export type PatchMode = "auto" | "small" | "structural";
export interface PatchOptions {
patchMode?: PatchMode;
rootPath?: string;
}
const SMALL_CHANGE_FILE_WINDOW = 3;
const STRUCTURAL_FILE_HINT =
/(^|\b)(index|config|cli|command|init|format|discover|patch|validate)(\b|\.)/i;
export async function patchFile( export async function patchFile(
filePath: string, filePath: string,
llmClient?: LLMClient, llmClient?: LLMClient,
cacheDir?: string, cacheDir?: string,
options: PatchOptions = {},
): Promise<void> { ): Promise<void> {
const dirPath = dirname(filePath); const rootPath = resolve(options.rootPath ?? cacheDir ?? process.cwd());
const mapPath = join(dirPath, ".pi-map.md"); const absFilePath = resolve(rootPath, filePath);
const dirPath = dirname(absFilePath);
const relDir = normalizeRelativePath(relative(rootPath, dirPath));
const entries = discoverProject(rootPath);
const entry = entries.find((candidate) => candidate.relativePath === relDir);
if (!existsSync(mapPath)) { if (!entry) {
// No map exists yet — would need to generate from scratch console.warn(`No discovered directory entry found for ${relDir}`);
console.warn(`No .pi-map.md found in ${dirPath}`);
return; return;
} }
const allFiles = readdirSync(dirPath).filter( const config = loadConfig(rootPath);
(f: string) => !f.startsWith(".") && !f.endsWith(".md"), const routingOpts = {
tagCap: config.tagCap,
workflowHintCap: config.workflowHintCap,
};
const mode = determinePatchMode(
entry,
absFilePath,
rootPath,
options.patchMode,
); );
const isSmallPackage = allFiles.length < SMALL_PACKAGE_THRESHOLD; const changedCtx = buildDirectoryContext(entries, entry);
await generateDirectoryArtifacts(
if (isSmallPackage) { entry,
// Full rewrite for small packages changedCtx,
const files = allFiles.filter((f) => {
const st = statSync(join(dirPath, f));
return st.isFile();
});
const relDir = relative(process.cwd(), dirPath) || ".";
await generateDirectoryMap(
{
dirPath,
relativePath: relDir,
files,
},
llmClient, llmClient,
cacheDir, cacheDir,
undefined,
routingOpts,
"both",
); );
for (const ancestor of getAncestorEntries(entries, changedCtx, entry)) {
const ancestorCtx = buildDirectoryContext(entries, ancestor);
await generateDirectoryArtifacts(
ancestor,
ancestorCtx,
llmClient,
cacheDir,
undefined,
routingOpts,
mode === "small" ? "index" : "both",
);
}
console.log( console.log(
`Full rewrite of ${mapPath} (small package: ${allFiles.length} files)`, `Patched ${joinArtifactPath(relDir, ".pi-map.md")} (patchMode: ${mode})`,
); );
} else {
// Section-level patch
const existing = parsePackageMap(readFileSync(mapPath, "utf8"));
const llmData = await extractFileLLM(filePath, llmClient, cacheDir);
const astData = await extractFileAST(filePath);
const fileName = basename(filePath);
const updatedFile = mergeFileData(fileName, llmData, astData);
// Replace the matching file entry
const idx = existing.files.findIndex((f) => f.name === updatedFile.name);
if (idx >= 0) {
existing.files[idx] = updatedFile;
} else {
existing.files.push(updatedFile);
} }
existing.dirty = `${new Date().toISOString()}: ${fileName} patched (section-level)`; function determinePatchMode(
writeFileSync(mapPath, renderPackageMap(existing)); entry: DirectoryEntry,
console.log(`Patched ${mapPath}`); absFilePath: string,
rootPath: string,
explicitMode: PatchMode = "auto",
): Exclude<PatchMode, "auto"> {
if (explicitMode !== "auto") {
return explicitMode;
}
const existingMapPath = resolve(entry.dirPath, ".pi-map.md");
const fileName = basename(absFilePath);
const childCount =
entry.relativePath === "."
? 0
: countDirectChildren(rootPath, entry.relativePath);
if (childCount > 0) {
return "structural";
}
if (entry.files.length > SMALL_CHANGE_FILE_WINDOW) {
return "structural";
}
if (STRUCTURAL_FILE_HINT.test(fileName)) {
return "structural";
}
if (existsSync(existingMapPath)) {
const existing = parsePackageMap(readFileSync(existingMapPath, "utf8"));
const existingFile = existing.files.find(
(candidate) => candidate.name === fileName,
);
if (!existingFile) {
return "structural";
} }
} }
return "small";
}
function countDirectChildren(rootPath: string, relDir: string): number {
const entries = discoverProject(rootPath);
return entries.filter((entry) => {
if (entry.relativePath === "." || entry.relativePath === relDir)
return false;
const parent = normalizeRelativePath(
entry.relativePath.split("/").slice(0, -1).join("/"),
);
return parent === relDir;
}).length;
}
function normalizeRelativePath(relPath: string): string {
if (!relPath || relPath === ".") return ".";
return relPath.replace(/\\/g, "/");
}
function joinArtifactPath(relDir: string, artifactName: string): string {
return relDir === "." ? artifactName : `${relDir}/${artifactName}`;
}
+670
View File
@@ -0,0 +1,670 @@
import { readdirSync, statSync, readFileSync } from "fs";
import { join, relative } from "path";
import type { SkillConfig, PromptInjectionMode } from "./config.js";
// ---------------------------------------------------------------------------
// Outgoing-context scanning (Slice 3)
// ---------------------------------------------------------------------------
export interface ReinjectDecision {
needed: boolean;
reason?: string;
}
/**
* Decide whether reinjection is needed for a given event.
* Checks mode, scans outgoing messages for the canonical marker,
* falls back to payload inspection, and evaluates relevant-turn triggers.
*/
export function shouldReinjectForEvent(
event: {
messages?: Array<{ content?: unknown; text?: string }>;
type?: string;
payload?: unknown;
},
mode: PromptInjectionMode,
): ReinjectDecision {
if (mode !== "strong" && mode !== "strict") {
return { needed: false, reason: "mode_does_not_require_reinjection" };
}
// Slice 3b: root-pair artifact changes invalidate prior injections
if (event.type === "artifact_change") {
return { needed: true, reason: "root_pair_artifact_changed" };
}
if (event.messages && outgoingMessagesHaveMarker(event.messages)) {
return { needed: false, reason: "marker_present_in_messages" };
}
if (event.payload && providerPayloadHasMarker(event.payload)) {
return { needed: false, reason: "marker_present_in_payload" };
}
if (event.type && isRelevantTurnForReinjection(event.type)) {
return { needed: true, reason: "relevant_turn_and_marker_absent" };
}
return { needed: false, reason: "not_a_relevant_turn" };
}
function extractTextFromMessage(m: {
content?: unknown;
text?: string;
}): string {
if (typeof m.content === "string") return m.content;
if (Array.isArray(m.content)) {
return m.content
.map((block: any) => {
if (typeof block === "string") return block;
if (typeof block.text === "string") return block.text;
if (typeof block.content === "string") return block.content;
return "";
})
.join("");
}
return m.text ?? "";
}
/**
* Scan an array of message-like objects for the canonical root-pair marker.
* Normalizes array-based content blocks (e.g., TextContent[]) to strings.
*/
export function outgoingMessagesHaveMarker(
messages: Array<{ content?: unknown; text?: string }>,
): boolean {
return messages.some((m) => hasRootPairMarker(extractTextFromMessage(m)));
}
/**
* Fallback inspection of a final provider payload for the canonical marker.
* Accepts either a string or an object that will be JSON-stringified.
*/
export function providerPayloadHasMarker(payload: unknown): boolean {
if (!payload) return false;
if (typeof payload === "string") {
return hasRootPairMarker(payload);
}
try {
return hasRootPairMarker(JSON.stringify(payload));
} catch {
return false;
}
}
/**
* Relevant-turn types that trigger reinjection in strong mode.
*/
export type RelevantTurnType =
| "agent_start"
| "edit_intent"
| "architecture_sensitive"
| "compaction"
| "artifact_change";
/**
* Determine whether a given event type is a relevant reinjection trigger.
*/
export function isRelevantTurnForReinjection(eventType: string): boolean {
const relevant: RelevantTurnType[] = [
"agent_start",
"edit_intent",
"architecture_sensitive",
"compaction",
"artifact_change",
];
return relevant.includes(eventType as RelevantTurnType);
}
function detectEditIntentFromText(combined: string): boolean {
const patterns = [
/\b(edit|modify|update|change|refactor|fix|patch|rewrite|delete|remove|add)\s+(the|a|this|that|these|those|file|code|function|method|class|module|component|line)\b/i,
/\b(write|create|generate)\s+(new|a|the)\s+(file|function|class|module|component)\b/i,
/```[\s\S]*?\b(edit|modify|update|change|refactor|fix|delete|remove)\b/i,
/\bapply\s+(the|a|this|that)\s+(change|edit|patch|fix)\b/i,
];
return patterns.some((p) => p.test(combined));
}
/**
* Heuristic detection of edit intent from message content.
*/
export function detectEditIntent(
messages: Array<{ content?: unknown; text?: string }>,
): boolean {
if (!messages || messages.length === 0) return false;
const combined = messages.map(extractTextFromMessage).join(" ");
return detectEditIntentFromText(combined);
}
function detectArchitectureSensitiveReasoningFromText(
combined: string,
): boolean {
const patterns = [
/\barchitectur(e|al)\b/i,
/\bdesign\s+(decision|pattern|choice|principle|review)\b/i,
/\b(restructure|reorganize|redesign|rearchitect)\b/i,
/\b(system|high.level|macro|structural)\s+(design|architecture|overview)\b/i,
/\bdependency\s+(injection|graph|cycle|inversion)\b/i,
/\b(api|interface|contract|schema|protocol)\s+(design|change|migration|breaking)\b/i,
/\b(microservice|monolith|modular|layered|hexagonal|clean)\s+arch/i,
/\bdata\s+(model|flow|pipeline|architecture)\b/i,
/\b(scalability|performance|security|maintainability)\s+(concern|tradeoff|decision)\b/i,
];
return patterns.some((p) => p.test(combined));
}
/**
* Heuristic detection of architecture-sensitive reasoning from message content.
*/
export function detectArchitectureSensitiveReasoning(
messages: Array<{ content?: unknown; text?: string }>,
): boolean {
if (!messages || messages.length === 0) return false;
const combined = messages.map(extractTextFromMessage).join(" ");
return detectArchitectureSensitiveReasoningFromText(combined);
}
// ---------------------------------------------------------------------------
// Root-pair artifact change detection
// ---------------------------------------------------------------------------
export interface RootPairMtimes {
mapMtime?: number;
indexMtime?: number;
}
/**
* Read current mtimes for the root pair artifacts, if they exist.
*/
export function getRootPairMtimes(cwd: string): RootPairMtimes {
const mapPath = join(cwd, ".pi-map.md");
const indexPath = join(cwd, ".pi-map.index.md");
const result: RootPairMtimes = {};
try {
result.mapMtime = statSync(mapPath).mtimeMs;
} catch {
// ignore
}
try {
result.indexMtime = statSync(indexPath).mtimeMs;
} catch {
// ignore
}
return result;
}
/**
* Compare current root-pair mtimes against previously recorded ones.
*/
export function rootPairChanged(
current: RootPairMtimes,
previous: RootPairMtimes,
): boolean {
return (
current.mapMtime !== previous.mapMtime ||
current.indexMtime !== previous.indexMtime
);
}
/**
* Canonical markers for injected root-pair content.
* These must be stable across turns and easy to scan in outgoing context.
*/
export const ROOT_PAIR_START_MARKER = "<!-- PI_MAP_ROOT_PAIR_START -->";
export const ROOT_PAIR_END_MARKER = "<!-- PI_MAP_ROOT_PAIR_END -->";
export const TRUST_BOUNDARY_TEXT =
"Trust boundary: index routes, map orients, source decides.";
/**
* Build a canonical root-pair block wrapping index and map content.
*/
export function buildRootPairBlock(
indexContent: string,
mapContent: string,
): string {
return [
ROOT_PAIR_START_MARKER,
"## Project Map Protocol",
"",
"1. Read this protocol and the root `.pi-map.index.md` first.",
"",
TRUST_BOUNDARY_TEXT,
"",
"### Root index",
indexContent,
"",
"### Root map",
mapContent,
ROOT_PAIR_END_MARKER,
].join("\n");
}
/**
* Check whether a message content string contains the canonical root-pair marker.
*/
export function hasRootPairMarker(content: string): boolean {
return content.includes(ROOT_PAIR_START_MARKER);
}
/**
* Build a lightweight user-visible pre-init startup hint.
* No synthetic artifact content is injected — only a prompt to run init.
*/
export function buildPreInitHint(): string {
return [
"📋 Project maps not initialized.",
"",
"The project-map extension is active. Run `project_map_init` to generate paired `.pi-map.md` and `.pi-map.index.md` artifacts for this project.",
"After init, the root pair will be preloaded automatically (default mode: strong). Source remains the final authority before edits.",
].join("\n");
}
/**
* Compute the effective injection budget in tokens.
* Uses the smaller of (percent of context window) and absolute cap.
* Falls back to absolute cap if context window is unknown.
*/
export function computeInjectionBudget(
config: Pick<SkillConfig, "contextBudgetPercent" | "contextBudgetMaxTokens">,
contextWindow?: number,
): number {
const absolute = config.contextBudgetMaxTokens;
if (contextWindow === undefined || contextWindow <= 0) {
return absolute;
}
const relative = Math.floor(
(contextWindow * config.contextBudgetPercent) / 100,
);
return Math.min(relative, absolute);
}
/**
* Determine whether the active mode permits pre-init hints.
*/
export function modeAllowsPreInitHint(mode: PromptInjectionMode): boolean {
return mode !== "off";
}
/**
* Determine whether the active mode permits automatic artifact injection after init.
*/
export function modeAllowsInjection(mode: PromptInjectionMode): boolean {
return mode === "strong" || mode === "strict";
}
/**
* Resolve whether a given mode requires the protocol path for sensitive actions.
*/
export function modeRequiresProtocolPath(mode: PromptInjectionMode): boolean {
return mode === "strict";
}
// ---------------------------------------------------------------------------
// Protocol-path detection (Slice 4)
// ---------------------------------------------------------------------------
/**
* Inline bypass marker prefix. Agents may include `[PI_MAP_BYPASS: reason]` in
* a message to proceed past a strict-mode guard.
*/
export const BYPASS_MARKER_PREFIX = "[PI_MAP_BYPASS:";
function extractTextFromPayload(payload: unknown): string {
if (!payload) return "";
if (typeof payload === "string") return payload;
try {
return JSON.stringify(payload);
} catch {
return "";
}
}
/**
* Check whether the outgoing context contains the full protocol path:
* canonical root-pair marker + trust-boundary instruction.
*/
export function hasProtocolPath(
messages?: Array<{ content?: unknown; text?: string }>,
payload?: unknown,
): boolean {
const sources: string[] = [];
if (messages) {
for (const m of messages) {
sources.push(extractTextFromMessage(m));
}
}
const payloadText = extractTextFromPayload(payload);
if (payloadText) {
sources.push(payloadText);
}
const combined = sources.join("\n");
return (
combined.includes(ROOT_PAIR_START_MARKER) &&
combined.includes(TRUST_BOUNDARY_TEXT)
);
}
/**
* Determine whether the current turn is a sensitive action.
*
* Uses explicit event types when available, with heuristic fallback from
* message content and provider payload for runtimes that do not emit
* `edit_intent` or `architecture_sensitive` event types.
*/
export function isSensitiveAction(
eventType?: string,
messages?: Array<{ content?: unknown; text?: string }>,
payload?: unknown,
): boolean {
const explicit: RelevantTurnType[] = [
"edit_intent",
"architecture_sensitive",
];
if (eventType && explicit.includes(eventType as RelevantTurnType)) {
return true;
}
const messageText =
messages && messages.length > 0
? messages.map(extractTextFromMessage).join("\n")
: "";
const payloadText = extractTextFromPayload(payload);
const combined =
messageText || payloadText ? `${messageText}\n${payloadText}`.trim() : "";
if (!combined) return false;
return (
detectEditIntentFromText(combined) ||
detectArchitectureSensitiveReasoningFromText(combined)
);
}
/**
* Check whether any message contains a valid explicit bypass marker.
* Empty or whitespace-only reasons are rejected.
*/
export function messagesHaveBypass(
messages?: Array<{ content?: unknown; text?: string }>,
payload?: unknown,
): boolean {
return extractBypassReason(messages, payload) !== undefined;
}
/**
* Extract the reason from the first `[PI_MAP_BYPASS: reason]` marker, if any.
* Returns `undefined` when the marker is absent or its reason is empty/whitespace.
*/
export function extractBypassReason(
messages?: Array<{ content?: unknown; text?: string }>,
payload?: unknown,
): string | undefined {
const parts: string[] = [];
if (messages) {
parts.push(messages.map(extractTextFromMessage).join("\n"));
}
const payloadText = extractTextFromPayload(payload);
if (payloadText) {
parts.push(payloadText);
}
const combined = parts.join("\n");
if (!combined) return undefined;
const escapedPrefix = BYPASS_MARKER_PREFIX.replace(
/[.*+?^${}()|[\]\\]/g,
"\\$&",
);
const match = new RegExp(`${escapedPrefix}\\s*([^\\]]+)\\]`).exec(combined);
if (!match) return undefined;
const reason = match[1].trim();
return reason.length > 0 ? reason : undefined;
}
export interface StrictBypassDecision {
guard: boolean;
reason?: string;
bypassMarker?: string;
}
/**
* Evaluate whether strict mode should block a sensitive turn with a visible
* bypass guard because the protocol path is missing.
*
* Returns `guard: false` for non-strict modes, non-sensitive turns, turns
* where the protocol path is present, or turns that include an explicit bypass
* marker.
*/
export function evaluateStrictBypass(
event:
| {
messages?: Array<{ content?: unknown; text?: string }>;
type?: string;
payload?: unknown;
}
| undefined
| null,
mode: PromptInjectionMode,
): StrictBypassDecision {
if (mode !== "strict") {
return { guard: false };
}
if (!event) {
return { guard: false };
}
if (!isSensitiveAction(event.type, event.messages, event.payload)) {
return { guard: false };
}
const bypassReason = extractBypassReason(event.messages, event.payload);
if (bypassReason !== undefined) {
return { guard: false, bypassMarker: bypassReason };
}
if (hasProtocolPath(event.messages, event.payload)) {
return { guard: false };
}
return {
guard: true,
reason: "protocol path missing for sensitive action",
};
}
/**
* Build a lightweight user-visible reminder for advisory mode after init.
* No root-pair content is injected automatically.
*/
export function buildAdvisoryReminder(): string {
return [
"📋 Project map advisory mode active.",
"",
"Root `.pi-map.index.md` and `.pi-map.md` are available but are not automatically injected. Read them manually when you need routing or orientation context, and remember that source remains the final authority before edits.",
].join("\n");
}
/**
* Build a visible strict-mode bypass guard for sensitive turns where the
* protocol path is missing.
*/
export function buildStrictBypassGuard(reason?: string): string {
return [
"🛑 Strict project-map guard",
"",
reason ||
"A sensitive action was detected without the project-map protocol path.",
"",
"The protocol path requires the root `.pi-map.index.md` / `.pi-map.md` pair plus the trust boundary (`index routes, map orients, source decides`) to be present in context.",
"",
"To proceed, either restore the project-map context or include an explicit bypass marker: `[PI_MAP_BYPASS: <brief justification>]`.",
].join("\n");
}
// ---------------------------------------------------------------------------
// Context-window discovery (Slice 2)
// ---------------------------------------------------------------------------
const KNOWN_MODEL_CONTEXT_WINDOWS: Record<string, number> = {
"gpt-4o": 128_000,
"gpt-4o-mini": 128_000,
"gpt-4-turbo": 128_000,
"gpt-4": 8_192,
"claude-3-5-sonnet": 200_000,
"claude-3-opus": 200_000,
"kimi-for-coding": 200_000,
k2p6: 1_000_000,
"kimi-k2-thinking": 256_000,
};
/**
* Discover the active model's context-window size from Pi runtime metadata.
* Returns `undefined` when unavailable so callers fall back to the absolute cap.
*/
export function discoverContextWindow(ctx: any): number | undefined {
const model = ctx?.model;
if (model) {
if (typeof model.contextWindow === "number" && model.contextWindow > 0) {
return model.contextWindow;
}
if (
typeof model.maxContextTokens === "number" &&
model.maxContextTokens > 0
) {
return model.maxContextTokens;
}
if (typeof model.id === "string") {
const known = KNOWN_MODEL_CONTEXT_WINDOWS[model.id];
if (known) return known;
}
}
return undefined;
}
// ---------------------------------------------------------------------------
// Token estimation (best-effort, deterministic)
// ---------------------------------------------------------------------------
/**
* Rough token estimate from character count.
* 1 token ≈ 4 chars for English/prose is a conservative heuristic.
*/
export function estimateTokens(text: string): number {
return Math.ceil(text.length / 4);
}
// ---------------------------------------------------------------------------
// Artifact-pair discovery
// ---------------------------------------------------------------------------
export interface ArtifactPair {
dir: string;
mapPath: string;
indexPath: string;
}
/**
* Find every directory under `cwd` that contains both `.pi-map.md` and
* `.pi-map.index.md`. Returns shallowest directories first for predictable
* structural coverage.
*/
export function findAllArtifactPairs(cwd: string): ArtifactPair[] {
const results: ArtifactPair[] = [];
function walk(dir: string) {
let entries: import("fs").Dirent[];
try {
entries = readdirSync(dir, { withFileTypes: true });
} catch {
return;
}
for (const entry of entries) {
if (
entry.isDirectory() &&
!entry.name.startsWith(".") &&
entry.name !== "node_modules"
) {
walk(join(dir, entry.name));
}
}
const mapPath = join(dir, ".pi-map.md");
const indexPath = join(dir, ".pi-map.index.md");
try {
statSync(mapPath);
statSync(indexPath);
results.push({
dir: relative(cwd, dir) || ".",
mapPath: relative(cwd, mapPath),
indexPath: relative(cwd, indexPath),
});
} catch {
// skip dirs without the paired artifacts
}
}
walk(cwd);
// shallow-first: sort by path depth then alphabetically
results.sort((a, b) => {
const depthA = a.dir.split(/[/\\]/).length;
const depthB = b.dir.split(/[/\\]/).length;
if (depthA !== depthB) return depthA - depthB;
return a.dir.localeCompare(b.dir);
});
return results;
}
// ---------------------------------------------------------------------------
// Budgeted expansion
// ---------------------------------------------------------------------------
/**
* Build the full injection payload for a project that already has artifacts.
*
* Guarantees:
* 1. Root pair is always present first.
* 2. Additional pairs are appended shallow-first while the budget allows.
* 3. A brief maintenance reminder is prepended.
*/
export function buildInjectionPayload(
cwd: string,
config: Pick<SkillConfig, "contextBudgetPercent" | "contextBudgetMaxTokens">,
contextWindow: number | undefined,
): { content: string; display: false } {
const budget = computeInjectionBudget(config, contextWindow);
const pairs = findAllArtifactPairs(cwd);
const rootIndex = pairs.findIndex((p) => p.dir === ".");
let rootPair: ArtifactPair | undefined;
if (rootIndex >= 0) {
rootPair = pairs.splice(rootIndex, 1)[0];
}
let usedTokens = 0;
const parts: string[] = [];
// Maintenance reminder (lightweight)
const reminder =
"📋 Project map active: Start with the root `.pi-map.index.md`, use indexes first for routing, read the local `.pi-map.md` plus source before edits, run `project_map_patch` after source edits, and run `project_map_validate` before freshness-sensitive architectural handoff. Trust boundary: index routes, map orients, source decides.";
parts.push(reminder);
usedTokens += estimateTokens(reminder);
// Root pair (guaranteed)
if (rootPair) {
const indexContent = readFileSync(join(cwd, rootPair.indexPath), "utf8");
const mapContent = readFileSync(join(cwd, rootPair.mapPath), "utf8");
const block = buildRootPairBlock(indexContent, mapContent);
parts.push(block);
usedTokens += estimateTokens(block);
}
// Budgeted expansion (shallow-first, deterministic)
for (const pair of pairs) {
const indexContent = readFileSync(join(cwd, pair.indexPath), "utf8");
const mapContent = readFileSync(join(cwd, pair.mapPath), "utf8");
const pairTokens =
estimateTokens(indexContent) + estimateTokens(mapContent);
if (usedTokens + pairTokens > budget) {
break;
}
parts.push(
`\n## ${pair.dir}\n\n### Index\n${indexContent}\n\n### Map\n${mapContent}`,
);
usedTokens += pairTokens;
}
return { content: parts.join("\n\n"), display: false };
}
+296
View File
@@ -0,0 +1,296 @@
import { readdirSync, readFileSync } from "fs";
import { join } from "path";
import { parseDirectoryMap, parseDirectoryIndex } from "./format.js";
import type { DirectoryArtifactModel } from "./directory-model.js";
const TOP_K = 3;
export interface RetrieveContextOptions {
topK?: number;
}
export function retrieveContext(
query: string,
rootPath: string,
opts: RetrieveContextOptions = {},
): string {
const topK = opts.topK ?? TOP_K;
const candidates = collectCandidates(rootPath);
const queryTerms = normalizeTerms(query);
const scored = candidates
.map((model) => ({
model,
score: scoreDirectory(model, queryTerms),
}))
.filter((c) => c.score > 0)
.sort((a, b) => b.score - a.score)
.slice(0, topK);
if (scored.length === 0) {
return renderNoResultsBundle(query);
}
return renderContextBundle(
query,
scored.map((s) => s.model),
);
}
function collectCandidates(rootPath: string): DirectoryArtifactModel[] {
const results: DirectoryArtifactModel[] = [];
function walk(dir: string) {
let entries: import("fs").Dirent[];
try {
entries = readdirSync(dir, { withFileTypes: true });
} catch {
return;
}
const hasIndex = entries.some(
(e) => e.isFile() && e.name === ".pi-map.index.md",
);
const hasMap = entries.some((e) => e.isFile() && e.name === ".pi-map.md");
if (hasIndex || hasMap) {
// Prefer index first, fall back to map, merge if both exist
let model: DirectoryArtifactModel | null = null;
if (hasIndex) {
try {
const text = readFileSync(join(dir, ".pi-map.index.md"), "utf8");
model = parseDirectoryIndex(text);
} catch {
// ignore parse error
}
}
if (hasMap) {
try {
const text = readFileSync(join(dir, ".pi-map.md"), "utf8");
const mapModel = parseDirectoryMap(text);
if (model) {
// Merge: index provides routing, map provides richer metadata
mergeModels(model, mapModel);
} else {
model = mapModel;
}
} catch {
// ignore parse error
}
}
if (model) {
results.push(model);
}
}
for (const entry of entries) {
if (
entry.isDirectory() &&
!entry.name.startsWith(".") &&
entry.name !== "node_modules"
) {
walk(join(dir, entry.name));
}
}
}
walk(rootPath);
return results;
}
function mergeModels(
index: DirectoryArtifactModel,
map: DirectoryArtifactModel,
): void {
// Use map's richer fields when index lacks them
if (!index.arch && map.arch) index.arch = map.arch;
if (index.tags.length === 0 && map.tags.length > 0) index.tags = map.tags;
if (index.symbols.length === 0 && map.symbols.length > 0)
index.symbols = map.symbols;
if (index.files.length === 0 && map.files.length > 0) index.files = map.files;
// Merge workflows, preferring index's workflow list but keeping map's extras
const indexTasks = new Set(index.workflows.map((w) => w.task));
for (const wf of map.workflows) {
if (!indexTasks.has(wf.task)) {
index.workflows.push(wf);
}
}
}
function normalizeTerms(query: string): string[] {
const terms = new Set<string>();
for (const word of query.toLowerCase().split(/[^a-z0-9]+/)) {
if (word.length > 1) {
terms.add(word);
for (const sub of splitCamelCase(word)) {
if (sub.length > 1) terms.add(sub);
}
}
}
return [...terms];
}
function splitCamelCase(str: string): string[] {
return str
.replace(/([a-z0-9])([A-Z])/g, "$1 $2")
.replace(/[_-]+/g, " ")
.toLowerCase()
.split(/\s+/)
.filter((w) => w.length > 1);
}
function scoreDirectory(
model: DirectoryArtifactModel,
queryTerms: string[],
): number {
let score = 0;
// Role matches
const roleWords = extractWords(model.role);
for (const term of queryTerms) {
if (roleWords.includes(term)) score += 3;
}
// Tag matches
for (const tag of model.tags) {
for (const term of queryTerms) {
if (tag.toLowerCase().includes(term)) score += 2;
}
}
// Symbol matches
for (const sym of model.symbols) {
for (const term of queryTerms) {
if (sym.toLowerCase().includes(term)) score += 2;
}
}
// Workflow task matches
for (const wf of model.workflows) {
for (const term of queryTerms) {
if (wf.task.toLowerCase().includes(term)) score += 2;
}
}
// File name and purpose matches
for (const file of model.files) {
const fileWords = extractWords(`${file.name} ${file.purpose}`);
const camelWords = splitCamelCase(file.name);
for (const term of queryTerms) {
if (fileWords.includes(term) || camelWords.includes(term)) score += 1;
}
}
// Directory path matches
const pathWords = extractWords(model.dir);
for (const term of queryTerms) {
if (pathWords.includes(term)) score += 1;
}
return score;
}
function extractWords(text: string): string[] {
return text
.toLowerCase()
.split(/[^a-z0-9]+/)
.filter((w) => w.length > 1);
}
function renderContextBundle(
query: string,
candidates: DirectoryArtifactModel[],
): string {
const lines: string[] = [];
lines.push(`# Context bundle: ${query}`);
lines.push("");
lines.push("## query");
lines.push(query);
lines.push("");
lines.push("## relevant indexes");
for (const c of candidates) {
lines.push(`- ${c.dir}/.pi-map.index.md`);
}
lines.push("");
lines.push("## relevant maps");
for (const c of candidates) {
lines.push(`- ${c.dir}/.pi-map.md`);
}
lines.push("");
lines.push("## likely files");
const seenFiles = new Set<string>();
for (const c of candidates) {
for (const f of c.files) {
const path = `${c.dir}/${f.name}`;
if (!seenFiles.has(path)) {
seenFiles.add(path);
lines.push(`- ${path}`);
}
}
}
if (seenFiles.size === 0) {
lines.push("-");
}
lines.push("");
// Symbols: only include if at least one candidate has scored symbols above threshold
const relevantSymbols = collectRelevantSymbols(candidates);
if (relevantSymbols.length > 0) {
lines.push("## relevant symbols");
for (const sym of relevantSymbols) {
lines.push(`- ${sym}`);
}
lines.push("");
}
lines.push("## instructions");
lines.push(
"Read the indexes first, then the strongest-match rich maps, then verify behavior from source before editing.",
);
return lines.join("\n");
}
function collectRelevantSymbols(
candidates: DirectoryArtifactModel[],
): string[] {
const seen = new Set<string>();
const result: string[] = [];
for (const c of candidates) {
for (const sym of c.symbols) {
if (!seen.has(sym)) {
seen.add(sym);
result.push(sym);
}
}
}
// Cap at a reasonable number to keep output compact
return result.slice(0, 12);
}
function renderNoResultsBundle(query: string): string {
return [
`# Context bundle: ${query}`,
"",
"## query",
query,
"",
"## relevant indexes",
"-",
"",
"## relevant maps",
"-",
"",
"## likely files",
"-",
"",
"## instructions",
"No relevant directories found. Try rephrasing the query, or run `project_map_validate` to check whether artifacts are stale.",
].join("\n");
}
+265
View File
@@ -0,0 +1,265 @@
import type {
DirectoryArtifactModel,
FileEntry,
RoutingMetadataOptions,
WorkflowHint,
} from "./directory-model.js";
const DEFAULT_TAG_CAP = 8;
const DEFAULT_WORKFLOW_HINT_CAP = 5;
const STOP_WORDS = new Set([
"the",
"and",
"for",
"with",
"from",
"into",
"onto",
"this",
"that",
"then",
"than",
"when",
"where",
"what",
"how",
"why",
"who",
"which",
"while",
"during",
"before",
"after",
"above",
"below",
"between",
"among",
"through",
"over",
"under",
"again",
"further",
"once",
"here",
"there",
"all",
"any",
"both",
"each",
"few",
"more",
"most",
"other",
"some",
"such",
"only",
"own",
"same",
"so",
"too",
"very",
"can",
"will",
"just",
"should",
"now",
"use",
"using",
"used",
"via",
"based",
"build",
"built",
"used",
"file",
"files",
"module",
"modules",
"function",
"functions",
"class",
"classes",
"export",
"exports",
"import",
"imports",
]);
export function populateRoutingMetadata(
model: DirectoryArtifactModel,
opts: RoutingMetadataOptions = {},
): void {
const tagCap = opts.tagCap ?? DEFAULT_TAG_CAP;
const workflowHintCap = opts.workflowHintCap ?? DEFAULT_WORKFLOW_HINT_CAP;
model.tags = generateTags(model.files, tagCap);
model.symbols = generateSymbols(model.files, tagCap);
model.workflows = generateWorkflows(model, workflowHintCap);
}
function generateTags(files: FileEntry[], cap: number): string[] {
const scores = new Map<string, number>();
for (const file of files) {
// Score words from file purpose
for (const word of extractWords(file.purpose)) {
scores.set(word, (scores.get(word) ?? 0) + 1);
}
// Score words from export names (camelCase split)
for (const exp of file.exports) {
const cleanExp = exp.replace(/^(class|func|method):/, "").split("(")[0];
for (const word of splitCamelCase(cleanExp)) {
if (word.length > 1) {
scores.set(word, (scores.get(word) ?? 0) + 2);
}
}
}
// Score dep path segments
for (const dep of file.deps) {
for (const segment of dep.split(/[/\-.]/)) {
const word = segment.toLowerCase();
if (word.length > 1 && !STOP_WORDS.has(word)) {
scores.set(word, (scores.get(word) ?? 0) + 1);
}
}
}
// Score from file name (stem, no extension)
const stem = file.name.replace(/\.[^.]+$/, "");
for (const word of splitCamelCase(stem)) {
if (word.length > 1) {
scores.set(word, (scores.get(word) ?? 0) + 2);
}
}
}
return Array.from(scores.entries())
.sort((a, b) => b[1] - a[1])
.map(([word]) => word)
.slice(0, cap);
}
function generateSymbols(files: FileEntry[], cap: number): string[] {
const scores = new Map<string, number>();
const seen = new Set<string>();
for (const file of files) {
for (const exp of file.exports) {
// Prefer clean identifiers over encoded DSL entries when possible
let symbol = exp;
let score = 1;
if (exp.startsWith("class:")) {
symbol = exp.slice(6).split(" ")[0];
score = 3;
} else if (exp.startsWith("func:")) {
symbol = exp.slice(5).split("(")[0];
score = 2;
} else if (exp.startsWith("method:")) {
symbol = exp.slice(7).split("(")[0];
score = 2;
}
if (!seen.has(symbol)) {
seen.add(symbol);
scores.set(symbol, (scores.get(symbol) ?? 0) + score);
}
}
}
return Array.from(scores.entries())
.sort((a, b) => b[1] - a[1])
.map(([name]) => name)
.slice(0, cap);
}
function generateWorkflows(
model: DirectoryArtifactModel,
cap: number,
): WorkflowHint[] {
const workflows: WorkflowHint[] = [];
const dirName = model.dir.split("/").pop() || model.dir;
const baseName = dirName === "." ? "project" : dirName;
// Only generate workflows when we have reasonable structural confidence
const hasSourceFiles = model.files.some((f) =>
/\.(ts|tsx|js|jsx|py|go|rs|java)$/.test(f.name),
);
if (!hasSourceFiles || model.files.length === 0) {
return workflows;
}
const sourceFiles = model.files.filter(
(f) =>
!/\.(test|spec)\./.test(f.name) && !/\.(md|json|yaml|yml)$/.test(f.name),
);
const testFiles = model.files.filter((f) => /\.(test|spec)\./.test(f.name));
const configFiles = model.files.filter(
(f) =>
f.name.includes("config") ||
/\.(json|yaml|yml|toml)$/.test(f.name) ||
f.name === ".env",
);
const cliFiles = model.files.filter(
(f) => f.name.includes("cli") || f.name.includes("command"),
);
if (sourceFiles.length > 0) {
workflows.push({
task: `change ${baseName} behavior`,
read: sourceFiles.slice(0, 3).map((f) => f.name),
});
}
if (testFiles.length > 0) {
workflows.push({
task: `update ${baseName} tests`,
read: testFiles.slice(0, 3).map((f) => f.name),
});
}
if (cliFiles.length > 0) {
workflows.push({
task: `change ${baseName} CLI`,
read: cliFiles.slice(0, 3).map((f) => f.name),
});
}
if (configFiles.length > 0) {
workflows.push({
task: `change ${baseName} config`,
read: configFiles.slice(0, 3).map((f) => f.name),
});
}
// Add a directory-navigation workflow for non-leaf directories
if (model.children.length > 0) {
workflows.push({
task: `explore ${baseName} subdirectories`,
index: model.children
.slice(0, 3)
.map((child) => `${child}/.pi-map.index.md`),
});
}
return workflows.slice(0, cap);
}
function extractWords(text: string): string[] {
return text
.toLowerCase()
.split(/[^a-z0-9]+/)
.filter((w) => w.length > 2 && !STOP_WORDS.has(w));
}
function splitCamelCase(str: string): string[] {
return str
.replace(/([a-z0-9])([A-Z])/g, "$1 $2")
.replace(/[_-]+/g, " ")
.toLowerCase()
.split(/\s+/)
.filter((w) => w.length > 1 && !STOP_WORDS.has(w));
}
+20
View File
@@ -0,0 +1,20 @@
# src/types (index)
dir: src/types
## role
Provides TypeScript type declarations for the Pi AI module's LLM chat completion functionality within the Pi runtime.
## parent
index: src/.pi-map.index.md
map: src/.pi-map.md
## children
-
## files
- pi-ai.d.ts
## links
index: src/types/.pi-map.index.md
map: src/types/.pi-map.md
## workflows
- change types behavior
read: pi-ai.d.ts
## dirty
-
+20
View File
@@ -0,0 +1,20 @@
# src/types
dir: src/types
index: src/types/.pi-map.index.md
## role
Provides TypeScript type declarations for the Pi AI module's LLM chat completion functionality within the Pi runtime.
## files
- pi-ai.d.ts | TypeScript declaration file for the Pi AI module's `complete` function that provides LLM chat completions within the Pi runtime | exp: complete
## arch
Minimal declaration-only types package using ambient module declarations (.d.ts) to define external API interfaces without implementation.
## tags
complete, pi, ai.d, typescript, declaration, provides, llm, chat
## symbols
- complete
## workflows
- change types behavior
read: pi-ai.d.ts
## dirty
-
+370 -57
View File
@@ -1,9 +1,17 @@
import { discoverProject } from "./discover.js";
import { parsePackageMap } from "./format.js";
import { existsSync, readFileSync } from "fs"; import { existsSync, readFileSync } from "fs";
import { join } from "path"; import { join, resolve } from "path";
import { discoverProject } from "./discover.js";
import type { DirectoryEntry } from "./discover.js";
import { parseDirectoryIndex, parseDirectoryMap } from "./format.js";
import { extractFileAST } from "./ast/ast-extract.js"; import { extractFileAST } from "./ast/ast-extract.js";
import { generateDirectoryMap } from "./init.js"; import {
generateDirectoryArtifacts,
buildDirectoryContext,
type ArtifactWriteMode,
} from "./init.js";
import type { LLMClient } from "./llm/llm-client.js";
import { loadConfig } from "./config.js";
import type { PatchMode } from "./patch.js";
export interface ValidationResult { export interface ValidationResult {
clean: boolean; clean: boolean;
@@ -11,81 +19,219 @@ export interface ValidationResult {
fixed?: number; fixed?: number;
} }
export interface ValidationOptions {
fix?: boolean;
verbose?: boolean;
llmClient?: LLMClient;
cacheDir?: string;
patchMode?: PatchMode;
}
export interface Discrepancy { export interface Discrepancy {
type: "missing" | "orphaned" | "stale-signature" | "dirty"; type:
| "missing"
| "orphaned"
| "stale-signature"
| "dirty"
| "stale-index"
| "stale-map"
| "broken-link"
| "structural";
path: string; path: string;
message: string; message: string;
} }
type RepairMode = Exclude<PatchMode, "auto">;
export async function validateMaps( export async function validateMaps(
rootPath: string, rootPath: string,
options?: { fix?: boolean; verbose?: boolean }, options: ValidationOptions = {},
): Promise<ValidationResult> { ): Promise<ValidationResult> {
const { fix = false, verbose = true } = options || {}; const root = resolve(rootPath);
const {
fix = false,
verbose = true,
llmClient,
cacheDir,
patchMode = "auto",
} = options;
const discrepancies: Discrepancy[] = []; const discrepancies: Discrepancy[] = [];
const entries = discoverProject(rootPath); const entries = discoverProject(root);
const dirsToFix = new Set<string>(); const config = loadConfig(root);
const routingOpts = {
tagCap: config.tagCap,
workflowHintCap: config.workflowHintCap,
};
const repairModes = new Map<string, RepairMode>();
for (const entry of entries) { for (const entry of entries) {
const ctx = buildDirectoryContext(entries, entry);
const mapPath = join(entry.dirPath, ".pi-map.md"); const mapPath = join(entry.dirPath, ".pi-map.md");
const indexPath = join(entry.dirPath, ".pi-map.index.md");
const expectedParent = ctx.parentMap.get(entry.relativePath);
const expectedChildren = [
...(ctx.childrenMap.get(entry.relativePath) ?? []),
].sort();
const structuralFlag = (reason: string, path = entry.relativePath) => {
discrepancies.push({ type: "structural", path, message: reason });
markRepairMode(repairModes, entry.dirPath, "structural");
};
if (!existsSync(mapPath)) { if (!existsSync(mapPath)) {
discrepancies.push({ discrepancies.push({
type: "missing", type: "stale-map",
path: entry.relativePath, path: mapPath,
message: "No .pi-map.md found", message: "missing .pi-map.md",
}); });
if (fix) dirsToFix.add(entry.dirPath); markRepairMode(repairModes, entry.dirPath, "structural");
continue;
}
if (!existsSync(indexPath)) {
discrepancies.push({
type: "stale-index",
path: indexPath,
message: "missing .pi-map.index.md",
});
markRepairMode(repairModes, entry.dirPath, "structural");
continue; continue;
} }
const mapData = parsePackageMap(readFileSync(mapPath, "utf8")); const mapText = readFileSync(mapPath, "utf8");
let mapNeedsRewrite = false; const indexText = readFileSync(indexPath, "utf8");
const mapData = parseDirectoryMap(mapText);
const indexData = parseDirectoryIndex(indexText);
// Check for dirty markers
if (mapData.dirty && mapData.dirty !== "-") { if (mapData.dirty && mapData.dirty !== "-") {
discrepancies.push({ discrepancies.push({
type: "dirty", type: "dirty",
path: mapPath, path: mapPath,
message: `Dirty: ${mapData.dirty}`, message: `Dirty: ${mapData.dirty}`,
}); });
if (fix) mapNeedsRewrite = true; markRepairMode(repairModes, entry.dirPath, "small");
} }
if (indexData.dirty && indexData.dirty !== "-") {
// Check for orphaned entries
for (const fileEntry of mapData.files) {
const filePath = join(entry.dirPath, fileEntry.name);
if (!existsSync(filePath)) {
discrepancies.push({ discrepancies.push({
type: "orphaned", type: "stale-index",
path: filePath, path: indexPath,
message: `File listed but deleted: ${fileEntry.name}`, message: `dirty index: ${indexData.dirty}`,
}); });
if (fix) mapNeedsRewrite = true; markRepairMode(repairModes, entry.dirPath, "small");
}
} }
// Check for new files not in map if (mapData.dir !== entry.relativePath) {
structuralFlag(`dir mismatch in map (${mapData.dir})`, mapPath);
}
if (indexData.dir !== entry.relativePath) {
structuralFlag(`dir mismatch in index (${indexData.dir})`, indexPath);
}
const expectedIndexLink = `index: ${entry.relativePath}/.pi-map.index.md`;
const expectedMapLink = `map: ${entry.relativePath}/.pi-map.md`;
if (!mapText.includes(expectedIndexLink)) {
discrepancies.push({
type: "broken-link",
path: mapPath,
message: `missing sibling index link (${expectedIndexLink})`,
});
markRepairMode(repairModes, entry.dirPath, "structural");
}
if (!indexText.includes(expectedMapLink)) {
discrepancies.push({
type: "broken-link",
path: indexPath,
message: `missing sibling map link (${expectedMapLink})`,
});
markRepairMode(repairModes, entry.dirPath, "structural");
}
const mapFiles = new Set(mapData.files.map((file) => file.name));
const indexFiles = new Set(indexData.files.map((file) => file.name));
const actualFiles = new Set(entry.files);
for (const file of entry.files) { for (const file of entry.files) {
if (!mapData.files.find((f) => f.name === file)) { if (!mapFiles.has(file)) {
discrepancies.push({ discrepancies.push({
type: "missing", type: "missing",
path: join(entry.dirPath, file), path: join(entry.dirPath, file),
message: `File not in .pi-map.md: ${file}`, message: `File not in .pi-map.md: ${file}`,
}); });
if (fix) mapNeedsRewrite = true; markRepairMode(repairModes, entry.dirPath, "small");
}
if (!indexFiles.has(file)) {
discrepancies.push({
type: "stale-index",
path: indexPath,
message: `file missing from index: ${file}`,
});
markRepairMode(repairModes, entry.dirPath, "small");
}
}
for (const file of mapFiles) {
if (!actualFiles.has(file)) {
discrepancies.push({
type: "orphaned",
path: join(entry.dirPath, file),
message: `File listed but deleted: ${file}`,
});
markRepairMode(repairModes, entry.dirPath, "small");
}
}
for (const file of indexFiles) {
if (!actualFiles.has(file)) {
discrepancies.push({
type: "stale-index",
path: indexPath,
message: `orphaned file entry in index: ${file}`,
});
markRepairMode(repairModes, entry.dirPath, "small");
}
}
if ((indexData.parent ?? undefined) !== expectedParent) {
discrepancies.push({
type: "broken-link",
path: indexPath,
message: `parent mismatch: expected ${expectedParent ?? "-"}`,
});
markRepairMode(repairModes, entry.dirPath, "structural");
}
if (!sameStringArrays(indexData.children, expectedChildren)) {
discrepancies.push({
type: "broken-link",
path: indexPath,
message: `children mismatch: expected ${expectedChildren.join(", ") || "-"}`,
});
markRepairMode(repairModes, entry.dirPath, "structural");
}
if (!sameWorkflowShapes(mapData.workflows, indexData.workflows)) {
discrepancies.push({
type: "structural",
path: entry.relativePath,
message: "map/index workflow disagreement",
});
markRepairMode(repairModes, entry.dirPath, "structural");
}
for (const target of collectWorkflowTargets(indexData)) {
if (!targetExists(root, target)) {
discrepancies.push({
type: "broken-link",
path: indexPath,
message: `workflow target missing: ${target}`,
});
markRepairMode(repairModes, entry.dirPath, "structural");
} }
} }
// Check signatures for code files
for (const fileEntry of mapData.files) { for (const fileEntry of mapData.files) {
const filePath = join(entry.dirPath, fileEntry.name); const filePath = join(entry.dirPath, fileEntry.name);
if (!existsSync(filePath)) continue; if (!existsSync(filePath)) continue;
const astData = await extractFileAST(filePath); const astData = await extractFileAST(filePath);
if (astData) { if (!astData) continue;
const listedExports = new Set(fileEntry.exports);
const actualExports = new Set(astData.exports); const actualExports = new Set(astData.exports);
const listedExports = new Set(
normalizeListedExports(fileEntry.exports, actualExports),
);
for (const exp of listedExports) { for (const exp of listedExports) {
if (!actualExports.has(exp)) { if (!actualExports.has(exp)) {
discrepancies.push({ discrepancies.push({
@@ -93,7 +239,7 @@ export async function validateMaps(
path: filePath, path: filePath,
message: `Missing export: ${exp}`, message: `Missing export: ${exp}`,
}); });
if (fix) mapNeedsRewrite = true; markRepairMode(repairModes, entry.dirPath, "structural");
} }
} }
for (const exp of actualExports) { for (const exp of actualExports) {
@@ -103,26 +249,41 @@ export async function validateMaps(
path: filePath, path: filePath,
message: `New export: ${exp}`, message: `New export: ${exp}`,
}); });
if (fix) mapNeedsRewrite = true; markRepairMode(repairModes, entry.dirPath, "structural");
} }
} }
} }
} }
if (fix && mapNeedsRewrite) {
dirsToFix.add(entry.dirPath);
}
}
// Apply fixes
let fixed = 0; let fixed = 0;
if (fix && dirsToFix.size > 0) { if (fix && repairModes.size > 0) {
for (const dirPath of dirsToFix) { if (!llmClient) {
const entry = entries.find((e) => e.dirPath === dirPath); throw new Error("validate --fix requires an LLM client");
if (entry) {
await generateDirectoryMap(entry);
fixed++;
} }
const repairPlan = new Map<string, ArtifactWriteMode>();
for (const [dirPath, inferredMode] of repairModes) {
const entry = entries.find((candidate) => candidate.dirPath === dirPath);
if (!entry) continue;
const effectiveMode = patchMode === "auto" ? inferredMode : patchMode;
addRepairChain(entries, entry, effectiveMode, repairPlan);
}
const orderedEntries = Array.from(repairPlan.entries()).sort(
(a, b) => depthOfPath(b[0]) - depthOfPath(a[0]),
);
for (const [dirPath, writeMode] of orderedEntries) {
const entry = entries.find((candidate) => candidate.dirPath === dirPath);
if (!entry) continue;
const ctx = buildDirectoryContext(entries, entry);
await generateDirectoryArtifacts(
entry,
ctx,
llmClient,
cacheDir,
undefined,
routingOpts,
writeMode,
);
fixed++;
} }
} }
@@ -130,24 +291,176 @@ export async function validateMaps(
clean: discrepancies.length === 0, clean: discrepancies.length === 0,
discrepancies, discrepancies,
}; };
if (fix) result.fixed = fixed;
if (fix) {
result.fixed = fixed;
}
if (verbose) { if (verbose) {
if (result.clean) { if (result.clean) {
console.log("All .pi-map.md files are clean."); console.log("All .pi-map.md/.pi-map.index.md files are clean.");
} else { } else {
console.log(`Found ${discrepancies.length} discrepancies:`); console.log(`Found ${discrepancies.length} discrepancies:`);
for (const d of discrepancies) { for (const discrepancy of discrepancies) {
console.log(` [${d.type}] ${d.path}: ${d.message}`); console.log(
` [${discrepancy.type}] ${discrepancy.path}: ${discrepancy.message}`,
);
} }
} }
if (fix && fixed > 0) { if (fix && fixed > 0) {
console.log(`Fixed ${fixed} director${fixed === 1 ? "y" : "ies"}.`); const fixModeLabel = patchMode === "auto" ? "auto" : patchMode;
console.log(
`Repaired affected chain (patchMode: ${fixModeLabel}) across ${fixed} directories.`,
);
} }
} }
return result; return result;
} }
function sameStringArrays(a: string[], b: string[]): boolean {
const left = [...a].sort();
const right = [...b].sort();
return JSON.stringify(left) === JSON.stringify(right);
}
function sameWorkflowShapes(
a: ReturnType<typeof parseDirectoryMap>["workflows"],
b: ReturnType<typeof parseDirectoryIndex>["workflows"],
): boolean {
const normalize = (workflow: (typeof a)[number]) => ({
task: workflow.task,
read: [...(workflow.read ?? [])].sort(),
index: [...(workflow.index ?? [])].sort(),
map: [...(workflow.map ?? [])].sort(),
files: [...(workflow.files ?? [])].sort(),
});
return JSON.stringify(a.map(normalize)) === JSON.stringify(b.map(normalize));
}
function collectWorkflowTargets(
indexData: ReturnType<typeof parseDirectoryIndex>,
): string[] {
const targets: string[] = [];
for (const workflow of indexData.workflows) {
targets.push(...(workflow.index ?? []));
targets.push(...(workflow.map ?? []));
targets.push(...(workflow.files ?? []));
}
return targets;
}
function targetExists(rootPath: string, target: string): boolean {
return existsSync(resolve(rootPath, target));
}
const BUILT_IN_GLOBALS = new Set([
"Array",
"Date",
"JSON",
"Math",
"Object",
"Promise",
"RegExp",
"String",
"Number",
"Boolean",
"Error",
"Map",
"Set",
"Symbol",
"WeakMap",
"WeakSet",
"ArrayBuffer",
"DataView",
"Float32Array",
"Float64Array",
"Int8Array",
"Int16Array",
"Int32Array",
"Uint8Array",
"Uint8ClampedArray",
"Uint16Array",
"Uint32Array",
"console",
"process",
"Buffer",
"undefined",
"null",
"Infinity",
"NaN",
"parseInt",
"parseFloat",
"isNaN",
"isFinite",
"encodeURI",
"encodeURIComponent",
"decodeURI",
"decodeURIComponent",
"eval",
"Function",
"Proxy",
"Reflect",
"Intl",
"BigInt",
]);
function normalizeListedExports(
exports: string[],
actualExports: Set<string>,
): string[] {
const normalized = new Set<string>();
for (const exp of exports) {
let candidate = exp.trim();
if (candidate.startsWith("call:") || candidate.startsWith("raise:"))
continue;
candidate = candidate.replace(/^(class|func|method):/, "");
candidate = candidate.split("(")[0]?.split(" ")[0] ?? candidate;
const match = candidate.match(/[A-Za-z_$][\w$]*/);
if (!match) continue;
const symbol = match[0];
if (
actualExports.has(symbol) ||
(/^[A-Z][A-Za-z0-9_$]*$/.test(symbol) && !BUILT_IN_GLOBALS.has(symbol)) ||
/^[A-Z0-9_]+$/.test(symbol)
) {
normalized.add(symbol);
}
}
return [...normalized];
}
function markRepairMode(
repairModes: Map<string, RepairMode>,
dirPath: string,
mode: RepairMode,
): void {
const existing = repairModes.get(dirPath);
if (!existing || existing === "small") {
repairModes.set(dirPath, mode);
}
}
function addRepairChain(
entries: DirectoryEntry[],
entry: DirectoryEntry,
mode: RepairMode,
plan: Map<string, ArtifactWriteMode>,
): void {
plan.set(entry.dirPath, "both");
const ctx = buildDirectoryContext(entries, entry);
let parent = ctx.parentMap.get(entry.relativePath);
while (parent) {
const ancestor = entries.find(
(candidate) => candidate.relativePath === parent,
);
if (!ancestor) break;
const desiredMode: ArtifactWriteMode = mode === "small" ? "index" : "both";
const existingMode = plan.get(ancestor.dirPath);
if (existingMode !== "both") {
plan.set(ancestor.dirPath, desiredMode);
}
parent = ctx.parentMap.get(parent);
}
}
function depthOfPath(dirPath: string): number {
return dirPath.split(/[\\/]/).filter(Boolean).length;
}
+37
View File
@@ -0,0 +1,37 @@
# tests (index)
dir: tests
## role
Comprehensive test suite for a project mapping tool that generates AI-readable codebase documentation with LLM integration, caching, and CLI query capabilities.
## parent
index: ./.pi-map.index.md
map: ./.pi-map.md
## children
-
## files
- ast-extract.test.ts
- cli.test.ts
- format.test.ts
- integration.test.ts
- llm-batch.test.ts
- llm-cache.test.ts
- llm-extract.test.ts
- llm-integration.test.ts
- merge.test.ts
- mock-llm.ts
- pi-extension.test.ts
- prompt-injection.test.ts
- retrieve.test.ts
- routing-metadata.test.ts
## links
index: tests/.pi-map.index.md
map: tests/.pi-map.md
## workflows
- change tests behavior
read: mock-llm.ts
- update tests tests
read: ast-extract.test.ts, cli.test.ts, format.test.ts
- change tests CLI
read: cli.test.ts
## dirty
-
+38
View File
@@ -0,0 +1,38 @@
# tests
dir: tests
index: tests/.pi-map.index.md
## role
Comprehensive test suite for a project mapping tool that generates AI-readable codebase documentation with LLM integration, caching, and CLI query capabilities.
## files
- ast-extract.test.ts | Tests AST extraction of TypeScript exports, imports, and dependency resolution with fallback for unsupported file types | dep: vitest, ../src/ast/ast-extract.js, fs, path, os, ast-extract.js
- cli.test.ts | Integration tests for a CLI tool that queries project context bundles from `.pi-map.md` and `.pi-map.index.md` files | dep: vitest, fs, path, os, child_process, url
- format.test.ts | Tests markdown rendering and parsing functions for package maps, directory maps, and directory indexes in a project mapping tool. | dep: vitest, ../src/format.js, ../src/directory-model.js
- integration.test.ts | Integration tests for a project mapping tool that generates and maintains .pi-map.md and .pi-map.index.md files across a codebase. | dep: vitest, fs, path, os, ../src/init.js, ../src/patch.js, ../src/validate.js, ./mock-llm.js, ../src/format.js
- llm-batch.test.ts | Unit tests for retry and batch processing utilities in an LLM module | dep: vitest, ../src/llm/llm-batch.js, ../src/llm-error.js
- llm-cache.test.ts | Tests a file-based caching system for LLM responses with get/set operations and cleanup. | dep: vitest, ../src/llm/llm-cache.js, fs, path, os
- llm-extract.test.ts | Unit tests for LLM-based file extraction with mock client, testing file size limits, binary detection, and response parsing | dep: vitest, ../src/llm/llm-extract.js, fs, path, os, ../src/llm/llm-client.js
- llm-integration.test.ts | Integration tests for LLM client functionality including Kimi API calls, file/package extraction, caching, parallel processing, and error handling | dep: vitest, fs, path, os, ../src/llm/llm-client.js, ../src/llm/llm-extract.js, ../src/llm/llm-batch.js, llm-client, llm-extract, llm-batch
- merge.test.ts | Tests that mergeFileData normalizes multi-line function/method parameters and return types into single-line export signatures | dep: vitest, ../src/merge.js
- mock-llm.ts | Provides mock LLM client implementations for testing purposes | exp: func:createMockFileClient(purpose) → LLMClient, func:createMockPackageClient() → LLMClient | dep: ../src/llm/llm-client.js, llm-client.js
- pi-extension.test.ts | Tests a Pi coding agent extension that manages project map initialization, patching, validation, reinitialization, and context retrieval with configurable prompt injection modes. | dep: vitest, fs, path, os, ../src/prompt-injection.js, ../pi-extension.js, @mariozechner/pi-coding-agent, @mariozechner/pi-ai, typebox
- prompt-injection.test.ts | Tests a prompt injection mitigation system that manages root-pair markers, context budgets, mode-based injection policies, and bypass detection for LLM interactions. | dep: vitest, fs, path, os, ../src/prompt-injection.js
- retrieve.test.ts | Tests the `retrieveContext` function that searches and ranks project map files to build context bundles for AI queries. | dep: vitest, fs, path, os, ../src/retrieve.js
- routing-metadata.test.ts | Tests the `populateRoutingMetadata` function which generates tags, symbols, and workflow hints from directory models for code navigation/routing purposes | dep: vitest, ../src/directory-model.js, ../src/routing-metadata.js
## arch
Layered testing architecture with unit, integration, and mock layers; uses file-based fixtures, mock LLM clients, and tests across AST extraction, LLM batching/caching, markdown rendering, context retrieval, and prompt injection security.
## tags
llm, js, src, tests, vitest, client, fs, path
## symbols
- createMockFileClient
- createMockPackageClient
## workflows
- change tests behavior
read: mock-llm.ts
- update tests tests
read: ast-extract.test.ts, cli.test.ts, format.test.ts
- change tests CLI
read: cli.test.ts
## dirty
-

Some files were not shown because too many files have changed in this diff Show More