feat: avoid duplicate project-map hint injection by checking context
The before_agent_start handler now scans the active session context via ctx.sessionManager.buildSessionContext() for an existing pi-project-map-hint custom message and skips injection when one is already present in the current branch. This prevents duplicate visible hints in advisory/pre-init modes and duplicate hidden hints in strong/strict modes. The hint is automatically re-injected after compaction or /tree navigation removes it from the active path. Also removes the unused hooks/on-prompt.ts prompt-text injector.
This commit is contained in:
Developer
@@ -5,5 +5,12 @@ coverage/
|
||||
.DS_Store
|
||||
.env
|
||||
.pi-map.md
|
||||
.pi-map.index.md
|
||||
# Local Pi runtime state
|
||||
.atl/
|
||||
.pi
|
||||
.cache/
|
||||
cache/
|
||||
subagent-outputs/
|
||||
IMPLEMENTATION_REPORT.md
|
||||
context.md
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
{}
|
||||
@@ -1,22 +1,174 @@
|
||||
# pi-project-map
|
||||
|
||||
Pi skill for hierarchical project analysis.
|
||||
> Pi skill and CLI for hierarchical project analysis.
|
||||
|
||||
## What it does
|
||||
`pi-project-map` generates and maintains paired, machine-readable analysis artifacts throughout a codebase so agents can navigate quickly, orient themselves, and then verify details from source.
|
||||
|
||||
Generates `.pi-map.md` files throughout your project — one per directory — containing a dense, machine-readable summary of that directory's files, exports, dependencies, and architecture. This gives Pi agents instant project comprehension without reading every source file.
|
||||
## What it is
|
||||
|
||||
## Quick Start
|
||||
For every non-ignored directory, the tool produces two files:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `.pi-map.index.md` | Routing-first index for deciding what to open next. |
|
||||
| `.pi-map.md` | Orientation-first rich map for understanding a directory. |
|
||||
|
||||
Together they form a **paired artifact model**:
|
||||
- indexes are small and routing-optimized
|
||||
- maps are denser and architecture-optimized
|
||||
- source remains the final authority
|
||||
|
||||
`pi-project-map` runs as both:
|
||||
- a **CLI** (`project-map`)
|
||||
- a **Pi extension** (`pi-extension.ts`) that registers tools and optional runtime prompt injection
|
||||
|
||||
## Quick start
|
||||
|
||||
Install:
|
||||
|
||||
```bash
|
||||
npm install -g pi-project-map
|
||||
```
|
||||
|
||||
Generate paired artifacts in a repo:
|
||||
|
||||
```bash
|
||||
cd my-project
|
||||
project-map init
|
||||
```
|
||||
|
||||
## Design
|
||||
For standalone CLI usage, provide an LLM provider/API key. For example:
|
||||
|
||||
See [design-doc.md](design-doc.md) for the full specification.
|
||||
```bash
|
||||
export OPENAI_API_KEY=...
|
||||
project-map init
|
||||
```
|
||||
|
||||
## Implementation Plan
|
||||
Inside Pi, the extension uses Pi's configured model automatically.
|
||||
|
||||
See [implementation-plan.md](implementation-plan.md) for the engineering roadmap.
|
||||
## Command overview
|
||||
|
||||
| CLI command | Pi tool | Purpose |
|
||||
|-------------|---------|---------|
|
||||
| `project-map init [path]` | `project_map_init` | Generate paired artifacts for the whole project or a subdirectory. |
|
||||
| `project-map patch <file>` | `project_map_patch` | Regenerate artifacts for the directory containing the changed file and refresh ancestors appropriately. |
|
||||
| `project-map validate [--fix]` | `project_map_validate` | Check paired artifacts for staleness or inconsistency. |
|
||||
| `project-map reinit [path]` | `project_map_reinit` | Force full regeneration of all artifacts. |
|
||||
| `project-map context <query>` | `project_map_context` | Return a ranked context bundle for a natural-language query. |
|
||||
|
||||
Typical workflow:
|
||||
|
||||
1. `project-map init` on first use
|
||||
2. after editing source, `project-map patch <changed-file>`
|
||||
3. before broad architectural decisions, `project-map validate`
|
||||
4. for targeted exploration, `project-map context "<query>"`
|
||||
|
||||
## Operating model
|
||||
|
||||
Follow a three-tier model when consuming project maps:
|
||||
|
||||
1. **Tier 0 — Protocol and root index**
|
||||
Start with the root `Project Map Protocol` and root `.pi-map.index.md`.
|
||||
2. **Tier 1 — Indexes and maps**
|
||||
Use indexes to route, then open the strongest-match `.pi-map.md` files for orientation.
|
||||
3. **Tier 2 — Source and tests**
|
||||
Read actual source, config, tests, and docs before editing or making exact runtime claims.
|
||||
|
||||
The trust boundary is always:
|
||||
|
||||
> **index routes, map orients, source decides.**
|
||||
|
||||
## Prompt injection policy
|
||||
|
||||
The Pi extension can automatically inject lightweight project-map guidance into the agent context. Behavior is controlled by `promptInjectionMode` in `.pi-project-map.json`.
|
||||
|
||||
### Before init
|
||||
|
||||
No synthetic map content is injected. The agent sees only a visible startup hint telling it to run `project_map_init`.
|
||||
|
||||
### After init
|
||||
|
||||
The root pair is guaranteed to load first:
|
||||
- root `.pi-map.index.md`
|
||||
- root `.pi-map.md`
|
||||
|
||||
Additional directory pairs are expanded only while the configured context budget allows.
|
||||
|
||||
### Mode ladder
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `off` | No automatic injection. |
|
||||
| `advisory` | Visible hints/reminders only; maps are read manually. |
|
||||
| `strong` | Root pair injection, budgeted expansion, reinjection on relevant turns. |
|
||||
| `strict` | Same as `strong`, plus a visible guard for sensitive turns when the protocol path is missing. |
|
||||
|
||||
The **protocol path** is present when outgoing context contains:
|
||||
- the canonical root-pair marker/block
|
||||
- the trust-boundary instruction
|
||||
|
||||
In `strict` mode, a sensitive action can be bypassed explicitly with:
|
||||
|
||||
```text
|
||||
[PI_MAP_BYPASS: <brief justification>]
|
||||
```
|
||||
|
||||
### Context budget
|
||||
|
||||
Default automatic-injection budget is the smaller of:
|
||||
- **15%** of the active model context window
|
||||
- **100,000 tokens** absolute cap
|
||||
|
||||
If the runtime cannot discover the model context window, it falls back to the absolute cap.
|
||||
|
||||
## Retrieval is separate
|
||||
|
||||
`project-map context <query>` and `project_map_context` are **separate, on-demand retrieval** paths. They do **not** replace automatic prompt injection.
|
||||
|
||||
Retrieval is deterministic and metadata-driven:
|
||||
1. score every directory's paired map/index metadata against the query
|
||||
2. keep the top matches (default: 3)
|
||||
3. return a compact markdown bundle with indexes, maps, likely files, and symbols
|
||||
|
||||
Use retrieval for targeted navigation when you already have a specific question.
|
||||
|
||||
## Configuration overview
|
||||
|
||||
Create `.pi-project-map.json` in the project root:
|
||||
|
||||
```json
|
||||
{
|
||||
"promptInjectionMode": "strong",
|
||||
"contextBudgetPercent": 15,
|
||||
"contextBudgetMaxTokens": 100000,
|
||||
"llmProvider": "openai",
|
||||
"llmModel": "gpt-4o-mini",
|
||||
"ignorePatterns": ["node_modules", ".git"],
|
||||
"tagCap": 8,
|
||||
"workflowHintCap": 5
|
||||
}
|
||||
```
|
||||
|
||||
Providing `ignorePatterns` replaces the built-in default list, so include any defaults you want to keep.
|
||||
|
||||
Key knobs:
|
||||
- `promptInjectionMode` — `off`, `advisory`, `strong`, `strict`
|
||||
- `contextBudgetPercent` — relative share of model context used for automatic injection
|
||||
- `contextBudgetMaxTokens` — hard absolute cap on automatic injection
|
||||
- `llmProvider` / `llmModel` / `llmBaseUrl` — standalone CLI provider settings
|
||||
- `ignorePatterns` — directories/files to skip
|
||||
- `tagCap` / `workflowHintCap` — routing metadata limits
|
||||
|
||||
## Documentation map
|
||||
|
||||
- [`SKILL.md`](SKILL.md) — skill definition and agent/operator instructions
|
||||
- [`usage-guide.md`](usage-guide.md) — practical workflows and examples
|
||||
- [`design-doc.md`](design-doc.md) — architecture and implementation details
|
||||
- [`troubleshooting.md`](troubleshooting.md) — common issues and recovery steps
|
||||
|
||||
## Known limitations
|
||||
|
||||
- token budgeting is best-effort, not tokenizer-exact
|
||||
- relevant-turn detection uses explicit event types plus heuristics
|
||||
- provider payload fallback depends on runtime serialization shapes
|
||||
- retrieval routes and orients; it never replaces source verification
|
||||
|
||||
@@ -1,108 +1,93 @@
|
||||
---
|
||||
name: pi-map
|
||||
description: Generates and maintains hierarchical, machine-readable project analysis files (.pi-map.md) for instant codebase comprehension. Use when working with medium-to-large codebases where understanding architecture, file relationships, and exports without reading every file is valuable. Automatically extracts symbols via AST and LLM heuristics.
|
||||
description: Generates and maintains hierarchical paired project-analysis artifacts (`.pi-map.index.md` + `.pi-map.md`) so Pi agents can navigate and orient in a codebase without reading every source file.
|
||||
---
|
||||
|
||||
# pi-project-map
|
||||
|
||||
A Pi skill that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file containing architectural context, exported symbols, and dependencies.
|
||||
A Pi skill that creates and maintains one paired analysis artifact set per non-ignored directory:
|
||||
|
||||
## What It Does
|
||||
- `.pi-map.index.md` — routing-first index
|
||||
- `.pi-map.md` — orientation-first rich map
|
||||
|
||||
- **Scans** your entire project and creates one `.pi-map.md` per directory
|
||||
- **Extracts** exports, imports, and dependencies via AST parsing (TypeScript, Python, Go) and LLM heuristics
|
||||
- **Updates** incrementally when files change (full rewrite for small packages, section-level patch for large)
|
||||
- **Validates** detects stale entries, missing files, orphaned entries, and changed signatures
|
||||
## What it does
|
||||
|
||||
## Quick Start
|
||||
- scans the project and emits one paired map/index set per directory
|
||||
- extracts per-file purpose, dependencies, and concepts via an LLM
|
||||
- extracts exact exports/imports via AST parsing where supported
|
||||
- patches artifacts incrementally after source edits
|
||||
- validates stale, missing, broken, or inconsistent paired artifacts
|
||||
- retrieves relevant context on demand via deterministic metadata scoring
|
||||
- injects lightweight project-map guidance into agent context according to a configurable mode ladder
|
||||
|
||||
```bash
|
||||
# Install globally
|
||||
npm install -g pi-project-map
|
||||
## Operating model
|
||||
|
||||
# Generate analysis files for the entire project
|
||||
project-map init
|
||||
### Tier 0 — protocol
|
||||
Always read the root `.pi-map.index.md` and the `Project Map Protocol` first.
|
||||
|
||||
# After editing a file, update its directory's analysis
|
||||
project-map patch src/components/Button.tsx
|
||||
### Tier 1 — routing
|
||||
Use indexes first to decide where to go next. Open the strongest-match `.pi-map.md` files for orientation.
|
||||
|
||||
# Check for staleness
|
||||
project-map validate
|
||||
### Tier 2 — source
|
||||
Read actual source, tests, config, and docs before editing or asserting exact runtime behavior.
|
||||
|
||||
# Force full regeneration
|
||||
project-map reinit
|
||||
```
|
||||
**Trust boundary:**
|
||||
|
||||
## Format
|
||||
> **index routes, map orients, source decides**
|
||||
|
||||
Each `.pi-map.md` uses dense markdown optimized for LLM consumption:
|
||||
## Agent instructions
|
||||
|
||||
```markdown
|
||||
# pkg/auth
|
||||
## role
|
||||
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
|
||||
## files
|
||||
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
|
||||
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
|
||||
## arch
|
||||
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
|
||||
## dirty
|
||||
-
|
||||
```
|
||||
When project-map artifacts exist in the repo:
|
||||
|
||||
### Abbreviations
|
||||
1. start with the root `.pi-map.index.md` and the `Project Map Protocol`
|
||||
2. use indexes first to route into the right directory
|
||||
3. read the local `.pi-map.md` plus relevant source before editing
|
||||
4. run `project_map_patch <file>` (tool) or `project-map patch <file>` (CLI) after each source edit
|
||||
5. run `project_map_validate` (tool) or `project-map validate` (CLI) before freshness-sensitive architectural decisions or final handoff
|
||||
6. for targeted exploration, use `project_map_context <query>` (tool) or `project-map context <query>` (CLI)
|
||||
7. in `strict` mode, only bypass the protocol-path guard with an explicit marker: `[PI_MAP_BYPASS: <brief justification>]`
|
||||
|
||||
| Abbreviation | Meaning |
|
||||
|-------------|---------|
|
||||
| `exp:` | Exported symbols |
|
||||
| `dep:` | Dependencies |
|
||||
| `pkg/` | Internal package reference |
|
||||
## Prompt injection modes
|
||||
|
||||
## Tools
|
||||
The Pi extension can inject project-map guidance automatically. Behavior is controlled by `promptInjectionMode` in `.pi-project-map.json`.
|
||||
|
||||
### `project-map:init [root]`
|
||||
Runs a full project scan and generates `.pi-map.md` files in every directory.
|
||||
### Before init
|
||||
If no `.pi-map.md` / `.pi-map.index.md` artifacts exist, the extension emits a lightweight visible hint to run `project_map_init`. No synthetic or fake map content is injected.
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
project-map init
|
||||
project-map init ~/my-project
|
||||
```
|
||||
### After init
|
||||
Once real artifacts exist, the runtime guarantees that the **root pair** is loaded first:
|
||||
|
||||
### `project-map:patch <file-path>`
|
||||
Updates the `.pi-map.md` for the directory containing the given file.
|
||||
- root `.pi-map.index.md`
|
||||
- root `.pi-map.md`
|
||||
|
||||
**Behavior:**
|
||||
- Small packages (< 10 files): full rewrite
|
||||
- Large packages (>= 10 files): section-level patch
|
||||
Additional directory pairs may be expanded while the configured context budget allows, in shallow-first order.
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
project-map patch src/components/Button.tsx
|
||||
```
|
||||
### Mode ladder
|
||||
|
||||
### `project-map:validate [root]`
|
||||
Checks all `.pi-map.md` files for staleness.
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `off` | No automatic injection. Use tools/CLI manually. |
|
||||
| `advisory` | Startup/init hints are shown. Root pair is not auto-loaded; read it manually when needed. |
|
||||
| `strong` (default) | Root pair is auto-loaded, expansion stays within budget, and reinjection runs on relevant turns. |
|
||||
| `strict` | Same as `strong`, but sensitive edits or architectural claims are guarded unless the protocol path is present or a bypass marker is provided. |
|
||||
|
||||
**Detects:**
|
||||
- Missing files (new files not yet in `.pi-map.md`)
|
||||
- Orphaned entries (files listed but deleted)
|
||||
- Stale signatures (exports changed since last scan)
|
||||
- Dirty markers (packages flagged for reconciliation)
|
||||
The **protocol path** means the outgoing context contains the canonical injected root-pair block and the trust-boundary text.
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
project-map validate
|
||||
```
|
||||
### Context budget
|
||||
Default budget: **15% of the active model context window**, capped at **100k tokens**. The smaller of the relative and absolute values wins. If the runtime cannot discover the context window, it uses the absolute cap.
|
||||
|
||||
### `project-map:reinit [path]`
|
||||
Force full re-initialization. Clears all dirty markers.
|
||||
## Retrieval usage
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
project-map reinit
|
||||
project-map reinit src/components
|
||||
```
|
||||
`project_map_context` (tool) and `project-map context` (CLI) are **on-demand retrieval**, separate from automatic injection.
|
||||
|
||||
They do not call the LLM. They score paired metadata against the query and return a compact markdown bundle with:
|
||||
|
||||
- relevant indexes
|
||||
- relevant maps
|
||||
- likely files
|
||||
- relevant symbols when useful
|
||||
|
||||
Always read the suggested indexes first, then maps, then verify critical behavior from source.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -110,43 +95,32 @@ Create `.pi-project-map.json` in the project root:
|
||||
|
||||
```json
|
||||
{
|
||||
"ignorePatterns": ["node_modules", ".git"],
|
||||
"smallPackageThreshold": 10,
|
||||
"contextBudget": 4000,
|
||||
"autoInjectPrompt": true
|
||||
"promptInjectionMode": "strong",
|
||||
"contextBudgetPercent": 15,
|
||||
"contextBudgetMaxTokens": 100000,
|
||||
"tagCap": 8,
|
||||
"workflowHintCap": 5,
|
||||
"llmProvider": "openai",
|
||||
"llmModel": "gpt-4o-mini",
|
||||
"ignorePatterns": ["node_modules", ".git", "dist", "build"]
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `ignorePatterns` | `node_modules`, `.git`, `dist`, etc. | Additional ignore patterns |
|
||||
| `smallPackageThreshold` | `10` | File count threshold for full rewrite vs patch |
|
||||
| `contextBudget` | `4000` | Max tokens to spend on analysis files |
|
||||
| `autoInjectPrompt` | `true` | Auto-inject maintenance instructions |
|
||||
Providing `ignorePatterns` replaces the built-in default list, so include any defaults you want to keep.
|
||||
|
||||
## Agent Instructions
|
||||
Knobs that matter in practice:
|
||||
- `promptInjectionMode` — `off` | `advisory` | `strong` | `strict`
|
||||
- `contextBudgetPercent` / `contextBudgetMaxTokens` — caps automatic map/index injection
|
||||
- `tagCap` / `workflowHintCap` — caps routing metadata per directory
|
||||
- `llmProvider` / `llmModel` / `llmBaseUrl` — standalone CLI only
|
||||
- `ignorePatterns` — discovery exclusions
|
||||
|
||||
When `.pi-map.md` files exist in the project:
|
||||
## Tools and commands
|
||||
|
||||
1. **Read them at session start** to build project understanding without scanning every file
|
||||
2. **Run `project-map:patch <path>`** after editing any source file
|
||||
3. **Run `project-map:validate`** if you suspect staleness before making architectural decisions
|
||||
4. **Trust the analysis** for orientation, but verify critical details by reading source when needed
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Run `project-map:init` after cloning a new repository
|
||||
- Run `project-map:reinit` periodically (daily/weekly) to catch changes made outside the agent
|
||||
- Add `.pi-map.md` to `.gitignore` — they are derived artifacts
|
||||
- For very large projects (> 1000 directories), consider running `init` on subdirectories
|
||||
|
||||
## Supported Languages
|
||||
|
||||
| Language | AST Parsing | Heuristic Extraction |
|
||||
|----------|------------|---------------------|
|
||||
| TypeScript / TSX | Full | Full |
|
||||
| JavaScript / JSX | Full | Full |
|
||||
| Python | Partial | Full |
|
||||
| Go | Partial | Full |
|
||||
| Rust | Partial | Full |
|
||||
| Other | - | Full (filename + regex patterns) |
|
||||
| Tool (Pi) | CLI command | Purpose |
|
||||
|-----------|-------------|---------|
|
||||
| `project_map_init` | `project-map init [path]` | Generate all paired artifacts. |
|
||||
| `project_map_patch` | `project-map patch <file>` | Regenerate the pair for the changed file's directory and refresh ancestors. |
|
||||
| `project_map_validate` | `project-map validate [--fix]` | Check paired artifacts for staleness and discrepancies; optionally repair. |
|
||||
| `project_map_reinit` | `project-map reinit [path]` | Force full regeneration. |
|
||||
| `project_map_context` | `project-map context <query>` | Retrieve a ranked context bundle for a natural-language query. |
|
||||
|
||||
+290
-306
@@ -1,351 +1,335 @@
|
||||
# Design Doc: Hierarchical Project Analysis Skill for Pi
|
||||
# Design Reference: pi-project-map
|
||||
|
||||
## 1. Goals and Success Criteria
|
||||
> Audience: maintainers and contributors.
|
||||
> Purpose: explain how `pi-project-map` works internally, not how to install or use it.
|
||||
|
||||
### Primary Goal
|
||||
Enable a Pi coding agent to understand a software project's architecture and code relationships without scanning the entire repository. The agent should have a compact, hierarchical "internal representation" of the project that it can consume in-context.
|
||||
## 1. Overview
|
||||
|
||||
### Success Criteria
|
||||
- The agent can orient itself in a new or familiar project without reading dozens of source files.
|
||||
- The agent understands cross-package dependencies, data flows, and architectural patterns from the analysis files alone.
|
||||
- Analysis files stay sufficiently fresh that the agent does not make decisions based on stale information.
|
||||
- The representation is token-dense: maximum information per token, optimized for LLM consumption, not human readability.
|
||||
`pi-project-map` is a TypeScript/Node.js skill package that generates and maintains hierarchical, paired project-map artifacts for AI coding agents:
|
||||
|
||||
## 2. Format Specification: Dense Markdown with Conventions
|
||||
- `.pi-map.index.md` — routing-first, sparse directory metadata
|
||||
- `.pi-map.md` — orientation-first, richer directory metadata
|
||||
|
||||
### Design Rationale
|
||||
- **Not JSON/YAML**: Brackets, quotes, and indentation add token overhead with no benefit to LLM comprehension.
|
||||
- **Not a custom DSL**: Fragile, requires a parser, and LLMs may hallucinate syntax.
|
||||
- **Dense markdown**: Hierarchical headings, bullet points, and abbreviations are natively understood by LLMs and extremely token-efficient.
|
||||
It runs as both:
|
||||
- a standalone CLI (`project-map`)
|
||||
- a Pi extension (`pi-extension.ts`)
|
||||
|
||||
### Structure
|
||||
Each directory in the project gets one analysis file named `.pi-map.md` (hidden by default, excluded from git via `.gitignore`).
|
||||
The extension registers tools and event hooks that keep the artifacts fresh and can inject them into agent context at runtime.
|
||||
|
||||
```markdown
|
||||
# <relative-path>
|
||||
## role
|
||||
<one-line package role> | Dep: <comma-separated upstream deps>
|
||||
## files
|
||||
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
|
||||
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
|
||||
## arch
|
||||
<free-form architectural notes: patterns, data flow, invariants, design decisions>
|
||||
## dirty
|
||||
<timestamp or flag indicating staleness>
|
||||
```
|
||||
### Core design principle
|
||||
|
||||
### Abbreviation Conventions
|
||||
| Abbreviation | Meaning |
|
||||
|-------------|---------|
|
||||
| `exp:` | exported symbols (functions, classes, types, constants) |
|
||||
| `dep:` | dependencies (other packages, files, or external libs) |
|
||||
| `pkg/` | project-internal package reference |
|
||||
| `ext/` | external dependency reference |
|
||||
| `->` | data flow direction |
|
||||
| `|` | field delimiter within a line |
|
||||
The artifacts are **navigation aids, not source-of-truth**. Source code is always the final authority.
|
||||
|
||||
### Example
|
||||
> **index routes, map orients, source decides.**
|
||||
|
||||
```markdown
|
||||
# pkg/auth
|
||||
## role
|
||||
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
|
||||
## files
|
||||
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
|
||||
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
|
||||
- types.ts | shared auth types | exp: AuthToken, UserClaims, Role
|
||||
## arch
|
||||
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
|
||||
Session state stored in Redis via db/sessions. No server-side JWT storage.
|
||||
## dirty
|
||||
-
|
||||
```
|
||||
## 2. Artifact model
|
||||
|
||||
### Rules
|
||||
- One file per directory, placed inside that directory.
|
||||
- Every non-excluded file in the directory gets one bullet under `## files`.
|
||||
- Subdirectories are referenced in `## role` via `Dep:` or in `## arch` as structural notes, not duplicated.
|
||||
- The `## dirty` section is empty (`-`) when clean, or contains a timestamp/flag when stale.
|
||||
Each non-ignored directory receives a matched pair.
|
||||
|
||||
## 3. Pipeline Architecture
|
||||
### 2.1 Shared model
|
||||
|
||||
### Hybrid Extraction: LLM + AST
|
||||
Both files are generated from the same in-memory `DirectoryArtifactModel`:
|
||||
|
||||
Two independent extraction layers contribute to the same output file.
|
||||
```ts
|
||||
interface DirectoryArtifactModel {
|
||||
dir: string;
|
||||
role: string;
|
||||
files: FileEntry[];
|
||||
arch: string;
|
||||
dirty?: string;
|
||||
isRoot: boolean;
|
||||
parent?: string;
|
||||
children: string[];
|
||||
tags: string[];
|
||||
symbols: string[];
|
||||
workflows: WorkflowHint[];
|
||||
}
|
||||
|
||||
#### Layer 1: LLM-Based Extraction (All Files)
|
||||
- **Input**: Raw file contents of every non-excluded file in the directory.
|
||||
- **Output**: Purpose description, architectural role, and cross-file relationships.
|
||||
- **Applies to**: Code files, config files, Dockerfiles, READMEs, YAML, JSON, shell scripts — everything.
|
||||
- **When it runs**: Once per file during init; again on changed files during patching.
|
||||
- **Implementation**: Calls an actual LLM (not regex heuristics). Inside Pi, it uses Pi's built-in LLM via the ExtensionAPI. Standalone CLI falls back to an external LLM API (OpenAI-compatible).
|
||||
|
||||
#### Layer 2: AST-Based Extraction (Code Files Only)
|
||||
- **Input**: Source code of files where a tree-sitter or LSP parser is available.
|
||||
- **Output**: Precise symbol lists (functions, classes, types), signatures, import/export graphs, class hierarchies.
|
||||
- **Applies to**: Supported languages only (TypeScript, Python, Go, Rust, etc.).
|
||||
- **When it runs**: Once per file during init; again on changed files during patching.
|
||||
|
||||
#### Merging
|
||||
The two layers merge into a single line per file under `## files`:
|
||||
|
||||
```
|
||||
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
|
||||
^ LLM ^ LLM ^ AST ^ AST + LLM
|
||||
```
|
||||
|
||||
- File name and purpose: LLM.
|
||||
- Exported symbols and signatures: AST (augmented by LLM if AST unavailable).
|
||||
- Dependency list: AST for imports; LLM for inferred architectural dependencies.
|
||||
|
||||
### LLM Client Architecture
|
||||
|
||||
The LLM client is abstracted behind a unified interface:
|
||||
|
||||
```typescript
|
||||
interface LLMClient {
|
||||
complete(prompt: string): Promise<string>;
|
||||
interface FileEntry {
|
||||
name: string;
|
||||
purpose: string;
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
}
|
||||
```
|
||||
|
||||
Two implementations:
|
||||
### 2.2 `.pi-map.md` (rich map)
|
||||
|
||||
1. **PiLLMClient** (Pi extension): Uses `ctx.model` or `ctx.modelRegistry` to invoke Pi's configured LLM. Called from `pi-extension.ts` when the skill runs inside Pi.
|
||||
2. **ExternalLLMClient** (standalone CLI): Calls an external OpenAI-compatible API. Configured via environment variable (e.g., `OPENAI_API_KEY`) or config file.
|
||||
Rendered by `src/format.ts` → `renderDirectoryMap()`.
|
||||
|
||||
### Caching
|
||||
Contains:
|
||||
- `dir:` line and sibling `index:` link
|
||||
- `Project Map Protocol` (root only)
|
||||
- `## role`
|
||||
- `## files`
|
||||
- `## arch`
|
||||
- `## tags`
|
||||
- `## symbols`
|
||||
- `## workflows`
|
||||
- `## dirty`
|
||||
|
||||
LLM results are cached to avoid re-querying unchanged files.
|
||||
### 2.3 `.pi-map.index.md` (index)
|
||||
|
||||
- **Key**: SHA-256 hash of file contents.
|
||||
- **Storage**: JSON file at `~/.cache/pi-project-map/llm-cache.json`.
|
||||
- **Behavior**: Before calling the LLM, compute the file hash and check the cache. If hit, reuse the cached result. If miss, call the LLM and store the result.
|
||||
- **Invalidation**: Cache entries are implicitly invalidated when the file content changes (because the hash changes). There is no TTL; the cache is append-only.
|
||||
Rendered by `src/format.ts` → `renderDirectoryIndex()`.
|
||||
|
||||
### Parallelization and Rate Limiting
|
||||
Contains:
|
||||
- same protocol (root only)
|
||||
- `## role`
|
||||
- `## parent`
|
||||
- `## children`
|
||||
- `## files`
|
||||
- `## links`
|
||||
- `## workflows`
|
||||
- `## dirty`
|
||||
|
||||
- **Concurrency**: 4-8 LLM calls in parallel, controlled by `p-limit`.
|
||||
- **Batch delays**: A small delay (e.g., 100ms) is inserted between batches to avoid triggering rate limits.
|
||||
- **Retry policy**: Each LLM call retries up to 3 times with exponential backoff (1s, 2s, 4s). If all retries fail, the entire operation stops with a hard error.
|
||||
### 2.4 Why a paired format?
|
||||
|
||||
### Error Handling
|
||||
- **indexes are cheap** — many can be loaded without consuming much context
|
||||
- **maps are dense** — loaded only after an index suggests relevance
|
||||
- **paired generation** guarantees structural consistency
|
||||
|
||||
- **Hard error on failure**: If an LLM call fails after all retries, `init` or `patch` stops immediately and prints a clear error. There is no heuristic fallback. The user must resolve the issue (set API key, wait for rate limit, check network).
|
||||
- **Context limit protection**: Files larger than the LLM's context window are truncated from the end (with a note in the prompt) before being sent.
|
||||
## 3. High-level architecture
|
||||
|
||||
### Init Pipeline
|
||||
```
|
||||
For each directory (depth-first):
|
||||
1. List all non-excluded files.
|
||||
2. For each file (parallel, 4-8 concurrent):
|
||||
a. Compute SHA-256 of file contents.
|
||||
b. Check disk cache. If hit, use cached result.
|
||||
c. If miss: call LLM (with retries/backoff) to extract purpose and role.
|
||||
d. Store result in cache.
|
||||
e. If code file + parser available: run AST extraction (symbols, imports).
|
||||
3. Merge per-file outputs into lines.
|
||||
4. Run LLM on merged lines + directory context to generate:
|
||||
- `## role` (package-level summary)
|
||||
- `## arch` (architectural notes)
|
||||
5. Write `.pi-map.md` to directory.
|
||||
### 3.1 Main modules
|
||||
|
||||
```text
|
||||
discover → directory tree, .gitignore-aware
|
||||
init → full generation
|
||||
llm-extract → LLM-based file/package analysis
|
||||
ast-extract → tree-sitter parsing
|
||||
merge → combine LLM + AST into FileEntry
|
||||
routing-metadata → tags, symbols, workflow hints
|
||||
format → render / parse the markdown pair
|
||||
patch → incremental update after edits
|
||||
validate → consistency checking with optional repair
|
||||
retrieve → deterministic query scoring
|
||||
prompt-injection → runtime context policy
|
||||
pi-extension → Pi tool/event registration
|
||||
cli → standalone command dispatcher
|
||||
config → defaults and .pi-project-map.json loader
|
||||
```
|
||||
|
||||
### Patch Pipeline
|
||||
```
|
||||
When agent edits file(s) in directory:
|
||||
1. Determine patch strategy:
|
||||
- If directory has < 10 files: full rewrite.
|
||||
- Else: section-level patch for changed file(s) only.
|
||||
2. For each changed file:
|
||||
a. Recompute SHA-256.
|
||||
b. Check cache. If miss or stale, call LLM with retries/backoff.
|
||||
3. Re-run AST extraction on changed file(s) if applicable.
|
||||
4. Update `## files` section (rewrite or patch).
|
||||
5. Update `## dirty` flag if full regeneration is deferred.
|
||||
### 3.2 Runtime modes
|
||||
|
||||
| Mode | Entry point | LLM client |
|
||||
|------|-------------|------------|
|
||||
| Pi extension | `pi-extension.ts` | `PiLLMClient` via Pi runtime |
|
||||
| Standalone CLI | `src/cli/cli.ts` | `ExternalLLMClient` or `KimiLLMClient` |
|
||||
|
||||
## 4. Extraction pipeline
|
||||
|
||||
### 4.1 Discovery
|
||||
|
||||
`src/discover.ts` walks the filesystem with `ignore`, merging built-in exclusions and `.gitignore`.
|
||||
|
||||
### 4.2 Per-directory generation
|
||||
|
||||
`src/init.ts` → `generateDirectoryArtifacts()`:
|
||||
|
||||
1. `processFiles()` runs in parallel over directory files
|
||||
2. for each file:
|
||||
- `extractFileLLM()` gets `purpose`, `deps`, `concepts`
|
||||
- `extractFileAST()` gets exports/imports/calls where possible
|
||||
- `mergeFileData()` combines both into a `FileEntry`
|
||||
3. `extractPackageLLM()` produces directory `role` and `arch`
|
||||
4. `createDirectoryModel()` builds the shared model
|
||||
5. `populateRoutingMetadata()` derives `tags`, `symbols`, `workflows`
|
||||
6. `writeDirectoryArtifacts()` writes both `.pi-map.md` and `.pi-map.index.md`
|
||||
|
||||
Directories are processed sequentially; files within a directory are processed concurrently.
|
||||
|
||||
### 4.3 LLM extraction
|
||||
|
||||
`src/llm/llm-extract.ts`:
|
||||
- prompts are minimal and line-oriented
|
||||
- binary files are skipped
|
||||
- files over 500KB are labeled large and skipped
|
||||
- source is truncated before prompting
|
||||
- results are cached by SHA-256 of file content
|
||||
- missing client throws `LLMError`
|
||||
|
||||
### 4.4 AST extraction
|
||||
|
||||
`src/ast/ast-extract.ts` uses `tree-sitter` for supported languages to extract:
|
||||
- imports / requires
|
||||
- exported classes, functions, constants
|
||||
- methods, parameters, return types
|
||||
- direct calls and raised exceptions
|
||||
|
||||
Unsupported languages fall back to LLM-only extraction.
|
||||
|
||||
### 4.5 Merging
|
||||
|
||||
`src/merge.ts`:
|
||||
- purpose/concepts come from the LLM
|
||||
- exports come from AST when available
|
||||
- rich AST symbols are encoded as compact DSL:
|
||||
- `class:Foo`
|
||||
- `method:bar(a: string) → number`
|
||||
- `call:baz`
|
||||
- `raise:Error`
|
||||
- deps are deduplicated union of AST + LLM deps
|
||||
|
||||
### 4.6 Routing metadata
|
||||
|
||||
`src/routing-metadata.ts` generates deterministic metadata used by retrieval and injection:
|
||||
- **tags**
|
||||
- **symbols**
|
||||
- **workflow hints**
|
||||
|
||||
Caps are configurable via `tagCap` and `workflowHintCap`.
|
||||
|
||||
## 5. Patch / validate / reinit behavior
|
||||
|
||||
### 5.1 Patch
|
||||
|
||||
`src/patch.ts`:
|
||||
1. resolve directory containing changed file
|
||||
2. rediscover project tree
|
||||
3. regenerate changed directory pair
|
||||
4. refresh ancestors according to patch mode
|
||||
|
||||
Patch mode:
|
||||
- **small** — refresh ancestor indexes only
|
||||
- **structural** — refresh ancestor map/index pairs
|
||||
|
||||
### 5.2 Validate
|
||||
|
||||
`src/validate.ts` compares artifacts against filesystem and AST.
|
||||
|
||||
Important discrepancy types:
|
||||
- `missing`
|
||||
- `orphaned`
|
||||
- `stale-signature`
|
||||
- `dirty`
|
||||
- `stale-map`
|
||||
- `stale-index`
|
||||
- `broken-link`
|
||||
- `structural`
|
||||
|
||||
With `--fix`, validate builds a repair plan and regenerates directories deepest-first.
|
||||
|
||||
### 5.3 Reinit
|
||||
|
||||
`reinitPath()` is the blunt instrument for widespread staleness.
|
||||
|
||||
## 6. Retrieval architecture
|
||||
|
||||
`src/retrieve.ts` implements deterministic, index-first context retrieval.
|
||||
|
||||
1. walk the project for paired artifacts
|
||||
2. parse indexes/maps into `DirectoryArtifactModel`
|
||||
3. normalize the query
|
||||
4. score every directory
|
||||
5. return top-K (default: 3) as a markdown bundle with:
|
||||
- relevant indexes
|
||||
- relevant maps
|
||||
- likely files
|
||||
- relevant symbols
|
||||
- instructions to verify from source
|
||||
|
||||
No LLM is used during retrieval. It is intentionally separate from automatic prompt injection.
|
||||
|
||||
## 7. Prompt injection architecture
|
||||
|
||||
`src/prompt-injection.ts` and `pi-extension.ts` implement runtime guidance injection.
|
||||
|
||||
### 7.1 Mode ladder
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `off` | No automatic injection |
|
||||
| `advisory` | Visible startup/init hints; no artifact preload |
|
||||
| `strong` (default) | Root pair preloaded, budgeted expansion, reinjection on relevant turns |
|
||||
| `strict` | Same as strong, plus bypass guard for sensitive edits/architecture reasoning without protocol path |
|
||||
|
||||
### 7.2 Event hooks
|
||||
|
||||
The extension currently registers:
|
||||
- `session_start`
|
||||
- `before_agent_start`
|
||||
- `context`
|
||||
|
||||
Payload fallback scanning is handled inside `context`-level decision logic; there is no separately registered `before_provider_request` hook in the current implementation.
|
||||
|
||||
### 7.3 Reinjection policy
|
||||
|
||||
`shouldReinjectForEvent()` decides whether to inject:
|
||||
- only active in `strong` or `strict`
|
||||
- skips if the canonical marker is already present in outgoing messages or payload
|
||||
- triggers on:
|
||||
- `agent_start`
|
||||
- `edit_intent`
|
||||
- `architecture_sensitive`
|
||||
- `compaction`
|
||||
- `artifact_change`
|
||||
- `artifact_change` always forces reinjection
|
||||
|
||||
`detectEditIntent()` and `detectArchitectureSensitiveReasoning()` provide heuristic fallback for generic turns.
|
||||
|
||||
In addition to marker-based deduplication, `before_agent_start` scans the active session context via `ctx.sessionManager.buildSessionContext()` for an existing `pi-project-map-hint` custom message. If one is already present in the current branch, the handler skips injection entirely. This prevents duplicate visible hints in advisory/pre-init modes and duplicate hidden hints in strong/strict modes when the session context already contains the guidance. The hint is automatically re-injected after compaction or `/tree` navigation removes it from the active path.
|
||||
|
||||
### 7.4 Protocol path and strict bypass
|
||||
|
||||
The **protocol path** is present when outgoing context contains:
|
||||
1. the canonical root-pair marker/block
|
||||
2. the trust-boundary text
|
||||
|
||||
In `strict` mode, a sensitive turn without the protocol path is blocked with a visible guard. The agent can override with:
|
||||
|
||||
```text
|
||||
[PI_MAP_BYPASS: brief justification]
|
||||
```
|
||||
|
||||
## 4. LLM Prompt Design
|
||||
Empty or whitespace reasons are rejected.
|
||||
|
||||
### File-Level Prompt
|
||||
### 7.5 Budgeted expansion
|
||||
|
||||
The LLM prompt for a single file is designed to produce a structured, concise analysis.
|
||||
`buildInjectionPayload()`:
|
||||
- computes budget as `min(relative, absolute)`
|
||||
- default is 15% of context window, capped at 100k tokens
|
||||
- always includes the root pair
|
||||
- adds additional pairs shallow-first until budget is exhausted
|
||||
- prepends a maintenance reminder
|
||||
|
||||
```
|
||||
You are analyzing a source file for a project map. Read the file below and summarize:
|
||||
Token estimation is best-effort: `ceil(char_count / 4)`.
|
||||
|
||||
1. PURPOSE: What does this file do? Describe its role in the project (2-3 sentences max).
|
||||
2. DEPENDENCIES: What does this file depend on? List internal modules/packages and external libraries.
|
||||
3. KEY CONCEPTS: Mention any important patterns, algorithms, or domain concepts.
|
||||
### 7.6 Context-window discovery
|
||||
|
||||
File path: <file-path>
|
||||
`discoverContextWindow()` inspects the Pi runtime model for context metadata and falls back to the absolute cap when unavailable.
|
||||
|
||||
```
|
||||
<file-contents-truncated>
|
||||
```
|
||||
## 8. Known limits and tradeoffs
|
||||
|
||||
Respond in this exact format:
|
||||
PURPOSE: <concise description>
|
||||
DEPS: <comma-separated list, or "none">
|
||||
CONCEPTS: <comma-separated list, or "none">
|
||||
```
|
||||
### Correctness vs cost
|
||||
- init/patch/repair make LLM calls
|
||||
- large repositories can be expensive
|
||||
- caching reduces duplicate work
|
||||
|
||||
### Package-Level Prompt
|
||||
### AST coverage
|
||||
- TypeScript/TSX, Python, and Go have the richest support
|
||||
- other languages may be partial or LLM-only
|
||||
|
||||
After all file summaries are collected for a directory, a second LLM call synthesizes the package role and architecture.
|
||||
### Token estimation
|
||||
- 4 chars/token is only a heuristic
|
||||
- oversized files may be truncated or skipped
|
||||
|
||||
```
|
||||
You are analyzing a directory in a software project. Below is a list of files in this directory with their purposes.
|
||||
### Staleness
|
||||
- there is no filesystem watcher
|
||||
- maps go stale when edits happen outside the patch flow
|
||||
- validate detects but does not prevent staleness
|
||||
|
||||
Directory: <dir-path>
|
||||
Files:
|
||||
- <file1>: <purpose1>
|
||||
- <file2>: <purpose2>
|
||||
...
|
||||
### Patch mode inference
|
||||
- auto-mode heuristics are good but imperfect
|
||||
- contributors can force structural mode when needed
|
||||
|
||||
Respond in this exact format:
|
||||
ROLE: <one-line description of this directory's role in the project>
|
||||
ARCH: <2-4 sentences describing architecture, data flow, patterns, and design decisions>
|
||||
```
|
||||
### Strict mode ergonomics
|
||||
- strict guards can be surprising on casual phrasing
|
||||
- bypass markers are intentionally explicit and user-visible
|
||||
|
||||
### Output Parsing
|
||||
### Retrieval scoring
|
||||
- deterministic scoring is reproducible but not semantic-search-smart
|
||||
- broader queries may still need manual browsing
|
||||
|
||||
The LLM client's response is parsed to extract `PURPOSE`, `DEPS`, `CONCEPTS`, `ROLE`, and `ARCH` fields. These are merged with AST data into the final `.pi-map.md` format.
|
||||
|
||||
### Context Limit Protection
|
||||
|
||||
- Files are truncated from the end if they exceed a configurable max token budget (default: 4000 tokens of source).
|
||||
- A marker `[...truncated]` is appended to the truncated content so the LLM knows it is not seeing the full file.
|
||||
- Very large binary or generated files are skipped entirely for LLM analysis (they still appear in `.pi-map.md` with a note like "Large/generated file").
|
||||
|
||||
## 5. Consumption Model
|
||||
|
||||
### Session Start
|
||||
1. Agent discovers all `.pi-map.md` files (e.g., via `find . -name ".pi-map.md"`).
|
||||
2. Agent reads **all** files into context. This is a one-time cost at session start.
|
||||
3. Agent constructs an internal mental model of the project hierarchy.
|
||||
|
||||
### During Session
|
||||
- An **auto-injected summary** stays in context (e.g., a condensed top-level `.pi-map.md` or a synthesized project overview).
|
||||
- When the agent needs deeper detail about a specific package, it already has the full `.pi-map.md` in memory from step 2.
|
||||
- If the agent enters a new package not yet loaded, it reads that package's `.pi-map.md` on demand.
|
||||
|
||||
### Context Management
|
||||
- For very large projects, the agent may summarize or prune the initial read, keeping only the top N levels of the hierarchy in active context.
|
||||
- The skill can provide a "context budget" parameter: max tokens to spend on analysis files.
|
||||
|
||||
## 6. Stale Data Mitigation
|
||||
|
||||
### Combined Strategy
|
||||
|
||||
#### 5.1 Dirty Markers
|
||||
- Whenever the agent edits a file, it appends a dirty flag to the directory's `.pi-map.md`:
|
||||
```markdown
|
||||
## dirty
|
||||
2024-06-09T14:32:00Z: tokens.ts modified
|
||||
```
|
||||
- A background or post-session reconciliation step regenerates dirty files.
|
||||
- The agent can also be instructed to reconcile before making architectural decisions.
|
||||
|
||||
#### 5.2 Periodic Full Re-init
|
||||
- On every new session start, or on a configurable schedule (e.g., daily), the skill offers to run a full re-scan.
|
||||
- This catches any changes made outside the agent's awareness (e.g., by other developers).
|
||||
|
||||
#### 5.3 Validation Command
|
||||
- A `validate` tool/command that the agent can invoke:
|
||||
- Checks for missing files (new files not in `.pi-map.md`).
|
||||
- Checks for orphaned entries (files listed but deleted).
|
||||
- Checks for changed signatures (AST mismatch between listed symbols and actual code).
|
||||
- Reports discrepancies and suggests corrections.
|
||||
|
||||
### Recovery
|
||||
- If validation finds staleness beyond a threshold (e.g., > 3 dirty packages), the skill recommends a full re-init.
|
||||
- The agent can also trigger re-init for a specific subtree.
|
||||
|
||||
## 7. Scope Boundaries and Non-Goals
|
||||
|
||||
### In Scope
|
||||
- Every directory in the project gets a `.pi-map.md` file.
|
||||
- Every non-excluded file gets analyzed by the LLM layer.
|
||||
- Code files get augmented by the AST layer where parsers exist.
|
||||
- Respect `.gitignore` and known junk patterns (node_modules, .git, dist, build, coverage, .next, .venv, __pycache__, .DS_Store).
|
||||
|
||||
### Out of Scope (Non-Goals)
|
||||
- **Human-readable documentation**: These files are machine-only. Human docs live elsewhere.
|
||||
- **Line-by-line code explanation**: The format captures symbols and architecture, not implementation details.
|
||||
- **Auto-regeneration on filesystem events**: The skill relies on agent-initiated updates and periodic re-init, not filesystem watchers.
|
||||
- **Cross-project analysis**: Each project is independent. No global index across repos.
|
||||
- **IDE integration**: This is a Pi agent skill, not a VS Code extension or LSP server.
|
||||
|
||||
## 8. Pi Skill Package Structure
|
||||
|
||||
```
|
||||
pi-project-map/
|
||||
├── SKILL.md # Skill definition for Pi
|
||||
├── package.json # npm package metadata
|
||||
├── src/
|
||||
│ ├── init.ts # Full project scan + generation
|
||||
│ ├── patch.ts # Incremental patch logic
|
||||
│ ├── validate.ts # Consistency checker
|
||||
│ ├── ast-extract.ts # Tree-sitter / LSP wrappers
|
||||
│ ├── llm-extract.ts # LLM prompt templates for extraction
|
||||
│ ├── merge.ts # Merge AST + LLM outputs
|
||||
│ ├── format.ts # Dense markdown formatter
|
||||
│ └── config.ts # Skill configuration (thresholds, ignore patterns)
|
||||
└── README.md # Setup and usage for humans
|
||||
```
|
||||
|
||||
### Custom Tools
|
||||
- `project-map:init` — Run full project scan. Creates all `.pi-map.md` files.
|
||||
- `project-map:patch <file-path>` — Update analysis for a specific file/directory.
|
||||
- `project-map:validate` — Run consistency check across all `.pi-map.md` files.
|
||||
- `project-map:reinit [path]` — Force re-initialization of entire project or subtree.
|
||||
|
||||
### Prompt Hook
|
||||
- On each prompt, the skill injects a lightweight custom message **only if it is not already present in the current branch of context**:
|
||||
> "If you modify any source file, run `project-map:patch <path>` to update the analysis. If you suspect staleness, run `project-map:validate`."
|
||||
- The extension checks `ctx.sessionManager.buildSessionContext()` for an existing `pi-project-map-hint` custom message and skips injection when one is found. This prevents duplicate hints after steering, follow-up messages, or multi-turn conversations. The hint is automatically re-injected after compaction or `/tree` navigation removes it from the active path.
|
||||
|
||||
## 9. Risks and Tradeoffs
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|-----------|--------|------------|
|
||||
| Token bloat (1000+ dirs) | Medium | High | Summary mode, lazy loading, context budget |
|
||||
| Stale analysis files | High | High | Dirty markers + periodic re-init + validation |
|
||||
| Agent trusts stale data | Medium | High | Clear instructions to validate before architectural decisions |
|
||||
| Expensive init on large repos | Medium | Medium | Parallelization, caching, optional incremental init |
|
||||
| Overlap with LSP/typedoc | Low | Low | This is agent-context, not IDE tooling. Different use case. |
|
||||
| AST parser unavailable | Medium | Low | Graceful fallback to LLM-only extraction |
|
||||
|
||||
## 10. Concrete Example: Full Project Snapshot
|
||||
|
||||
```
|
||||
project-root/
|
||||
├── .pi-map.md
|
||||
├── src/
|
||||
│ ├── .pi-map.md
|
||||
│ ├── auth/
|
||||
│ │ ├── .pi-map.md
|
||||
│ │ ├── tokens.ts
|
||||
│ │ ├── middleware.ts
|
||||
│ │ └── types.ts
|
||||
│ └── db/
|
||||
│ ├── .pi-map.md
|
||||
│ ├── connection.ts
|
||||
│ └── migrations/
|
||||
│ ├── .pi-map.md
|
||||
│ └── 001_init.sql
|
||||
├── docker/
|
||||
│ ├── .pi-map.md
|
||||
│ ├── Dockerfile
|
||||
│ └── docker-compose.yml
|
||||
└── README.md
|
||||
```
|
||||
|
||||
Each `.pi-map.md` follows the format in Section 2, creating a navigable hierarchy.
|
||||
|
||||
## 11. Future Extensions
|
||||
|
||||
- **Cross-reference graph**: A top-level `project-graph.md` linking all packages with dependency arrows.
|
||||
- **Search index**: A lightweight FTS5 index over all `.pi-map.md` files for fast symbol lookup.
|
||||
- **Diff-aware patching**: Only re-run LLM on changed functions, not entire files.
|
||||
- **Multi-repo workspaces**: Support monorepos with independent package boundaries.
|
||||
### Cache
|
||||
- cache grows unless manually cleaned
|
||||
- corrupted cache files are recovered by starting fresh
|
||||
|
||||
@@ -1,241 +0,0 @@
|
||||
# Implementation Plan: Hierarchical Project Analysis Skill for Pi
|
||||
|
||||
## Overview
|
||||
|
||||
Build a Pi skill package (`pi-project-map`) that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file. The skill provides custom tools for init, patch, validate, and re-init, plus a prompt hook that ensures the agent keeps analysis files in sync.
|
||||
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
- **M1: Foundation and Format** — ✅ Complete
|
||||
- **M2: Heuristic Extraction** — ✅ Complete (placeholder only; to be replaced by real LLM)
|
||||
- **M3: AST Extraction** — ✅ Complete
|
||||
- **M4: Patch and Update** — ✅ Complete
|
||||
- **M5: Validation and `--fix`** — ✅ Complete
|
||||
- **M6: Pi Skill Integration** — ✅ Complete (basic)
|
||||
- **M7: Proper LLM Integration** — 🔄 In Progress / Next up
|
||||
|
||||
The remaining major work is to replace the heuristic "LLM" extraction with **actual LLM API calls**.
|
||||
|
||||
---
|
||||
|
||||
## Remaining Milestone: M7 — Proper LLM Integration
|
||||
|
||||
**Goal**: Replace the regex heuristic `llm-extract.ts` with real LLM calls, including dual-provider support (Pi native + external fallback), disk caching, parallelization, and retries.
|
||||
|
||||
**Estimated time**: 1 week of focused work.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: LLM Client Abstraction (`src/llm-client.ts`)
|
||||
|
||||
Create a unified interface for LLM calls.
|
||||
|
||||
```typescript
|
||||
interface LLMClient {
|
||||
complete(prompt: string): Promise<string>;
|
||||
}
|
||||
```
|
||||
|
||||
Sub-tasks:
|
||||
- Define the `LLMClient` interface.
|
||||
- Add a factory function `createLLMClient(mode: 'pi' | 'external', options)`.
|
||||
- Handle mode selection:
|
||||
- `'pi'`: used inside `pi-extension.ts` with Pi's model registry.
|
||||
- `'external'`: used in standalone CLI with OpenAI-compatible API.
|
||||
|
||||
**Acceptance Criteria**
|
||||
- `LLMClient` interface exists and compiles.
|
||||
- Factory correctly selects implementation based on mode.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: External LLM Client (`src/external-llm-client.ts`)
|
||||
|
||||
Implement the standalone CLI LLM client using an OpenAI-compatible API.
|
||||
|
||||
Sub-tasks:
|
||||
- Add `openai` (or a lightweight fetch-based client) as a dependency.
|
||||
- Read configuration from:
|
||||
- Environment variable `OPENAI_API_KEY` (or `ANTHROPIC_API_KEY`, etc.)
|
||||
- Config file `.pi-project-map.json` fields: `llmProvider`, `llmModel`, `llmBaseUrl`
|
||||
- Implement `complete(prompt)` using chat completions API.
|
||||
- Default to `gpt-4o-mini` or similar cheap model.
|
||||
|
||||
**Acceptance Criteria**
|
||||
- `project-map init` works standalone with `OPENAI_API_KEY` set.
|
||||
- Missing API key produces a clear, actionable error message.
|
||||
- Failed API call throws a descriptive `LLMError`.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Pi LLM Client (`src/pi-llm-client.ts`)
|
||||
|
||||
Implement the Pi-native LLM client for use inside the extension.
|
||||
|
||||
Sub-tasks:
|
||||
- Accept Pi's `ExtensionContext` or model registry as a constructor argument.
|
||||
- Use `ctx.model` / `ctx.modelRegistry` to get the configured model and API key.
|
||||
- Call the provider directly (likely using the same OpenAI-compatible endpoint Pi uses).
|
||||
- If Pi does not expose direct LLM calls, fall back to emitting a tool call / follow-up message pattern.
|
||||
|
||||
**Acceptance Criteria**
|
||||
- Pi extension can successfully call an LLM when running inside Pi.
|
||||
- Errors surface clearly to the user.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Disk Cache (`src/llm-cache.ts`)
|
||||
|
||||
Implement persistent SHA-256 → LLM result caching.
|
||||
|
||||
Sub-tasks:
|
||||
- Cache directory: `~/.cache/pi-project-map/` (create if missing).
|
||||
- Cache file: `llm-cache.json` (simple JSON object).
|
||||
- Functions:
|
||||
- `getCached(hash: string): string | null`
|
||||
- `setCached(hash: string, result: string): void`
|
||||
- Ensure atomic writes (write to temp file, rename).
|
||||
- Add cache size limit (e.g., 10,000 entries, LRU eviction).
|
||||
|
||||
**Acceptance Criteria**
|
||||
- Second `init` run on unchanged files does not call LLM.
|
||||
- Cache persists across process restarts.
|
||||
- Corrupted cache file does not crash the tool.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Parallelization and Retries (`src/llm-batch.ts`)
|
||||
|
||||
Run LLM requests in parallel with retries and backoff.
|
||||
|
||||
Sub-tasks:
|
||||
- Add `p-limit` dependency for concurrency control.
|
||||
- Default concurrency: 4 (configurable via `.pi-project-map.json` `llmConcurrency`).
|
||||
- Add small delay (100ms) between batches.
|
||||
- Implement retry logic: 3 retries, delays 1s → 2s → 4s.
|
||||
- On final failure, throw a hard error and stop the entire process.
|
||||
|
||||
**Acceptance Criteria**
|
||||
- 100-file project completes significantly faster than sequential.
|
||||
- Simulated transient failures are retried and recovered.
|
||||
- Persistent failure stops the tool with a clear error.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Rewrite `llm-extract.ts` to Use Real LLM
|
||||
|
||||
Replace regex heuristics with LLM calls.
|
||||
|
||||
Sub-tasks:
|
||||
- Accept an `LLMClient` in `extractFileLLM` and `extractPackageLLM`.
|
||||
- Check disk cache before calling LLM.
|
||||
- Construct file-level prompt (see design doc Section 4).
|
||||
- Parse response into `purpose`, `deps`, `concepts`.
|
||||
- Construct package-level prompt.
|
||||
- Parse response into `role`, `arch`.
|
||||
- Remove heuristic code from production path; keep only for test mocks if useful.
|
||||
|
||||
**Acceptance Criteria**
|
||||
- `llm-extract.ts` calls the LLM client for every file.
|
||||
- Output includes rich, non-trivial descriptions for typical files.
|
||||
- Unit tests mock the LLM client to verify prompt structure and parsing.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Context Limit Handling
|
||||
|
||||
Protect against oversized files.
|
||||
|
||||
Sub-tasks:
|
||||
- Measure prompt + file content tokens (approximate: 1 token ≈ 4 chars for ASCII).
|
||||
- If file exceeds max context budget (configurable, default 4000 tokens), truncate from the end.
|
||||
- Append `[...truncated]` marker in the prompt.
|
||||
- Skip LLM for binary/generated files over a hard limit (e.g., 50KB) and mark them as "Large/generated file".
|
||||
|
||||
**Acceptance Criteria**
|
||||
- A 1MB minified JS file does not crash or consume excessive tokens.
|
||||
- Truncated files still produce useful output.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Update CLI and Extension
|
||||
|
||||
Wire the new LLM client into all entry points.
|
||||
|
||||
Sub-tasks:
|
||||
- `src/cli.ts`: create external LLM client, pass into `initProject` / `patchFile`.
|
||||
- `pi-extension.ts`: create Pi LLM client, pass into tools.
|
||||
- Update `init.ts` and `patch.ts` signatures to accept an optional `LLMClient`.
|
||||
- Add CLI flag `--llm-provider=openai` for explicit selection.
|
||||
- Update error handling to catch `LLMError` and print helpful messages.
|
||||
|
||||
**Acceptance Criteria**
|
||||
- CLI works with external API key.
|
||||
- Extension works inside Pi (if Pi exposes LLM access).
|
||||
- Clear errors on misconfiguration.
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Update Tests
|
||||
|
||||
Sub-tasks:
|
||||
- Replace heuristic tests with mocked LLM client tests.
|
||||
- Add integration test: create a fake LLM client, run `initProject`, verify output contains LLM-provided text.
|
||||
- Add cache test: verify cache hit skips LLM call.
|
||||
- Add retry test: verify transient failures retry, persistent failures hard-stop.
|
||||
|
||||
**Acceptance Criteria**
|
||||
- All tests pass.
|
||||
- Test coverage includes: LLM client, cache, batching, prompt parsing, error handling.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing and Dependencies
|
||||
|
||||
```
|
||||
Task 1 (LLMClient interface)
|
||||
├── Task 2 (External client)
|
||||
├── Task 3 (Pi client)
|
||||
├── Task 4 (Cache)
|
||||
├── Task 5 (Batch + retries)
|
||||
│
|
||||
├── Task 6 (Rewrite llm-extract.ts)
|
||||
├── Task 7 (Context limits)
|
||||
│
|
||||
├── Task 8 (Wire CLI + extension)
|
||||
└── Task 9 (Tests)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Validation Criteria (for LLM Integration)
|
||||
|
||||
1. **Real LLM calls**: `llm-extract.ts` invokes the configured LLM client for every file.
|
||||
2. **Cache hit**: Second `init` on unchanged repo completes with zero LLM calls.
|
||||
3. **Parallel speed**: 100-file project init completes in under 30 seconds (assuming average LLM latency 500ms).
|
||||
4. **Retry works**: Transient 429/5xx errors are retried; permanent failures stop with a clear error.
|
||||
5. **Context limit safety**: Files > max token budget are truncated, not rejected.
|
||||
6. **Dual provider**: CLI uses external API; Pi extension uses Pi's LLM.
|
||||
7. **Quality**: LLM output is visibly richer than the old heuristic output (verified by manual inspection).
|
||||
|
||||
---
|
||||
|
||||
## Rollout Plan
|
||||
|
||||
1. **Test on real projects** (Day 1-2): Run `init` on 2-3 real codebases with the new LLM integration.
|
||||
2. **Cost audit** (Day 3): Measure token usage per project; adjust defaults if too expensive.
|
||||
3. **Prompt tuning** (Day 4-5): Iterate prompt design based on output quality.
|
||||
4. **Release** (Day 6-7): Publish updated npm package, update Pi extension docs.
|
||||
|
||||
---
|
||||
|
||||
## Completed Milestones (for reference)
|
||||
|
||||
- **M1**: Foundation and Format
|
||||
- **M2**: Heuristic Extraction (placeholder, to be replaced by M7)
|
||||
- **M3**: AST Extraction
|
||||
- **M4**: Patch and Update
|
||||
- **M5**: Validation and `--fix`
|
||||
- **M6**: Pi Skill Integration
|
||||
@@ -0,0 +1,10 @@
|
||||
# Apply Progress
|
||||
|
||||
Status: complete
|
||||
|
||||
Summary: Implemented paired map/index artifacts, layered routing/orientation protocol, pair-aware patch/validate/reinit flows, config caps, and docs/runtime guidance.
|
||||
|
||||
Implemented commits:
|
||||
- c6064f8 Implement layered maps and context retrieval
|
||||
|
||||
Stale-checkbox reconciliation: historical implementation completed in committed work above; tasks.md reconciled to checked state on 2026-06-11.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Archive Report
|
||||
|
||||
Status: archived
|
||||
|
||||
Archived path: openspec/changes/archive/2026-06-11-layered-map-protocol
|
||||
|
||||
Archive mode: manual archive fallback in openspec-only repo with legacy flat change specs; canonical sync marked not-applicable in sync-report.md.
|
||||
|
||||
Inputs preserved in archive: proposal.md, spec.md, design.md, tasks.md, apply-progress.md, verify-report.md, sync-report.md.
|
||||
@@ -0,0 +1,200 @@
|
||||
# Design: Layered Map Protocol
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Design** |
|
||||
| Based on | [Spec](spec.md) |
|
||||
| Next | Tasks |
|
||||
|
||||
## Design summary
|
||||
|
||||
This change introduces a paired, navigation-first artifact model without changing the core mission of `pi-project-map`. The implementation should keep the existing discover → analyze → merge → render pipeline, but route both outputs through a shared intermediate directory model:
|
||||
|
||||
- `.pi-map.index.md` = routing-first view
|
||||
- `.pi-map.md` = orientation-first view
|
||||
|
||||
Source remains the final authority.
|
||||
|
||||
## Affected areas
|
||||
|
||||
### Source files likely to change
|
||||
- `src/format.ts`
|
||||
- `src/init.ts`
|
||||
- `src/patch.ts`
|
||||
- `src/validate.ts`
|
||||
- `pi-extension.ts`
|
||||
- `README.md`
|
||||
- `SKILL.md`
|
||||
- `design-doc.md`
|
||||
- CLI argument parsing for `validate --fix` if not already supported
|
||||
|
||||
### New modules likely to appear
|
||||
- `src/root-index.ts` or equivalent shared index generation helper
|
||||
- `src/directory-model.ts` or equivalent shared intermediate model helper
|
||||
- optional config support for workflow/tag caps if not already present in config handling
|
||||
|
||||
## Architecture changes
|
||||
|
||||
### 1. Shared intermediate model
|
||||
Build one structured directory model per mapped directory, then render two views from it.
|
||||
|
||||
The model should carry at least:
|
||||
- directory identity (`dir`)
|
||||
- sibling/parent/child relationships
|
||||
- likely files
|
||||
- role/arch summaries
|
||||
- tags
|
||||
- prioritized symbols
|
||||
- workflow candidates in normalized schema
|
||||
- stale/freshness markers
|
||||
|
||||
This shared model is the main guard against map/index drift.
|
||||
|
||||
### 2. Universal paired artifacts
|
||||
During init/reinit, generate both `.pi-map.md` and `.pi-map.index.md` for every non-ignored directory.
|
||||
|
||||
#### Index rendering target
|
||||
Indexes should be small and role-first, using this fixed order:
|
||||
- `# <relative-path> (index)`
|
||||
- `## role`
|
||||
- `## parent`
|
||||
- `## children`
|
||||
- `## files`
|
||||
- `## links`
|
||||
- `## workflows`
|
||||
- `## dirty`
|
||||
|
||||
Index content should include:
|
||||
- `dir`
|
||||
- short role summary
|
||||
- parent link
|
||||
- child directory links
|
||||
- likely files
|
||||
- explicit sibling/child `index:` / `map:` links
|
||||
- up to configured workflow hints
|
||||
- dirty marker
|
||||
|
||||
Indexes must omit dependency edges and keep architectural prose minimal.
|
||||
|
||||
#### Rich-map rendering target
|
||||
Maps should remain dense but richer, using this fixed order:
|
||||
- `# <relative-path>`
|
||||
- `## role`
|
||||
- `## files`
|
||||
- `## arch`
|
||||
- `## tags`
|
||||
- `## symbols`
|
||||
- `## workflows`
|
||||
- `## dirty`
|
||||
|
||||
Map content should include:
|
||||
- `dir`
|
||||
- sibling `index:` link near the top
|
||||
- slightly richer file lines than today when useful
|
||||
- prioritized symbol lists in large directories
|
||||
|
||||
### 3. Root artifact behavior
|
||||
Root `.pi-map.index.md` is the Tier 0 routing artifact.
|
||||
|
||||
Root `.pi-map.md` remains rich and must:
|
||||
- point to root index,
|
||||
- restate the trust boundary,
|
||||
- be suitable for automatic loading on architecture/system or ambiguous tasks.
|
||||
|
||||
Both root artifacts should explicitly encode the Tier 0 behavior.
|
||||
|
||||
### 4. Workflow synthesis
|
||||
Workflow hints may be LLM-heavy, but they must be normalized into a deterministic shape before rendering.
|
||||
|
||||
Suggested normalized fields:
|
||||
- `task`
|
||||
- `read`
|
||||
- `index`
|
||||
- `map`
|
||||
- optional `files`
|
||||
|
||||
Indexes should render workflow hints in compact task→route form.
|
||||
|
||||
Because the user wants cross-repo workflow routing allowed, workflow synthesis may target other directories outside the local subtree when the hints are strong.
|
||||
|
||||
### 5. Configuration
|
||||
Paired mode is the only mode for now, but these knobs should be configurable:
|
||||
- workflow-hint cap (default 5)
|
||||
- rich-map tag cap (default 8)
|
||||
|
||||
Configuration belongs in shared config handling rather than ad hoc generator constants.
|
||||
|
||||
Suggested config shape for v1:
|
||||
|
||||
```yaml
|
||||
workflowHintCap: 5
|
||||
tagCap: 8
|
||||
```
|
||||
|
||||
### 6. Patch behavior
|
||||
Patch should always regenerate, never hand-edit, generated artifacts.
|
||||
|
||||
#### Changed directory
|
||||
Always regenerate both the local `.pi-map.md` and `.pi-map.index.md`.
|
||||
|
||||
#### Ancestor strategy
|
||||
Use auto-detected patch sizing with explicit override available from both CLI and tool surfaces.
|
||||
|
||||
- **Small default:** up to 3 related files, no structure/routing shift
|
||||
- refresh ancestor indexes only
|
||||
- **Large/structural:** export changes, directory shape changes, routing metadata impact, or wider cross-area effect
|
||||
- refresh both ancestor indexes and ancestor maps
|
||||
|
||||
This suggests a helper that computes the affected chain and the required artifact depth for each ancestor.
|
||||
|
||||
### 7. Validation behavior
|
||||
Validation should become pair-aware, not just file-list-aware.
|
||||
|
||||
Checks should include:
|
||||
- missing sibling artifact
|
||||
- sibling `dir` mismatch
|
||||
- broken parent/child/sibling references
|
||||
- stale `dirty` markers
|
||||
- stale or missing likely-file references
|
||||
- disagreement between map/index routing references
|
||||
|
||||
Validation failures for missing/stale indexes are hard failures.
|
||||
|
||||
When validation classifies a patch as structural, the output should explain the main reason concisely (for example export change, directory-shape change, or routing impact).
|
||||
|
||||
Repair should be exposed through `validate --fix`, which regenerates the affected chain by default.
|
||||
|
||||
Validation and repair messaging should stay concise and inline, with the main structural reason plus the chosen `patchMode` when repair runs.
|
||||
|
||||
### 8. Prompt and doc integration
|
||||
`pi-extension.ts`, `SKILL.md`, and docs should explicitly teach:
|
||||
- Tier 0 = protocol + root index
|
||||
- indexes first, maps next
|
||||
- local rich map + source before edits
|
||||
- validate before freshness-sensitive handoff
|
||||
- `index routes, map orients, source decides`
|
||||
|
||||
### 9. External dependency boundary
|
||||
This design should stay self-contained.
|
||||
|
||||
- No vector store is required.
|
||||
- No Engram dependency is required.
|
||||
- Retrieval remains local, deterministic, and based on generated paired artifacts.
|
||||
|
||||
## Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| Pair generation increases implementation size | Keep a shared intermediate model and split delivery into slices |
|
||||
| Workflow hints become noisy | Omit uncertain hints and normalize schema before rendering |
|
||||
| Ancestor refresh logic becomes brittle | Centralize affected-chain computation and test small vs structural cases |
|
||||
| Pair drift creates false confidence | Use common `dir` identity and pair-aware validation |
|
||||
| Config knobs spread inconsistently | Keep workflow/tag caps in shared config loading |
|
||||
|
||||
## Settled v1 defaults
|
||||
|
||||
- Root `.pi-map.md` points to root `.pi-map.index.md` rather than repeating route summaries.
|
||||
- The shared intermediate directory model remains internal; behavior is tested through rendered outputs.
|
||||
- Validation surfaces structural reasons as concise inline messages.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Proposal: Layered Map Protocol
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Proposal** |
|
||||
| Based on | User interview + current design doc |
|
||||
| Next | Spec |
|
||||
|
||||
## Problem
|
||||
|
||||
The current design centers consumption on discovering all `.pi-map.md` files and loading them at session start, with pruning for larger repositories. That scales poorly and frames maps as bulk context instead of as a navigation system.
|
||||
|
||||
The repo currently lacks:
|
||||
|
||||
1. a clear root routing artifact,
|
||||
2. an explicit trust/use protocol,
|
||||
3. a layered loading model,
|
||||
4. fast per-directory navigation artifacts,
|
||||
5. a clean split between routing metadata and richer orientation metadata.
|
||||
|
||||
## Proposed change
|
||||
|
||||
Adopt a layered navigation model built around a **paired artifact** for every non-ignored directory:
|
||||
|
||||
- `.pi-map.index.md` → fast routing
|
||||
- `.pi-map.md` → richer orientation
|
||||
|
||||
### Tier model
|
||||
- **Tier 0 — always injected:** Project Map Protocol + root `.pi-map.index.md`
|
||||
- **Tier 1 — task-start load:** likely relevant directory indexes first, then rich maps for strongest matches
|
||||
- **Tier 2 — exact verification:** source files, tests, configs, and docs
|
||||
|
||||
### Paired artifacts per directory
|
||||
Every non-ignored directory should generate both:
|
||||
- a rich `.pi-map.md`
|
||||
- a lightweight `.pi-map.index.md`
|
||||
|
||||
The index is for movement. The map is for understanding. Source remains the final authority.
|
||||
|
||||
This change stays self-contained: it does not add a vector store, Engram dependency, or other external memory/retrieval backend.
|
||||
|
||||
### Operating model
|
||||
- Root `.pi-map.md` remains rich, but points to root `.pi-map.index.md` for traversal.
|
||||
- Each index points to its sibling rich map.
|
||||
- Each rich map points back to its sibling index.
|
||||
- If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
|
||||
|
||||
## In scope
|
||||
|
||||
- [ ] Generate `.pi-map.index.md` for every non-ignored directory
|
||||
- [ ] Keep `.pi-map.md` as the richer sibling artifact for every non-ignored directory
|
||||
- [ ] Emit a Project Map Protocol in docs/prompt guidance and in generated root artifacts
|
||||
- [ ] Replace eager "read all maps" guidance with layered retrieval guidance
|
||||
- [ ] Define fixed index/map responsibilities and section order
|
||||
- [ ] Cascade freshness and repair logic across changed directory pairs and affected ancestors
|
||||
- [ ] Add strict validation for missing/stale/broken index-map pairs
|
||||
- [ ] Update docs and skill guidance to reflect the new model
|
||||
|
||||
## Out of scope
|
||||
|
||||
- [ ] Add a query-driven retrieval command such as `project-map context <query>`
|
||||
- [ ] Preserve backward compatibility for agents that only understand the old preload-only model
|
||||
- [ ] Replace source verification with map-based authority
|
||||
- [ ] Redesign the core AST/LLM extraction pipeline beyond what is needed to feed the new artifacts
|
||||
|
||||
## Decisions from grilling
|
||||
|
||||
| Topic | Decision |
|
||||
|---|---|
|
||||
| Change split | Two specs |
|
||||
| This change slug | `layered-map-protocol` |
|
||||
| Follow-up change slug | `map-context-retrieval` |
|
||||
| Backward compatibility required | No |
|
||||
| Artifact model | Every non-ignored directory gets both `.pi-map.md` and `.pi-map.index.md` |
|
||||
| Root Tier 0 | Project Map Protocol + root `.pi-map.index.md` |
|
||||
| Root rich map | Auto-read for architecture/system questions and ambiguous tasks |
|
||||
| Root relationship | Root map points to root index |
|
||||
| Retrieval mode | Hybrid auto-load, indexes first |
|
||||
| Patch strategy | Changed dir updates both; ancestors refresh by size/structure rules |
|
||||
| Validation | Hard-fail on missing/stale indexes, repair via `validate --fix` |
|
||||
| Prompt trust boundary | index routes, map orients, source decides |
|
||||
| Extra retrieval backends | None in this change |
|
||||
| SDD mode | Interactive |
|
||||
| Artifact store | OpenSpec |
|
||||
| PR strategy | Auto-forecast |
|
||||
| Review budget | 400 lines |
|
||||
|
||||
## Success criteria
|
||||
|
||||
- [ ] Every non-ignored directory has both a fast index and a richer map
|
||||
- [ ] The root artifacts clearly tell an agent how to navigate and when to read source
|
||||
- [ ] The documented consumption model no longer requires reading every `.pi-map.md` up front
|
||||
- [ ] Validation fails on stale or missing paired artifacts and offers a repair path
|
||||
- [ ] Patch/refresh behavior keeps routing trustworthy without requiring full reinit for small changes
|
||||
@@ -0,0 +1,229 @@
|
||||
# Spec: Layered Map Protocol
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Spec** |
|
||||
| Based on | [Proposal](proposal.md) |
|
||||
| Next | Design |
|
||||
|
||||
## Overview
|
||||
|
||||
`pi-project-map` must move from a bulk-preload model to a layered navigation model. The system should always provide a small root protocol plus root index, selectively load likely relevant directory indexes first, load rich maps next for the strongest matches, and require source verification before edits or precise runtime claims.
|
||||
|
||||
## Decisions
|
||||
|
||||
| # | Question | Answer |
|
||||
|---|---|---|
|
||||
| 1 | Per-directory artifact model | Every non-ignored directory gets both `.pi-map.md` and `.pi-map.index.md` |
|
||||
| 2 | Tier 0 default | Project Map Protocol + root `.pi-map.index.md` |
|
||||
| 3 | Root rich map auto-read | Yes for architecture/system questions and ambiguous tasks |
|
||||
| 4 | Retrieval style before query tool exists | Hybrid auto-load, indexes first |
|
||||
| 5 | Backward compatibility with old preload-only model | Not required |
|
||||
| 6 | Query-driven context bundling | Deferred to follow-up spec `map-context-retrieval` |
|
||||
| 7 | Mode shape | Paired artifact mode is the only mode for now |
|
||||
| 8 | Index section naming | Simple fixed names: `role`, `parent`, `children`, `files`, `links`, `workflows`, `dirty` |
|
||||
| 9 | Rich map section naming | Current-plus fixed names: `role`, `files`, `arch`, `tags`, `symbols`, `workflows`, `dirty` |
|
||||
| 10 | Patch-size override surface | Available from both CLI and tool surfaces |
|
||||
| 11 | Structural classification output | Validation explains the main structural reason concisely |
|
||||
| 12 | Extra retrieval backends | No vector store or Engram dependency in this change |
|
||||
|
||||
## Functional requirements
|
||||
|
||||
### 1. Paired artifacts per directory
|
||||
Every non-ignored directory must generate two artifacts:
|
||||
- `.pi-map.index.md` — quick-routing index
|
||||
- `.pi-map.md` — rich orientation document
|
||||
|
||||
Both artifacts must share a common identity marker such as `dir: <relative-path>`.
|
||||
|
||||
### 2. Root entrypoint
|
||||
The tool must generate a root navigation layer in both root artifacts.
|
||||
|
||||
#### Root Tier 0 behavior
|
||||
Tier 0 must always include:
|
||||
- Project Map Protocol
|
||||
- root `.pi-map.index.md`
|
||||
|
||||
The generated root artifacts must explicitly state this behavior.
|
||||
|
||||
#### Root rich map behavior
|
||||
Root `.pi-map.md` must:
|
||||
- point to root `.pi-map.index.md` for traversal,
|
||||
- restate the trust boundary,
|
||||
- be suitable for automatic loading on architecture/system questions and ambiguous tasks.
|
||||
|
||||
### 3. Project Map Protocol
|
||||
The generated system must define an explicit protocol with at least these rules:
|
||||
1. Read the Project Map Protocol and root `.pi-map.index.md` first.
|
||||
2. Use `index:` / `map:` references to open relevant directory indexes and maps.
|
||||
3. Load indexes before rich maps during task-start navigation.
|
||||
4. Read the local rich map and actual source before editing.
|
||||
5. Treat non-empty `## dirty` sections in either artifact as stale.
|
||||
6. If source and generated artifacts disagree, trust source.
|
||||
7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.
|
||||
8. After editing source, run `project_map_patch` for each changed file.
|
||||
9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.
|
||||
|
||||
### 4. Index contract
|
||||
Every `.pi-map.index.md` must use a fixed section order and remain optimized for fast routing.
|
||||
|
||||
#### Required behavior
|
||||
- Keep architectural prose minimal.
|
||||
- Omit uncertain workflow/file hints rather than labeling confidence.
|
||||
- Allow up to 5 workflow hints by default, with configuration support.
|
||||
- Do not include dependency edges in indexes.
|
||||
- Include explicit parent link when a parent mapped directory exists.
|
||||
- For child directory entries, point to both child `index:` and child `map:`.
|
||||
- Leaf-directory indexes remain tiny but still follow the same contract.
|
||||
|
||||
#### Fixed section order
|
||||
Each index must use this fixed order:
|
||||
1. `## role`
|
||||
2. `## parent`
|
||||
3. `## children`
|
||||
4. `## files`
|
||||
5. `## links`
|
||||
6. `## workflows`
|
||||
7. `## dirty`
|
||||
|
||||
#### Minimum contents
|
||||
Each index must minimally support:
|
||||
- title header in the form `# <relative-path> (index)`
|
||||
- `dir` identity marker
|
||||
- short role summary
|
||||
- parent link where applicable
|
||||
- child directories where applicable
|
||||
- likely files
|
||||
- explicit sibling/child `index:` / `map:` links
|
||||
- short workflow hints in hybrid task→route form
|
||||
- `## dirty`
|
||||
|
||||
### 5. Rich map contract
|
||||
Every `.pi-map.md` must use a fixed section order and remain optimized for dense orientation.
|
||||
|
||||
#### Required behavior
|
||||
- Rich maps must explicitly link to their sibling `.pi-map.index.md`.
|
||||
- Rich maps may keep some routing overlap, but indexes remain primary for navigation.
|
||||
- `files` entries should be slightly richer than the current dense line format when useful.
|
||||
- Tags should default to up to 8, with configuration support.
|
||||
- Symbols should be prioritized for density in large directories using structural importance plus LLM refinement.
|
||||
- Cross-repo workflow routing should live mostly in indexes, not maps.
|
||||
|
||||
#### Fixed section order
|
||||
Each rich map must use this fixed order:
|
||||
1. `## role`
|
||||
2. `## files`
|
||||
3. `## arch`
|
||||
4. `## tags`
|
||||
5. `## symbols`
|
||||
6. `## workflows`
|
||||
7. `## dirty`
|
||||
|
||||
#### Minimum contents
|
||||
Each rich map must minimally support:
|
||||
- title header in the form `# <relative-path>`
|
||||
- `dir` identity marker
|
||||
- sibling `index:` link near the top
|
||||
- richer but still compact workflow guidance
|
||||
- `## dirty`
|
||||
|
||||
### 6. Workflow generation
|
||||
Workflow hints may be LLM-heavy, but they must be normalized into a deterministic schema before rendering.
|
||||
|
||||
#### Index workflow rules
|
||||
- Primary form is hybrid task→route, for example: `change CLI behavior -> read: src/.pi-map.index.md, src/cli.ts`
|
||||
- Cross-repo routing is allowed.
|
||||
- Uncertain hints should be omitted.
|
||||
- Indexes should optimize for fast routing, not standalone completeness.
|
||||
|
||||
### 7. Shared generation model
|
||||
Map and index generation must come from a shared intermediate directory model rather than unrelated passes.
|
||||
|
||||
The renderer may present different views, but the underlying directory identity and source facts must stay aligned.
|
||||
|
||||
### 8. Patch behavior
|
||||
`project_map_patch` must never manually edit generated artifacts; it must regenerate them.
|
||||
|
||||
#### Changed directory
|
||||
For every changed source file, patch must update both artifacts for the changed directory.
|
||||
|
||||
#### Ancestor refresh behavior
|
||||
Patch sizing must be auto-detected with an explicit override path available from both CLI and tool surfaces.
|
||||
|
||||
- **Small change default:** up to 3 closely related files and no structural/routing shift
|
||||
- changed directory: refresh map + index
|
||||
- ancestors: refresh indexes only
|
||||
- **Large/structural change:** exports, directory shape, routing metadata, or broader cross-area impact
|
||||
- changed directory: refresh map + index
|
||||
- ancestors: refresh both map + index
|
||||
|
||||
### 9. Validation behavior
|
||||
`project_map_validate` must hard-fail when paired artifacts are missing or structurally stale.
|
||||
|
||||
Validation output should prefer concise inline wording, for example:
|
||||
- `[structural] src/foo.ts: export change`
|
||||
- `[stale-index] src/.pi-map.index.md: likely files out of date`
|
||||
|
||||
Validation must check at least:
|
||||
- missing `.pi-map.index.md` or `.pi-map.md`
|
||||
- stale `## dirty` sections in either artifact
|
||||
- broken `index:` / `map:` references
|
||||
- stale `files:` pointers
|
||||
- map/index pair disagreement on routing hints or referenced areas
|
||||
- sibling pair identity mismatches
|
||||
|
||||
When validation classifies a patch as structural, it should report the main reason concisely, such as export change, directory-shape change, or routing impact.
|
||||
|
||||
#### Repair
|
||||
Repair must be exposed through `project-map validate --fix`.
|
||||
|
||||
Default repair scope is the affected directory pair plus the necessary ancestor chain.
|
||||
Repair summaries should also report the chosen `patchMode` concisely, for example: `repaired affected chain (patchMode: structural)`.
|
||||
|
||||
### 10. Visibility and policy
|
||||
`.pi-map.index.md` must follow the same hidden/ignored policy as `.pi-map.md`.
|
||||
|
||||
Agents should never manually edit either artifact directly.
|
||||
|
||||
## Non-functional requirements
|
||||
|
||||
- Prefer density over completeness.
|
||||
- Keep indexes routing-first and maps understanding-first.
|
||||
- Keep the protocol simple enough to emit in root artifacts and prompt guidance.
|
||||
- Support configuration for top-level `workflowHintCap` and `tagCap` keys.
|
||||
- Do not require a vector store, Engram, or other extra retrieval backend for this design.
|
||||
|
||||
## User flows
|
||||
|
||||
### Flow 1: Agent starts work on a repo
|
||||
1. Agent reads the Project Map Protocol and root `.pi-map.index.md`
|
||||
2. For architecture/system or ambiguous tasks, agent also reads root `.pi-map.md`
|
||||
3. Agent opens likely relevant directory indexes first
|
||||
4. Agent opens the strongest-match rich maps
|
||||
5. Agent reads source only when exact behavior or editing is involved
|
||||
|
||||
### Flow 2: Agent edits a file
|
||||
1. Agent routes with index data
|
||||
2. Agent reads the local rich map and actual source
|
||||
3. Agent edits source
|
||||
4. Agent runs `project_map_patch` for changed files
|
||||
5. Patch refreshes the changed directory pair and the required ancestor chain
|
||||
|
||||
### Flow 3: Validation and repair
|
||||
1. Agent runs `project_map_validate`
|
||||
2. Missing/stale pair failures are reported as hard errors
|
||||
3. Agent runs `project-map validate --fix` when repair is desired
|
||||
4. The affected chain is regenerated
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Every non-ignored directory produces both `.pi-map.md` and `.pi-map.index.md`
|
||||
- [ ] Tier 0 behavior is both documented and emitted in generated root artifacts
|
||||
- [ ] Indexes have a fixed routing-first contract with parent/child/sibling links
|
||||
- [ ] Rich maps have a fixed orientation-first contract with sibling index links
|
||||
- [ ] `project_map_patch` refreshes both changed-directory artifacts and size-appropriate ancestor artifacts
|
||||
- [ ] `project_map_validate` hard-fails on missing/stale paired artifacts and supports `--fix`
|
||||
- [ ] Workflow-hint count and tag cap are configurable
|
||||
- [ ] Prompt guidance expresses: index routes, map orients, source decides
|
||||
@@ -0,0 +1,7 @@
|
||||
# Sync Report
|
||||
|
||||
Status: NOT-APPLICABLE
|
||||
|
||||
Reason: This repository uses legacy flat change artifacts (proposal.md/spec.md/design.md/tasks.md) and does not maintain a canonical openspec/specs/ tree for these changes. No canonical spec sync was performed.
|
||||
|
||||
User-approved fallback: archive completed change as an audit record without canonical spec sync.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Tasks: Layered Map Protocol
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Tasks** |
|
||||
| Based on | [Design](design.md) |
|
||||
| Next | Apply |
|
||||
|
||||
## Delivery slices
|
||||
|
||||
### Slice 1: Shared model and paired rendering
|
||||
**Scope**: shared intermediate directory model, universal paired artifacts, fixed section names/order, root Tier 0 entrypoint
|
||||
**Review goal**: establish the core paired architecture without retrieval-command work
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Add a shared intermediate directory model for paired map/index generation
|
||||
2. [ ] Extend render/parse support for paired `.pi-map.md` and `.pi-map.index.md` outputs with the agreed fixed section names/order
|
||||
3. [ ] Add generation support for `.pi-map.index.md` in every non-ignored directory
|
||||
4. [ ] Enrich root artifacts with explicit Tier 0 behavior and sibling links
|
||||
5. [ ] Add/update tests for map/index rendering, fixed order, and sibling identity
|
||||
|
||||
### Slice 2: Routing metadata and workflow normalization
|
||||
**Scope**: likely files, parent/child/sibling links, workflow schema, rich-map metadata density rules
|
||||
**Review goal**: make the navigation model operational on real output
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Add index-first routing metadata generation from AST/structure + LLM judgment
|
||||
2. [ ] Normalize workflow hints into a deterministic schema before rendering
|
||||
3. [ ] Add parent/child/sibling `index:` / `map:` references
|
||||
4. [ ] Implement rich-map density rules for slightly richer file lines, tag cap, and prioritized symbols
|
||||
5. [ ] Add/update tests for routing hints, leaf indexes, and workflow omission on low confidence
|
||||
|
||||
### Slice 3: Patch sizing, cascade refresh, and repair
|
||||
**Scope**: changed-directory pair refresh, ancestor-chain refresh rules, validation hard failures, `validate --fix`
|
||||
**Review goal**: make freshness and repair trustworthy
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Make `project_map_patch` always regenerate both artifacts for the changed directory
|
||||
2. [ ] Add auto-detected small vs structural patch sizing with explicit override in both CLI and tool surfaces
|
||||
3. [ ] Refresh ancestor indexes for small changes and ancestor map+index pairs for structural changes
|
||||
4. [ ] Expand validation for pair-aware checks, hard failures on missing/stale indexes, and concise structural-reason output
|
||||
5. [ ] Add/confirm `project-map validate --fix` repair flow for affected-chain regeneration
|
||||
6. [ ] Add/update tests for patch classification, ancestor refresh, and repair behavior
|
||||
|
||||
### Slice 4: Prompt guidance, config knobs, and docs
|
||||
**Scope**: Project Map Protocol, Tier 0 runtime guidance, workflow/tag config, docs rewrite
|
||||
**Review goal**: align runtime behavior and documentation with the final operating model
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Update `pi-extension.ts` prompt guidance to teach the layered paired protocol
|
||||
2. [ ] Update `SKILL.md` to describe indexes as routing and maps as orientation
|
||||
3. [ ] Update `README.md` and `design-doc.md` consumption guidance
|
||||
4. [ ] Add config support for workflow-hint cap and rich-map tag cap
|
||||
5. [ ] Keep the paired-artifact model self-contained with no vector-store or Engram dependency
|
||||
6. [ ] Add/update tests for config-driven caps and root Tier 0 guidance
|
||||
|
||||
## Acceptance checklist
|
||||
|
||||
- [x] Every non-ignored directory has both `.pi-map.md` and `.pi-map.index.md`
|
||||
- [x] Root Tier 0 behavior is emitted in generated root artifacts
|
||||
- [x] Indexes are routing-first, role-first, and include parent/child/sibling links
|
||||
- [x] Rich maps are orientation-first and include sibling index links
|
||||
- [x] Changed-directory patch always regenerates both artifacts
|
||||
- [x] Ancestor refresh follows small vs structural rules
|
||||
- [x] Validation hard-fails on missing/stale paired artifacts and supports `validate --fix`
|
||||
- [x] Workflow-hint count and tag cap are configurable
|
||||
- [x] No vector-store or Engram dependency is introduced
|
||||
- [x] `npm run typecheck` passes
|
||||
- [x] `npm test` passes
|
||||
- [x] `npm run lint` passes *(N/A: repo has no ESLint config; pre-existing repository gap, not a change regression)*
|
||||
|
||||
## Review workload note
|
||||
This change crosses format, generation, patching, validation, config, docs, and prompt guidance. Keep implementation in narrow, reviewable slices and avoid one oversized PR.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Verify Report
|
||||
|
||||
Status: PASS
|
||||
|
||||
Change: layered-map-protocol
|
||||
|
||||
Verified summary: Implemented paired map/index artifacts, layered routing/orientation protocol, pair-aware patch/validate/reinit flows, config caps, and docs/runtime guidance.
|
||||
|
||||
Evidence commands:
|
||||
- npm run typecheck
|
||||
- npx vitest run
|
||||
- npm run build
|
||||
- node dist/cli.js validate .
|
||||
|
||||
Current validation state:
|
||||
- npm run typecheck: PASS
|
||||
- npx vitest run: PASS (262/262)
|
||||
- npm run build: PASS
|
||||
- node dist/cli.js validate .: PASS
|
||||
|
||||
Notes:
|
||||
- Repo-wide lint remains unavailable because the repository has no ESLint config; treated as a pre-existing repository-level gap, not a change regression.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Apply Progress
|
||||
|
||||
Status: complete
|
||||
|
||||
Summary: Implemented deterministic index-first context retrieval via tool and CLI, plus retrieval docs/skill guidance.
|
||||
|
||||
Implemented commits:
|
||||
- c6064f8 Implement layered maps and context retrieval
|
||||
|
||||
Stale-checkbox reconciliation: historical implementation completed in committed work above; tasks.md reconciled to checked state on 2026-06-11.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Archive Report
|
||||
|
||||
Status: archived
|
||||
|
||||
Archived path: openspec/changes/archive/2026-06-11-map-context-retrieval
|
||||
|
||||
Archive mode: manual archive fallback in openspec-only repo with legacy flat change specs; canonical sync marked not-applicable in sync-report.md.
|
||||
|
||||
Inputs preserved in archive: proposal.md, spec.md, design.md, tasks.md, apply-progress.md, verify-report.md, sync-report.md.
|
||||
@@ -0,0 +1,65 @@
|
||||
# Design: Map Context Retrieval
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Design** |
|
||||
| Based on | [Spec](spec.md) |
|
||||
| Next | Tasks |
|
||||
|
||||
## Design summary
|
||||
|
||||
This change adds a lightweight retrieval layer on top of paired project-map metadata. It should not require a new external storage system, vector store, or Engram dependency. Instead, it should scan root and directory indexes first, expand to rich maps and files from the strongest candidates, and emit an agent-friendly bundle.
|
||||
|
||||
## Likely implementation areas
|
||||
|
||||
- `pi-extension.ts` for the first-class tool surface
|
||||
- `src/index.ts` for exports
|
||||
- new retrieval module, e.g. `src/context.ts` or `src/retrieve.ts`
|
||||
- shared parsing/model code from the layered protocol
|
||||
- `src/cli/*` for follow-up CLI wiring
|
||||
- docs and skill guidance
|
||||
|
||||
## Retrieval pipeline
|
||||
|
||||
1. Accept a natural-language `query`
|
||||
2. Read root `.pi-map.index.md` and, when needed, root `.pi-map.md`
|
||||
3. Parse paired index/map metadata through the shared format/model layer
|
||||
4. Score candidate directories primarily from indexes
|
||||
5. Keep the top 3 candidates by default
|
||||
6. Expand strongest candidates to rich maps, likely files, and symbols
|
||||
7. Emit a compact markdown bundle with stable section order and retrieval-specific title `# Context bundle: <query>`
|
||||
|
||||
## Candidate scoring inputs
|
||||
|
||||
- direct term matches in workflow hints
|
||||
- direct term matches in likely files
|
||||
- parent/child/sibling link context
|
||||
- direct term matches in tags
|
||||
- direct term matches in symbols
|
||||
- path/name similarity
|
||||
- root workflow routing hits
|
||||
|
||||
A first implementation can use deterministic weighted lexical scoring.
|
||||
|
||||
Candidate-selection reasoning does not need to be exposed by default.
|
||||
|
||||
## Parsing strategy
|
||||
|
||||
Reuse the paired-artifact parser/model from the layered protocol. Avoid retrieval-specific ad hoc parsing.
|
||||
|
||||
## Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| Retrieval becomes too fuzzy to trust | Keep output advisory and always direct agent back to source |
|
||||
| Pair parser complexity grows | Extend shared format/model logic instead of command-local parsing |
|
||||
| Command output becomes too large | Limit result count and keep instructions terse |
|
||||
|
||||
## Open choices
|
||||
|
||||
1. Exact scoring weights for workflow hints vs files vs tags vs symbols
|
||||
2. Whether the tool should support optional structured output later
|
||||
3. Whether broad architecture queries should be allowed to exceed the usual top-3 default in a later version
|
||||
4. Whether later versions should support optional root-rich-map inclusion as a flag
|
||||
@@ -0,0 +1,53 @@
|
||||
# Proposal: Map Context Retrieval
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Proposal** |
|
||||
| Based on | Follow-up to `layered-map-protocol` |
|
||||
| Next | Spec |
|
||||
|
||||
## Problem
|
||||
|
||||
After the layered map protocol lands, the agent will know it should load root protocol + root index first, then route through per-directory indexes and rich maps. But the repo will still lack an ergonomic retrieval primitive that can turn a natural-language task into a compact context bundle.
|
||||
|
||||
Without that helper, the agent still has to manually inspect directory indexes, rank candidate areas, and expand to rich maps/files. That weakens the value of the new routing layer.
|
||||
|
||||
## Proposed change
|
||||
|
||||
Add a retrieval tool and command shape:
|
||||
|
||||
```bash
|
||||
project-map context "<user task>"
|
||||
```
|
||||
|
||||
The first-class surface should be a Pi tool. CLI support can follow the same shape.
|
||||
|
||||
The retrieval result should return a compact context bundle containing:
|
||||
- relevant indexes,
|
||||
- strongest-match rich maps,
|
||||
- likely source files,
|
||||
- relevant symbols when useful,
|
||||
- short instructions on what to read next.
|
||||
|
||||
This remains a local metadata-driven retrieval feature, not a vector-store or Engram-backed memory system.
|
||||
## In scope
|
||||
|
||||
- [ ] Add Pi tool support for `project-map context <query>`
|
||||
- [ ] Add CLI support using the same shape after the tool contract is stable
|
||||
- [ ] Rank results using paired index/map metadata from the layered protocol
|
||||
- [ ] Return a compact, markdown-first, agent-friendly context bundle
|
||||
- [ ] Document recommended usage from prompts and docs
|
||||
## Out of scope
|
||||
|
||||
- [ ] Full semantic search across arbitrary source contents
|
||||
- [ ] Replacing source verification with map-based answers
|
||||
- [ ] Building a heavyweight external indexer or vector database
|
||||
|
||||
## Success criteria
|
||||
|
||||
- [ ] A natural-language task can resolve to likely indexes/maps/files/symbols without manual repo scanning
|
||||
- [ ] Output is compact enough to inject directly into the next step
|
||||
- [ ] Retrieval builds on generated paired metadata rather than bypassing it
|
||||
- [ ] The first version is tool-first and markdown-first
|
||||
@@ -0,0 +1,127 @@
|
||||
# Spec: Map Context Retrieval
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Spec** |
|
||||
| Based on | [Proposal](proposal.md) |
|
||||
| Next | Design |
|
||||
|
||||
## Overview
|
||||
|
||||
Add a retrieval-oriented `context` command that turns a user task into a compact project-map routing bundle. This change depends on the paired root/per-directory index+map metadata introduced by `layered-map-protocol`.
|
||||
|
||||
## Dependency
|
||||
|
||||
This change should not begin implementation before `layered-map-protocol` has landed enough metadata to support routing and ranking.
|
||||
|
||||
## Functional requirements
|
||||
|
||||
### 1. Tool-first surface
|
||||
Pi should expose `project-map context` as a first-class tool action so agents can request a context bundle directly.
|
||||
|
||||
For v1, the tool input should be minimal:
|
||||
- `query: string`
|
||||
|
||||
### 2. CLI follow-up surface
|
||||
The CLI should support the same conceptual shape:
|
||||
|
||||
```bash
|
||||
project-map context "<query>"
|
||||
```
|
||||
|
||||
CLI support may follow after the tool contract is stable.
|
||||
The CLI should mirror the same main input name: `query`.
|
||||
### 3. Bundle contents
|
||||
A context bundle must contain, at minimum:
|
||||
- relevant indexes
|
||||
- strongest-match relevant maps
|
||||
- likely files
|
||||
- relevant symbols when useful
|
||||
- short next-step instructions
|
||||
### 4. Ranking behavior
|
||||
The command must rank candidates using generated metadata such as:
|
||||
- root workflow hints
|
||||
- directory index routing hints
|
||||
- parent/child/sibling link structure
|
||||
- package tags
|
||||
- symbol references
|
||||
- task/workflow entries
|
||||
- `files:` / `index:` / `map:` targets
|
||||
|
||||
Indexes should be the first routing layer. Rich maps and symbols should expand from the strongest routed candidates.
|
||||
|
||||
By default, retrieval should usually surface the top 3 candidate directories, with adaptive omission of weaker rich maps when confidence drops.
|
||||
### 5. Trust boundary
|
||||
The bundle must instruct the agent to:
|
||||
- read indexes first for orientation,
|
||||
- read rich maps next when deeper context is needed,
|
||||
- read source before editing or asserting exact behavior.
|
||||
|
||||
## Output shape
|
||||
|
||||
The first version should be markdown-first, with stable, schema-like sections optimized for LLM consumption rather than human prose.
|
||||
|
||||
Default section order:
|
||||
1. `query`
|
||||
2. `relevant indexes`
|
||||
3. `relevant maps`
|
||||
4. `likely files`
|
||||
5. `relevant symbols`
|
||||
6. `instructions`
|
||||
|
||||
The retrieval title format should remain retrieval-specific:
|
||||
- `# Context bundle: <query>`
|
||||
|
||||
A representative response shape:
|
||||
|
||||
```markdown
|
||||
# Context bundle: validation stale signatures
|
||||
|
||||
## query
|
||||
validation stale signatures
|
||||
|
||||
## relevant indexes
|
||||
- .pi-map.index.md
|
||||
- src/.pi-map.index.md
|
||||
- tests/.pi-map.index.md
|
||||
|
||||
## relevant maps
|
||||
- src/.pi-map.md
|
||||
- tests/.pi-map.md
|
||||
|
||||
## likely files
|
||||
- src/validate.ts
|
||||
- src/ast/ast-extract.ts
|
||||
- tests/validate.test.ts
|
||||
|
||||
## relevant symbols
|
||||
- validateMaps
|
||||
- generateDirectoryMap
|
||||
- extractFileAST
|
||||
|
||||
## instructions
|
||||
Read the indexes first, then the strongest-match rich maps, then verify behavior from source before editing.
|
||||
```
|
||||
|
||||
The tool should omit weak/unhelpful symbol output rather than forcing a noisy symbol section.
|
||||
## Non-functional requirements
|
||||
|
||||
- Output must be compact enough for prompt injection.
|
||||
- Ranking should be deterministic enough to test.
|
||||
- The implementation should prefer lightweight heuristics over heavyweight indexing.
|
||||
- Retrieval should reuse the paired-artifact parsing layer instead of ad hoc string slicing.
|
||||
- Retrieval should not require a vector store, Engram, or another external memory backend.
|
||||
- The first version should accept natural language queries first; structured filters can come later.
|
||||
- Candidate explanations are not required by default.
|
||||
- Result-count override and root-rich-map inclusion can come later if needed, but are not part of v1.
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pi exposes `project-map context <query>` as a tool-first surface
|
||||
- [ ] CLI support follows the same shape after the tool contract is stable
|
||||
- [ ] Retrieval output includes indexes, strongest-match maps, files, and instructions
|
||||
- [ ] Symbol hints appear only when paired metadata makes them useful
|
||||
- [ ] Ranking respects the index-first routing model and usually returns top 3 candidates
|
||||
- [ ] Docs show when to use the retrieval command
|
||||
- [ ] Trust-boundary guidance remains explicit
|
||||
@@ -0,0 +1,7 @@
|
||||
# Sync Report
|
||||
|
||||
Status: NOT-APPLICABLE
|
||||
|
||||
Reason: This repository uses legacy flat change artifacts (proposal.md/spec.md/design.md/tasks.md) and does not maintain a canonical openspec/specs/ tree for these changes. No canonical spec sync was performed.
|
||||
|
||||
User-approved fallback: archive completed change as an audit record without canonical spec sync.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Tasks: Map Context Retrieval
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Tasks** |
|
||||
| Based on | [Design](design.md) |
|
||||
| Next | Apply |
|
||||
|
||||
## Tasks
|
||||
|
||||
1. [ ] Add Pi tool support for `project-map context` with `query` as the main input
|
||||
2. [ ] Add retrieval module for scoring paired index/map metadata
|
||||
3. [ ] Reuse or extend shared paired-artifact parsing/model code
|
||||
4. [ ] Rank candidate directories from indexes first, then expand to maps/files/symbols
|
||||
5. [ ] Emit compact markdown-first context bundles with stable section order and retrieval-specific `Context bundle` title
|
||||
6. [ ] Add tests for retrieval ranking, top-3 default behavior, and output shape
|
||||
7. [ ] Add CLI support after the tool contract is stable
|
||||
8. [ ] Update docs and skill guidance for retrieval usage
|
||||
## Acceptance checklist
|
||||
|
||||
- [x] Tool works for natural-language queries via `query`
|
||||
- [x] Output includes relevant indexes, strongest-match maps, likely files, symbols when useful, and instructions
|
||||
- [x] Retrieval depends on paired metadata rather than raw source scanning
|
||||
- [x] Ranking follows the index-first routing model with a top-3 default
|
||||
- [x] `npm run typecheck` passes
|
||||
- [x] `npm test` passes
|
||||
- [x] `npm run lint` passes *(N/A: repo has no ESLint config; pre-existing repository gap, not a change regression)*
|
||||
|
||||
## Sequencing note
|
||||
This change is intentionally follow-up work. It should start after the layered map protocol produces reliable paired routing metadata.
|
||||
|
||||
The first delivery surface is the Pi tool. CLI support follows the stabilized tool contract.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Verify Report
|
||||
|
||||
Status: PASS
|
||||
|
||||
Change: map-context-retrieval
|
||||
|
||||
Verified summary: Implemented deterministic index-first context retrieval via tool and CLI, plus retrieval docs/skill guidance.
|
||||
|
||||
Evidence commands:
|
||||
- npm run typecheck
|
||||
- npx vitest run
|
||||
- npm run build
|
||||
- node dist/cli.js context "validation routing"
|
||||
- node dist/cli.js validate .
|
||||
|
||||
Current validation state:
|
||||
- npm run typecheck: PASS
|
||||
- npx vitest run: PASS (262/262)
|
||||
- npm run build: PASS
|
||||
- node dist/cli.js validate .: PASS
|
||||
|
||||
Notes:
|
||||
- Repo-wide lint remains unavailable because the repository has no ESLint config; treated as a pre-existing repository-level gap, not a change regression.
|
||||
@@ -0,0 +1,15 @@
|
||||
# Apply Progress
|
||||
|
||||
Status: complete
|
||||
|
||||
Summary: Implemented prompt-injection slices 1-5: mode/config surface, root-pair preload, reinjection/dedupe, strict/advisory/strong/off semantics, and documentation/runtime alignment.
|
||||
|
||||
Implemented commits:
|
||||
- 56560d9 feat(prompt): implement prompt injection slice 1
|
||||
- 621434b feat(prompt): implement prompt injection slice 2
|
||||
- 19666c9 feat(prompt): implement prompt injection slice 3
|
||||
- 58e8bd3 feat(prompt): implement prompt injection slice 4
|
||||
- 11365fa docs(prompt): align prompt injection guidance and runtime copy
|
||||
- c11d49d spec(prompt): add project map prompt injection change set
|
||||
|
||||
Stale-checkbox reconciliation: historical implementation completed in committed work above; tasks.md reconciled to checked state on 2026-06-11.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Archive Report
|
||||
|
||||
Status: archived
|
||||
|
||||
Archived path: openspec/changes/archive/2026-06-11-project-map-prompt-injection
|
||||
|
||||
Archive mode: manual archive fallback in openspec-only repo with legacy flat change specs; canonical sync marked not-applicable in sync-report.md.
|
||||
|
||||
Inputs preserved in archive: proposal.md, spec.md, design.md, tasks.md, apply-progress.md, verify-report.md, sync-report.md.
|
||||
@@ -0,0 +1,235 @@
|
||||
# Design: Project Map Prompt Injection
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Design** |
|
||||
| Based on | [Spec](spec.md) |
|
||||
| Next | Tasks |
|
||||
|
||||
## Design summary
|
||||
|
||||
This change adds a runtime guidance layer on top of the paired map/index artifact system. The implementation should not redesign map generation or retrieval. Instead, it should add a well-scoped injection pipeline that:
|
||||
- emits honest startup hints before init,
|
||||
- preloads the root pair after init,
|
||||
- expands under a hybrid budget,
|
||||
- avoids redundant reinjection by scanning real outgoing context,
|
||||
- varies guidance strength through explicit modes,
|
||||
- proves correctness with integration-heavy validation.
|
||||
|
||||
## Affected areas
|
||||
|
||||
### Source files likely to change
|
||||
- `pi-extension.ts`
|
||||
- shared config handling (`src/config.ts` or equivalent)
|
||||
- runtime helpers for map/index discovery and injection selection
|
||||
- tests covering extension lifecycle and per-turn behavior
|
||||
- docs/runtime guidance surfaces if needed
|
||||
|
||||
### New modules likely to appear
|
||||
- `src/prompt-injection.ts` or equivalent runtime helper
|
||||
- optional helper for canonical marker construction / detection
|
||||
- optional helper for token-budget estimation across paired artifacts
|
||||
|
||||
## Architecture changes
|
||||
|
||||
### 1. Injection policy helper
|
||||
Centralize prompt-injection logic in one helper rather than scattering it across hooks.
|
||||
|
||||
Suggested responsibilities:
|
||||
- determine current mode (`off` / `advisory` / `strong` / `strict`),
|
||||
- detect whether artifacts exist,
|
||||
- build pre-init startup hint,
|
||||
- build post-init root-pair payload,
|
||||
- estimate expansion budget,
|
||||
- choose additional artifacts under the budget,
|
||||
- construct a canonical injected marker/block,
|
||||
- scan outgoing context or payload for that marker,
|
||||
- decide whether reinjection is required.
|
||||
|
||||
This helper is the main guard against drift between startup hints, context hooks, and strict-mode checks.
|
||||
|
||||
### 2. Injection surfaces
|
||||
Use different extension surfaces for different responsibilities.
|
||||
|
||||
#### Pre-init / startup hint
|
||||
Use `before_agent_start` for lightweight startup guidance before real artifacts exist.
|
||||
|
||||
Required behavior:
|
||||
- inject only a hint,
|
||||
- tell the agent to run `project_map_init`,
|
||||
- keep the hint visible and inspectable.
|
||||
|
||||
#### Post-init root-pair preload
|
||||
Use `before_agent_start` to guarantee the initial post-init root-pair preload for a prompt.
|
||||
|
||||
Required behavior:
|
||||
- inject root `.pi-map.index.md`,
|
||||
- inject root `.pi-map.md`,
|
||||
- optionally append brief protocol text only if needed by the selected mode.
|
||||
|
||||
#### Relevant-turn reinjection
|
||||
Use `context` for relevant-turn checks.
|
||||
|
||||
Required behavior:
|
||||
- inspect `event.messages`,
|
||||
- decide whether the canonical block is already present,
|
||||
- only add the root pair / budgeted expansion if absent,
|
||||
- avoid rescanning on every trivial turn in `strong` mode.
|
||||
|
||||
#### Payload fallback
|
||||
Use `before_provider_request` only as a fallback or debugging surface when message-layer detection is insufficient.
|
||||
|
||||
### 3. Canonical marker design
|
||||
Deduplication depends on stable canonical detection.
|
||||
|
||||
The implementation should stamp injected content with a canonical marker block.
|
||||
|
||||
Recommended v1 shape:
|
||||
- a deterministic wrapper such as `<!-- PI_MAP_ROOT_PAIR_START -->` / `<!-- PI_MAP_ROOT_PAIR_END -->`,
|
||||
- normalized artifact identity lines for root `.pi-map.index.md` and root `.pi-map.md`,
|
||||
- trust-boundary text within the same wrapped block when the active mode requires it.
|
||||
|
||||
Marker rules:
|
||||
- stable across turns,
|
||||
- independent of provider formatting quirks,
|
||||
- easy to scan in both `event.messages` and final provider payload,
|
||||
- robust enough that root-pair presence can be detected without brittle full-text matching.
|
||||
|
||||
### 4. Budgeting
|
||||
Budgeting should be deterministic and layered.
|
||||
|
||||
#### Context-window discovery and fallback
|
||||
- Prefer active-model context-window metadata exposed by Pi runtime/model selection.
|
||||
- If context-window metadata is unavailable, fall back to the configured absolute cap.
|
||||
- The fallback path should be explicit in logs/debug behavior so budget decisions remain auditable.
|
||||
|
||||
#### Required order
|
||||
1. compute effective budget from relative percentage + optional absolute cap,
|
||||
2. reserve the root pair first,
|
||||
3. expand outward using a deterministic traversal order,
|
||||
4. stop when the budget would be exceeded.
|
||||
|
||||
#### Traversal strategy
|
||||
The spec does not force exact traversal heuristics, but the design should prefer:
|
||||
- root pair first,
|
||||
- shallow structural coverage before deep leaves,
|
||||
- predictable order over opaque scoring.
|
||||
|
||||
This keeps automatic injection understandable and auditable.
|
||||
|
||||
### 5. Mode control surface
|
||||
Expose a config/runtime setting for the four modes.
|
||||
|
||||
Because the current project config is loaded from flat JSON in `.pi-project-map.json`, v1 should prefer a flat compatible shape rather than forcing an immediate nested/YAML migration.
|
||||
|
||||
Suggested v1 config shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"promptInjectionMode": "strong",
|
||||
"contextBudgetPercent": 15,
|
||||
"contextBudgetMaxTokens": 100000
|
||||
}
|
||||
```
|
||||
|
||||
Migration note:
|
||||
- keep existing flat JSON loading in `src/config.ts`,
|
||||
- treat these as new additive keys,
|
||||
- decide explicitly whether existing `contextBudget` is deprecated, ignored for injection, or retained only for older LLM-analysis paths.
|
||||
|
||||
Optional runtime UX may later mirror thinking-level controls, but v1 implementation can start with config-driven mode selection as long as the semantics are the same.
|
||||
|
||||
### 6. Mode semantics
|
||||
#### `off`
|
||||
- no automatic artifact injection,
|
||||
- no startup/init hint beyond existing tool/docs discovery.
|
||||
|
||||
#### `advisory`
|
||||
- startup/init hints enabled,
|
||||
- optional root-pair preload,
|
||||
- light reminders,
|
||||
- weaker reinjection behavior.
|
||||
|
||||
#### `strong`
|
||||
- root-pair preload required,
|
||||
- budgeted expansion required,
|
||||
- relevant-turn reinjection checks required,
|
||||
- reminders before edits and architecture-sensitive reasoning.
|
||||
|
||||
#### `strict`
|
||||
- everything in `strong`, plus:
|
||||
- when protocol path is missing during sensitive actions, require explicit bypass justification.
|
||||
|
||||
### 7. Relevant-turn detection
|
||||
`strong` mode should not rescan on every turn.
|
||||
|
||||
Relevant-turn triggers should be mapped to concrete runtime signals where possible:
|
||||
- agent start,
|
||||
- edit intent / edit tool preparation,
|
||||
- architecture-sensitive planning prompts,
|
||||
- compaction completion,
|
||||
- root-pair artifact change detection.
|
||||
|
||||
This will likely require some combination of:
|
||||
- hook-local heuristics,
|
||||
- observed tool calls,
|
||||
- file timestamp/hash checks for root artifacts,
|
||||
- compaction event handling.
|
||||
|
||||
### 8. Visibility model
|
||||
The design should preserve mixed visibility.
|
||||
|
||||
- startup hints: visible and inspectable,
|
||||
- raw injected artifact blocks: agent-visible by default,
|
||||
- the fact that automatic injection exists should remain discoverable.
|
||||
|
||||
The implementation may use hidden custom message types for artifact payloads, but should not make startup/init behavior opaque.
|
||||
|
||||
### 9. Validation strategy
|
||||
This change is validation-heavy.
|
||||
|
||||
Unit tests alone are not enough, because the main risk is runtime interaction between hooks, message mutation, compaction, and reinjection.
|
||||
|
||||
#### Integration focus areas
|
||||
- startup before init,
|
||||
- startup after init,
|
||||
- root-pair marker insertion,
|
||||
- reinjection suppression when marker already exists,
|
||||
- reinjection after compaction,
|
||||
- reinjection after artifact mutation,
|
||||
- mode differences,
|
||||
- strict-mode bypass path,
|
||||
- mixed visibility expectations,
|
||||
- synthetic event-sequence coverage for relevant-turn heuristics such as edit-intent, architecture-sensitive reasoning, compaction, and artifact invalidation.
|
||||
|
||||
### 10. Documentation impact
|
||||
Runtime guidance docs must align with the spec, but retrieval docs remain separate.
|
||||
|
||||
Docs should teach:
|
||||
- startup hint before init,
|
||||
- root-pair automatic preload after init,
|
||||
- trust boundary,
|
||||
- mode ladder,
|
||||
- relevant-turn reinjection behavior,
|
||||
- integration-test importance.
|
||||
|
||||
## Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| Message-level scanning misses provider serialization quirks | Add payload fallback via `before_provider_request` |
|
||||
| Root-pair marker becomes brittle | Use deterministic boundaries and normalized artifact identity lines |
|
||||
| 15% + 100k is too aggressive in some fleets | Keep both knobs configurable and document the opinionated default |
|
||||
| Relevant-turn detection becomes fuzzy | Centralize detection heuristics and prove them with integration tests |
|
||||
| `strict` mode causes friction | Keep `strong` as default and isolate strict-only bypass behavior |
|
||||
|
||||
## Settled defaults
|
||||
|
||||
- Root pair is always guaranteed after init.
|
||||
- Reinjection avoidance is based on canonical outgoing-context scanning.
|
||||
- Retrieval remains out of scope.
|
||||
- Mixed visibility is the intended baseline.
|
||||
- Four modes exist, with `strong` as default.
|
||||
- Default budget is 15% of active context window, capped at 100k tokens.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Proposal: Project Map Prompt Injection
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Proposal** |
|
||||
| Based on | Grill Me checkpoint + runtime/doc inspection |
|
||||
| Next | Spec |
|
||||
|
||||
## Problem
|
||||
|
||||
`pi-project-map` currently provides strong generated artifacts and tool surfaces, but weak automatic runtime guidance.
|
||||
|
||||
Today the repo has:
|
||||
- a one-time `before_agent_start` hint,
|
||||
- a `session_start` dirty-state UI notification,
|
||||
- tool metadata and docs,
|
||||
- generated `.pi-map.md` / `.pi-map.index.md` artifacts.
|
||||
|
||||
But it lacks a clear spec for:
|
||||
1. when maps/indexes should be injected automatically,
|
||||
2. how much should be injected under a context budget,
|
||||
3. how reinjection should avoid wasting prompt space,
|
||||
4. how visible that guidance should be to the user,
|
||||
5. how strong enforcement should be,
|
||||
6. how to validate this behavior with integration tests.
|
||||
|
||||
## Proposed change
|
||||
|
||||
Adopt a runtime prompt-injection policy for project-map that:
|
||||
- always injects the **root pair** after init,
|
||||
- expands outward under a **hybrid context budget cap**,
|
||||
- avoids redundant reinjection by scanning the actual outgoing context,
|
||||
- keeps **retrieval guidance separate** from this spec,
|
||||
- exposes **four configurable guidance modes** (`off`, `advisory`, `strong`, `strict`),
|
||||
- uses **mixed visibility**: user-visible startup hints, agent-visible artifact injection,
|
||||
- requires **extensive integration tests** for reinjection and context-scanning behavior.
|
||||
|
||||
## In scope
|
||||
|
||||
- [ ] Define pre-init startup hint behavior
|
||||
- [ ] Define post-init automatic root-pair preload behavior
|
||||
- [ ] Define hybrid budget defaults and config shape
|
||||
- [ ] Define outgoing-context scanning for reinjection avoidance
|
||||
- [ ] Define visibility model for hints vs injected artifacts
|
||||
- [ ] Define configurable guidance-strength modes and default
|
||||
- [ ] Define relevant-turn reinjection triggers for `strong`
|
||||
- [ ] Define strict-mode bypass-justification behavior at the spec level
|
||||
- [ ] Define validation and integration-test expectations
|
||||
- [ ] Define implementation slices for runtime hooks/config/tests/docs
|
||||
|
||||
## Out of scope
|
||||
|
||||
- [ ] Redesign retrieval ranking or `project_map_context`
|
||||
- [ ] Merge retrieval behavior into this policy spec
|
||||
- [ ] Redesign paired artifact contents
|
||||
- [ ] Introduce vector stores, Engram, or external retrieval backends
|
||||
- [ ] Guarantee provider-agnostic perfect token counting beyond best-effort budgeting
|
||||
|
||||
## Decisions from grilling
|
||||
|
||||
| Topic | Decision |
|
||||
|---|---|
|
||||
| Change slug | `project-map-prompt-injection` |
|
||||
| Scope split | Retrieval remains a separate spec |
|
||||
| Pre-init | Inject only a lightweight `project_map_init` hint |
|
||||
| Fake/synthetic maps before init | No |
|
||||
| Guaranteed minimum | Always inject root `.pi-map.index.md` + root `.pi-map.md` |
|
||||
| Budget model | Hybrid cap |
|
||||
| Budget config | Relative percentage + optional absolute cap; smaller wins |
|
||||
| Budget default | 15% of context window, capped at 100k tokens |
|
||||
| Reinjection avoidance | Canonical marker scanning in actual outgoing context |
|
||||
| Visibility | Mixed |
|
||||
| Mode set | `off`, `advisory`, `strong`, `strict` |
|
||||
| Default mode | `strong` |
|
||||
| `strong` triggers | Relevant turns only |
|
||||
| Validation | Extensive integration tests required |
|
||||
|
||||
## Success criteria
|
||||
|
||||
- [ ] The spec clearly defines what is injected before and after init
|
||||
- [ ] The spec clearly defines how much is injected and how budgets are applied
|
||||
- [ ] The spec clearly defines when reinjection checks happen and how duplicates are avoided
|
||||
- [ ] The spec clearly defines mode semantics and the default mode
|
||||
- [ ] The spec clearly separates runtime injection policy from retrieval behavior
|
||||
- [ ] The implementation plan can be executed in narrow, reviewable slices
|
||||
- [ ] The validation section makes integration coverage a first-class requirement
|
||||
@@ -0,0 +1,226 @@
|
||||
# Spec: Project Map Prompt Injection
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Spec** |
|
||||
| Based on | [Proposal](proposal.md) |
|
||||
| Next | Design |
|
||||
|
||||
## Overview
|
||||
|
||||
`pi-project-map` must define a deliberate automatic runtime guidance model for map/index usage. After init, the system should preload the root pair, expand under a bounded context budget, avoid redundant reinjection by inspecting actual outgoing context, and scale enforcement through configurable guidance modes.
|
||||
|
||||
This spec covers **automatic injection and maintenance guidance only**. Retrieval behavior remains covered by `map-context-retrieval`.
|
||||
|
||||
## Decisions
|
||||
|
||||
| # | Question | Answer |
|
||||
|---|---|---|
|
||||
| 1 | Retrieval included in this spec | No |
|
||||
| 2 | Pre-init behavior | Lightweight init hint only |
|
||||
| 3 | Synthetic map content before init | No |
|
||||
| 4 | Guaranteed post-init minimum | Always inject root pair |
|
||||
| 5 | Budget model | Hybrid cap |
|
||||
| 6 | Budget knobs | Relative percentage + optional absolute cap |
|
||||
| 7 | Budget default | 15% of context window, capped at 100k tokens |
|
||||
| 8 | Reinjection avoidance | Canonical marker scan in outgoing context/payload |
|
||||
| 9 | Visibility | Mixed |
|
||||
| 10 | Mode set | `off`, `advisory`, `strong`, `strict` |
|
||||
| 11 | Default mode | `strong` |
|
||||
| 12 | `strong` reinjection cadence | Relevant turns only |
|
||||
| 13 | Validation expectation | Extensive integration coverage |
|
||||
|
||||
## Functional requirements
|
||||
|
||||
### 1. Pre-init behavior
|
||||
Before generated map/index artifacts exist, the extension must inject only a lightweight startup hint.
|
||||
|
||||
#### Required behavior
|
||||
- The hint must tell the agent that the project-map extension is active.
|
||||
- The hint must instruct the agent to run `project_map_init`.
|
||||
- The hint must not claim that real map/index artifacts already exist.
|
||||
- The system must not inject synthetic or fake map content before real artifacts are generated.
|
||||
|
||||
### 2. Post-init guaranteed preload
|
||||
After real artifacts exist, the system must guarantee a minimum preload of the root pair:
|
||||
- root `.pi-map.index.md`
|
||||
- root `.pi-map.md`
|
||||
|
||||
This root pair is the minimum automatic preload before budgeted expansion begins.
|
||||
|
||||
### 3. Trust boundary
|
||||
The runtime policy must encode the trust boundary:
|
||||
|
||||
> **index routes, map orients, source decides**
|
||||
|
||||
Required implications:
|
||||
1. Indexes are navigation aids, not final authority.
|
||||
2. Maps provide orientation and architectural context, not final authority.
|
||||
3. Source must remain the final authority before edits or exact behavioral claims.
|
||||
4. If injected artifacts and source disagree, source wins.
|
||||
|
||||
### 4. Budget model
|
||||
Automatic expansion beyond the root pair must use a hybrid context budget cap.
|
||||
|
||||
#### Knobs
|
||||
The system must support:
|
||||
- a **relative context-budget percentage**,
|
||||
- an **optional absolute token cap**.
|
||||
|
||||
If both are configured, the smaller effective budget wins.
|
||||
|
||||
#### Default budget
|
||||
The default must be:
|
||||
- **15%** of the active model context window,
|
||||
- **100k tokens** absolute cap,
|
||||
- use the smaller effective budget.
|
||||
|
||||
#### Context-window discovery and fallback
|
||||
- The runtime should derive the active model context window from Pi model metadata when available.
|
||||
- If the active model context window is unavailable, the runtime must still honor the absolute cap.
|
||||
- In that fallback case, implementations may skip the relative calculation and use the absolute cap as the effective budget.
|
||||
|
||||
#### Expansion rules
|
||||
- Root pair injection happens before budgeted expansion.
|
||||
- Additional map/index artifacts are added only while the budget allows.
|
||||
- The expansion strategy should prefer shallow, high-value structural coverage over deep indiscriminate expansion.
|
||||
- The spec does not require exact provider-token parity; best-effort budgeting is acceptable if it is deterministic and auditable.
|
||||
|
||||
### 5. Reinjection avoidance
|
||||
The extension must avoid redundant reinjection once the root pair is already in active outgoing context.
|
||||
|
||||
#### Required definition
|
||||
"Already in context" must be defined by scanning the **actual outgoing context**, not only by session guesses.
|
||||
|
||||
#### Required behavior
|
||||
- Before reinjecting, scan per-turn `event.messages` for a stable canonical marker or normalized injected root-pair block.
|
||||
- If message-layer evidence is insufficient, the runtime may additionally inspect the final provider payload.
|
||||
- Inject only when the canonical root-pair marker/block is absent.
|
||||
- The spec must allow implementation via explicit scanning logic even if Pi does not expose a convenience API.
|
||||
|
||||
### 6. Visibility model
|
||||
The system must use mixed visibility.
|
||||
|
||||
#### Required behavior
|
||||
- Lightweight startup/init hints should be user-visible and inspectable.
|
||||
- Automatic artifact injection may remain agent-visible by default.
|
||||
- The spec should preserve debuggability: implementations should make the presence of automatic guidance discoverable, even when raw artifact blocks are not fully dumped to the user on every turn.
|
||||
|
||||
### 7. Guidance-strength modes
|
||||
The system must support four named modes:
|
||||
- `off`
|
||||
- `advisory`
|
||||
- `strong`
|
||||
- `strict`
|
||||
|
||||
#### Protocol path definition
|
||||
For this spec, the **protocol path** is present when the current outgoing context contains:
|
||||
1. the canonical injected root-pair block, or an equivalent canonical marker proving that root `.pi-map.index.md` and root `.pi-map.md` are already present, and
|
||||
2. the trust-boundary instruction establishing that `index routes, map orients, source decides`.
|
||||
|
||||
If either element is missing for a sensitive action, the protocol path is missing.
|
||||
|
||||
#### Mode semantics
|
||||
- **`off`**
|
||||
- no automatic injection beyond tool/docs discovery.
|
||||
- **`advisory`**
|
||||
- inject startup/init hints,
|
||||
- allow optional root-pair preload,
|
||||
- use light reminders.
|
||||
- **`strong`**
|
||||
- inject root pair,
|
||||
- expand under the configured budget,
|
||||
- run reinjection checks on relevant turns,
|
||||
- remind before edits or architecture-sensitive reasoning.
|
||||
- **`strict`**
|
||||
- same as `strong`, plus:
|
||||
- require explicit bypass justification before sensitive edits or architectural claims when the protocol path is missing.
|
||||
|
||||
#### Default mode
|
||||
The default mode must be **`strong`**.
|
||||
|
||||
### 8. `strong`-mode trigger timing
|
||||
In `strong` mode, reinjection checks must happen on relevant turns only.
|
||||
|
||||
Required triggers:
|
||||
- agent start,
|
||||
- before edits,
|
||||
- before architecture-sensitive reasoning,
|
||||
- after compaction,
|
||||
- after root-pair artifact changes.
|
||||
|
||||
Required non-trigger:
|
||||
- do not rescan on every trivial turn.
|
||||
|
||||
### 9. Implementation surfaces
|
||||
The runtime may achieve this behavior through a combination of:
|
||||
- startup hooks,
|
||||
- per-turn context hooks,
|
||||
- provider-payload hooks,
|
||||
- prompt guidance,
|
||||
- generated root artifacts.
|
||||
|
||||
The spec intentionally does **not** require one exact implementation mechanism, but it does require the observable runtime behavior above.
|
||||
|
||||
### 10. Validation requirements
|
||||
The implementation must be proven with extensive integration coverage.
|
||||
|
||||
At minimum, integration coverage must validate:
|
||||
- pre-init hint behavior,
|
||||
- post-init root-pair injection,
|
||||
- budgeted expansion behavior,
|
||||
- canonical-marker dedupe,
|
||||
- reinjection after compaction,
|
||||
- reinjection after root-pair artifact changes,
|
||||
- mixed visibility behavior,
|
||||
- guidance-mode differences,
|
||||
- strict-mode bypass behavior where implemented.
|
||||
|
||||
## Non-functional requirements
|
||||
|
||||
- Keep the policy explicit and auditable.
|
||||
- Keep retrieval out of scope for this spec.
|
||||
- Prefer deterministic behavior over opaque heuristics where possible.
|
||||
- Optimize for modern large-context models, while retaining a hard ceiling.
|
||||
- Preserve source as final authority.
|
||||
|
||||
## User flows
|
||||
|
||||
### Flow 1: New repo, maps not initialized
|
||||
1. Agent starts in a repo without project-map artifacts.
|
||||
2. Runtime injects a lightweight visible startup hint.
|
||||
3. Agent is instructed to run `project_map_init`.
|
||||
4. No fake artifact content is injected.
|
||||
|
||||
### Flow 2: Normal initialized repo in `strong` mode
|
||||
1. Agent starts.
|
||||
2. Runtime checks outgoing context for canonical root-pair marker.
|
||||
3. If absent, inject root pair and budgeted expansion.
|
||||
4. On relevant later turns, runtime rescans only when trigger conditions apply.
|
||||
5. Before edits, agent is reminded to use injected context and then verify source.
|
||||
|
||||
### Flow 3: Compaction or artifact invalidation
|
||||
1. Context compacts or root-pair artifacts change.
|
||||
2. Runtime treats that as a relevant reinjection trigger.
|
||||
3. Runtime rescans outgoing context.
|
||||
4. If canonical root-pair block is absent, inject again.
|
||||
|
||||
### Flow 4: Strict-mode sensitive action
|
||||
1. Agent approaches a sensitive edit or architectural claim.
|
||||
2. Runtime checks whether the protocol path is present.
|
||||
3. If not, runtime requires explicit bypass justification before proceeding.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pre-init behavior is hint-only and never injects fake map content
|
||||
- [ ] Post-init behavior always guarantees root pair preload before budgeted expansion
|
||||
- [ ] Budgeting supports both relative and absolute caps, with the smaller effective budget winning
|
||||
- [ ] Default budget is 15% of active context window, capped at 100k tokens, with absolute-cap fallback when context-window metadata is unavailable
|
||||
- [ ] Reinjection avoidance is based on actual outgoing-context scanning
|
||||
- [ ] Mixed visibility is honored
|
||||
- [ ] Four guidance modes exist with `strong` as default
|
||||
- [ ] `strong` checks fire on relevant turns only
|
||||
- [ ] `strict` adds bypass-justification semantics for missing protocol path on sensitive actions
|
||||
- [ ] Extensive integration tests cover context scanning and reinjection behavior
|
||||
@@ -0,0 +1,7 @@
|
||||
# Sync Report
|
||||
|
||||
Status: NOT-APPLICABLE
|
||||
|
||||
Reason: This repository uses legacy flat change artifacts (proposal.md/spec.md/design.md/tasks.md) and does not maintain a canonical openspec/specs/ tree for these changes. No canonical spec sync was performed.
|
||||
|
||||
User-approved fallback: archive completed change as an audit record without canonical spec sync.
|
||||
@@ -0,0 +1,90 @@
|
||||
# Tasks: Project Map Prompt Injection
|
||||
|
||||
## Status
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | **Tasks** |
|
||||
| Based on | [Design](design.md) |
|
||||
| Next | Apply |
|
||||
|
||||
## Delivery slices
|
||||
|
||||
### Slice 1: Injection policy scaffold and config surface
|
||||
**Scope**: mode/config scaffolding, pre-init hint behavior, canonical helper boundaries
|
||||
**Review goal**: establish the policy control surface without yet wiring the full runtime pipeline
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Add a shared prompt-injection policy helper/module
|
||||
2. [ ] Add config support for injection mode, context-budget percent, and absolute cap using additive flat JSON keys compatible with existing `.pi-project-map.json` loading
|
||||
3. [ ] Decide and document how existing `contextBudget` interacts with the new injection-budget knobs
|
||||
4. [ ] Implement pre-init startup-hint behavior with no synthetic artifact injection
|
||||
5. [ ] Define canonical marker/block construction for injected root-pair content
|
||||
6. [ ] Add/update tests for config loading and pre-init hint behavior
|
||||
|
||||
### Slice 2: Root-pair preload and budgeted expansion
|
||||
**Scope**: post-init preload, budget calculation, artifact selection under cap
|
||||
**Review goal**: make automatic injection materially real after init
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Implement guaranteed root-pair preload after init
|
||||
2. [ ] Implement effective-budget calculation using percent + absolute cap with smaller-wins semantics
|
||||
3. [ ] Implement active-model context-window discovery and explicit absolute-cap fallback when metadata is unavailable
|
||||
4. [ ] Implement deterministic budgeted expansion beyond the root pair
|
||||
5. [ ] Keep retrieval explicitly out of this injection path
|
||||
6. [ ] Add/update tests for root-pair guarantee, context-window fallback, and budget-capped expansion
|
||||
|
||||
### Slice 3: Reinjection avoidance and relevant-turn checks
|
||||
**Scope**: outgoing-context scanning, relevant-turn triggers, compaction/artifact invalidation
|
||||
**Review goal**: avoid wasteful reinjection while preserving strong guidance
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Implement canonical marker scanning over `event.messages`
|
||||
2. [ ] Add fallback inspection of final provider payload when needed
|
||||
3. [ ] Implement relevant-turn reinjection triggers for `strong` mode
|
||||
4. [ ] Reinject after compaction and after root-pair artifact changes
|
||||
5. [ ] Add/update integration tests for dedupe, reinjection suppression, and reinjection after invalidation
|
||||
6. [ ] Include synthetic event-sequence coverage for edit-intent, architecture-sensitive reasoning, compaction, and artifact-change heuristics
|
||||
|
||||
### Slice 4: Mode semantics, visibility, and strict-path behavior
|
||||
**Scope**: off/advisory/strong/strict semantics, mixed visibility, strict bypass behavior
|
||||
**Review goal**: make the mode ladder operational and reviewable
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Implement mode-specific behavior for `off`, `advisory`, `strong`, and `strict`
|
||||
2. [ ] Define and enforce the spec meaning of a missing `protocol path` during sensitive actions
|
||||
3. [ ] Keep startup hints user-visible and artifact injection agent-visible by default
|
||||
4. [ ] Implement strict-mode explicit bypass-justification behavior for sensitive edits/architectural claims
|
||||
5. [ ] Add/update integration tests for visibility, protocol-path detection, and mode differences
|
||||
6. [ ] Verify that `strong` remains the default behavior
|
||||
|
||||
### Slice 5: Documentation and runtime alignment
|
||||
**Scope**: docs, runtime guidance text, implementation notes
|
||||
**Review goal**: align user-facing/runtime-facing guidance with the frozen spec
|
||||
|
||||
**Tasks**:
|
||||
1. [ ] Update runtime guidance strings to reflect init-hint behavior, root-pair preload, trust boundary, and mode ladder
|
||||
2. [ ] Update docs/skill guidance for the prompt-injection policy
|
||||
3. [ ] Keep retrieval guidance separate from these docs or clearly reference it as separate
|
||||
4. [ ] Document the opinionated default budget and configurability
|
||||
5. [ ] Summarize integration-test expectations and known risks
|
||||
|
||||
## Acceptance checklist
|
||||
|
||||
- [x] Before init, only a lightweight `project_map_init` hint is injected
|
||||
- [x] No synthetic map/index artifact content is injected before init
|
||||
- [x] After init, root `.pi-map.index.md` and root `.pi-map.md` are always guaranteed before budgeted expansion
|
||||
- [x] Expansion uses a hybrid cap with both relative and absolute knobs, smaller effective budget wins
|
||||
- [x] Default budget is 15% of active context window, capped at 100k tokens
|
||||
- [x] Reinjection avoidance is based on canonical marker scanning in actual outgoing context
|
||||
- [x] `strong` checks only relevant turns, not every trivial turn
|
||||
- [x] Modes `off`, `advisory`, `strong`, and `strict` are implemented with the agreed semantics
|
||||
- [x] Mixed visibility behavior is preserved
|
||||
- [x] Retrieval behavior remains separate from this spec’s implementation scope
|
||||
- [x] Extensive integration tests validate context scanning and reinjection behavior
|
||||
- [x] `npm run typecheck` passes
|
||||
- [x] `npm test` passes
|
||||
- [x] `npm run lint` passes *(N/A: repo has no ESLint config; pre-existing repository gap, not a change regression)*
|
||||
|
||||
## Review workload note
|
||||
This change mixes runtime hook behavior, prompt budgeting, context dedupe, visibility policy, and strict-mode enforcement. Keep delivery narrow and test-heavy. Avoid collapsing this into one oversized implementation slice.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Verify Report
|
||||
|
||||
Status: PASS
|
||||
|
||||
Change: project-map-prompt-injection
|
||||
|
||||
Verified summary: Implemented prompt-injection slices 1-5: mode/config surface, root-pair preload, reinjection/dedupe, strict/advisory/strong/off semantics, and documentation/runtime alignment.
|
||||
|
||||
Evidence commands:
|
||||
- npm run typecheck
|
||||
- npx vitest run
|
||||
- npm run build
|
||||
- node dist/cli.js validate .
|
||||
|
||||
Current validation state:
|
||||
- npm run typecheck: PASS
|
||||
- npx vitest run: PASS (262/262)
|
||||
- npm run build: PASS
|
||||
- node dist/cli.js validate .: PASS
|
||||
|
||||
Notes:
|
||||
- Repo-wide lint remains unavailable because the repository has no ESLint config; treated as a pre-existing repository-level gap, not a change regression.
|
||||
@@ -0,0 +1,30 @@
|
||||
project:
|
||||
name: pi-project-map
|
||||
description: Pi skill and CLI for generating hierarchical .pi-map.md files for AI-oriented codebase comprehension
|
||||
|
||||
stack:
|
||||
runtime:
|
||||
language: TypeScript
|
||||
platform: Node.js
|
||||
cli:
|
||||
entrypoint: dist/cli.js
|
||||
extension:
|
||||
entrypoint: pi-extension.ts
|
||||
testing:
|
||||
framework: Vitest
|
||||
commands:
|
||||
- npm test
|
||||
- npm run typecheck
|
||||
- npm run lint
|
||||
|
||||
sdd:
|
||||
execution_mode: interactive
|
||||
artifact_store: openspec
|
||||
chained_pr_strategy: auto-forecast
|
||||
review_budget_lines: 400
|
||||
|
||||
phase_rules:
|
||||
explore_before_proposal: true
|
||||
spec_before_design: true
|
||||
design_before_tasks: true
|
||||
verify_before_archive: true
|
||||
@@ -0,0 +1,20 @@
|
||||
# OpenSpec Project Context: pi-project-map
|
||||
|
||||
## Product
|
||||
`pi-project-map` is a Pi skill plus CLI that generates hierarchical `.pi-map.md` files throughout a repository so coding agents can orient quickly without reading every source file.
|
||||
|
||||
## Current architecture
|
||||
- `src/init.ts`: full project discovery and map generation
|
||||
- `src/patch.ts`: incremental map updates after file edits
|
||||
- `src/validate.ts`: map freshness and discrepancy checks
|
||||
- `src/format.ts`: markdown map rendering/parsing
|
||||
- `pi-extension.ts`: Pi tool registration and prompt guidance
|
||||
|
||||
## Conventions
|
||||
- `.pi-map.md` files are orientation aids, not runtime truth.
|
||||
- Exact behavior must still be verified from source before editing or making precise claims.
|
||||
- Tooling changes should preserve the core `init` / `patch` / `validate` / `reinit` workflow unless a spec explicitly changes it.
|
||||
- Keep review slices small; prefer staged, reviewable changes over one large refactor.
|
||||
|
||||
## Active planning theme
|
||||
Move from "load all maps up front" toward a layered retrieval model where the root artifacts teach the agent how to navigate, then package maps and source files are loaded on demand or by focused auto-selection.
|
||||
Generated
+608
-568
File diff suppressed because it is too large
Load Diff
+3
-2
@@ -45,7 +45,8 @@
|
||||
"openai": "^6.42.0",
|
||||
"p-limit": "^7.3.0",
|
||||
"picocolors": "^1.1.1",
|
||||
"tree-sitter": "^0.21.0",
|
||||
"tree-sitter-typescript": "^0.21.0"
|
||||
"tree-sitter": "^0.22.4",
|
||||
"tree-sitter-python": "^0.23.6",
|
||||
"tree-sitter-typescript": "^0.23.2"
|
||||
}
|
||||
}
|
||||
|
||||
+280
-19
@@ -2,9 +2,27 @@ import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
|
||||
import { Type } from "typebox";
|
||||
import { readFileSync, readdirSync, statSync } from "fs";
|
||||
import { join, relative } from "path";
|
||||
import { initProject, patchFile, validateMaps, reinitPath } from "./src/index.js";
|
||||
import { createLLMClient } from "./src/llm-client.js";
|
||||
import { LLMError } from "./src/llm-error.js";
|
||||
import {
|
||||
initProject,
|
||||
patchFile,
|
||||
validateMaps,
|
||||
reinitPath,
|
||||
retrieveContext,
|
||||
buildPreInitHint,
|
||||
buildAdvisoryReminder,
|
||||
buildStrictBypassGuard,
|
||||
modeAllowsPreInitHint,
|
||||
modeAllowsInjection,
|
||||
evaluateStrictBypass,
|
||||
discoverContextWindow,
|
||||
buildInjectionPayload,
|
||||
shouldReinjectForEvent,
|
||||
getRootPairMtimes,
|
||||
rootPairChanged,
|
||||
} from "./src/index.js";
|
||||
import { loadConfig } from "./src/config.js";
|
||||
import { createLLMClient } from "./src/llm/llm-client.js";
|
||||
import { LLMError } from "./src/llm/llm-error.js";
|
||||
|
||||
/**
|
||||
* Get the LLM client for Pi runtime.
|
||||
@@ -50,6 +68,19 @@ function isDirty(content: string): boolean {
|
||||
return content.includes("## dirty") && !content.includes("## dirty\n-");
|
||||
}
|
||||
|
||||
function renderProgressBar(
|
||||
completed: number,
|
||||
total: number,
|
||||
currentFile?: string,
|
||||
width = 20,
|
||||
): string {
|
||||
const pct = total > 0 ? completed / total : 0;
|
||||
const filled = Math.round(width * pct);
|
||||
const bar = "█".repeat(filled) + "░".repeat(width - filled);
|
||||
const file = currentFile ? ` → ${currentFile}` : "";
|
||||
return `[${bar}] ${completed}/${total}${file}`;
|
||||
}
|
||||
|
||||
const HINT_CUSTOM_TYPE = "pi-project-map-hint";
|
||||
|
||||
function hintAlreadyInContext(ctx: any): boolean {
|
||||
@@ -70,16 +101,17 @@ function hintAlreadyInContext(ctx: any): boolean {
|
||||
}
|
||||
|
||||
export default function (pi: ExtensionAPI) {
|
||||
let lastRootPairMtimes: import("./src/index.js").RootPairMtimes = {};
|
||||
pi.registerTool({
|
||||
name: "project_map_init",
|
||||
label: "Project Map Init",
|
||||
description:
|
||||
"Generate .pi-map.md analysis files for the entire project or a subdirectory",
|
||||
"Generate paired .pi-map.md and .pi-map.index.md analysis files for the entire project or a subdirectory",
|
||||
promptSnippet:
|
||||
"Initialize project analysis files for codebase understanding",
|
||||
"Initialize paired project map/index artifacts for codebase understanding",
|
||||
promptGuidelines: [
|
||||
"Use project_map_init when starting work on a new project or after significant restructuring",
|
||||
"Run project_map_init when .pi-map.md files are missing or severely outdated",
|
||||
"Run project_map_init when .pi-map.md / .pi-map.index.md files are missing or severely outdated",
|
||||
],
|
||||
parameters: Type.Object({
|
||||
path: Type.Optional(
|
||||
@@ -92,7 +124,29 @@ export default function (pi: ExtensionAPI) {
|
||||
try {
|
||||
const targetPath = params.path || ctx.cwd;
|
||||
const client = getPiLLMClient(ctx);
|
||||
await initProject(targetPath, { verbose: false, llmClient: client });
|
||||
await initProject(targetPath, {
|
||||
verbose: false,
|
||||
llmClient: client,
|
||||
cacheDir: ctx.cwd,
|
||||
onProgress: (info) => {
|
||||
const bar = renderProgressBar(
|
||||
info.completed,
|
||||
info.total,
|
||||
info.currentFile,
|
||||
);
|
||||
_onUpdate?.({
|
||||
content: [{ type: "text", text: bar }],
|
||||
details: {
|
||||
progress:
|
||||
info.total > 0
|
||||
? Math.round((info.completed / info.total) * 100)
|
||||
: 0,
|
||||
file: info.currentFile,
|
||||
dir: info.dir,
|
||||
},
|
||||
});
|
||||
},
|
||||
});
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
@@ -116,8 +170,9 @@ export default function (pi: ExtensionAPI) {
|
||||
name: "project_map_patch",
|
||||
label: "Project Map Patch",
|
||||
description:
|
||||
"Update .pi-map.md for the directory containing a changed file",
|
||||
promptSnippet: "Update project analysis after editing a source file",
|
||||
"Update the paired .pi-map.md / .pi-map.index.md artifacts for the directory containing a changed file",
|
||||
promptSnippet:
|
||||
"Update paired project map/index artifacts after editing a source file",
|
||||
promptGuidelines: [
|
||||
"Use project_map_patch immediately after editing any source file",
|
||||
"Pass the absolute or relative path of the modified file",
|
||||
@@ -130,7 +185,7 @@ export default function (pi: ExtensionAPI) {
|
||||
async execute(_toolCallId, params, _signal, _onUpdate, ctx) {
|
||||
try {
|
||||
const client = getPiLLMClient(ctx);
|
||||
await patchFile(params.file_path, client);
|
||||
await patchFile(params.file_path, client, ctx.cwd);
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
@@ -153,8 +208,9 @@ export default function (pi: ExtensionAPI) {
|
||||
pi.registerTool({
|
||||
name: "project_map_validate",
|
||||
label: "Project Map Validate",
|
||||
description: "Check all .pi-map.md files for staleness and discrepancies",
|
||||
promptSnippet: "Validate project analysis files for accuracy",
|
||||
description:
|
||||
"Check all .pi-map.md / .pi-map.index.md files for staleness and discrepancies",
|
||||
promptSnippet: "Validate paired project map/index artifacts for accuracy",
|
||||
promptGuidelines: [
|
||||
"Use project_map_validate before making architectural decisions if you suspect stale data",
|
||||
"Use project_map_validate to detect files that were deleted or added outside the agent",
|
||||
@@ -196,8 +252,10 @@ export default function (pi: ExtensionAPI) {
|
||||
pi.registerTool({
|
||||
name: "project_map_reinit",
|
||||
label: "Project Map Reinit",
|
||||
description: "Force full regeneration of all .pi-map.md files",
|
||||
promptSnippet: "Force full regeneration of project analysis files",
|
||||
description:
|
||||
"Force full regeneration of all .pi-map.md / .pi-map.index.md artifacts",
|
||||
promptSnippet:
|
||||
"Force full regeneration of paired project map/index artifacts",
|
||||
promptGuidelines: [
|
||||
"Use project_map_reinit when validation shows widespread staleness",
|
||||
"Use project_map_reinit after pulling major changes from version control",
|
||||
@@ -213,7 +271,29 @@ export default function (pi: ExtensionAPI) {
|
||||
try {
|
||||
const targetPath = params.path || ctx.cwd;
|
||||
const client = getPiLLMClient(ctx);
|
||||
await reinitPath(targetPath, { verbose: false, llmClient: client });
|
||||
await reinitPath(targetPath, {
|
||||
verbose: false,
|
||||
llmClient: client,
|
||||
cacheDir: ctx.cwd,
|
||||
onProgress: (info) => {
|
||||
const bar = renderProgressBar(
|
||||
info.completed,
|
||||
info.total,
|
||||
info.currentFile,
|
||||
);
|
||||
_onUpdate?.({
|
||||
content: [{ type: "text", text: bar }],
|
||||
details: {
|
||||
progress:
|
||||
info.total > 0
|
||||
? Math.round((info.completed / info.total) * 100)
|
||||
: 0,
|
||||
file: info.currentFile,
|
||||
dir: info.dir,
|
||||
},
|
||||
});
|
||||
},
|
||||
});
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
@@ -233,6 +313,41 @@ export default function (pi: ExtensionAPI) {
|
||||
},
|
||||
});
|
||||
|
||||
pi.registerTool({
|
||||
name: "project_map_context",
|
||||
label: "Project Map Context",
|
||||
description:
|
||||
"Retrieve a compact markdown context bundle for a natural-language query using paired project map/index metadata",
|
||||
promptSnippet:
|
||||
"Get relevant project context for a task without reading every source file",
|
||||
promptGuidelines: [
|
||||
"Use project_map_context when you need to understand a task area before diving into source",
|
||||
"Pass a concise query describing the feature, bug, or area you want to explore",
|
||||
"Always read the suggested indexes first, then maps, then verify from source",
|
||||
],
|
||||
parameters: Type.Object({
|
||||
query: Type.String({
|
||||
description:
|
||||
"Natural-language query describing the task or area to explore",
|
||||
}),
|
||||
}),
|
||||
async execute(_toolCallId, params, _signal, _onUpdate, ctx) {
|
||||
try {
|
||||
const bundle = retrieveContext(params.query, ctx.cwd);
|
||||
return {
|
||||
content: [{ type: "text", text: bundle }],
|
||||
details: { success: true },
|
||||
};
|
||||
} catch (err: any) {
|
||||
const msg = err instanceof LLMError ? err.message : String(err);
|
||||
return {
|
||||
content: [{ type: "text", text: `Error: ${msg}` }],
|
||||
details: { success: false, error: msg },
|
||||
};
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
// Auto-load .pi-map.md files on session start
|
||||
pi.on("session_start", async (_event, ctx) => {
|
||||
const mapFiles = findPiMapFiles(ctx.cwd);
|
||||
@@ -259,16 +374,162 @@ export default function (pi: ExtensionAPI) {
|
||||
// within the current branch of context. Re-inject after compaction or
|
||||
// tree navigation removes the hint from the active path.
|
||||
pi.on("before_agent_start", async (_event, _ctx) => {
|
||||
const config = loadConfig(_ctx.cwd);
|
||||
const mapFiles = findPiMapFiles(_ctx.cwd);
|
||||
if (mapFiles.length === 0) return {};
|
||||
|
||||
// Mode is off: no injection at all
|
||||
if (config.promptInjectionMode === "off") {
|
||||
return {};
|
||||
}
|
||||
|
||||
// No maps exist yet: show visible pre-init hint, but only if mode allows it
|
||||
if (mapFiles.length === 0) {
|
||||
if (!modeAllowsPreInitHint(config.promptInjectionMode)) {
|
||||
return {};
|
||||
}
|
||||
if (hintAlreadyInContext(_ctx)) return {};
|
||||
return {
|
||||
message: {
|
||||
customType: HINT_CUSTOM_TYPE,
|
||||
content: buildPreInitHint(),
|
||||
display: true,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Slice 4: advisory mode shows a visible lightweight reminder after init.
|
||||
// No root-pair preload, no per-turn reinjection.
|
||||
if (config.promptInjectionMode === "advisory") {
|
||||
if (hintAlreadyInContext(_ctx)) return {};
|
||||
return {
|
||||
message: {
|
||||
customType: HINT_CUSTOM_TYPE,
|
||||
content: buildAdvisoryReminder(),
|
||||
display: true,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Maps exist but mode does not permit automatic artifact injection.
|
||||
if (!modeAllowsInjection(config.promptInjectionMode)) {
|
||||
return {};
|
||||
}
|
||||
|
||||
// Slice 3b: detect root-pair artifact changes
|
||||
const currentMtimes = getRootPairMtimes(_ctx.cwd);
|
||||
const hasPrevious =
|
||||
lastRootPairMtimes.mapMtime !== undefined ||
|
||||
lastRootPairMtimes.indexMtime !== undefined;
|
||||
const artifactChanged =
|
||||
hasPrevious && rootPairChanged(currentMtimes, lastRootPairMtimes);
|
||||
lastRootPairMtimes = currentMtimes;
|
||||
|
||||
// Slice 3a/3b: avoid redundant reinjection by scanning outgoing context
|
||||
const eventType = artifactChanged ? "artifact_change" : "agent_start";
|
||||
const decision = shouldReinjectForEvent(
|
||||
{
|
||||
messages: _event?.messages,
|
||||
type: eventType,
|
||||
},
|
||||
config.promptInjectionMode,
|
||||
);
|
||||
if (!decision.needed) {
|
||||
return {};
|
||||
}
|
||||
|
||||
if (hintAlreadyInContext(_ctx)) return {};
|
||||
|
||||
// Slice 2: post-init root-pair preload + budgeted expansion
|
||||
const contextWindow = discoverContextWindow(_ctx);
|
||||
const payload = buildInjectionPayload(_ctx.cwd, config, contextWindow);
|
||||
return {
|
||||
message: {
|
||||
customType: HINT_CUSTOM_TYPE,
|
||||
content:
|
||||
"📋 Project map active: If you modify any source file, run `project_map_patch` with the file path. If you suspect staleness, run `project_map_validate`.",
|
||||
display: false,
|
||||
content: payload.content,
|
||||
display: payload.display,
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
// Per-turn context scanning for reinjection in strong/strict modes
|
||||
pi.on("context", async (event: any, ctx: any) => {
|
||||
const config = loadConfig(ctx.cwd);
|
||||
const mapFiles = findPiMapFiles(ctx.cwd);
|
||||
|
||||
if (mapFiles.length === 0) {
|
||||
return {};
|
||||
}
|
||||
|
||||
if (!modeAllowsInjection(config.promptInjectionMode)) {
|
||||
return {};
|
||||
}
|
||||
|
||||
// Slice 3b: detect root-pair artifact changes early. Invalidation always
|
||||
// forces reinjection, even in strict mode, because the context is stale.
|
||||
const currentMtimes = getRootPairMtimes(ctx.cwd);
|
||||
const hasPrevious =
|
||||
lastRootPairMtimes.mapMtime !== undefined ||
|
||||
lastRootPairMtimes.indexMtime !== undefined;
|
||||
const artifactChanged =
|
||||
hasPrevious && rootPairChanged(currentMtimes, lastRootPairMtimes);
|
||||
lastRootPairMtimes = currentMtimes;
|
||||
|
||||
if (artifactChanged) {
|
||||
const decision = shouldReinjectForEvent(
|
||||
{
|
||||
messages: event?.messages,
|
||||
type: "artifact_change",
|
||||
payload: event?.payload,
|
||||
},
|
||||
config.promptInjectionMode,
|
||||
);
|
||||
if (!decision.needed) {
|
||||
return {};
|
||||
}
|
||||
const contextWindow = discoverContextWindow(ctx);
|
||||
const payload = buildInjectionPayload(ctx.cwd, config, contextWindow);
|
||||
return {
|
||||
message: {
|
||||
customType: "pi-project-map-hint",
|
||||
content: payload.content,
|
||||
display: payload.display,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Slice 4: strict-mode bypass guard for sensitive actions with missing protocol path.
|
||||
if (config.promptInjectionMode === "strict") {
|
||||
const bypass = evaluateStrictBypass(event, config.promptInjectionMode);
|
||||
if (bypass.guard) {
|
||||
return {
|
||||
message: {
|
||||
customType: "pi-project-map-hint",
|
||||
content: buildStrictBypassGuard(bypass.reason),
|
||||
display: true,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const decision = shouldReinjectForEvent(
|
||||
{
|
||||
messages: event?.messages,
|
||||
type: event?.type,
|
||||
payload: event?.payload,
|
||||
},
|
||||
config.promptInjectionMode,
|
||||
);
|
||||
if (!decision.needed) {
|
||||
return {};
|
||||
}
|
||||
|
||||
const contextWindow = discoverContextWindow(ctx);
|
||||
const payload = buildInjectionPayload(ctx.cwd, config, contextWindow);
|
||||
return {
|
||||
message: {
|
||||
customType: "pi-project-map-hint",
|
||||
content: payload.content,
|
||||
display: payload.display,
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
@@ -1,238 +0,0 @@
|
||||
import { readFileSync } from "fs";
|
||||
import { extname } from "path";
|
||||
|
||||
interface ASTFileData {
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
}
|
||||
|
||||
const LANGUAGE_MAP: Record<string, string> = {
|
||||
".ts": "typescript",
|
||||
".tsx": "tsx",
|
||||
".js": "javascript",
|
||||
".jsx": "javascript",
|
||||
".mjs": "javascript",
|
||||
".py": "python",
|
||||
".go": "go",
|
||||
".rs": "rust",
|
||||
".java": "java",
|
||||
".c": "c",
|
||||
".cpp": "cpp",
|
||||
".h": "c",
|
||||
".rb": "ruby",
|
||||
};
|
||||
|
||||
export async function extractFileAST(
|
||||
filePath: string,
|
||||
): Promise<ASTFileData | null> {
|
||||
const ext = extname(filePath).toLowerCase();
|
||||
const langName = LANGUAGE_MAP[ext];
|
||||
if (!langName) return null;
|
||||
|
||||
try {
|
||||
const Parser = require("tree-sitter");
|
||||
let grammar: unknown;
|
||||
|
||||
if (langName === "typescript" || langName === "tsx") {
|
||||
const ts = require("tree-sitter-typescript");
|
||||
grammar = langName === "tsx" ? ts.tsx : ts.typescript;
|
||||
} else {
|
||||
// For other languages, try to require the grammar package
|
||||
try {
|
||||
const pkg = require(`tree-sitter-${langName}`);
|
||||
// Some packages export { language }, others export the grammar directly
|
||||
grammar = pkg.language || pkg.default || pkg;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
if (!grammar) return null;
|
||||
|
||||
const parser = new Parser();
|
||||
parser.setLanguage(grammar);
|
||||
|
||||
const content = readFileSync(filePath, "utf8");
|
||||
const tree = parser.parse(content);
|
||||
|
||||
const exports = extractExportsFromTree(tree, langName);
|
||||
const deps = extractDepsFromTree(tree, langName);
|
||||
|
||||
return { exports, deps };
|
||||
} catch {
|
||||
// Graceful fallback if tree-sitter fails
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function extractExportsFromTree(tree: Tree, langName: string): string[] {
|
||||
const exports: string[] = [];
|
||||
const root = tree.rootNode;
|
||||
|
||||
function visit(node: SyntaxNode) {
|
||||
if (
|
||||
langName === "typescript" ||
|
||||
langName === "tsx" ||
|
||||
langName === "javascript"
|
||||
) {
|
||||
if (node.type === "export_statement") {
|
||||
// export function foo
|
||||
// export class Foo
|
||||
// export const foo
|
||||
// export { foo, bar }
|
||||
// export default foo
|
||||
const declaration = node.childForFieldName?.("declaration");
|
||||
if (declaration) {
|
||||
const nameNode = findIdentifier(declaration);
|
||||
if (nameNode) exports.push(nameNode.text);
|
||||
} else {
|
||||
// export { ... }
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child?.type === "export_clause") {
|
||||
for (let j = 0; j < child.childCount; j++) {
|
||||
const spec = child.child(j);
|
||||
if (spec?.type === "export_specifier") {
|
||||
const nameNode = spec.childForFieldName?.("name");
|
||||
if (nameNode) exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if (langName === "python") {
|
||||
if (
|
||||
node.type === "function_definition" ||
|
||||
node.type === "class_definition"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode) exports.push(nameNode.text);
|
||||
}
|
||||
} else if (langName === "go") {
|
||||
if (
|
||||
node.type === "function_declaration" ||
|
||||
node.type === "type_declaration" ||
|
||||
node.type === "var_declaration" ||
|
||||
node.type === "const_declaration"
|
||||
) {
|
||||
const nameNode = findIdentifier(node);
|
||||
if (nameNode && /^[A-Z]/.test(nameNode.text)) {
|
||||
exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
} else if (langName === "rust") {
|
||||
if (
|
||||
node.type === "function_item" ||
|
||||
node.type === "struct_item" ||
|
||||
node.type === "enum_item"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode) exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) visit(child);
|
||||
}
|
||||
}
|
||||
|
||||
visit(root);
|
||||
return [...new Set(exports)];
|
||||
}
|
||||
|
||||
function extractDepsFromTree(tree: Tree, langName: string): string[] {
|
||||
const deps: string[] = [];
|
||||
const root = tree.rootNode;
|
||||
|
||||
function visit(node: SyntaxNode) {
|
||||
if (
|
||||
langName === "typescript" ||
|
||||
langName === "tsx" ||
|
||||
langName === "javascript"
|
||||
) {
|
||||
if (
|
||||
node.type === "import_statement" ||
|
||||
node.type === "import_declaration"
|
||||
) {
|
||||
const source = node.childForFieldName?.("source");
|
||||
if (source) {
|
||||
const text = source.text;
|
||||
// Remove quotes
|
||||
deps.push(text.slice(1, -1));
|
||||
}
|
||||
}
|
||||
// CommonJS: require("...")
|
||||
if (node.type === "call_expression") {
|
||||
const func = node.childForFieldName?.("function");
|
||||
if (func?.text === "require") {
|
||||
const args = node.childForFieldName?.("arguments");
|
||||
if (args && args.childCount > 0) {
|
||||
const firstArg = args.child(0);
|
||||
if (firstArg?.type === "string") {
|
||||
deps.push(firstArg.text.slice(1, -1));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if (langName === "python") {
|
||||
if (
|
||||
node.type === "import_statement" ||
|
||||
node.type === "import_from_statement"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode) deps.push(nameNode.text);
|
||||
}
|
||||
} else if (langName === "go") {
|
||||
if (node.type === "import_spec") {
|
||||
const pathNode = node.childForFieldName?.("path");
|
||||
if (pathNode) deps.push(pathNode.text.slice(1, -1));
|
||||
}
|
||||
} else if (langName === "rust") {
|
||||
if (node.type === "use_declaration") {
|
||||
const argument = node.childForFieldName?.("argument");
|
||||
if (argument) deps.push(argument.text);
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) visit(child);
|
||||
}
|
||||
}
|
||||
|
||||
visit(root);
|
||||
return [...new Set(deps)];
|
||||
}
|
||||
|
||||
function findIdentifier(node: SyntaxNode): SyntaxNode | null {
|
||||
if (
|
||||
node.type === "identifier" ||
|
||||
node.type === "type_identifier" ||
|
||||
node.type === "property_identifier"
|
||||
) {
|
||||
return node;
|
||||
}
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) {
|
||||
const found = findIdentifier(child);
|
||||
if (found) return found;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Type definitions for tree-sitter nodes (simplified)
|
||||
interface Tree {
|
||||
rootNode: SyntaxNode;
|
||||
}
|
||||
|
||||
interface SyntaxNode {
|
||||
type: string;
|
||||
text: string;
|
||||
childCount: number;
|
||||
childForFieldName?(name: string): SyntaxNode | null;
|
||||
child(index: number): SyntaxNode | null;
|
||||
}
|
||||
@@ -0,0 +1,651 @@
|
||||
import { readFileSync } from "fs";
|
||||
import { extname } from "path";
|
||||
|
||||
interface MethodData {
|
||||
name: string;
|
||||
params: string[];
|
||||
returns?: string;
|
||||
calls: string[];
|
||||
raises: string[];
|
||||
}
|
||||
|
||||
interface ClassData {
|
||||
name: string;
|
||||
methods: MethodData[];
|
||||
}
|
||||
|
||||
export interface ASTFileData {
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
classes: ClassData[];
|
||||
functions: MethodData[];
|
||||
}
|
||||
|
||||
const LANGUAGE_MAP: Record<string, string> = {
|
||||
".ts": "typescript",
|
||||
".tsx": "tsx",
|
||||
".js": "javascript",
|
||||
".jsx": "javascript",
|
||||
".mjs": "javascript",
|
||||
".py": "python",
|
||||
".go": "go",
|
||||
".rs": "rust",
|
||||
".java": "java",
|
||||
".c": "c",
|
||||
".cpp": "cpp",
|
||||
".h": "c",
|
||||
".rb": "ruby",
|
||||
};
|
||||
|
||||
export async function extractFileAST(
|
||||
filePath: string,
|
||||
): Promise<ASTFileData | null> {
|
||||
const ext = extname(filePath).toLowerCase();
|
||||
const langName = LANGUAGE_MAP[ext];
|
||||
if (!langName) return null;
|
||||
|
||||
try {
|
||||
const Parser = require("tree-sitter");
|
||||
let grammar: unknown;
|
||||
|
||||
if (langName === "typescript" || langName === "tsx") {
|
||||
const ts = require("tree-sitter-typescript");
|
||||
grammar = langName === "tsx" ? ts.tsx : ts.typescript;
|
||||
} else {
|
||||
try {
|
||||
const pkg = require(`tree-sitter-${langName}`);
|
||||
// Some grammars export the language directly (e.g. python),
|
||||
// others export { typescript, tsx } (e.g. typescript)
|
||||
grammar = pkg.typescript || pkg.tsx || pkg.go || pkg;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
if (!grammar) return null;
|
||||
|
||||
const parser = new Parser();
|
||||
parser.setLanguage(grammar);
|
||||
|
||||
const content = readFileSync(filePath, "utf8");
|
||||
const tree = parser.parse(content);
|
||||
|
||||
if (langName === "python") {
|
||||
return extractPythonData(tree, content);
|
||||
}
|
||||
if (
|
||||
langName === "typescript" ||
|
||||
langName === "tsx" ||
|
||||
langName === "javascript"
|
||||
) {
|
||||
return extractTypeScriptData(tree, content);
|
||||
}
|
||||
if (langName === "go") {
|
||||
return extractGoData(tree, content);
|
||||
}
|
||||
|
||||
// Fallback for other languages
|
||||
const exports = extractExportsFromTree(tree, langName);
|
||||
const deps = extractDepsFromTree(tree, langName);
|
||||
return { exports, deps, classes: [], functions: [] };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// PYTHON
|
||||
// ============================================================================
|
||||
|
||||
function extractPythonData(tree: Tree, source: string): ASTFileData {
|
||||
const classes: ClassData[] = [];
|
||||
const exports: string[] = [];
|
||||
const deps: string[] = [];
|
||||
const functions: MethodData[] = [];
|
||||
|
||||
function visit(node: SyntaxNode) {
|
||||
// Imports
|
||||
if (
|
||||
node.type === "import_statement" ||
|
||||
node.type === "import_from_statement"
|
||||
) {
|
||||
const moduleNode = node.childForFieldName?.("module_name");
|
||||
if (moduleNode) {
|
||||
deps.push(moduleNode.text);
|
||||
} else {
|
||||
// import a, b, c
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child?.type === "dotted_name" || child?.type === "identifier") {
|
||||
deps.push(child.text);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Classes
|
||||
if (node.type === "class_definition") {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (!nameNode) return;
|
||||
const className = nameNode.text;
|
||||
exports.push(className);
|
||||
|
||||
const methods: MethodData[] = [];
|
||||
const body = node.childForFieldName?.("body");
|
||||
if (body) {
|
||||
for (let i = 0; i < body.childCount; i++) {
|
||||
const child = body.child(i);
|
||||
if (child?.type === "function_definition") {
|
||||
const method = extractPythonMethod(child, source);
|
||||
if (method) methods.push(method);
|
||||
}
|
||||
}
|
||||
}
|
||||
classes.push({ name: className, methods });
|
||||
return; // don't recurse into class body
|
||||
}
|
||||
|
||||
// Top-level functions
|
||||
if (node.type === "function_definition") {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode) {
|
||||
exports.push(nameNode.text);
|
||||
const method = extractPythonMethod(node, source);
|
||||
if (method) functions.push(method);
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) visit(child);
|
||||
}
|
||||
}
|
||||
|
||||
visit(tree.rootNode);
|
||||
return {
|
||||
exports: [...new Set(exports)],
|
||||
deps: [...new Set(deps)],
|
||||
classes,
|
||||
functions,
|
||||
};
|
||||
}
|
||||
|
||||
function extractPythonMethod(
|
||||
node: SyntaxNode,
|
||||
_source: string,
|
||||
): MethodData | null {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (!nameNode) return null;
|
||||
|
||||
const params: string[] = [];
|
||||
const parameters = node.childForFieldName?.("parameters");
|
||||
if (parameters) {
|
||||
for (let i = 0; i < parameters.childCount; i++) {
|
||||
const param = parameters.child(i);
|
||||
if (param?.type === "identifier" || param?.type === "typed_parameter") {
|
||||
params.push(param.text);
|
||||
} else if (param?.type === "typed_default_parameter") {
|
||||
// name: type = default
|
||||
const nameChild = param.childForFieldName?.("name");
|
||||
if (nameChild) params.push(nameChild.text);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Return type
|
||||
let returns: string | undefined;
|
||||
const returnType = node.childForFieldName?.("return_type");
|
||||
if (returnType) {
|
||||
returns = returnType.text.replace(/^->\s*/, "");
|
||||
}
|
||||
|
||||
// Calls and raises
|
||||
const calls: string[] = [];
|
||||
const raises: string[] = [];
|
||||
const body = node.childForFieldName?.("body");
|
||||
if (body) {
|
||||
extractPythonCallsAndRaises(body, calls, raises);
|
||||
}
|
||||
|
||||
return {
|
||||
name: nameNode.text,
|
||||
params,
|
||||
returns,
|
||||
calls: dedupeCallChains(calls),
|
||||
raises: [...new Set(raises)],
|
||||
};
|
||||
}
|
||||
|
||||
function dedupeCallChains(calls: string[]): string[] {
|
||||
const unique = [...new Set(calls)];
|
||||
// Remove shorter calls that are prefixes of longer ones
|
||||
return unique.filter(
|
||||
(call) =>
|
||||
!unique.some(
|
||||
(other) =>
|
||||
other !== call &&
|
||||
(other.startsWith(`${call}.`) || other.startsWith(`${call}(`)),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
function extractPythonCallsAndRaises(
|
||||
node: SyntaxNode,
|
||||
calls: string[],
|
||||
raises: string[],
|
||||
) {
|
||||
if (node.type === "call") {
|
||||
const func = node.childForFieldName?.("function");
|
||||
if (func) {
|
||||
const callStr = extractCallChain(func);
|
||||
if (callStr) calls.push(callStr);
|
||||
}
|
||||
}
|
||||
if (node.type === "raise_statement") {
|
||||
const exc = node.child(1);
|
||||
if (exc) {
|
||||
const excType =
|
||||
exc.type === "call" ? exc.childForFieldName?.("function") : exc;
|
||||
if (excType) raises.push(excType.text);
|
||||
}
|
||||
}
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child && node.type !== "raise_statement")
|
||||
extractPythonCallsAndRaises(child, calls, raises);
|
||||
}
|
||||
}
|
||||
|
||||
function extractCallChain(node: SyntaxNode): string | null {
|
||||
if (
|
||||
node.type === "identifier" ||
|
||||
node.type === "attribute" ||
|
||||
node.type === "member_expression" ||
|
||||
node.type === "property_identifier" ||
|
||||
node.type === "type_identifier"
|
||||
) {
|
||||
return node.text.replace(/\s+/g, " ").trim();
|
||||
}
|
||||
if (node.type === "call" || node.type === "call_expression") {
|
||||
const func = node.childForFieldName?.("function");
|
||||
if (func) return extractCallChain(func);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// TYPESCRIPT / JAVASCRIPT
|
||||
// ============================================================================
|
||||
|
||||
function extractTypeScriptData(tree: Tree, source: string): ASTFileData {
|
||||
const classes: ClassData[] = [];
|
||||
const exports: string[] = [];
|
||||
const deps: string[] = [];
|
||||
const functions: MethodData[] = [];
|
||||
|
||||
function visit(node: SyntaxNode) {
|
||||
// Imports
|
||||
if (
|
||||
node.type === "import_statement" ||
|
||||
node.type === "import_declaration"
|
||||
) {
|
||||
const sourceNode = node.childForFieldName?.("source");
|
||||
if (sourceNode) {
|
||||
deps.push(sourceNode.text.slice(1, -1)); // remove quotes
|
||||
}
|
||||
}
|
||||
if (node.type === "call_expression") {
|
||||
const func = node.childForFieldName?.("function");
|
||||
if (func?.text === "require") {
|
||||
const args = node.childForFieldName?.("arguments");
|
||||
if (args && args.childCount > 0) {
|
||||
const firstArg = args.child(0);
|
||||
if (firstArg?.type === "string") {
|
||||
deps.push(firstArg.text.slice(1, -1));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Classes
|
||||
if (node.type === "class_declaration" || node.type === "class") {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (!nameNode) return;
|
||||
const className = nameNode.text;
|
||||
exports.push(className);
|
||||
|
||||
const methods: MethodData[] = [];
|
||||
const body = node.childForFieldName?.("body");
|
||||
if (body) {
|
||||
for (let i = 0; i < body.childCount; i++) {
|
||||
const child = body.child(i);
|
||||
if (
|
||||
child?.type === "method_definition" ||
|
||||
child?.type === "function_definition"
|
||||
) {
|
||||
const method = extractTSMethod(child, source);
|
||||
if (method) methods.push(method);
|
||||
}
|
||||
}
|
||||
}
|
||||
classes.push({ name: className, methods });
|
||||
}
|
||||
|
||||
// Exported functions/consts
|
||||
if (
|
||||
node.type === "export_statement" ||
|
||||
node.type === "export_declaration"
|
||||
) {
|
||||
const declaration = node.childForFieldName?.("declaration");
|
||||
if (declaration) {
|
||||
const nameNode = findIdentifier(declaration);
|
||||
if (nameNode) {
|
||||
exports.push(nameNode.text);
|
||||
if (
|
||||
declaration.type === "function_declaration" ||
|
||||
declaration.type === "function"
|
||||
) {
|
||||
const method = extractTSMethod(declaration, source);
|
||||
if (method) functions.push(method);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) visit(child);
|
||||
}
|
||||
}
|
||||
|
||||
visit(tree.rootNode);
|
||||
return {
|
||||
exports: [...new Set(exports)],
|
||||
deps: [...new Set(deps)],
|
||||
classes,
|
||||
functions,
|
||||
};
|
||||
}
|
||||
|
||||
function extractTSMethod(node: SyntaxNode, _source: string): MethodData | null {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (!nameNode) return null;
|
||||
|
||||
const params: string[] = [];
|
||||
const parameters = node.childForFieldName?.("parameters");
|
||||
if (parameters) {
|
||||
for (let i = 0; i < parameters.childCount; i++) {
|
||||
const param = parameters.child(i);
|
||||
if (
|
||||
param?.type === "identifier" ||
|
||||
param?.type === "required_parameter" ||
|
||||
param?.type === "optional_parameter"
|
||||
) {
|
||||
const name =
|
||||
param.childForFieldName?.("pattern") ||
|
||||
param.childForFieldName?.("name");
|
||||
if (name) {
|
||||
const typeAnnotation = param.childForFieldName?.("type");
|
||||
if (typeAnnotation) {
|
||||
const typeText = typeAnnotation.text.replace(/^:\s*/, "");
|
||||
params.push(`${name.text}: ${typeText}`);
|
||||
} else {
|
||||
params.push(name.text);
|
||||
}
|
||||
} else {
|
||||
params.push(param.text);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Return type
|
||||
let returns: string | undefined;
|
||||
const returnType = node.childForFieldName?.("return_type");
|
||||
if (returnType) {
|
||||
returns = returnType.text.replace(/^:\s*/, "");
|
||||
}
|
||||
|
||||
// Calls and raises
|
||||
const calls: string[] = [];
|
||||
const raises: string[] = [];
|
||||
const body = node.childForFieldName?.("body");
|
||||
if (body) {
|
||||
extractTSCallsAndThrows(body, calls, raises);
|
||||
}
|
||||
|
||||
return {
|
||||
name: nameNode.text,
|
||||
params,
|
||||
returns,
|
||||
calls: dedupeCallChains(calls),
|
||||
raises: [...new Set(raises)],
|
||||
};
|
||||
}
|
||||
|
||||
function extractTSCallsAndThrows(
|
||||
node: SyntaxNode,
|
||||
calls: string[],
|
||||
raises: string[],
|
||||
) {
|
||||
if (node.type === "call_expression") {
|
||||
const func = node.childForFieldName?.("function");
|
||||
if (func) {
|
||||
const callStr = extractCallChain(func);
|
||||
if (callStr) calls.push(callStr);
|
||||
}
|
||||
}
|
||||
if (node.type === "throw_statement") {
|
||||
const exc = node.child(1);
|
||||
if (exc) {
|
||||
// Strip 'new ' prefix and extract just the error type
|
||||
const text = exc.text.replace(/^new\s+/, "");
|
||||
const match = text.match(/^(\w+)/);
|
||||
if (match) raises.push(match[1]);
|
||||
}
|
||||
}
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child && node.type !== "throw_statement")
|
||||
extractTSCallsAndThrows(child, calls, raises);
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// GO
|
||||
// ============================================================================
|
||||
|
||||
function extractGoData(tree: Tree, _source: string): ASTFileData {
|
||||
const exports: string[] = [];
|
||||
const deps: string[] = [];
|
||||
|
||||
function visit(node: SyntaxNode) {
|
||||
if (node.type === "import_spec") {
|
||||
const pathNode = node.childForFieldName?.("path");
|
||||
if (pathNode) deps.push(pathNode.text.slice(1, -1));
|
||||
}
|
||||
|
||||
if (
|
||||
node.type === "function_declaration" ||
|
||||
node.type === "method_declaration"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode && /^[A-Z]/.test(nameNode.text)) {
|
||||
exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
|
||||
if (node.type === "type_declaration") {
|
||||
const spec = node.childForFieldName?.("spec");
|
||||
if (spec) {
|
||||
const nameNode = spec.childForFieldName?.("name");
|
||||
if (nameNode && /^[A-Z]/.test(nameNode.text)) {
|
||||
exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) visit(child);
|
||||
}
|
||||
}
|
||||
|
||||
visit(tree.rootNode);
|
||||
return {
|
||||
exports: [...new Set(exports)],
|
||||
deps: [...new Set(deps)],
|
||||
classes: [],
|
||||
functions: [],
|
||||
};
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// GENERIC FALLBACK
|
||||
// ============================================================================
|
||||
|
||||
function extractExportsFromTree(tree: Tree, langName: string): string[] {
|
||||
const exports: string[] = [];
|
||||
const root = tree.rootNode;
|
||||
|
||||
function visit(node: SyntaxNode) {
|
||||
if (
|
||||
langName === "typescript" ||
|
||||
langName === "tsx" ||
|
||||
langName === "javascript"
|
||||
) {
|
||||
if (node.type === "export_statement") {
|
||||
const declaration = node.childForFieldName?.("declaration");
|
||||
if (declaration) {
|
||||
const nameNode = findIdentifier(declaration);
|
||||
if (nameNode) exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
} else if (langName === "python") {
|
||||
if (
|
||||
node.type === "function_definition" ||
|
||||
node.type === "class_definition"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode) exports.push(nameNode.text);
|
||||
}
|
||||
} else if (langName === "go") {
|
||||
if (
|
||||
node.type === "function_declaration" ||
|
||||
node.type === "type_declaration"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode && /^[A-Z]/.test(nameNode.text)) {
|
||||
exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
} else if (langName === "rust") {
|
||||
if (
|
||||
node.type === "function_item" ||
|
||||
node.type === "struct_item" ||
|
||||
node.type === "enum_item"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode) exports.push(nameNode.text);
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) visit(child);
|
||||
}
|
||||
}
|
||||
|
||||
visit(root);
|
||||
return [...new Set(exports)];
|
||||
}
|
||||
|
||||
function extractDepsFromTree(tree: Tree, langName: string): string[] {
|
||||
const deps: string[] = [];
|
||||
const root = tree.rootNode;
|
||||
|
||||
function visit(node: SyntaxNode) {
|
||||
if (
|
||||
langName === "typescript" ||
|
||||
langName === "tsx" ||
|
||||
langName === "javascript"
|
||||
) {
|
||||
if (
|
||||
node.type === "import_statement" ||
|
||||
node.type === "import_declaration"
|
||||
) {
|
||||
const source = node.childForFieldName?.("source");
|
||||
if (source) deps.push(source.text.slice(1, -1));
|
||||
}
|
||||
if (node.type === "call_expression") {
|
||||
const func = node.childForFieldName?.("function");
|
||||
if (func?.text === "require") {
|
||||
const args = node.childForFieldName?.("arguments");
|
||||
if (args && args.childCount > 0) {
|
||||
const firstArg = args.child(0);
|
||||
if (firstArg?.type === "string") {
|
||||
deps.push(firstArg.text.slice(1, -1));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if (langName === "python") {
|
||||
if (
|
||||
node.type === "import_statement" ||
|
||||
node.type === "import_from_statement"
|
||||
) {
|
||||
const nameNode = node.childForFieldName?.("name");
|
||||
if (nameNode) deps.push(nameNode.text);
|
||||
}
|
||||
} else if (langName === "go") {
|
||||
if (node.type === "import_spec") {
|
||||
const pathNode = node.childForFieldName?.("path");
|
||||
if (pathNode) deps.push(pathNode.text.slice(1, -1));
|
||||
}
|
||||
} else if (langName === "rust") {
|
||||
if (node.type === "use_declaration") {
|
||||
const argument = node.childForFieldName?.("argument");
|
||||
if (argument) deps.push(argument.text);
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) visit(child);
|
||||
}
|
||||
}
|
||||
|
||||
visit(root);
|
||||
return [...new Set(deps)];
|
||||
}
|
||||
|
||||
function findIdentifier(node: SyntaxNode): SyntaxNode | null {
|
||||
if (
|
||||
node.type === "identifier" ||
|
||||
node.type === "type_identifier" ||
|
||||
node.type === "property_identifier"
|
||||
) {
|
||||
return node;
|
||||
}
|
||||
for (let i = 0; i < node.childCount; i++) {
|
||||
const child = node.child(i);
|
||||
if (child) {
|
||||
const found = findIdentifier(child);
|
||||
if (found) return found;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Type definitions for tree-sitter nodes (simplified)
|
||||
interface Tree {
|
||||
rootNode: SyntaxNode;
|
||||
}
|
||||
|
||||
interface SyntaxNode {
|
||||
type: string;
|
||||
text: string;
|
||||
childCount: number;
|
||||
childForFieldName?(name: string): SyntaxNode | null;
|
||||
child(index: number): SyntaxNode | null;
|
||||
}
|
||||
+1
-200
@@ -1,201 +1,2 @@
|
||||
#!/usr/bin/env node
|
||||
import { initProject } from "./init.js";
|
||||
import { patchFile } from "./patch.js";
|
||||
import { validateMaps } from "./validate.js";
|
||||
import { reinitPath } from "./init.js";
|
||||
import { discoverProject } from "./discover.js";
|
||||
import { createLLMClient, LLMError } from "./llm-client.js";
|
||||
import { loadConfig } from "./config.js";
|
||||
import pc from "picocolors";
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const command = args[0];
|
||||
|
||||
function printUsage() {
|
||||
console.log(`${pc.bold("project-map")} — hierarchical project analysis for Pi agents
|
||||
`);
|
||||
console.log(`${pc.bold("Usage:")}`);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("init")} [path] Generate .pi-map.md files for all directories`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("patch")} <file> Update analysis for a changed file's directory`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("validate")} [--fix] [path] Check for stale/missing/orphaned entries`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("reinit")} [path] Force full regeneration`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("--help")} Show this help message`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("--version")} Show version\n`,
|
||||
);
|
||||
console.log(`${pc.bold("Options:")}`);
|
||||
console.log(
|
||||
` --llm-provider=openai|kimi LLM provider (default: config or openai)`,
|
||||
);
|
||||
console.log(
|
||||
` --llm-model=<model> LLM model name (or set LLM_MODEL env var)`,
|
||||
);
|
||||
console.log(` --llm-base-url=<url> Custom base URL for LLM API\n`);
|
||||
console.log(`${pc.bold("Examples:")}`);
|
||||
console.log(` project-map init`);
|
||||
console.log(` project-map patch src/components/Button.tsx`);
|
||||
console.log(` project-map validate --fix`);
|
||||
console.log(` project-map reinit`);
|
||||
console.log(
|
||||
` project-map init --llm-provider=kimi --llm-model=kimi-k2-6`,
|
||||
);
|
||||
}
|
||||
|
||||
function printVersion() {
|
||||
const pkg = require("../package.json");
|
||||
console.log(pkg.version);
|
||||
}
|
||||
|
||||
function formatCount(count: number, label: string): string {
|
||||
const plural = label.endsWith("y")
|
||||
? `${label.slice(0, -1)}ies`
|
||||
: `${label}${count === 1 ? "" : "s"}`;
|
||||
return `${pc.bold(String(count))} ${count === 1 ? label : plural}`;
|
||||
}
|
||||
|
||||
function parseArgs(args: string[]): {
|
||||
path: string;
|
||||
fix: boolean;
|
||||
llmProvider?: string;
|
||||
llmModel?: string;
|
||||
llmBaseUrl?: string;
|
||||
positional: string[];
|
||||
} {
|
||||
let path = ".";
|
||||
let fix = false;
|
||||
let llmProvider: string | undefined;
|
||||
let llmModel: string | undefined;
|
||||
let llmBaseUrl: string | undefined;
|
||||
const positional: string[] = [];
|
||||
|
||||
for (const arg of args.slice(1)) {
|
||||
if (arg === "--fix") {
|
||||
fix = true;
|
||||
} else if (arg.startsWith("--llm-provider=")) {
|
||||
llmProvider = arg.slice("--llm-provider=".length);
|
||||
} else if (arg.startsWith("--llm-model=")) {
|
||||
llmModel = arg.slice("--llm-model=".length);
|
||||
} else if (arg.startsWith("--llm-base-url=")) {
|
||||
llmBaseUrl = arg.slice("--llm-base-url=".length);
|
||||
} else if (!arg.startsWith("-")) {
|
||||
positional.push(arg);
|
||||
path = arg;
|
||||
}
|
||||
}
|
||||
|
||||
return { path, fix, llmProvider, llmModel, llmBaseUrl, positional };
|
||||
}
|
||||
|
||||
function createClientFromArgs(args: ReturnType<typeof parseArgs>) {
|
||||
const config = loadConfig();
|
||||
const provider = (args.llmProvider || config.llmProvider) as
|
||||
| "openai"
|
||||
| "kimi"
|
||||
| "pi";
|
||||
return createLLMClient(provider, {
|
||||
model: args.llmModel || config.llmModel || process.env.LLM_MODEL,
|
||||
baseUrl: args.llmBaseUrl || config.llmBaseUrl,
|
||||
});
|
||||
}
|
||||
|
||||
async function main() {
|
||||
if (!command || command === "--help" || command === "-h") {
|
||||
printUsage();
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (command === "--version" || command === "-v") {
|
||||
printVersion();
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const parsed = parseArgs(args);
|
||||
|
||||
switch (command) {
|
||||
case "init": {
|
||||
const targetPath = parsed.positional[0] || ".";
|
||||
const start = Date.now();
|
||||
const entries = discoverProject(targetPath);
|
||||
console.log(`Scanning ${formatCount(entries.length, "directory")}...`);
|
||||
const client = createClientFromArgs(parsed);
|
||||
await initProject(targetPath, { verbose: false, llmClient: client });
|
||||
const elapsed = ((Date.now() - start) / 1000).toFixed(1);
|
||||
console.log(
|
||||
`${pc.green("✓")} Generated ${formatCount(entries.length, ".pi-map.md file")} in ${elapsed}s`,
|
||||
);
|
||||
break;
|
||||
}
|
||||
case "patch": {
|
||||
if (!parsed.positional[0]) {
|
||||
console.error(
|
||||
`${pc.red("Error:")} Missing file path. Usage: project-map patch <file>`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
const client = createClientFromArgs(parsed);
|
||||
await patchFile(parsed.positional[0], client);
|
||||
console.log(`${pc.green("✓")} Patched`);
|
||||
break;
|
||||
}
|
||||
case "validate": {
|
||||
const { path, fix } = parsed;
|
||||
const result = await validateMaps(path, { fix, verbose: true });
|
||||
if (result.clean) {
|
||||
console.log(`${pc.green("✓")} All .pi-map.md files are clean.`);
|
||||
} else {
|
||||
const counts: Record<string, number> = {};
|
||||
for (const d of result.discrepancies) {
|
||||
counts[d.type] = (counts[d.type] || 0) + 1;
|
||||
}
|
||||
const summary = Object.entries(counts)
|
||||
.map(([type, count]) => `${count} ${type}`)
|
||||
.join(", ");
|
||||
const fixMsg =
|
||||
fix && result.fixed !== undefined
|
||||
? ` (${pc.green("✓")} fixed ${formatCount(result.fixed, "directory")})`
|
||||
: "";
|
||||
console.log(
|
||||
`${pc.yellow("⚠")} Found ${formatCount(result.discrepancies.length, "discrepancy")}: ${summary}${fixMsg}`,
|
||||
);
|
||||
}
|
||||
process.exit(result.clean ? 0 : 1);
|
||||
break;
|
||||
}
|
||||
case "reinit": {
|
||||
const targetPath = parsed.positional[0] || ".";
|
||||
const start = Date.now();
|
||||
const entries = discoverProject(targetPath);
|
||||
console.log(
|
||||
`Regenerating ${formatCount(entries.length, ".pi-map.md file")}...`,
|
||||
);
|
||||
const client = createClientFromArgs(parsed);
|
||||
await reinitPath(targetPath, { verbose: false, llmClient: client });
|
||||
const elapsed = ((Date.now() - start) / 1000).toFixed(1);
|
||||
console.log(`${pc.green("✓")} Regenerated in ${elapsed}s`);
|
||||
break;
|
||||
}
|
||||
default:
|
||||
console.error(`${pc.red("Error:")} Unknown command "${command}"`);
|
||||
console.error(`Run ${pc.cyan("project-map --help")} for usage.`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
if (err instanceof LLMError) {
|
||||
console.error(`${pc.red("LLM Error:")} ${err.message}`);
|
||||
} else {
|
||||
console.error(`${pc.red("Error:")} ${err.message}`);
|
||||
}
|
||||
process.exit(1);
|
||||
});
|
||||
import "./cli/cli.js";
|
||||
|
||||
+285
@@ -0,0 +1,285 @@
|
||||
#!/usr/bin/env node
|
||||
import { initProject } from "../init.js";
|
||||
import { patchFile } from "../patch.js";
|
||||
import { validateMaps } from "../validate.js";
|
||||
import { reinitPath } from "../init.js";
|
||||
import { discoverProject } from "../discover.js";
|
||||
import { retrieveContext } from "../retrieve.js";
|
||||
import { createLLMClient, LLMError } from "../llm/llm-client.js";
|
||||
import { loadConfig } from "../config.js";
|
||||
import pc from "picocolors";
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const command = args[0];
|
||||
|
||||
type PatchMode = "auto" | "small" | "structural";
|
||||
|
||||
function printUsage() {
|
||||
console.log(`${pc.bold("project-map")} — hierarchical project analysis for Pi agents
|
||||
`);
|
||||
console.log(`${pc.bold("Usage:")}`);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("init")} [path] Generate .pi-map.md files for all directories`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("patch")} <file> Update analysis for a changed file's directory`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("validate")} [--fix] [path] Check for stale/missing/orphaned entries`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("reinit")} [path] Force full regeneration`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("--help")} Show this help message`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("context")} <query> Retrieve relevant project context for a query`,
|
||||
);
|
||||
console.log(
|
||||
` project-map ${pc.cyan("--version")} Show version\n`,
|
||||
);
|
||||
console.log(`${pc.bold("Options:")}`);
|
||||
console.log(
|
||||
` --llm-provider=openai|kimi LLM provider (default: config or openai)`,
|
||||
);
|
||||
console.log(
|
||||
` --llm-model=<model> LLM model name (or set LLM_MODEL env var)`,
|
||||
);
|
||||
console.log(` --llm-base-url=<url> Custom base URL for LLM API`);
|
||||
console.log(
|
||||
` --patch-mode=<mode> Patch/repair mode: auto|small|structural\n`,
|
||||
);
|
||||
console.log(`${pc.bold("Examples:")}`);
|
||||
console.log(` project-map init`);
|
||||
console.log(` project-map patch src/components/Button.tsx`);
|
||||
console.log(` project-map validate --fix`);
|
||||
console.log(` project-map reinit`);
|
||||
console.log(` project-map context "authentication logic"`);
|
||||
console.log(` project-map init --llm-provider=kimi --llm-model=kimi-k2-6`);
|
||||
}
|
||||
|
||||
function printVersion() {
|
||||
const pkg = require("../../package.json");
|
||||
console.log(pkg.version);
|
||||
}
|
||||
|
||||
function formatCount(count: number, label: string): string {
|
||||
const plural = label.endsWith("y")
|
||||
? `${label.slice(0, -1)}ies`
|
||||
: `${label}${count === 1 ? "" : "s"}`;
|
||||
return `${pc.bold(String(count))} ${count === 1 ? label : plural}`;
|
||||
}
|
||||
|
||||
function renderProgressBar(
|
||||
completed: number,
|
||||
total: number,
|
||||
currentFile?: string,
|
||||
width = 30,
|
||||
): string {
|
||||
const pct = total > 0 ? completed / total : 0;
|
||||
const filled = Math.round(width * pct);
|
||||
const bar = "█".repeat(filled) + "░".repeat(width - filled);
|
||||
const file = currentFile ? ` | ${pc.dim(currentFile)}` : "";
|
||||
return `[${pc.cyan(bar)}] ${completed}/${total}${file}`;
|
||||
}
|
||||
|
||||
function parseArgs(args: string[]): {
|
||||
path: string;
|
||||
fix: boolean;
|
||||
llmProvider?: string;
|
||||
llmModel?: string;
|
||||
llmBaseUrl?: string;
|
||||
patchMode?: PatchMode;
|
||||
positional: string[];
|
||||
} {
|
||||
let path = ".";
|
||||
let fix = false;
|
||||
let llmProvider: string | undefined;
|
||||
let llmModel: string | undefined;
|
||||
let llmBaseUrl: string | undefined;
|
||||
let patchMode: PatchMode | undefined;
|
||||
const positional: string[] = [];
|
||||
|
||||
for (const arg of args.slice(1)) {
|
||||
if (arg === "--fix") {
|
||||
fix = true;
|
||||
} else if (arg.startsWith("--llm-provider=")) {
|
||||
llmProvider = arg.slice("--llm-provider=".length);
|
||||
} else if (arg.startsWith("--llm-model=")) {
|
||||
llmModel = arg.slice("--llm-model=".length);
|
||||
} else if (arg.startsWith("--llm-base-url=")) {
|
||||
llmBaseUrl = arg.slice("--llm-base-url=".length);
|
||||
} else if (arg.startsWith("--patch-mode=")) {
|
||||
patchMode = arg.slice("--patch-mode=".length) as PatchMode;
|
||||
} else if (!arg.startsWith("-")) {
|
||||
positional.push(arg);
|
||||
path = arg;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
path,
|
||||
fix,
|
||||
llmProvider,
|
||||
llmModel,
|
||||
llmBaseUrl,
|
||||
patchMode,
|
||||
positional,
|
||||
};
|
||||
}
|
||||
|
||||
function createClientFromArgs(args: ReturnType<typeof parseArgs>) {
|
||||
const config = loadConfig();
|
||||
const provider = (args.llmProvider || config.llmProvider) as
|
||||
| "openai"
|
||||
| "kimi"
|
||||
| "pi";
|
||||
return createLLMClient(provider, {
|
||||
model: args.llmModel || config.llmModel || process.env.LLM_MODEL,
|
||||
baseUrl: args.llmBaseUrl || config.llmBaseUrl,
|
||||
});
|
||||
}
|
||||
|
||||
async function main() {
|
||||
if (!command || command === "--help" || command === "-h") {
|
||||
printUsage();
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (command === "--version" || command === "-v") {
|
||||
printVersion();
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const parsed = parseArgs(args);
|
||||
|
||||
switch (command) {
|
||||
case "init": {
|
||||
const targetPath = parsed.positional[0] || ".";
|
||||
const start = Date.now();
|
||||
const entries = discoverProject(targetPath);
|
||||
console.log(`Scanning ${formatCount(entries.length, "directory")}...`);
|
||||
const client = createClientFromArgs(parsed);
|
||||
let lastLine = "";
|
||||
await initProject(targetPath, {
|
||||
verbose: false,
|
||||
llmClient: client,
|
||||
cacheDir: targetPath,
|
||||
onProgress: (info) => {
|
||||
const line = renderProgressBar(
|
||||
info.completed,
|
||||
info.total,
|
||||
info.currentFile,
|
||||
);
|
||||
process.stdout.write(`\r${line.padEnd(lastLine.length)}`);
|
||||
lastLine = line;
|
||||
},
|
||||
});
|
||||
process.stdout.write("\n");
|
||||
const elapsed = ((Date.now() - start) / 1000).toFixed(1);
|
||||
console.log(
|
||||
`${pc.green("✓")} Generated ${formatCount(entries.length, ".pi-map.md file")} in ${elapsed}s`,
|
||||
);
|
||||
break;
|
||||
}
|
||||
case "patch": {
|
||||
if (!parsed.positional[0]) {
|
||||
console.error(
|
||||
`${pc.red("Error:")} Missing file path. Usage: project-map patch <file>`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
const client = createClientFromArgs(parsed);
|
||||
await patchFile(parsed.positional[0], client, process.cwd(), {
|
||||
patchMode: parsed.patchMode,
|
||||
rootPath: process.cwd(),
|
||||
});
|
||||
console.log(`${pc.green("✓")} Patched`);
|
||||
break;
|
||||
}
|
||||
case "validate": {
|
||||
const { path, fix } = parsed;
|
||||
const result = await validateMaps(path, {
|
||||
fix,
|
||||
verbose: true,
|
||||
llmClient: fix ? createClientFromArgs(parsed) : undefined,
|
||||
cacheDir: process.cwd(),
|
||||
patchMode: parsed.patchMode,
|
||||
});
|
||||
if (result.clean) {
|
||||
console.log(`${pc.green("✓")} All .pi-map.md files are clean.`);
|
||||
} else {
|
||||
const counts: Record<string, number> = {};
|
||||
for (const d of result.discrepancies) {
|
||||
counts[d.type] = (counts[d.type] || 0) + 1;
|
||||
}
|
||||
const summary = Object.entries(counts)
|
||||
.map(([type, count]) => `${count} ${type}`)
|
||||
.join(", ");
|
||||
const fixMsg =
|
||||
fix && result.fixed !== undefined
|
||||
? ` (${pc.green("✓")} fixed ${formatCount(result.fixed, "directory")})`
|
||||
: "";
|
||||
console.log(
|
||||
`${pc.yellow("⚠")} Found ${formatCount(result.discrepancies.length, "discrepancy")}: ${summary}${fixMsg}`,
|
||||
);
|
||||
}
|
||||
process.exit(result.clean ? 0 : 1);
|
||||
break;
|
||||
}
|
||||
case "context": {
|
||||
if (!parsed.positional[0]) {
|
||||
console.error(
|
||||
`${pc.red("Error:")} Missing query. Usage: project-map context <query>`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
const query = parsed.positional[0];
|
||||
const bundle = retrieveContext(query, process.cwd());
|
||||
console.log(bundle);
|
||||
break;
|
||||
}
|
||||
case "reinit": {
|
||||
const targetPath = parsed.positional[0] || ".";
|
||||
const start = Date.now();
|
||||
const entries = discoverProject(targetPath);
|
||||
console.log(
|
||||
`Regenerating ${formatCount(entries.length, ".pi-map.md file")}...`,
|
||||
);
|
||||
const client = createClientFromArgs(parsed);
|
||||
let lastLine = "";
|
||||
await reinitPath(targetPath, {
|
||||
verbose: false,
|
||||
llmClient: client,
|
||||
cacheDir: targetPath,
|
||||
onProgress: (info) => {
|
||||
const line = renderProgressBar(
|
||||
info.completed,
|
||||
info.total,
|
||||
info.currentFile,
|
||||
);
|
||||
process.stdout.write(`\r${line.padEnd(lastLine.length)}`);
|
||||
lastLine = line;
|
||||
},
|
||||
});
|
||||
process.stdout.write("\n");
|
||||
const elapsed = ((Date.now() - start) / 1000).toFixed(1);
|
||||
console.log(`${pc.green("✓")} Regenerated in ${elapsed}s`);
|
||||
break;
|
||||
}
|
||||
default:
|
||||
console.error(`${pc.red("Error:")} Unknown command "${command}"`);
|
||||
console.error(`Run ${pc.cyan("project-map --help")} for usage.`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
if (err instanceof LLMError) {
|
||||
console.error(`${pc.red("LLM Error:")} ${err.message}`);
|
||||
} else {
|
||||
console.error(`${pc.red("Error:")} ${err.message}`);
|
||||
}
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,6 +1,8 @@
|
||||
import { existsSync, readFileSync } from "fs";
|
||||
import { join } from "path";
|
||||
|
||||
export type PromptInjectionMode = "off" | "advisory" | "strong" | "strict";
|
||||
|
||||
export interface SkillConfig {
|
||||
ignorePatterns: string[];
|
||||
smallPackageThreshold: number;
|
||||
@@ -9,6 +11,11 @@ export interface SkillConfig {
|
||||
llmBaseUrl?: string;
|
||||
contextBudget: number;
|
||||
autoInjectPrompt: boolean;
|
||||
tagCap: number;
|
||||
workflowHintCap: number;
|
||||
promptInjectionMode: PromptInjectionMode;
|
||||
contextBudgetPercent: number;
|
||||
contextBudgetMaxTokens: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_CONFIG: SkillConfig = {
|
||||
@@ -24,6 +31,7 @@ export const DEFAULT_CONFIG: SkillConfig = {
|
||||
".DS_Store",
|
||||
"*.log",
|
||||
".pi-map.md",
|
||||
".pi-map.index.md",
|
||||
".cache",
|
||||
"tmp",
|
||||
"temp",
|
||||
@@ -38,6 +46,11 @@ export const DEFAULT_CONFIG: SkillConfig = {
|
||||
llmModel: "gpt-4o-mini",
|
||||
contextBudget: 4000,
|
||||
autoInjectPrompt: true,
|
||||
tagCap: 8,
|
||||
workflowHintCap: 5,
|
||||
promptInjectionMode: "strong",
|
||||
contextBudgetPercent: 15,
|
||||
contextBudgetMaxTokens: 100_000,
|
||||
};
|
||||
|
||||
export function loadConfig(cwd: string = process.cwd()): SkillConfig {
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
export interface FileEntry {
|
||||
name: string;
|
||||
purpose: string;
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
}
|
||||
|
||||
export interface DirectoryArtifactModel {
|
||||
dir: string;
|
||||
role: string;
|
||||
files: FileEntry[];
|
||||
arch: string;
|
||||
dirty?: string;
|
||||
isRoot: boolean;
|
||||
parent?: string;
|
||||
children: string[];
|
||||
tags: string[];
|
||||
symbols: string[];
|
||||
workflows: WorkflowHint[];
|
||||
}
|
||||
|
||||
export interface WorkflowHint {
|
||||
task: string;
|
||||
read?: string[];
|
||||
index?: string[];
|
||||
map?: string[];
|
||||
files?: string[];
|
||||
}
|
||||
|
||||
export const PROJECT_MAP_PROTOCOL_LINES = [
|
||||
"## Project Map Protocol",
|
||||
"",
|
||||
"1. Read this protocol and the root `.pi-map.index.md` first.",
|
||||
"2. Use `index:` / `map:` references to open relevant directory indexes and maps.",
|
||||
"3. Load indexes before rich maps during task-start navigation.",
|
||||
"4. Read the local rich map and actual source before editing.",
|
||||
"5. Treat non-empty `## dirty` sections in either artifact as stale.",
|
||||
"6. If source and generated artifacts disagree, trust source.",
|
||||
"7. If map and index disagree, trust neither blindly; verify from source and regenerate the pair.",
|
||||
"8. After editing source, run `project_map_patch` for each changed file.",
|
||||
"9. Before broad architectural claims or final handoff, run `project_map_validate` when freshness matters.",
|
||||
"",
|
||||
"Trust boundary: index routes, map orients, source decides.",
|
||||
];
|
||||
|
||||
export interface RoutingMetadataOptions {
|
||||
tagCap?: number;
|
||||
workflowHintCap?: number;
|
||||
}
|
||||
|
||||
export function createDirectoryModel(opts: {
|
||||
dir: string;
|
||||
role: string;
|
||||
files: FileEntry[];
|
||||
arch: string;
|
||||
parent?: string;
|
||||
children?: string[];
|
||||
isRoot?: boolean;
|
||||
dirty?: string;
|
||||
}): DirectoryArtifactModel {
|
||||
return {
|
||||
dir: opts.dir,
|
||||
role: opts.role,
|
||||
files: opts.files,
|
||||
arch: opts.arch,
|
||||
parent: opts.parent,
|
||||
children: opts.children ?? [],
|
||||
isRoot: opts.isRoot ?? false,
|
||||
dirty: opts.dirty,
|
||||
tags: [],
|
||||
symbols: [],
|
||||
workflows: [],
|
||||
};
|
||||
}
|
||||
@@ -14,6 +14,7 @@ const DEFAULT_IGNORE = [
|
||||
".DS_Store",
|
||||
"*.log",
|
||||
".pi-map.md",
|
||||
".pi-map.index.md",
|
||||
".cache",
|
||||
"tmp",
|
||||
"temp",
|
||||
|
||||
+531
-31
@@ -1,3 +1,6 @@
|
||||
import type { DirectoryArtifactModel, FileEntry } from "./directory-model.js";
|
||||
import { PROJECT_MAP_PROTOCOL_LINES } from "./directory-model.js";
|
||||
|
||||
export interface PackageMapData {
|
||||
path: string;
|
||||
role: string;
|
||||
@@ -6,20 +9,64 @@ export interface PackageMapData {
|
||||
dirty?: string;
|
||||
}
|
||||
|
||||
export interface FileEntry {
|
||||
name: string;
|
||||
purpose: string;
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
export function renderPackageMap(data: PackageMapData): string {
|
||||
const model = convertPackageMapToModel(data);
|
||||
return renderDirectoryMap(model);
|
||||
}
|
||||
|
||||
export function renderPackageMap(data: PackageMapData): string {
|
||||
export function parsePackageMap(markdown: string): PackageMapData {
|
||||
const model = parseDirectoryMap(markdown);
|
||||
return convertModelToPackageMap(model);
|
||||
}
|
||||
|
||||
function convertPackageMapToModel(
|
||||
data: PackageMapData,
|
||||
): DirectoryArtifactModel {
|
||||
return {
|
||||
dir: data.path,
|
||||
role: data.role,
|
||||
files: data.files,
|
||||
arch: data.arch,
|
||||
dirty: data.dirty,
|
||||
isRoot: data.path === ".",
|
||||
parent: undefined,
|
||||
children: [],
|
||||
tags: [],
|
||||
symbols: [],
|
||||
workflows: [],
|
||||
};
|
||||
}
|
||||
|
||||
function convertModelToPackageMap(
|
||||
model: DirectoryArtifactModel,
|
||||
): PackageMapData {
|
||||
return {
|
||||
path: model.dir,
|
||||
role: model.role,
|
||||
files: model.files,
|
||||
arch: model.arch,
|
||||
dirty: model.dirty,
|
||||
};
|
||||
}
|
||||
|
||||
export function renderDirectoryMap(model: DirectoryArtifactModel): string {
|
||||
const lines: string[] = [];
|
||||
lines.push(`# ${data.path}`);
|
||||
lines.push(`# ${model.dir}`);
|
||||
lines.push(`dir: ${model.dir}`);
|
||||
lines.push("");
|
||||
lines.push(`index: ${model.dir}/.pi-map.index.md`);
|
||||
lines.push("");
|
||||
|
||||
if (model.isRoot) {
|
||||
lines.push(...PROJECT_MAP_PROTOCOL_LINES);
|
||||
lines.push("");
|
||||
}
|
||||
|
||||
lines.push(`## role`);
|
||||
lines.push(data.role);
|
||||
lines.push(model.role);
|
||||
|
||||
lines.push(`## files`);
|
||||
for (const file of data.files) {
|
||||
for (const file of model.files) {
|
||||
const exp =
|
||||
file.exports.length > 0 ? `exp: ${file.exports.join(", ")}` : "";
|
||||
const dep = file.deps.length > 0 ? `dep: ${file.deps.join(", ")}` : "";
|
||||
@@ -28,28 +75,88 @@ export function renderPackageMap(data: PackageMapData): string {
|
||||
if (dep) parts.push(dep);
|
||||
lines.push(parts.join(" | "));
|
||||
}
|
||||
|
||||
lines.push(`## arch`);
|
||||
lines.push(data.arch);
|
||||
lines.push(model.arch);
|
||||
|
||||
// Slice 2: tags, symbols, workflows — scaffold empty sections for now
|
||||
lines.push(`## tags`);
|
||||
if (model.tags.length > 0) {
|
||||
lines.push(model.tags.join(", "));
|
||||
} else {
|
||||
lines.push("-");
|
||||
}
|
||||
|
||||
lines.push(`## symbols`);
|
||||
if (model.symbols.length > 0) {
|
||||
for (const sym of model.symbols) {
|
||||
lines.push(`- ${sym}`);
|
||||
}
|
||||
} else {
|
||||
lines.push("-");
|
||||
}
|
||||
|
||||
lines.push(`## workflows`);
|
||||
if (model.workflows.length > 0) {
|
||||
for (const wf of model.workflows) {
|
||||
lines.push(`- ${wf.task}`);
|
||||
if (wf.read && wf.read.length > 0) {
|
||||
lines.push(` read: ${wf.read.join(", ")}`);
|
||||
}
|
||||
if (wf.index && wf.index.length > 0) {
|
||||
lines.push(` index: ${wf.index.join(", ")}`);
|
||||
}
|
||||
if (wf.map && wf.map.length > 0) {
|
||||
lines.push(` map: ${wf.map.join(", ")}`);
|
||||
}
|
||||
if (wf.files && wf.files.length > 0) {
|
||||
lines.push(` files: ${wf.files.join(", ")}`);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
lines.push("-");
|
||||
}
|
||||
|
||||
lines.push(`## dirty`);
|
||||
lines.push(data.dirty || "-");
|
||||
lines.push(model.dirty || "-");
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
export function parsePackageMap(markdown: string): PackageMapData {
|
||||
export function parseDirectoryMap(markdown: string): DirectoryArtifactModel {
|
||||
const lines = markdown.split("\n").map((l) => l.trimEnd());
|
||||
const result: PackageMapData = {
|
||||
path: "",
|
||||
const result: DirectoryArtifactModel = {
|
||||
dir: "",
|
||||
role: "",
|
||||
files: [],
|
||||
arch: "",
|
||||
dirty: "-",
|
||||
isRoot: false,
|
||||
parent: undefined,
|
||||
children: [],
|
||||
tags: [],
|
||||
symbols: [],
|
||||
workflows: [],
|
||||
};
|
||||
|
||||
let section: "none" | "role" | "files" | "arch" | "dirty" = "none";
|
||||
let section:
|
||||
| "none"
|
||||
| "role"
|
||||
| "files"
|
||||
| "arch"
|
||||
| "tags"
|
||||
| "symbols"
|
||||
| "workflows"
|
||||
| "dirty" = "none";
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.startsWith("# ")) {
|
||||
result.path = line.slice(2).trim();
|
||||
result.dir = line.slice(2).trim();
|
||||
result.isRoot = result.dir === ".";
|
||||
continue;
|
||||
}
|
||||
if (line.startsWith("dir: ")) {
|
||||
result.dir = line.slice(5).trim();
|
||||
result.isRoot = result.dir === ".";
|
||||
continue;
|
||||
}
|
||||
if (line === "## role") {
|
||||
@@ -64,6 +171,18 @@ export function parsePackageMap(markdown: string): PackageMapData {
|
||||
section = "arch";
|
||||
continue;
|
||||
}
|
||||
if (line === "## tags") {
|
||||
section = "tags";
|
||||
continue;
|
||||
}
|
||||
if (line === "## symbols") {
|
||||
section = "symbols";
|
||||
continue;
|
||||
}
|
||||
if (line === "## workflows") {
|
||||
section = "workflows";
|
||||
continue;
|
||||
}
|
||||
if (line === "## dirty") {
|
||||
section = "dirty";
|
||||
continue;
|
||||
@@ -83,6 +202,70 @@ export function parsePackageMap(markdown: string): PackageMapData {
|
||||
case "arch":
|
||||
result.arch = result.arch ? `${result.arch}\n${line}` : line;
|
||||
break;
|
||||
case "tags":
|
||||
if (line !== "-") {
|
||||
result.tags.push(
|
||||
...line
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean),
|
||||
);
|
||||
}
|
||||
break;
|
||||
case "symbols":
|
||||
if (line !== "-" && line.startsWith("- ")) {
|
||||
result.symbols.push(line.slice(2).trim());
|
||||
}
|
||||
break;
|
||||
case "workflows": {
|
||||
if (line !== "-" && line.startsWith("- ")) {
|
||||
const task = line.slice(2).trim();
|
||||
result.workflows.push({ task });
|
||||
} else if (line.startsWith(" read: ") && result.workflows.length > 0) {
|
||||
const reads = line
|
||||
.slice(8)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.read = reads;
|
||||
} else if (
|
||||
line.startsWith(" index: ") &&
|
||||
result.workflows.length > 0
|
||||
) {
|
||||
const indices = line
|
||||
.slice(9)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.index = indices;
|
||||
} else if (line.startsWith(" map: ") && result.workflows.length > 0) {
|
||||
const maps = line
|
||||
.slice(7)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.map = maps;
|
||||
} else if (
|
||||
line.startsWith(" files: ") &&
|
||||
result.workflows.length > 0
|
||||
) {
|
||||
const files = line
|
||||
.slice(9)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.files = files;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case "dirty":
|
||||
result.dirty = line === "-" ? undefined : line;
|
||||
break;
|
||||
@@ -92,10 +275,115 @@ export function parsePackageMap(markdown: string): PackageMapData {
|
||||
return result;
|
||||
}
|
||||
|
||||
function splitTopLevel(str: string, delimiter: string): string[] {
|
||||
const result: string[] = [];
|
||||
let current = "";
|
||||
let depthParen = 0;
|
||||
let depthBracket = 0;
|
||||
let depthBrace = 0;
|
||||
let depthAngle = 0;
|
||||
let inString: '"' | "'" | null = null;
|
||||
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
const ch = str[i];
|
||||
const prev = str[i - 1];
|
||||
const next = str[i + 1];
|
||||
|
||||
if (inString) {
|
||||
if (ch === inString && prev !== "\\") {
|
||||
inString = null;
|
||||
}
|
||||
current += ch;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === '"' || ch === "'") {
|
||||
const isWordApostrophe =
|
||||
ch === "'" &&
|
||||
/[A-Za-z0-9_$]/.test(prev ?? "") &&
|
||||
/[A-Za-z0-9_$]/.test(next ?? "");
|
||||
if (isWordApostrophe) {
|
||||
current += ch;
|
||||
continue;
|
||||
}
|
||||
inString = ch;
|
||||
current += ch;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === "(") depthParen++;
|
||||
if (ch === ")") depthParen = Math.max(0, depthParen - 1);
|
||||
if (ch === "[") depthBracket++;
|
||||
if (ch === "]") depthBracket = Math.max(0, depthBracket - 1);
|
||||
if (ch === "{") depthBrace++;
|
||||
if (ch === "}") depthBrace = Math.max(0, depthBrace - 1);
|
||||
if (ch === "<") depthAngle++;
|
||||
if (ch === ">") depthAngle = Math.max(0, depthAngle - 1);
|
||||
|
||||
if (
|
||||
delimiter.length === 1 &&
|
||||
ch === delimiter &&
|
||||
depthParen === 0 &&
|
||||
depthBracket === 0 &&
|
||||
depthBrace === 0 &&
|
||||
depthAngle === 0
|
||||
) {
|
||||
result.push(current.trim());
|
||||
current = "";
|
||||
continue;
|
||||
}
|
||||
|
||||
if (
|
||||
delimiter.length > 1 &&
|
||||
str.startsWith(delimiter, i) &&
|
||||
depthParen === 0 &&
|
||||
depthBracket === 0 &&
|
||||
depthBrace === 0 &&
|
||||
depthAngle === 0
|
||||
) {
|
||||
result.push(current.trim());
|
||||
current = "";
|
||||
i += delimiter.length - 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
current += ch;
|
||||
}
|
||||
|
||||
if (current.trim()) result.push(current.trim());
|
||||
return result;
|
||||
}
|
||||
|
||||
function splitRespectingNesting(str: string): string[] {
|
||||
return splitTopLevel(str, ",");
|
||||
}
|
||||
|
||||
function splitFieldsRespectingNesting(str: string): string[] {
|
||||
const rawParts = splitTopLevel(str, " | ");
|
||||
const fields: string[] = [];
|
||||
|
||||
for (const part of rawParts) {
|
||||
const trimmed = part.trim();
|
||||
if (
|
||||
fields.length < 2 ||
|
||||
trimmed.startsWith("exp: ") ||
|
||||
trimmed.startsWith("dep: ")
|
||||
) {
|
||||
fields.push(trimmed);
|
||||
} else if (fields.length > 0) {
|
||||
fields[fields.length - 1] = `${fields[fields.length - 1]} | ${trimmed}`;
|
||||
}
|
||||
}
|
||||
|
||||
return fields;
|
||||
}
|
||||
|
||||
function parseFileLine(line: string): FileEntry | null {
|
||||
// Format: - filename | purpose | exp: ... | dep: ...
|
||||
const withoutPrefix = line.slice(2).trim();
|
||||
const parts = withoutPrefix.split(" | ").map((p) => p.trim());
|
||||
const parts = splitFieldsRespectingNesting(withoutPrefix).map((p) =>
|
||||
p.trim(),
|
||||
);
|
||||
|
||||
if (parts.length < 2) return null;
|
||||
|
||||
@@ -107,23 +395,235 @@ function parseFileLine(line: string): FileEntry | null {
|
||||
for (let i = 2; i < parts.length; i++) {
|
||||
const part = parts[i];
|
||||
if (part.startsWith("exp: ")) {
|
||||
exports.push(
|
||||
...part
|
||||
.slice(5)
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean),
|
||||
);
|
||||
exports.push(...splitRespectingNesting(part.slice(5)).filter(Boolean));
|
||||
} else if (part.startsWith("dep: ")) {
|
||||
deps.push(
|
||||
...part
|
||||
.slice(5)
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean),
|
||||
);
|
||||
deps.push(...splitRespectingNesting(part.slice(5)).filter(Boolean));
|
||||
}
|
||||
}
|
||||
|
||||
return { name, purpose, exports, deps };
|
||||
}
|
||||
|
||||
export function renderDirectoryIndex(model: DirectoryArtifactModel): string {
|
||||
const lines: string[] = [];
|
||||
lines.push(`# ${model.dir} (index)`);
|
||||
lines.push(`dir: ${model.dir}`);
|
||||
lines.push("");
|
||||
|
||||
if (model.isRoot) {
|
||||
lines.push(...PROJECT_MAP_PROTOCOL_LINES);
|
||||
lines.push("");
|
||||
}
|
||||
|
||||
lines.push(`## role`);
|
||||
lines.push(model.role);
|
||||
|
||||
lines.push(`## parent`);
|
||||
if (model.parent) {
|
||||
lines.push(`index: ${model.parent}/.pi-map.index.md`);
|
||||
lines.push(`map: ${model.parent}/.pi-map.md`);
|
||||
} else {
|
||||
lines.push("-");
|
||||
}
|
||||
|
||||
lines.push(`## children`);
|
||||
if (model.children.length > 0) {
|
||||
for (const child of model.children) {
|
||||
lines.push(`- ${child}`);
|
||||
lines.push(` index: ${child}/.pi-map.index.md`);
|
||||
lines.push(` map: ${child}/.pi-map.md`);
|
||||
}
|
||||
} else {
|
||||
lines.push("-");
|
||||
}
|
||||
|
||||
lines.push(`## files`);
|
||||
for (const file of model.files) {
|
||||
lines.push(`- ${file.name}`);
|
||||
}
|
||||
|
||||
lines.push(`## links`);
|
||||
lines.push(`index: ${model.dir}/.pi-map.index.md`);
|
||||
lines.push(`map: ${model.dir}/.pi-map.md`);
|
||||
|
||||
lines.push(`## workflows`);
|
||||
if (model.workflows.length > 0) {
|
||||
for (const wf of model.workflows) {
|
||||
lines.push(`- ${wf.task}`);
|
||||
if (wf.read && wf.read.length > 0) {
|
||||
lines.push(` read: ${wf.read.join(", ")}`);
|
||||
}
|
||||
if (wf.index && wf.index.length > 0) {
|
||||
lines.push(` index: ${wf.index.join(", ")}`);
|
||||
}
|
||||
if (wf.map && wf.map.length > 0) {
|
||||
lines.push(` map: ${wf.map.join(", ")}`);
|
||||
}
|
||||
if (wf.files && wf.files.length > 0) {
|
||||
lines.push(` files: ${wf.files.join(", ")}`);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
lines.push("-");
|
||||
}
|
||||
|
||||
lines.push(`## dirty`);
|
||||
lines.push(model.dirty || "-");
|
||||
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
export function parseDirectoryIndex(markdown: string): DirectoryArtifactModel {
|
||||
const lines = markdown.split("\n").map((l) => l.trimEnd());
|
||||
const result: DirectoryArtifactModel = {
|
||||
dir: "",
|
||||
role: "",
|
||||
files: [],
|
||||
arch: "",
|
||||
dirty: "-",
|
||||
isRoot: false,
|
||||
parent: undefined,
|
||||
children: [],
|
||||
tags: [],
|
||||
symbols: [],
|
||||
workflows: [],
|
||||
};
|
||||
|
||||
let section:
|
||||
| "none"
|
||||
| "role"
|
||||
| "parent"
|
||||
| "children"
|
||||
| "files"
|
||||
| "links"
|
||||
| "workflows"
|
||||
| "dirty" = "none";
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.startsWith("# ")) {
|
||||
const title = line.slice(2).trim();
|
||||
// Strip " (index)" suffix if present
|
||||
result.dir = title.replace(/ \(index\)$/, "");
|
||||
result.isRoot = result.dir === ".";
|
||||
continue;
|
||||
}
|
||||
if (line.startsWith("dir: ")) {
|
||||
result.dir = line.slice(5).trim();
|
||||
result.isRoot = result.dir === ".";
|
||||
continue;
|
||||
}
|
||||
if (line === "## role") {
|
||||
section = "role";
|
||||
continue;
|
||||
}
|
||||
if (line === "## parent") {
|
||||
section = "parent";
|
||||
continue;
|
||||
}
|
||||
if (line === "## children") {
|
||||
section = "children";
|
||||
continue;
|
||||
}
|
||||
if (line === "## files") {
|
||||
section = "files";
|
||||
continue;
|
||||
}
|
||||
if (line === "## links") {
|
||||
section = "links";
|
||||
continue;
|
||||
}
|
||||
if (line === "## workflows") {
|
||||
section = "workflows";
|
||||
continue;
|
||||
}
|
||||
if (line === "## dirty") {
|
||||
section = "dirty";
|
||||
continue;
|
||||
}
|
||||
if (line === "") continue;
|
||||
|
||||
switch (section) {
|
||||
case "role":
|
||||
result.role = line;
|
||||
break;
|
||||
case "parent":
|
||||
if (line !== "-" && line.startsWith("index: ")) {
|
||||
result.parent = line.slice(7).trim().replace("/.pi-map.index.md", "");
|
||||
}
|
||||
break;
|
||||
case "children":
|
||||
if (line !== "-" && line.startsWith("- ")) {
|
||||
const childName = line.slice(2).trim();
|
||||
result.children.push(childName);
|
||||
}
|
||||
break;
|
||||
case "files":
|
||||
if (line !== "-" && line.startsWith("- ")) {
|
||||
result.files.push({
|
||||
name: line.slice(2).trim(),
|
||||
purpose: "",
|
||||
exports: [],
|
||||
deps: [],
|
||||
});
|
||||
}
|
||||
break;
|
||||
case "links":
|
||||
// Parse sibling links; no-op for now
|
||||
break;
|
||||
case "workflows": {
|
||||
if (line !== "-" && line.startsWith("- ")) {
|
||||
const task = line.slice(2).trim();
|
||||
result.workflows.push({ task });
|
||||
} else if (line.startsWith(" read: ") && result.workflows.length > 0) {
|
||||
const reads = line
|
||||
.slice(8)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.read = reads;
|
||||
} else if (
|
||||
line.startsWith(" index: ") &&
|
||||
result.workflows.length > 0
|
||||
) {
|
||||
const indices = line
|
||||
.slice(9)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.index = indices;
|
||||
} else if (line.startsWith(" map: ") && result.workflows.length > 0) {
|
||||
const maps = line
|
||||
.slice(7)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.map = maps;
|
||||
} else if (
|
||||
line.startsWith(" files: ") &&
|
||||
result.workflows.length > 0
|
||||
) {
|
||||
const files = line
|
||||
.slice(9)
|
||||
.trim()
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
const last = result.workflows[result.workflows.length - 1];
|
||||
if (last) last.files = files;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case "dirty":
|
||||
result.dirty = line === "-" ? undefined : line;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
+60
-4
@@ -1,5 +1,61 @@
|
||||
// Main entry point for pi-project-map skill
|
||||
export { initProject, reinitPath } from "./init.js";
|
||||
export { patchFile } from "./patch.js";
|
||||
export { validateMaps } from "./validate.js";
|
||||
export { renderPackageMap, parsePackageMap } from "./format.js";
|
||||
export { initProject, reinitPath, generateDirectoryArtifacts } from "./init.js";
|
||||
export { patchFile, type PatchMode, type PatchOptions } from "./patch.js";
|
||||
export {
|
||||
validateMaps,
|
||||
type ValidationOptions,
|
||||
type ValidationResult,
|
||||
type Discrepancy,
|
||||
} from "./validate.js";
|
||||
export {
|
||||
renderPackageMap,
|
||||
parsePackageMap,
|
||||
renderDirectoryMap,
|
||||
parseDirectoryMap,
|
||||
renderDirectoryIndex,
|
||||
parseDirectoryIndex,
|
||||
} from "./format.js";
|
||||
export type { PackageMapData } from "./format.js";
|
||||
export {
|
||||
createDirectoryModel,
|
||||
type DirectoryArtifactModel,
|
||||
type FileEntry,
|
||||
} from "./directory-model.js";
|
||||
export { populateRoutingMetadata } from "./routing-metadata.js";
|
||||
export { retrieveContext } from "./retrieve.js";
|
||||
export {
|
||||
buildRootPairBlock,
|
||||
hasRootPairMarker,
|
||||
buildPreInitHint,
|
||||
computeInjectionBudget,
|
||||
modeAllowsPreInitHint,
|
||||
modeAllowsInjection,
|
||||
modeRequiresProtocolPath,
|
||||
hasProtocolPath,
|
||||
isSensitiveAction,
|
||||
messagesHaveBypass,
|
||||
extractBypassReason,
|
||||
evaluateStrictBypass,
|
||||
buildAdvisoryReminder,
|
||||
buildStrictBypassGuard,
|
||||
discoverContextWindow,
|
||||
estimateTokens,
|
||||
findAllArtifactPairs,
|
||||
buildInjectionPayload,
|
||||
outgoingMessagesHaveMarker,
|
||||
providerPayloadHasMarker,
|
||||
isRelevantTurnForReinjection,
|
||||
shouldReinjectForEvent,
|
||||
detectEditIntent,
|
||||
detectArchitectureSensitiveReasoning,
|
||||
getRootPairMtimes,
|
||||
rootPairChanged,
|
||||
type RootPairMtimes,
|
||||
type RelevantTurnType,
|
||||
type ReinjectDecision,
|
||||
type StrictBypassDecision,
|
||||
ROOT_PAIR_START_MARKER,
|
||||
ROOT_PAIR_END_MARKER,
|
||||
TRUST_BOUNDARY_TEXT,
|
||||
BYPASS_MARKER_PREFIX,
|
||||
} from "./prompt-injection.js";
|
||||
|
||||
+229
-28
@@ -1,19 +1,38 @@
|
||||
import { discoverProject, type DirectoryEntry } from "./discover.js";
|
||||
import {
|
||||
renderPackageMap,
|
||||
type PackageMapData,
|
||||
type FileEntry,
|
||||
} from "./format.js";
|
||||
import { extractFileLLM, extractPackageLLM } from "./llm-extract.js";
|
||||
import { extractFileAST } from "./ast-extract.js";
|
||||
|
||||
export { discoverProject };
|
||||
import type { FileEntry } from "./directory-model.js";
|
||||
import { renderDirectoryIndex, renderDirectoryMap } from "./format.js";
|
||||
import { extractFileLLM, extractPackageLLM } from "./llm/llm-extract.js";
|
||||
import { extractFileAST } from "./ast/ast-extract.js";
|
||||
import { mergeFileData } from "./merge.js";
|
||||
import { processFiles } from "./llm/llm-batch.js";
|
||||
import { writeFileSync } from "fs";
|
||||
import { join } from "path";
|
||||
import type { LLMClient } from "./llm-client.js";
|
||||
import type { LLMClient } from "./llm/llm-client.js";
|
||||
import {
|
||||
createDirectoryModel,
|
||||
type DirectoryArtifactModel,
|
||||
type RoutingMetadataOptions,
|
||||
} from "./directory-model.js";
|
||||
import { populateRoutingMetadata } from "./routing-metadata.js";
|
||||
import { loadConfig } from "./config.js";
|
||||
|
||||
export interface ProgressInfo {
|
||||
message: string;
|
||||
completed: number;
|
||||
total: number;
|
||||
currentFile?: string;
|
||||
dir?: string;
|
||||
}
|
||||
|
||||
export interface InitOptions {
|
||||
verbose?: boolean;
|
||||
llmClient?: LLMClient;
|
||||
cacheDir?: string;
|
||||
onProgress?: (info: ProgressInfo) => void;
|
||||
tagCap?: number;
|
||||
workflowHintCap?: number;
|
||||
}
|
||||
|
||||
export async function initProject(
|
||||
@@ -21,51 +40,233 @@ export async function initProject(
|
||||
options: InitOptions = {},
|
||||
): Promise<void> {
|
||||
const entries = discoverProject(rootPath);
|
||||
const totalFiles = entries.reduce((sum, e) => sum + e.files.length, 0);
|
||||
let globalCompleted = 0;
|
||||
|
||||
for (const entry of entries) {
|
||||
await generateDirectoryMap(entry, options.llmClient);
|
||||
// Load project config and apply defaults when not overridden in options
|
||||
const config = loadConfig(rootPath);
|
||||
const routingOpts: RoutingMetadataOptions = {
|
||||
tagCap: options.tagCap ?? config.tagCap,
|
||||
workflowHintCap: options.workflowHintCap ?? config.workflowHintCap,
|
||||
};
|
||||
|
||||
options.onProgress?.({
|
||||
message: `Scanning ${entries.length} directories (${totalFiles} files)...`,
|
||||
completed: 0,
|
||||
total: totalFiles,
|
||||
});
|
||||
|
||||
// Build directory relationships
|
||||
const dirSet = new Set(entries.map((e) => e.relativePath));
|
||||
const parentMap = buildParentMap(entries);
|
||||
const childrenMap = buildChildrenMap(entries);
|
||||
|
||||
for (let i = 0; i < entries.length; i++) {
|
||||
const entry = entries[i];
|
||||
await generateDirectoryArtifacts(
|
||||
entry,
|
||||
{
|
||||
dirSet,
|
||||
parentMap,
|
||||
childrenMap,
|
||||
isRoot: entry.relativePath === ".",
|
||||
},
|
||||
options.llmClient,
|
||||
options.cacheDir,
|
||||
(info) => {
|
||||
globalCompleted =
|
||||
info.completed +
|
||||
entries.slice(0, i).reduce((sum, e) => sum + e.files.length, 0);
|
||||
options.onProgress?.({
|
||||
...info,
|
||||
completed: globalCompleted,
|
||||
total: totalFiles,
|
||||
dir: entry.relativePath,
|
||||
});
|
||||
},
|
||||
routingOpts,
|
||||
);
|
||||
}
|
||||
|
||||
options.onProgress?.({
|
||||
message: `Generated ${entries.length} directory map/index pairs`,
|
||||
completed: totalFiles,
|
||||
total: totalFiles,
|
||||
});
|
||||
if (options.verbose !== false) {
|
||||
console.log(`Generated ${entries.length} .pi-map.md files`);
|
||||
console.log(`Generated ${entries.length} directory map/index pairs`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function generateDirectoryMap(
|
||||
entry: DirectoryEntry,
|
||||
llmClient?: LLMClient,
|
||||
): Promise<FileEntry[]> {
|
||||
const fileData: FileEntry[] = [];
|
||||
for (const file of entry.files) {
|
||||
const filePath = join(entry.dirPath, file);
|
||||
const llmData = await extractFileLLM(filePath, llmClient);
|
||||
const astData = await extractFileAST(filePath);
|
||||
fileData.push(mergeFileData(file, llmData, astData));
|
||||
export interface DirectoryContext {
|
||||
dirSet: Set<string>;
|
||||
parentMap: Map<string, string | undefined>;
|
||||
childrenMap: Map<string, string[]>;
|
||||
isRoot: boolean;
|
||||
}
|
||||
|
||||
export type ArtifactWriteMode = "both" | "map" | "index";
|
||||
|
||||
export function buildDirectoryContext(
|
||||
entries: DirectoryEntry[],
|
||||
targetEntry: DirectoryEntry,
|
||||
): DirectoryContext {
|
||||
return {
|
||||
dirSet: new Set(entries.map((e) => e.relativePath)),
|
||||
parentMap: buildParentMap(entries),
|
||||
childrenMap: buildChildrenMap(entries),
|
||||
isRoot: targetEntry.relativePath === ".",
|
||||
};
|
||||
}
|
||||
|
||||
function buildParentMap(
|
||||
entries: DirectoryEntry[],
|
||||
): Map<string, string | undefined> {
|
||||
const map = new Map<string, string | undefined>();
|
||||
for (const entry of entries) {
|
||||
const rel = entry.relativePath;
|
||||
if (rel === ".") {
|
||||
map.set(rel, undefined);
|
||||
} else {
|
||||
const lastSep = rel.lastIndexOf("/");
|
||||
map.set(rel, lastSep >= 0 ? rel.slice(0, lastSep) : ".");
|
||||
}
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
function buildChildrenMap(entries: DirectoryEntry[]): Map<string, string[]> {
|
||||
const map = new Map<string, string[]>();
|
||||
for (const entry of entries) {
|
||||
map.set(entry.relativePath, []);
|
||||
}
|
||||
for (const entry of entries) {
|
||||
const rel = entry.relativePath;
|
||||
if (rel === ".") continue;
|
||||
const lastSep = rel.lastIndexOf("/");
|
||||
const parent = lastSep >= 0 ? rel.slice(0, lastSep) : ".";
|
||||
const siblings = map.get(parent);
|
||||
if (siblings) {
|
||||
siblings.push(rel);
|
||||
}
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
export async function buildDirectoryArtifactModel(
|
||||
entry: DirectoryEntry,
|
||||
ctx: DirectoryContext,
|
||||
llmClient?: LLMClient,
|
||||
cacheDir?: string,
|
||||
onProgress?: (info: ProgressInfo) => void,
|
||||
routingOpts?: RoutingMetadataOptions,
|
||||
): Promise<DirectoryArtifactModel> {
|
||||
const fileData = await processFiles(
|
||||
entry.files,
|
||||
async (file) => {
|
||||
const filePath = join(entry.dirPath, file);
|
||||
const llmData = await extractFileLLM(filePath, llmClient, cacheDir);
|
||||
const astData = await extractFileAST(filePath);
|
||||
return mergeFileData(file, llmData, astData);
|
||||
},
|
||||
{ concurrency: 4, batchDelayMs: 100, maxRetries: 2 },
|
||||
(completed, total, currentFile) => {
|
||||
onProgress?.({
|
||||
message: ` → ${currentFile}`,
|
||||
completed,
|
||||
total,
|
||||
currentFile,
|
||||
dir: entry.relativePath,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
const packageData = await extractPackageLLM(
|
||||
entry.relativePath,
|
||||
fileData,
|
||||
llmClient,
|
||||
cacheDir,
|
||||
);
|
||||
|
||||
const mapData: PackageMapData = {
|
||||
path: entry.relativePath,
|
||||
const model: DirectoryArtifactModel = createDirectoryModel({
|
||||
dir: entry.relativePath,
|
||||
role: packageData.role,
|
||||
files: fileData,
|
||||
arch: packageData.arch,
|
||||
parent: ctx.parentMap.get(entry.relativePath),
|
||||
children: ctx.childrenMap.get(entry.relativePath) ?? [],
|
||||
isRoot: ctx.isRoot,
|
||||
dirty: "-",
|
||||
};
|
||||
});
|
||||
|
||||
const outPath = join(entry.dirPath, ".pi-map.md");
|
||||
writeFileSync(outPath, renderPackageMap(mapData));
|
||||
return fileData;
|
||||
populateRoutingMetadata(model, routingOpts);
|
||||
return model;
|
||||
}
|
||||
|
||||
export function writeDirectoryArtifacts(
|
||||
entry: DirectoryEntry,
|
||||
model: DirectoryArtifactModel,
|
||||
writeMode: ArtifactWriteMode = "both",
|
||||
): void {
|
||||
const mapPath = join(entry.dirPath, ".pi-map.md");
|
||||
const indexPath = join(entry.dirPath, ".pi-map.index.md");
|
||||
|
||||
if (writeMode === "both" || writeMode === "map") {
|
||||
writeFileSync(mapPath, renderDirectoryMap(model));
|
||||
}
|
||||
if (writeMode === "both" || writeMode === "index") {
|
||||
writeFileSync(indexPath, renderDirectoryIndex(model));
|
||||
}
|
||||
}
|
||||
|
||||
export async function generateDirectoryArtifacts(
|
||||
entry: DirectoryEntry,
|
||||
ctx: DirectoryContext,
|
||||
llmClient?: LLMClient,
|
||||
cacheDir?: string,
|
||||
onProgress?: (info: ProgressInfo) => void,
|
||||
routingOpts?: RoutingMetadataOptions,
|
||||
writeMode: ArtifactWriteMode = "both",
|
||||
): Promise<FileEntry[]> {
|
||||
const model = await buildDirectoryArtifactModel(
|
||||
entry,
|
||||
ctx,
|
||||
llmClient,
|
||||
cacheDir,
|
||||
onProgress,
|
||||
routingOpts,
|
||||
);
|
||||
writeDirectoryArtifacts(entry, model, writeMode);
|
||||
return model.files;
|
||||
}
|
||||
|
||||
export async function reinitPath(
|
||||
path: string,
|
||||
options: InitOptions = {},
|
||||
): Promise<void> {
|
||||
// Full regeneration clears all dirty markers by overwriting every .pi-map.md
|
||||
// Full regeneration clears all dirty markers by overwriting every map/index pair
|
||||
await initProject(path, options);
|
||||
}
|
||||
|
||||
// Backward-compatible wrapper for patch/validate compatibility
|
||||
export async function generateDirectoryMap(
|
||||
entry: DirectoryEntry,
|
||||
llmClient?: LLMClient,
|
||||
cacheDir?: string,
|
||||
): Promise<FileEntry[]> {
|
||||
const entries = [entry];
|
||||
const ctx = buildDirectoryContext(entries, entry);
|
||||
const config = loadConfig(entry.dirPath);
|
||||
const routingOpts: RoutingMetadataOptions = {
|
||||
tagCap: config.tagCap,
|
||||
workflowHintCap: config.workflowHintCap,
|
||||
};
|
||||
return generateDirectoryArtifacts(
|
||||
entry,
|
||||
ctx,
|
||||
llmClient,
|
||||
cacheDir,
|
||||
undefined,
|
||||
routingOpts,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
import { readFileSync, writeFileSync, existsSync, mkdirSync, renameSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { homedir } from "os";
|
||||
|
||||
const CACHE_DIR = join(homedir(), ".cache", "pi-project-map");
|
||||
const CACHE_FILE = join(CACHE_DIR, "llm-cache.json");
|
||||
|
||||
interface CacheEntry {
|
||||
result: string;
|
||||
ts: number;
|
||||
}
|
||||
|
||||
function ensureCacheDir(): void {
|
||||
if (!existsSync(CACHE_DIR)) {
|
||||
mkdirSync(CACHE_DIR, { recursive: true });
|
||||
}
|
||||
}
|
||||
|
||||
function loadCache(): Record<string, CacheEntry> {
|
||||
if (!existsSync(CACHE_FILE)) return {};
|
||||
try {
|
||||
const raw = readFileSync(CACHE_FILE, "utf8");
|
||||
return JSON.parse(raw) as Record<string, CacheEntry>;
|
||||
} catch {
|
||||
// Corrupted cache — start fresh
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
function saveCache(cache: Record<string, CacheEntry>): void {
|
||||
ensureCacheDir();
|
||||
const tmp = CACHE_FILE + ".tmp";
|
||||
writeFileSync(tmp, JSON.stringify(cache, null, 2));
|
||||
renameSync(tmp, CACHE_FILE);
|
||||
}
|
||||
|
||||
export function getCached(hash: string): string | undefined {
|
||||
const cache = loadCache();
|
||||
return cache[hash]?.result;
|
||||
}
|
||||
|
||||
export function setCached(hash: string, result: string): void {
|
||||
const cache = loadCache();
|
||||
cache[hash] = { result, ts: Date.now() };
|
||||
saveCache(cache);
|
||||
}
|
||||
@@ -1,385 +0,0 @@
|
||||
import { readFileSync, statSync } from "fs";
|
||||
import { createHash } from "crypto";
|
||||
import { extname, basename } from "path";
|
||||
import type { LLMClient } from "./llm-client.js";
|
||||
import { getCached, setCached } from "./llm-cache.js";
|
||||
import { LLMError } from "./llm-error.js";
|
||||
|
||||
interface LLMFileData {
|
||||
purpose: string;
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
concepts: string[];
|
||||
}
|
||||
|
||||
interface LLMPackageData {
|
||||
role: string;
|
||||
arch: string;
|
||||
}
|
||||
|
||||
const MAX_FILE_SIZE = 50 * 1024; // 50KB
|
||||
const CONTEXT_BUDGET = 4000; // tokens
|
||||
const CHARS_PER_TOKEN = 4; // approximate for ASCII
|
||||
|
||||
// Heuristic patterns for common file types (used as fallback + for tests)
|
||||
const FILE_TYPE_PURPOSES: Record<string, string> = {
|
||||
".ts": "TypeScript module",
|
||||
".tsx": "React component",
|
||||
".js": "JavaScript module",
|
||||
".jsx": "React component",
|
||||
".py": "Python module",
|
||||
".go": "Go module",
|
||||
".rs": "Rust module",
|
||||
".java": "Java class",
|
||||
".kt": "Kotlin class",
|
||||
".swift": "Swift module",
|
||||
".c": "C source",
|
||||
".cpp": "C++ source",
|
||||
".h": "C/C++ header",
|
||||
".rb": "Ruby module",
|
||||
".php": "PHP script",
|
||||
".sh": "Shell script",
|
||||
".md": "Documentation",
|
||||
".json": "Configuration",
|
||||
".yaml": "Configuration",
|
||||
".yml": "Configuration",
|
||||
".toml": "Configuration",
|
||||
".ini": "Configuration",
|
||||
".env": "Environment config",
|
||||
".dockerfile": "Docker image definition",
|
||||
dockerfile: "Docker image definition",
|
||||
".sql": "Database schema/queries",
|
||||
".css": "Stylesheet",
|
||||
".scss": "SCSS stylesheet",
|
||||
".less": "LESS stylesheet",
|
||||
".html": "HTML template",
|
||||
".vue": "Vue component",
|
||||
".svelte": "Svelte component",
|
||||
};
|
||||
|
||||
function truncateForContext(content: string, promptLength: number): string {
|
||||
const maxChars = CONTEXT_BUDGET * CHARS_PER_TOKEN - promptLength;
|
||||
if (content.length <= maxChars) return content;
|
||||
return content.slice(0, maxChars - 20) + "\n[...truncated]";
|
||||
}
|
||||
|
||||
function buildFilePrompt(filePath: string, content: string): string {
|
||||
const name = basename(filePath);
|
||||
const ext = extname(filePath).toLowerCase();
|
||||
const typeHint = FILE_TYPE_PURPOSES[ext] || FILE_TYPE_PURPOSES[name.toLowerCase()] || ext || "file";
|
||||
|
||||
return `Analyze this ${typeHint} file. Respond in this exact format (one line each):
|
||||
PURPOSE: <concise one-sentence description of what this file does>
|
||||
DEPS: <comma-separated list of key dependencies/modules it relies on, or "none">
|
||||
CONCEPTS: <comma-separated list of key concepts/patterns used, or "none">
|
||||
|
||||
File: ${name}
|
||||
\`\`\`
|
||||
${content}
|
||||
\`\`\`
|
||||
`;
|
||||
}
|
||||
|
||||
function parseFileResponse(response: string): { purpose: string; deps: string[]; concepts: string[] } {
|
||||
const lines = response.split("\n");
|
||||
let purpose = "";
|
||||
let deps: string[] = [];
|
||||
let concepts: string[] = [];
|
||||
|
||||
for (const line of lines) {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.startsWith("PURPOSE:")) {
|
||||
purpose = trimmed.slice("PURPOSE:".length).trim();
|
||||
} else if (trimmed.startsWith("DEPS:")) {
|
||||
const depsStr = trimmed.slice("DEPS:".length).trim();
|
||||
deps = depsStr === "none" ? [] : depsStr.split(",").map((s) => s.trim()).filter(Boolean);
|
||||
} else if (trimmed.startsWith("CONCEPTS:")) {
|
||||
const conceptsStr = trimmed.slice("CONCEPTS:".length).trim();
|
||||
concepts = conceptsStr === "none" ? [] : conceptsStr.split(",").map((s) => s.trim()).filter(Boolean);
|
||||
}
|
||||
}
|
||||
|
||||
return { purpose, deps, concepts };
|
||||
}
|
||||
|
||||
function buildPackagePrompt(relativePath: string, fileSummaries: { name: string; purpose: string }[]): string {
|
||||
const filesList = fileSummaries.map((f) => `- ${f.name}: ${f.purpose}`).join("\n");
|
||||
return `Analyze this code package/directory. Respond in this exact format (one line each):
|
||||
ROLE: <concise one-sentence description of this package's role in the project>
|
||||
ARCH: <concise description of architecture/patterns used in this package>
|
||||
|
||||
Directory: ${relativePath}
|
||||
Files:
|
||||
${filesList}
|
||||
`;
|
||||
}
|
||||
|
||||
function parsePackageResponse(response: string): { role: string; arch: string } {
|
||||
const lines = response.split("\n");
|
||||
let role = "";
|
||||
let arch = "";
|
||||
|
||||
for (const line of lines) {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.startsWith("ROLE:")) {
|
||||
role = trimmed.slice("ROLE:".length).trim();
|
||||
} else if (trimmed.startsWith("ARCH:")) {
|
||||
arch = trimmed.slice("ARCH:".length).trim();
|
||||
}
|
||||
}
|
||||
|
||||
return { role, arch };
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// PRODUCTION: Real LLM calls
|
||||
// ============================================================================
|
||||
|
||||
export async function extractFileLLM(
|
||||
filePath: string,
|
||||
client?: LLMClient,
|
||||
): Promise<LLMFileData> {
|
||||
const content = readFileSync(filePath, "utf8");
|
||||
const hash = createHash("sha256").update(content).digest("hex");
|
||||
|
||||
// Check disk cache
|
||||
const cached = getCached(hash);
|
||||
if (cached) {
|
||||
const parsed = parseFileResponse(cached);
|
||||
return {
|
||||
purpose: parsed.purpose,
|
||||
exports: [], // AST provides precise exports
|
||||
deps: parsed.deps,
|
||||
concepts: parsed.concepts,
|
||||
};
|
||||
}
|
||||
|
||||
// Skip very large files
|
||||
const size = statSync(filePath).size;
|
||||
if (size > MAX_FILE_SIZE) {
|
||||
return {
|
||||
purpose: "Large/generated file",
|
||||
exports: [],
|
||||
deps: [],
|
||||
concepts: [],
|
||||
};
|
||||
}
|
||||
|
||||
// If no LLM client provided, fall back to heuristics (for backward compat / tests)
|
||||
if (!client) {
|
||||
return extractFileHeuristic(filePath, content);
|
||||
}
|
||||
|
||||
const prompt = buildFilePrompt(filePath, truncateForContext(content, 200));
|
||||
const response = await client.complete(prompt);
|
||||
setCached(hash, response);
|
||||
|
||||
const parsed = parseFileResponse(response);
|
||||
return {
|
||||
purpose: parsed.purpose,
|
||||
exports: [], // AST provides precise exports
|
||||
deps: parsed.deps,
|
||||
concepts: parsed.concepts,
|
||||
};
|
||||
}
|
||||
|
||||
export async function extractPackageLLM(
|
||||
relativePath: string,
|
||||
fileData: { name: string; purpose: string }[],
|
||||
client?: LLMClient,
|
||||
): Promise<LLMPackageData> {
|
||||
if (!client) {
|
||||
return extractPackageHeuristic(relativePath, fileData);
|
||||
}
|
||||
|
||||
const prompt = buildPackagePrompt(relativePath, fileData);
|
||||
const response = await client.complete(prompt);
|
||||
const parsed = parsePackageResponse(response);
|
||||
|
||||
return {
|
||||
role: parsed.role || dirNameToRole(relativePath),
|
||||
arch: parsed.arch || `Contains ${fileData.length} files.`,
|
||||
};
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// HEURISTIC FALLBACK (for tests / no-LLM mode)
|
||||
// ============================================================================
|
||||
|
||||
export async function extractFileHeuristic(
|
||||
filePath: string,
|
||||
content?: string,
|
||||
): Promise<LLMFileData> {
|
||||
const fileContent = content ?? readFileSync(filePath, "utf8");
|
||||
const ext = extname(filePath).toLowerCase();
|
||||
const name = basename(filePath);
|
||||
const baseName = basename(filePath, ext);
|
||||
|
||||
const exports = extractExportsHeuristic(fileContent, ext, name);
|
||||
const deps = extractDepsHeuristic(fileContent, ext);
|
||||
const purpose = generatePurpose(name, ext, baseName, exports);
|
||||
|
||||
return { purpose, exports, deps, concepts: [] };
|
||||
}
|
||||
|
||||
export async function extractPackageHeuristic(
|
||||
relativePath: string,
|
||||
fileData: { name: string; purpose: string }[],
|
||||
): Promise<LLMPackageData> {
|
||||
const dirName = basename(relativePath);
|
||||
const role = dirNameToRole(dirName);
|
||||
|
||||
const purposes = fileData.map((f) => f.purpose);
|
||||
const hasTests = purposes.some((p) => p.includes("Test"));
|
||||
const hasTypes = purposes.some((p) => p.includes("Type"));
|
||||
const hasComponents = purposes.some(
|
||||
(p) => p.includes("component") || p.includes("Component"),
|
||||
);
|
||||
const hasUtils = purposes.some((p) => p.includes("Utility"));
|
||||
|
||||
let arch = "";
|
||||
if (hasTests) arch += "Contains tests. ";
|
||||
if (hasTypes) arch += "Defines shared types. ";
|
||||
if (hasComponents) arch += "Component-based architecture. ";
|
||||
if (hasUtils) arch += "Shared utilities. ";
|
||||
if (!arch) arch = `Contains ${fileData.length} files.`;
|
||||
|
||||
return { role, arch: arch.trim() };
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// INTERNAL HEURISTIC HELPERS
|
||||
// ============================================================================
|
||||
|
||||
function dirNameToRole(dirName: string): string {
|
||||
if (dirName === ".") return "Project root";
|
||||
if (dirName === "src" || dirName === "lib" || dirName === "source") return "Source code";
|
||||
if (dirName === "test" || dirName === "tests" || dirName === "spec") return "Test suite";
|
||||
if (dirName === "docs" || dirName === "doc") return "Documentation";
|
||||
if (dirName === "config" || dirName === "configuration") return "Configuration";
|
||||
if (dirName === "utils" || dirName === "helpers" || dirName === "util") return "Utility functions";
|
||||
if (dirName === "types" || dirName === "type") return "Type definitions";
|
||||
if (dirName === "components" || dirName === "component") return "UI components";
|
||||
if (dirName === "hooks" || dirName === "hook") return "Custom hooks";
|
||||
if (dirName === "api" || dirName === "apis") return "API endpoints/handlers";
|
||||
if (dirName === "db" || dirName === "database" || dirName === "models") return "Database layer";
|
||||
if (dirName === "auth" || dirName === "authentication") return "Authentication layer";
|
||||
return `Package ${dirName}`;
|
||||
}
|
||||
|
||||
function extractExportsHeuristic(content: string, ext: string, _filename: string): string[] {
|
||||
const exports: string[] = [];
|
||||
|
||||
if ([".ts", ".tsx", ".js", ".jsx", ".mjs"].includes(ext)) {
|
||||
const exportRegex = /(?:^|\n)\s*export\s+(?:default\s+)?(?:async\s+)?(?:function\s+|class\s+|const\s+|let\s+|var\s+|interface\s+|type\s+|enum\s+)?([A-Za-z_$][A-Za-z0-9_$]*)/g;
|
||||
let match: RegExpExecArray | null;
|
||||
match = exportRegex.exec(content);
|
||||
while (match !== null) {
|
||||
exports.push(match[1]);
|
||||
match = exportRegex.exec(content);
|
||||
}
|
||||
|
||||
const namedExportRegex = /(?:^|\n)\s*export\s*\{\s*([^}]+)\s*\}/g;
|
||||
match = namedExportRegex.exec(content);
|
||||
while (match !== null) {
|
||||
const names = match[1].split(",").map((s) => s.trim().split(/\s+as\s+/)[0].trim());
|
||||
exports.push(...names);
|
||||
match = namedExportRegex.exec(content);
|
||||
}
|
||||
} else if (ext === ".py") {
|
||||
const pyRegex = /^(?:async\s+)?def\s+([A-Za-z_][A-Za-z0-9_]*)|class\s+([A-Za-z_][A-Za-z0-9_]*)/gm;
|
||||
let match: RegExpExecArray | null = pyRegex.exec(content);
|
||||
while (match !== null) {
|
||||
exports.push(match[1] || match[2]);
|
||||
match = pyRegex.exec(content);
|
||||
}
|
||||
} else if (ext === ".go") {
|
||||
const goRegex = /^(?:func|type|var|const)\s+([A-Z][A-Za-z0-9_]*)/gm;
|
||||
let match: RegExpExecArray | null = goRegex.exec(content);
|
||||
while (match !== null) {
|
||||
exports.push(match[1]);
|
||||
match = goRegex.exec(content);
|
||||
}
|
||||
} else if (ext === ".rs") {
|
||||
const rsRegex = /pub\s+(?:fn|struct|enum|trait|type|const|static|use)\s+([A-Za-z_][A-Za-z0-9_]*)/g;
|
||||
let match: RegExpExecArray | null = rsRegex.exec(content);
|
||||
while (match !== null) {
|
||||
exports.push(match[1]);
|
||||
match = rsRegex.exec(content);
|
||||
}
|
||||
}
|
||||
|
||||
return [...new Set(exports)];
|
||||
}
|
||||
|
||||
function extractDepsHeuristic(content: string, ext: string): string[] {
|
||||
const deps: string[] = [];
|
||||
|
||||
if ([".ts", ".tsx", ".js", ".jsx", ".mjs"].includes(ext)) {
|
||||
const importRegex = /import\s+(?:(?:type\s+)?\{[^}]*\}|\*\s+as\s+\w+|\w+)\s+from\s+['"]([^'"]+)['"]/g;
|
||||
let match: RegExpExecArray | null = importRegex.exec(content);
|
||||
while (match !== null) {
|
||||
deps.push(match[1]);
|
||||
match = importRegex.exec(content);
|
||||
}
|
||||
const requireRegex = /require\s*\(\s*['"]([^'"]+)['"]\s*\)/g;
|
||||
match = requireRegex.exec(content);
|
||||
while (match !== null) {
|
||||
deps.push(match[1]);
|
||||
match = requireRegex.exec(content);
|
||||
}
|
||||
} else if (ext === ".py") {
|
||||
const pyImportRegex = /^(?:from|import)\s+([A-Za-z_][A-Za-z0-9_.]*)/gm;
|
||||
let match: RegExpExecArray | null = pyImportRegex.exec(content);
|
||||
while (match !== null) {
|
||||
deps.push(match[1]);
|
||||
match = pyImportRegex.exec(content);
|
||||
}
|
||||
} else if (ext === ".go") {
|
||||
const goImportRegex = /"([^"]+)"/g;
|
||||
let match: RegExpExecArray | null = goImportRegex.exec(content);
|
||||
while (match !== null) {
|
||||
if (match[1].includes("/")) deps.push(match[1]);
|
||||
match = goImportRegex.exec(content);
|
||||
}
|
||||
} else if (ext === ".rs") {
|
||||
const rsUseRegex = /use\s+([A-Za-z_][A-Za-z0-9_:]*)/g;
|
||||
let match: RegExpExecArray | null = rsUseRegex.exec(content);
|
||||
while (match !== null) {
|
||||
deps.push(match[1]);
|
||||
match = rsUseRegex.exec(content);
|
||||
}
|
||||
}
|
||||
|
||||
return [...new Set(deps)];
|
||||
}
|
||||
|
||||
function generatePurpose(name: string, ext: string, _baseName: string, exports: string[]): string {
|
||||
if (/test|spec/i.test(name) && exports.length === 0) return "Test suite";
|
||||
if (/config|settings/i.test(name)) return "Configuration";
|
||||
if (/util|helper/i.test(name)) return "Utility functions";
|
||||
if (/types?\.d?\.ts$/.test(name)) return "Type definitions";
|
||||
if (/index\./.test(name)) return "Module entry point";
|
||||
if (/middleware/.test(name)) return "Middleware";
|
||||
if (/route/.test(name)) return "Route handlers";
|
||||
if (/controller/.test(name)) return "Controller";
|
||||
if (/service/.test(name)) return "Service layer";
|
||||
if (/model/.test(name)) return "Data model";
|
||||
if (/schema/.test(name)) return "Data schema";
|
||||
if (/component/.test(name) || /\.tsx$/.test(name) || /\.vue$/.test(name) || /\.svelte$/.test(name)) {
|
||||
return "UI component";
|
||||
}
|
||||
if (/hook|use[A-Z]/.test(name)) return "React hook";
|
||||
if (/style|\.css|\.scss|\.less/.test(name)) return "Styling";
|
||||
if (/docker/i.test(name)) return "Container definition";
|
||||
if (/\.env/.test(name)) return "Environment variables";
|
||||
if (/readme/i.test(name)) return "Project documentation";
|
||||
|
||||
if (exports.length > 0) {
|
||||
const firstFew = exports.slice(0, 3).join(", ");
|
||||
if (exports.length <= 3) return `Exports: ${firstFew}`;
|
||||
return `Exports ${exports.length} symbols: ${firstFew}...`;
|
||||
}
|
||||
|
||||
return FILE_TYPE_PURPOSES[ext] || (ext ? `${ext.slice(1).toUpperCase()} file` : `${name} file`);
|
||||
}
|
||||
@@ -45,11 +45,12 @@ export async function processFiles<T, R>(
|
||||
files: T[],
|
||||
processor: (file: T) => Promise<R>,
|
||||
options: BatchOptions = {},
|
||||
onProgress?: (completed: number, total: number, currentFile: T) => void,
|
||||
): Promise<R[]> {
|
||||
const opts = { ...DEFAULT_OPTIONS, ...options };
|
||||
const limit = pLimit(opts.concurrency);
|
||||
|
||||
const results: R[] = [];
|
||||
let completed = 0;
|
||||
let batchCount = 0;
|
||||
|
||||
const tasks = files.map((file, index) =>
|
||||
@@ -59,7 +60,10 @@ export async function processFiles<T, R>(
|
||||
batchCount++;
|
||||
await sleep(opts.batchDelayMs);
|
||||
}
|
||||
return withRetry(() => processor(file), opts);
|
||||
const result = await withRetry(() => processor(file), opts);
|
||||
completed++;
|
||||
onProgress?.(completed, files.length, file);
|
||||
return result;
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
import {
|
||||
readFileSync,
|
||||
writeFileSync,
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
renameSync,
|
||||
} from "fs";
|
||||
import { dirname, join } from "path";
|
||||
|
||||
const CACHE_FILE = "llm-cache.json";
|
||||
|
||||
interface CacheEntry {
|
||||
result: string;
|
||||
ts: number;
|
||||
}
|
||||
|
||||
function getCachePath(cacheDir?: string): string {
|
||||
const base = cacheDir || process.cwd();
|
||||
return join(base, ".cache", CACHE_FILE);
|
||||
}
|
||||
|
||||
function ensureCacheDir(cachePath: string): void {
|
||||
const dir = dirname(cachePath);
|
||||
if (!existsSync(dir)) {
|
||||
mkdirSync(dir, { recursive: true });
|
||||
}
|
||||
}
|
||||
|
||||
function loadCache(cachePath: string): Record<string, CacheEntry> {
|
||||
if (!existsSync(cachePath)) return {};
|
||||
try {
|
||||
const raw = readFileSync(cachePath, "utf8");
|
||||
return JSON.parse(raw) as Record<string, CacheEntry>;
|
||||
} catch {
|
||||
// Corrupted cache — start fresh
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
function saveCache(cachePath: string, cache: Record<string, CacheEntry>): void {
|
||||
ensureCacheDir(cachePath);
|
||||
const tmp = `${cachePath}.tmp`;
|
||||
writeFileSync(tmp, JSON.stringify(cache, null, 2));
|
||||
renameSync(tmp, cachePath);
|
||||
}
|
||||
|
||||
export function getCached(hash: string, cacheDir?: string): string | undefined {
|
||||
const cachePath = getCachePath(cacheDir);
|
||||
const cache = loadCache(cachePath);
|
||||
return cache[hash]?.result;
|
||||
}
|
||||
|
||||
export function setCached(
|
||||
hash: string,
|
||||
result: string,
|
||||
cacheDir?: string,
|
||||
): void {
|
||||
const cachePath = getCachePath(cacheDir);
|
||||
const cache = loadCache(cachePath);
|
||||
cache[hash] = { result, ts: Date.now() };
|
||||
saveCache(cachePath, cache);
|
||||
}
|
||||
@@ -0,0 +1,284 @@
|
||||
import { readFileSync, statSync } from "fs";
|
||||
import { createHash } from "crypto";
|
||||
import { extname, basename } from "path";
|
||||
import type { LLMClient } from "./llm-client.js";
|
||||
import { getCached, setCached } from "./llm-cache.js";
|
||||
import { LLMError } from "./llm-error.js";
|
||||
|
||||
interface LLMFileData {
|
||||
purpose: string;
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
concepts: string[];
|
||||
}
|
||||
|
||||
interface LLMPackageData {
|
||||
role: string;
|
||||
arch: string;
|
||||
}
|
||||
|
||||
const MAX_FILE_SIZE = 500 * 1024; // 500KB
|
||||
const CONTEXT_BUDGET = 4000; // tokens
|
||||
const CHARS_PER_TOKEN = 4; // approximate for ASCII
|
||||
|
||||
// Known binary extensions — skip without reading content
|
||||
const BINARY_EXTENSIONS = new Set([
|
||||
".png",
|
||||
".jpg",
|
||||
".jpeg",
|
||||
".gif",
|
||||
".bmp",
|
||||
".webp",
|
||||
".ico",
|
||||
".svgz",
|
||||
".mp3",
|
||||
".mp4",
|
||||
".avi",
|
||||
".mov",
|
||||
".mkv",
|
||||
".flv",
|
||||
".wmv",
|
||||
".wav",
|
||||
".ogg",
|
||||
".flac",
|
||||
".aac",
|
||||
".wma",
|
||||
".zip",
|
||||
".tar",
|
||||
".gz",
|
||||
".bz2",
|
||||
".xz",
|
||||
".7z",
|
||||
".rar",
|
||||
".exe",
|
||||
".dll",
|
||||
".so",
|
||||
".dylib",
|
||||
".bin",
|
||||
".pdf",
|
||||
".doc",
|
||||
".docx",
|
||||
".xls",
|
||||
".xlsx",
|
||||
".ppt",
|
||||
".pptx",
|
||||
".wasm",
|
||||
".class",
|
||||
".jar",
|
||||
".o",
|
||||
".a",
|
||||
".ttf",
|
||||
".otf",
|
||||
".woff",
|
||||
".woff2",
|
||||
".eot",
|
||||
".db",
|
||||
".sqlite",
|
||||
".sqlite3",
|
||||
]);
|
||||
|
||||
function isBinaryFile(filePath: string): boolean {
|
||||
// Fast-path: check extension
|
||||
const ext = extname(filePath).toLowerCase();
|
||||
if (BINARY_EXTENSIONS.has(ext)) return true;
|
||||
|
||||
// Fallback: sniff first 8KB for null bytes or non-printable ratio
|
||||
try {
|
||||
const buf = readFileSync(filePath).subarray(0, 8192);
|
||||
let nonPrintable = 0;
|
||||
for (let i = 0; i < buf.length; i++) {
|
||||
const b = buf[i];
|
||||
if (b === 0) return true; // null byte = definitely binary
|
||||
if (b < 0x20 && b !== 0x09 && b !== 0x0a && b !== 0x0d) {
|
||||
nonPrintable++;
|
||||
}
|
||||
}
|
||||
// If >30% non-printable, treat as binary
|
||||
return nonPrintable / buf.length > 0.3;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function truncateForContext(content: string, promptLength: number): string {
|
||||
const maxChars = CONTEXT_BUDGET * CHARS_PER_TOKEN - promptLength;
|
||||
if (content.length <= maxChars) return content;
|
||||
return `${content.slice(0, maxChars - 20)}\n[...truncated]`;
|
||||
}
|
||||
|
||||
function buildFilePrompt(filePath: string, content: string): string {
|
||||
const name = basename(filePath);
|
||||
return `Analyze this file. Respond in this exact format (one line each):
|
||||
PURPOSE: <concise one-sentence description of what this file does>
|
||||
DEPS: <comma-separated list of key dependencies/modules it relies on, or "none">
|
||||
CONCEPTS: <comma-separated list of key concepts/patterns used, or "none">
|
||||
|
||||
File: ${name}
|
||||
\`\`\`
|
||||
${content}
|
||||
\`\`\`
|
||||
`;
|
||||
}
|
||||
|
||||
function parseFileResponse(response: string): {
|
||||
purpose: string;
|
||||
deps: string[];
|
||||
concepts: string[];
|
||||
} {
|
||||
const lines = response.split("\n");
|
||||
let purpose = "";
|
||||
let deps: string[] = [];
|
||||
let concepts: string[] = [];
|
||||
|
||||
for (const line of lines) {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.startsWith("PURPOSE:")) {
|
||||
purpose = trimmed.slice("PURPOSE:".length).trim();
|
||||
} else if (trimmed.startsWith("DEPS:")) {
|
||||
const depsStr = trimmed.slice("DEPS:".length).trim();
|
||||
deps =
|
||||
depsStr === "none"
|
||||
? []
|
||||
: depsStr
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
} else if (trimmed.startsWith("CONCEPTS:")) {
|
||||
const conceptsStr = trimmed.slice("CONCEPTS:".length).trim();
|
||||
concepts =
|
||||
conceptsStr === "none"
|
||||
? []
|
||||
: conceptsStr
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
}
|
||||
|
||||
return { purpose, deps, concepts };
|
||||
}
|
||||
|
||||
function buildPackagePrompt(
|
||||
relativePath: string,
|
||||
fileSummaries: { name: string; purpose: string }[],
|
||||
): string {
|
||||
const filesList = fileSummaries
|
||||
.map((f) => `- ${f.name}: ${f.purpose}`)
|
||||
.join("\n");
|
||||
return `Analyze this code package/directory. Respond in this exact format (one line each):
|
||||
ROLE: <concise one-sentence description of this package's role in the project>
|
||||
ARCH: <concise description of architecture/patterns used in this package>
|
||||
|
||||
Directory: ${relativePath}
|
||||
Files:
|
||||
${filesList}
|
||||
`;
|
||||
}
|
||||
|
||||
function parsePackageResponse(response: string): {
|
||||
role: string;
|
||||
arch: string;
|
||||
} {
|
||||
const lines = response.split("\n");
|
||||
let role = "";
|
||||
let arch = "";
|
||||
|
||||
for (const line of lines) {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.startsWith("ROLE:")) {
|
||||
role = trimmed.slice("ROLE:".length).trim();
|
||||
} else if (trimmed.startsWith("ARCH:")) {
|
||||
arch = trimmed.slice("ARCH:".length).trim();
|
||||
}
|
||||
}
|
||||
|
||||
return { role, arch };
|
||||
}
|
||||
|
||||
export async function extractFileLLM(
|
||||
filePath: string,
|
||||
client?: LLMClient,
|
||||
cacheDir?: string,
|
||||
): Promise<LLMFileData> {
|
||||
// LLM-only: require a client
|
||||
if (!client) {
|
||||
throw new LLMError(
|
||||
"No LLM client configured. " +
|
||||
"Set OPENAI_API_KEY / KIMI_API_KEY environment variable, " +
|
||||
"or run inside Pi with a configured model.",
|
||||
);
|
||||
}
|
||||
|
||||
// Skip binary files
|
||||
if (isBinaryFile(filePath)) {
|
||||
return {
|
||||
purpose: "Binary file",
|
||||
exports: [],
|
||||
deps: [],
|
||||
concepts: [],
|
||||
};
|
||||
}
|
||||
|
||||
const content = readFileSync(filePath, "utf8");
|
||||
const hash = createHash("sha256").update(content).digest("hex");
|
||||
|
||||
// Check disk cache
|
||||
const cached = getCached(hash, cacheDir);
|
||||
if (cached) {
|
||||
const parsed = parseFileResponse(cached);
|
||||
return {
|
||||
purpose: parsed.purpose,
|
||||
exports: [], // AST provides precise exports
|
||||
deps: parsed.deps,
|
||||
concepts: parsed.concepts,
|
||||
};
|
||||
}
|
||||
|
||||
// Skip very large files
|
||||
const size = statSync(filePath).size;
|
||||
if (size > MAX_FILE_SIZE) {
|
||||
return {
|
||||
purpose: "Large file",
|
||||
exports: [],
|
||||
deps: [],
|
||||
concepts: [],
|
||||
};
|
||||
}
|
||||
|
||||
const prompt = buildFilePrompt(filePath, truncateForContext(content, 200));
|
||||
const response = await client.complete(prompt);
|
||||
setCached(hash, response, cacheDir);
|
||||
|
||||
const parsed = parseFileResponse(response);
|
||||
return {
|
||||
purpose: parsed.purpose,
|
||||
exports: [], // AST provides precise exports
|
||||
deps: parsed.deps,
|
||||
concepts: parsed.concepts,
|
||||
};
|
||||
}
|
||||
|
||||
export async function extractPackageLLM(
|
||||
relativePath: string,
|
||||
fileData: { name: string; purpose: string }[],
|
||||
client?: LLMClient,
|
||||
_cacheDir?: string,
|
||||
): Promise<LLMPackageData> {
|
||||
// LLM-only: require a client
|
||||
if (!client) {
|
||||
throw new LLMError(
|
||||
"No LLM client configured. " +
|
||||
"Set OPENAI_API_KEY / KIMI_API_KEY environment variable, " +
|
||||
"or run inside Pi with a configured model.",
|
||||
);
|
||||
}
|
||||
|
||||
const prompt = buildPackagePrompt(relativePath, fileData);
|
||||
const response = await client.complete(prompt);
|
||||
const parsed = parsePackageResponse(response);
|
||||
|
||||
return {
|
||||
role: parsed.role || `Package ${basename(relativePath)}`,
|
||||
arch: parsed.arch || `Contains ${fileData.length} files.`,
|
||||
};
|
||||
}
|
||||
@@ -34,6 +34,15 @@ export class PiLLMClient implements LLMClient {
|
||||
// Dynamically import Pi's AI module (available in the Pi runtime)
|
||||
const { complete } = await import("@mariozechner/pi-ai");
|
||||
|
||||
// Resolve auth through Pi's model registry (handles /login, env vars, etc.)
|
||||
const auth = await ctx.modelRegistry?.getApiKeyAndHeaders?.(model);
|
||||
if (auth && !auth.ok) {
|
||||
throw new LLMError(
|
||||
`Pi LLM auth error: ${auth.error}. ` +
|
||||
"Run /login in Pi to configure authentication.",
|
||||
);
|
||||
}
|
||||
|
||||
const response = await complete(
|
||||
model,
|
||||
{
|
||||
@@ -50,6 +59,7 @@ export class PiLLMClient implements LLMClient {
|
||||
{
|
||||
temperature: 0.1,
|
||||
maxTokens: 256,
|
||||
...(auth?.ok ? { apiKey: auth.apiKey, headers: auth.headers } : {}),
|
||||
},
|
||||
);
|
||||
|
||||
+67
-2
@@ -1,4 +1,4 @@
|
||||
import type { FileEntry } from "./format.js";
|
||||
import type { FileEntry } from "./directory-model.js";
|
||||
|
||||
interface LLMFileData {
|
||||
purpose: string;
|
||||
@@ -9,6 +9,23 @@ interface LLMFileData {
|
||||
interface ASTFileData {
|
||||
exports: string[];
|
||||
deps: string[];
|
||||
classes: Array<{
|
||||
name: string;
|
||||
methods: Array<{
|
||||
name: string;
|
||||
params: string[];
|
||||
returns?: string;
|
||||
calls: string[];
|
||||
raises: string[];
|
||||
}>;
|
||||
}>;
|
||||
functions: Array<{
|
||||
name: string;
|
||||
params: string[];
|
||||
returns?: string;
|
||||
calls: string[];
|
||||
raises: string[];
|
||||
}>;
|
||||
}
|
||||
|
||||
export function mergeFileData(
|
||||
@@ -16,10 +33,58 @@ export function mergeFileData(
|
||||
llm: LLMFileData,
|
||||
ast: ASTFileData | null,
|
||||
): FileEntry {
|
||||
const mergedExports = ast?.exports ?? llm.exports;
|
||||
|
||||
// Remove simple export names that will be replaced by rich AST entries
|
||||
const richNames = new Set<string>();
|
||||
if (ast) {
|
||||
for (const cls of ast.classes) richNames.add(cls.name);
|
||||
for (const func of ast.functions) richNames.add(func.name);
|
||||
}
|
||||
const dedupedExports = mergedExports.filter((e) => !richNames.has(e));
|
||||
|
||||
// Encode classes into exports using compact DSL
|
||||
if (ast && ast.classes.length > 0) {
|
||||
for (const cls of ast.classes) {
|
||||
const classExports: string[] = [`class:${cls.name}`];
|
||||
for (const method of cls.methods) {
|
||||
const paramStr = method.params.join(", ").replace(/\s+/g, " ");
|
||||
const returnStr = method.returns
|
||||
? ` → ${method.returns.replace(/\s+/g, " ")}`
|
||||
: "";
|
||||
classExports.push(`method:${method.name}(${paramStr})${returnStr}`);
|
||||
for (const call of method.calls) {
|
||||
classExports.push(`call:${call}`);
|
||||
}
|
||||
for (const raise of method.raises) {
|
||||
classExports.push(`raise:${raise}`);
|
||||
}
|
||||
}
|
||||
dedupedExports.push(...classExports);
|
||||
}
|
||||
}
|
||||
|
||||
// Encode top-level functions
|
||||
if (ast && ast.functions.length > 0) {
|
||||
for (const func of ast.functions) {
|
||||
const paramStr = func.params.join(", ").replace(/\s+/g, " ");
|
||||
const returnStr = func.returns
|
||||
? ` → ${func.returns.replace(/\s+/g, " ")}`
|
||||
: "";
|
||||
dedupedExports.push(`func:${func.name}(${paramStr})${returnStr}`);
|
||||
for (const call of func.calls) {
|
||||
dedupedExports.push(`call:${call}`);
|
||||
}
|
||||
for (const raise of func.raises) {
|
||||
dedupedExports.push(`raise:${raise}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
name: fileName,
|
||||
purpose: llm.purpose,
|
||||
exports: ast?.exports ?? llm.exports,
|
||||
exports: dedupedExports,
|
||||
deps: [...new Set([...(ast?.deps ?? []), ...llm.deps])],
|
||||
};
|
||||
}
|
||||
|
||||
+146
-53
@@ -1,69 +1,162 @@
|
||||
import { dirname, join, basename, relative } from "path";
|
||||
import { existsSync, readFileSync, writeFileSync } from "fs";
|
||||
import { parsePackageMap, renderPackageMap } from "./format.js";
|
||||
import { extractFileLLM } from "./llm-extract.js";
|
||||
import { extractFileAST } from "./ast-extract.js";
|
||||
import { mergeFileData } from "./merge.js";
|
||||
import { generateDirectoryMap } from "./init.js";
|
||||
import { readdirSync, statSync } from "fs";
|
||||
import type { LLMClient } from "./llm-client.js";
|
||||
import { basename, dirname, relative, resolve } from "path";
|
||||
import { existsSync, readFileSync } from "fs";
|
||||
import { parsePackageMap } from "./format.js";
|
||||
import {
|
||||
discoverProject,
|
||||
generateDirectoryArtifacts,
|
||||
buildDirectoryContext,
|
||||
} from "./init.js";
|
||||
import type { DirectoryEntry } from "./discover.js";
|
||||
import type { LLMClient } from "./llm/llm-client.js";
|
||||
import { loadConfig } from "./config.js";
|
||||
|
||||
const SMALL_PACKAGE_THRESHOLD = 10;
|
||||
export type PatchMode = "auto" | "small" | "structural";
|
||||
|
||||
export interface PatchOptions {
|
||||
patchMode?: PatchMode;
|
||||
rootPath?: string;
|
||||
}
|
||||
|
||||
const SMALL_CHANGE_FILE_WINDOW = 3;
|
||||
const STRUCTURAL_FILE_HINT =
|
||||
/(^|\b)(index|config|cli|command|init|format|discover|patch|validate)(\b|\.)/i;
|
||||
|
||||
export async function patchFile(
|
||||
filePath: string,
|
||||
llmClient?: LLMClient,
|
||||
cacheDir?: string,
|
||||
options: PatchOptions = {},
|
||||
): Promise<void> {
|
||||
const dirPath = dirname(filePath);
|
||||
const mapPath = join(dirPath, ".pi-map.md");
|
||||
const rootPath = resolve(options.rootPath ?? cacheDir ?? process.cwd());
|
||||
const absFilePath = resolve(rootPath, filePath);
|
||||
const dirPath = dirname(absFilePath);
|
||||
const relDir = normalizeRelativePath(relative(rootPath, dirPath));
|
||||
const entries = discoverProject(rootPath);
|
||||
const entry = entries.find((candidate) => candidate.relativePath === relDir);
|
||||
|
||||
if (!existsSync(mapPath)) {
|
||||
// No map exists yet — would need to generate from scratch
|
||||
console.warn(`No .pi-map.md found in ${dirPath}`);
|
||||
if (!entry) {
|
||||
console.warn(`No discovered directory entry found for ${relDir}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const allFiles = readdirSync(dirPath).filter(
|
||||
(f: string) => !f.startsWith(".") && !f.endsWith(".md"),
|
||||
const config = loadConfig(rootPath);
|
||||
const routingOpts = {
|
||||
tagCap: config.tagCap,
|
||||
workflowHintCap: config.workflowHintCap,
|
||||
};
|
||||
const mode = determinePatchMode(
|
||||
entry,
|
||||
absFilePath,
|
||||
rootPath,
|
||||
options.patchMode,
|
||||
);
|
||||
const changedCtx = buildDirectoryContext(entries, entry);
|
||||
await generateDirectoryArtifacts(
|
||||
entry,
|
||||
changedCtx,
|
||||
llmClient,
|
||||
cacheDir,
|
||||
undefined,
|
||||
routingOpts,
|
||||
"both",
|
||||
);
|
||||
const isSmallPackage = allFiles.length < SMALL_PACKAGE_THRESHOLD;
|
||||
|
||||
if (isSmallPackage) {
|
||||
// Full rewrite for small packages
|
||||
const files = allFiles.filter((f) => {
|
||||
const st = statSync(join(dirPath, f));
|
||||
return st.isFile();
|
||||
});
|
||||
const relDir = relative(process.cwd(), dirPath) || ".";
|
||||
await generateDirectoryMap(
|
||||
{
|
||||
dirPath,
|
||||
relativePath: relDir,
|
||||
files,
|
||||
},
|
||||
for (const ancestor of getAncestorEntries(entries, changedCtx, entry)) {
|
||||
const ancestorCtx = buildDirectoryContext(entries, ancestor);
|
||||
await generateDirectoryArtifacts(
|
||||
ancestor,
|
||||
ancestorCtx,
|
||||
llmClient,
|
||||
cacheDir,
|
||||
undefined,
|
||||
routingOpts,
|
||||
mode === "small" ? "index" : "both",
|
||||
);
|
||||
console.log(
|
||||
`Full rewrite of ${mapPath} (small package: ${allFiles.length} files)`,
|
||||
);
|
||||
} else {
|
||||
// Section-level patch
|
||||
const existing = parsePackageMap(readFileSync(mapPath, "utf8"));
|
||||
const llmData = await extractFileLLM(filePath, llmClient);
|
||||
const astData = await extractFileAST(filePath);
|
||||
const fileName = basename(filePath);
|
||||
const updatedFile = mergeFileData(fileName, llmData, astData);
|
||||
|
||||
// Replace the matching file entry
|
||||
const idx = existing.files.findIndex((f) => f.name === updatedFile.name);
|
||||
if (idx >= 0) {
|
||||
existing.files[idx] = updatedFile;
|
||||
} else {
|
||||
existing.files.push(updatedFile);
|
||||
}
|
||||
|
||||
existing.dirty = `${new Date().toISOString()}: ${fileName} patched (section-level)`;
|
||||
writeFileSync(mapPath, renderPackageMap(existing));
|
||||
console.log(`Patched ${mapPath}`);
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Patched ${joinArtifactPath(relDir, ".pi-map.md")} (patchMode: ${mode})`,
|
||||
);
|
||||
}
|
||||
|
||||
function determinePatchMode(
|
||||
entry: DirectoryEntry,
|
||||
absFilePath: string,
|
||||
rootPath: string,
|
||||
explicitMode: PatchMode = "auto",
|
||||
): Exclude<PatchMode, "auto"> {
|
||||
if (explicitMode !== "auto") {
|
||||
return explicitMode;
|
||||
}
|
||||
|
||||
const existingMapPath = resolve(entry.dirPath, ".pi-map.md");
|
||||
const fileName = basename(absFilePath);
|
||||
const childCount =
|
||||
entry.relativePath === "."
|
||||
? 0
|
||||
: countDirectChildren(rootPath, entry.relativePath);
|
||||
|
||||
if (childCount > 0) {
|
||||
return "structural";
|
||||
}
|
||||
|
||||
if (entry.files.length > SMALL_CHANGE_FILE_WINDOW) {
|
||||
return "structural";
|
||||
}
|
||||
|
||||
if (STRUCTURAL_FILE_HINT.test(fileName)) {
|
||||
return "structural";
|
||||
}
|
||||
|
||||
if (existsSync(existingMapPath)) {
|
||||
const existing = parsePackageMap(readFileSync(existingMapPath, "utf8"));
|
||||
const existingFile = existing.files.find(
|
||||
(candidate) => candidate.name === fileName,
|
||||
);
|
||||
if (!existingFile) {
|
||||
return "structural";
|
||||
}
|
||||
}
|
||||
|
||||
return "small";
|
||||
}
|
||||
|
||||
function countDirectChildren(rootPath: string, relDir: string): number {
|
||||
const entries = discoverProject(rootPath);
|
||||
return entries.filter((entry) => {
|
||||
if (entry.relativePath === "." || entry.relativePath === relDir)
|
||||
return false;
|
||||
const parent = normalizeRelativePath(
|
||||
entry.relativePath.split("/").slice(0, -1).join("/"),
|
||||
);
|
||||
return parent === relDir;
|
||||
}).length;
|
||||
}
|
||||
|
||||
function getAncestorEntries(
|
||||
entries: DirectoryEntry[],
|
||||
ctx: ReturnType<typeof buildDirectoryContext>,
|
||||
entry: DirectoryEntry,
|
||||
): DirectoryEntry[] {
|
||||
const result: DirectoryEntry[] = [];
|
||||
let parent = ctx.parentMap.get(entry.relativePath);
|
||||
while (parent) {
|
||||
const ancestor = entries.find(
|
||||
(candidate) => candidate.relativePath === parent,
|
||||
);
|
||||
if (ancestor) {
|
||||
result.push(ancestor);
|
||||
}
|
||||
parent = ctx.parentMap.get(parent);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function normalizeRelativePath(relPath: string): string {
|
||||
if (!relPath || relPath === ".") return ".";
|
||||
return relPath.replace(/\\/g, "/");
|
||||
}
|
||||
|
||||
function joinArtifactPath(relDir: string, artifactName: string): string {
|
||||
return relDir === "." ? artifactName : `${relDir}/${artifactName}`;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,670 @@
|
||||
import { readdirSync, statSync, readFileSync } from "fs";
|
||||
import { join, relative } from "path";
|
||||
import type { SkillConfig, PromptInjectionMode } from "./config.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Outgoing-context scanning (Slice 3)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ReinjectDecision {
|
||||
needed: boolean;
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether reinjection is needed for a given event.
|
||||
* Checks mode, scans outgoing messages for the canonical marker,
|
||||
* falls back to payload inspection, and evaluates relevant-turn triggers.
|
||||
*/
|
||||
export function shouldReinjectForEvent(
|
||||
event: {
|
||||
messages?: Array<{ content?: unknown; text?: string }>;
|
||||
type?: string;
|
||||
payload?: unknown;
|
||||
},
|
||||
mode: PromptInjectionMode,
|
||||
): ReinjectDecision {
|
||||
if (mode !== "strong" && mode !== "strict") {
|
||||
return { needed: false, reason: "mode_does_not_require_reinjection" };
|
||||
}
|
||||
|
||||
// Slice 3b: root-pair artifact changes invalidate prior injections
|
||||
if (event.type === "artifact_change") {
|
||||
return { needed: true, reason: "root_pair_artifact_changed" };
|
||||
}
|
||||
|
||||
if (event.messages && outgoingMessagesHaveMarker(event.messages)) {
|
||||
return { needed: false, reason: "marker_present_in_messages" };
|
||||
}
|
||||
|
||||
if (event.payload && providerPayloadHasMarker(event.payload)) {
|
||||
return { needed: false, reason: "marker_present_in_payload" };
|
||||
}
|
||||
|
||||
if (event.type && isRelevantTurnForReinjection(event.type)) {
|
||||
return { needed: true, reason: "relevant_turn_and_marker_absent" };
|
||||
}
|
||||
|
||||
return { needed: false, reason: "not_a_relevant_turn" };
|
||||
}
|
||||
|
||||
function extractTextFromMessage(m: {
|
||||
content?: unknown;
|
||||
text?: string;
|
||||
}): string {
|
||||
if (typeof m.content === "string") return m.content;
|
||||
if (Array.isArray(m.content)) {
|
||||
return m.content
|
||||
.map((block: any) => {
|
||||
if (typeof block === "string") return block;
|
||||
if (typeof block.text === "string") return block.text;
|
||||
if (typeof block.content === "string") return block.content;
|
||||
return "";
|
||||
})
|
||||
.join("");
|
||||
}
|
||||
return m.text ?? "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan an array of message-like objects for the canonical root-pair marker.
|
||||
* Normalizes array-based content blocks (e.g., TextContent[]) to strings.
|
||||
*/
|
||||
export function outgoingMessagesHaveMarker(
|
||||
messages: Array<{ content?: unknown; text?: string }>,
|
||||
): boolean {
|
||||
return messages.some((m) => hasRootPairMarker(extractTextFromMessage(m)));
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback inspection of a final provider payload for the canonical marker.
|
||||
* Accepts either a string or an object that will be JSON-stringified.
|
||||
*/
|
||||
export function providerPayloadHasMarker(payload: unknown): boolean {
|
||||
if (!payload) return false;
|
||||
if (typeof payload === "string") {
|
||||
return hasRootPairMarker(payload);
|
||||
}
|
||||
try {
|
||||
return hasRootPairMarker(JSON.stringify(payload));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Relevant-turn types that trigger reinjection in strong mode.
|
||||
*/
|
||||
export type RelevantTurnType =
|
||||
| "agent_start"
|
||||
| "edit_intent"
|
||||
| "architecture_sensitive"
|
||||
| "compaction"
|
||||
| "artifact_change";
|
||||
|
||||
/**
|
||||
* Determine whether a given event type is a relevant reinjection trigger.
|
||||
*/
|
||||
export function isRelevantTurnForReinjection(eventType: string): boolean {
|
||||
const relevant: RelevantTurnType[] = [
|
||||
"agent_start",
|
||||
"edit_intent",
|
||||
"architecture_sensitive",
|
||||
"compaction",
|
||||
"artifact_change",
|
||||
];
|
||||
return relevant.includes(eventType as RelevantTurnType);
|
||||
}
|
||||
|
||||
function detectEditIntentFromText(combined: string): boolean {
|
||||
const patterns = [
|
||||
/\b(edit|modify|update|change|refactor|fix|patch|rewrite|delete|remove|add)\s+(the|a|this|that|these|those|file|code|function|method|class|module|component|line)\b/i,
|
||||
/\b(write|create|generate)\s+(new|a|the)\s+(file|function|class|module|component)\b/i,
|
||||
/```[\s\S]*?\b(edit|modify|update|change|refactor|fix|delete|remove)\b/i,
|
||||
/\bapply\s+(the|a|this|that)\s+(change|edit|patch|fix)\b/i,
|
||||
];
|
||||
return patterns.some((p) => p.test(combined));
|
||||
}
|
||||
|
||||
/**
|
||||
* Heuristic detection of edit intent from message content.
|
||||
*/
|
||||
export function detectEditIntent(
|
||||
messages: Array<{ content?: unknown; text?: string }>,
|
||||
): boolean {
|
||||
if (!messages || messages.length === 0) return false;
|
||||
const combined = messages.map(extractTextFromMessage).join(" ");
|
||||
return detectEditIntentFromText(combined);
|
||||
}
|
||||
|
||||
function detectArchitectureSensitiveReasoningFromText(
|
||||
combined: string,
|
||||
): boolean {
|
||||
const patterns = [
|
||||
/\barchitectur(e|al)\b/i,
|
||||
/\bdesign\s+(decision|pattern|choice|principle|review)\b/i,
|
||||
/\b(restructure|reorganize|redesign|rearchitect)\b/i,
|
||||
/\b(system|high.level|macro|structural)\s+(design|architecture|overview)\b/i,
|
||||
/\bdependency\s+(injection|graph|cycle|inversion)\b/i,
|
||||
/\b(api|interface|contract|schema|protocol)\s+(design|change|migration|breaking)\b/i,
|
||||
/\b(microservice|monolith|modular|layered|hexagonal|clean)\s+arch/i,
|
||||
/\bdata\s+(model|flow|pipeline|architecture)\b/i,
|
||||
/\b(scalability|performance|security|maintainability)\s+(concern|tradeoff|decision)\b/i,
|
||||
];
|
||||
return patterns.some((p) => p.test(combined));
|
||||
}
|
||||
|
||||
/**
|
||||
* Heuristic detection of architecture-sensitive reasoning from message content.
|
||||
*/
|
||||
export function detectArchitectureSensitiveReasoning(
|
||||
messages: Array<{ content?: unknown; text?: string }>,
|
||||
): boolean {
|
||||
if (!messages || messages.length === 0) return false;
|
||||
const combined = messages.map(extractTextFromMessage).join(" ");
|
||||
return detectArchitectureSensitiveReasoningFromText(combined);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Root-pair artifact change detection
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface RootPairMtimes {
|
||||
mapMtime?: number;
|
||||
indexMtime?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read current mtimes for the root pair artifacts, if they exist.
|
||||
*/
|
||||
export function getRootPairMtimes(cwd: string): RootPairMtimes {
|
||||
const mapPath = join(cwd, ".pi-map.md");
|
||||
const indexPath = join(cwd, ".pi-map.index.md");
|
||||
const result: RootPairMtimes = {};
|
||||
try {
|
||||
result.mapMtime = statSync(mapPath).mtimeMs;
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
try {
|
||||
result.indexMtime = statSync(indexPath).mtimeMs;
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare current root-pair mtimes against previously recorded ones.
|
||||
*/
|
||||
export function rootPairChanged(
|
||||
current: RootPairMtimes,
|
||||
previous: RootPairMtimes,
|
||||
): boolean {
|
||||
return (
|
||||
current.mapMtime !== previous.mapMtime ||
|
||||
current.indexMtime !== previous.indexMtime
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Canonical markers for injected root-pair content.
|
||||
* These must be stable across turns and easy to scan in outgoing context.
|
||||
*/
|
||||
export const ROOT_PAIR_START_MARKER = "<!-- PI_MAP_ROOT_PAIR_START -->";
|
||||
export const ROOT_PAIR_END_MARKER = "<!-- PI_MAP_ROOT_PAIR_END -->";
|
||||
export const TRUST_BOUNDARY_TEXT =
|
||||
"Trust boundary: index routes, map orients, source decides.";
|
||||
|
||||
/**
|
||||
* Build a canonical root-pair block wrapping index and map content.
|
||||
*/
|
||||
export function buildRootPairBlock(
|
||||
indexContent: string,
|
||||
mapContent: string,
|
||||
): string {
|
||||
return [
|
||||
ROOT_PAIR_START_MARKER,
|
||||
"## Project Map Protocol",
|
||||
"",
|
||||
"1. Read this protocol and the root `.pi-map.index.md` first.",
|
||||
"",
|
||||
TRUST_BOUNDARY_TEXT,
|
||||
"",
|
||||
"### Root index",
|
||||
indexContent,
|
||||
"",
|
||||
"### Root map",
|
||||
mapContent,
|
||||
ROOT_PAIR_END_MARKER,
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether a message content string contains the canonical root-pair marker.
|
||||
*/
|
||||
export function hasRootPairMarker(content: string): boolean {
|
||||
return content.includes(ROOT_PAIR_START_MARKER);
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a lightweight user-visible pre-init startup hint.
|
||||
* No synthetic artifact content is injected — only a prompt to run init.
|
||||
*/
|
||||
export function buildPreInitHint(): string {
|
||||
return [
|
||||
"📋 Project maps not initialized.",
|
||||
"",
|
||||
"The project-map extension is active. Run `project_map_init` to generate paired `.pi-map.md` and `.pi-map.index.md` artifacts for this project.",
|
||||
"After init, the root pair will be preloaded automatically (default mode: strong). Source remains the final authority before edits.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute the effective injection budget in tokens.
|
||||
* Uses the smaller of (percent of context window) and absolute cap.
|
||||
* Falls back to absolute cap if context window is unknown.
|
||||
*/
|
||||
export function computeInjectionBudget(
|
||||
config: Pick<SkillConfig, "contextBudgetPercent" | "contextBudgetMaxTokens">,
|
||||
contextWindow?: number,
|
||||
): number {
|
||||
const absolute = config.contextBudgetMaxTokens;
|
||||
if (contextWindow === undefined || contextWindow <= 0) {
|
||||
return absolute;
|
||||
}
|
||||
const relative = Math.floor(
|
||||
(contextWindow * config.contextBudgetPercent) / 100,
|
||||
);
|
||||
return Math.min(relative, absolute);
|
||||
}
|
||||
|
||||
/**
|
||||
* Determine whether the active mode permits pre-init hints.
|
||||
*/
|
||||
export function modeAllowsPreInitHint(mode: PromptInjectionMode): boolean {
|
||||
return mode !== "off";
|
||||
}
|
||||
|
||||
/**
|
||||
* Determine whether the active mode permits automatic artifact injection after init.
|
||||
*/
|
||||
export function modeAllowsInjection(mode: PromptInjectionMode): boolean {
|
||||
return mode === "strong" || mode === "strict";
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve whether a given mode requires the protocol path for sensitive actions.
|
||||
*/
|
||||
export function modeRequiresProtocolPath(mode: PromptInjectionMode): boolean {
|
||||
return mode === "strict";
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Protocol-path detection (Slice 4)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Inline bypass marker prefix. Agents may include `[PI_MAP_BYPASS: reason]` in
|
||||
* a message to proceed past a strict-mode guard.
|
||||
*/
|
||||
export const BYPASS_MARKER_PREFIX = "[PI_MAP_BYPASS:";
|
||||
|
||||
function extractTextFromPayload(payload: unknown): string {
|
||||
if (!payload) return "";
|
||||
if (typeof payload === "string") return payload;
|
||||
try {
|
||||
return JSON.stringify(payload);
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether the outgoing context contains the full protocol path:
|
||||
* canonical root-pair marker + trust-boundary instruction.
|
||||
*/
|
||||
export function hasProtocolPath(
|
||||
messages?: Array<{ content?: unknown; text?: string }>,
|
||||
payload?: unknown,
|
||||
): boolean {
|
||||
const sources: string[] = [];
|
||||
if (messages) {
|
||||
for (const m of messages) {
|
||||
sources.push(extractTextFromMessage(m));
|
||||
}
|
||||
}
|
||||
const payloadText = extractTextFromPayload(payload);
|
||||
if (payloadText) {
|
||||
sources.push(payloadText);
|
||||
}
|
||||
const combined = sources.join("\n");
|
||||
return (
|
||||
combined.includes(ROOT_PAIR_START_MARKER) &&
|
||||
combined.includes(TRUST_BOUNDARY_TEXT)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Determine whether the current turn is a sensitive action.
|
||||
*
|
||||
* Uses explicit event types when available, with heuristic fallback from
|
||||
* message content and provider payload for runtimes that do not emit
|
||||
* `edit_intent` or `architecture_sensitive` event types.
|
||||
*/
|
||||
export function isSensitiveAction(
|
||||
eventType?: string,
|
||||
messages?: Array<{ content?: unknown; text?: string }>,
|
||||
payload?: unknown,
|
||||
): boolean {
|
||||
const explicit: RelevantTurnType[] = [
|
||||
"edit_intent",
|
||||
"architecture_sensitive",
|
||||
];
|
||||
if (eventType && explicit.includes(eventType as RelevantTurnType)) {
|
||||
return true;
|
||||
}
|
||||
const messageText =
|
||||
messages && messages.length > 0
|
||||
? messages.map(extractTextFromMessage).join("\n")
|
||||
: "";
|
||||
const payloadText = extractTextFromPayload(payload);
|
||||
const combined =
|
||||
messageText || payloadText ? `${messageText}\n${payloadText}`.trim() : "";
|
||||
if (!combined) return false;
|
||||
return (
|
||||
detectEditIntentFromText(combined) ||
|
||||
detectArchitectureSensitiveReasoningFromText(combined)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether any message contains a valid explicit bypass marker.
|
||||
* Empty or whitespace-only reasons are rejected.
|
||||
*/
|
||||
export function messagesHaveBypass(
|
||||
messages?: Array<{ content?: unknown; text?: string }>,
|
||||
payload?: unknown,
|
||||
): boolean {
|
||||
return extractBypassReason(messages, payload) !== undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract the reason from the first `[PI_MAP_BYPASS: reason]` marker, if any.
|
||||
* Returns `undefined` when the marker is absent or its reason is empty/whitespace.
|
||||
*/
|
||||
export function extractBypassReason(
|
||||
messages?: Array<{ content?: unknown; text?: string }>,
|
||||
payload?: unknown,
|
||||
): string | undefined {
|
||||
const parts: string[] = [];
|
||||
if (messages) {
|
||||
parts.push(messages.map(extractTextFromMessage).join("\n"));
|
||||
}
|
||||
const payloadText = extractTextFromPayload(payload);
|
||||
if (payloadText) {
|
||||
parts.push(payloadText);
|
||||
}
|
||||
const combined = parts.join("\n");
|
||||
if (!combined) return undefined;
|
||||
const escapedPrefix = BYPASS_MARKER_PREFIX.replace(
|
||||
/[.*+?^${}()|[\]\\]/g,
|
||||
"\\$&",
|
||||
);
|
||||
const match = new RegExp(`${escapedPrefix}\\s*([^\\]]+)\\]`).exec(combined);
|
||||
if (!match) return undefined;
|
||||
const reason = match[1].trim();
|
||||
return reason.length > 0 ? reason : undefined;
|
||||
}
|
||||
|
||||
export interface StrictBypassDecision {
|
||||
guard: boolean;
|
||||
reason?: string;
|
||||
bypassMarker?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Evaluate whether strict mode should block a sensitive turn with a visible
|
||||
* bypass guard because the protocol path is missing.
|
||||
*
|
||||
* Returns `guard: false` for non-strict modes, non-sensitive turns, turns
|
||||
* where the protocol path is present, or turns that include an explicit bypass
|
||||
* marker.
|
||||
*/
|
||||
export function evaluateStrictBypass(
|
||||
event:
|
||||
| {
|
||||
messages?: Array<{ content?: unknown; text?: string }>;
|
||||
type?: string;
|
||||
payload?: unknown;
|
||||
}
|
||||
| undefined
|
||||
| null,
|
||||
mode: PromptInjectionMode,
|
||||
): StrictBypassDecision {
|
||||
if (mode !== "strict") {
|
||||
return { guard: false };
|
||||
}
|
||||
if (!event) {
|
||||
return { guard: false };
|
||||
}
|
||||
if (!isSensitiveAction(event.type, event.messages, event.payload)) {
|
||||
return { guard: false };
|
||||
}
|
||||
const bypassReason = extractBypassReason(event.messages, event.payload);
|
||||
if (bypassReason !== undefined) {
|
||||
return { guard: false, bypassMarker: bypassReason };
|
||||
}
|
||||
if (hasProtocolPath(event.messages, event.payload)) {
|
||||
return { guard: false };
|
||||
}
|
||||
return {
|
||||
guard: true,
|
||||
reason: "protocol path missing for sensitive action",
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a lightweight user-visible reminder for advisory mode after init.
|
||||
* No root-pair content is injected automatically.
|
||||
*/
|
||||
export function buildAdvisoryReminder(): string {
|
||||
return [
|
||||
"📋 Project map advisory mode active.",
|
||||
"",
|
||||
"Root `.pi-map.index.md` and `.pi-map.md` are available but are not automatically injected. Read them manually when you need routing or orientation context, and remember that source remains the final authority before edits.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a visible strict-mode bypass guard for sensitive turns where the
|
||||
* protocol path is missing.
|
||||
*/
|
||||
export function buildStrictBypassGuard(reason?: string): string {
|
||||
return [
|
||||
"🛑 Strict project-map guard",
|
||||
"",
|
||||
reason ||
|
||||
"A sensitive action was detected without the project-map protocol path.",
|
||||
"",
|
||||
"The protocol path requires the root `.pi-map.index.md` / `.pi-map.md` pair plus the trust boundary (`index routes, map orients, source decides`) to be present in context.",
|
||||
"",
|
||||
"To proceed, either restore the project-map context or include an explicit bypass marker: `[PI_MAP_BYPASS: <brief justification>]`.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Context-window discovery (Slice 2)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const KNOWN_MODEL_CONTEXT_WINDOWS: Record<string, number> = {
|
||||
"gpt-4o": 128_000,
|
||||
"gpt-4o-mini": 128_000,
|
||||
"gpt-4-turbo": 128_000,
|
||||
"gpt-4": 8_192,
|
||||
"claude-3-5-sonnet": 200_000,
|
||||
"claude-3-opus": 200_000,
|
||||
"kimi-for-coding": 200_000,
|
||||
k2p6: 1_000_000,
|
||||
"kimi-k2-thinking": 256_000,
|
||||
};
|
||||
|
||||
/**
|
||||
* Discover the active model's context-window size from Pi runtime metadata.
|
||||
* Returns `undefined` when unavailable so callers fall back to the absolute cap.
|
||||
*/
|
||||
export function discoverContextWindow(ctx: any): number | undefined {
|
||||
const model = ctx?.model;
|
||||
if (model) {
|
||||
if (typeof model.contextWindow === "number" && model.contextWindow > 0) {
|
||||
return model.contextWindow;
|
||||
}
|
||||
if (
|
||||
typeof model.maxContextTokens === "number" &&
|
||||
model.maxContextTokens > 0
|
||||
) {
|
||||
return model.maxContextTokens;
|
||||
}
|
||||
if (typeof model.id === "string") {
|
||||
const known = KNOWN_MODEL_CONTEXT_WINDOWS[model.id];
|
||||
if (known) return known;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Token estimation (best-effort, deterministic)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Rough token estimate from character count.
|
||||
* 1 token ≈ 4 chars for English/prose is a conservative heuristic.
|
||||
*/
|
||||
export function estimateTokens(text: string): number {
|
||||
return Math.ceil(text.length / 4);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Artifact-pair discovery
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ArtifactPair {
|
||||
dir: string;
|
||||
mapPath: string;
|
||||
indexPath: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find every directory under `cwd` that contains both `.pi-map.md` and
|
||||
* `.pi-map.index.md`. Returns shallowest directories first for predictable
|
||||
* structural coverage.
|
||||
*/
|
||||
export function findAllArtifactPairs(cwd: string): ArtifactPair[] {
|
||||
const results: ArtifactPair[] = [];
|
||||
|
||||
function walk(dir: string) {
|
||||
let entries: import("fs").Dirent[];
|
||||
try {
|
||||
entries = readdirSync(dir, { withFileTypes: true });
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (
|
||||
entry.isDirectory() &&
|
||||
!entry.name.startsWith(".") &&
|
||||
entry.name !== "node_modules"
|
||||
) {
|
||||
walk(join(dir, entry.name));
|
||||
}
|
||||
}
|
||||
const mapPath = join(dir, ".pi-map.md");
|
||||
const indexPath = join(dir, ".pi-map.index.md");
|
||||
try {
|
||||
statSync(mapPath);
|
||||
statSync(indexPath);
|
||||
results.push({
|
||||
dir: relative(cwd, dir) || ".",
|
||||
mapPath: relative(cwd, mapPath),
|
||||
indexPath: relative(cwd, indexPath),
|
||||
});
|
||||
} catch {
|
||||
// skip dirs without the paired artifacts
|
||||
}
|
||||
}
|
||||
|
||||
walk(cwd);
|
||||
|
||||
// shallow-first: sort by path depth then alphabetically
|
||||
results.sort((a, b) => {
|
||||
const depthA = a.dir.split(/[/\\]/).length;
|
||||
const depthB = b.dir.split(/[/\\]/).length;
|
||||
if (depthA !== depthB) return depthA - depthB;
|
||||
return a.dir.localeCompare(b.dir);
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Budgeted expansion
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Build the full injection payload for a project that already has artifacts.
|
||||
*
|
||||
* Guarantees:
|
||||
* 1. Root pair is always present first.
|
||||
* 2. Additional pairs are appended shallow-first while the budget allows.
|
||||
* 3. A brief maintenance reminder is prepended.
|
||||
*/
|
||||
export function buildInjectionPayload(
|
||||
cwd: string,
|
||||
config: Pick<SkillConfig, "contextBudgetPercent" | "contextBudgetMaxTokens">,
|
||||
contextWindow: number | undefined,
|
||||
): { content: string; display: false } {
|
||||
const budget = computeInjectionBudget(config, contextWindow);
|
||||
|
||||
const pairs = findAllArtifactPairs(cwd);
|
||||
const rootIndex = pairs.findIndex((p) => p.dir === ".");
|
||||
let rootPair: ArtifactPair | undefined;
|
||||
if (rootIndex >= 0) {
|
||||
rootPair = pairs.splice(rootIndex, 1)[0];
|
||||
}
|
||||
|
||||
let usedTokens = 0;
|
||||
const parts: string[] = [];
|
||||
|
||||
// Maintenance reminder (lightweight)
|
||||
const reminder =
|
||||
"📋 Project map active: Start with the root `.pi-map.index.md`, use indexes first for routing, read the local `.pi-map.md` plus source before edits, run `project_map_patch` after source edits, and run `project_map_validate` before freshness-sensitive architectural handoff. Trust boundary: index routes, map orients, source decides.";
|
||||
parts.push(reminder);
|
||||
usedTokens += estimateTokens(reminder);
|
||||
|
||||
// Root pair (guaranteed)
|
||||
if (rootPair) {
|
||||
const indexContent = readFileSync(join(cwd, rootPair.indexPath), "utf8");
|
||||
const mapContent = readFileSync(join(cwd, rootPair.mapPath), "utf8");
|
||||
const block = buildRootPairBlock(indexContent, mapContent);
|
||||
parts.push(block);
|
||||
usedTokens += estimateTokens(block);
|
||||
}
|
||||
|
||||
// Budgeted expansion (shallow-first, deterministic)
|
||||
for (const pair of pairs) {
|
||||
const indexContent = readFileSync(join(cwd, pair.indexPath), "utf8");
|
||||
const mapContent = readFileSync(join(cwd, pair.mapPath), "utf8");
|
||||
const pairTokens =
|
||||
estimateTokens(indexContent) + estimateTokens(mapContent);
|
||||
if (usedTokens + pairTokens > budget) {
|
||||
break;
|
||||
}
|
||||
parts.push(
|
||||
`\n## ${pair.dir}\n\n### Index\n${indexContent}\n\n### Map\n${mapContent}`,
|
||||
);
|
||||
usedTokens += pairTokens;
|
||||
}
|
||||
|
||||
return { content: parts.join("\n\n"), display: false };
|
||||
}
|
||||
+296
@@ -0,0 +1,296 @@
|
||||
import { readdirSync, readFileSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { parseDirectoryMap, parseDirectoryIndex } from "./format.js";
|
||||
import type { DirectoryArtifactModel } from "./directory-model.js";
|
||||
|
||||
const TOP_K = 3;
|
||||
|
||||
export interface RetrieveContextOptions {
|
||||
topK?: number;
|
||||
}
|
||||
|
||||
export function retrieveContext(
|
||||
query: string,
|
||||
rootPath: string,
|
||||
opts: RetrieveContextOptions = {},
|
||||
): string {
|
||||
const topK = opts.topK ?? TOP_K;
|
||||
const candidates = collectCandidates(rootPath);
|
||||
const queryTerms = normalizeTerms(query);
|
||||
|
||||
const scored = candidates
|
||||
.map((model) => ({
|
||||
model,
|
||||
score: scoreDirectory(model, queryTerms),
|
||||
}))
|
||||
.filter((c) => c.score > 0)
|
||||
.sort((a, b) => b.score - a.score)
|
||||
.slice(0, topK);
|
||||
|
||||
if (scored.length === 0) {
|
||||
return renderNoResultsBundle(query);
|
||||
}
|
||||
|
||||
return renderContextBundle(
|
||||
query,
|
||||
scored.map((s) => s.model),
|
||||
);
|
||||
}
|
||||
|
||||
function collectCandidates(rootPath: string): DirectoryArtifactModel[] {
|
||||
const results: DirectoryArtifactModel[] = [];
|
||||
function walk(dir: string) {
|
||||
let entries: import("fs").Dirent[];
|
||||
try {
|
||||
entries = readdirSync(dir, { withFileTypes: true });
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
|
||||
const hasIndex = entries.some(
|
||||
(e) => e.isFile() && e.name === ".pi-map.index.md",
|
||||
);
|
||||
const hasMap = entries.some((e) => e.isFile() && e.name === ".pi-map.md");
|
||||
|
||||
if (hasIndex || hasMap) {
|
||||
// Prefer index first, fall back to map, merge if both exist
|
||||
let model: DirectoryArtifactModel | null = null;
|
||||
|
||||
if (hasIndex) {
|
||||
try {
|
||||
const text = readFileSync(join(dir, ".pi-map.index.md"), "utf8");
|
||||
model = parseDirectoryIndex(text);
|
||||
} catch {
|
||||
// ignore parse error
|
||||
}
|
||||
}
|
||||
|
||||
if (hasMap) {
|
||||
try {
|
||||
const text = readFileSync(join(dir, ".pi-map.md"), "utf8");
|
||||
const mapModel = parseDirectoryMap(text);
|
||||
if (model) {
|
||||
// Merge: index provides routing, map provides richer metadata
|
||||
mergeModels(model, mapModel);
|
||||
} else {
|
||||
model = mapModel;
|
||||
}
|
||||
} catch {
|
||||
// ignore parse error
|
||||
}
|
||||
}
|
||||
|
||||
if (model) {
|
||||
results.push(model);
|
||||
}
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
if (
|
||||
entry.isDirectory() &&
|
||||
!entry.name.startsWith(".") &&
|
||||
entry.name !== "node_modules"
|
||||
) {
|
||||
walk(join(dir, entry.name));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
walk(rootPath);
|
||||
return results;
|
||||
}
|
||||
|
||||
function mergeModels(
|
||||
index: DirectoryArtifactModel,
|
||||
map: DirectoryArtifactModel,
|
||||
): void {
|
||||
// Use map's richer fields when index lacks them
|
||||
if (!index.arch && map.arch) index.arch = map.arch;
|
||||
if (index.tags.length === 0 && map.tags.length > 0) index.tags = map.tags;
|
||||
if (index.symbols.length === 0 && map.symbols.length > 0)
|
||||
index.symbols = map.symbols;
|
||||
if (index.files.length === 0 && map.files.length > 0) index.files = map.files;
|
||||
// Merge workflows, preferring index's workflow list but keeping map's extras
|
||||
const indexTasks = new Set(index.workflows.map((w) => w.task));
|
||||
for (const wf of map.workflows) {
|
||||
if (!indexTasks.has(wf.task)) {
|
||||
index.workflows.push(wf);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeTerms(query: string): string[] {
|
||||
const terms = new Set<string>();
|
||||
for (const word of query.toLowerCase().split(/[^a-z0-9]+/)) {
|
||||
if (word.length > 1) {
|
||||
terms.add(word);
|
||||
for (const sub of splitCamelCase(word)) {
|
||||
if (sub.length > 1) terms.add(sub);
|
||||
}
|
||||
}
|
||||
}
|
||||
return [...terms];
|
||||
}
|
||||
|
||||
function splitCamelCase(str: string): string[] {
|
||||
return str
|
||||
.replace(/([a-z0-9])([A-Z])/g, "$1 $2")
|
||||
.replace(/[_-]+/g, " ")
|
||||
.toLowerCase()
|
||||
.split(/\s+/)
|
||||
.filter((w) => w.length > 1);
|
||||
}
|
||||
|
||||
function scoreDirectory(
|
||||
model: DirectoryArtifactModel,
|
||||
queryTerms: string[],
|
||||
): number {
|
||||
let score = 0;
|
||||
|
||||
// Role matches
|
||||
const roleWords = extractWords(model.role);
|
||||
for (const term of queryTerms) {
|
||||
if (roleWords.includes(term)) score += 3;
|
||||
}
|
||||
|
||||
// Tag matches
|
||||
for (const tag of model.tags) {
|
||||
for (const term of queryTerms) {
|
||||
if (tag.toLowerCase().includes(term)) score += 2;
|
||||
}
|
||||
}
|
||||
|
||||
// Symbol matches
|
||||
for (const sym of model.symbols) {
|
||||
for (const term of queryTerms) {
|
||||
if (sym.toLowerCase().includes(term)) score += 2;
|
||||
}
|
||||
}
|
||||
|
||||
// Workflow task matches
|
||||
for (const wf of model.workflows) {
|
||||
for (const term of queryTerms) {
|
||||
if (wf.task.toLowerCase().includes(term)) score += 2;
|
||||
}
|
||||
}
|
||||
|
||||
// File name and purpose matches
|
||||
for (const file of model.files) {
|
||||
const fileWords = extractWords(`${file.name} ${file.purpose}`);
|
||||
const camelWords = splitCamelCase(file.name);
|
||||
for (const term of queryTerms) {
|
||||
if (fileWords.includes(term) || camelWords.includes(term)) score += 1;
|
||||
}
|
||||
}
|
||||
|
||||
// Directory path matches
|
||||
const pathWords = extractWords(model.dir);
|
||||
for (const term of queryTerms) {
|
||||
if (pathWords.includes(term)) score += 1;
|
||||
}
|
||||
|
||||
return score;
|
||||
}
|
||||
|
||||
function extractWords(text: string): string[] {
|
||||
return text
|
||||
.toLowerCase()
|
||||
.split(/[^a-z0-9]+/)
|
||||
.filter((w) => w.length > 1);
|
||||
}
|
||||
|
||||
function renderContextBundle(
|
||||
query: string,
|
||||
candidates: DirectoryArtifactModel[],
|
||||
): string {
|
||||
const lines: string[] = [];
|
||||
lines.push(`# Context bundle: ${query}`);
|
||||
lines.push("");
|
||||
|
||||
lines.push("## query");
|
||||
lines.push(query);
|
||||
lines.push("");
|
||||
|
||||
lines.push("## relevant indexes");
|
||||
for (const c of candidates) {
|
||||
lines.push(`- ${c.dir}/.pi-map.index.md`);
|
||||
}
|
||||
lines.push("");
|
||||
|
||||
lines.push("## relevant maps");
|
||||
for (const c of candidates) {
|
||||
lines.push(`- ${c.dir}/.pi-map.md`);
|
||||
}
|
||||
lines.push("");
|
||||
|
||||
lines.push("## likely files");
|
||||
const seenFiles = new Set<string>();
|
||||
for (const c of candidates) {
|
||||
for (const f of c.files) {
|
||||
const path = `${c.dir}/${f.name}`;
|
||||
if (!seenFiles.has(path)) {
|
||||
seenFiles.add(path);
|
||||
lines.push(`- ${path}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (seenFiles.size === 0) {
|
||||
lines.push("-");
|
||||
}
|
||||
lines.push("");
|
||||
|
||||
// Symbols: only include if at least one candidate has scored symbols above threshold
|
||||
const relevantSymbols = collectRelevantSymbols(candidates);
|
||||
if (relevantSymbols.length > 0) {
|
||||
lines.push("## relevant symbols");
|
||||
for (const sym of relevantSymbols) {
|
||||
lines.push(`- ${sym}`);
|
||||
}
|
||||
lines.push("");
|
||||
}
|
||||
|
||||
lines.push("## instructions");
|
||||
lines.push(
|
||||
"Read the indexes first, then the strongest-match rich maps, then verify behavior from source before editing.",
|
||||
);
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function collectRelevantSymbols(
|
||||
candidates: DirectoryArtifactModel[],
|
||||
): string[] {
|
||||
const seen = new Set<string>();
|
||||
const result: string[] = [];
|
||||
for (const c of candidates) {
|
||||
for (const sym of c.symbols) {
|
||||
if (!seen.has(sym)) {
|
||||
seen.add(sym);
|
||||
result.push(sym);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Cap at a reasonable number to keep output compact
|
||||
return result.slice(0, 12);
|
||||
}
|
||||
|
||||
function renderNoResultsBundle(query: string): string {
|
||||
return [
|
||||
`# Context bundle: ${query}`,
|
||||
"",
|
||||
"## query",
|
||||
query,
|
||||
"",
|
||||
"## relevant indexes",
|
||||
"-",
|
||||
"",
|
||||
"## relevant maps",
|
||||
"-",
|
||||
"",
|
||||
"## likely files",
|
||||
"-",
|
||||
"",
|
||||
"## instructions",
|
||||
"No relevant directories found. Try rephrasing the query or run `project_map_reinit` if artifacts are stale.",
|
||||
].join("\n");
|
||||
}
|
||||
@@ -0,0 +1,265 @@
|
||||
import type {
|
||||
DirectoryArtifactModel,
|
||||
FileEntry,
|
||||
RoutingMetadataOptions,
|
||||
WorkflowHint,
|
||||
} from "./directory-model.js";
|
||||
|
||||
const DEFAULT_TAG_CAP = 8;
|
||||
const DEFAULT_WORKFLOW_HINT_CAP = 5;
|
||||
|
||||
const STOP_WORDS = new Set([
|
||||
"the",
|
||||
"and",
|
||||
"for",
|
||||
"with",
|
||||
"from",
|
||||
"into",
|
||||
"onto",
|
||||
"this",
|
||||
"that",
|
||||
"then",
|
||||
"than",
|
||||
"when",
|
||||
"where",
|
||||
"what",
|
||||
"how",
|
||||
"why",
|
||||
"who",
|
||||
"which",
|
||||
"while",
|
||||
"during",
|
||||
"before",
|
||||
"after",
|
||||
"above",
|
||||
"below",
|
||||
"between",
|
||||
"among",
|
||||
"through",
|
||||
"over",
|
||||
"under",
|
||||
"again",
|
||||
"further",
|
||||
"once",
|
||||
"here",
|
||||
"there",
|
||||
"all",
|
||||
"any",
|
||||
"both",
|
||||
"each",
|
||||
"few",
|
||||
"more",
|
||||
"most",
|
||||
"other",
|
||||
"some",
|
||||
"such",
|
||||
"only",
|
||||
"own",
|
||||
"same",
|
||||
"so",
|
||||
"too",
|
||||
"very",
|
||||
"can",
|
||||
"will",
|
||||
"just",
|
||||
"should",
|
||||
"now",
|
||||
"use",
|
||||
"using",
|
||||
"used",
|
||||
"via",
|
||||
"based",
|
||||
"build",
|
||||
"built",
|
||||
"used",
|
||||
"file",
|
||||
"files",
|
||||
"module",
|
||||
"modules",
|
||||
"function",
|
||||
"functions",
|
||||
"class",
|
||||
"classes",
|
||||
"export",
|
||||
"exports",
|
||||
"import",
|
||||
"imports",
|
||||
]);
|
||||
|
||||
export function populateRoutingMetadata(
|
||||
model: DirectoryArtifactModel,
|
||||
opts: RoutingMetadataOptions = {},
|
||||
): void {
|
||||
const tagCap = opts.tagCap ?? DEFAULT_TAG_CAP;
|
||||
const workflowHintCap = opts.workflowHintCap ?? DEFAULT_WORKFLOW_HINT_CAP;
|
||||
|
||||
model.tags = generateTags(model.files, tagCap);
|
||||
model.symbols = generateSymbols(model.files, tagCap);
|
||||
model.workflows = generateWorkflows(model, workflowHintCap);
|
||||
}
|
||||
|
||||
function generateTags(files: FileEntry[], cap: number): string[] {
|
||||
const scores = new Map<string, number>();
|
||||
|
||||
for (const file of files) {
|
||||
// Score words from file purpose
|
||||
for (const word of extractWords(file.purpose)) {
|
||||
scores.set(word, (scores.get(word) ?? 0) + 1);
|
||||
}
|
||||
|
||||
// Score words from export names (camelCase split)
|
||||
for (const exp of file.exports) {
|
||||
const cleanExp = exp.replace(/^(class|func|method):/, "").split("(")[0];
|
||||
for (const word of splitCamelCase(cleanExp)) {
|
||||
if (word.length > 1) {
|
||||
scores.set(word, (scores.get(word) ?? 0) + 2);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Score dep path segments
|
||||
for (const dep of file.deps) {
|
||||
for (const segment of dep.split(/[/\-.]/)) {
|
||||
const word = segment.toLowerCase();
|
||||
if (word.length > 1 && !STOP_WORDS.has(word)) {
|
||||
scores.set(word, (scores.get(word) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Score from file name (stem, no extension)
|
||||
const stem = file.name.replace(/\.[^.]+$/, "");
|
||||
for (const word of splitCamelCase(stem)) {
|
||||
if (word.length > 1) {
|
||||
scores.set(word, (scores.get(word) ?? 0) + 2);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return Array.from(scores.entries())
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.map(([word]) => word)
|
||||
.slice(0, cap);
|
||||
}
|
||||
|
||||
function generateSymbols(files: FileEntry[], cap: number): string[] {
|
||||
const scores = new Map<string, number>();
|
||||
const seen = new Set<string>();
|
||||
|
||||
for (const file of files) {
|
||||
for (const exp of file.exports) {
|
||||
// Prefer clean identifiers over encoded DSL entries when possible
|
||||
let symbol = exp;
|
||||
let score = 1;
|
||||
|
||||
if (exp.startsWith("class:")) {
|
||||
symbol = exp.slice(6).split(" ")[0];
|
||||
score = 3;
|
||||
} else if (exp.startsWith("func:")) {
|
||||
symbol = exp.slice(5).split("(")[0];
|
||||
score = 2;
|
||||
} else if (exp.startsWith("method:")) {
|
||||
symbol = exp.slice(7).split("(")[0];
|
||||
score = 2;
|
||||
}
|
||||
|
||||
if (!seen.has(symbol)) {
|
||||
seen.add(symbol);
|
||||
scores.set(symbol, (scores.get(symbol) ?? 0) + score);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return Array.from(scores.entries())
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.map(([name]) => name)
|
||||
.slice(0, cap);
|
||||
}
|
||||
|
||||
function generateWorkflows(
|
||||
model: DirectoryArtifactModel,
|
||||
cap: number,
|
||||
): WorkflowHint[] {
|
||||
const workflows: WorkflowHint[] = [];
|
||||
const dirName = model.dir.split("/").pop() || model.dir;
|
||||
const baseName = dirName === "." ? "project" : dirName;
|
||||
|
||||
// Only generate workflows when we have reasonable structural confidence
|
||||
const hasSourceFiles = model.files.some((f) =>
|
||||
/\.(ts|tsx|js|jsx|py|go|rs|java)$/.test(f.name),
|
||||
);
|
||||
if (!hasSourceFiles || model.files.length === 0) {
|
||||
return workflows;
|
||||
}
|
||||
|
||||
const sourceFiles = model.files.filter(
|
||||
(f) =>
|
||||
!/\.(test|spec)\./.test(f.name) && !/\.(md|json|yaml|yml)$/.test(f.name),
|
||||
);
|
||||
const testFiles = model.files.filter((f) => /\.(test|spec)\./.test(f.name));
|
||||
const configFiles = model.files.filter(
|
||||
(f) =>
|
||||
f.name.includes("config") ||
|
||||
/\.(json|yaml|yml|toml)$/.test(f.name) ||
|
||||
f.name === ".env",
|
||||
);
|
||||
const cliFiles = model.files.filter(
|
||||
(f) => f.name.includes("cli") || f.name.includes("command"),
|
||||
);
|
||||
|
||||
if (sourceFiles.length > 0) {
|
||||
workflows.push({
|
||||
task: `change ${baseName} behavior`,
|
||||
read: sourceFiles.slice(0, 3).map((f) => f.name),
|
||||
});
|
||||
}
|
||||
|
||||
if (testFiles.length > 0) {
|
||||
workflows.push({
|
||||
task: `update ${baseName} tests`,
|
||||
read: testFiles.slice(0, 3).map((f) => f.name),
|
||||
});
|
||||
}
|
||||
|
||||
if (cliFiles.length > 0) {
|
||||
workflows.push({
|
||||
task: `change ${baseName} CLI`,
|
||||
read: cliFiles.slice(0, 3).map((f) => f.name),
|
||||
});
|
||||
}
|
||||
|
||||
if (configFiles.length > 0) {
|
||||
workflows.push({
|
||||
task: `change ${baseName} config`,
|
||||
read: configFiles.slice(0, 3).map((f) => f.name),
|
||||
});
|
||||
}
|
||||
|
||||
// Add a directory-navigation workflow for non-leaf directories
|
||||
if (model.children.length > 0) {
|
||||
workflows.push({
|
||||
task: `explore ${baseName} subdirectories`,
|
||||
index: model.children
|
||||
.slice(0, 3)
|
||||
.map((child) => `${child}/.pi-map.index.md`),
|
||||
});
|
||||
}
|
||||
|
||||
return workflows.slice(0, cap);
|
||||
}
|
||||
|
||||
function extractWords(text: string): string[] {
|
||||
return text
|
||||
.toLowerCase()
|
||||
.split(/[^a-z0-9]+/)
|
||||
.filter((w) => w.length > 2 && !STOP_WORDS.has(w));
|
||||
}
|
||||
|
||||
function splitCamelCase(str: string): string[] {
|
||||
return str
|
||||
.replace(/([a-z0-9])([A-Z])/g, "$1 $2")
|
||||
.replace(/[_-]+/g, " ")
|
||||
.toLowerCase()
|
||||
.split(/\s+/)
|
||||
.filter((w) => w.length > 1 && !STOP_WORDS.has(w));
|
||||
}
|
||||
+390
-77
@@ -1,9 +1,17 @@
|
||||
import { discoverProject } from "./discover.js";
|
||||
import { parsePackageMap } from "./format.js";
|
||||
import { existsSync, readFileSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { extractFileAST } from "./ast-extract.js";
|
||||
import { generateDirectoryMap } from "./init.js";
|
||||
import { join, resolve } from "path";
|
||||
import { discoverProject } from "./discover.js";
|
||||
import type { DirectoryEntry } from "./discover.js";
|
||||
import { parseDirectoryIndex, parseDirectoryMap } from "./format.js";
|
||||
import { extractFileAST } from "./ast/ast-extract.js";
|
||||
import {
|
||||
generateDirectoryArtifacts,
|
||||
buildDirectoryContext,
|
||||
type ArtifactWriteMode,
|
||||
} from "./init.js";
|
||||
import type { LLMClient } from "./llm/llm-client.js";
|
||||
import { loadConfig } from "./config.js";
|
||||
import type { PatchMode } from "./patch.js";
|
||||
|
||||
export interface ValidationResult {
|
||||
clean: boolean;
|
||||
@@ -11,118 +19,271 @@ export interface ValidationResult {
|
||||
fixed?: number;
|
||||
}
|
||||
|
||||
export interface ValidationOptions {
|
||||
fix?: boolean;
|
||||
verbose?: boolean;
|
||||
llmClient?: LLMClient;
|
||||
cacheDir?: string;
|
||||
patchMode?: PatchMode;
|
||||
}
|
||||
|
||||
export interface Discrepancy {
|
||||
type: "missing" | "orphaned" | "stale-signature" | "dirty";
|
||||
type:
|
||||
| "missing"
|
||||
| "orphaned"
|
||||
| "stale-signature"
|
||||
| "dirty"
|
||||
| "stale-index"
|
||||
| "stale-map"
|
||||
| "broken-link"
|
||||
| "structural";
|
||||
path: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
type RepairMode = Exclude<PatchMode, "auto">;
|
||||
|
||||
export async function validateMaps(
|
||||
rootPath: string,
|
||||
options?: { fix?: boolean; verbose?: boolean },
|
||||
options: ValidationOptions = {},
|
||||
): Promise<ValidationResult> {
|
||||
const { fix = false, verbose = true } = options || {};
|
||||
const root = resolve(rootPath);
|
||||
const {
|
||||
fix = false,
|
||||
verbose = true,
|
||||
llmClient,
|
||||
cacheDir,
|
||||
patchMode = "auto",
|
||||
} = options;
|
||||
const discrepancies: Discrepancy[] = [];
|
||||
const entries = discoverProject(rootPath);
|
||||
const dirsToFix = new Set<string>();
|
||||
const entries = discoverProject(root);
|
||||
const config = loadConfig(root);
|
||||
const routingOpts = {
|
||||
tagCap: config.tagCap,
|
||||
workflowHintCap: config.workflowHintCap,
|
||||
};
|
||||
const repairModes = new Map<string, RepairMode>();
|
||||
|
||||
for (const entry of entries) {
|
||||
const ctx = buildDirectoryContext(entries, entry);
|
||||
const mapPath = join(entry.dirPath, ".pi-map.md");
|
||||
const indexPath = join(entry.dirPath, ".pi-map.index.md");
|
||||
const expectedParent = ctx.parentMap.get(entry.relativePath);
|
||||
const expectedChildren = [
|
||||
...(ctx.childrenMap.get(entry.relativePath) ?? []),
|
||||
].sort();
|
||||
const structuralFlag = (reason: string, path = entry.relativePath) => {
|
||||
discrepancies.push({ type: "structural", path, message: reason });
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
};
|
||||
|
||||
if (!existsSync(mapPath)) {
|
||||
discrepancies.push({
|
||||
type: "missing",
|
||||
path: entry.relativePath,
|
||||
message: "No .pi-map.md found",
|
||||
type: "stale-map",
|
||||
path: mapPath,
|
||||
message: "missing .pi-map.md",
|
||||
});
|
||||
if (fix) dirsToFix.add(entry.dirPath);
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
continue;
|
||||
}
|
||||
if (!existsSync(indexPath)) {
|
||||
discrepancies.push({
|
||||
type: "stale-index",
|
||||
path: indexPath,
|
||||
message: "missing .pi-map.index.md",
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
continue;
|
||||
}
|
||||
|
||||
const mapData = parsePackageMap(readFileSync(mapPath, "utf8"));
|
||||
let mapNeedsRewrite = false;
|
||||
const mapText = readFileSync(mapPath, "utf8");
|
||||
const indexText = readFileSync(indexPath, "utf8");
|
||||
const mapData = parseDirectoryMap(mapText);
|
||||
const indexData = parseDirectoryIndex(indexText);
|
||||
|
||||
// Check for dirty markers
|
||||
if (mapData.dirty && mapData.dirty !== "-") {
|
||||
discrepancies.push({
|
||||
type: "dirty",
|
||||
path: mapPath,
|
||||
message: `Dirty: ${mapData.dirty}`,
|
||||
});
|
||||
if (fix) mapNeedsRewrite = true;
|
||||
markRepairMode(repairModes, entry.dirPath, "small");
|
||||
}
|
||||
if (indexData.dirty && indexData.dirty !== "-") {
|
||||
discrepancies.push({
|
||||
type: "stale-index",
|
||||
path: indexPath,
|
||||
message: `dirty index: ${indexData.dirty}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "small");
|
||||
}
|
||||
|
||||
// Check for orphaned entries
|
||||
for (const fileEntry of mapData.files) {
|
||||
const filePath = join(entry.dirPath, fileEntry.name);
|
||||
if (!existsSync(filePath)) {
|
||||
discrepancies.push({
|
||||
type: "orphaned",
|
||||
path: filePath,
|
||||
message: `File listed but deleted: ${fileEntry.name}`,
|
||||
});
|
||||
if (fix) mapNeedsRewrite = true;
|
||||
}
|
||||
if (mapData.dir !== entry.relativePath) {
|
||||
structuralFlag(`dir mismatch in map (${mapData.dir})`, mapPath);
|
||||
}
|
||||
if (indexData.dir !== entry.relativePath) {
|
||||
structuralFlag(`dir mismatch in index (${indexData.dir})`, indexPath);
|
||||
}
|
||||
|
||||
// Check for new files not in map
|
||||
const expectedIndexLink = `index: ${entry.relativePath}/.pi-map.index.md`;
|
||||
const expectedMapLink = `map: ${entry.relativePath}/.pi-map.md`;
|
||||
if (!mapText.includes(expectedIndexLink)) {
|
||||
discrepancies.push({
|
||||
type: "broken-link",
|
||||
path: mapPath,
|
||||
message: `missing sibling index link (${expectedIndexLink})`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
if (!indexText.includes(expectedMapLink)) {
|
||||
discrepancies.push({
|
||||
type: "broken-link",
|
||||
path: indexPath,
|
||||
message: `missing sibling map link (${expectedMapLink})`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
|
||||
const mapFiles = new Set(mapData.files.map((file) => file.name));
|
||||
const indexFiles = new Set(indexData.files.map((file) => file.name));
|
||||
const actualFiles = new Set(entry.files);
|
||||
for (const file of entry.files) {
|
||||
if (!mapData.files.find((f) => f.name === file)) {
|
||||
if (!mapFiles.has(file)) {
|
||||
discrepancies.push({
|
||||
type: "missing",
|
||||
path: join(entry.dirPath, file),
|
||||
message: `File not in .pi-map.md: ${file}`,
|
||||
});
|
||||
if (fix) mapNeedsRewrite = true;
|
||||
markRepairMode(repairModes, entry.dirPath, "small");
|
||||
}
|
||||
if (!indexFiles.has(file)) {
|
||||
discrepancies.push({
|
||||
type: "stale-index",
|
||||
path: indexPath,
|
||||
message: `file missing from index: ${file}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "small");
|
||||
}
|
||||
}
|
||||
for (const file of mapFiles) {
|
||||
if (!actualFiles.has(file)) {
|
||||
discrepancies.push({
|
||||
type: "orphaned",
|
||||
path: join(entry.dirPath, file),
|
||||
message: `File listed but deleted: ${file}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "small");
|
||||
}
|
||||
}
|
||||
for (const file of indexFiles) {
|
||||
if (!actualFiles.has(file)) {
|
||||
discrepancies.push({
|
||||
type: "stale-index",
|
||||
path: indexPath,
|
||||
message: `orphaned file entry in index: ${file}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "small");
|
||||
}
|
||||
}
|
||||
|
||||
if ((indexData.parent ?? undefined) !== expectedParent) {
|
||||
discrepancies.push({
|
||||
type: "broken-link",
|
||||
path: indexPath,
|
||||
message: `parent mismatch: expected ${expectedParent ?? "-"}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
if (!sameStringArrays(indexData.children, expectedChildren)) {
|
||||
discrepancies.push({
|
||||
type: "broken-link",
|
||||
path: indexPath,
|
||||
message: `children mismatch: expected ${expectedChildren.join(", ") || "-"}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
|
||||
if (!sameWorkflowShapes(mapData.workflows, indexData.workflows)) {
|
||||
discrepancies.push({
|
||||
type: "structural",
|
||||
path: entry.relativePath,
|
||||
message: "map/index workflow disagreement",
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
|
||||
for (const target of collectWorkflowTargets(indexData)) {
|
||||
if (!targetExists(root, target)) {
|
||||
discrepancies.push({
|
||||
type: "broken-link",
|
||||
path: indexPath,
|
||||
message: `workflow target missing: ${target}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
}
|
||||
|
||||
// Check signatures for code files
|
||||
for (const fileEntry of mapData.files) {
|
||||
const filePath = join(entry.dirPath, fileEntry.name);
|
||||
if (!existsSync(filePath)) continue;
|
||||
|
||||
const astData = await extractFileAST(filePath);
|
||||
if (astData) {
|
||||
const listedExports = new Set(fileEntry.exports);
|
||||
const actualExports = new Set(astData.exports);
|
||||
|
||||
for (const exp of listedExports) {
|
||||
if (!actualExports.has(exp)) {
|
||||
discrepancies.push({
|
||||
type: "stale-signature",
|
||||
path: filePath,
|
||||
message: `Missing export: ${exp}`,
|
||||
});
|
||||
if (fix) mapNeedsRewrite = true;
|
||||
}
|
||||
}
|
||||
for (const exp of actualExports) {
|
||||
if (!listedExports.has(exp)) {
|
||||
discrepancies.push({
|
||||
type: "stale-signature",
|
||||
path: filePath,
|
||||
message: `New export: ${exp}`,
|
||||
});
|
||||
if (fix) mapNeedsRewrite = true;
|
||||
}
|
||||
if (!astData) continue;
|
||||
const actualExports = new Set(astData.exports);
|
||||
const listedExports = new Set(
|
||||
normalizeListedExports(fileEntry.exports, actualExports),
|
||||
);
|
||||
for (const exp of listedExports) {
|
||||
if (!actualExports.has(exp)) {
|
||||
discrepancies.push({
|
||||
type: "stale-signature",
|
||||
path: filePath,
|
||||
message: `Missing export: ${exp}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
}
|
||||
for (const exp of actualExports) {
|
||||
if (!listedExports.has(exp)) {
|
||||
discrepancies.push({
|
||||
type: "stale-signature",
|
||||
path: filePath,
|
||||
message: `New export: ${exp}`,
|
||||
});
|
||||
markRepairMode(repairModes, entry.dirPath, "structural");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (fix && mapNeedsRewrite) {
|
||||
dirsToFix.add(entry.dirPath);
|
||||
}
|
||||
}
|
||||
|
||||
// Apply fixes
|
||||
let fixed = 0;
|
||||
if (fix && dirsToFix.size > 0) {
|
||||
for (const dirPath of dirsToFix) {
|
||||
const entry = entries.find((e) => e.dirPath === dirPath);
|
||||
if (entry) {
|
||||
await generateDirectoryMap(entry);
|
||||
fixed++;
|
||||
}
|
||||
if (fix && repairModes.size > 0) {
|
||||
if (!llmClient) {
|
||||
throw new Error("validate --fix requires an LLM client");
|
||||
}
|
||||
const repairPlan = new Map<string, ArtifactWriteMode>();
|
||||
for (const [dirPath, inferredMode] of repairModes) {
|
||||
const entry = entries.find((candidate) => candidate.dirPath === dirPath);
|
||||
if (!entry) continue;
|
||||
const effectiveMode = patchMode === "auto" ? inferredMode : patchMode;
|
||||
addRepairChain(entries, entry, effectiveMode, repairPlan);
|
||||
}
|
||||
const orderedEntries = Array.from(repairPlan.entries()).sort(
|
||||
(a, b) => depthOfPath(b[0]) - depthOfPath(a[0]),
|
||||
);
|
||||
for (const [dirPath, writeMode] of orderedEntries) {
|
||||
const entry = entries.find((candidate) => candidate.dirPath === dirPath);
|
||||
if (!entry) continue;
|
||||
const ctx = buildDirectoryContext(entries, entry);
|
||||
await generateDirectoryArtifacts(
|
||||
entry,
|
||||
ctx,
|
||||
llmClient,
|
||||
cacheDir,
|
||||
undefined,
|
||||
routingOpts,
|
||||
writeMode,
|
||||
);
|
||||
fixed++;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -130,24 +291,176 @@ export async function validateMaps(
|
||||
clean: discrepancies.length === 0,
|
||||
discrepancies,
|
||||
};
|
||||
|
||||
if (fix) {
|
||||
result.fixed = fixed;
|
||||
}
|
||||
if (fix) result.fixed = fixed;
|
||||
|
||||
if (verbose) {
|
||||
if (result.clean) {
|
||||
console.log("All .pi-map.md files are clean.");
|
||||
console.log("All .pi-map.md/.pi-map.index.md files are clean.");
|
||||
} else {
|
||||
console.log(`Found ${discrepancies.length} discrepancies:`);
|
||||
for (const d of discrepancies) {
|
||||
console.log(` [${d.type}] ${d.path}: ${d.message}`);
|
||||
for (const discrepancy of discrepancies) {
|
||||
console.log(
|
||||
` [${discrepancy.type}] ${discrepancy.path}: ${discrepancy.message}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (fix && fixed > 0) {
|
||||
console.log(`Fixed ${fixed} director${fixed === 1 ? "y" : "ies"}.`);
|
||||
const fixModeLabel = patchMode === "auto" ? "auto" : patchMode;
|
||||
console.log(
|
||||
`Repaired affected chain (patchMode: ${fixModeLabel}) across ${fixed} directories.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
function sameStringArrays(a: string[], b: string[]): boolean {
|
||||
const left = [...a].sort();
|
||||
const right = [...b].sort();
|
||||
return JSON.stringify(left) === JSON.stringify(right);
|
||||
}
|
||||
|
||||
function sameWorkflowShapes(
|
||||
a: ReturnType<typeof parseDirectoryMap>["workflows"],
|
||||
b: ReturnType<typeof parseDirectoryIndex>["workflows"],
|
||||
): boolean {
|
||||
const normalize = (workflow: (typeof a)[number]) => ({
|
||||
task: workflow.task,
|
||||
read: [...(workflow.read ?? [])].sort(),
|
||||
index: [...(workflow.index ?? [])].sort(),
|
||||
map: [...(workflow.map ?? [])].sort(),
|
||||
files: [...(workflow.files ?? [])].sort(),
|
||||
});
|
||||
return JSON.stringify(a.map(normalize)) === JSON.stringify(b.map(normalize));
|
||||
}
|
||||
|
||||
function collectWorkflowTargets(
|
||||
indexData: ReturnType<typeof parseDirectoryIndex>,
|
||||
): string[] {
|
||||
const targets: string[] = [];
|
||||
for (const workflow of indexData.workflows) {
|
||||
targets.push(...(workflow.index ?? []));
|
||||
targets.push(...(workflow.map ?? []));
|
||||
targets.push(...(workflow.files ?? []));
|
||||
}
|
||||
return targets;
|
||||
}
|
||||
|
||||
function targetExists(rootPath: string, target: string): boolean {
|
||||
return existsSync(resolve(rootPath, target));
|
||||
}
|
||||
|
||||
const BUILT_IN_GLOBALS = new Set([
|
||||
"Array",
|
||||
"Date",
|
||||
"JSON",
|
||||
"Math",
|
||||
"Object",
|
||||
"Promise",
|
||||
"RegExp",
|
||||
"String",
|
||||
"Number",
|
||||
"Boolean",
|
||||
"Error",
|
||||
"Map",
|
||||
"Set",
|
||||
"Symbol",
|
||||
"WeakMap",
|
||||
"WeakSet",
|
||||
"ArrayBuffer",
|
||||
"DataView",
|
||||
"Float32Array",
|
||||
"Float64Array",
|
||||
"Int8Array",
|
||||
"Int16Array",
|
||||
"Int32Array",
|
||||
"Uint8Array",
|
||||
"Uint8ClampedArray",
|
||||
"Uint16Array",
|
||||
"Uint32Array",
|
||||
"console",
|
||||
"process",
|
||||
"Buffer",
|
||||
"undefined",
|
||||
"null",
|
||||
"Infinity",
|
||||
"NaN",
|
||||
"parseInt",
|
||||
"parseFloat",
|
||||
"isNaN",
|
||||
"isFinite",
|
||||
"encodeURI",
|
||||
"encodeURIComponent",
|
||||
"decodeURI",
|
||||
"decodeURIComponent",
|
||||
"eval",
|
||||
"Function",
|
||||
"Proxy",
|
||||
"Reflect",
|
||||
"Intl",
|
||||
"BigInt",
|
||||
]);
|
||||
|
||||
function normalizeListedExports(
|
||||
exports: string[],
|
||||
actualExports: Set<string>,
|
||||
): string[] {
|
||||
const normalized = new Set<string>();
|
||||
for (const exp of exports) {
|
||||
let candidate = exp.trim();
|
||||
if (candidate.startsWith("call:") || candidate.startsWith("raise:"))
|
||||
continue;
|
||||
candidate = candidate.replace(/^(class|func|method):/, "");
|
||||
candidate = candidate.split("(")[0]?.split(" ")[0] ?? candidate;
|
||||
const match = candidate.match(/[A-Za-z_$][\w$]*/);
|
||||
if (!match) continue;
|
||||
const symbol = match[0];
|
||||
if (
|
||||
actualExports.has(symbol) ||
|
||||
(/^[A-Z][A-Za-z0-9_$]*$/.test(symbol) && !BUILT_IN_GLOBALS.has(symbol)) ||
|
||||
/^[A-Z0-9_]+$/.test(symbol)
|
||||
) {
|
||||
normalized.add(symbol);
|
||||
}
|
||||
}
|
||||
return [...normalized];
|
||||
}
|
||||
|
||||
function markRepairMode(
|
||||
repairModes: Map<string, RepairMode>,
|
||||
dirPath: string,
|
||||
mode: RepairMode,
|
||||
): void {
|
||||
const existing = repairModes.get(dirPath);
|
||||
if (!existing || existing === "small") {
|
||||
repairModes.set(dirPath, mode);
|
||||
}
|
||||
}
|
||||
|
||||
function addRepairChain(
|
||||
entries: DirectoryEntry[],
|
||||
entry: DirectoryEntry,
|
||||
mode: RepairMode,
|
||||
plan: Map<string, ArtifactWriteMode>,
|
||||
): void {
|
||||
plan.set(entry.dirPath, "both");
|
||||
const ctx = buildDirectoryContext(entries, entry);
|
||||
let parent = ctx.parentMap.get(entry.relativePath);
|
||||
while (parent) {
|
||||
const ancestor = entries.find(
|
||||
(candidate) => candidate.relativePath === parent,
|
||||
);
|
||||
if (!ancestor) break;
|
||||
const desiredMode: ArtifactWriteMode = mode === "small" ? "index" : "both";
|
||||
const existingMode = plan.get(ancestor.dirPath);
|
||||
if (existingMode !== "both") {
|
||||
plan.set(ancestor.dirPath, desiredMode);
|
||||
}
|
||||
parent = ctx.parentMap.get(parent);
|
||||
}
|
||||
}
|
||||
|
||||
function depthOfPath(dirPath: string): number {
|
||||
return dirPath.split(/[\\/]/).filter(Boolean).length;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { extractFileAST } from "../src/ast-extract.js";
|
||||
import { extractFileAST } from "../src/ast/ast-extract.js";
|
||||
import { mkdtempSync, writeFileSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { tmpdir } from "os";
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from "vitest";
|
||||
import { mkdtempSync, writeFileSync, mkdirSync, rmSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { tmpdir } from "os";
|
||||
import { execSync } from "child_process";
|
||||
import { fileURLToPath } from "url";
|
||||
|
||||
const projectRoot = fileURLToPath(new URL("..", import.meta.url));
|
||||
|
||||
function runCli(
|
||||
args: string,
|
||||
cwd: string,
|
||||
): { stdout: string; stderr: string; exitCode: number } {
|
||||
try {
|
||||
const stdout = execSync(
|
||||
`npx tsx ${join(projectRoot, "src/cli/cli.ts")} ${args}`,
|
||||
{
|
||||
cwd,
|
||||
encoding: "utf8",
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
},
|
||||
);
|
||||
return { stdout: stdout.trim(), stderr: "", exitCode: 0 };
|
||||
} catch (err: any) {
|
||||
return {
|
||||
stdout: err.stdout?.toString().trim() || "",
|
||||
stderr: err.stderr?.toString().trim() || "",
|
||||
exitCode: err.status || 1,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
describe("cli context", () => {
|
||||
let dir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(join(tmpdir(), "pi-map-cli-"));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(dir, { recursive: true });
|
||||
});
|
||||
|
||||
it("context command returns a bundle for a matching query", () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
mkdirSync(join(dir, "src", "auth"));
|
||||
writeFileSync(
|
||||
join(dir, "src", "auth", "tokens.ts"),
|
||||
`export function validateToken() {}\n`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(dir, "src", "auth", ".pi-map.md"),
|
||||
`# src/auth
|
||||
dir: src/auth
|
||||
index: src/auth/.pi-map.index.md
|
||||
## role
|
||||
Authentication and token validation.
|
||||
## files
|
||||
- tokens.ts | Token validation | exp: validateToken | dep: -
|
||||
## arch
|
||||
Guard pattern.
|
||||
## tags
|
||||
auth, token, validate
|
||||
## symbols
|
||||
validateToken
|
||||
## workflows
|
||||
- validate token
|
||||
files: tokens.ts
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(dir, "src", "auth", ".pi-map.index.md"),
|
||||
`# src/auth (index)
|
||||
dir: src/auth
|
||||
## role
|
||||
Auth layer.
|
||||
## parent
|
||||
index: ./.pi-map.index.md
|
||||
map: ./.pi-map.md
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- tokens.ts
|
||||
## links
|
||||
index: src/auth/.pi-map.index.md
|
||||
map: src/auth/.pi-map.md
|
||||
## workflows
|
||||
- validate token
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
|
||||
const { stdout, exitCode } = runCli('context "token validation"', dir);
|
||||
expect(exitCode).toBe(0);
|
||||
expect(stdout).toContain("# Context bundle: token validation");
|
||||
expect(stdout).toContain("## relevant indexes");
|
||||
expect(stdout).toContain("src/auth/.pi-map.index.md");
|
||||
expect(stdout).toContain("## relevant maps");
|
||||
expect(stdout).toContain("src/auth/.pi-map.md");
|
||||
expect(stdout).toContain("## likely files");
|
||||
expect(stdout).toContain("src/auth/tokens.ts");
|
||||
expect(stdout).toContain("## instructions");
|
||||
});
|
||||
|
||||
it("context command omits symbols section when no symbols exist", () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(
|
||||
join(dir, "src", ".pi-map.md"),
|
||||
`# src
|
||||
dir: src
|
||||
index: src/.pi-map.index.md
|
||||
## role
|
||||
Core source.
|
||||
## files
|
||||
- index.ts | Entry | exp: main | dep: -
|
||||
## arch
|
||||
Entrypoint.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(dir, "src", ".pi-map.index.md"),
|
||||
`# src (index)
|
||||
dir: src
|
||||
## role
|
||||
Core source.
|
||||
## parent
|
||||
-
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- index.ts
|
||||
## links
|
||||
index: src/.pi-map.index.md
|
||||
map: src/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
|
||||
const { stdout, exitCode } = runCli('context "core source"', dir);
|
||||
expect(exitCode).toBe(0);
|
||||
expect(stdout).toContain("# Context bundle: core source");
|
||||
expect(stdout).not.toContain("## relevant symbols");
|
||||
});
|
||||
|
||||
it("context command returns no-results bundle when nothing matches", () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(
|
||||
join(dir, "src", ".pi-map.md"),
|
||||
`# src
|
||||
dir: src
|
||||
index: src/.pi-map.index.md
|
||||
## role
|
||||
Core source.
|
||||
## files
|
||||
- index.ts | Entry | exp: main | dep: -
|
||||
## arch
|
||||
Entrypoint.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(dir, "src", ".pi-map.index.md"),
|
||||
`# src (index)
|
||||
dir: src
|
||||
## role
|
||||
Core source.
|
||||
## parent
|
||||
-
|
||||
## children
|
||||
-
|
||||
## files
|
||||
- index.ts
|
||||
## links
|
||||
index: src/.pi-map.index.md
|
||||
map: src/.pi-map.md
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
|
||||
const { stdout, exitCode } = runCli('context "zzzzzzzz"', dir);
|
||||
expect(exitCode).toBe(0);
|
||||
expect(stdout).toContain("# Context bundle: zzzzzzzz");
|
||||
expect(stdout).toContain("No relevant directories found");
|
||||
});
|
||||
|
||||
it("context command errors when query is missing", () => {
|
||||
const { stderr, exitCode } = runCli("context", dir);
|
||||
expect(exitCode).toBe(1);
|
||||
expect(stderr).toContain("Missing query");
|
||||
});
|
||||
});
|
||||
@@ -2,8 +2,13 @@ import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
renderPackageMap,
|
||||
parsePackageMap,
|
||||
renderDirectoryMap,
|
||||
parseDirectoryMap,
|
||||
renderDirectoryIndex,
|
||||
parseDirectoryIndex,
|
||||
type PackageMapData,
|
||||
} from "../src/format.js";
|
||||
import { createDirectoryModel } from "../src/directory-model.js";
|
||||
|
||||
const sampleData: PackageMapData = {
|
||||
path: "pkg/auth",
|
||||
@@ -42,6 +47,9 @@ describe("format", () => {
|
||||
"- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions",
|
||||
);
|
||||
expect(output).toContain("## arch");
|
||||
expect(output).toContain("## tags");
|
||||
expect(output).toContain("## symbols");
|
||||
expect(output).toContain("## workflows");
|
||||
expect(output).toContain("## dirty");
|
||||
expect(output).toContain("-");
|
||||
});
|
||||
@@ -94,6 +102,12 @@ Test package
|
||||
Line one.
|
||||
Line two.
|
||||
Line three.
|
||||
## tags
|
||||
-
|
||||
## symbols
|
||||
-
|
||||
## workflows
|
||||
-
|
||||
## dirty
|
||||
-
|
||||
`;
|
||||
@@ -101,3 +115,278 @@ Line three.
|
||||
expect(parsed.arch).toBe("Line one.\nLine two.\nLine three.");
|
||||
});
|
||||
});
|
||||
|
||||
describe("paired artifacts", () => {
|
||||
it("renders and parses a directory map with tags/symbols/workflows", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/cli",
|
||||
role: "CLI entrypoint and command parsing",
|
||||
files: [
|
||||
{
|
||||
name: "cli.ts",
|
||||
purpose: "CLI entrypoint",
|
||||
exports: ["main"],
|
||||
deps: ["commander"],
|
||||
},
|
||||
],
|
||||
arch: "Command-line interface built on commander.js",
|
||||
});
|
||||
model.tags = ["cli", "entrypoint"];
|
||||
model.symbols = ["main"];
|
||||
model.workflows = [{ task: "add command", read: ["cli.ts"] }];
|
||||
|
||||
const rendered = renderDirectoryMap(model);
|
||||
expect(rendered).toContain("# src/cli");
|
||||
expect(rendered).toContain("## tags");
|
||||
expect(rendered).toContain("cli, entrypoint");
|
||||
expect(rendered).toContain("## symbols");
|
||||
expect(rendered).toContain("- main");
|
||||
expect(rendered).toContain("## workflows");
|
||||
expect(rendered).toContain("- add command");
|
||||
|
||||
const parsed = parseDirectoryMap(rendered);
|
||||
expect(parsed.dir).toBe("src/cli");
|
||||
expect(parsed.tags).toEqual(["cli", "entrypoint"]);
|
||||
expect(parsed.symbols).toEqual(["main"]);
|
||||
expect(parsed.workflows).toHaveLength(1);
|
||||
expect(parsed.workflows[0].task).toBe("add command");
|
||||
});
|
||||
|
||||
it("renders and parses a directory index with parent/children/links", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src",
|
||||
role: "Core source code",
|
||||
files: [
|
||||
{ name: "index.ts", purpose: "Main exports", exports: [], deps: [] },
|
||||
],
|
||||
arch: "Source code root",
|
||||
parent: ".",
|
||||
children: ["src/cli", "src/lib"],
|
||||
});
|
||||
|
||||
const rendered = renderDirectoryIndex(model);
|
||||
expect(rendered).toContain("# src (index)");
|
||||
expect(rendered).toContain("dir: src");
|
||||
expect(rendered).toContain("## parent");
|
||||
expect(rendered).toContain("index: ./.pi-map.index.md");
|
||||
expect(rendered).toContain("map: ./.pi-map.md");
|
||||
expect(rendered).toContain("## children");
|
||||
expect(rendered).toContain("- src/cli");
|
||||
expect(rendered).toContain("index: src/cli/.pi-map.index.md");
|
||||
expect(rendered).toContain("map: src/cli/.pi-map.md");
|
||||
expect(rendered).toContain("## links");
|
||||
expect(rendered).toContain("index: src/.pi-map.index.md");
|
||||
expect(rendered).toContain("map: src/.pi-map.md");
|
||||
|
||||
const parsed = parseDirectoryIndex(rendered);
|
||||
expect(parsed.dir).toBe("src");
|
||||
expect(parsed.parent).toBe(".");
|
||||
expect(parsed.children).toEqual(["src/cli", "src/lib"]);
|
||||
});
|
||||
|
||||
it("renders a root index with Project Map Protocol", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: ".",
|
||||
role: "Project root",
|
||||
files: [],
|
||||
arch: "Root architecture",
|
||||
isRoot: true,
|
||||
});
|
||||
|
||||
const rendered = renderDirectoryIndex(model);
|
||||
expect(rendered).toContain("# . (index)");
|
||||
expect(rendered).toContain("dir: .");
|
||||
});
|
||||
|
||||
it("round-trips an empty leaf index", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/utils",
|
||||
role: "Utilities",
|
||||
files: [
|
||||
{ name: "helpers.ts", purpose: "Helpers", exports: [], deps: [] },
|
||||
],
|
||||
arch: "Shared helpers",
|
||||
});
|
||||
|
||||
const rendered = renderDirectoryIndex(model);
|
||||
expect(rendered).toContain("# src/utils (index)");
|
||||
expect(rendered).toContain("## children");
|
||||
expect(rendered).toContain("-");
|
||||
|
||||
const parsed = parseDirectoryIndex(rendered);
|
||||
expect(parsed.dir).toBe("src/utils");
|
||||
expect(parsed.children).toEqual([]);
|
||||
expect(parsed.files).toHaveLength(1);
|
||||
expect(parsed.files[0].name).toBe("helpers.ts");
|
||||
});
|
||||
|
||||
it("round-trips workflows with read continuations in map", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/cli",
|
||||
role: "CLI entrypoint",
|
||||
files: [{ name: "cli.ts", purpose: "CLI", exports: ["main"], deps: [] }],
|
||||
arch: "CLI",
|
||||
});
|
||||
model.workflows = [
|
||||
{ task: "add command", read: ["cli.ts", "commands.ts"] },
|
||||
{ task: "update flags", read: ["cli.ts"] },
|
||||
];
|
||||
|
||||
const rendered = renderDirectoryMap(model);
|
||||
expect(rendered).toContain("- add command");
|
||||
expect(rendered).toContain(" read: cli.ts, commands.ts");
|
||||
expect(rendered).toContain("- update flags");
|
||||
expect(rendered).toContain(" read: cli.ts");
|
||||
|
||||
const parsed = parseDirectoryMap(rendered);
|
||||
expect(parsed.workflows).toHaveLength(2);
|
||||
expect(parsed.workflows[0].task).toBe("add command");
|
||||
expect(parsed.workflows[0].read).toEqual(["cli.ts", "commands.ts"]);
|
||||
expect(parsed.workflows[1].task).toBe("update flags");
|
||||
expect(parsed.workflows[1].read).toEqual(["cli.ts"]);
|
||||
});
|
||||
|
||||
it("round-trips workflows with read continuations in index", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/cli",
|
||||
role: "CLI entrypoint",
|
||||
files: [{ name: "cli.ts", purpose: "CLI", exports: ["main"], deps: [] }],
|
||||
arch: "CLI",
|
||||
});
|
||||
model.workflows = [
|
||||
{ task: "add command", read: ["cli.ts", "commands.ts"] },
|
||||
];
|
||||
|
||||
const rendered = renderDirectoryIndex(model);
|
||||
expect(rendered).toContain("- add command");
|
||||
expect(rendered).toContain(" read: cli.ts, commands.ts");
|
||||
|
||||
const parsed = parseDirectoryIndex(rendered);
|
||||
expect(parsed.workflows).toHaveLength(1);
|
||||
expect(parsed.workflows[0].task).toBe("add command");
|
||||
expect(parsed.workflows[0].read).toEqual(["cli.ts", "commands.ts"]);
|
||||
});
|
||||
|
||||
it("round-trips workflows with index, map, and files continuations", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src",
|
||||
role: "Source root",
|
||||
files: [{ name: "index.ts", purpose: "Exports", exports: [], deps: [] }],
|
||||
arch: "Source",
|
||||
children: ["src/auth", "src/cli"],
|
||||
});
|
||||
model.workflows = [
|
||||
{
|
||||
task: "explore subdirectories",
|
||||
index: ["src/auth/.pi-map.index.md", "src/cli/.pi-map.index.md"],
|
||||
map: ["src/auth/.pi-map.md"],
|
||||
files: ["src/index.ts"],
|
||||
},
|
||||
];
|
||||
|
||||
const mapRendered = renderDirectoryMap(model);
|
||||
expect(mapRendered).toContain("- explore subdirectories");
|
||||
expect(mapRendered).toContain(
|
||||
" index: src/auth/.pi-map.index.md, src/cli/.pi-map.index.md",
|
||||
);
|
||||
expect(mapRendered).toContain(" map: src/auth/.pi-map.md");
|
||||
expect(mapRendered).toContain(" files: src/index.ts");
|
||||
|
||||
const mapParsed = parseDirectoryMap(mapRendered);
|
||||
expect(mapParsed.workflows).toHaveLength(1);
|
||||
expect(mapParsed.workflows[0].index).toEqual([
|
||||
"src/auth/.pi-map.index.md",
|
||||
"src/cli/.pi-map.index.md",
|
||||
]);
|
||||
expect(mapParsed.workflows[0].map).toEqual(["src/auth/.pi-map.md"]);
|
||||
expect(mapParsed.workflows[0].files).toEqual(["src/index.ts"]);
|
||||
|
||||
const indexRendered = renderDirectoryIndex(model);
|
||||
expect(indexRendered).toContain("- explore subdirectories");
|
||||
expect(indexRendered).toContain(
|
||||
" index: src/auth/.pi-map.index.md, src/cli/.pi-map.index.md",
|
||||
);
|
||||
|
||||
const indexParsed = parseDirectoryIndex(indexRendered);
|
||||
expect(indexParsed.workflows).toHaveLength(1);
|
||||
expect(indexParsed.workflows[0].index).toEqual([
|
||||
"src/auth/.pi-map.index.md",
|
||||
"src/cli/.pi-map.index.md",
|
||||
]);
|
||||
});
|
||||
|
||||
it("renderer includes dir and index preamble directly", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/core",
|
||||
role: "Core logic",
|
||||
files: [],
|
||||
arch: "Core",
|
||||
});
|
||||
|
||||
const rendered = renderDirectoryMap(model);
|
||||
expect(rendered).toContain("dir: src/core");
|
||||
expect(rendered).toContain("index: src/core/.pi-map.index.md");
|
||||
});
|
||||
|
||||
it("root renderer includes Project Map Protocol directly", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: ".",
|
||||
role: "Project root",
|
||||
files: [],
|
||||
arch: "Root",
|
||||
isRoot: true,
|
||||
});
|
||||
|
||||
const mapRendered = renderDirectoryMap(model);
|
||||
expect(mapRendered).toContain("## Project Map Protocol");
|
||||
expect(mapRendered).toContain(
|
||||
"Trust boundary: index routes, map orients, source decides.",
|
||||
);
|
||||
|
||||
const indexRendered = renderDirectoryIndex(model);
|
||||
expect(indexRendered).toContain("## Project Map Protocol");
|
||||
});
|
||||
|
||||
it("parses file lines with commas inside signatures", () => {
|
||||
const mapText = `# src
|
||||
## files
|
||||
- format.ts | Renders maps | exp: PackageMapData, func:renderPackageMap(data: PackageMapData) → string, call:convertPackageMapToModel, func:renderDirectoryIndex(model: DirectoryArtifactModel) → string | dep: ./model.js
|
||||
## arch
|
||||
Test
|
||||
## dirty
|
||||
-
|
||||
`;
|
||||
const parsed = parseDirectoryMap(mapText);
|
||||
expect(parsed.files).toHaveLength(1);
|
||||
expect(parsed.files[0].exports).toEqual([
|
||||
"PackageMapData",
|
||||
"func:renderPackageMap(data: PackageMapData) → string",
|
||||
"call:convertPackageMapToModel",
|
||||
"func:renderDirectoryIndex(model: DirectoryArtifactModel) → string",
|
||||
]);
|
||||
expect(parsed.files[0].deps).toEqual(["./model.js"]);
|
||||
});
|
||||
|
||||
it("parses file lines with pipes and commas inside type signatures", () => {
|
||||
const mapText = `# src
|
||||
## files
|
||||
- llm-cache.ts | Cache helpers | exp: func:getCached(hash: string, cacheDir: string) → string | undefined, func:setCached(hash: string, result: string, cacheDir: string) → void | dep: fs, path
|
||||
- user.ts | User helpers | exp: User, func:createUser(data: Omit<User, "id" | "createdAt">) → User, func:serializeUser(user: User) → string | dep: ../utils/validation.js
|
||||
## arch
|
||||
Test
|
||||
## dirty
|
||||
-
|
||||
`;
|
||||
const parsed = parseDirectoryMap(mapText);
|
||||
expect(parsed.files).toHaveLength(2);
|
||||
expect(parsed.files[0].exports).toEqual([
|
||||
"func:getCached(hash: string, cacheDir: string) → string | undefined",
|
||||
"func:setCached(hash: string, result: string, cacheDir: string) → void",
|
||||
]);
|
||||
expect(parsed.files[1].exports).toEqual([
|
||||
"User",
|
||||
'func:createUser(data: Omit<User, "id" | "createdAt">) → User',
|
||||
"func:serializeUser(user: User) → string",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
+263
-25
@@ -11,6 +11,7 @@ import { tmpdir } from "os";
|
||||
import { initProject } from "../src/init.js";
|
||||
import { patchFile } from "../src/patch.js";
|
||||
import { validateMaps } from "../src/validate.js";
|
||||
import { createMockFileClient } from "./mock-llm.js";
|
||||
|
||||
describe("integration", () => {
|
||||
let dir: string;
|
||||
@@ -23,61 +24,157 @@ describe("integration", () => {
|
||||
rmSync(dir, { recursive: true });
|
||||
});
|
||||
|
||||
it("init creates .pi-map.md files", async () => {
|
||||
it("init creates both .pi-map.md and .pi-map.index.md files", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "index.ts"), `export function foo() {}\n`);
|
||||
await initProject(dir);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const map = readFileSync(join(dir, "src", ".pi-map.md"), "utf8");
|
||||
const index = readFileSync(join(dir, "src", ".pi-map.index.md"), "utf8");
|
||||
expect(map).toContain("# src");
|
||||
expect(map).toContain("foo");
|
||||
expect(index).toContain("# src (index)");
|
||||
expect(index).toContain("dir: src");
|
||||
});
|
||||
|
||||
it("root artifacts contain Project Map Protocol", async () => {
|
||||
writeFileSync(join(dir, "package.json"), `{}\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const rootMap = readFileSync(join(dir, ".pi-map.md"), "utf8");
|
||||
const rootIndex = readFileSync(join(dir, ".pi-map.index.md"), "utf8");
|
||||
|
||||
expect(rootMap).toContain("## Project Map Protocol");
|
||||
expect(rootMap).toContain("index: ./.pi-map.index.md");
|
||||
expect(rootIndex).toContain("## Project Map Protocol");
|
||||
expect(rootIndex).toContain("map: ./.pi-map.md");
|
||||
});
|
||||
|
||||
it("non-root map contains index link", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "index.ts"), `export function foo() {}\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const map = readFileSync(join(dir, "src", ".pi-map.md"), "utf8");
|
||||
expect(map).toContain("index: src/.pi-map.index.md");
|
||||
expect(map).toContain("dir: src");
|
||||
});
|
||||
|
||||
it("index contains parent and children when applicable", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
mkdirSync(join(dir, "src", "utils"));
|
||||
writeFileSync(join(dir, "src", "index.ts"), `export function foo() {}\n`);
|
||||
writeFileSync(
|
||||
join(dir, "src", "utils", "helpers.ts"),
|
||||
`export const h = 1;\n`,
|
||||
);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const srcIndex = readFileSync(join(dir, "src", ".pi-map.index.md"), "utf8");
|
||||
expect(srcIndex).toContain("## parent");
|
||||
expect(srcIndex).toContain("index: ./.pi-map.index.md");
|
||||
|
||||
expect(srcIndex).toContain("## children");
|
||||
expect(srcIndex).toContain("- src/utils");
|
||||
|
||||
const utilsIndex = readFileSync(
|
||||
join(dir, "src", "utils", ".pi-map.index.md"),
|
||||
"utf8",
|
||||
);
|
||||
expect(utilsIndex).toContain("## parent");
|
||||
expect(utilsIndex).toContain("index: src/.pi-map.index.md");
|
||||
});
|
||||
|
||||
it("patch updates a file entry", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "index.ts"), `export function foo() {}\n`);
|
||||
writeFileSync(join(dir, "src", "utils.ts"), `export const bar = 1;\n`);
|
||||
await initProject(dir);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
// Modify a file
|
||||
writeFileSync(
|
||||
join(dir, "src", "index.ts"),
|
||||
`export function foo() {}\nexport function baz() {}\n`,
|
||||
);
|
||||
await patchFile(join(dir, "src", "index.ts"));
|
||||
await patchFile(join(dir, "src", "index.ts"), client, dir);
|
||||
|
||||
const map = readFileSync(join(dir, "src", ".pi-map.md"), "utf8");
|
||||
expect(map).toContain("baz");
|
||||
});
|
||||
|
||||
it("patch adds dirty marker for large packages", async () => {
|
||||
it("patch with small mode refreshes ancestor indexes but preserves ancestor maps", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
// Create 11 files so it's a "large" package
|
||||
for (let i = 0; i < 11; i++) {
|
||||
writeFileSync(
|
||||
join(dir, "src", `file${i}.ts`),
|
||||
`export const x${i} = ${i};\n`,
|
||||
);
|
||||
}
|
||||
await initProject(dir);
|
||||
writeFileSync(join(dir, "package.json"), `{}\n`);
|
||||
writeFileSync(join(dir, "src", "index.ts"), `export const a = 1;\n`);
|
||||
writeFileSync(join(dir, "src", "helper.ts"), `export const b = 2;\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
// Modify a file
|
||||
writeFileSync(
|
||||
join(dir, "src", "file0.ts"),
|
||||
`export const x0 = 0;\nexport const y = 99;\n`,
|
||||
join(dir, ".pi-map.md"),
|
||||
`${readFileSync(join(dir, ".pi-map.md"), "utf8")}\nSENTINEL_ROOT_MAP\n`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(dir, ".pi-map.index.md"),
|
||||
`${readFileSync(join(dir, ".pi-map.index.md"), "utf8")}\nSENTINEL_ROOT_INDEX\n`,
|
||||
);
|
||||
await patchFile(join(dir, "src", "file0.ts"));
|
||||
|
||||
const map = readFileSync(join(dir, "src", ".pi-map.md"), "utf8");
|
||||
expect(map).toContain("y");
|
||||
expect(map).toContain("dirty");
|
||||
expect(map).toContain("patched");
|
||||
writeFileSync(
|
||||
join(dir, "src", "index.ts"),
|
||||
`export const a = 1;\nexport const c = 3;\n`,
|
||||
);
|
||||
await patchFile(join(dir, "src", "index.ts"), client, dir, {
|
||||
rootPath: dir,
|
||||
patchMode: "small",
|
||||
});
|
||||
|
||||
const rootMap = readFileSync(join(dir, ".pi-map.md"), "utf8");
|
||||
const rootIndex = readFileSync(join(dir, ".pi-map.index.md"), "utf8");
|
||||
expect(rootMap).toContain("SENTINEL_ROOT_MAP");
|
||||
expect(rootIndex).not.toContain("SENTINEL_ROOT_INDEX");
|
||||
});
|
||||
|
||||
it("patch with structural mode refreshes ancestor maps and indexes", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "package.json"), `{}\n`);
|
||||
writeFileSync(join(dir, "src", "index.ts"), `export const a = 1;\n`);
|
||||
writeFileSync(join(dir, "src", "helper.ts"), `export const b = 2;\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
writeFileSync(
|
||||
join(dir, ".pi-map.md"),
|
||||
`${readFileSync(join(dir, ".pi-map.md"), "utf8")}\nSENTINEL_ROOT_MAP\n`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(dir, ".pi-map.index.md"),
|
||||
`${readFileSync(join(dir, ".pi-map.index.md"), "utf8")}\nSENTINEL_ROOT_INDEX\n`,
|
||||
);
|
||||
|
||||
writeFileSync(
|
||||
join(dir, "src", "index.ts"),
|
||||
`export const a = 1;\nexport const c = 3;\n`,
|
||||
);
|
||||
await patchFile(join(dir, "src", "index.ts"), client, dir, {
|
||||
rootPath: dir,
|
||||
patchMode: "structural",
|
||||
});
|
||||
|
||||
const rootMap = readFileSync(join(dir, ".pi-map.md"), "utf8");
|
||||
const rootIndex = readFileSync(join(dir, ".pi-map.index.md"), "utf8");
|
||||
expect(rootMap).not.toContain("SENTINEL_ROOT_MAP");
|
||||
expect(rootIndex).not.toContain("SENTINEL_ROOT_INDEX");
|
||||
});
|
||||
|
||||
it("validate detects new files", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
|
||||
await initProject(dir);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
// Add new file
|
||||
writeFileSync(join(dir, "src", "b.ts"), `export const b = 2;\n`);
|
||||
@@ -91,7 +188,8 @@ describe("integration", () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
|
||||
writeFileSync(join(dir, "src", "b.ts"), `export const b = 2;\n`);
|
||||
await initProject(dir);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
// Delete a file
|
||||
rmSync(join(dir, "src", "b.ts"));
|
||||
@@ -104,7 +202,8 @@ describe("integration", () => {
|
||||
it("validate detects changed signatures", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
|
||||
await initProject(dir);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
// Change exports
|
||||
writeFileSync(
|
||||
@@ -118,4 +217,143 @@ describe("integration", () => {
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("init populates routing metadata in rich maps", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(
|
||||
join(dir, "src", "index.ts"),
|
||||
`export function init() {}\nexport function run() {}\n`,
|
||||
);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const map = readFileSync(join(dir, "src", ".pi-map.md"), "utf8");
|
||||
// Tags and symbols should be scaffolded or populated
|
||||
expect(map).toContain("## tags");
|
||||
expect(map).toContain("## symbols");
|
||||
expect(map).toContain("## workflows");
|
||||
});
|
||||
|
||||
it("leaf directory index is tiny but present", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
mkdirSync(join(dir, "src", "utils"));
|
||||
writeFileSync(
|
||||
join(dir, "src", "utils", "helpers.ts"),
|
||||
`export const h = 1;\n`,
|
||||
);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const index = readFileSync(
|
||||
join(dir, "src", "utils", ".pi-map.index.md"),
|
||||
"utf8",
|
||||
);
|
||||
// Leaf index should have empty children but still follow contract
|
||||
expect(index).toContain("## children");
|
||||
expect(index).toContain("-");
|
||||
expect(index).toContain("## parent");
|
||||
expect(index).toContain("index: src/.pi-map.index.md");
|
||||
});
|
||||
|
||||
it("non-source directories omit uncertain workflows", async () => {
|
||||
mkdirSync(join(dir, "docs"));
|
||||
writeFileSync(join(dir, "docs", "README.md"), `# README\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const map = readFileSync(join(dir, "docs", ".pi-map.md"), "utf8");
|
||||
// No source files means workflows section should be empty/omitted
|
||||
const workflowsMatch = map.match(/## workflows\n([^#]*)/);
|
||||
if (workflowsMatch) {
|
||||
expect(workflowsMatch[1].trim()).toBe("-");
|
||||
}
|
||||
});
|
||||
|
||||
it("init loads config caps and applies them to routing metadata", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
for (let i = 0; i < 20; i++) {
|
||||
writeFileSync(
|
||||
join(dir, "src", `file${i}.ts`),
|
||||
`export function fn${i}() {}\n`,
|
||||
);
|
||||
}
|
||||
writeFileSync(
|
||||
join(dir, ".pi-project-map.json"),
|
||||
JSON.stringify({ tagCap: 3, workflowHintCap: 2 }),
|
||||
);
|
||||
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const map = readFileSync(join(dir, "src", ".pi-map.md"), "utf8");
|
||||
const parsedMap = (await import("../src/format.js")).parseDirectoryMap(map);
|
||||
expect(parsedMap.tags.length).toBeLessThanOrEqual(3);
|
||||
expect(parsedMap.symbols.length).toBeLessThanOrEqual(3);
|
||||
|
||||
const index = readFileSync(join(dir, "src", ".pi-map.index.md"), "utf8");
|
||||
const parsedIndex = (await import("../src/format.js")).parseDirectoryIndex(
|
||||
index,
|
||||
);
|
||||
expect(parsedIndex.workflows.length).toBeLessThanOrEqual(2);
|
||||
});
|
||||
|
||||
it("validate hard-fails when an index is missing", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
rmSync(join(dir, "src", ".pi-map.index.md"));
|
||||
|
||||
const result = await validateMaps(dir, { verbose: false });
|
||||
expect(result.clean).toBe(false);
|
||||
expect(result.discrepancies.some((d) => d.type === "stale-index")).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("validate detects broken sibling links", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
const mapPath = join(dir, "src", ".pi-map.md");
|
||||
writeFileSync(
|
||||
mapPath,
|
||||
readFileSync(mapPath, "utf8").replace(
|
||||
"index: src/.pi-map.index.md",
|
||||
"index: src/bad-index.md",
|
||||
),
|
||||
);
|
||||
|
||||
const result = await validateMaps(dir, { verbose: false });
|
||||
expect(result.clean).toBe(false);
|
||||
expect(result.discrepancies.some((d) => d.type === "broken-link")).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("validate --fix repairs the affected chain", async () => {
|
||||
mkdirSync(join(dir, "src"));
|
||||
writeFileSync(join(dir, "package.json"), `{}\n`);
|
||||
writeFileSync(join(dir, "src", "a.ts"), `export const a = 1;\n`);
|
||||
const client = createMockFileClient();
|
||||
await initProject(dir, { llmClient: client, verbose: false });
|
||||
|
||||
rmSync(join(dir, "src", ".pi-map.index.md"));
|
||||
const result = await validateMaps(dir, {
|
||||
fix: true,
|
||||
verbose: false,
|
||||
llmClient: client,
|
||||
cacheDir: dir,
|
||||
patchMode: "small",
|
||||
});
|
||||
|
||||
expect(result.clean).toBe(false);
|
||||
expect(result.fixed).toBeGreaterThan(0);
|
||||
expect(
|
||||
readFileSync(join(dir, "src", ".pi-map.index.md"), "utf8"),
|
||||
).toContain("# src (index)");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { withRetry, processFiles } from "../src/llm-batch.js";
|
||||
import { withRetry, processFiles } from "../src/llm/llm-batch.js";
|
||||
import { LLMError } from "../src/llm-error.js";
|
||||
|
||||
describe("withRetry", () => {
|
||||
|
||||
+14
-10
@@ -1,10 +1,10 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from "vitest";
|
||||
import { getCached, setCached } from "../src/llm-cache.js";
|
||||
import { existsSync, unlinkSync, rmdirSync } from "fs";
|
||||
import { getCached, setCached } from "../src/llm/llm-cache.js";
|
||||
import { existsSync, unlinkSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { homedir } from "os";
|
||||
import { tmpdir } from "os";
|
||||
|
||||
const TEST_CACHE_DIR = join(homedir(), ".cache", "pi-project-map");
|
||||
const TEST_CACHE_DIR = join(tmpdir(), "pi-project-map-test-cache");
|
||||
const TEST_CACHE_FILE = join(TEST_CACHE_DIR, "llm-cache.json");
|
||||
|
||||
describe("llm-cache", () => {
|
||||
@@ -21,20 +21,24 @@ describe("llm-cache", () => {
|
||||
});
|
||||
|
||||
it("returns undefined for missing entries", () => {
|
||||
const result = getCached("nonexistent-hash");
|
||||
const result = getCached("nonexistent-hash", TEST_CACHE_DIR);
|
||||
expect(result).toBeUndefined();
|
||||
});
|
||||
|
||||
it("stores and retrieves cached results", () => {
|
||||
setCached("abc123", "PURPOSE: test\nDEPS: none\nCONCEPTS: none");
|
||||
const result = getCached("abc123");
|
||||
setCached(
|
||||
"abc123",
|
||||
"PURPOSE: test\nDEPS: none\nCONCEPTS: none",
|
||||
TEST_CACHE_DIR,
|
||||
);
|
||||
const result = getCached("abc123", TEST_CACHE_DIR);
|
||||
expect(result).toBe("PURPOSE: test\nDEPS: none\nCONCEPTS: none");
|
||||
});
|
||||
|
||||
it("overwrites existing entries", () => {
|
||||
setCached("abc123", "old");
|
||||
setCached("abc123", "new");
|
||||
const result = getCached("abc123");
|
||||
setCached("abc123", "old", TEST_CACHE_DIR);
|
||||
setCached("abc123", "new", TEST_CACHE_DIR);
|
||||
const result = getCached("abc123", TEST_CACHE_DIR);
|
||||
expect(result).toBe("new");
|
||||
});
|
||||
});
|
||||
|
||||
+47
-87
@@ -1,67 +1,17 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { extractFileLLM, extractFileHeuristic } from "../src/llm-extract.js";
|
||||
import { extractFileLLM } from "../src/llm/llm-extract.js";
|
||||
import { writeFileSync, mkdtempSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { tmpdir } from "os";
|
||||
import type { LLMClient } from "../src/llm-client.js";
|
||||
import type { LLMClient } from "../src/llm/llm-client.js";
|
||||
|
||||
describe("llm-extract heuristics", () => {
|
||||
it("extracts TypeScript exports", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "test.ts");
|
||||
writeFileSync(
|
||||
file,
|
||||
`export function foo() {}
|
||||
export class Bar {}
|
||||
export const baz = 1;
|
||||
export type Qux = string;
|
||||
export { a, b as c };
|
||||
`,
|
||||
);
|
||||
const result = await extractFileLLM(file);
|
||||
expect(result.exports).toContain("foo");
|
||||
expect(result.exports).toContain("Bar");
|
||||
expect(result.exports).toContain("baz");
|
||||
expect(result.exports).toContain("Qux");
|
||||
expect(result.exports).toContain("a");
|
||||
expect(result.exports).toContain("b");
|
||||
});
|
||||
|
||||
it("extracts TypeScript imports", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "test.ts");
|
||||
writeFileSync(
|
||||
file,
|
||||
`import { foo } from "./bar";
|
||||
import * as baz from "baz-lib";
|
||||
import type { Qux } from "qux";
|
||||
const x = require("legacy");
|
||||
`,
|
||||
);
|
||||
const result = await extractFileLLM(file);
|
||||
expect(result.deps).toContain("./bar");
|
||||
expect(result.deps).toContain("baz-lib");
|
||||
expect(result.deps).toContain("qux");
|
||||
expect(result.deps).toContain("legacy");
|
||||
});
|
||||
|
||||
it("infers purpose from filename patterns", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "userController.ts");
|
||||
writeFileSync(file, `export class UserController {}`);
|
||||
const result = await extractFileLLM(file);
|
||||
expect(result.purpose).toMatch(/Controller|Exports/);
|
||||
});
|
||||
|
||||
it("handles non-code files", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "Dockerfile");
|
||||
writeFileSync(file, `FROM node:20\nWORKDIR /app`);
|
||||
const result = await extractFileLLM(file);
|
||||
expect(result.purpose).toBe("Container definition");
|
||||
expect(result.exports).toEqual([]);
|
||||
});
|
||||
});
|
||||
function createMockClient(response: string): LLMClient {
|
||||
return {
|
||||
async complete() {
|
||||
return response;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe("llm-extract with mock client", () => {
|
||||
it("uses LLM client when provided", async () => {
|
||||
@@ -69,54 +19,64 @@ describe("llm-extract with mock client", () => {
|
||||
const file = join(dir, "test.ts");
|
||||
writeFileSync(file, `export const foo = 1;`);
|
||||
|
||||
const mockClient: LLMClient = {
|
||||
async complete() {
|
||||
return "PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing";
|
||||
},
|
||||
};
|
||||
const mockClient = createMockClient(
|
||||
"PURPOSE: Test file\nDEPS: none\nCONCEPTS: testing",
|
||||
);
|
||||
|
||||
const result = await extractFileLLM(file, mockClient);
|
||||
const result = await extractFileLLM(file, mockClient, tmpdir());
|
||||
expect(result.purpose).toBe("Test file");
|
||||
expect(result.deps).toEqual([]);
|
||||
expect(result.concepts).toContain("testing");
|
||||
});
|
||||
|
||||
it("throws without LLM client", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "test.ts");
|
||||
writeFileSync(file, `export const foo = 1;`);
|
||||
|
||||
await expect(extractFileLLM(file)).rejects.toThrow("No LLM client configured");
|
||||
});
|
||||
|
||||
it("skips large files", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "big.ts");
|
||||
writeFileSync(file, "x".repeat(60 * 1024));
|
||||
writeFileSync(file, "x".repeat(600 * 1024));
|
||||
|
||||
const mockClient: LLMClient = {
|
||||
async complete() {
|
||||
return "PURPOSE: Should not call\nDEPS: none\nCONCEPTS: none";
|
||||
},
|
||||
};
|
||||
const mockClient = createMockClient(
|
||||
"PURPOSE: Should not call\nDEPS: none\nCONCEPTS: none",
|
||||
);
|
||||
|
||||
const result = await extractFileLLM(file, mockClient);
|
||||
expect(result.purpose).toBe("Large/generated file");
|
||||
expect(result.purpose).toBe("Large file");
|
||||
});
|
||||
|
||||
it("falls back to heuristics without client", async () => {
|
||||
it("skips binary files", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "utils.ts");
|
||||
writeFileSync(file, `export function helper() {}`);
|
||||
const file = join(dir, "image.png");
|
||||
// Write some binary-looking content with null bytes
|
||||
const buf = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
writeFileSync(file, buf);
|
||||
|
||||
const result = await extractFileLLM(file);
|
||||
expect(result.purpose).toBe("Utility functions");
|
||||
expect(result.exports).toContain("helper");
|
||||
const mockClient = createMockClient(
|
||||
"PURPOSE: Should not call\nDEPS: none\nCONCEPTS: none",
|
||||
);
|
||||
|
||||
const result = await extractFileLLM(file, mockClient);
|
||||
expect(result.purpose).toBe("Binary file");
|
||||
});
|
||||
});
|
||||
|
||||
describe("extractFileHeuristic", () => {
|
||||
it("returns structured data", async () => {
|
||||
it("parses response with deps and concepts", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-"));
|
||||
const file = join(dir, "test.ts");
|
||||
writeFileSync(file, `export const x = 1;`);
|
||||
writeFileSync(file, `import { foo } from "bar";\nexport const x = 1;`);
|
||||
|
||||
const result = await extractFileHeuristic(file);
|
||||
expect(result.purpose).toBeDefined();
|
||||
expect(Array.isArray(result.exports)).toBe(true);
|
||||
expect(Array.isArray(result.deps)).toBe(true);
|
||||
expect(Array.isArray(result.concepts)).toBe(true);
|
||||
const mockClient = createMockClient(
|
||||
"PURPOSE: Config module\nDEPS: bar, baz\nCONCEPTS: constants, config",
|
||||
);
|
||||
|
||||
const result = await extractFileLLM(file, mockClient, tmpdir());
|
||||
expect(result.purpose).toBe("Config module");
|
||||
expect(result.deps).toEqual(["bar", "baz"]);
|
||||
expect(result.concepts).toEqual(["constants", "config"]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2,9 +2,9 @@ import { describe, it, expect } from "vitest";
|
||||
import { writeFileSync, mkdtempSync, readFileSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { tmpdir } from "os";
|
||||
import { createLLMClient } from "../src/llm-client.js";
|
||||
import { extractFileLLM, extractPackageLLM } from "../src/llm-extract.js";
|
||||
import { processFiles } from "../src/llm-batch.js";
|
||||
import { createLLMClient } from "../src/llm/llm-client.js";
|
||||
import { extractFileLLM, extractPackageLLM } from "../src/llm/llm-extract.js";
|
||||
import { processFiles } from "../src/llm/llm-batch.js";
|
||||
|
||||
// Load .env file manually (no dotenv dependency needed)
|
||||
function loadEnv(): Record<string, string> {
|
||||
@@ -55,7 +55,7 @@ describe.skipIf(!hasKimiKey)("LLM integration with Kimi", () => {
|
||||
);
|
||||
|
||||
const client = createLLMClient("kimi", { model: kimiModel });
|
||||
const result = await extractFileLLM(file, client);
|
||||
const result = await extractFileLLM(file, client, dir);
|
||||
|
||||
expect(result.purpose).toBeTruthy();
|
||||
expect(result.purpose.length).toBeGreaterThan(5);
|
||||
@@ -90,12 +90,12 @@ describe.skipIf(!hasKimiKey)("LLM integration with Kimi", () => {
|
||||
writeFileSync(file, `export const version = "1.0.0";`);
|
||||
|
||||
const client = createLLMClient("kimi", { model: kimiModel });
|
||||
const result1 = await extractFileLLM(file, client);
|
||||
const result1 = await extractFileLLM(file, client, dir);
|
||||
expect(result1.purpose).toBeTruthy();
|
||||
|
||||
// Second call should hit cache — much faster
|
||||
const start = Date.now();
|
||||
const result2 = await extractFileLLM(file, client);
|
||||
const result2 = await extractFileLLM(file, client, dir);
|
||||
const elapsed = Date.now() - start;
|
||||
|
||||
expect(result2.purpose).toBe(result1.purpose);
|
||||
@@ -119,7 +119,7 @@ describe.skipIf(!hasKimiKey)("LLM integration with Kimi", () => {
|
||||
const start = Date.now();
|
||||
const results = await processFiles(
|
||||
files,
|
||||
async (f) => extractFileLLM(f, client),
|
||||
async (f) => extractFileLLM(f, client, dir),
|
||||
{ concurrency: 3, maxRetries: 1, retryDelaysMs: [2000] },
|
||||
);
|
||||
const elapsed = Date.now() - start;
|
||||
@@ -135,7 +135,7 @@ describe.skipIf(!hasKimiKey)("LLM integration with Kimi", () => {
|
||||
it("skips large files without calling LLM", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "pi-map-llm-"));
|
||||
const file = join(dir, "big.ts");
|
||||
writeFileSync(file, "x".repeat(60 * 1024));
|
||||
writeFileSync(file, "x".repeat(600 * 1024));
|
||||
|
||||
let calls = 0;
|
||||
const trackingClient = createLLMClient("kimi", { model: kimiModel });
|
||||
@@ -145,8 +145,8 @@ describe.skipIf(!hasKimiKey)("LLM integration with Kimi", () => {
|
||||
return originalComplete(...args);
|
||||
};
|
||||
|
||||
const result = await extractFileLLM(file, trackingClient);
|
||||
expect(result.purpose).toBe("Large/generated file");
|
||||
const result = await extractFileLLM(file, trackingClient, dir);
|
||||
expect(result.purpose).toBe("Large file");
|
||||
expect(calls).toBe(0); // Should never call LLM for large files
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { mergeFileData } from "../src/merge.js";
|
||||
|
||||
describe("mergeFileData", () => {
|
||||
it("collapses multi-line parameters into single-line func signatures", () => {
|
||||
const result = mergeFileData(
|
||||
"api.ts",
|
||||
{ purpose: "API helpers", exports: [], deps: [] },
|
||||
{
|
||||
exports: ["shouldReinjectForEvent"],
|
||||
deps: [],
|
||||
classes: [],
|
||||
functions: [
|
||||
{
|
||||
name: "shouldReinjectForEvent",
|
||||
params: [
|
||||
"event: {\n\t\tmessages?: Array<{ content?: unknown; text?: string }>;\n\t\ttype?: string;\n\t\tpayload?: unknown;\n\t}",
|
||||
"mode: PromptInjectionMode",
|
||||
],
|
||||
returns: "ReinjectDecision",
|
||||
calls: ["outgoingMessagesHaveMarker"],
|
||||
raises: [],
|
||||
},
|
||||
],
|
||||
},
|
||||
);
|
||||
|
||||
const funcExport = result.exports.find((e) =>
|
||||
e.startsWith("func:shouldReinjectForEvent"),
|
||||
);
|
||||
expect(funcExport).toBeDefined();
|
||||
expect(funcExport).not.toContain("\n");
|
||||
expect(funcExport).not.toContain("\t");
|
||||
expect(funcExport).toBe(
|
||||
"func:shouldReinjectForEvent(event: { messages?: Array<{ content?: unknown; text?: string }>; type?: string; payload?: unknown; }, mode: PromptInjectionMode) → ReinjectDecision",
|
||||
);
|
||||
});
|
||||
|
||||
it("collapses multi-line return types into single-line func signatures", () => {
|
||||
const result = mergeFileData(
|
||||
"types.ts",
|
||||
{ purpose: "Types", exports: [], deps: [] },
|
||||
{
|
||||
exports: ["complexReturn"],
|
||||
deps: [],
|
||||
classes: [],
|
||||
functions: [
|
||||
{
|
||||
name: "complexReturn",
|
||||
params: ["x: number"],
|
||||
returns: "{\n a: string;\n b: number;\n}",
|
||||
calls: [],
|
||||
raises: [],
|
||||
},
|
||||
],
|
||||
},
|
||||
);
|
||||
|
||||
const funcExport = result.exports.find((e) =>
|
||||
e.startsWith("func:complexReturn"),
|
||||
);
|
||||
expect(funcExport).toBeDefined();
|
||||
expect(funcExport).not.toContain("\n");
|
||||
expect(funcExport).toBe(
|
||||
"func:complexReturn(x: number) → { a: string; b: number; }",
|
||||
);
|
||||
});
|
||||
|
||||
it("collapses multi-line method parameters into single-line method signatures", () => {
|
||||
const result = mergeFileData(
|
||||
"class.ts",
|
||||
{ purpose: "Class file", exports: [], deps: [] },
|
||||
{
|
||||
exports: ["MyClass"],
|
||||
deps: [],
|
||||
classes: [
|
||||
{
|
||||
name: "MyClass",
|
||||
methods: [
|
||||
{
|
||||
name: "doThing",
|
||||
params: ["opts: {\n a: string;\n b: number;\n}"],
|
||||
returns: "void",
|
||||
calls: [],
|
||||
raises: [],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
functions: [],
|
||||
},
|
||||
);
|
||||
|
||||
const methodExport = result.exports.find((e) =>
|
||||
e.startsWith("method:doThing"),
|
||||
);
|
||||
expect(methodExport).toBeDefined();
|
||||
expect(methodExport).not.toContain("\n");
|
||||
expect(methodExport).toBe(
|
||||
"method:doThing(opts: { a: string; b: number; }) → void",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
import type { LLMClient } from "../src/llm/llm-client.js";
|
||||
|
||||
export function createMockFileClient(purpose = "Test file"): LLMClient {
|
||||
return {
|
||||
async complete() {
|
||||
return `PURPOSE: ${purpose}\nDEPS: none\nCONCEPTS: testing`;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function createMockPackageClient(): LLMClient {
|
||||
return {
|
||||
async complete() {
|
||||
return "ROLE: Test package\nARCH: Test architecture";
|
||||
},
|
||||
};
|
||||
}
|
||||
+1020
-12
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,244 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from "vitest";
|
||||
import { mkdtempSync, writeFileSync, mkdirSync, rmSync } from "fs";
|
||||
import { join } from "path";
|
||||
import { tmpdir } from "os";
|
||||
import { retrieveContext } from "../src/retrieve.js";
|
||||
|
||||
describe("retrieve", () => {
|
||||
let dir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(join(tmpdir(), "pi-ret-"));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(dir, { recursive: true });
|
||||
});
|
||||
|
||||
function writeMap(
|
||||
relDir: string,
|
||||
role: string,
|
||||
files: { name: string; purpose: string; exports?: string[] }[],
|
||||
tags?: string[],
|
||||
symbols?: string[],
|
||||
) {
|
||||
const d = join(dir, relDir);
|
||||
if (!d.startsWith(dir)) throw new Error("invalid path");
|
||||
mkdirSync(d, { recursive: true });
|
||||
const fileLines = files
|
||||
.map((f) => {
|
||||
const exp = f.exports?.length ? ` | exp: ${f.exports.join(", ")}` : "";
|
||||
return `- ${f.name} | ${f.purpose}${exp}`;
|
||||
})
|
||||
.join("\n");
|
||||
const tagLine = tags?.length ? tags.join(", ") : "-";
|
||||
const symLines = symbols?.length
|
||||
? symbols.map((s) => `- ${s}`).join("\n")
|
||||
: "-";
|
||||
writeFileSync(
|
||||
join(d, ".pi-map.md"),
|
||||
`# ${relDir}
|
||||
dir: ${relDir}
|
||||
|
||||
index: ${relDir}/.pi-map.index.md
|
||||
|
||||
## role
|
||||
${role}
|
||||
|
||||
## files
|
||||
${fileLines}
|
||||
|
||||
## arch
|
||||
Test arch
|
||||
|
||||
## tags
|
||||
${tagLine}
|
||||
|
||||
## symbols
|
||||
${symLines}
|
||||
|
||||
## workflows
|
||||
-
|
||||
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(d, ".pi-map.index.md"),
|
||||
`# ${relDir} (index)
|
||||
dir: ${relDir}
|
||||
|
||||
## role
|
||||
${role}
|
||||
|
||||
## parent
|
||||
-
|
||||
|
||||
## children
|
||||
-
|
||||
|
||||
## files
|
||||
${files.map((f) => `- ${f.name}`).join("\n")}
|
||||
|
||||
## links
|
||||
index: ${relDir}/.pi-map.index.md
|
||||
map: ${relDir}/.pi-map.md
|
||||
|
||||
## workflows
|
||||
-
|
||||
|
||||
## dirty
|
||||
-
|
||||
`,
|
||||
);
|
||||
}
|
||||
|
||||
it("returns empty bundle when no maps exist", () => {
|
||||
const bundle = retrieveContext("auth", dir);
|
||||
expect(bundle).toContain("# Context bundle: auth");
|
||||
expect(bundle).toContain("No relevant directories found");
|
||||
});
|
||||
|
||||
it("ranks directories by query relevance", () => {
|
||||
writeMap(
|
||||
"src/auth",
|
||||
"Auth layer: JWT issuance, validation, refresh.",
|
||||
[{ name: "tokens.ts", purpose: "JWT gen/val", exports: ["issueToken"] }],
|
||||
["auth", "jwt"],
|
||||
["issueToken"],
|
||||
);
|
||||
writeMap(
|
||||
"src/utils",
|
||||
"Shared utilities and helpers.",
|
||||
[{ name: "helpers.ts", purpose: "Helpers" }],
|
||||
["utils"],
|
||||
);
|
||||
|
||||
const bundle = retrieveContext("jwt validation", dir);
|
||||
expect(bundle).toContain("src/auth/.pi-map.index.md");
|
||||
expect(bundle).toContain("src/auth/.pi-map.md");
|
||||
});
|
||||
|
||||
it("limits results to top 3 by default", () => {
|
||||
for (let i = 0; i < 5; i++) {
|
||||
writeMap(
|
||||
`src/pkg${i}`,
|
||||
`Package ${i} logic`,
|
||||
[{ name: `file${i}.ts`, purpose: `Feature ${i}` }],
|
||||
[`pkg${i}`],
|
||||
);
|
||||
}
|
||||
|
||||
// Query matching all
|
||||
const bundle = retrieveContext("pkg", dir);
|
||||
const indexMatches = bundle.match(/\.pi-map\.index\.md/g) || [];
|
||||
expect(indexMatches.length).toBeLessThanOrEqual(3);
|
||||
});
|
||||
|
||||
it("includes likely files from matched directories", () => {
|
||||
writeMap(
|
||||
"src/auth",
|
||||
"Auth layer",
|
||||
[
|
||||
{ name: "tokens.ts", purpose: "JWT tokens", exports: ["issueToken"] },
|
||||
{
|
||||
name: "middleware.ts",
|
||||
purpose: "Auth middleware",
|
||||
exports: ["requireAuth"],
|
||||
},
|
||||
],
|
||||
["auth"],
|
||||
["issueToken", "requireAuth"],
|
||||
);
|
||||
|
||||
const bundle = retrieveContext("auth middleware", dir);
|
||||
expect(bundle).toContain("src/auth/tokens.ts");
|
||||
expect(bundle).toContain("src/auth/middleware.ts");
|
||||
});
|
||||
|
||||
it("includes relevant symbols when present", () => {
|
||||
writeMap(
|
||||
"src/auth",
|
||||
"Auth layer",
|
||||
[{ name: "tokens.ts", purpose: "JWT tokens" }],
|
||||
["auth"],
|
||||
["issueToken", "verifyToken"],
|
||||
);
|
||||
|
||||
const bundle = retrieveContext("auth", dir);
|
||||
expect(bundle).toContain("## relevant symbols");
|
||||
expect(bundle).toContain("issueToken");
|
||||
expect(bundle).toContain("verifyToken");
|
||||
});
|
||||
|
||||
it("omits symbol section when no symbols exist", () => {
|
||||
writeMap(
|
||||
"src/utils",
|
||||
"Utilities",
|
||||
[{ name: "helpers.ts", purpose: "Helpers" }],
|
||||
["utils"],
|
||||
[],
|
||||
);
|
||||
|
||||
const bundle = retrieveContext("utils", dir);
|
||||
expect(bundle).not.toContain("## relevant symbols");
|
||||
});
|
||||
|
||||
it("includes instructions in every bundle", () => {
|
||||
writeMap(
|
||||
"src/cli",
|
||||
"CLI entrypoint",
|
||||
[{ name: "cli.ts", purpose: "CLI" }],
|
||||
["cli"],
|
||||
);
|
||||
|
||||
const bundle = retrieveContext("cli", dir);
|
||||
expect(bundle).toContain("## instructions");
|
||||
expect(bundle).toContain("Read the indexes first");
|
||||
});
|
||||
|
||||
it("uses stable section order", () => {
|
||||
writeMap(
|
||||
"src/auth",
|
||||
"Auth layer",
|
||||
[{ name: "tokens.ts", purpose: "JWT tokens" }],
|
||||
["auth"],
|
||||
["issueToken"],
|
||||
);
|
||||
|
||||
const bundle = retrieveContext("auth", dir);
|
||||
const queryIdx = bundle.indexOf("## query");
|
||||
const indexIdx = bundle.indexOf("## relevant indexes");
|
||||
const mapIdx = bundle.indexOf("## relevant maps");
|
||||
const fileIdx = bundle.indexOf("## likely files");
|
||||
const symIdx = bundle.indexOf("## relevant symbols");
|
||||
const instIdx = bundle.indexOf("## instructions");
|
||||
|
||||
expect(queryIdx).toBeGreaterThan(-1);
|
||||
expect(indexIdx).toBeGreaterThan(queryIdx);
|
||||
expect(mapIdx).toBeGreaterThan(indexIdx);
|
||||
expect(fileIdx).toBeGreaterThan(mapIdx);
|
||||
expect(symIdx).toBeGreaterThan(fileIdx);
|
||||
expect(instIdx).toBeGreaterThan(symIdx);
|
||||
});
|
||||
|
||||
it("scores file names and purposes", () => {
|
||||
writeMap(
|
||||
"src/auth",
|
||||
"Auth layer",
|
||||
[
|
||||
{
|
||||
name: "validateToken.ts",
|
||||
purpose: "Token validation logic",
|
||||
exports: ["validateToken"],
|
||||
},
|
||||
],
|
||||
[],
|
||||
[],
|
||||
);
|
||||
|
||||
const bundle = retrieveContext("validate token", dir);
|
||||
expect(bundle).toContain("src/auth/.pi-map.index.md");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,219 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { createDirectoryModel } from "../src/directory-model.js";
|
||||
import { populateRoutingMetadata } from "../src/routing-metadata.js";
|
||||
|
||||
describe("routing metadata", () => {
|
||||
it("generates tags from file purposes and exports", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/auth",
|
||||
role: "Auth layer",
|
||||
files: [
|
||||
{
|
||||
name: "tokens.ts",
|
||||
purpose: "JWT generation and validation",
|
||||
exports: ["issueToken", "verifyToken"],
|
||||
deps: ["crypto"],
|
||||
},
|
||||
],
|
||||
arch: "Auth",
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
expect(model.tags.length).toBeGreaterThan(0);
|
||||
expect(model.tags).toContain("jwt");
|
||||
expect(model.tags).toContain("token");
|
||||
});
|
||||
|
||||
it("generates symbols from exports", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/utils",
|
||||
role: "Utilities",
|
||||
files: [
|
||||
{
|
||||
name: "helpers.ts",
|
||||
purpose: "Helpers",
|
||||
exports: ["formatDate", "parseUrl", "formatDate"],
|
||||
deps: [],
|
||||
},
|
||||
],
|
||||
arch: "Utils",
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
expect(model.symbols.length).toBeGreaterThan(0);
|
||||
expect(model.symbols).toContain("formatDate");
|
||||
expect(model.symbols).toContain("parseUrl");
|
||||
});
|
||||
|
||||
it("caps tags at default limit", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/big",
|
||||
role: "Big module",
|
||||
files: Array.from({ length: 20 }, (_, i) => ({
|
||||
name: `file${i}.ts`,
|
||||
purpose: `Purpose ${i} with many unique words ${i}`,
|
||||
exports: [`export${i}A`, `export${i}B`],
|
||||
deps: [`dep${i}`],
|
||||
})),
|
||||
arch: "Big",
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
expect(model.tags.length).toBeLessThanOrEqual(8);
|
||||
});
|
||||
|
||||
it("caps symbols at default limit", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/big",
|
||||
role: "Big module",
|
||||
files: Array.from({ length: 20 }, (_, i) => ({
|
||||
name: `file${i}.ts`,
|
||||
purpose: `Purpose ${i}`,
|
||||
exports: [`export${i}A`, `export${i}B`],
|
||||
deps: [],
|
||||
})),
|
||||
arch: "Big",
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
expect(model.symbols.length).toBeLessThanOrEqual(8);
|
||||
});
|
||||
|
||||
it("generates workflow hints for source directories", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/cli",
|
||||
role: "CLI",
|
||||
files: [
|
||||
{
|
||||
name: "cli.ts",
|
||||
purpose: "CLI entrypoint",
|
||||
exports: ["main"],
|
||||
deps: [],
|
||||
},
|
||||
{
|
||||
name: "cli.test.ts",
|
||||
purpose: "CLI tests",
|
||||
exports: [],
|
||||
deps: [],
|
||||
},
|
||||
],
|
||||
arch: "CLI",
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
expect(model.workflows.length).toBeGreaterThan(0);
|
||||
const changeBehavior = model.workflows.find((w) =>
|
||||
w.task.includes("change"),
|
||||
);
|
||||
expect(changeBehavior).toBeDefined();
|
||||
});
|
||||
|
||||
it("omits workflows for non-source directories", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "docs",
|
||||
role: "Documentation",
|
||||
files: [
|
||||
{
|
||||
name: "README.md",
|
||||
purpose: "Readme",
|
||||
exports: [],
|
||||
deps: [],
|
||||
},
|
||||
],
|
||||
arch: "Docs",
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
expect(model.workflows).toEqual([]);
|
||||
});
|
||||
|
||||
it("caps workflow hints at default limit", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/cli",
|
||||
role: "CLI",
|
||||
files: [
|
||||
{ name: "cli.ts", purpose: "CLI", exports: ["main"], deps: [] },
|
||||
{ name: "config.ts", purpose: "Config", exports: [], deps: [] },
|
||||
{ name: "cli.test.ts", purpose: "Tests", exports: [], deps: [] },
|
||||
{ name: "commands.ts", purpose: "Commands", exports: [], deps: [] },
|
||||
],
|
||||
arch: "CLI",
|
||||
});
|
||||
model.children = ["src/cli/sub"];
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
expect(model.workflows.length).toBeLessThanOrEqual(5);
|
||||
});
|
||||
|
||||
it("workflow hints include read targets for relevant files", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/auth",
|
||||
role: "Auth",
|
||||
files: [
|
||||
{
|
||||
name: "tokens.ts",
|
||||
purpose: "Tokens",
|
||||
exports: ["issueToken"],
|
||||
deps: [],
|
||||
},
|
||||
{
|
||||
name: "tokens.test.ts",
|
||||
purpose: "Token tests",
|
||||
exports: [],
|
||||
deps: [],
|
||||
},
|
||||
],
|
||||
arch: "Auth",
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
const testWorkflow = model.workflows.find((w) => w.task.includes("test"));
|
||||
expect(testWorkflow).toBeDefined();
|
||||
expect(testWorkflow!.read).toContain("tokens.test.ts");
|
||||
});
|
||||
|
||||
it("workflow hints include index targets for directories with children", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src",
|
||||
role: "Source",
|
||||
files: [
|
||||
{
|
||||
name: "index.ts",
|
||||
purpose: "Exports",
|
||||
exports: [],
|
||||
deps: [],
|
||||
},
|
||||
],
|
||||
arch: "Source",
|
||||
children: ["src/auth", "src/cli"],
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model);
|
||||
const exploreWorkflow = model.workflows.find((w) =>
|
||||
w.task.includes("explore"),
|
||||
);
|
||||
expect(exploreWorkflow).toBeDefined();
|
||||
expect(exploreWorkflow!.index).toContain("src/auth/.pi-map.index.md");
|
||||
expect(exploreWorkflow!.index).toContain("src/cli/.pi-map.index.md");
|
||||
});
|
||||
|
||||
it("respects explicit tagCap and workflowHintCap options", () => {
|
||||
const model = createDirectoryModel({
|
||||
dir: "src/big",
|
||||
role: "Big module",
|
||||
files: Array.from({ length: 20 }, (_, i) => ({
|
||||
name: `file${i}.ts`,
|
||||
purpose: `Purpose ${i} with many unique words ${i}`,
|
||||
exports: [`export${i}A`, `export${i}B`],
|
||||
deps: [`dep${i}`],
|
||||
})),
|
||||
arch: "Big",
|
||||
children: ["src/big/sub"],
|
||||
});
|
||||
|
||||
populateRoutingMetadata(model, { tagCap: 3, workflowHintCap: 2 });
|
||||
expect(model.tags.length).toBeLessThanOrEqual(3);
|
||||
expect(model.symbols.length).toBeLessThanOrEqual(3);
|
||||
expect(model.workflows.length).toBeLessThanOrEqual(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,185 @@
|
||||
# Troubleshooting pi-project-map
|
||||
|
||||
## Stale map issues
|
||||
|
||||
### Symptom
|
||||
`project_map_validate` reports `missing`, `orphaned`, or `stale-signature` discrepancies, or a directory map describes files that no longer exist.
|
||||
|
||||
### Likely cause
|
||||
A source file was edited, added, or deleted without running `project_map_patch` or `project_map_reinit`.
|
||||
|
||||
### What to do
|
||||
1. run `project_map_validate` to inspect discrepancies
|
||||
2. if the list is small and localized, run `project_map_patch <changed-file>`
|
||||
3. if the list is large or structural, run `project_map_reinit`
|
||||
4. re-run `project_map_validate` to confirm clean state
|
||||
|
||||
### Prevention
|
||||
- patch immediately after editing source
|
||||
- in Pi, use the `project_map_patch` tool with the changed file path
|
||||
- watch for the `session_start` dirty warning
|
||||
|
||||
## Validate discrepancies
|
||||
|
||||
### Symptom
|
||||
`validate --fix` fails with `validate --fix requires an LLM client`.
|
||||
|
||||
### Cause
|
||||
Repair needs an LLM client to regenerate stale artifacts.
|
||||
|
||||
### Fix
|
||||
- CLI: provide `--llm-provider=...` and the required API key
|
||||
- Pi: use the tool surface so the runtime provides the LLM client
|
||||
|
||||
### Symptom
|
||||
Many `stale-signature` discrepancies appear.
|
||||
|
||||
### Cause
|
||||
AST exports no longer match listed exports, or generated signatures are stale.
|
||||
|
||||
### Fix
|
||||
- patch the changed file/directory
|
||||
- if widespread, reinit and validate again
|
||||
|
||||
### Symptom
|
||||
`broken-link` appears after moving directories.
|
||||
|
||||
### Fix
|
||||
Run `project_map_reinit`.
|
||||
|
||||
## Prompt injection mode surprises
|
||||
|
||||
### Symptom
|
||||
No project-map context appears in a Pi session.
|
||||
|
||||
### Checks before init
|
||||
1. confirm `.pi-project-map.json` does not set `promptInjectionMode` to `off`
|
||||
2. confirm root `.pi-map.md` / `.pi-map.index.md` do not exist yet
|
||||
3. confirm the session is at agent start; the pre-init hint is emitted from `before_agent_start`
|
||||
|
||||
### Checks after init
|
||||
1. confirm root `.pi-map.md` and `.pi-map.index.md` exist
|
||||
2. confirm mode is `strong` or `strict` if you expect automatic artifact injection
|
||||
3. confirm the outgoing context does not already contain the root-pair marker
|
||||
4. confirm the turn is actually relevant (`agent_start`, `edit_intent`, `architecture_sensitive`, `compaction`, `artifact_change`)
|
||||
|
||||
### Symptom
|
||||
Root pair is injected repeatedly.
|
||||
|
||||
### Likely cause
|
||||
Marker scanning failed or artifact invalidation forced reinjection.
|
||||
|
||||
### Fix
|
||||
- inspect whether `<!-- PI_MAP_ROOT_PAIR_START -->` is present in outgoing context
|
||||
- check whether root artifacts changed on disk
|
||||
- confirm payload/message serialization still exposes marker text
|
||||
|
||||
### Symptom
|
||||
`advisory` mode shows a reminder but no maps are loaded.
|
||||
|
||||
### Expected behavior
|
||||
That is intentional. Advisory mode is reminder-only; read the root pair manually.
|
||||
|
||||
## Strict bypass confusion
|
||||
|
||||
### Symptom
|
||||
A sensitive edit is blocked with the strict guard.
|
||||
|
||||
### What is happening
|
||||
In `strict` mode, the turn was classified as sensitive and the protocol path is missing from outgoing context.
|
||||
|
||||
### Resolution options
|
||||
1. restore the protocol path by ensuring the root pair and trust boundary are present in context
|
||||
2. include an explicit bypass marker:
|
||||
|
||||
```text
|
||||
[PI_MAP_BYPASS: emergency fix needed without full context]
|
||||
```
|
||||
|
||||
3. switch to `strong` mode if strict is too noisy for the task
|
||||
|
||||
### Symptom
|
||||
Bypass marker is ignored.
|
||||
|
||||
### Cause
|
||||
The marker must contain a non-empty reason. Empty or whitespace-only reasons are rejected.
|
||||
|
||||
### Symptom
|
||||
Strict guard triggers on obviously non-sensitive turns.
|
||||
|
||||
### Cause
|
||||
Heuristic sensitive-turn detection can false-positive on words like `refactor`, `redesign`, or `dependency graph`.
|
||||
|
||||
## LLM / provider / cache issues
|
||||
|
||||
### Symptom
|
||||
`project-map init` fails with `No LLM client configured`.
|
||||
|
||||
### CLI fix
|
||||
- set `OPENAI_API_KEY` for OpenAI provider
|
||||
- set `KIMI_API_KEY` for Kimi provider
|
||||
- or configure provider/model in `.pi-project-map.json`
|
||||
|
||||
### Pi fix
|
||||
- select a model in Pi
|
||||
- log in if the provider requires credentials
|
||||
- ensure Pi runtime LLM access is working
|
||||
|
||||
### Symptom
|
||||
External/provider request fails.
|
||||
|
||||
### Checks
|
||||
1. API key validity
|
||||
2. network connectivity
|
||||
3. `llmBaseUrl` correctness if using a proxy
|
||||
4. provider rate limits
|
||||
|
||||
### Symptom
|
||||
Second init still makes many LLM calls.
|
||||
|
||||
### Cause
|
||||
Cache miss.
|
||||
|
||||
### Checks
|
||||
- consistent cache directory between runs
|
||||
- cache file exists and is readable
|
||||
- file contents did not change
|
||||
|
||||
### Symptom
|
||||
Cache file is huge.
|
||||
|
||||
### Cause
|
||||
There is no eviction policy yet.
|
||||
|
||||
### Fix
|
||||
Remove the local cache directory if needed.
|
||||
|
||||
## Test / lint expectations
|
||||
|
||||
### Main checks
|
||||
|
||||
```bash
|
||||
npm test
|
||||
npm run typecheck
|
||||
npm run build
|
||||
node dist/cli.js validate .
|
||||
```
|
||||
|
||||
### Lint note
|
||||
`npm run lint` is currently not a reliable gate if the repo has no ESLint config. Treat missing-lint-config failures as repository hygiene issues, not feature regressions.
|
||||
|
||||
### Common failures
|
||||
|
||||
| Failure | Likely cause |
|
||||
|---------|--------------|
|
||||
| Pi LLM client tests fail | Pi runtime or mocks drifted |
|
||||
| Prompt-injection sequence tests fail | Reinjection / strict logic changed |
|
||||
| Validate tests fail | Format parsing, merge encoding, or discovery rules changed |
|
||||
| AST tests fail | tree-sitter grammar/runtime drift |
|
||||
|
||||
## Known limitations
|
||||
|
||||
- token budgeting is heuristic, not tokenizer-exact
|
||||
- retrieval is deterministic, not semantic-search-driven
|
||||
- strict mode is a prompt-level guard, not a hard tool sandbox
|
||||
- payload scanning assumes string or JSON-serializable structures
|
||||
+198
@@ -0,0 +1,198 @@
|
||||
# pi-project-map Usage Guide
|
||||
|
||||
For Pi agents and advanced users who want predictable, low-friction navigation and maintenance of project-map artifacts.
|
||||
|
||||
## When to use each command
|
||||
|
||||
| Situation | Command |
|
||||
|-----------|---------|
|
||||
| New project, no `.pi-map.md` files yet, or artifacts are severely outdated | `project_map_init` / `project-map init` |
|
||||
| You just edited one or more source files | `project_map_patch <file>` / `project-map patch <file>` |
|
||||
| You suspect stale data, or you are about to make an architectural decision | `project_map_validate` / `project-map validate` |
|
||||
| Validation shows widespread staleness, or you pulled major changes from version control | `project_map_reinit` / `project-map reinit` |
|
||||
| You have a specific question like "where is auth handled?" | `project_map_context <query>` / `project-map context <query>` |
|
||||
|
||||
## Command behavior
|
||||
|
||||
### init
|
||||
Run once when you start work on a repo, or after large restructuring. It discovers every non-ignored directory, analyzes files with LLM + AST, and writes both `.pi-map.md` and `.pi-map.index.md` for every directory.
|
||||
|
||||
### patch
|
||||
Run **immediately after editing a source file**. The command regenerates the pair for that file’s directory and refreshes ancestor artifacts.
|
||||
|
||||
Patch mode is chosen automatically:
|
||||
- **small** — refresh ancestor indexes only
|
||||
- **structural** — refresh ancestor map/index pairs
|
||||
|
||||
You can force a mode with `project-map patch <file> --patch-mode=small|structural`.
|
||||
`auto` remains the default.
|
||||
|
||||
### validate
|
||||
Run before architectural decisions, broad refactors, or final handoff. It checks for:
|
||||
- missing/orphaned files
|
||||
- stale signatures
|
||||
- dirty markers
|
||||
- broken parent/children/sibling links
|
||||
- map/index disagreements
|
||||
|
||||
Use `project-map validate --fix` to repair affected chains. `--fix` requires an LLM client.
|
||||
|
||||
### reinit
|
||||
Use sparingly. It regenerates every pair from scratch and is the blunt instrument for widespread staleness.
|
||||
|
||||
### context
|
||||
Use when you know what you are looking for:
|
||||
|
||||
```bash
|
||||
project-map context "authentication logic"
|
||||
project-map context "routing metadata generation"
|
||||
project-map context "LLM client error handling"
|
||||
```
|
||||
|
||||
The returned bundle is deterministic and ranked. Read indexes first, then strongest-match maps, then the actual source files.
|
||||
|
||||
## Practical workflows
|
||||
|
||||
### Starting a new task in a mapped project
|
||||
|
||||
1. read the root `.pi-map.index.md` and the `Project Map Protocol`
|
||||
2. use the root index to find the relevant child directory
|
||||
3. read that directory’s `.pi-map.index.md`, then its `.pi-map.md`
|
||||
4. read the relevant source files
|
||||
5. edit source
|
||||
6. run `project_map_patch <changed-file>`
|
||||
7. run tests/build
|
||||
8. run `project_map_validate` before architectural summary or handoff
|
||||
|
||||
### Exploring an unfamiliar area
|
||||
|
||||
```bash
|
||||
project-map context "how is validation implemented"
|
||||
```
|
||||
|
||||
Treat the returned bundle as a ranked entry point, not as truth.
|
||||
|
||||
### After pulling changes from version control
|
||||
|
||||
```bash
|
||||
project-map validate
|
||||
# if many discrepancies:
|
||||
project-map reinit
|
||||
```
|
||||
|
||||
### Before a big refactor
|
||||
|
||||
```bash
|
||||
project-map validate
|
||||
# if needed:
|
||||
project-map reinit
|
||||
```
|
||||
|
||||
## Prompt injection behavior before and after init
|
||||
|
||||
### Before init
|
||||
If no paired artifacts exist:
|
||||
- `off`: nothing happens
|
||||
- `advisory`, `strong`, `strict`: a visible hint appears telling you to run `project_map_init`
|
||||
|
||||
No map content is fabricated.
|
||||
|
||||
### After init
|
||||
Once the root pair exists:
|
||||
- `off`: no automatic injection
|
||||
- `advisory`: a visible reminder that maps are available, but the root pair is **not** auto-loaded
|
||||
- `strong`: root pair is auto-loaded; additional pairs are added within the budget; reinjection happens on relevant turns
|
||||
- `strict`: same as `strong`, plus enforcement of the protocol path for sensitive actions
|
||||
|
||||
Relevant turns that trigger reinjection in `strong`/`strict`:
|
||||
- agent start
|
||||
- edit intent
|
||||
- architecture-sensitive reasoning
|
||||
- compaction
|
||||
- root-pair artifact changes
|
||||
|
||||
## Advisory vs strong vs strict in practice
|
||||
|
||||
| Concern | Use |
|
||||
|---------|-----|
|
||||
| You want maps available but do not want automatic context expansion | `advisory` |
|
||||
| Normal daily work; you want routing/orientation preloaded without friction | `strong` (default) |
|
||||
| High-stakes codebase or you want explicit justification before bypassing context discipline | `strict` |
|
||||
| You prefer fully manual control | `off` |
|
||||
|
||||
In `strict`, if you attempt a sensitive edit or architectural claim without the protocol path in context, a guard appears. To proceed, either restore the project-map context or include:
|
||||
|
||||
```text
|
||||
[PI_MAP_BYPASS: editing a one-line comment, map context not needed]
|
||||
```
|
||||
|
||||
Use bypass markers sparingly.
|
||||
|
||||
## Example task flows
|
||||
|
||||
### Fix a bug in `src/utils/validation.ts`
|
||||
|
||||
```text
|
||||
1. project-map context "validation utilities"
|
||||
2. Read src/utils/.pi-map.index.md and .pi-map.md
|
||||
3. Read src/utils/validation.ts
|
||||
4. Edit the file
|
||||
5. project_map_patch src/utils/validation.ts
|
||||
6. Run tests
|
||||
7. project_map_validate
|
||||
```
|
||||
|
||||
### Add a new file to `src/llm/`
|
||||
|
||||
```text
|
||||
1. Create src/llm/new-client.ts
|
||||
2. Implement the file
|
||||
3. project_map_patch src/llm/new-client.ts
|
||||
4. project_map_validate
|
||||
```
|
||||
|
||||
### Review architecture before approving a PR
|
||||
|
||||
```text
|
||||
1. project-map validate
|
||||
2. If clean, read root .pi-map.md and key directory maps
|
||||
3. Cross-check claims against source
|
||||
4. If stale, run project-map reinit first
|
||||
```
|
||||
|
||||
## Retrieval vs automatic injection
|
||||
|
||||
| | Automatic injection | `project_map_context` / `project-map context` |
|
||||
|---|---|---|
|
||||
| Trigger | Configured mode + relevant turn | Explicit request |
|
||||
| Content | Root pair + budgeted expansion | Top-ranked directories for a query |
|
||||
| Cost | No LLM call; reads artifacts | No LLM call; deterministic scoring |
|
||||
| Best use | Maintain baseline orientation | Targeted navigation for a specific task |
|
||||
|
||||
Use both together: injection for baseline orientation, retrieval for focused entry points.
|
||||
|
||||
## Keeping artifacts fresh
|
||||
|
||||
- patch after every edit
|
||||
- validate before architectural claims
|
||||
- reinit when many artifacts are stale or after large merges
|
||||
- watch for the Pi extension warning about dirty packages on session start
|
||||
|
||||
## Configuration quick reference
|
||||
|
||||
```json
|
||||
{
|
||||
"promptInjectionMode": "strong",
|
||||
"contextBudgetPercent": 15,
|
||||
"contextBudgetMaxTokens": 100000,
|
||||
"tagCap": 8,
|
||||
"workflowHintCap": 5,
|
||||
"ignorePatterns": ["node_modules", ".git", "dist", "build"]
|
||||
}
|
||||
```
|
||||
|
||||
Providing `ignorePatterns` replaces the built-in default list, so include any defaults you want to keep.
|
||||
|
||||
- lower `contextBudgetPercent` / `contextBudgetMaxTokens` to reduce token use
|
||||
- raise them if you want deeper auto-loaded context in large projects
|
||||
- `strict` is the safest enforcement mode; `strong` is the best default for everyday work
|
||||
Reference in New Issue
Block a user