Initial commit: pi-project-map skill scaffolding
- Design doc with full spec (dense markdown format, hybrid AST+LLM pipeline, consumption model, stale data mitigation) - Implementation plan with 6 milestones and rollout strategy - TypeScript package structure with all source stubs - CLI entry point, formatter, discover, init, patch, validate, extract, merge - Pi SKILL.md with tool definitions and format documentation
This commit is contained in:
@@ -0,0 +1,6 @@
|
|||||||
|
node_modules/
|
||||||
|
dist/
|
||||||
|
coverage/
|
||||||
|
*.log
|
||||||
|
.DS_Store
|
||||||
|
.env
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
# pi-project-map
|
||||||
|
|
||||||
|
Pi skill for hierarchical project analysis.
|
||||||
|
|
||||||
|
## What it does
|
||||||
|
|
||||||
|
Generates `.pi-map.md` files throughout your project — one per directory — containing a dense, machine-readable summary of that directory's files, exports, dependencies, and architecture. This gives Pi agents instant project comprehension without reading every source file.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install -g pi-project-map
|
||||||
|
project-map init
|
||||||
|
```
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
See [design-doc.md](design-doc.md) for the full specification.
|
||||||
|
|
||||||
|
## Implementation Plan
|
||||||
|
|
||||||
|
See [implementation-plan.md](implementation-plan.md) for the engineering roadmap.
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# pi-project-map
|
||||||
|
|
||||||
|
A Pi skill that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file containing architectural context, exported symbols, and dependencies.
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
### `project-map:init [root]`
|
||||||
|
Runs a full project scan and generates `.pi-map.md` files in every directory.
|
||||||
|
|
||||||
|
### `project-map:patch <file-path>`
|
||||||
|
Updates the `.pi-map.md` for the directory containing the given file. Uses full rewrite for small packages (< 10 files) or section-level patch for larger packages.
|
||||||
|
|
||||||
|
### `project-map:validate [root]`
|
||||||
|
Checks all `.pi-map.md` files for staleness: missing files, orphaned entries, changed signatures, and dirty markers.
|
||||||
|
|
||||||
|
### `project-map:reinit [path]`
|
||||||
|
Force full re-initialization of the entire project or a specific subtree. Clears all dirty markers.
|
||||||
|
|
||||||
|
## Format
|
||||||
|
|
||||||
|
Each `.pi-map.md` uses dense markdown with conventions:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# pkg/auth
|
||||||
|
## role
|
||||||
|
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
|
||||||
|
## files
|
||||||
|
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
|
||||||
|
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
|
||||||
|
## arch
|
||||||
|
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
|
||||||
|
## dirty
|
||||||
|
-
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Create `.pi-project-map.json` in the project root:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"ignorePatterns": ["node_modules", ".git"],
|
||||||
|
"smallPackageThreshold": 10,
|
||||||
|
"llmModel": "gpt-4o-mini",
|
||||||
|
"contextBudget": 4000,
|
||||||
|
"autoInjectPrompt": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install -g pi-project-map
|
||||||
|
```
|
||||||
|
|
||||||
|
Then add to your Pi skills configuration.
|
||||||
+258
@@ -0,0 +1,258 @@
|
|||||||
|
# Design Doc: Hierarchical Project Analysis Skill for Pi
|
||||||
|
|
||||||
|
## 1. Goals and Success Criteria
|
||||||
|
|
||||||
|
### Primary Goal
|
||||||
|
Enable a Pi coding agent to understand a software project's architecture and code relationships without scanning the entire repository. The agent should have a compact, hierarchical "internal representation" of the project that it can consume in-context.
|
||||||
|
|
||||||
|
### Success Criteria
|
||||||
|
- The agent can orient itself in a new or familiar project without reading dozens of source files.
|
||||||
|
- The agent understands cross-package dependencies, data flows, and architectural patterns from the analysis files alone.
|
||||||
|
- Analysis files stay sufficiently fresh that the agent does not make decisions based on stale information.
|
||||||
|
- The representation is token-dense: maximum information per token, optimized for LLM consumption, not human readability.
|
||||||
|
|
||||||
|
## 2. Format Specification: Dense Markdown with Conventions
|
||||||
|
|
||||||
|
### Design Rationale
|
||||||
|
- **Not JSON/YAML**: Brackets, quotes, and indentation add token overhead with no benefit to LLM comprehension.
|
||||||
|
- **Not a custom DSL**: Fragile, requires a parser, and LLMs may hallucinate syntax.
|
||||||
|
- **Dense markdown**: Hierarchical headings, bullet points, and abbreviations are natively understood by LLMs and extremely token-efficient.
|
||||||
|
|
||||||
|
### Structure
|
||||||
|
Each directory in the project gets one analysis file named `.pi-map.md` (hidden by default, excluded from git via `.gitignore`).
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# <relative-path>
|
||||||
|
## role
|
||||||
|
<one-line package role> | Dep: <comma-separated upstream deps>
|
||||||
|
## files
|
||||||
|
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
|
||||||
|
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
|
||||||
|
## arch
|
||||||
|
<free-form architectural notes: patterns, data flow, invariants, design decisions>
|
||||||
|
## dirty
|
||||||
|
<timestamp or flag indicating staleness>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Abbreviation Conventions
|
||||||
|
| Abbreviation | Meaning |
|
||||||
|
|-------------|---------|
|
||||||
|
| `exp:` | exported symbols (functions, classes, types, constants) |
|
||||||
|
| `dep:` | dependencies (other packages, files, or external libs) |
|
||||||
|
| `pkg/` | project-internal package reference |
|
||||||
|
| `ext/` | external dependency reference |
|
||||||
|
| `->` | data flow direction |
|
||||||
|
| `|` | field delimiter within a line |
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# pkg/auth
|
||||||
|
## role
|
||||||
|
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
|
||||||
|
## files
|
||||||
|
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
|
||||||
|
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
|
||||||
|
- types.ts | shared auth types | exp: AuthToken, UserClaims, Role
|
||||||
|
## arch
|
||||||
|
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
|
||||||
|
Session state stored in Redis via db/sessions. No server-side JWT storage.
|
||||||
|
## dirty
|
||||||
|
-
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rules
|
||||||
|
- One file per directory, placed inside that directory.
|
||||||
|
- Every non-excluded file in the directory gets one bullet under `## files`.
|
||||||
|
- Subdirectories are referenced in `## role` via `Dep:` or in `## arch` as structural notes, not duplicated.
|
||||||
|
- The `## dirty` section is empty (`-`) when clean, or contains a timestamp/flag when stale.
|
||||||
|
|
||||||
|
## 3. Pipeline Architecture
|
||||||
|
|
||||||
|
### Hybrid Extraction: LLM + AST
|
||||||
|
|
||||||
|
Two independent extraction layers contribute to the same output file.
|
||||||
|
|
||||||
|
#### Layer 1: LLM-Based Extraction (All Files)
|
||||||
|
- **Input**: Raw file contents of every non-excluded file in the directory.
|
||||||
|
- **Output**: One-line purpose description, architectural role, and cross-file relationships.
|
||||||
|
- **Applies to**: Code files, config files, Dockerfiles, READMEs, YAML, JSON, shell scripts — everything.
|
||||||
|
- **When it runs**: Once per file during init; again on changed files during patching.
|
||||||
|
|
||||||
|
#### Layer 2: AST-Based Extraction (Code Files Only)
|
||||||
|
- **Input**: Source code of files where a tree-sitter or LSP parser is available.
|
||||||
|
- **Output**: Precise symbol lists (functions, classes, types), signatures, import/export graphs, class hierarchies.
|
||||||
|
- **Applies to**: Supported languages only (TypeScript, Python, Go, Rust, etc.).
|
||||||
|
- **When it runs**: Once per file during init; again on changed files during patching.
|
||||||
|
|
||||||
|
#### Merging
|
||||||
|
The two layers merge into a single line per file under `## files`:
|
||||||
|
|
||||||
|
```
|
||||||
|
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
|
||||||
|
^ LLM ^ LLM ^ AST ^ AST + LLM
|
||||||
|
```
|
||||||
|
|
||||||
|
- File name and one-line purpose: LLM.
|
||||||
|
- Exported symbols and signatures: AST (augmented by LLM if AST unavailable).
|
||||||
|
- Dependency list: AST for imports; LLM for inferred architectural dependencies.
|
||||||
|
|
||||||
|
### Init Pipeline
|
||||||
|
```
|
||||||
|
For each directory (depth-first):
|
||||||
|
1. List all non-excluded files.
|
||||||
|
2. For each file:
|
||||||
|
a. Run LLM extraction (purpose, role).
|
||||||
|
b. If code file + parser available: run AST extraction (symbols, imports).
|
||||||
|
3. Merge per-file outputs into lines.
|
||||||
|
4. Run LLM on merged lines + directory context to generate:
|
||||||
|
- `## role` (package-level summary)
|
||||||
|
- `## arch` (architectural notes)
|
||||||
|
5. Write `.pi-map.md` to directory.
|
||||||
|
```
|
||||||
|
|
||||||
|
### Patch Pipeline
|
||||||
|
```
|
||||||
|
When agent edits file(s) in directory:
|
||||||
|
1. Determine patch strategy:
|
||||||
|
- If directory has < 10 files: full rewrite.
|
||||||
|
- Else: section-level patch for changed file(s) only.
|
||||||
|
2. Re-run LLM extraction on changed file(s).
|
||||||
|
3. Re-run AST extraction on changed file(s) if applicable.
|
||||||
|
4. Update `## files` section (rewrite or patch).
|
||||||
|
5. Update `## dirty` flag if full regeneration is deferred.
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. Consumption Model
|
||||||
|
|
||||||
|
### Session Start
|
||||||
|
1. Agent discovers all `.pi-map.md` files (e.g., via `find . -name ".pi-map.md"`).
|
||||||
|
2. Agent reads **all** files into context. This is a one-time cost at session start.
|
||||||
|
3. Agent constructs an internal mental model of the project hierarchy.
|
||||||
|
|
||||||
|
### During Session
|
||||||
|
- An **auto-injected summary** stays in context (e.g., a condensed top-level `.pi-map.md` or a synthesized project overview).
|
||||||
|
- When the agent needs deeper detail about a specific package, it already has the full `.pi-map.md` in memory from step 2.
|
||||||
|
- If the agent enters a new package not yet loaded, it reads that package's `.pi-map.md` on demand.
|
||||||
|
|
||||||
|
### Context Management
|
||||||
|
- For very large projects, the agent may summarize or prune the initial read, keeping only the top N levels of the hierarchy in active context.
|
||||||
|
- The skill can provide a "context budget" parameter: max tokens to spend on analysis files.
|
||||||
|
|
||||||
|
## 5. Stale Data Mitigation
|
||||||
|
|
||||||
|
### Combined Strategy
|
||||||
|
|
||||||
|
#### 5.1 Dirty Markers
|
||||||
|
- Whenever the agent edits a file, it appends a dirty flag to the directory's `.pi-map.md`:
|
||||||
|
```markdown
|
||||||
|
## dirty
|
||||||
|
2024-06-09T14:32:00Z: tokens.ts modified
|
||||||
|
```
|
||||||
|
- A background or post-session reconciliation step regenerates dirty files.
|
||||||
|
- The agent can also be instructed to reconcile before making architectural decisions.
|
||||||
|
|
||||||
|
#### 5.2 Periodic Full Re-init
|
||||||
|
- On every new session start, or on a configurable schedule (e.g., daily), the skill offers to run a full re-scan.
|
||||||
|
- This catches any changes made outside the agent's awareness (e.g., by other developers).
|
||||||
|
|
||||||
|
#### 5.3 Validation Command
|
||||||
|
- A `validate` tool/command that the agent can invoke:
|
||||||
|
- Checks for missing files (new files not in `.pi-map.md`).
|
||||||
|
- Checks for orphaned entries (files listed but deleted).
|
||||||
|
- Checks for changed signatures (AST mismatch between listed symbols and actual code).
|
||||||
|
- Reports discrepancies and suggests corrections.
|
||||||
|
|
||||||
|
### Recovery
|
||||||
|
- If validation finds staleness beyond a threshold (e.g., > 3 dirty packages), the skill recommends a full re-init.
|
||||||
|
- The agent can also trigger re-init for a specific subtree.
|
||||||
|
|
||||||
|
## 6. Scope Boundaries and Non-Goals
|
||||||
|
|
||||||
|
### In Scope
|
||||||
|
- Every directory in the project gets a `.pi-map.md` file.
|
||||||
|
- Every non-excluded file gets analyzed by the LLM layer.
|
||||||
|
- Code files get augmented by the AST layer where parsers exist.
|
||||||
|
- Respect `.gitignore` and known junk patterns (node_modules, .git, dist, build, coverage, .next, .venv, __pycache__, .DS_Store).
|
||||||
|
|
||||||
|
### Out of Scope (Non-Goals)
|
||||||
|
- **Human-readable documentation**: These files are machine-only. Human docs live elsewhere.
|
||||||
|
- **Line-by-line code explanation**: The format captures symbols and architecture, not implementation details.
|
||||||
|
- **Auto-regeneration on filesystem events**: The skill relies on agent-initiated updates and periodic re-init, not filesystem watchers.
|
||||||
|
- **Cross-project analysis**: Each project is independent. No global index across repos.
|
||||||
|
- **IDE integration**: This is a Pi agent skill, not a VS Code extension or LSP server.
|
||||||
|
|
||||||
|
## 7. Pi Skill Package Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
pi-project-map/
|
||||||
|
├── SKILL.md # Skill definition for Pi
|
||||||
|
├── package.json # npm package metadata
|
||||||
|
├── src/
|
||||||
|
│ ├── init.ts # Full project scan + generation
|
||||||
|
│ ├── patch.ts # Incremental patch logic
|
||||||
|
│ ├── validate.ts # Consistency checker
|
||||||
|
│ ├── ast-extract.ts # Tree-sitter / LSP wrappers
|
||||||
|
│ ├── llm-extract.ts # LLM prompt templates for extraction
|
||||||
|
│ ├── merge.ts # Merge AST + LLM outputs
|
||||||
|
│ ├── format.ts # Dense markdown formatter
|
||||||
|
│ └── config.ts # Skill configuration (thresholds, ignore patterns)
|
||||||
|
├── hooks/
|
||||||
|
│ └── on-prompt.ts # Injects maintenance command into prompts
|
||||||
|
└── README.md # Setup and usage for humans
|
||||||
|
```
|
||||||
|
|
||||||
|
### Custom Tools
|
||||||
|
- `project-map:init` — Run full project scan. Creates all `.pi-map.md` files.
|
||||||
|
- `project-map:patch <file-path>` — Update analysis for a specific file/directory.
|
||||||
|
- `project-map:validate` — Run consistency check across all `.pi-map.md` files.
|
||||||
|
- `project-map:reinit [path]` — Force re-initialization of entire project or subtree.
|
||||||
|
|
||||||
|
### Prompt Hook
|
||||||
|
- On every prompt, the skill appends a lightweight instruction:
|
||||||
|
> "If you modify any source file, run `project-map:patch <path>` to update the analysis. If you suspect staleness, run `project-map:validate`."
|
||||||
|
|
||||||
|
## 8. Risks and Tradeoffs
|
||||||
|
|
||||||
|
| Risk | Likelihood | Impact | Mitigation |
|
||||||
|
|------|-----------|--------|------------|
|
||||||
|
| Token bloat (1000+ dirs) | Medium | High | Summary mode, lazy loading, context budget |
|
||||||
|
| Stale analysis files | High | High | Dirty markers + periodic re-init + validation |
|
||||||
|
| Agent trusts stale data | Medium | High | Clear instructions to validate before architectural decisions |
|
||||||
|
| Expensive init on large repos | Medium | Medium | Parallelization, caching, optional incremental init |
|
||||||
|
| Overlap with LSP/typedoc | Low | Low | This is agent-context, not IDE tooling. Different use case. |
|
||||||
|
| AST parser unavailable | Medium | Low | Graceful fallback to LLM-only extraction |
|
||||||
|
|
||||||
|
## 9. Concrete Example: Full Project Snapshot
|
||||||
|
|
||||||
|
```
|
||||||
|
project-root/
|
||||||
|
├── .pi-map.md
|
||||||
|
├── src/
|
||||||
|
│ ├── .pi-map.md
|
||||||
|
│ ├── auth/
|
||||||
|
│ │ ├── .pi-map.md
|
||||||
|
│ │ ├── tokens.ts
|
||||||
|
│ │ ├── middleware.ts
|
||||||
|
│ │ └── types.ts
|
||||||
|
│ └── db/
|
||||||
|
│ ├── .pi-map.md
|
||||||
|
│ ├── connection.ts
|
||||||
|
│ └── migrations/
|
||||||
|
│ ├── .pi-map.md
|
||||||
|
│ └── 001_init.sql
|
||||||
|
├── docker/
|
||||||
|
│ ├── .pi-map.md
|
||||||
|
│ ├── Dockerfile
|
||||||
|
│ └── docker-compose.yml
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Each `.pi-map.md` follows the format in Section 2, creating a navigable hierarchy.
|
||||||
|
|
||||||
|
## 10. Future Extensions
|
||||||
|
|
||||||
|
- **Cross-reference graph**: A top-level `project-graph.md` linking all packages with dependency arrows.
|
||||||
|
- **Search index**: A lightweight FTS5 index over all `.pi-map.md` files for fast symbol lookup.
|
||||||
|
- **Diff-aware patching**: Only re-run LLM on changed functions, not entire files.
|
||||||
|
- **Multi-repo workspaces**: Support monorepos with independent package boundaries.
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
// Pi skill prompt hook
|
||||||
|
// Injected into every prompt to remind the agent to maintain .pi-map.md files
|
||||||
|
|
||||||
|
export const MAINTENANCE_INSTRUCTION = `
|
||||||
|
If you modify any source file, run \`project-map:patch <file-path>\` to update the analysis.
|
||||||
|
If you suspect staleness, run \`project-map:validate\`.
|
||||||
|
`;
|
||||||
|
|
||||||
|
export function injectPrompt(originalPrompt: string): string {
|
||||||
|
return `${originalPrompt}\n\n---\n${MAINTENANCE_INSTRUCTION}`;
|
||||||
|
}
|
||||||
@@ -0,0 +1,246 @@
|
|||||||
|
# Implementation Plan: Hierarchical Project Analysis Skill for Pi
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Build a Pi skill package (`pi-project-map`) that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file. The skill provides custom tools for init, patch, validate, and re-init, plus a prompt hook that ensures the agent keeps analysis files in sync.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestones
|
||||||
|
|
||||||
|
### M1: Foundation and Format (Week 1)
|
||||||
|
**Goal**: A working dense-markdown formatter and the ability to generate `.pi-map.md` files from a directory listing.
|
||||||
|
|
||||||
|
#### Tasks
|
||||||
|
1. **Bootstrap the npm package**
|
||||||
|
- Create `package.json`, `tsconfig.json`, basic CLI entry point.
|
||||||
|
- Set up test runner (vitest or jest).
|
||||||
|
- Create stub `SKILL.md` for Pi integration.
|
||||||
|
|
||||||
|
2. **Implement the formatter (`src/format.ts`)**
|
||||||
|
- Define the `.pi-map.md` schema as a TypeScript interface.
|
||||||
|
- Implement `renderPackageMap(data) -> string` that outputs dense markdown.
|
||||||
|
- Implement `parsePackageMap(markdown) -> data` for reading existing files.
|
||||||
|
- Unit tests for round-trip serialization.
|
||||||
|
|
||||||
|
3. **Implement file discovery (`src/discover.ts`)**
|
||||||
|
- Walk a directory tree.
|
||||||
|
- Respect `.gitignore` and a built-in ignore list (node_modules, .git, dist, etc.).
|
||||||
|
- Return a flat list of `(dirPath, filePaths[])` tuples.
|
||||||
|
- Unit tests with fixture directories.
|
||||||
|
|
||||||
|
4. **Implement init skeleton (`src/init.ts`)**
|
||||||
|
- Given a project root, create one `.pi-map.md` per directory.
|
||||||
|
- Populate `## files` with filenames only (no analysis yet).
|
||||||
|
- Populate `## role` and `## arch` with placeholder text.
|
||||||
|
- Command: `project-map:init --root <path>`
|
||||||
|
|
||||||
|
**Acceptance Criteria**
|
||||||
|
- Running `project-map:init` on a test repo creates `.pi-map.md` in every directory.
|
||||||
|
- Files match `.gitignore` rules correctly.
|
||||||
|
- Output format follows the dense-markdown spec from the design doc.
|
||||||
|
|
||||||
|
**Dependencies**: None.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### M2: LLM Extraction Layer (Week 2)
|
||||||
|
**Goal**: The LLM layer generates one-line purposes and package-level architecture notes.
|
||||||
|
|
||||||
|
#### Tasks
|
||||||
|
1. **Implement LLM file extraction (`src/llm-extract.ts`)**
|
||||||
|
- Prompt template: given a file's contents and path, return a one-line purpose and a list of exported symbols (if any).
|
||||||
|
- Prompt template: given a file's contents, infer its dependencies.
|
||||||
|
- Integrate with Pi's LLM tool calls (or a generic OpenAI-compatible client for standalone testing).
|
||||||
|
- Add caching: hash file contents, skip LLM call if unchanged.
|
||||||
|
|
||||||
|
2. **Implement LLM package extraction (`src/llm-extract.ts`)**
|
||||||
|
- Prompt template: given all per-file summaries in a directory, generate `## role` and `## arch`.
|
||||||
|
- Keep prompts token-efficient (truncate large files, focus on headers/exports).
|
||||||
|
|
||||||
|
3. **Wire LLM layer into init (`src/init.ts`)**
|
||||||
|
- After file discovery, run LLM extraction on every file.
|
||||||
|
- Then run package-level LLM extraction.
|
||||||
|
- Write fully populated `.pi-map.md` files.
|
||||||
|
|
||||||
|
4. **Add configuration (`src/config.ts`)**
|
||||||
|
- Configurable ignore patterns.
|
||||||
|
- Configurable LLM model and token limits.
|
||||||
|
- Configurable context budget for large projects.
|
||||||
|
|
||||||
|
**Acceptance Criteria**
|
||||||
|
- `project-map:init` produces `.pi-map.md` files with meaningful `## files`, `## role`, and `## arch` sections.
|
||||||
|
- Caching works: second run on unchanged repo is fast.
|
||||||
|
- Unit tests mock LLM responses to verify prompt structure and output parsing.
|
||||||
|
|
||||||
|
**Dependencies**: M1.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### M3: AST Extraction Layer (Week 3)
|
||||||
|
**Goal**: Code files get precise symbol and dependency data from tree-sitter or LSP.
|
||||||
|
|
||||||
|
#### Tasks
|
||||||
|
1. **Set up tree-sitter (`src/ast-extract.ts`)**
|
||||||
|
- Add tree-sitter dependencies for target languages (TypeScript, Python, Go, Rust as first-class).
|
||||||
|
- Implement `extractSymbols(filePath, language) -> { exports, imports, types }`.
|
||||||
|
- Implement `extractDependencies(filePath, language) -> string[]`.
|
||||||
|
|
||||||
|
2. **Language detection**
|
||||||
|
- Map file extensions to tree-sitter grammars.
|
||||||
|
- Graceful fallback to LLM-only if no parser available.
|
||||||
|
|
||||||
|
3. **Merge AST + LLM outputs (`src/merge.ts`)**
|
||||||
|
- Combine AST symbols with LLM purpose descriptions.
|
||||||
|
- AST provides `exp:` and `dep:`; LLM provides the one-line purpose.
|
||||||
|
- If AST and LLM disagree on dependencies, prefer AST for imports, LLM for inferred architectural deps.
|
||||||
|
|
||||||
|
4. **Wire AST into init (`src/init.ts`)**
|
||||||
|
- For each code file, run AST extraction in parallel with LLM extraction.
|
||||||
|
- Merge results before formatting.
|
||||||
|
|
||||||
|
**Acceptance Criteria**
|
||||||
|
- TypeScript/Python files show precise exported function/class names in `exp:`.
|
||||||
|
- Import statements are captured in `dep:`.
|
||||||
|
- Files with no AST parser still get LLM-only analysis.
|
||||||
|
- Unit tests with sample source files verify symbol extraction accuracy.
|
||||||
|
|
||||||
|
**Dependencies**: M2.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### M4: Patch and Update (Week 4)
|
||||||
|
**Goal**: The agent can incrementally update `.pi-map.md` files after editing source files.
|
||||||
|
|
||||||
|
#### Tasks
|
||||||
|
1. **Implement patch strategy (`src/patch.ts`)**
|
||||||
|
- Determine directory size (file count).
|
||||||
|
- If < 10 files: trigger full rewrite of that directory's `.pi-map.md`.
|
||||||
|
- If >= 10 files: parse existing `.pi-map.md`, identify the section(s) for changed file(s), and rewrite only those lines.
|
||||||
|
|
||||||
|
2. **Implement dirty markers (`src/patch.ts`)**
|
||||||
|
- When a section-level patch occurs, append a dirty note:
|
||||||
|
```markdown
|
||||||
|
## dirty
|
||||||
|
2024-06-09T14:32:00Z: tokens.ts patched (section-level)
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Command: `project-map:patch <file-path>`**
|
||||||
|
- Given a changed file path, find its directory.
|
||||||
|
- Re-run LLM + AST extraction on that file only.
|
||||||
|
- Apply patch strategy.
|
||||||
|
- Update `.pi-map.md`.
|
||||||
|
|
||||||
|
4. **Command: `project-map:reinit [path]`**
|
||||||
|
- Force full re-init of entire project or a subtree.
|
||||||
|
- Clear all dirty markers.
|
||||||
|
|
||||||
|
**Acceptance Criteria**
|
||||||
|
- Editing one file in a small package triggers full rewrite of that package's `.pi-map.md`.
|
||||||
|
- Editing one file in a large package patches only the relevant line.
|
||||||
|
- Dirty markers are correctly added on section-level patches.
|
||||||
|
- Re-init clears dirty markers and regenerates everything.
|
||||||
|
|
||||||
|
**Dependencies**: M3.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### M5: Validation and Consistency (Week 5)
|
||||||
|
**Goal**: The agent can detect and recover from stale analysis files.
|
||||||
|
|
||||||
|
#### Tasks
|
||||||
|
1. **Implement validation (`src/validate.ts`)**
|
||||||
|
- Walk all `.pi-map.md` files.
|
||||||
|
- For each listed file, check existence (catch deletions).
|
||||||
|
- For each directory, check for new files not in `.pi-map.md`.
|
||||||
|
- For code files with AST support, compare listed `exp:` against actual symbols.
|
||||||
|
- Report discrepancies with severity (missing, orphaned, stale-signature).
|
||||||
|
|
||||||
|
2. **Implement reconciliation (`src/validate.ts`)**
|
||||||
|
- Option `--fix`: automatically patch or flag discrepancies.
|
||||||
|
- Option `--reinit-if <threshold>`: if > N discrepancies, recommend full re-init.
|
||||||
|
|
||||||
|
3. **Command: `project-map:validate`**
|
||||||
|
- Run checks and print a report.
|
||||||
|
- Exit code 0 if clean, 1 if discrepancies found.
|
||||||
|
|
||||||
|
**Acceptance Criteria**
|
||||||
|
- Validation detects a deleted file that still appears in `.pi-map.md`.
|
||||||
|
- Validation detects a new file not yet in `.pi-map.md`.
|
||||||
|
- Validation detects a changed function signature.
|
||||||
|
- `--fix` patches or flags all found issues.
|
||||||
|
|
||||||
|
**Dependencies**: M4.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### M6: Pi Integration (Week 6)
|
||||||
|
**Goal**: The skill is installable and functional within the Pi agent harness.
|
||||||
|
|
||||||
|
#### Tasks
|
||||||
|
1. **Finalize `SKILL.md`**
|
||||||
|
- Document all custom tools: `project-map:init`, `project-map:patch`, `project-map:validate`, `project-map:reinit`.
|
||||||
|
- Document the dense-markdown format for the agent.
|
||||||
|
- Include usage examples and configuration options.
|
||||||
|
|
||||||
|
2. **Implement prompt hook (`hooks/on-prompt.ts`)**
|
||||||
|
- Inject a lightweight instruction into every prompt:
|
||||||
|
> "If you modify any source file, run `project-map:patch <path>`. If you suspect staleness, run `project-map:validate`."
|
||||||
|
- Make injection configurable (toggle on/off, customize message).
|
||||||
|
|
||||||
|
3. **Auto-inject session-start behavior**
|
||||||
|
- On session start, if `.pi-map.md` files exist, the skill auto-reads them into context.
|
||||||
|
- If they don't exist, the skill offers to run `project-map:init`.
|
||||||
|
- If dirty markers exist, the skill warns the agent.
|
||||||
|
|
||||||
|
4. **Packaging and publishing**
|
||||||
|
- Ensure `package.json` has correct `bin` entries.
|
||||||
|
- Write human README with install instructions.
|
||||||
|
- Tag and publish to npm (or internal registry).
|
||||||
|
|
||||||
|
**Acceptance Criteria**
|
||||||
|
- Installing the skill in Pi makes the four tools available.
|
||||||
|
- The prompt hook injects maintenance instructions reliably.
|
||||||
|
- Session start auto-detects and loads `.pi-map.md` files.
|
||||||
|
- The skill can be published and installed via npm.
|
||||||
|
|
||||||
|
**Dependencies**: M5.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sequencing and Dependencies
|
||||||
|
|
||||||
|
```
|
||||||
|
M1 (Foundation) → M2 (LLM) → M3 (AST) → M4 (Patch) → M5 (Validate) → M6 (Pi Integration)
|
||||||
|
```
|
||||||
|
|
||||||
|
No parallel tracks — each milestone builds on the previous. Total estimated time: 6 weeks at a steady pace, or 3-4 weeks with focused effort.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Validation Criteria (Overall)
|
||||||
|
|
||||||
|
1. **Functional**: `project-map:init` correctly generates `.pi-map.md` for a test repo of 50+ files across 10+ directories.
|
||||||
|
2. **Accuracy**: AST-extracted symbols match actual source code (verified by unit tests).
|
||||||
|
3. **Freshness**: After editing a file, `project-map:patch` updates the analysis within 5 seconds.
|
||||||
|
4. **Consistency**: `project-map:validate` detects 100% of synthetic staleness scenarios (deleted files, new files, signature changes).
|
||||||
|
5. **Token efficiency**: A 100-file project's full `.pi-map.md` set fits within a 4k token budget.
|
||||||
|
6. **Pi integration**: The skill installs cleanly, tools are discoverable, and the prompt hook works.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Rollout Plan
|
||||||
|
|
||||||
|
1. **Internal dogfooding** (Week 7): Use the skill on 2-3 real projects. Gather feedback on format density and agent behavior.
|
||||||
|
2. **Format refinement** (Week 8): Adjust abbreviations, section structure, and dirty-marker format based on dogfooding.
|
||||||
|
3. **Open beta** (Week 9): Share with a small group of Pi users. Collect bug reports and feature requests.
|
||||||
|
4. **v1.0 release** (Week 10): Stable API, documented format, published to npm.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Open Questions for Implementation
|
||||||
|
|
||||||
|
- Which tree-sitter grammars to bundle by default? (Start with TypeScript, Python, Go.)
|
||||||
|
- Should the LLM extraction use the agent's current model or a dedicated cheaper model?
|
||||||
|
- How to handle monorepos with multiple `.gitignore` files at different depths?
|
||||||
|
- Should `.pi-map.md` files be committed to git or kept in `.gitignore`? (Recommendation: `.gitignore` — they are derived artifacts.)
|
||||||
Generated
+3588
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,39 @@
|
|||||||
|
{
|
||||||
|
"name": "pi-project-map",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"description": "Pi skill for hierarchical project analysis — generates and maintains .pi-map.md files",
|
||||||
|
"main": "dist/index.js",
|
||||||
|
"types": "dist/index.d.ts",
|
||||||
|
"bin": {
|
||||||
|
"project-map": "dist/cli.js"
|
||||||
|
},
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsc",
|
||||||
|
"dev": "tsc --watch",
|
||||||
|
"test": "vitest",
|
||||||
|
"lint": "eslint src/**/*.ts",
|
||||||
|
"typecheck": "tsc --noEmit"
|
||||||
|
},
|
||||||
|
"keywords": [
|
||||||
|
"pi",
|
||||||
|
"skill",
|
||||||
|
"project-analysis",
|
||||||
|
"code-intelligence",
|
||||||
|
"agent-context"
|
||||||
|
],
|
||||||
|
"author": "",
|
||||||
|
"license": "MIT",
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^20.0.0",
|
||||||
|
"typescript": "^5.3.0",
|
||||||
|
"vitest": "^1.0.0",
|
||||||
|
"eslint": "^8.0.0",
|
||||||
|
"@typescript-eslint/eslint-plugin": "^6.0.0",
|
||||||
|
"@typescript-eslint/parser": "^6.0.0"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"tree-sitter": "^0.21.0",
|
||||||
|
"tree-sitter-typescript": "^0.21.0",
|
||||||
|
"ignore": "^5.3.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
import { readFileSync } from 'fs';
|
||||||
|
|
||||||
|
interface ASTFileData {
|
||||||
|
exports: string[];
|
||||||
|
deps: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function extractFileAST(filePath: string): Promise<ASTFileData | null> {
|
||||||
|
// TODO: integrate tree-sitter
|
||||||
|
// Detect language from extension
|
||||||
|
// Parse and extract symbols
|
||||||
|
return null;
|
||||||
|
}
|
||||||
+32
@@ -0,0 +1,32 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
import { initProject } from './init.js';
|
||||||
|
import { patchFile } from './patch.js';
|
||||||
|
import { validateMaps } from './validate.js';
|
||||||
|
import { reinitPath } from './init.js';
|
||||||
|
|
||||||
|
const args = process.argv.slice(2);
|
||||||
|
const command = args[0];
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
switch (command) {
|
||||||
|
case 'init':
|
||||||
|
await initProject(args[1] || '.');
|
||||||
|
break;
|
||||||
|
case 'patch':
|
||||||
|
await patchFile(args[1]);
|
||||||
|
break;
|
||||||
|
case 'validate': {
|
||||||
|
const result = await validateMaps(args[1] || '.');
|
||||||
|
process.exit(result.clean ? 0 : 1);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case 'reinit':
|
||||||
|
await reinitPath(args[1] || '.');
|
||||||
|
break;
|
||||||
|
default:
|
||||||
|
console.log(`Usage: project-map <init|patch|validate|reinit> [path]`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main();
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
export interface SkillConfig {
|
||||||
|
ignorePatterns: string[];
|
||||||
|
smallPackageThreshold: number;
|
||||||
|
llmModel: string;
|
||||||
|
contextBudget: number;
|
||||||
|
autoInjectPrompt: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const DEFAULT_CONFIG: SkillConfig = {
|
||||||
|
ignorePatterns: [
|
||||||
|
'node_modules', '.git', 'dist', 'build', 'coverage',
|
||||||
|
'.next', '.venv', '__pycache__', '.DS_Store', '*.log'
|
||||||
|
],
|
||||||
|
smallPackageThreshold: 10,
|
||||||
|
llmModel: 'gpt-4o-mini',
|
||||||
|
contextBudget: 4000,
|
||||||
|
autoInjectPrompt: true
|
||||||
|
};
|
||||||
|
|
||||||
|
export function loadConfig(): SkillConfig {
|
||||||
|
// TODO: load from .pi-project-map.json or similar
|
||||||
|
return DEFAULT_CONFIG;
|
||||||
|
}
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
import { readdirSync, statSync } from 'fs';
|
||||||
|
import { join, relative } from 'path';
|
||||||
|
import ignore from 'ignore';
|
||||||
|
|
||||||
|
const DEFAULT_IGNORE = [
|
||||||
|
'node_modules', '.git', 'dist', 'build', 'coverage',
|
||||||
|
'.next', '.venv', '__pycache__', '.DS_Store', '*.log'
|
||||||
|
];
|
||||||
|
|
||||||
|
export interface DirectoryEntry {
|
||||||
|
dirPath: string;
|
||||||
|
relativePath: string;
|
||||||
|
files: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function discoverProject(rootPath: string): DirectoryEntry[] {
|
||||||
|
const ig = ignore().add(DEFAULT_IGNORE);
|
||||||
|
const gitignorePath = join(rootPath, '.gitignore');
|
||||||
|
try {
|
||||||
|
const gitignoreContent = require('fs').readFileSync(gitignorePath, 'utf8');
|
||||||
|
ig.add(gitignoreContent);
|
||||||
|
} catch { /* no .gitignore */ }
|
||||||
|
|
||||||
|
const entries: DirectoryEntry[] = [];
|
||||||
|
|
||||||
|
function walk(dir: string) {
|
||||||
|
const relDir = relative(rootPath, dir) || '.';
|
||||||
|
if (ig.ignores(relDir)) return;
|
||||||
|
|
||||||
|
const items = readdirSync(dir);
|
||||||
|
const files: string[] = [];
|
||||||
|
const subdirs: string[] = [];
|
||||||
|
|
||||||
|
for (const item of items) {
|
||||||
|
const relPath = join(relDir, item);
|
||||||
|
if (ig.ignores(relPath)) continue;
|
||||||
|
|
||||||
|
const fullPath = join(dir, item);
|
||||||
|
const st = statSync(fullPath);
|
||||||
|
if (st.isDirectory()) {
|
||||||
|
subdirs.push(fullPath);
|
||||||
|
} else {
|
||||||
|
files.push(item);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
entries.push({ dirPath: dir, relativePath: relDir, files });
|
||||||
|
|
||||||
|
for (const subdir of subdirs) {
|
||||||
|
walk(subdir);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
walk(rootPath);
|
||||||
|
return entries;
|
||||||
|
}
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
export interface PackageMapData {
|
||||||
|
path: string;
|
||||||
|
role: string;
|
||||||
|
files: FileEntry[];
|
||||||
|
arch: string;
|
||||||
|
dirty?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface FileEntry {
|
||||||
|
name: string;
|
||||||
|
purpose: string;
|
||||||
|
exports: string[];
|
||||||
|
deps: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function renderPackageMap(data: PackageMapData): string {
|
||||||
|
const lines: string[] = [];
|
||||||
|
lines.push(`# ${data.path}`);
|
||||||
|
lines.push(`## role`);
|
||||||
|
lines.push(data.role);
|
||||||
|
lines.push(`## files`);
|
||||||
|
for (const file of data.files) {
|
||||||
|
const exp = file.exports.length > 0 ? `exp: ${file.exports.join(', ')}` : '';
|
||||||
|
const dep = file.deps.length > 0 ? `dep: ${file.deps.join(', ')}` : '';
|
||||||
|
const parts = [`- ${file.name} | ${file.purpose}`];
|
||||||
|
if (exp) parts.push(exp);
|
||||||
|
if (dep) parts.push(dep);
|
||||||
|
lines.push(parts.join(' | '));
|
||||||
|
}
|
||||||
|
lines.push(`## arch`);
|
||||||
|
lines.push(data.arch);
|
||||||
|
lines.push(`## dirty`);
|
||||||
|
lines.push(data.dirty || '-');
|
||||||
|
return `${lines.join('\n')}\n`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parsePackageMap(_markdown: string): PackageMapData {
|
||||||
|
// TODO: implement robust parser
|
||||||
|
throw new Error('parsePackageMap not yet implemented');
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
// Main entry point for pi-project-map skill
|
||||||
|
export { initProject } from './init.js';
|
||||||
|
export { patchFile } from './patch.js';
|
||||||
|
export { validateMaps } from './validate.js';
|
||||||
|
export { renderPackageMap, parsePackageMap } from './format.js';
|
||||||
+42
@@ -0,0 +1,42 @@
|
|||||||
|
import { discoverProject } from './discover.js';
|
||||||
|
import { renderPackageMap, type PackageMapData } from './format.js';
|
||||||
|
import { extractFileLLM } from './llm-extract.js';
|
||||||
|
import { extractFileAST } from './ast-extract.js';
|
||||||
|
import { mergeFileData } from './merge.js';
|
||||||
|
import { extractPackageLLM } from './llm-extract.js';
|
||||||
|
import { writeFileSync } from 'fs';
|
||||||
|
import { join } from 'path';
|
||||||
|
|
||||||
|
export async function initProject(rootPath: string): Promise<void> {
|
||||||
|
const entries = discoverProject(rootPath);
|
||||||
|
|
||||||
|
for (const entry of entries) {
|
||||||
|
const fileData = [];
|
||||||
|
for (const file of entry.files) {
|
||||||
|
const filePath = join(entry.dirPath, file);
|
||||||
|
const llmData = await extractFileLLM(filePath);
|
||||||
|
const astData = await extractFileAST(filePath);
|
||||||
|
fileData.push(mergeFileData(file, llmData, astData));
|
||||||
|
}
|
||||||
|
|
||||||
|
const packageData = await extractPackageLLM(entry.relativePath, fileData);
|
||||||
|
|
||||||
|
const mapData: PackageMapData = {
|
||||||
|
path: entry.relativePath,
|
||||||
|
role: packageData.role,
|
||||||
|
files: fileData,
|
||||||
|
arch: packageData.arch,
|
||||||
|
dirty: '-'
|
||||||
|
};
|
||||||
|
|
||||||
|
const outPath = join(entry.dirPath, '.pi-map.md');
|
||||||
|
writeFileSync(outPath, renderPackageMap(mapData));
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`Generated ${entries.length} .pi-map.md files`);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function reinitPath(path: string): Promise<void> {
|
||||||
|
// TODO: clear dirty markers and force regeneration
|
||||||
|
await initProject(path);
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import { readFileSync } from 'fs';
|
||||||
|
import { createHash } from 'crypto';
|
||||||
|
|
||||||
|
interface LLMFileData {
|
||||||
|
purpose: string;
|
||||||
|
exports: string[];
|
||||||
|
deps: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
interface LLMPackageData {
|
||||||
|
role: string;
|
||||||
|
arch: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Simple in-memory cache
|
||||||
|
const cache = new Map<string, LLMFileData>();
|
||||||
|
|
||||||
|
export async function extractFileLLM(filePath: string): Promise<LLMFileData> {
|
||||||
|
const content = readFileSync(filePath, 'utf8');
|
||||||
|
const hash = createHash('sha256').update(content).digest('hex');
|
||||||
|
|
||||||
|
if (cache.has(hash)) {
|
||||||
|
return cache.get(hash)!;
|
||||||
|
}
|
||||||
|
|
||||||
|
// TODO: integrate with Pi's LLM tool or a generic client
|
||||||
|
// For now, return placeholder data
|
||||||
|
const result: LLMFileData = {
|
||||||
|
purpose: 'TODO: analyze with LLM',
|
||||||
|
exports: [],
|
||||||
|
deps: []
|
||||||
|
};
|
||||||
|
|
||||||
|
cache.set(hash, result);
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function extractPackageLLM(
|
||||||
|
relativePath: string,
|
||||||
|
fileData: { name: string; purpose: string }[]
|
||||||
|
): Promise<LLMPackageData> {
|
||||||
|
// TODO: integrate with Pi's LLM tool
|
||||||
|
// For now, return placeholder data
|
||||||
|
return {
|
||||||
|
role: `TODO: analyze package ${relativePath}`,
|
||||||
|
arch: 'TODO: architectural analysis'
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
import type { FileEntry } from './format.js';
|
||||||
|
|
||||||
|
interface LLMFileData {
|
||||||
|
purpose: string;
|
||||||
|
exports: string[];
|
||||||
|
deps: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ASTFileData {
|
||||||
|
exports: string[];
|
||||||
|
deps: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function mergeFileData(
|
||||||
|
fileName: string,
|
||||||
|
llm: LLMFileData,
|
||||||
|
ast: ASTFileData | null
|
||||||
|
): FileEntry {
|
||||||
|
return {
|
||||||
|
name: fileName,
|
||||||
|
purpose: llm.purpose,
|
||||||
|
exports: ast?.exports ?? llm.exports,
|
||||||
|
deps: [...new Set([...(ast?.deps ?? []), ...llm.deps])]
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import { dirname, join } from 'path';
|
||||||
|
import { existsSync, readFileSync, writeFileSync } from 'fs';
|
||||||
|
import { parsePackageMap, renderPackageMap } from './format.js';
|
||||||
|
import { extractFileLLM } from './llm-extract.js';
|
||||||
|
import { extractFileAST } from './ast-extract.js';
|
||||||
|
import { mergeFileData } from './merge.js';
|
||||||
|
import { readdirSync } from 'fs';
|
||||||
|
|
||||||
|
const SMALL_PACKAGE_THRESHOLD = 10;
|
||||||
|
|
||||||
|
export async function patchFile(filePath: string): Promise<void> {
|
||||||
|
const dirPath = dirname(filePath);
|
||||||
|
const mapPath = join(dirPath, '.pi-map.md');
|
||||||
|
|
||||||
|
if (!existsSync(mapPath)) {
|
||||||
|
// No map exists yet — would need to generate from scratch
|
||||||
|
console.warn(`No .pi-map.md found in ${dirPath}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const allFiles = readdirSync(dirPath).filter((f: string) => !f.startsWith('.') && !f.endsWith('.md'));
|
||||||
|
const isSmallPackage = allFiles.length < SMALL_PACKAGE_THRESHOLD;
|
||||||
|
|
||||||
|
if (isSmallPackage) {
|
||||||
|
// Full rewrite for small packages
|
||||||
|
// TODO: import and reuse init logic for a single directory
|
||||||
|
console.log(`Full rewrite of ${mapPath} (small package: ${allFiles.length} files)`);
|
||||||
|
} else {
|
||||||
|
// Section-level patch
|
||||||
|
const existing = parsePackageMap(readFileSync(mapPath, 'utf8'));
|
||||||
|
const llmData = await extractFileLLM(filePath);
|
||||||
|
const astData = await extractFileAST(filePath);
|
||||||
|
const fileName = filePath.split('/').pop()!;
|
||||||
|
const updatedFile = mergeFileData(fileName, llmData, astData);
|
||||||
|
|
||||||
|
// Replace the matching file entry
|
||||||
|
const idx = existing.files.findIndex(f => f.name === updatedFile.name);
|
||||||
|
if (idx >= 0) {
|
||||||
|
existing.files[idx] = updatedFile;
|
||||||
|
} else {
|
||||||
|
existing.files.push(updatedFile);
|
||||||
|
}
|
||||||
|
|
||||||
|
existing.dirty = `${new Date().toISOString()}: ${fileName} patched (section-level)`;
|
||||||
|
writeFileSync(mapPath, renderPackageMap(existing));
|
||||||
|
console.log(`Patched ${mapPath}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
import { discoverProject } from './discover.js';
|
||||||
|
import { parsePackageMap } from './format.js';
|
||||||
|
import { existsSync, readFileSync } from 'fs';
|
||||||
|
import { join } from 'path';
|
||||||
|
import { extractFileAST } from './ast-extract.js';
|
||||||
|
|
||||||
|
export interface ValidationResult {
|
||||||
|
clean: boolean;
|
||||||
|
discrepancies: Discrepancy[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface Discrepancy {
|
||||||
|
type: 'missing' | 'orphaned' | 'stale-signature' | 'dirty';
|
||||||
|
path: string;
|
||||||
|
message: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function validateMaps(rootPath: string): Promise<ValidationResult> {
|
||||||
|
const discrepancies: Discrepancy[] = [];
|
||||||
|
const entries = discoverProject(rootPath);
|
||||||
|
|
||||||
|
for (const entry of entries) {
|
||||||
|
const mapPath = join(entry.dirPath, '.pi-map.md');
|
||||||
|
if (!existsSync(mapPath)) {
|
||||||
|
discrepancies.push({ type: 'missing', path: entry.relativePath, message: 'No .pi-map.md found' });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const mapData = parsePackageMap(readFileSync(mapPath, 'utf8'));
|
||||||
|
|
||||||
|
// Check for dirty markers
|
||||||
|
if (mapData.dirty && mapData.dirty !== '-') {
|
||||||
|
discrepancies.push({ type: 'dirty', path: mapPath, message: `Dirty: ${mapData.dirty}` });
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check for orphaned entries
|
||||||
|
for (const fileEntry of mapData.files) {
|
||||||
|
const filePath = join(entry.dirPath, fileEntry.name);
|
||||||
|
if (!existsSync(filePath)) {
|
||||||
|
discrepancies.push({ type: 'orphaned', path: filePath, message: `File listed but deleted: ${fileEntry.name}` });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check for new files not in map
|
||||||
|
for (const file of entry.files) {
|
||||||
|
if (!mapData.files.find(f => f.name === file)) {
|
||||||
|
discrepancies.push({ type: 'missing', path: join(entry.dirPath, file), message: `File not in .pi-map.md: ${file}` });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check signatures for code files
|
||||||
|
for (const fileEntry of mapData.files) {
|
||||||
|
const filePath = join(entry.dirPath, fileEntry.name);
|
||||||
|
if (!existsSync(filePath)) continue;
|
||||||
|
|
||||||
|
const astData = await extractFileAST(filePath);
|
||||||
|
if (astData) {
|
||||||
|
const listedExports = new Set(fileEntry.exports);
|
||||||
|
const actualExports = new Set(astData.exports);
|
||||||
|
|
||||||
|
for (const exp of listedExports) {
|
||||||
|
if (!actualExports.has(exp)) {
|
||||||
|
discrepancies.push({ type: 'stale-signature', path: filePath, message: `Missing export: ${exp}` });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const exp of actualExports) {
|
||||||
|
if (!listedExports.has(exp)) {
|
||||||
|
discrepancies.push({ type: 'stale-signature', path: filePath, message: `New export: ${exp}` });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const result: ValidationResult = {
|
||||||
|
clean: discrepancies.length === 0,
|
||||||
|
discrepancies
|
||||||
|
};
|
||||||
|
|
||||||
|
if (result.clean) {
|
||||||
|
console.log('All .pi-map.md files are clean.');
|
||||||
|
} else {
|
||||||
|
console.log(`Found ${discrepancies.length} discrepancies:`);
|
||||||
|
for (const d of discrepancies) {
|
||||||
|
console.log(` [${d.type}] ${d.path}: ${d.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2022",
|
||||||
|
"module": "NodeNext",
|
||||||
|
"moduleResolution": "NodeNext",
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"strict": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"rootDir": "./src",
|
||||||
|
"declaration": true,
|
||||||
|
"declarationMap": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"resolveJsonModule": true
|
||||||
|
},
|
||||||
|
"include": ["src/**/*"],
|
||||||
|
"exclude": ["node_modules", "dist"]
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user