Initial commit: pi-project-map skill scaffolding

- Design doc with full spec (dense markdown format, hybrid AST+LLM pipeline,
  consumption model, stale data mitigation)
- Implementation plan with 6 milestones and rollout strategy
- TypeScript package structure with all source stubs
- CLI entry point, formatter, discover, init, patch, validate, extract, merge
- Pi SKILL.md with tool definitions and format documentation
This commit is contained in:
2026-06-09 17:45:19 +02:00
commit fd958671ca
20 changed files with 4667 additions and 0 deletions
+6
View File
@@ -0,0 +1,6 @@
node_modules/
dist/
coverage/
*.log
.DS_Store
.env
+22
View File
@@ -0,0 +1,22 @@
# pi-project-map
Pi skill for hierarchical project analysis.
## What it does
Generates `.pi-map.md` files throughout your project — one per directory — containing a dense, machine-readable summary of that directory's files, exports, dependencies, and architecture. This gives Pi agents instant project comprehension without reading every source file.
## Quick Start
```bash
npm install -g pi-project-map
project-map init
```
## Design
See [design-doc.md](design-doc.md) for the full specification.
## Implementation Plan
See [implementation-plan.md](implementation-plan.md) for the engineering roadmap.
+56
View File
@@ -0,0 +1,56 @@
# pi-project-map
A Pi skill that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file containing architectural context, exported symbols, and dependencies.
## Tools
### `project-map:init [root]`
Runs a full project scan and generates `.pi-map.md` files in every directory.
### `project-map:patch <file-path>`
Updates the `.pi-map.md` for the directory containing the given file. Uses full rewrite for small packages (< 10 files) or section-level patch for larger packages.
### `project-map:validate [root]`
Checks all `.pi-map.md` files for staleness: missing files, orphaned entries, changed signatures, and dirty markers.
### `project-map:reinit [path]`
Force full re-initialization of the entire project or a specific subtree. Clears all dirty markers.
## Format
Each `.pi-map.md` uses dense markdown with conventions:
```markdown
# pkg/auth
## role
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
## files
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
## arch
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
## dirty
-
```
## Configuration
Create `.pi-project-map.json` in the project root:
```json
{
"ignorePatterns": ["node_modules", ".git"],
"smallPackageThreshold": 10,
"llmModel": "gpt-4o-mini",
"contextBudget": 4000,
"autoInjectPrompt": true
}
```
## Installation
```bash
npm install -g pi-project-map
```
Then add to your Pi skills configuration.
+258
View File
@@ -0,0 +1,258 @@
# Design Doc: Hierarchical Project Analysis Skill for Pi
## 1. Goals and Success Criteria
### Primary Goal
Enable a Pi coding agent to understand a software project's architecture and code relationships without scanning the entire repository. The agent should have a compact, hierarchical "internal representation" of the project that it can consume in-context.
### Success Criteria
- The agent can orient itself in a new or familiar project without reading dozens of source files.
- The agent understands cross-package dependencies, data flows, and architectural patterns from the analysis files alone.
- Analysis files stay sufficiently fresh that the agent does not make decisions based on stale information.
- The representation is token-dense: maximum information per token, optimized for LLM consumption, not human readability.
## 2. Format Specification: Dense Markdown with Conventions
### Design Rationale
- **Not JSON/YAML**: Brackets, quotes, and indentation add token overhead with no benefit to LLM comprehension.
- **Not a custom DSL**: Fragile, requires a parser, and LLMs may hallucinate syntax.
- **Dense markdown**: Hierarchical headings, bullet points, and abbreviations are natively understood by LLMs and extremely token-efficient.
### Structure
Each directory in the project gets one analysis file named `.pi-map.md` (hidden by default, excluded from git via `.gitignore`).
```markdown
# <relative-path>
## role
<one-line package role> | Dep: <comma-separated upstream deps>
## files
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
- <filename> | <one-line purpose> | exp: <exported symbols> | dep: <internal/external deps>
## arch
<free-form architectural notes: patterns, data flow, invariants, design decisions>
## dirty
<timestamp or flag indicating staleness>
```
### Abbreviation Conventions
| Abbreviation | Meaning |
|-------------|---------|
| `exp:` | exported symbols (functions, classes, types, constants) |
| `dep:` | dependencies (other packages, files, or external libs) |
| `pkg/` | project-internal package reference |
| `ext/` | external dependency reference |
| `->` | data flow direction |
| `|` | field delimiter within a line |
### Example
```markdown
# pkg/auth
## role
Auth layer: JWT issuance, validation, refresh. Stateless. Dep: pkg/crypto, pkg/db.
## files
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
- middleware.ts | HTTP auth guard | exp: requireAuth, requireRole | dep: tokens/verifyToken
- types.ts | shared auth types | exp: AuthToken, UserClaims, Role
## arch
Guard pattern on routes. Tokens short-lived (15m), refresh long-lived (7d). Rotation on every use.
Session state stored in Redis via db/sessions. No server-side JWT storage.
## dirty
-
```
### Rules
- One file per directory, placed inside that directory.
- Every non-excluded file in the directory gets one bullet under `## files`.
- Subdirectories are referenced in `## role` via `Dep:` or in `## arch` as structural notes, not duplicated.
- The `## dirty` section is empty (`-`) when clean, or contains a timestamp/flag when stale.
## 3. Pipeline Architecture
### Hybrid Extraction: LLM + AST
Two independent extraction layers contribute to the same output file.
#### Layer 1: LLM-Based Extraction (All Files)
- **Input**: Raw file contents of every non-excluded file in the directory.
- **Output**: One-line purpose description, architectural role, and cross-file relationships.
- **Applies to**: Code files, config files, Dockerfiles, READMEs, YAML, JSON, shell scripts — everything.
- **When it runs**: Once per file during init; again on changed files during patching.
#### Layer 2: AST-Based Extraction (Code Files Only)
- **Input**: Source code of files where a tree-sitter or LSP parser is available.
- **Output**: Precise symbol lists (functions, classes, types), signatures, import/export graphs, class hierarchies.
- **Applies to**: Supported languages only (TypeScript, Python, Go, Rust, etc.).
- **When it runs**: Once per file during init; again on changed files during patching.
#### Merging
The two layers merge into a single line per file under `## files`:
```
- tokens.ts | JWT gen/val | exp: issueToken, verifyToken, refreshToken | dep: crypto/hmac, db/sessions
^ LLM ^ LLM ^ AST ^ AST + LLM
```
- File name and one-line purpose: LLM.
- Exported symbols and signatures: AST (augmented by LLM if AST unavailable).
- Dependency list: AST for imports; LLM for inferred architectural dependencies.
### Init Pipeline
```
For each directory (depth-first):
1. List all non-excluded files.
2. For each file:
a. Run LLM extraction (purpose, role).
b. If code file + parser available: run AST extraction (symbols, imports).
3. Merge per-file outputs into lines.
4. Run LLM on merged lines + directory context to generate:
- `## role` (package-level summary)
- `## arch` (architectural notes)
5. Write `.pi-map.md` to directory.
```
### Patch Pipeline
```
When agent edits file(s) in directory:
1. Determine patch strategy:
- If directory has < 10 files: full rewrite.
- Else: section-level patch for changed file(s) only.
2. Re-run LLM extraction on changed file(s).
3. Re-run AST extraction on changed file(s) if applicable.
4. Update `## files` section (rewrite or patch).
5. Update `## dirty` flag if full regeneration is deferred.
```
## 4. Consumption Model
### Session Start
1. Agent discovers all `.pi-map.md` files (e.g., via `find . -name ".pi-map.md"`).
2. Agent reads **all** files into context. This is a one-time cost at session start.
3. Agent constructs an internal mental model of the project hierarchy.
### During Session
- An **auto-injected summary** stays in context (e.g., a condensed top-level `.pi-map.md` or a synthesized project overview).
- When the agent needs deeper detail about a specific package, it already has the full `.pi-map.md` in memory from step 2.
- If the agent enters a new package not yet loaded, it reads that package's `.pi-map.md` on demand.
### Context Management
- For very large projects, the agent may summarize or prune the initial read, keeping only the top N levels of the hierarchy in active context.
- The skill can provide a "context budget" parameter: max tokens to spend on analysis files.
## 5. Stale Data Mitigation
### Combined Strategy
#### 5.1 Dirty Markers
- Whenever the agent edits a file, it appends a dirty flag to the directory's `.pi-map.md`:
```markdown
## dirty
2024-06-09T14:32:00Z: tokens.ts modified
```
- A background or post-session reconciliation step regenerates dirty files.
- The agent can also be instructed to reconcile before making architectural decisions.
#### 5.2 Periodic Full Re-init
- On every new session start, or on a configurable schedule (e.g., daily), the skill offers to run a full re-scan.
- This catches any changes made outside the agent's awareness (e.g., by other developers).
#### 5.3 Validation Command
- A `validate` tool/command that the agent can invoke:
- Checks for missing files (new files not in `.pi-map.md`).
- Checks for orphaned entries (files listed but deleted).
- Checks for changed signatures (AST mismatch between listed symbols and actual code).
- Reports discrepancies and suggests corrections.
### Recovery
- If validation finds staleness beyond a threshold (e.g., > 3 dirty packages), the skill recommends a full re-init.
- The agent can also trigger re-init for a specific subtree.
## 6. Scope Boundaries and Non-Goals
### In Scope
- Every directory in the project gets a `.pi-map.md` file.
- Every non-excluded file gets analyzed by the LLM layer.
- Code files get augmented by the AST layer where parsers exist.
- Respect `.gitignore` and known junk patterns (node_modules, .git, dist, build, coverage, .next, .venv, __pycache__, .DS_Store).
### Out of Scope (Non-Goals)
- **Human-readable documentation**: These files are machine-only. Human docs live elsewhere.
- **Line-by-line code explanation**: The format captures symbols and architecture, not implementation details.
- **Auto-regeneration on filesystem events**: The skill relies on agent-initiated updates and periodic re-init, not filesystem watchers.
- **Cross-project analysis**: Each project is independent. No global index across repos.
- **IDE integration**: This is a Pi agent skill, not a VS Code extension or LSP server.
## 7. Pi Skill Package Structure
```
pi-project-map/
├── SKILL.md # Skill definition for Pi
├── package.json # npm package metadata
├── src/
│ ├── init.ts # Full project scan + generation
│ ├── patch.ts # Incremental patch logic
│ ├── validate.ts # Consistency checker
│ ├── ast-extract.ts # Tree-sitter / LSP wrappers
│ ├── llm-extract.ts # LLM prompt templates for extraction
│ ├── merge.ts # Merge AST + LLM outputs
│ ├── format.ts # Dense markdown formatter
│ └── config.ts # Skill configuration (thresholds, ignore patterns)
├── hooks/
│ └── on-prompt.ts # Injects maintenance command into prompts
└── README.md # Setup and usage for humans
```
### Custom Tools
- `project-map:init` — Run full project scan. Creates all `.pi-map.md` files.
- `project-map:patch <file-path>` — Update analysis for a specific file/directory.
- `project-map:validate` — Run consistency check across all `.pi-map.md` files.
- `project-map:reinit [path]` — Force re-initialization of entire project or subtree.
### Prompt Hook
- On every prompt, the skill appends a lightweight instruction:
> "If you modify any source file, run `project-map:patch <path>` to update the analysis. If you suspect staleness, run `project-map:validate`."
## 8. Risks and Tradeoffs
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Token bloat (1000+ dirs) | Medium | High | Summary mode, lazy loading, context budget |
| Stale analysis files | High | High | Dirty markers + periodic re-init + validation |
| Agent trusts stale data | Medium | High | Clear instructions to validate before architectural decisions |
| Expensive init on large repos | Medium | Medium | Parallelization, caching, optional incremental init |
| Overlap with LSP/typedoc | Low | Low | This is agent-context, not IDE tooling. Different use case. |
| AST parser unavailable | Medium | Low | Graceful fallback to LLM-only extraction |
## 9. Concrete Example: Full Project Snapshot
```
project-root/
├── .pi-map.md
├── src/
│ ├── .pi-map.md
│ ├── auth/
│ │ ├── .pi-map.md
│ │ ├── tokens.ts
│ │ ├── middleware.ts
│ │ └── types.ts
│ └── db/
│ ├── .pi-map.md
│ ├── connection.ts
│ └── migrations/
│ ├── .pi-map.md
│ └── 001_init.sql
├── docker/
│ ├── .pi-map.md
│ ├── Dockerfile
│ └── docker-compose.yml
└── README.md
```
Each `.pi-map.md` follows the format in Section 2, creating a navigable hierarchy.
## 10. Future Extensions
- **Cross-reference graph**: A top-level `project-graph.md` linking all packages with dependency arrows.
- **Search index**: A lightweight FTS5 index over all `.pi-map.md` files for fast symbol lookup.
- **Diff-aware patching**: Only re-run LLM on changed functions, not entire files.
- **Multi-repo workspaces**: Support monorepos with independent package boundaries.
+11
View File
@@ -0,0 +1,11 @@
// Pi skill prompt hook
// Injected into every prompt to remind the agent to maintain .pi-map.md files
export const MAINTENANCE_INSTRUCTION = `
If you modify any source file, run \`project-map:patch <file-path>\` to update the analysis.
If you suspect staleness, run \`project-map:validate\`.
`;
export function injectPrompt(originalPrompt: string): string {
return `${originalPrompt}\n\n---\n${MAINTENANCE_INSTRUCTION}`;
}
+246
View File
@@ -0,0 +1,246 @@
# Implementation Plan: Hierarchical Project Analysis Skill for Pi
## Overview
Build a Pi skill package (`pi-project-map`) that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file. The skill provides custom tools for init, patch, validate, and re-init, plus a prompt hook that ensures the agent keeps analysis files in sync.
---
## Milestones
### M1: Foundation and Format (Week 1)
**Goal**: A working dense-markdown formatter and the ability to generate `.pi-map.md` files from a directory listing.
#### Tasks
1. **Bootstrap the npm package**
- Create `package.json`, `tsconfig.json`, basic CLI entry point.
- Set up test runner (vitest or jest).
- Create stub `SKILL.md` for Pi integration.
2. **Implement the formatter (`src/format.ts`)**
- Define the `.pi-map.md` schema as a TypeScript interface.
- Implement `renderPackageMap(data) -> string` that outputs dense markdown.
- Implement `parsePackageMap(markdown) -> data` for reading existing files.
- Unit tests for round-trip serialization.
3. **Implement file discovery (`src/discover.ts`)**
- Walk a directory tree.
- Respect `.gitignore` and a built-in ignore list (node_modules, .git, dist, etc.).
- Return a flat list of `(dirPath, filePaths[])` tuples.
- Unit tests with fixture directories.
4. **Implement init skeleton (`src/init.ts`)**
- Given a project root, create one `.pi-map.md` per directory.
- Populate `## files` with filenames only (no analysis yet).
- Populate `## role` and `## arch` with placeholder text.
- Command: `project-map:init --root <path>`
**Acceptance Criteria**
- Running `project-map:init` on a test repo creates `.pi-map.md` in every directory.
- Files match `.gitignore` rules correctly.
- Output format follows the dense-markdown spec from the design doc.
**Dependencies**: None.
---
### M2: LLM Extraction Layer (Week 2)
**Goal**: The LLM layer generates one-line purposes and package-level architecture notes.
#### Tasks
1. **Implement LLM file extraction (`src/llm-extract.ts`)**
- Prompt template: given a file's contents and path, return a one-line purpose and a list of exported symbols (if any).
- Prompt template: given a file's contents, infer its dependencies.
- Integrate with Pi's LLM tool calls (or a generic OpenAI-compatible client for standalone testing).
- Add caching: hash file contents, skip LLM call if unchanged.
2. **Implement LLM package extraction (`src/llm-extract.ts`)**
- Prompt template: given all per-file summaries in a directory, generate `## role` and `## arch`.
- Keep prompts token-efficient (truncate large files, focus on headers/exports).
3. **Wire LLM layer into init (`src/init.ts`)**
- After file discovery, run LLM extraction on every file.
- Then run package-level LLM extraction.
- Write fully populated `.pi-map.md` files.
4. **Add configuration (`src/config.ts`)**
- Configurable ignore patterns.
- Configurable LLM model and token limits.
- Configurable context budget for large projects.
**Acceptance Criteria**
- `project-map:init` produces `.pi-map.md` files with meaningful `## files`, `## role`, and `## arch` sections.
- Caching works: second run on unchanged repo is fast.
- Unit tests mock LLM responses to verify prompt structure and output parsing.
**Dependencies**: M1.
---
### M3: AST Extraction Layer (Week 3)
**Goal**: Code files get precise symbol and dependency data from tree-sitter or LSP.
#### Tasks
1. **Set up tree-sitter (`src/ast-extract.ts`)**
- Add tree-sitter dependencies for target languages (TypeScript, Python, Go, Rust as first-class).
- Implement `extractSymbols(filePath, language) -> { exports, imports, types }`.
- Implement `extractDependencies(filePath, language) -> string[]`.
2. **Language detection**
- Map file extensions to tree-sitter grammars.
- Graceful fallback to LLM-only if no parser available.
3. **Merge AST + LLM outputs (`src/merge.ts`)**
- Combine AST symbols with LLM purpose descriptions.
- AST provides `exp:` and `dep:`; LLM provides the one-line purpose.
- If AST and LLM disagree on dependencies, prefer AST for imports, LLM for inferred architectural deps.
4. **Wire AST into init (`src/init.ts`)**
- For each code file, run AST extraction in parallel with LLM extraction.
- Merge results before formatting.
**Acceptance Criteria**
- TypeScript/Python files show precise exported function/class names in `exp:`.
- Import statements are captured in `dep:`.
- Files with no AST parser still get LLM-only analysis.
- Unit tests with sample source files verify symbol extraction accuracy.
**Dependencies**: M2.
---
### M4: Patch and Update (Week 4)
**Goal**: The agent can incrementally update `.pi-map.md` files after editing source files.
#### Tasks
1. **Implement patch strategy (`src/patch.ts`)**
- Determine directory size (file count).
- If < 10 files: trigger full rewrite of that directory's `.pi-map.md`.
- If >= 10 files: parse existing `.pi-map.md`, identify the section(s) for changed file(s), and rewrite only those lines.
2. **Implement dirty markers (`src/patch.ts`)**
- When a section-level patch occurs, append a dirty note:
```markdown
## dirty
2024-06-09T14:32:00Z: tokens.ts patched (section-level)
```
3. **Command: `project-map:patch <file-path>`**
- Given a changed file path, find its directory.
- Re-run LLM + AST extraction on that file only.
- Apply patch strategy.
- Update `.pi-map.md`.
4. **Command: `project-map:reinit [path]`**
- Force full re-init of entire project or a subtree.
- Clear all dirty markers.
**Acceptance Criteria**
- Editing one file in a small package triggers full rewrite of that package's `.pi-map.md`.
- Editing one file in a large package patches only the relevant line.
- Dirty markers are correctly added on section-level patches.
- Re-init clears dirty markers and regenerates everything.
**Dependencies**: M3.
---
### M5: Validation and Consistency (Week 5)
**Goal**: The agent can detect and recover from stale analysis files.
#### Tasks
1. **Implement validation (`src/validate.ts`)**
- Walk all `.pi-map.md` files.
- For each listed file, check existence (catch deletions).
- For each directory, check for new files not in `.pi-map.md`.
- For code files with AST support, compare listed `exp:` against actual symbols.
- Report discrepancies with severity (missing, orphaned, stale-signature).
2. **Implement reconciliation (`src/validate.ts`)**
- Option `--fix`: automatically patch or flag discrepancies.
- Option `--reinit-if <threshold>`: if > N discrepancies, recommend full re-init.
3. **Command: `project-map:validate`**
- Run checks and print a report.
- Exit code 0 if clean, 1 if discrepancies found.
**Acceptance Criteria**
- Validation detects a deleted file that still appears in `.pi-map.md`.
- Validation detects a new file not yet in `.pi-map.md`.
- Validation detects a changed function signature.
- `--fix` patches or flags all found issues.
**Dependencies**: M4.
---
### M6: Pi Integration (Week 6)
**Goal**: The skill is installable and functional within the Pi agent harness.
#### Tasks
1. **Finalize `SKILL.md`**
- Document all custom tools: `project-map:init`, `project-map:patch`, `project-map:validate`, `project-map:reinit`.
- Document the dense-markdown format for the agent.
- Include usage examples and configuration options.
2. **Implement prompt hook (`hooks/on-prompt.ts`)**
- Inject a lightweight instruction into every prompt:
> "If you modify any source file, run `project-map:patch <path>`. If you suspect staleness, run `project-map:validate`."
- Make injection configurable (toggle on/off, customize message).
3. **Auto-inject session-start behavior**
- On session start, if `.pi-map.md` files exist, the skill auto-reads them into context.
- If they don't exist, the skill offers to run `project-map:init`.
- If dirty markers exist, the skill warns the agent.
4. **Packaging and publishing**
- Ensure `package.json` has correct `bin` entries.
- Write human README with install instructions.
- Tag and publish to npm (or internal registry).
**Acceptance Criteria**
- Installing the skill in Pi makes the four tools available.
- The prompt hook injects maintenance instructions reliably.
- Session start auto-detects and loads `.pi-map.md` files.
- The skill can be published and installed via npm.
**Dependencies**: M5.
---
## Sequencing and Dependencies
```
M1 (Foundation) → M2 (LLM) → M3 (AST) → M4 (Patch) → M5 (Validate) → M6 (Pi Integration)
```
No parallel tracks — each milestone builds on the previous. Total estimated time: 6 weeks at a steady pace, or 3-4 weeks with focused effort.
---
## Validation Criteria (Overall)
1. **Functional**: `project-map:init` correctly generates `.pi-map.md` for a test repo of 50+ files across 10+ directories.
2. **Accuracy**: AST-extracted symbols match actual source code (verified by unit tests).
3. **Freshness**: After editing a file, `project-map:patch` updates the analysis within 5 seconds.
4. **Consistency**: `project-map:validate` detects 100% of synthetic staleness scenarios (deleted files, new files, signature changes).
5. **Token efficiency**: A 100-file project's full `.pi-map.md` set fits within a 4k token budget.
6. **Pi integration**: The skill installs cleanly, tools are discoverable, and the prompt hook works.
---
## Rollout Plan
1. **Internal dogfooding** (Week 7): Use the skill on 2-3 real projects. Gather feedback on format density and agent behavior.
2. **Format refinement** (Week 8): Adjust abbreviations, section structure, and dirty-marker format based on dogfooding.
3. **Open beta** (Week 9): Share with a small group of Pi users. Collect bug reports and feature requests.
4. **v1.0 release** (Week 10): Stable API, documented format, published to npm.
---
## Open Questions for Implementation
- Which tree-sitter grammars to bundle by default? (Start with TypeScript, Python, Go.)
- Should the LLM extraction use the agent's current model or a dedicated cheaper model?
- How to handle monorepos with multiple `.gitignore` files at different depths?
- Should `.pi-map.md` files be committed to git or kept in `.gitignore`? (Recommendation: `.gitignore` — they are derived artifacts.)
+3588
View File
File diff suppressed because it is too large Load Diff
+39
View File
@@ -0,0 +1,39 @@
{
"name": "pi-project-map",
"version": "0.1.0",
"description": "Pi skill for hierarchical project analysis — generates and maintains .pi-map.md files",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"bin": {
"project-map": "dist/cli.js"
},
"scripts": {
"build": "tsc",
"dev": "tsc --watch",
"test": "vitest",
"lint": "eslint src/**/*.ts",
"typecheck": "tsc --noEmit"
},
"keywords": [
"pi",
"skill",
"project-analysis",
"code-intelligence",
"agent-context"
],
"author": "",
"license": "MIT",
"devDependencies": {
"@types/node": "^20.0.0",
"typescript": "^5.3.0",
"vitest": "^1.0.0",
"eslint": "^8.0.0",
"@typescript-eslint/eslint-plugin": "^6.0.0",
"@typescript-eslint/parser": "^6.0.0"
},
"dependencies": {
"tree-sitter": "^0.21.0",
"tree-sitter-typescript": "^0.21.0",
"ignore": "^5.3.0"
}
}
+13
View File
@@ -0,0 +1,13 @@
import { readFileSync } from 'fs';
interface ASTFileData {
exports: string[];
deps: string[];
}
export async function extractFileAST(filePath: string): Promise<ASTFileData | null> {
// TODO: integrate tree-sitter
// Detect language from extension
// Parse and extract symbols
return null;
}
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env node
import { initProject } from './init.js';
import { patchFile } from './patch.js';
import { validateMaps } from './validate.js';
import { reinitPath } from './init.js';
const args = process.argv.slice(2);
const command = args[0];
async function main() {
switch (command) {
case 'init':
await initProject(args[1] || '.');
break;
case 'patch':
await patchFile(args[1]);
break;
case 'validate': {
const result = await validateMaps(args[1] || '.');
process.exit(result.clean ? 0 : 1);
break;
}
case 'reinit':
await reinitPath(args[1] || '.');
break;
default:
console.log(`Usage: project-map <init|patch|validate|reinit> [path]`);
process.exit(1);
}
}
main();
+23
View File
@@ -0,0 +1,23 @@
export interface SkillConfig {
ignorePatterns: string[];
smallPackageThreshold: number;
llmModel: string;
contextBudget: number;
autoInjectPrompt: boolean;
}
export const DEFAULT_CONFIG: SkillConfig = {
ignorePatterns: [
'node_modules', '.git', 'dist', 'build', 'coverage',
'.next', '.venv', '__pycache__', '.DS_Store', '*.log'
],
smallPackageThreshold: 10,
llmModel: 'gpt-4o-mini',
contextBudget: 4000,
autoInjectPrompt: true
};
export function loadConfig(): SkillConfig {
// TODO: load from .pi-project-map.json or similar
return DEFAULT_CONFIG;
}
+56
View File
@@ -0,0 +1,56 @@
import { readdirSync, statSync } from 'fs';
import { join, relative } from 'path';
import ignore from 'ignore';
const DEFAULT_IGNORE = [
'node_modules', '.git', 'dist', 'build', 'coverage',
'.next', '.venv', '__pycache__', '.DS_Store', '*.log'
];
export interface DirectoryEntry {
dirPath: string;
relativePath: string;
files: string[];
}
export function discoverProject(rootPath: string): DirectoryEntry[] {
const ig = ignore().add(DEFAULT_IGNORE);
const gitignorePath = join(rootPath, '.gitignore');
try {
const gitignoreContent = require('fs').readFileSync(gitignorePath, 'utf8');
ig.add(gitignoreContent);
} catch { /* no .gitignore */ }
const entries: DirectoryEntry[] = [];
function walk(dir: string) {
const relDir = relative(rootPath, dir) || '.';
if (ig.ignores(relDir)) return;
const items = readdirSync(dir);
const files: string[] = [];
const subdirs: string[] = [];
for (const item of items) {
const relPath = join(relDir, item);
if (ig.ignores(relPath)) continue;
const fullPath = join(dir, item);
const st = statSync(fullPath);
if (st.isDirectory()) {
subdirs.push(fullPath);
} else {
files.push(item);
}
}
entries.push({ dirPath: dir, relativePath: relDir, files });
for (const subdir of subdirs) {
walk(subdir);
}
}
walk(rootPath);
return entries;
}
+40
View File
@@ -0,0 +1,40 @@
export interface PackageMapData {
path: string;
role: string;
files: FileEntry[];
arch: string;
dirty?: string;
}
export interface FileEntry {
name: string;
purpose: string;
exports: string[];
deps: string[];
}
export function renderPackageMap(data: PackageMapData): string {
const lines: string[] = [];
lines.push(`# ${data.path}`);
lines.push(`## role`);
lines.push(data.role);
lines.push(`## files`);
for (const file of data.files) {
const exp = file.exports.length > 0 ? `exp: ${file.exports.join(', ')}` : '';
const dep = file.deps.length > 0 ? `dep: ${file.deps.join(', ')}` : '';
const parts = [`- ${file.name} | ${file.purpose}`];
if (exp) parts.push(exp);
if (dep) parts.push(dep);
lines.push(parts.join(' | '));
}
lines.push(`## arch`);
lines.push(data.arch);
lines.push(`## dirty`);
lines.push(data.dirty || '-');
return `${lines.join('\n')}\n`;
}
export function parsePackageMap(_markdown: string): PackageMapData {
// TODO: implement robust parser
throw new Error('parsePackageMap not yet implemented');
}
+5
View File
@@ -0,0 +1,5 @@
// Main entry point for pi-project-map skill
export { initProject } from './init.js';
export { patchFile } from './patch.js';
export { validateMaps } from './validate.js';
export { renderPackageMap, parsePackageMap } from './format.js';
+42
View File
@@ -0,0 +1,42 @@
import { discoverProject } from './discover.js';
import { renderPackageMap, type PackageMapData } from './format.js';
import { extractFileLLM } from './llm-extract.js';
import { extractFileAST } from './ast-extract.js';
import { mergeFileData } from './merge.js';
import { extractPackageLLM } from './llm-extract.js';
import { writeFileSync } from 'fs';
import { join } from 'path';
export async function initProject(rootPath: string): Promise<void> {
const entries = discoverProject(rootPath);
for (const entry of entries) {
const fileData = [];
for (const file of entry.files) {
const filePath = join(entry.dirPath, file);
const llmData = await extractFileLLM(filePath);
const astData = await extractFileAST(filePath);
fileData.push(mergeFileData(file, llmData, astData));
}
const packageData = await extractPackageLLM(entry.relativePath, fileData);
const mapData: PackageMapData = {
path: entry.relativePath,
role: packageData.role,
files: fileData,
arch: packageData.arch,
dirty: '-'
};
const outPath = join(entry.dirPath, '.pi-map.md');
writeFileSync(outPath, renderPackageMap(mapData));
}
console.log(`Generated ${entries.length} .pi-map.md files`);
}
export async function reinitPath(path: string): Promise<void> {
// TODO: clear dirty markers and force regeneration
await initProject(path);
}
+48
View File
@@ -0,0 +1,48 @@
import { readFileSync } from 'fs';
import { createHash } from 'crypto';
interface LLMFileData {
purpose: string;
exports: string[];
deps: string[];
}
interface LLMPackageData {
role: string;
arch: string;
}
// Simple in-memory cache
const cache = new Map<string, LLMFileData>();
export async function extractFileLLM(filePath: string): Promise<LLMFileData> {
const content = readFileSync(filePath, 'utf8');
const hash = createHash('sha256').update(content).digest('hex');
if (cache.has(hash)) {
return cache.get(hash)!;
}
// TODO: integrate with Pi's LLM tool or a generic client
// For now, return placeholder data
const result: LLMFileData = {
purpose: 'TODO: analyze with LLM',
exports: [],
deps: []
};
cache.set(hash, result);
return result;
}
export async function extractPackageLLM(
relativePath: string,
fileData: { name: string; purpose: string }[]
): Promise<LLMPackageData> {
// TODO: integrate with Pi's LLM tool
// For now, return placeholder data
return {
role: `TODO: analyze package ${relativePath}`,
arch: 'TODO: architectural analysis'
};
}
+25
View File
@@ -0,0 +1,25 @@
import type { FileEntry } from './format.js';
interface LLMFileData {
purpose: string;
exports: string[];
deps: string[];
}
interface ASTFileData {
exports: string[];
deps: string[];
}
export function mergeFileData(
fileName: string,
llm: LLMFileData,
ast: ASTFileData | null
): FileEntry {
return {
name: fileName,
purpose: llm.purpose,
exports: ast?.exports ?? llm.exports,
deps: [...new Set([...(ast?.deps ?? []), ...llm.deps])]
};
}
+48
View File
@@ -0,0 +1,48 @@
import { dirname, join } from 'path';
import { existsSync, readFileSync, writeFileSync } from 'fs';
import { parsePackageMap, renderPackageMap } from './format.js';
import { extractFileLLM } from './llm-extract.js';
import { extractFileAST } from './ast-extract.js';
import { mergeFileData } from './merge.js';
import { readdirSync } from 'fs';
const SMALL_PACKAGE_THRESHOLD = 10;
export async function patchFile(filePath: string): Promise<void> {
const dirPath = dirname(filePath);
const mapPath = join(dirPath, '.pi-map.md');
if (!existsSync(mapPath)) {
// No map exists yet — would need to generate from scratch
console.warn(`No .pi-map.md found in ${dirPath}`);
return;
}
const allFiles = readdirSync(dirPath).filter((f: string) => !f.startsWith('.') && !f.endsWith('.md'));
const isSmallPackage = allFiles.length < SMALL_PACKAGE_THRESHOLD;
if (isSmallPackage) {
// Full rewrite for small packages
// TODO: import and reuse init logic for a single directory
console.log(`Full rewrite of ${mapPath} (small package: ${allFiles.length} files)`);
} else {
// Section-level patch
const existing = parsePackageMap(readFileSync(mapPath, 'utf8'));
const llmData = await extractFileLLM(filePath);
const astData = await extractFileAST(filePath);
const fileName = filePath.split('/').pop()!;
const updatedFile = mergeFileData(fileName, llmData, astData);
// Replace the matching file entry
const idx = existing.files.findIndex(f => f.name === updatedFile.name);
if (idx >= 0) {
existing.files[idx] = updatedFile;
} else {
existing.files.push(updatedFile);
}
existing.dirty = `${new Date().toISOString()}: ${fileName} patched (section-level)`;
writeFileSync(mapPath, renderPackageMap(existing));
console.log(`Patched ${mapPath}`);
}
}
+90
View File
@@ -0,0 +1,90 @@
import { discoverProject } from './discover.js';
import { parsePackageMap } from './format.js';
import { existsSync, readFileSync } from 'fs';
import { join } from 'path';
import { extractFileAST } from './ast-extract.js';
export interface ValidationResult {
clean: boolean;
discrepancies: Discrepancy[];
}
export interface Discrepancy {
type: 'missing' | 'orphaned' | 'stale-signature' | 'dirty';
path: string;
message: string;
}
export async function validateMaps(rootPath: string): Promise<ValidationResult> {
const discrepancies: Discrepancy[] = [];
const entries = discoverProject(rootPath);
for (const entry of entries) {
const mapPath = join(entry.dirPath, '.pi-map.md');
if (!existsSync(mapPath)) {
discrepancies.push({ type: 'missing', path: entry.relativePath, message: 'No .pi-map.md found' });
continue;
}
const mapData = parsePackageMap(readFileSync(mapPath, 'utf8'));
// Check for dirty markers
if (mapData.dirty && mapData.dirty !== '-') {
discrepancies.push({ type: 'dirty', path: mapPath, message: `Dirty: ${mapData.dirty}` });
}
// Check for orphaned entries
for (const fileEntry of mapData.files) {
const filePath = join(entry.dirPath, fileEntry.name);
if (!existsSync(filePath)) {
discrepancies.push({ type: 'orphaned', path: filePath, message: `File listed but deleted: ${fileEntry.name}` });
}
}
// Check for new files not in map
for (const file of entry.files) {
if (!mapData.files.find(f => f.name === file)) {
discrepancies.push({ type: 'missing', path: join(entry.dirPath, file), message: `File not in .pi-map.md: ${file}` });
}
}
// Check signatures for code files
for (const fileEntry of mapData.files) {
const filePath = join(entry.dirPath, fileEntry.name);
if (!existsSync(filePath)) continue;
const astData = await extractFileAST(filePath);
if (astData) {
const listedExports = new Set(fileEntry.exports);
const actualExports = new Set(astData.exports);
for (const exp of listedExports) {
if (!actualExports.has(exp)) {
discrepancies.push({ type: 'stale-signature', path: filePath, message: `Missing export: ${exp}` });
}
}
for (const exp of actualExports) {
if (!listedExports.has(exp)) {
discrepancies.push({ type: 'stale-signature', path: filePath, message: `New export: ${exp}` });
}
}
}
}
}
const result: ValidationResult = {
clean: discrepancies.length === 0,
discrepancies
};
if (result.clean) {
console.log('All .pi-map.md files are clean.');
} else {
console.log(`Found ${discrepancies.length} discrepancies:`);
for (const d of discrepancies) {
console.log(` [${d.type}] ${d.path}: ${d.message}`);
}
}
return result;
}
+19
View File
@@ -0,0 +1,19 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"esModuleInterop": true,
"strict": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"outDir": "./dist",
"rootDir": "./src",
"declaration": true,
"declarationMap": true,
"sourceMap": true,
"resolveJsonModule": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}