# Implementation Plan: Hierarchical Project Analysis Skill for Pi

## Overview

Build a Pi skill package (`pi-project-map`) that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a `.pi-map.md` file. The skill provides custom tools for init, patch, validate, and re-init, plus a prompt hook that ensures the agent keeps analysis files in sync.

---

## Milestones

### M1: Foundation and Format (Week 1)
**Goal**: A working dense-markdown formatter and the ability to generate `.pi-map.md` files from a directory listing.

#### Tasks
1. **Bootstrap the npm package**
   - Create `package.json`, `tsconfig.json`, basic CLI entry point.
   - Set up test runner (vitest or jest).
   - Create stub `SKILL.md` for Pi integration.

2. **Implement the formatter (`src/format.ts`)**
   - Define the `.pi-map.md` schema as a TypeScript interface.
   - Implement `renderPackageMap(data) -> string` that outputs dense markdown.
   - Implement `parsePackageMap(markdown) -> data` for reading existing files.
   - Unit tests for round-trip serialization.

3. **Implement file discovery (`src/discover.ts`)**
   - Walk a directory tree.
   - Respect `.gitignore` and a built-in ignore list (node_modules, .git, dist, etc.).
   - Return a flat list of `(dirPath, filePaths[])` tuples.
   - Unit tests with fixture directories.

4. **Implement init skeleton (`src/init.ts`)**
   - Given a project root, create one `.pi-map.md` per directory.
   - Populate `## files` with filenames only (no analysis yet).
   - Populate `## role` and `## arch` with placeholder text.
   - Command: `project-map:init --root <path>`

**Acceptance Criteria**
- Running `project-map:init` on a test repo creates `.pi-map.md` in every directory.
- Files match `.gitignore` rules correctly.
- Output format follows the dense-markdown spec from the design doc.

**Dependencies**: None.

---

### M2: LLM Extraction Layer (Week 2)
**Goal**: The LLM layer generates one-line purposes and package-level architecture notes.

#### Tasks
1. **Implement LLM file extraction (`src/llm-extract.ts`)**
   - Prompt template: given a file's contents and path, return a one-line purpose and a list of exported symbols (if any).
   - Prompt template: given a file's contents, infer its dependencies.
   - Integrate with Pi's LLM tool calls (or a generic OpenAI-compatible client for standalone testing).
   - Add caching: hash file contents, skip LLM call if unchanged.

2. **Implement LLM package extraction (`src/llm-extract.ts`)**
   - Prompt template: given all per-file summaries in a directory, generate `## role` and `## arch`.
   - Keep prompts token-efficient (truncate large files, focus on headers/exports).

3. **Wire LLM layer into init (`src/init.ts`)**
   - After file discovery, run LLM extraction on every file.
   - Then run package-level LLM extraction.
   - Write fully populated `.pi-map.md` files.

4. **Add configuration (`src/config.ts`)**
   - Configurable ignore patterns.
   - Configurable LLM model and token limits.
   - Configurable context budget for large projects.

**Acceptance Criteria**
- `project-map:init` produces `.pi-map.md` files with meaningful `## files`, `## role`, and `## arch` sections.
- Caching works: second run on unchanged repo is fast.
- Unit tests mock LLM responses to verify prompt structure and output parsing.

**Dependencies**: M1.

---

### M3: AST Extraction Layer (Week 3)
**Goal**: Code files get precise symbol and dependency data from tree-sitter or LSP.

#### Tasks
1. **Set up tree-sitter (`src/ast-extract.ts`)**
   - Add tree-sitter dependencies for target languages (TypeScript, Python, Go, Rust as first-class).
   - Implement `extractSymbols(filePath, language) -> { exports, imports, types }`.
   - Implement `extractDependencies(filePath, language) -> string[]`.

2. **Language detection**
   - Map file extensions to tree-sitter grammars.
   - Graceful fallback to LLM-only if no parser available.

3. **Merge AST + LLM outputs (`src/merge.ts`)**
   - Combine AST symbols with LLM purpose descriptions.
   - AST provides `exp:` and `dep:`; LLM provides the one-line purpose.
   - If AST and LLM disagree on dependencies, prefer AST for imports, LLM for inferred architectural deps.

4. **Wire AST into init (`src/init.ts`)**
   - For each code file, run AST extraction in parallel with LLM extraction.
   - Merge results before formatting.

**Acceptance Criteria**
- TypeScript/Python files show precise exported function/class names in `exp:`.
- Import statements are captured in `dep:`.
- Files with no AST parser still get LLM-only analysis.
- Unit tests with sample source files verify symbol extraction accuracy.

**Dependencies**: M2.

---

### M4: Patch and Update (Week 4)
**Goal**: The agent can incrementally update `.pi-map.md` files after editing source files.

#### Tasks
1. **Implement patch strategy (`src/patch.ts`)**
   - Determine directory size (file count).
   - If < 10 files: trigger full rewrite of that directory's `.pi-map.md`.
   - If >= 10 files: parse existing `.pi-map.md`, identify the section(s) for changed file(s), and rewrite only those lines.

2. **Implement dirty markers (`src/patch.ts`)**
   - When a section-level patch occurs, append a dirty note:
     ```markdown
     ## dirty
     2024-06-09T14:32:00Z: tokens.ts patched (section-level)
     ```

3. **Command: `project-map:patch <file-path>`**
   - Given a changed file path, find its directory.
   - Re-run LLM + AST extraction on that file only.
   - Apply patch strategy.
   - Update `.pi-map.md`.

4. **Command: `project-map:reinit [path]`**
   - Force full re-init of entire project or a subtree.
   - Clear all dirty markers.

**Acceptance Criteria**
- Editing one file in a small package triggers full rewrite of that package's `.pi-map.md`.
- Editing one file in a large package patches only the relevant line.
- Dirty markers are correctly added on section-level patches.
- Re-init clears dirty markers and regenerates everything.

**Dependencies**: M3.

---

### M5: Validation and Consistency (Week 5)
**Goal**: The agent can detect and recover from stale analysis files.

#### Tasks
1. **Implement validation (`src/validate.ts`)**
   - Walk all `.pi-map.md` files.
   - For each listed file, check existence (catch deletions).
   - For each directory, check for new files not in `.pi-map.md`.
   - For code files with AST support, compare listed `exp:` against actual symbols.
   - Report discrepancies with severity (missing, orphaned, stale-signature).

2. **Implement reconciliation (`src/validate.ts`)**
   - Option `--fix`: automatically patch or flag discrepancies.
   - Option `--reinit-if <threshold>`: if > N discrepancies, recommend full re-init.

3. **Command: `project-map:validate`**
   - Run checks and print a report.
   - Exit code 0 if clean, 1 if discrepancies found.

**Acceptance Criteria**
- Validation detects a deleted file that still appears in `.pi-map.md`.
- Validation detects a new file not yet in `.pi-map.md`.
- Validation detects a changed function signature.
- `--fix` patches or flags all found issues.

**Dependencies**: M4.

---

### M6: Pi Integration (Week 6)
**Goal**: The skill is installable and functional within the Pi agent harness.

#### Tasks
1. **Finalize `SKILL.md`**
   - Document all custom tools: `project-map:init`, `project-map:patch`, `project-map:validate`, `project-map:reinit`.
   - Document the dense-markdown format for the agent.
   - Include usage examples and configuration options.

2. **Implement prompt hook (`hooks/on-prompt.ts`)**
   - Inject a lightweight instruction into every prompt:
     > "If you modify any source file, run `project-map:patch <path>`. If you suspect staleness, run `project-map:validate`."
   - Make injection configurable (toggle on/off, customize message).

3. **Auto-inject session-start behavior**
   - On session start, if `.pi-map.md` files exist, the skill auto-reads them into context.
   - If they don't exist, the skill offers to run `project-map:init`.
   - If dirty markers exist, the skill warns the agent.

4. **Packaging and publishing**
   - Ensure `package.json` has correct `bin` entries.
   - Write human README with install instructions.
   - Tag and publish to npm (or internal registry).

**Acceptance Criteria**
- Installing the skill in Pi makes the four tools available.
- The prompt hook injects maintenance instructions reliably.
- Session start auto-detects and loads `.pi-map.md` files.
- The skill can be published and installed via npm.

**Dependencies**: M5.

---

## Sequencing and Dependencies

```
M1 (Foundation) → M2 (LLM) → M3 (AST) → M4 (Patch) → M5 (Validate) → M6 (Pi Integration)
```

No parallel tracks — each milestone builds on the previous. Total estimated time: 6 weeks at a steady pace, or 3-4 weeks with focused effort.

---

## Validation Criteria (Overall)

1. **Functional**: `project-map:init` correctly generates `.pi-map.md` for a test repo of 50+ files across 10+ directories.
2. **Accuracy**: AST-extracted symbols match actual source code (verified by unit tests).
3. **Freshness**: After editing a file, `project-map:patch` updates the analysis within 5 seconds.
4. **Consistency**: `project-map:validate` detects 100% of synthetic staleness scenarios (deleted files, new files, signature changes).
5. **Token efficiency**: A 100-file project's full `.pi-map.md` set fits within a 4k token budget.
6. **Pi integration**: The skill installs cleanly, tools are discoverable, and the prompt hook works.

---

## Rollout Plan

1. **Internal dogfooding** (Week 7): Use the skill on 2-3 real projects. Gather feedback on format density and agent behavior.
2. **Format refinement** (Week 8): Adjust abbreviations, section structure, and dirty-marker format based on dogfooding.
3. **Open beta** (Week 9): Share with a small group of Pi users. Collect bug reports and feature requests.
4. **v1.0 release** (Week 10): Stable API, documented format, published to npm.

---

## Open Questions for Implementation

- Which tree-sitter grammars to bundle by default? (Start with TypeScript, Python, Go.)
- Should the LLM extraction use the agent's current model or a dedicated cheaper model?
- How to handle monorepos with multiple `.gitignore` files at different depths?
- Should `.pi-map.md` files be committed to git or kept in `.gitignore`? (Recommendation: `.gitignore` — they are derived artifacts.)