pi-map/implementation-plan.md

Implementation Plan: Hierarchical Project Analysis Skill for Pi

Overview

Build a Pi skill package (pi-project-map) that generates and maintains a hierarchical, machine-readable analysis of a software project. Each directory gets a .pi-map.md file. The skill provides custom tools for init, patch, validate, and re-init, plus a prompt hook that ensures the agent keeps analysis files in sync.

---

Current Status

• M1: Foundation and Format — ✅ Complete
• M2: Heuristic Extraction — ✅ Complete (placeholder only; to be replaced by real LLM)
• M3: AST Extraction — ✅ Complete
• M4: Patch and Update — ✅ Complete
• M5: Validation and --fix — ✅ Complete
• M6: Pi Skill Integration — ✅ Complete (basic)
• M7: Proper LLM Integration — 🔄 In Progress / Next up

The remaining major work is to replace the heuristic "LLM" extraction with actual LLM API calls.

---

Remaining Milestone: M7 — Proper LLM Integration

Goal: Replace the regex heuristic llm-extract.ts with real LLM calls, including dual-provider support (Pi native + external fallback), disk caching, parallelization, and retries.

Estimated time: 1 week of focused work.

---

Task 1: LLM Client Abstraction (src/llm-client.ts)

Create a unified interface for LLM calls.

interface LLMClient {
  complete(prompt: string): Promise<string>;
}

Sub-tasks:

• Define the LLMClient interface.
• Add a factory function createLLMClient(mode: 'pi' | 'external', options).
• Handle mode selection:
  • 'pi': used inside pi-extension.ts with Pi's model registry.
  • 'external': used in standalone CLI with OpenAI-compatible API.

Acceptance Criteria

• LLMClient interface exists and compiles.
• Factory correctly selects implementation based on mode.

---

Task 2: External LLM Client (src/external-llm-client.ts)

Implement the standalone CLI LLM client using an OpenAI-compatible API.

Sub-tasks:

• Add openai (or a lightweight fetch-based client) as a dependency.
• Read configuration from:
  • Environment variable OPENAI_API_KEY (or ANTHROPIC_API_KEY, etc.)
  • Config file .pi-project-map.json fields: llmProvider, llmModel, llmBaseUrl
• Implement complete(prompt) using chat completions API.
• Default to gpt-4o-mini or similar cheap model.

Acceptance Criteria

• project-map init works standalone with OPENAI_API_KEY set.
• Missing API key produces a clear, actionable error message.
• Failed API call throws a descriptive LLMError.

---

Task 3: Pi LLM Client (src/pi-llm-client.ts)

Implement the Pi-native LLM client for use inside the extension.

Sub-tasks:

• Accept Pi's ExtensionContext or model registry as a constructor argument.
• Use ctx.model / ctx.modelRegistry to get the configured model and API key.
• Call the provider directly (likely using the same OpenAI-compatible endpoint Pi uses).
• If Pi does not expose direct LLM calls, fall back to emitting a tool call / follow-up message pattern.

Acceptance Criteria

• Pi extension can successfully call an LLM when running inside Pi.
• Errors surface clearly to the user.

---

Task 4: Disk Cache (src/llm-cache.ts)

Implement persistent SHA-256 → LLM result caching.

Sub-tasks:

• Cache directory: ~/.cache/pi-project-map/ (create if missing).
• Cache file: llm-cache.json (simple JSON object).
• Functions:
  • getCached(hash: string): string | null
  • setCached(hash: string, result: string): void
• Ensure atomic writes (write to temp file, rename).
• Add cache size limit (e.g., 10,000 entries, LRU eviction).

Acceptance Criteria

• Second init run on unchanged files does not call LLM.
• Cache persists across process restarts.
• Corrupted cache file does not crash the tool.

---

Task 5: Parallelization and Retries (src/llm-batch.ts)

Run LLM requests in parallel with retries and backoff.

Sub-tasks:

• Add p-limit dependency for concurrency control.
• Default concurrency: 4 (configurable via .pi-project-map.json llmConcurrency).
• Add small delay (100ms) between batches.
• Implement retry logic: 3 retries, delays 1s → 2s → 4s.
• On final failure, throw a hard error and stop the entire process.

Acceptance Criteria

• 100-file project completes significantly faster than sequential.
• Simulated transient failures are retried and recovered.
• Persistent failure stops the tool with a clear error.

---

Task 6: Rewrite llm-extract.ts to Use Real LLM

Replace regex heuristics with LLM calls.

Sub-tasks:

• Accept an LLMClient in extractFileLLM and extractPackageLLM.
• Check disk cache before calling LLM.
• Construct file-level prompt (see design doc Section 4).
• Parse response into purpose, deps, concepts.
• Construct package-level prompt.
• Parse response into role, arch.
• Remove heuristic code from production path; keep only for test mocks if useful.

Acceptance Criteria

• llm-extract.ts calls the LLM client for every file.
• Output includes rich, non-trivial descriptions for typical files.
• Unit tests mock the LLM client to verify prompt structure and parsing.

---

Task 7: Context Limit Handling

Protect against oversized files.

Sub-tasks:

• Measure prompt + file content tokens (approximate: 1 token ≈ 4 chars for ASCII).
• If file exceeds max context budget (configurable, default 4000 tokens), truncate from the end.
• Append [...truncated] marker in the prompt.
• Skip LLM for binary/generated files over a hard limit (e.g., 50KB) and mark them as "Large/generated file".

Acceptance Criteria

• A 1MB minified JS file does not crash or consume excessive tokens.
• Truncated files still produce useful output.

---

Task 8: Update CLI and Extension

Wire the new LLM client into all entry points.

Sub-tasks:

• src/cli.ts: create external LLM client, pass into initProject / patchFile.
• pi-extension.ts: create Pi LLM client, pass into tools.
• Update init.ts and patch.ts signatures to accept an optional LLMClient.
• Add CLI flag --llm-provider=openai for explicit selection.
• Update error handling to catch LLMError and print helpful messages.

Acceptance Criteria

• CLI works with external API key.
• Extension works inside Pi (if Pi exposes LLM access).
• Clear errors on misconfiguration.

---

Task 9: Update Tests

Sub-tasks:

• Replace heuristic tests with mocked LLM client tests.
• Add integration test: create a fake LLM client, run initProject, verify output contains LLM-provided text.
• Add cache test: verify cache hit skips LLM call.
• Add retry test: verify transient failures retry, persistent failures hard-stop.

Acceptance Criteria

• All tests pass.
• Test coverage includes: LLM client, cache, batching, prompt parsing, error handling.

---

Sequencing and Dependencies

Task 1 (LLMClient interface)
  ├── Task 2 (External client)
  ├── Task 3 (Pi client)
  ├── Task 4 (Cache)
  ├── Task 5 (Batch + retries)
  │
  ├── Task 6 (Rewrite llm-extract.ts)
  ├── Task 7 (Context limits)
  │
  ├── Task 8 (Wire CLI + extension)
  └── Task 9 (Tests)

---

Validation Criteria (for LLM Integration)

1. Real LLM calls: llm-extract.ts invokes the configured LLM client for every file.
2. Cache hit: Second init on unchanged repo completes with zero LLM calls.
3. Parallel speed: 100-file project init completes in under 30 seconds (assuming average LLM latency 500ms).
4. Retry works: Transient 429/5xx errors are retried; permanent failures stop with a clear error.
5. Context limit safety: Files > max token budget are truncated, not rejected.
6. Dual provider: CLI uses external API; Pi extension uses Pi's LLM.
7. Quality: LLM output is visibly richer than the old heuristic output (verified by manual inspection).

---

Rollout Plan

1. Test on real projects (Day 1-2): Run init on 2-3 real codebases with the new LLM integration.
2. Cost audit (Day 3): Measure token usage per project; adjust defaults if too expensive.
3. Prompt tuning (Day 4-5): Iterate prompt design based on output quality.
4. Release (Day 6-7): Publish updated npm package, update Pi extension docs.

---

Completed Milestones (for reference)

• M1: Foundation and Format
• M2: Heuristic Extraction (placeholder, to be replaced by M7)
• M3: AST Extraction
• M4: Patch and Update
• M5: Validation and --fix
• M6: Pi Skill Integration