Update design doc and implementation plan for proper LLM integration

- design-doc.md: Rewrote Section 3 (Pipeline Architecture) to describe the
  real LLM integration: dual-provider LLM client (Pi native + external),
  SHA-256 disk cache in ~/.cache/pi-project-map/, parallelization with
  4-8 concurrent requests, retries with exponential backoff, hard-error
  policy, context limit protection. Added new Section 4 with concrete
  file-level and package-level prompts.
- implementation-plan.md: Replaced old milestone schedule with current
  status and detailed M7 tasks for LLM integration: LLM client abstraction,
  external API client, Pi LLM client, disk cache, parallel batching with
  retries, rewriting llm-extract.ts, context limits, CLI/extension wiring,
  and tests.
- Minor formatting cleanup in pi-extension.ts and src/cli.ts

All 16 tests pass. TypeScript compiles clean.

implementation-plan.md

@@ -6,241 +6,236 @@ Build a Pi skill package (`pi-project-map`) that generates and maintains a hiera
 
 ---
 
-## Milestones
+## Current Status
 
-### M1: Foundation and Format (Week 1)
-**Goal**: A working dense-markdown formatter and the ability to generate `.pi-map.md` files from a directory listing.
+- **M1: Foundation and Format** — ✅ Complete
+- **M2: Heuristic Extraction** — ✅ Complete (placeholder only; to be replaced by real LLM)
+- **M3: AST Extraction** — ✅ Complete
+- **M4: Patch and Update** — ✅ Complete
+- **M5: Validation and `--fix`** — ✅ Complete
+- **M6: Pi Skill Integration** — ✅ Complete (basic)
+- **M7: Proper LLM Integration** — 🔄 In Progress / Next up
 
-#### Tasks
-1. **Bootstrap the npm package**
-   - Create `package.json`, `tsconfig.json`, basic CLI entry point.
-   - Set up test runner (vitest or jest).
-   - Create stub `SKILL.md` for Pi integration.
-
-2. **Implement the formatter (`src/format.ts`)**
-   - Define the `.pi-map.md` schema as a TypeScript interface.
-   - Implement `renderPackageMap(data) -> string` that outputs dense markdown.
-   - Implement `parsePackageMap(markdown) -> data` for reading existing files.
-   - Unit tests for round-trip serialization.
-
-3. **Implement file discovery (`src/discover.ts`)**
-   - Walk a directory tree.
-   - Respect `.gitignore` and a built-in ignore list (node_modules, .git, dist, etc.).
-   - Return a flat list of `(dirPath, filePaths[])` tuples.
-   - Unit tests with fixture directories.
-
-4. **Implement init skeleton (`src/init.ts`)**
-   - Given a project root, create one `.pi-map.md` per directory.
-   - Populate `## files` with filenames only (no analysis yet).
-   - Populate `## role` and `## arch` with placeholder text.
-   - Command: `project-map:init --root <path>`
-
-**Acceptance Criteria**
-- Running `project-map:init` on a test repo creates `.pi-map.md` in every directory.
-- Files match `.gitignore` rules correctly.
-- Output format follows the dense-markdown spec from the design doc.
-
-**Dependencies**: None.
+The remaining major work is to replace the heuristic "LLM" extraction with **actual LLM API calls**.
 
 ---
 
-### M2: LLM Extraction Layer (Week 2)
-**Goal**: The LLM layer generates one-line purposes and package-level architecture notes.
+## Remaining Milestone: M7 — Proper LLM Integration
 
-#### Tasks
-1. **Implement LLM file extraction (`src/llm-extract.ts`)**
-   - Prompt template: given a file's contents and path, return a one-line purpose and a list of exported symbols (if any).
-   - Prompt template: given a file's contents, infer its dependencies.
-   - Integrate with Pi's LLM tool calls (or a generic OpenAI-compatible client for standalone testing).
-   - Add caching: hash file contents, skip LLM call if unchanged.
+**Goal**: Replace the regex heuristic `llm-extract.ts` with real LLM calls, including dual-provider support (Pi native + external fallback), disk caching, parallelization, and retries.
 
-2. **Implement LLM package extraction (`src/llm-extract.ts`)**
-   - Prompt template: given all per-file summaries in a directory, generate `## role` and `## arch`.
-   - Keep prompts token-efficient (truncate large files, focus on headers/exports).
-
-3. **Wire LLM layer into init (`src/init.ts`)**
-   - After file discovery, run LLM extraction on every file.
-   - Then run package-level LLM extraction.
-   - Write fully populated `.pi-map.md` files.
-
-4. **Add configuration (`src/config.ts`)**
-   - Configurable ignore patterns.
-   - Configurable LLM model and token limits.
-   - Configurable context budget for large projects.
-
-**Acceptance Criteria**
-- `project-map:init` produces `.pi-map.md` files with meaningful `## files`, `## role`, and `## arch` sections.
-- Caching works: second run on unchanged repo is fast.
-- Unit tests mock LLM responses to verify prompt structure and output parsing.
-
-**Dependencies**: M1.
+**Estimated time**: 1 week of focused work.
 
 ---
 
-### M3: AST Extraction Layer (Week 3)
-**Goal**: Code files get precise symbol and dependency data from tree-sitter or LSP.
+### Task 1: LLM Client Abstraction (`src/llm-client.ts`)
 
-#### Tasks
-1. **Set up tree-sitter (`src/ast-extract.ts`)**
-   - Add tree-sitter dependencies for target languages (TypeScript, Python, Go, Rust as first-class).
-   - Implement `extractSymbols(filePath, language) -> { exports, imports, types }`.
-   - Implement `extractDependencies(filePath, language) -> string[]`.
+Create a unified interface for LLM calls.
 
-2. **Language detection**
-   - Map file extensions to tree-sitter grammars.
-   - Graceful fallback to LLM-only if no parser available.
+```typescript
+interface LLMClient {
+  complete(prompt: string): Promise<string>;
+}
+```
 
-3. **Merge AST + LLM outputs (`src/merge.ts`)**
-   - Combine AST symbols with LLM purpose descriptions.
-   - AST provides `exp:` and `dep:`; LLM provides the one-line purpose.
-   - If AST and LLM disagree on dependencies, prefer AST for imports, LLM for inferred architectural deps.
-
-4. **Wire AST into init (`src/init.ts`)**
-   - For each code file, run AST extraction in parallel with LLM extraction.
-   - Merge results before formatting.
+Sub-tasks:
+- Define the `LLMClient` interface.
+- Add a factory function `createLLMClient(mode: 'pi' | 'external', options)`.
+- Handle mode selection:
+  - `'pi'`: used inside `pi-extension.ts` with Pi's model registry.
+  - `'external'`: used in standalone CLI with OpenAI-compatible API.
 
 **Acceptance Criteria**
-- TypeScript/Python files show precise exported function/class names in `exp:`.
-- Import statements are captured in `dep:`.
-- Files with no AST parser still get LLM-only analysis.
-- Unit tests with sample source files verify symbol extraction accuracy.
-
-**Dependencies**: M2.
+- `LLMClient` interface exists and compiles.
+- Factory correctly selects implementation based on mode.
 
 ---
 
-### M4: Patch and Update (Week 4)
-**Goal**: The agent can incrementally update `.pi-map.md` files after editing source files.
+### Task 2: External LLM Client (`src/external-llm-client.ts`)
 
-#### Tasks
-1. **Implement patch strategy (`src/patch.ts`)**
-   - Determine directory size (file count).
-   - If < 10 files: trigger full rewrite of that directory's `.pi-map.md`.
-   - If >= 10 files: parse existing `.pi-map.md`, identify the section(s) for changed file(s), and rewrite only those lines.
+Implement the standalone CLI LLM client using an OpenAI-compatible API.
 
-2. **Implement dirty markers (`src/patch.ts`)**
-   - When a section-level patch occurs, append a dirty note:
-     ```markdown
-     ## dirty
-     2024-06-09T14:32:00Z: tokens.ts patched (section-level)
-     ```
-
-3. **Command: `project-map:patch <file-path>`**
-   - Given a changed file path, find its directory.
-   - Re-run LLM + AST extraction on that file only.
-   - Apply patch strategy.
-   - Update `.pi-map.md`.
-
-4. **Command: `project-map:reinit [path]`**
-   - Force full re-init of entire project or a subtree.
-   - Clear all dirty markers.
+Sub-tasks:
+- Add `openai` (or a lightweight fetch-based client) as a dependency.
+- Read configuration from:
+  - Environment variable `OPENAI_API_KEY` (or `ANTHROPIC_API_KEY`, etc.)
+  - Config file `.pi-project-map.json` fields: `llmProvider`, `llmModel`, `llmBaseUrl`
+- Implement `complete(prompt)` using chat completions API.
+- Default to `gpt-4o-mini` or similar cheap model.
 
 **Acceptance Criteria**
-- Editing one file in a small package triggers full rewrite of that package's `.pi-map.md`.
-- Editing one file in a large package patches only the relevant line.
-- Dirty markers are correctly added on section-level patches.
-- Re-init clears dirty markers and regenerates everything.
-
-**Dependencies**: M3.
+- `project-map init` works standalone with `OPENAI_API_KEY` set.
+- Missing API key produces a clear, actionable error message.
+- Failed API call throws a descriptive `LLMError`.
 
 ---
 
-### M5: Validation and Consistency (Week 5)
-**Goal**: The agent can detect and recover from stale analysis files.
+### Task 3: Pi LLM Client (`src/pi-llm-client.ts`)
 
-#### Tasks
-1. **Implement validation (`src/validate.ts`)**
-   - Walk all `.pi-map.md` files.
-   - For each listed file, check existence (catch deletions).
-   - For each directory, check for new files not in `.pi-map.md`.
-   - For code files with AST support, compare listed `exp:` against actual symbols.
-   - Report discrepancies with severity (missing, orphaned, stale-signature).
+Implement the Pi-native LLM client for use inside the extension.
 
-2. **Implement reconciliation (`src/validate.ts`)**
-   - Option `--fix`: automatically patch or flag discrepancies.
-   - Option `--reinit-if <threshold>`: if > N discrepancies, recommend full re-init.
-
-3. **Command: `project-map:validate`**
-   - Run checks and print a report.
-   - Exit code 0 if clean, 1 if discrepancies found.
+Sub-tasks:
+- Accept Pi's `ExtensionContext` or model registry as a constructor argument.
+- Use `ctx.model` / `ctx.modelRegistry` to get the configured model and API key.
+- Call the provider directly (likely using the same OpenAI-compatible endpoint Pi uses).
+- If Pi does not expose direct LLM calls, fall back to emitting a tool call / follow-up message pattern.
 
 **Acceptance Criteria**
-- Validation detects a deleted file that still appears in `.pi-map.md`.
-- Validation detects a new file not yet in `.pi-map.md`.
-- Validation detects a changed function signature.
-- `--fix` patches or flags all found issues.
-
-**Dependencies**: M4.
+- Pi extension can successfully call an LLM when running inside Pi.
+- Errors surface clearly to the user.
 
 ---
 
-### M6: Pi Integration (Week 6)
-**Goal**: The skill is installable and functional within the Pi agent harness.
+### Task 4: Disk Cache (`src/llm-cache.ts`)
 
-#### Tasks
-1. **Finalize `SKILL.md`**
-   - Document all custom tools: `project-map:init`, `project-map:patch`, `project-map:validate`, `project-map:reinit`.
-   - Document the dense-markdown format for the agent.
-   - Include usage examples and configuration options.
+Implement persistent SHA-256 → LLM result caching.
 
-2. **Implement prompt hook (`hooks/on-prompt.ts`)**
-   - Inject a lightweight instruction into every prompt:
-     > "If you modify any source file, run `project-map:patch <path>`. If you suspect staleness, run `project-map:validate`."
-   - Make injection configurable (toggle on/off, customize message).
-
-3. **Auto-inject session-start behavior**
-   - On session start, if `.pi-map.md` files exist, the skill auto-reads them into context.
-   - If they don't exist, the skill offers to run `project-map:init`.
-   - If dirty markers exist, the skill warns the agent.
-
-4. **Packaging and publishing**
-   - Ensure `package.json` has correct `bin` entries.
-   - Write human README with install instructions.
-   - Tag and publish to npm (or internal registry).
+Sub-tasks:
+- Cache directory: `~/.cache/pi-project-map/` (create if missing).
+- Cache file: `llm-cache.json` (simple JSON object).
+- Functions:
+  - `getCached(hash: string): string | null`
+  - `setCached(hash: string, result: string): void`
+- Ensure atomic writes (write to temp file, rename).
+- Add cache size limit (e.g., 10,000 entries, LRU eviction).
 
 **Acceptance Criteria**
-- Installing the skill in Pi makes the four tools available.
-- The prompt hook injects maintenance instructions reliably.
-- Session start auto-detects and loads `.pi-map.md` files.
-- The skill can be published and installed via npm.
+- Second `init` run on unchanged files does not call LLM.
+- Cache persists across process restarts.
+- Corrupted cache file does not crash the tool.
 
-**Dependencies**: M5.
+---
+
+### Task 5: Parallelization and Retries (`src/llm-batch.ts`)
+
+Run LLM requests in parallel with retries and backoff.
+
+Sub-tasks:
+- Add `p-limit` dependency for concurrency control.
+- Default concurrency: 4 (configurable via `.pi-project-map.json` `llmConcurrency`).
+- Add small delay (100ms) between batches.
+- Implement retry logic: 3 retries, delays 1s → 2s → 4s.
+- On final failure, throw a hard error and stop the entire process.
+
+**Acceptance Criteria**
+- 100-file project completes significantly faster than sequential.
+- Simulated transient failures are retried and recovered.
+- Persistent failure stops the tool with a clear error.
+
+---
+
+### Task 6: Rewrite `llm-extract.ts` to Use Real LLM
+
+Replace regex heuristics with LLM calls.
+
+Sub-tasks:
+- Accept an `LLMClient` in `extractFileLLM` and `extractPackageLLM`.
+- Check disk cache before calling LLM.
+- Construct file-level prompt (see design doc Section 4).
+- Parse response into `purpose`, `deps`, `concepts`.
+- Construct package-level prompt.
+- Parse response into `role`, `arch`.
+- Remove heuristic code from production path; keep only for test mocks if useful.
+
+**Acceptance Criteria**
+- `llm-extract.ts` calls the LLM client for every file.
+- Output includes rich, non-trivial descriptions for typical files.
+- Unit tests mock the LLM client to verify prompt structure and parsing.
+
+---
+
+### Task 7: Context Limit Handling
+
+Protect against oversized files.
+
+Sub-tasks:
+- Measure prompt + file content tokens (approximate: 1 token ≈ 4 chars for ASCII).
+- If file exceeds max context budget (configurable, default 4000 tokens), truncate from the end.
+- Append `[...truncated]` marker in the prompt.
+- Skip LLM for binary/generated files over a hard limit (e.g., 50KB) and mark them as "Large/generated file".
+
+**Acceptance Criteria**
+- A 1MB minified JS file does not crash or consume excessive tokens.
+- Truncated files still produce useful output.
+
+---
+
+### Task 8: Update CLI and Extension
+
+Wire the new LLM client into all entry points.
+
+Sub-tasks:
+- `src/cli.ts`: create external LLM client, pass into `initProject` / `patchFile`.
+- `pi-extension.ts`: create Pi LLM client, pass into tools.
+- Update `init.ts` and `patch.ts` signatures to accept an optional `LLMClient`.
+- Add CLI flag `--llm-provider=openai` for explicit selection.
+- Update error handling to catch `LLMError` and print helpful messages.
+
+**Acceptance Criteria**
+- CLI works with external API key.
+- Extension works inside Pi (if Pi exposes LLM access).
+- Clear errors on misconfiguration.
+
+---
+
+### Task 9: Update Tests
+
+Sub-tasks:
+- Replace heuristic tests with mocked LLM client tests.
+- Add integration test: create a fake LLM client, run `initProject`, verify output contains LLM-provided text.
+- Add cache test: verify cache hit skips LLM call.
+- Add retry test: verify transient failures retry, persistent failures hard-stop.
+
+**Acceptance Criteria**
+- All tests pass.
+- Test coverage includes: LLM client, cache, batching, prompt parsing, error handling.
 
 ---
 
 ## Sequencing and Dependencies
 
 ```
-M1 (Foundation) → M2 (LLM) → M3 (AST) → M4 (Patch) → M5 (Validate) → M6 (Pi Integration)
+Task 1 (LLMClient interface)
+  ├── Task 2 (External client)
+  ├── Task 3 (Pi client)
+  ├── Task 4 (Cache)
+  ├── Task 5 (Batch + retries)
+  │
+  ├── Task 6 (Rewrite llm-extract.ts)
+  ├── Task 7 (Context limits)
+  │
+  ├── Task 8 (Wire CLI + extension)
+  └── Task 9 (Tests)
 ```
 
-No parallel tracks — each milestone builds on the previous. Total estimated time: 6 weeks at a steady pace, or 3-4 weeks with focused effort.
-
 ---
 
-## Validation Criteria (Overall)
+## Validation Criteria (for LLM Integration)
 
-1. **Functional**: `project-map:init` correctly generates `.pi-map.md` for a test repo of 50+ files across 10+ directories.
-2. **Accuracy**: AST-extracted symbols match actual source code (verified by unit tests).
-3. **Freshness**: After editing a file, `project-map:patch` updates the analysis within 5 seconds.
-4. **Consistency**: `project-map:validate` detects 100% of synthetic staleness scenarios (deleted files, new files, signature changes).
-5. **Token efficiency**: A 100-file project's full `.pi-map.md` set fits within a 4k token budget.
-6. **Pi integration**: The skill installs cleanly, tools are discoverable, and the prompt hook works.
+1. **Real LLM calls**: `llm-extract.ts` invokes the configured LLM client for every file.
+2. **Cache hit**: Second `init` on unchanged repo completes with zero LLM calls.
+3. **Parallel speed**: 100-file project init completes in under 30 seconds (assuming average LLM latency 500ms).
+4. **Retry works**: Transient 429/5xx errors are retried; permanent failures stop with a clear error.
+5. **Context limit safety**: Files > max token budget are truncated, not rejected.
+6. **Dual provider**: CLI uses external API; Pi extension uses Pi's LLM.
+7. **Quality**: LLM output is visibly richer than the old heuristic output (verified by manual inspection).
 
 ---
 
 ## Rollout Plan
 
-1. **Internal dogfooding** (Week 7): Use the skill on 2-3 real projects. Gather feedback on format density and agent behavior.
-2. **Format refinement** (Week 8): Adjust abbreviations, section structure, and dirty-marker format based on dogfooding.
-3. **Open beta** (Week 9): Share with a small group of Pi users. Collect bug reports and feature requests.
-4. **v1.0 release** (Week 10): Stable API, documented format, published to npm.
+1. **Test on real projects** (Day 1-2): Run `init` on 2-3 real codebases with the new LLM integration.
+2. **Cost audit** (Day 3): Measure token usage per project; adjust defaults if too expensive.
+3. **Prompt tuning** (Day 4-5): Iterate prompt design based on output quality.
+4. **Release** (Day 6-7): Publish updated npm package, update Pi extension docs.
 
 ---
 
-## Open Questions for Implementation
+## Completed Milestones (for reference)
 
-- Which tree-sitter grammars to bundle by default? (Start with TypeScript, Python, Go.)
-- Should the LLM extraction use the agent's current model or a dedicated cheaper model?
-- How to handle monorepos with multiple `.gitignore` files at different depths?
-- Should `.pi-map.md` files be committed to git or kept in `.gitignore`? (Recommendation: `.gitignore` — they are derived artifacts.)
+- **M1**: Foundation and Format
+- **M2**: Heuristic Extraction (placeholder, to be replaced by M7)
+- **M3**: AST Extraction
+- **M4**: Patch and Update
+- **M5**: Validation and `--fix`
+- **M6**: Pi Skill Integration