alex/pi-map
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update design doc and implementation plan for proper LLM integration

Browse Source 
- design-doc.md: Rewrote Section 3 (Pipeline Architecture) to describe the
  real LLM integration: dual-provider LLM client (Pi native + external),
  SHA-256 disk cache in ~/.cache/pi-project-map/, parallelization with
  4-8 concurrent requests, retries with exponential backoff, hard-error
  policy, context limit protection. Added new Section 4 with concrete
  file-level and package-level prompts.
- implementation-plan.md: Replaced old milestone schedule with current
  status and detailed M7 tasks for LLM integration: LLM client abstraction,
  external API client, Pi LLM client, disk cache, parallel batching with
  retries, rewriting llm-extract.ts, context limits, CLI/extension wiring,
  and tests.
- Minor formatting cleanup in pi-extension.ts and src/cli.ts

All 16 tests pass. TypeScript compiles clean.
This commit is contained in:
Alex
2026-06-09 21:45:44 +02:00
parent 93c2ac60c5
commit 7b67205d43
4 changed files with 326 additions and 210 deletions
Download Patch File Download Diff File Expand all files Collapse all files
design-doc.md
+107 -13
View File
@@ -75,9 +75,10 @@ Two independent extraction layers contribute to the same output file.
#### Layer 1: LLM-Based Extraction (All Files)
- **Input**: Raw file contents of every non-excluded file in the directory.
- **Output**: One-line purpose description, architectural role, and cross-file relationships.
- **Output**: Purpose description, architectural role, and cross-file relationships.
- **Applies to**: Code files, config files, Dockerfiles, READMEs, YAML, JSON, shell scripts — everything.
- **When it runs**: Once per file during init; again on changed files during patching.
- **Implementation**: Calls an actual LLM (not regex heuristics). Inside Pi, it uses Pi's built-in LLM via the ExtensionAPI. Standalone CLI falls back to an external LLM API (OpenAI-compatible).
#### Layer 2: AST-Based Extraction (Code Files Only)
- **Input**: Source code of files where a tree-sitter or LSP parser is available.
@@ -93,17 +94,55 @@ The two layers merge into a single line per file under `## files`:
^ LLM ^ LLM ^ AST ^ AST + LLM
```
- File name and one-line purpose: LLM.
- File name and purpose: LLM.
- Exported symbols and signatures: AST (augmented by LLM if AST unavailable).
- Dependency list: AST for imports; LLM for inferred architectural dependencies.
### LLM Client Architecture
The LLM client is abstracted behind a unified interface:
```typescript
interface LLMClient {
complete(prompt: string): Promise<string>;
}
```
Two implementations:
1. **PiLLMClient** (Pi extension): Uses `ctx.model` or `ctx.modelRegistry` to invoke Pi's configured LLM. Called from `pi-extension.ts` when the skill runs inside Pi.
2. **ExternalLLMClient** (standalone CLI): Calls an external OpenAI-compatible API. Configured via environment variable (e.g., `OPENAI_API_KEY`) or config file.
### Caching
LLM results are cached to avoid re-querying unchanged files.
- **Key**: SHA-256 hash of file contents.
- **Storage**: JSON file at `~/.cache/pi-project-map/llm-cache.json`.
- **Behavior**: Before calling the LLM, compute the file hash and check the cache. If hit, reuse the cached result. If miss, call the LLM and store the result.
- **Invalidation**: Cache entries are implicitly invalidated when the file content changes (because the hash changes). There is no TTL; the cache is append-only.
### Parallelization and Rate Limiting
- **Concurrency**: 4-8 LLM calls in parallel, controlled by `p-limit`.
- **Batch delays**: A small delay (e.g., 100ms) is inserted between batches to avoid triggering rate limits.
- **Retry policy**: Each LLM call retries up to 3 times with exponential backoff (1s, 2s, 4s). If all retries fail, the entire operation stops with a hard error.
### Error Handling
- **Hard error on failure**: If an LLM call fails after all retries, `init` or `patch` stops immediately and prints a clear error. There is no heuristic fallback. The user must resolve the issue (set API key, wait for rate limit, check network).
- **Context limit protection**: Files larger than the LLM's context window are truncated from the end (with a note in the prompt) before being sent.
### Init Pipeline
```
For each directory (depth-first):
1. List all non-excluded files.
2. For each file:
a. Run LLM extraction (purpose, role).
b. If code file + parser available: run AST extraction (symbols, imports).
2. For each file (parallel, 4-8 concurrent):
a. Compute SHA-256 of file contents.
b. Check disk cache. If hit, use cached result.
c. If miss: call LLM (with retries/backoff) to extract purpose and role.
d. Store result in cache.
e. If code file + parser available: run AST extraction (symbols, imports).
3. Merge per-file outputs into lines.
4. Run LLM on merged lines + directory context to generate:
- `## role` (package-level summary)
@@ -117,13 +156,68 @@ When agent edits file(s) in directory:
1. Determine patch strategy:
- If directory has < 10 files: full rewrite.
- Else: section-level patch for changed file(s) only.
2. Re-run LLM extraction on changed file(s).
2. For each changed file:
a. Recompute SHA-256.
b. Check cache. If miss or stale, call LLM with retries/backoff.
3. Re-run AST extraction on changed file(s) if applicable.
4. Update `## files` section (rewrite or patch).
5. Update `## dirty` flag if full regeneration is deferred.
```
## 4. Consumption Model
## 4. LLM Prompt Design
### File-Level Prompt
The LLM prompt for a single file is designed to produce a structured, concise analysis.
```
You are analyzing a source file for a project map. Read the file below and summarize:
1. PURPOSE: What does this file do? Describe its role in the project (2-3 sentences max).
2. DEPENDENCIES: What does this file depend on? List internal modules/packages and external libraries.
3. KEY CONCEPTS: Mention any important patterns, algorithms, or domain concepts.
File path: <file-path>
```
<file-contents-truncated>
```
Respond in this exact format:
PURPOSE: <concise description>
DEPS: <comma-separated list, or "none">
CONCEPTS: <comma-separated list, or "none">
```
### Package-Level Prompt
After all file summaries are collected for a directory, a second LLM call synthesizes the package role and architecture.
```
You are analyzing a directory in a software project. Below is a list of files in this directory with their purposes.
Directory: <dir-path>
Files:
- <file1>: <purpose1>
- <file2>: <purpose2>
...
Respond in this exact format:
ROLE: <one-line description of this directory's role in the project>
ARCH: <2-4 sentences describing architecture, data flow, patterns, and design decisions>
```
### Output Parsing
The LLM client's response is parsed to extract `PURPOSE`, `DEPS`, `CONCEPTS`, `ROLE`, and `ARCH` fields. These are merged with AST data into the final `.pi-map.md` format.
### Context Limit Protection
- Files are truncated from the end if they exceed a configurable max token budget (default: 4000 tokens of source).
- A marker `[...truncated]` is appended to the truncated content so the LLM knows it is not seeing the full file.
- Very large binary or generated files are skipped entirely for LLM analysis (they still appear in `.pi-map.md` with a note like "Large/generated file").
## 5. Consumption Model
### Session Start
1. Agent discovers all `.pi-map.md` files (e.g., via `find . -name ".pi-map.md"`).
@@ -139,7 +233,7 @@ When agent edits file(s) in directory:
- For very large projects, the agent may summarize or prune the initial read, keeping only the top N levels of the hierarchy in active context.
- The skill can provide a "context budget" parameter: max tokens to spend on analysis files.
## 5. Stale Data Mitigation
## 6. Stale Data Mitigation
### Combined Strategy
@@ -167,7 +261,7 @@ When agent edits file(s) in directory:
- If validation finds staleness beyond a threshold (e.g., > 3 dirty packages), the skill recommends a full re-init.
- The agent can also trigger re-init for a specific subtree.
## 6. Scope Boundaries and Non-Goals
## 7. Scope Boundaries and Non-Goals
### In Scope
- Every directory in the project gets a `.pi-map.md` file.
@@ -182,7 +276,7 @@ When agent edits file(s) in directory:
- **Cross-project analysis**: Each project is independent. No global index across repos.
- **IDE integration**: This is a Pi agent skill, not a VS Code extension or LSP server.
## 7. Pi Skill Package Structure
## 8. Pi Skill Package Structure
```
pi-project-map/
@@ -212,7 +306,7 @@ pi-project-map/
- On every prompt, the skill appends a lightweight instruction:
> "If you modify any source file, run `project-map:patch <path>` to update the analysis. If you suspect staleness, run `project-map:validate`."
## 8. Risks and Tradeoffs
## 9. Risks and Tradeoffs
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
@@ -223,7 +317,7 @@ pi-project-map/
| Overlap with LSP/typedoc | Low | Low | This is agent-context, not IDE tooling. Different use case. |
| AST parser unavailable | Medium | Low | Graceful fallback to LLM-only extraction |
## 9. Concrete Example: Full Project Snapshot
## 10. Concrete Example: Full Project Snapshot
```
project-root/
@@ -250,7 +344,7 @@ project-root/
Each `.pi-map.md` follows the format in Section 2, creating a navigable hierarchy.
## 10. Future Extensions
## 11. Future Extensions
- **Cross-reference graph**: A top-level `project-graph.md` linking all packages with dependency arrows.
- **Search index**: A lightweight FTS5 index over all `.pi-map.md` files for fast symbol lookup.