feat: smart git URL parsing for browser URLs
- Add git URL parsing utilities (extract_base_repo_url, parse_git_url) - Support GitHub, GitLab, Bitbucket browser URL detection - Add /projects/repositories/parse-url endpoint - Enhance repository creation to detect browser URLs and suggest corrections - Add real-time URL validation in frontend with debouncing - Show visual indicators (green/yellow/red) for URL validity - Display inline suggestions with 'Use Suggested' button - Add comprehensive unit tests for URL parsing - Quality gates: ruff ✓, mypy ✓, typecheck ✓, lint ✓, build ✓
This commit is contained in:
@@ -0,0 +1,154 @@
|
||||
# Smart Git URL Parsing - Design
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
User pastes URL → Frontend validates → Backend validates → Clone repo
|
||||
↓ ↓
|
||||
Show suggestions Parse & suggest
|
||||
```
|
||||
|
||||
## URL Detection Logic
|
||||
|
||||
### Patterns to Detect
|
||||
|
||||
1. **GitHub/GitLab/Bitbucket browser URLs**
|
||||
- `https://github.com/owner/repo/tree/branch-name`
|
||||
- `https://github.com/owner/repo/blob/branch/path/to/file`
|
||||
- `https://github.com/owner/repo/pull/123`
|
||||
- `https://gitlab.com/owner/repo/-/tree/branch`
|
||||
- `https://bitbucket.org/owner/repo/src/branch/`
|
||||
|
||||
2. **URLs with query parameters**
|
||||
- `https://github.com/owner/repo?tab=readme-ov-file`
|
||||
- `https://github.com/owner/repo.git?branch=develop`
|
||||
|
||||
3. **Valid clone URLs (should pass through)**
|
||||
- `https://github.com/owner/repo.git`
|
||||
- `git@github.com:owner/repo.git`
|
||||
- `https://github.com/owner/repo` (without .git)
|
||||
|
||||
### URL Parsing Rules
|
||||
|
||||
```python
|
||||
def extract_base_repo_url(url: str) -> str | None:
|
||||
"""Extract base repository URL from a browser/git URL.
|
||||
|
||||
Examples:
|
||||
https://github.com/user/repo/tree/main → https://github.com/user/repo.git
|
||||
https://github.com/user/repo.git → https://github.com/user/repo.git
|
||||
git@github.com:user/repo.git → git@github.com:user/repo.git
|
||||
https://gitlab.com/user/repo/-/blob/main/README.md → https://gitlab.com/user/repo.git
|
||||
|
||||
Returns None if URL doesn't match known patterns.
|
||||
"""
|
||||
...
|
||||
```
|
||||
|
||||
**Algorithm:**
|
||||
1. Remove query parameters
|
||||
2. Detect host (github.com, gitlab.com, bitbucket.org, etc.)
|
||||
3. For GitHub: Remove `/tree/*`, `/blob/*`, `/pull/*`, `/issues/*` paths
|
||||
4. For GitLab: Remove `/-/tree/*`, `/-/blob/*` paths
|
||||
5. For Bitbucket: Remove `/src/*` paths
|
||||
6. Ensure `.git` suffix
|
||||
7. Return cleaned URL or None
|
||||
|
||||
## API Changes
|
||||
|
||||
### POST /repositories (enhanced)
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"project_id": "uuid",
|
||||
"name": "my-repo",
|
||||
"remote_url": "https://github.com/user/repo/tree/main",
|
||||
"is_mirror": false
|
||||
}
|
||||
```
|
||||
|
||||
**New Response for Non-Repo URLs (422):**
|
||||
```json
|
||||
{
|
||||
"detail": "URL appears to be a browser URL, not a git clone URL",
|
||||
"suggested_url": "https://github.com/user/repo.git",
|
||||
"original_url": "https://github.com/user/repo/tree/main",
|
||||
"needs_confirmation": true
|
||||
}
|
||||
```
|
||||
|
||||
### New Endpoint: POST /repositories/parse-url
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"url": "https://github.com/user/repo/tree/main"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"original_url": "https://github.com/user/repo/tree/main",
|
||||
"base_url": "https://github.com/user/repo.git",
|
||||
"is_valid_repo_url": false,
|
||||
"needs_parsing": true,
|
||||
"message": "This looks like a browser URL. Did you mean to clone https://github.com/user/repo.git?"
|
||||
}
|
||||
```
|
||||
|
||||
## Frontend Flow
|
||||
|
||||
### Repository Creation Dialog (Enhanced)
|
||||
|
||||
1. **User pastes URL**
|
||||
2. **Frontend calls `/repositories/parse-url`** (debounced)
|
||||
3. **If URL needs parsing:**
|
||||
- Show yellow warning indicator
|
||||
- Display: "This looks like a browser URL"
|
||||
- Show suggested URL with "Use this instead" button
|
||||
- Allow user to proceed with original URL anyway
|
||||
4. **If URL is valid:**
|
||||
- Show green checkmark
|
||||
- Proceed normally
|
||||
5. **User clicks "Create"**
|
||||
6. **If backend returns 422 with suggestion:**
|
||||
- Show confirmation dialog with suggested URL
|
||||
- Options: "Use suggested URL", "Use original", "Cancel"
|
||||
|
||||
### UI Components
|
||||
|
||||
**URLInput Component:**
|
||||
- Input field with validation status icon
|
||||
- Shows inline suggestions when URL is detected as browser URL
|
||||
- Green/yellow/red border based on validation
|
||||
|
||||
**URLCorrectionDialog Component:**
|
||||
- Modal dialog for confirming URL correction
|
||||
- Shows before/after comparison
|
||||
- Clear action buttons
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. **Backend utilities** - URL parsing functions with tests
|
||||
2. **Backend endpoint** - `/repositories/parse-url`
|
||||
3. **Backend validation** - Enhanced POST /repositories with suggestion response
|
||||
4. **Frontend URL input** - Enhanced input with validation feedback
|
||||
5. **Frontend dialog** - Confirmation dialog for URL corrections
|
||||
6. **Integration** - Wire up parse-url endpoint to frontend
|
||||
7. **Tests** - Unit tests for URL parsing, integration tests for flow
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Invalid URLs
|
||||
- Completely malformed URLs: Return 400 with clear message
|
||||
- Unsupported hosts: Return 400 with "Unsupported git host"
|
||||
- Private repos (auth needed): Return 401/403 with auth instructions
|
||||
- Non-existent repos: Return 404 (from git clone failure)
|
||||
|
||||
### Clone Failures
|
||||
- Network issues: Retry with exponential backoff
|
||||
- Auth required: Prompt for credentials
|
||||
- Large repos: Show progress indicator
|
||||
- Timeout: Increase timeout for large repos
|
||||
Reference in New Issue
Block a user