634 lines
17 KiB
Markdown
634 lines
17 KiB
Markdown
# RFCP - Iteration 3.2.0: Comprehensive Performance & Bug Fixes
|
|
|
|
## Overview
|
|
|
|
Major iteration combining performance optimizations, UI fixes, and bug resolutions. This addresses the Detailed preset timeout, stuck progress bar, app close issues, region data problems, and UX improvements.
|
|
|
|
**Scope:** Backend optimizations + Frontend fixes + Electron fixes + Data validation
|
|
|
|
---
|
|
|
|
## Part 1: Performance Optimizations
|
|
|
|
### 1.1 Adaptive Resolution (CRITICAL)
|
|
|
|
**Problem:** 10km radius with 200m resolution = ~7850 points → timeout
|
|
|
|
**Solution:** Distance-based adaptive resolution
|
|
|
|
**File:** `backend/app/services/coverage_service.py` (or grid generation code)
|
|
|
|
```python
|
|
def get_adaptive_resolution(base_resolution: float, distance_from_tx: float) -> float:
|
|
"""
|
|
Adaptive resolution based on distance from transmitter.
|
|
|
|
Close to TX: use user's chosen resolution (details matter)
|
|
Far from TX: use coarser resolution (macro view sufficient)
|
|
"""
|
|
if distance_from_tx < 2000: # < 2km
|
|
return base_resolution # User's choice (e.g., 200m)
|
|
elif distance_from_tx < 5000: # 2-5km
|
|
return max(base_resolution, 300) # At least 300m
|
|
else: # > 5km
|
|
return max(base_resolution, 500) # At least 500m
|
|
|
|
|
|
def generate_adaptive_grid(center_lat, center_lon, radius_m, base_resolution):
|
|
"""
|
|
Generate grid with adaptive resolution zones.
|
|
|
|
Instead of uniform grid, create concentric zones with different resolutions.
|
|
"""
|
|
points = []
|
|
|
|
# Zone 1: Inner (< 2km) - full resolution
|
|
inner_points = generate_grid_ring(center_lat, center_lon, 0, 2000, base_resolution)
|
|
points.extend(inner_points)
|
|
|
|
# Zone 2: Middle (2-5km) - medium resolution
|
|
if radius_m > 2000:
|
|
medium_res = max(base_resolution, 300)
|
|
middle_points = generate_grid_ring(center_lat, center_lon, 2000, min(5000, radius_m), medium_res)
|
|
points.extend(middle_points)
|
|
|
|
# Zone 3: Outer (5km+) - coarse resolution
|
|
if radius_m > 5000:
|
|
coarse_res = max(base_resolution, 500)
|
|
outer_points = generate_grid_ring(center_lat, center_lon, 5000, radius_m, coarse_res)
|
|
points.extend(outer_points)
|
|
|
|
return points
|
|
```
|
|
|
|
**Expected result:**
|
|
- 10km with 200m base: ~7850 → ~2500 points (3x reduction)
|
|
- Combined with LOD: 10km detailed should complete in ~60s
|
|
|
|
### 1.2 Radial Preview Mode (NEW FEATURE)
|
|
|
|
**Purpose:** Instant preview using 360 radial spokes instead of full grid
|
|
|
|
**File:** `backend/app/services/coverage_service.py`
|
|
|
|
```python
|
|
def calculate_radial_preview(
|
|
tx_lat: float, tx_lon: float, tx_height: float,
|
|
radius_m: float, frequency_mhz: float,
|
|
num_spokes: int = 360, # 1 degree resolution
|
|
points_per_spoke: int = 50
|
|
) -> List[dict]:
|
|
"""
|
|
Fast radial preview calculation.
|
|
|
|
Instead of grid, calculate along 360 radial lines from TX.
|
|
Much faster because:
|
|
- Terrain profile can be cached per spoke
|
|
- No building calculations (terrain only)
|
|
- Linear interpolation between points
|
|
"""
|
|
results = []
|
|
|
|
for angle_deg in range(num_spokes):
|
|
angle_rad = math.radians(angle_deg)
|
|
|
|
# Calculate points along this spoke
|
|
for i in range(points_per_spoke):
|
|
distance = (i + 1) * (radius_m / points_per_spoke)
|
|
|
|
# Calculate point position
|
|
rx_lat, rx_lon = move_point(tx_lat, tx_lon, distance, angle_deg)
|
|
|
|
# Simple terrain-only calculation (no buildings)
|
|
path_loss = calculate_terrain_path_loss(
|
|
tx_lat, tx_lon, tx_height,
|
|
rx_lat, rx_lon, 1.5, # Standard UE height
|
|
frequency_mhz
|
|
)
|
|
|
|
results.append({
|
|
'lat': rx_lat,
|
|
'lon': rx_lon,
|
|
'rsrp': tx_power_dbm - path_loss,
|
|
'distance': distance,
|
|
'angle': angle_deg
|
|
})
|
|
|
|
return results
|
|
```
|
|
|
|
**Add to API:** New endpoint or parameter `?mode=preview`
|
|
|
|
---
|
|
|
|
## Part 2: Bug Fixes
|
|
|
|
### 2.1 Progress Bar Stuck at "Initializing 5%" (CRITICAL)
|
|
|
|
**Problem:** Progress never updates past 5%
|
|
|
|
**Root Cause:** WebSocket messages not reaching frontend OR React state not updating
|
|
|
|
**Debug & Fix Steps:**
|
|
|
|
**Step 1: Backend - Verify messages are sent**
|
|
|
|
File: `backend/app/api/websocket.py` or `backend/app/services/coverage_service.py`
|
|
|
|
```python
|
|
import logging
|
|
logger = logging.getLogger(__name__)
|
|
|
|
async def send_progress(websocket, progress: int, status: str):
|
|
"""Send progress with logging"""
|
|
message = {"type": "progress", "progress": progress, "status": status}
|
|
logger.info(f"[WS] Sending progress: {progress}% - {status}")
|
|
await websocket.send_json(message)
|
|
await asyncio.sleep(0) # Yield to event loop
|
|
```
|
|
|
|
**Step 2: Frontend - Check WebSocket handling**
|
|
|
|
File: `frontend/src/services/websocket.ts` or similar
|
|
|
|
```typescript
|
|
// Add logging
|
|
socket.onmessage = (event) => {
|
|
const data = JSON.parse(event.data);
|
|
console.log('[WS] Received:', data);
|
|
|
|
if (data.type === 'progress') {
|
|
console.log('[WS] Progress update:', data.progress, data.status);
|
|
// Update store
|
|
setCoverageProgress(data.progress, data.status);
|
|
}
|
|
};
|
|
```
|
|
|
|
**Step 3: Frontend - Check React state update**
|
|
|
|
File: `frontend/src/store/coverage.ts` or state management
|
|
|
|
```typescript
|
|
// Ensure state updates trigger re-render
|
|
setCoverageProgress: (progress: number, status: string) => {
|
|
console.log('[Store] Setting progress:', progress, status);
|
|
set({
|
|
progress: progress, // Must be new value, not mutation
|
|
progressStatus: status
|
|
});
|
|
}
|
|
```
|
|
|
|
**Step 4: Frontend - Check component subscription**
|
|
|
|
File: `frontend/src/App.tsx` or progress display component
|
|
|
|
```typescript
|
|
// Ensure component subscribes to store changes
|
|
const progress = useCoverageStore((state) => state.progress);
|
|
const progressStatus = useCoverageStore((state) => state.progressStatus);
|
|
|
|
useEffect(() => {
|
|
console.log('[UI] Progress changed:', progress, progressStatus);
|
|
}, [progress, progressStatus]);
|
|
```
|
|
|
|
### 2.2 App Close Button Broken (Electron)
|
|
|
|
**Problem:** Clicking X kills backend but Electron window stays open
|
|
|
|
**File:** `desktop/main.js`
|
|
|
|
```javascript
|
|
const { app, BrowserWindow } = require('electron');
|
|
const { spawn } = require('child_process');
|
|
|
|
let mainWindow;
|
|
let backendProcess;
|
|
|
|
function createWindow() {
|
|
mainWindow = new BrowserWindow({
|
|
// ... existing config
|
|
});
|
|
|
|
// Handle window close
|
|
mainWindow.on('close', async (event) => {
|
|
event.preventDefault(); // Prevent immediate close
|
|
|
|
console.log('[Electron] Window closing, cleaning up...');
|
|
|
|
// Kill backend process
|
|
if (backendProcess) {
|
|
console.log('[Electron] Killing backend process...');
|
|
backendProcess.kill('SIGTERM');
|
|
|
|
// Wait for graceful shutdown
|
|
await new Promise(resolve => setTimeout(resolve, 1000));
|
|
|
|
// Force kill if still running
|
|
if (!backendProcess.killed) {
|
|
backendProcess.kill('SIGKILL');
|
|
}
|
|
}
|
|
|
|
// Now actually close
|
|
mainWindow.destroy();
|
|
});
|
|
|
|
mainWindow.on('closed', () => {
|
|
mainWindow = null;
|
|
});
|
|
}
|
|
|
|
// Ensure app quits when all windows closed
|
|
app.on('window-all-closed', () => {
|
|
console.log('[Electron] All windows closed, quitting app');
|
|
app.quit();
|
|
});
|
|
|
|
// Cleanup on app quit
|
|
app.on('before-quit', () => {
|
|
console.log('[Electron] App quitting, final cleanup');
|
|
if (backendProcess && !backendProcess.killed) {
|
|
backendProcess.kill('SIGKILL');
|
|
}
|
|
});
|
|
```
|
|
|
|
### 2.3 Memory Leak (1328 MB not released)
|
|
|
|
**Problem:** Memory not freed after calculation
|
|
|
|
**File:** `backend/app/services/parallel_coverage_service.py` or `backend/app/parallel/manager.py`
|
|
|
|
```python
|
|
import gc
|
|
from multiprocessing import shared_memory
|
|
|
|
class SharedMemoryManager:
|
|
def __init__(self):
|
|
self._shared_blocks = []
|
|
|
|
def create_shared_block(self, name, size):
|
|
shm = shared_memory.SharedMemory(name=name, create=True, size=size)
|
|
self._shared_blocks.append(shm)
|
|
return shm
|
|
|
|
def cleanup(self):
|
|
"""Explicitly cleanup all shared memory blocks"""
|
|
for shm in self._shared_blocks:
|
|
try:
|
|
shm.close()
|
|
shm.unlink() # Important! Actually frees the memory
|
|
except Exception as e:
|
|
logger.warning(f"Error cleaning up shared memory: {e}")
|
|
self._shared_blocks.clear()
|
|
|
|
# Force garbage collection
|
|
gc.collect()
|
|
|
|
|
|
# In coverage calculation:
|
|
async def calculate_coverage(...):
|
|
shm_manager = SharedMemoryManager()
|
|
try:
|
|
# ... calculation code
|
|
pass
|
|
finally:
|
|
# ALWAYS cleanup, even on error/timeout
|
|
shm_manager.cleanup()
|
|
logger.info("[MEMORY] Shared memory cleaned up")
|
|
```
|
|
|
|
### 2.4 Region Data / Map Cache Issues
|
|
|
|
**Problem:** Western Ukraine region hangs or produces "No coverage points found"
|
|
|
|
**Diagnosis steps:**
|
|
|
|
**Step 1: Check ProgramData folder structure**
|
|
|
|
```python
|
|
# Add diagnostic endpoint or startup check
|
|
import os
|
|
|
|
def diagnose_data_folders():
|
|
"""Check data folder structure and validity"""
|
|
|
|
# Common locations
|
|
locations = [
|
|
os.path.expandvars(r'%PROGRAMDATA%\RFCP'),
|
|
os.path.expandvars(r'%APPDATA%\RFCP'),
|
|
os.path.expandvars(r'%LOCALAPPDATA%\RFCP'),
|
|
'./data',
|
|
'../data'
|
|
]
|
|
|
|
report = {}
|
|
for loc in locations:
|
|
if os.path.exists(loc):
|
|
report[loc] = {
|
|
'exists': True,
|
|
'files': os.listdir(loc),
|
|
'size_mb': sum(
|
|
os.path.getsize(os.path.join(loc, f))
|
|
for f in os.listdir(loc)
|
|
if os.path.isfile(os.path.join(loc, f))
|
|
) / 1024 / 1024
|
|
}
|
|
|
|
# Check terrain tiles
|
|
terrain_dir = os.path.join(loc, 'terrain')
|
|
if os.path.exists(terrain_dir):
|
|
tiles = [f for f in os.listdir(terrain_dir) if f.endswith('.hgt')]
|
|
report[loc]['terrain_tiles'] = len(tiles)
|
|
|
|
# Check OSM cache
|
|
osm_dir = os.path.join(loc, 'osm_cache')
|
|
if os.path.exists(osm_dir):
|
|
cache_files = os.listdir(osm_dir)
|
|
report[loc]['osm_cache_files'] = len(cache_files)
|
|
|
|
return report
|
|
```
|
|
|
|
**Step 2: Validate terrain tiles**
|
|
|
|
```python
|
|
def validate_terrain_tile(filepath: str) -> dict:
|
|
"""Check if terrain tile is valid"""
|
|
import struct
|
|
|
|
result = {
|
|
'path': filepath,
|
|
'exists': os.path.exists(filepath),
|
|
'valid': False,
|
|
'error': None
|
|
}
|
|
|
|
if not result['exists']:
|
|
result['error'] = 'File not found'
|
|
return result
|
|
|
|
try:
|
|
size = os.path.getsize(filepath)
|
|
|
|
# SRTM1 (1 arc-second): 3601x3601x2 = 25,934,402 bytes
|
|
# SRTM3 (3 arc-second): 1201x1201x2 = 2,884,802 bytes
|
|
|
|
if size == 25934402:
|
|
result['type'] = 'SRTM1'
|
|
result['valid'] = True
|
|
elif size == 2884802:
|
|
result['type'] = 'SRTM3'
|
|
result['valid'] = True
|
|
else:
|
|
result['error'] = f'Unexpected size: {size} bytes'
|
|
|
|
except Exception as e:
|
|
result['error'] = str(e)
|
|
|
|
return result
|
|
```
|
|
|
|
**Step 3: Fix OSM cache for regions**
|
|
|
|
```python
|
|
def get_osm_cache_key(bbox: tuple, data_type: str) -> str:
|
|
"""Generate consistent cache key for OSM data"""
|
|
# Round to avoid floating point issues
|
|
lat_min = round(bbox[0], 4)
|
|
lon_min = round(bbox[1], 4)
|
|
lat_max = round(bbox[2], 4)
|
|
lon_max = round(bbox[3], 4)
|
|
|
|
return f"{data_type}_{lat_min}_{lon_min}_{lat_max}_{lon_max}.json"
|
|
|
|
|
|
def validate_osm_cache(cache_path: str) -> bool:
|
|
"""Check if cached OSM data is valid"""
|
|
try:
|
|
with open(cache_path, 'r') as f:
|
|
data = json.load(f)
|
|
|
|
# Check structure
|
|
if not isinstance(data, dict):
|
|
return False
|
|
if 'elements' not in data and not isinstance(data, list):
|
|
return False
|
|
|
|
return True
|
|
except:
|
|
return False
|
|
```
|
|
|
|
---
|
|
|
|
## Part 3: UI Fixes
|
|
|
|
### 3.1 Calculate Button Position
|
|
|
|
**Problem:** Button overlaps with scrollbar
|
|
|
|
**File:** `frontend/src/components/CoverageSettings.tsx` or similar
|
|
|
|
```tsx
|
|
// Move Calculate button outside scrollable area
|
|
// Or add right margin
|
|
|
|
<div className="coverage-settings-panel">
|
|
<div className="scrollable-content">
|
|
{/* All settings */}
|
|
</div>
|
|
|
|
<div className="fixed-footer" style={{
|
|
padding: '16px',
|
|
borderTop: '1px solid var(--border-color)',
|
|
marginRight: '16px' // Space for scrollbar
|
|
}}>
|
|
<button
|
|
className="calculate-button"
|
|
onClick={handleCalculate}
|
|
style={{ width: '100%' }}
|
|
>
|
|
Calculate Coverage
|
|
</button>
|
|
</div>
|
|
</div>
|
|
```
|
|
|
|
### 3.2 Site Drag - Move Sectors Together
|
|
|
|
**Problem:** Dragging site doesn't move its sectors
|
|
|
|
**File:** `frontend/src/components/map/SiteMarker.tsx` or site handling code
|
|
|
|
```typescript
|
|
const handleSiteDrag = (siteId: string, newLat: number, newLon: number) => {
|
|
const site = getSite(siteId);
|
|
if (!site) return;
|
|
|
|
// Calculate delta
|
|
const deltaLat = newLat - site.lat;
|
|
const deltaLon = newLon - site.lon;
|
|
|
|
// Update site position
|
|
updateSite(siteId, { lat: newLat, lon: newLon });
|
|
|
|
// Update all sectors of this site
|
|
const sectors = getSectorsForSite(siteId);
|
|
sectors.forEach(sector => {
|
|
updateSector(sector.id, {
|
|
lat: sector.lat + deltaLat,
|
|
lon: sector.lon + deltaLon
|
|
});
|
|
});
|
|
};
|
|
```
|
|
|
|
### 3.3 Site Delete - Remove Sectors Together
|
|
|
|
**Problem:** Deleting site doesn't remove its sectors
|
|
|
|
**File:** `frontend/src/store/sites.ts` or site management
|
|
|
|
```typescript
|
|
const deleteSite = (siteId: string) => {
|
|
// First, delete all sectors belonging to this site
|
|
const sectors = get().sectors.filter(s => s.siteId === siteId);
|
|
sectors.forEach(sector => {
|
|
deleteSector(sector.id);
|
|
});
|
|
|
|
// Then delete the site
|
|
set(state => ({
|
|
sites: state.sites.filter(s => s.id !== siteId)
|
|
}));
|
|
};
|
|
```
|
|
|
|
---
|
|
|
|
## Part 4: Testing Checklist
|
|
|
|
### Performance Tests
|
|
- [ ] 5km Standard: < 10 seconds
|
|
- [ ] 5km Detailed: < 60 seconds
|
|
- [ ] 10km Standard: < 30 seconds
|
|
- [ ] 10km Detailed: < 120 seconds (was timeout)
|
|
- [ ] Radial preview (any radius): < 5 seconds
|
|
|
|
### Bug Fix Tests
|
|
- [ ] Progress bar updates from 5% → 100%
|
|
- [ ] App closes completely when clicking X
|
|
- [ ] Memory returns to baseline after calculation
|
|
- [ ] Western Ukraine region calculates successfully
|
|
- [ ] All terrain tiles validate correctly
|
|
- [ ] OSM cache files are valid JSON
|
|
|
|
### UI Tests
|
|
- [ ] Calculate button not overlapping scrollbar
|
|
- [ ] Dragging site moves all its sectors
|
|
- [ ] Deleting site removes all its sectors
|
|
- [ ] No console errors in browser DevTools
|
|
|
|
### Data Validation
|
|
- [ ] Run `diagnose_data_folders()` - check output
|
|
- [ ] Validate all terrain tiles in cache
|
|
- [ ] Validate all OSM cache files
|
|
- [ ] Check for corrupted files and remove them
|
|
|
|
---
|
|
|
|
## Build & Deploy
|
|
|
|
```powershell
|
|
# Backend
|
|
cd D:\root\rfcp\backend
|
|
pip install -e .
|
|
|
|
# Frontend
|
|
cd D:\root\rfcp\frontend
|
|
npm run build
|
|
|
|
# Electron
|
|
cd D:\root\rfcp\desktop
|
|
npm run build
|
|
|
|
# Full test
|
|
cd D:\root\rfcp\installer
|
|
.\test-detailed-quick.bat
|
|
|
|
# If works, rebuild installer
|
|
pyinstaller rfcp-server.spec --clean
|
|
```
|
|
|
|
---
|
|
|
|
## Commit Message
|
|
|
|
```
|
|
feat: Iteration 3.2.0 - Performance & Bug Fixes
|
|
|
|
Performance:
|
|
- Add adaptive resolution (distance-based grid density)
|
|
- Add radial preview mode (360 spokes, instant feedback)
|
|
- Expected: 10km Detailed ~60s (was timeout)
|
|
|
|
Bug Fixes:
|
|
- Fix progress bar stuck at 5% (WebSocket + React state)
|
|
- Fix app close button (Electron lifecycle)
|
|
- Fix memory leak (SharedMemory cleanup)
|
|
- Fix region data issues (cache validation)
|
|
|
|
UI Improvements:
|
|
- Move Calculate button (scrollbar overlap)
|
|
- Site drag moves all sectors
|
|
- Site delete removes all sectors
|
|
|
|
Data Validation:
|
|
- Add terrain tile validation
|
|
- Add OSM cache validation
|
|
- Add diagnostic reporting
|
|
```
|
|
|
|
---
|
|
|
|
## Success Criteria
|
|
|
|
1. **10km Detailed completes** without timeout (~60-120s acceptable)
|
|
2. **Progress bar works** - shows actual progress 5% → 100%
|
|
3. **App closes cleanly** - no orphan processes
|
|
4. **Memory released** - returns to baseline after calculation
|
|
5. **All regions work** - Western Ukraine calculates successfully
|
|
6. **Site management** - drag/delete affects sectors correctly
|
|
|
|
---
|
|
|
|
## Priority Order for Implementation
|
|
|
|
1. **Adaptive Resolution** - biggest performance impact
|
|
2. **Progress bar fix** - critical UX issue
|
|
3. **App close fix** - annoying bug
|
|
4. **Site drag/delete sectors** - quick win
|
|
5. **Calculate button position** - quick win
|
|
6. **Memory leak** - important but complex
|
|
7. **Region data validation** - diagnostic + fix
|
|
8. **Radial preview** - nice to have
|
|
|
|
---
|
|
|
|
## Notes for Claude Code
|
|
|
|
- This is a large iteration - take it step by step
|
|
- Test after each major change
|
|
- Backend and frontend changes may need coordination
|
|
- Electron changes require rebuild of desktop app
|
|
- Data validation can be added as debug endpoint first
|
|
- If stuck on one issue, move to next and come back
|
|
|
|
---
|
|
|
|
*"Big iteration, big impact. Let's make RFCP production-ready!"* 🚀
|