@mytec: iter3.4.0 start
This commit is contained in:
@@ -0,0 +1,633 @@
|
||||
# RFCP - Iteration 3.2.0: Comprehensive Performance & Bug Fixes
|
||||
|
||||
## Overview
|
||||
|
||||
Major iteration combining performance optimizations, UI fixes, and bug resolutions. This addresses the Detailed preset timeout, stuck progress bar, app close issues, region data problems, and UX improvements.
|
||||
|
||||
**Scope:** Backend optimizations + Frontend fixes + Electron fixes + Data validation
|
||||
|
||||
---
|
||||
|
||||
## Part 1: Performance Optimizations
|
||||
|
||||
### 1.1 Adaptive Resolution (CRITICAL)
|
||||
|
||||
**Problem:** 10km radius with 200m resolution = ~7850 points → timeout
|
||||
|
||||
**Solution:** Distance-based adaptive resolution
|
||||
|
||||
**File:** `backend/app/services/coverage_service.py` (or grid generation code)
|
||||
|
||||
```python
|
||||
def get_adaptive_resolution(base_resolution: float, distance_from_tx: float) -> float:
|
||||
"""
|
||||
Adaptive resolution based on distance from transmitter.
|
||||
|
||||
Close to TX: use user's chosen resolution (details matter)
|
||||
Far from TX: use coarser resolution (macro view sufficient)
|
||||
"""
|
||||
if distance_from_tx < 2000: # < 2km
|
||||
return base_resolution # User's choice (e.g., 200m)
|
||||
elif distance_from_tx < 5000: # 2-5km
|
||||
return max(base_resolution, 300) # At least 300m
|
||||
else: # > 5km
|
||||
return max(base_resolution, 500) # At least 500m
|
||||
|
||||
|
||||
def generate_adaptive_grid(center_lat, center_lon, radius_m, base_resolution):
|
||||
"""
|
||||
Generate grid with adaptive resolution zones.
|
||||
|
||||
Instead of uniform grid, create concentric zones with different resolutions.
|
||||
"""
|
||||
points = []
|
||||
|
||||
# Zone 1: Inner (< 2km) - full resolution
|
||||
inner_points = generate_grid_ring(center_lat, center_lon, 0, 2000, base_resolution)
|
||||
points.extend(inner_points)
|
||||
|
||||
# Zone 2: Middle (2-5km) - medium resolution
|
||||
if radius_m > 2000:
|
||||
medium_res = max(base_resolution, 300)
|
||||
middle_points = generate_grid_ring(center_lat, center_lon, 2000, min(5000, radius_m), medium_res)
|
||||
points.extend(middle_points)
|
||||
|
||||
# Zone 3: Outer (5km+) - coarse resolution
|
||||
if radius_m > 5000:
|
||||
coarse_res = max(base_resolution, 500)
|
||||
outer_points = generate_grid_ring(center_lat, center_lon, 5000, radius_m, coarse_res)
|
||||
points.extend(outer_points)
|
||||
|
||||
return points
|
||||
```
|
||||
|
||||
**Expected result:**
|
||||
- 10km with 200m base: ~7850 → ~2500 points (3x reduction)
|
||||
- Combined with LOD: 10km detailed should complete in ~60s
|
||||
|
||||
### 1.2 Radial Preview Mode (NEW FEATURE)
|
||||
|
||||
**Purpose:** Instant preview using 360 radial spokes instead of full grid
|
||||
|
||||
**File:** `backend/app/services/coverage_service.py`
|
||||
|
||||
```python
|
||||
def calculate_radial_preview(
|
||||
tx_lat: float, tx_lon: float, tx_height: float,
|
||||
radius_m: float, frequency_mhz: float,
|
||||
num_spokes: int = 360, # 1 degree resolution
|
||||
points_per_spoke: int = 50
|
||||
) -> List[dict]:
|
||||
"""
|
||||
Fast radial preview calculation.
|
||||
|
||||
Instead of grid, calculate along 360 radial lines from TX.
|
||||
Much faster because:
|
||||
- Terrain profile can be cached per spoke
|
||||
- No building calculations (terrain only)
|
||||
- Linear interpolation between points
|
||||
"""
|
||||
results = []
|
||||
|
||||
for angle_deg in range(num_spokes):
|
||||
angle_rad = math.radians(angle_deg)
|
||||
|
||||
# Calculate points along this spoke
|
||||
for i in range(points_per_spoke):
|
||||
distance = (i + 1) * (radius_m / points_per_spoke)
|
||||
|
||||
# Calculate point position
|
||||
rx_lat, rx_lon = move_point(tx_lat, tx_lon, distance, angle_deg)
|
||||
|
||||
# Simple terrain-only calculation (no buildings)
|
||||
path_loss = calculate_terrain_path_loss(
|
||||
tx_lat, tx_lon, tx_height,
|
||||
rx_lat, rx_lon, 1.5, # Standard UE height
|
||||
frequency_mhz
|
||||
)
|
||||
|
||||
results.append({
|
||||
'lat': rx_lat,
|
||||
'lon': rx_lon,
|
||||
'rsrp': tx_power_dbm - path_loss,
|
||||
'distance': distance,
|
||||
'angle': angle_deg
|
||||
})
|
||||
|
||||
return results
|
||||
```
|
||||
|
||||
**Add to API:** New endpoint or parameter `?mode=preview`
|
||||
|
||||
---
|
||||
|
||||
## Part 2: Bug Fixes
|
||||
|
||||
### 2.1 Progress Bar Stuck at "Initializing 5%" (CRITICAL)
|
||||
|
||||
**Problem:** Progress never updates past 5%
|
||||
|
||||
**Root Cause:** WebSocket messages not reaching frontend OR React state not updating
|
||||
|
||||
**Debug & Fix Steps:**
|
||||
|
||||
**Step 1: Backend - Verify messages are sent**
|
||||
|
||||
File: `backend/app/api/websocket.py` or `backend/app/services/coverage_service.py`
|
||||
|
||||
```python
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
async def send_progress(websocket, progress: int, status: str):
|
||||
"""Send progress with logging"""
|
||||
message = {"type": "progress", "progress": progress, "status": status}
|
||||
logger.info(f"[WS] Sending progress: {progress}% - {status}")
|
||||
await websocket.send_json(message)
|
||||
await asyncio.sleep(0) # Yield to event loop
|
||||
```
|
||||
|
||||
**Step 2: Frontend - Check WebSocket handling**
|
||||
|
||||
File: `frontend/src/services/websocket.ts` or similar
|
||||
|
||||
```typescript
|
||||
// Add logging
|
||||
socket.onmessage = (event) => {
|
||||
const data = JSON.parse(event.data);
|
||||
console.log('[WS] Received:', data);
|
||||
|
||||
if (data.type === 'progress') {
|
||||
console.log('[WS] Progress update:', data.progress, data.status);
|
||||
// Update store
|
||||
setCoverageProgress(data.progress, data.status);
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**Step 3: Frontend - Check React state update**
|
||||
|
||||
File: `frontend/src/store/coverage.ts` or state management
|
||||
|
||||
```typescript
|
||||
// Ensure state updates trigger re-render
|
||||
setCoverageProgress: (progress: number, status: string) => {
|
||||
console.log('[Store] Setting progress:', progress, status);
|
||||
set({
|
||||
progress: progress, // Must be new value, not mutation
|
||||
progressStatus: status
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**Step 4: Frontend - Check component subscription**
|
||||
|
||||
File: `frontend/src/App.tsx` or progress display component
|
||||
|
||||
```typescript
|
||||
// Ensure component subscribes to store changes
|
||||
const progress = useCoverageStore((state) => state.progress);
|
||||
const progressStatus = useCoverageStore((state) => state.progressStatus);
|
||||
|
||||
useEffect(() => {
|
||||
console.log('[UI] Progress changed:', progress, progressStatus);
|
||||
}, [progress, progressStatus]);
|
||||
```
|
||||
|
||||
### 2.2 App Close Button Broken (Electron)
|
||||
|
||||
**Problem:** Clicking X kills backend but Electron window stays open
|
||||
|
||||
**File:** `desktop/main.js`
|
||||
|
||||
```javascript
|
||||
const { app, BrowserWindow } = require('electron');
|
||||
const { spawn } = require('child_process');
|
||||
|
||||
let mainWindow;
|
||||
let backendProcess;
|
||||
|
||||
function createWindow() {
|
||||
mainWindow = new BrowserWindow({
|
||||
// ... existing config
|
||||
});
|
||||
|
||||
// Handle window close
|
||||
mainWindow.on('close', async (event) => {
|
||||
event.preventDefault(); // Prevent immediate close
|
||||
|
||||
console.log('[Electron] Window closing, cleaning up...');
|
||||
|
||||
// Kill backend process
|
||||
if (backendProcess) {
|
||||
console.log('[Electron] Killing backend process...');
|
||||
backendProcess.kill('SIGTERM');
|
||||
|
||||
// Wait for graceful shutdown
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
|
||||
// Force kill if still running
|
||||
if (!backendProcess.killed) {
|
||||
backendProcess.kill('SIGKILL');
|
||||
}
|
||||
}
|
||||
|
||||
// Now actually close
|
||||
mainWindow.destroy();
|
||||
});
|
||||
|
||||
mainWindow.on('closed', () => {
|
||||
mainWindow = null;
|
||||
});
|
||||
}
|
||||
|
||||
// Ensure app quits when all windows closed
|
||||
app.on('window-all-closed', () => {
|
||||
console.log('[Electron] All windows closed, quitting app');
|
||||
app.quit();
|
||||
});
|
||||
|
||||
// Cleanup on app quit
|
||||
app.on('before-quit', () => {
|
||||
console.log('[Electron] App quitting, final cleanup');
|
||||
if (backendProcess && !backendProcess.killed) {
|
||||
backendProcess.kill('SIGKILL');
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### 2.3 Memory Leak (1328 MB not released)
|
||||
|
||||
**Problem:** Memory not freed after calculation
|
||||
|
||||
**File:** `backend/app/services/parallel_coverage_service.py` or `backend/app/parallel/manager.py`
|
||||
|
||||
```python
|
||||
import gc
|
||||
from multiprocessing import shared_memory
|
||||
|
||||
class SharedMemoryManager:
|
||||
def __init__(self):
|
||||
self._shared_blocks = []
|
||||
|
||||
def create_shared_block(self, name, size):
|
||||
shm = shared_memory.SharedMemory(name=name, create=True, size=size)
|
||||
self._shared_blocks.append(shm)
|
||||
return shm
|
||||
|
||||
def cleanup(self):
|
||||
"""Explicitly cleanup all shared memory blocks"""
|
||||
for shm in self._shared_blocks:
|
||||
try:
|
||||
shm.close()
|
||||
shm.unlink() # Important! Actually frees the memory
|
||||
except Exception as e:
|
||||
logger.warning(f"Error cleaning up shared memory: {e}")
|
||||
self._shared_blocks.clear()
|
||||
|
||||
# Force garbage collection
|
||||
gc.collect()
|
||||
|
||||
|
||||
# In coverage calculation:
|
||||
async def calculate_coverage(...):
|
||||
shm_manager = SharedMemoryManager()
|
||||
try:
|
||||
# ... calculation code
|
||||
pass
|
||||
finally:
|
||||
# ALWAYS cleanup, even on error/timeout
|
||||
shm_manager.cleanup()
|
||||
logger.info("[MEMORY] Shared memory cleaned up")
|
||||
```
|
||||
|
||||
### 2.4 Region Data / Map Cache Issues
|
||||
|
||||
**Problem:** Western Ukraine region hangs or produces "No coverage points found"
|
||||
|
||||
**Diagnosis steps:**
|
||||
|
||||
**Step 1: Check ProgramData folder structure**
|
||||
|
||||
```python
|
||||
# Add diagnostic endpoint or startup check
|
||||
import os
|
||||
|
||||
def diagnose_data_folders():
|
||||
"""Check data folder structure and validity"""
|
||||
|
||||
# Common locations
|
||||
locations = [
|
||||
os.path.expandvars(r'%PROGRAMDATA%\RFCP'),
|
||||
os.path.expandvars(r'%APPDATA%\RFCP'),
|
||||
os.path.expandvars(r'%LOCALAPPDATA%\RFCP'),
|
||||
'./data',
|
||||
'../data'
|
||||
]
|
||||
|
||||
report = {}
|
||||
for loc in locations:
|
||||
if os.path.exists(loc):
|
||||
report[loc] = {
|
||||
'exists': True,
|
||||
'files': os.listdir(loc),
|
||||
'size_mb': sum(
|
||||
os.path.getsize(os.path.join(loc, f))
|
||||
for f in os.listdir(loc)
|
||||
if os.path.isfile(os.path.join(loc, f))
|
||||
) / 1024 / 1024
|
||||
}
|
||||
|
||||
# Check terrain tiles
|
||||
terrain_dir = os.path.join(loc, 'terrain')
|
||||
if os.path.exists(terrain_dir):
|
||||
tiles = [f for f in os.listdir(terrain_dir) if f.endswith('.hgt')]
|
||||
report[loc]['terrain_tiles'] = len(tiles)
|
||||
|
||||
# Check OSM cache
|
||||
osm_dir = os.path.join(loc, 'osm_cache')
|
||||
if os.path.exists(osm_dir):
|
||||
cache_files = os.listdir(osm_dir)
|
||||
report[loc]['osm_cache_files'] = len(cache_files)
|
||||
|
||||
return report
|
||||
```
|
||||
|
||||
**Step 2: Validate terrain tiles**
|
||||
|
||||
```python
|
||||
def validate_terrain_tile(filepath: str) -> dict:
|
||||
"""Check if terrain tile is valid"""
|
||||
import struct
|
||||
|
||||
result = {
|
||||
'path': filepath,
|
||||
'exists': os.path.exists(filepath),
|
||||
'valid': False,
|
||||
'error': None
|
||||
}
|
||||
|
||||
if not result['exists']:
|
||||
result['error'] = 'File not found'
|
||||
return result
|
||||
|
||||
try:
|
||||
size = os.path.getsize(filepath)
|
||||
|
||||
# SRTM1 (1 arc-second): 3601x3601x2 = 25,934,402 bytes
|
||||
# SRTM3 (3 arc-second): 1201x1201x2 = 2,884,802 bytes
|
||||
|
||||
if size == 25934402:
|
||||
result['type'] = 'SRTM1'
|
||||
result['valid'] = True
|
||||
elif size == 2884802:
|
||||
result['type'] = 'SRTM3'
|
||||
result['valid'] = True
|
||||
else:
|
||||
result['error'] = f'Unexpected size: {size} bytes'
|
||||
|
||||
except Exception as e:
|
||||
result['error'] = str(e)
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
**Step 3: Fix OSM cache for regions**
|
||||
|
||||
```python
|
||||
def get_osm_cache_key(bbox: tuple, data_type: str) -> str:
|
||||
"""Generate consistent cache key for OSM data"""
|
||||
# Round to avoid floating point issues
|
||||
lat_min = round(bbox[0], 4)
|
||||
lon_min = round(bbox[1], 4)
|
||||
lat_max = round(bbox[2], 4)
|
||||
lon_max = round(bbox[3], 4)
|
||||
|
||||
return f"{data_type}_{lat_min}_{lon_min}_{lat_max}_{lon_max}.json"
|
||||
|
||||
|
||||
def validate_osm_cache(cache_path: str) -> bool:
|
||||
"""Check if cached OSM data is valid"""
|
||||
try:
|
||||
with open(cache_path, 'r') as f:
|
||||
data = json.load(f)
|
||||
|
||||
# Check structure
|
||||
if not isinstance(data, dict):
|
||||
return False
|
||||
if 'elements' not in data and not isinstance(data, list):
|
||||
return False
|
||||
|
||||
return True
|
||||
except:
|
||||
return False
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 3: UI Fixes
|
||||
|
||||
### 3.1 Calculate Button Position
|
||||
|
||||
**Problem:** Button overlaps with scrollbar
|
||||
|
||||
**File:** `frontend/src/components/CoverageSettings.tsx` or similar
|
||||
|
||||
```tsx
|
||||
// Move Calculate button outside scrollable area
|
||||
// Or add right margin
|
||||
|
||||
<div className="coverage-settings-panel">
|
||||
<div className="scrollable-content">
|
||||
{/* All settings */}
|
||||
</div>
|
||||
|
||||
<div className="fixed-footer" style={{
|
||||
padding: '16px',
|
||||
borderTop: '1px solid var(--border-color)',
|
||||
marginRight: '16px' // Space for scrollbar
|
||||
}}>
|
||||
<button
|
||||
className="calculate-button"
|
||||
onClick={handleCalculate}
|
||||
style={{ width: '100%' }}
|
||||
>
|
||||
Calculate Coverage
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
### 3.2 Site Drag - Move Sectors Together
|
||||
|
||||
**Problem:** Dragging site doesn't move its sectors
|
||||
|
||||
**File:** `frontend/src/components/map/SiteMarker.tsx` or site handling code
|
||||
|
||||
```typescript
|
||||
const handleSiteDrag = (siteId: string, newLat: number, newLon: number) => {
|
||||
const site = getSite(siteId);
|
||||
if (!site) return;
|
||||
|
||||
// Calculate delta
|
||||
const deltaLat = newLat - site.lat;
|
||||
const deltaLon = newLon - site.lon;
|
||||
|
||||
// Update site position
|
||||
updateSite(siteId, { lat: newLat, lon: newLon });
|
||||
|
||||
// Update all sectors of this site
|
||||
const sectors = getSectorsForSite(siteId);
|
||||
sectors.forEach(sector => {
|
||||
updateSector(sector.id, {
|
||||
lat: sector.lat + deltaLat,
|
||||
lon: sector.lon + deltaLon
|
||||
});
|
||||
});
|
||||
};
|
||||
```
|
||||
|
||||
### 3.3 Site Delete - Remove Sectors Together
|
||||
|
||||
**Problem:** Deleting site doesn't remove its sectors
|
||||
|
||||
**File:** `frontend/src/store/sites.ts` or site management
|
||||
|
||||
```typescript
|
||||
const deleteSite = (siteId: string) => {
|
||||
// First, delete all sectors belonging to this site
|
||||
const sectors = get().sectors.filter(s => s.siteId === siteId);
|
||||
sectors.forEach(sector => {
|
||||
deleteSector(sector.id);
|
||||
});
|
||||
|
||||
// Then delete the site
|
||||
set(state => ({
|
||||
sites: state.sites.filter(s => s.id !== siteId)
|
||||
}));
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 4: Testing Checklist
|
||||
|
||||
### Performance Tests
|
||||
- [ ] 5km Standard: < 10 seconds
|
||||
- [ ] 5km Detailed: < 60 seconds
|
||||
- [ ] 10km Standard: < 30 seconds
|
||||
- [ ] 10km Detailed: < 120 seconds (was timeout)
|
||||
- [ ] Radial preview (any radius): < 5 seconds
|
||||
|
||||
### Bug Fix Tests
|
||||
- [ ] Progress bar updates from 5% → 100%
|
||||
- [ ] App closes completely when clicking X
|
||||
- [ ] Memory returns to baseline after calculation
|
||||
- [ ] Western Ukraine region calculates successfully
|
||||
- [ ] All terrain tiles validate correctly
|
||||
- [ ] OSM cache files are valid JSON
|
||||
|
||||
### UI Tests
|
||||
- [ ] Calculate button not overlapping scrollbar
|
||||
- [ ] Dragging site moves all its sectors
|
||||
- [ ] Deleting site removes all its sectors
|
||||
- [ ] No console errors in browser DevTools
|
||||
|
||||
### Data Validation
|
||||
- [ ] Run `diagnose_data_folders()` - check output
|
||||
- [ ] Validate all terrain tiles in cache
|
||||
- [ ] Validate all OSM cache files
|
||||
- [ ] Check for corrupted files and remove them
|
||||
|
||||
---
|
||||
|
||||
## Build & Deploy
|
||||
|
||||
```powershell
|
||||
# Backend
|
||||
cd D:\root\rfcp\backend
|
||||
pip install -e .
|
||||
|
||||
# Frontend
|
||||
cd D:\root\rfcp\frontend
|
||||
npm run build
|
||||
|
||||
# Electron
|
||||
cd D:\root\rfcp\desktop
|
||||
npm run build
|
||||
|
||||
# Full test
|
||||
cd D:\root\rfcp\installer
|
||||
.\test-detailed-quick.bat
|
||||
|
||||
# If works, rebuild installer
|
||||
pyinstaller rfcp-server.spec --clean
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Commit Message
|
||||
|
||||
```
|
||||
feat: Iteration 3.2.0 - Performance & Bug Fixes
|
||||
|
||||
Performance:
|
||||
- Add adaptive resolution (distance-based grid density)
|
||||
- Add radial preview mode (360 spokes, instant feedback)
|
||||
- Expected: 10km Detailed ~60s (was timeout)
|
||||
|
||||
Bug Fixes:
|
||||
- Fix progress bar stuck at 5% (WebSocket + React state)
|
||||
- Fix app close button (Electron lifecycle)
|
||||
- Fix memory leak (SharedMemory cleanup)
|
||||
- Fix region data issues (cache validation)
|
||||
|
||||
UI Improvements:
|
||||
- Move Calculate button (scrollbar overlap)
|
||||
- Site drag moves all sectors
|
||||
- Site delete removes all sectors
|
||||
|
||||
Data Validation:
|
||||
- Add terrain tile validation
|
||||
- Add OSM cache validation
|
||||
- Add diagnostic reporting
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. **10km Detailed completes** without timeout (~60-120s acceptable)
|
||||
2. **Progress bar works** - shows actual progress 5% → 100%
|
||||
3. **App closes cleanly** - no orphan processes
|
||||
4. **Memory released** - returns to baseline after calculation
|
||||
5. **All regions work** - Western Ukraine calculates successfully
|
||||
6. **Site management** - drag/delete affects sectors correctly
|
||||
|
||||
---
|
||||
|
||||
## Priority Order for Implementation
|
||||
|
||||
1. **Adaptive Resolution** - biggest performance impact
|
||||
2. **Progress bar fix** - critical UX issue
|
||||
3. **App close fix** - annoying bug
|
||||
4. **Site drag/delete sectors** - quick win
|
||||
5. **Calculate button position** - quick win
|
||||
6. **Memory leak** - important but complex
|
||||
7. **Region data validation** - diagnostic + fix
|
||||
8. **Radial preview** - nice to have
|
||||
|
||||
---
|
||||
|
||||
## Notes for Claude Code
|
||||
|
||||
- This is a large iteration - take it step by step
|
||||
- Test after each major change
|
||||
- Backend and frontend changes may need coordination
|
||||
- Electron changes require rebuild of desktop app
|
||||
- Data validation can be added as debug endpoint first
|
||||
- If stuck on one issue, move to next and come back
|
||||
|
||||
---
|
||||
|
||||
*"Big iteration, big impact. Let's make RFCP production-ready!"* 🚀
|
||||
Reference in New Issue
Block a user