@mytec: iter3.4.0 start
This commit is contained in:
332
docs/devlog/installer/RFCP-Iteration-3.1.0-LOD-Optimization.md
Normal file
332
docs/devlog/installer/RFCP-Iteration-3.1.0-LOD-Optimization.md
Normal file
@@ -0,0 +1,332 @@
|
||||
# RFCP - Iteration 3.1.0: LOD (Level of Detail) Optimization
|
||||
|
||||
## Overview
|
||||
|
||||
Detailed preset times out at 300s because dominant_path_service calculates expensive geometry for ALL 868 points. This iteration adds distance-based LOD to skip or simplify calculations for distant points, reducing total time to <60s.
|
||||
|
||||
**Current:** 302.8ms/point × 868 points = 262s (TIMEOUT)
|
||||
**Target:** ~33s total (8x speedup)
|
||||
|
||||
---
|
||||
|
||||
## Issues Identified
|
||||
|
||||
**Problem 1: All points get full dominant_path calculation**
|
||||
- Root Cause: No distance-based filtering
|
||||
- Impact: Points >3km from TX still check 25+ buildings × 150+ walls
|
||||
- At these distances, building-level detail provides minimal accuracy benefit
|
||||
|
||||
**Problem 2: dominant_path is O(points × buildings × walls)**
|
||||
- Root Cause: Algorithmic complexity
|
||||
- Impact: 868 × 25 × 150 = 3.2M intersection checks
|
||||
- Each check is ~0.1ms = 320 seconds theoretical minimum
|
||||
|
||||
---
|
||||
|
||||
## Solution: Distance-Based LOD
|
||||
|
||||
### LOD Levels
|
||||
|
||||
```
|
||||
Distance > 3km → LOD_NONE → Skip dominant_path entirely (0 buildings)
|
||||
Distance 1.5-3km → LOD_SIMPLIFIED → Check only 5 nearest buildings
|
||||
Distance < 1.5km → LOD_FULL → Full calculation (current behavior)
|
||||
```
|
||||
|
||||
### Expected Performance
|
||||
|
||||
| LOD Level | Distance | Points (~) | Time/point | Total |
|
||||
|-------------|-----------|------------|------------|---------|
|
||||
| NONE | >3km | 600 (70%) | ~2ms | 1.2s |
|
||||
| SIMPLIFIED | 1.5-3km | 180 (20%) | ~30ms | 5.4s |
|
||||
| FULL | <1.5km | 88 (10%) | ~300ms | 26.4s |
|
||||
| **TOTAL** | | 868 | | **~33s**|
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
### Step 1: Add LOD constants to dominant_path_service.py
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
**Add at top of file (after imports):**
|
||||
|
||||
```python
|
||||
from enum import Enum
|
||||
|
||||
class LODLevel(Enum):
|
||||
"""Level of Detail for dominant path calculations"""
|
||||
NONE = "none" # Skip dominant path entirely
|
||||
SIMPLIFIED = "simplified" # Check only nearest buildings
|
||||
FULL = "full" # Full calculation
|
||||
|
||||
# LOD distance thresholds (meters)
|
||||
LOD_THRESHOLD_NONE = 3000 # >3km: skip dominant path
|
||||
LOD_THRESHOLD_SIMPLIFIED = 1500 # 1.5-3km: simplified mode
|
||||
|
||||
# Simplified mode limits
|
||||
SIMPLIFIED_MAX_BUILDINGS = 5
|
||||
SIMPLIFIED_MAX_WALLS = 50
|
||||
```
|
||||
|
||||
### Step 2: Add get_lod_level() function
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
**Add function:**
|
||||
|
||||
```python
|
||||
def get_lod_level(distance_m: float) -> LODLevel:
|
||||
"""
|
||||
Determine LOD level based on TX-RX distance.
|
||||
|
||||
At long distances, building-level multipath contributes
|
||||
minimally to path loss - macro propagation models suffice.
|
||||
"""
|
||||
if distance_m > LOD_THRESHOLD_NONE:
|
||||
return LODLevel.NONE
|
||||
elif distance_m > LOD_THRESHOLD_SIMPLIFIED:
|
||||
return LODLevel.SIMPLIFIED
|
||||
else:
|
||||
return LODLevel.FULL
|
||||
```
|
||||
|
||||
### Step 3: Create find_dominant_path_with_lod() wrapper
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
**Add function (this wraps existing logic):**
|
||||
|
||||
```python
|
||||
def find_dominant_path_with_lod(
|
||||
tx_lat: float, tx_lon: float, tx_height: float,
|
||||
rx_lat: float, rx_lon: float, rx_height: float,
|
||||
frequency_mhz: float,
|
||||
buildings: list,
|
||||
distance_m: float = None
|
||||
) -> dict:
|
||||
"""
|
||||
Find dominant path with LOD optimization.
|
||||
|
||||
Args:
|
||||
tx_lat, tx_lon, tx_height: Transmitter position
|
||||
rx_lat, rx_lon, rx_height: Receiver position
|
||||
frequency_mhz: Operating frequency
|
||||
buildings: List of building dicts from OSM
|
||||
distance_m: Pre-calculated TX-RX distance (optional, saves recalc)
|
||||
|
||||
Returns:
|
||||
dict with:
|
||||
- path_loss_db: Additional path loss from buildings (0 if skipped)
|
||||
- lod_level: Which LOD was applied
|
||||
- buildings_checked: How many buildings were evaluated
|
||||
- walls_checked: How many walls were evaluated
|
||||
- skipped: True if dominant_path was skipped entirely
|
||||
"""
|
||||
from app.services.terrain_service import TerrainService
|
||||
|
||||
# Calculate distance if not provided
|
||||
if distance_m is None:
|
||||
distance_m = TerrainService.haversine_distance(tx_lat, tx_lon, rx_lat, rx_lon)
|
||||
|
||||
lod = get_lod_level(distance_m)
|
||||
|
||||
# LOD_NONE: Skip dominant path entirely
|
||||
if lod == LODLevel.NONE:
|
||||
return {
|
||||
"path_loss_db": 0.0,
|
||||
"lod_level": "none",
|
||||
"buildings_checked": 0,
|
||||
"walls_checked": 0,
|
||||
"skipped": True
|
||||
}
|
||||
|
||||
# Filter buildings for LOD_SIMPLIFIED
|
||||
buildings_to_check = buildings
|
||||
if lod == LODLevel.SIMPLIFIED and buildings:
|
||||
if len(buildings) > SIMPLIFIED_MAX_BUILDINGS:
|
||||
# Sort by distance to path midpoint and take nearest
|
||||
mid_lat = (tx_lat + rx_lat) / 2
|
||||
mid_lon = (tx_lon + rx_lon) / 2
|
||||
|
||||
buildings_with_dist = []
|
||||
for b in buildings:
|
||||
# Get building centroid from geometry
|
||||
geom = b.get('geometry', {})
|
||||
coords = geom.get('coordinates', [[]])[0] if isinstance(geom, dict) else b.get('geometry', [[]])
|
||||
|
||||
if coords and len(coords) > 0:
|
||||
# Handle both formats: [[lon,lat],...] or [{'lon':..,'lat':..},...]
|
||||
if isinstance(coords[0], (list, tuple)):
|
||||
blat = sum(c[1] for c in coords) / len(coords)
|
||||
blon = sum(c[0] for c in coords) / len(coords)
|
||||
else:
|
||||
blat = sum(c.get('lat', c.get('y', 0)) for c in coords) / len(coords)
|
||||
blon = sum(c.get('lon', c.get('x', 0)) for c in coords) / len(coords)
|
||||
|
||||
dist = TerrainService.haversine_distance(mid_lat, mid_lon, blat, blon)
|
||||
buildings_with_dist.append((dist, b))
|
||||
|
||||
buildings_with_dist.sort(key=lambda x: x[0])
|
||||
buildings_to_check = [b for _, b in buildings_with_dist[:SIMPLIFIED_MAX_BUILDINGS]]
|
||||
|
||||
# Call existing dominant path function
|
||||
# Look for existing function: find_dominant_path_vectorized, find_dominant_paths, etc.
|
||||
try:
|
||||
# Try vectorized version first
|
||||
result = find_dominant_path_vectorized(
|
||||
tx_lat, tx_lon,
|
||||
rx_lat, rx_lon,
|
||||
buildings_to_check,
|
||||
frequency_mhz
|
||||
)
|
||||
except (NameError, AttributeError):
|
||||
# Fall back to sync version if vectorized not available
|
||||
try:
|
||||
result = dominant_path_service.find_dominant_paths(
|
||||
tx_lat, tx_lon, tx_height,
|
||||
rx_lat, rx_lon, rx_height,
|
||||
frequency_mhz,
|
||||
buildings_to_check
|
||||
)
|
||||
except:
|
||||
# If no dominant path function works, return zero loss
|
||||
result = {"path_loss_db": 0.0}
|
||||
|
||||
# Ensure result is dict
|
||||
if not isinstance(result, dict):
|
||||
result = {"path_loss_db": float(result) if result else 0.0}
|
||||
|
||||
# Add LOD metadata
|
||||
result["lod_level"] = lod.value
|
||||
result["buildings_checked"] = len(buildings_to_check)
|
||||
result["skipped"] = False
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
### Step 4: Add logging for LOD decisions
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
**Add after LOD decision (inside find_dominant_path_with_lod):**
|
||||
|
||||
```python
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Add this right after lod = get_lod_level(distance_m):
|
||||
if lod == LODLevel.NONE:
|
||||
logger.debug(f"[DOMINANT_PATH] LOD=none, dist={distance_m:.0f}m, skipped")
|
||||
elif lod == LODLevel.SIMPLIFIED:
|
||||
logger.debug(f"[DOMINANT_PATH] LOD=simplified, dist={distance_m:.0f}m, buildings={len(buildings_to_check)}")
|
||||
else:
|
||||
logger.debug(f"[DOMINANT_PATH] LOD=full, dist={distance_m:.0f}m, buildings={len(buildings_to_check)}")
|
||||
```
|
||||
|
||||
### Step 5: Update coverage calculation to use LOD wrapper
|
||||
|
||||
**File:** `backend/app/services/coverage_service.py` OR `backend/app/services/parallel_coverage_service.py`
|
||||
|
||||
**Find where dominant_path is called and replace with LOD version:**
|
||||
|
||||
```python
|
||||
# BEFORE (find lines like this):
|
||||
dominant_result = find_dominant_path_vectorized(tx, rx, buildings, ...)
|
||||
# or
|
||||
dominant_result = dominant_path_service.find_dominant_paths(...)
|
||||
|
||||
# AFTER (replace with):
|
||||
from app.services.dominant_path_service import find_dominant_path_with_lod
|
||||
|
||||
dominant_result = find_dominant_path_with_lod(
|
||||
tx_lat, tx_lon, tx_height,
|
||||
rx_lat, rx_lon, rx_height,
|
||||
frequency_mhz,
|
||||
buildings,
|
||||
distance_m=point_distance # Pass pre-calculated distance if available
|
||||
)
|
||||
|
||||
# Use the result
|
||||
if not dominant_result.get("skipped", False):
|
||||
total_loss += dominant_result.get("path_loss_db", 0.0)
|
||||
```
|
||||
|
||||
### Step 6: Update worker function (if using parallel processing)
|
||||
|
||||
**File:** `backend/app/parallel/worker.py` OR wherever worker calculates points
|
||||
|
||||
**Same pattern - use find_dominant_path_with_lod instead of direct calls.**
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] LODLevel enum imports correctly
|
||||
- [ ] get_lod_level(4000) returns LODLevel.NONE
|
||||
- [ ] get_lod_level(2000) returns LODLevel.SIMPLIFIED
|
||||
- [ ] get_lod_level(1000) returns LODLevel.FULL
|
||||
- [ ] Detailed preset completes without timeout
|
||||
- [ ] Detailed preset time < 90 seconds (target: ~33s)
|
||||
- [ ] Standard preset still works (regression check)
|
||||
- [ ] Logs show LOD decisions: "LOD=none", "LOD=simplified", "LOD=full"
|
||||
- [ ] Coverage map looks reasonable (no obvious artifacts at LOD boundaries)
|
||||
|
||||
---
|
||||
|
||||
## Build & Deploy
|
||||
|
||||
```powershell
|
||||
# Backend
|
||||
cd D:\root\rfcp\backend
|
||||
pip install -e .
|
||||
|
||||
# Test
|
||||
cd D:\root\rfcp\installer
|
||||
.\test-detailed-quick.bat
|
||||
|
||||
# If works, rebuild executable
|
||||
cd D:\root\rfcp\installer
|
||||
pyinstaller rfcp-server.spec --clean
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Commit Message
|
||||
|
||||
```
|
||||
feat(backend): add LOD optimization for dominant_path (v3.1.0)
|
||||
|
||||
- Add LODLevel enum (NONE, SIMPLIFIED, FULL)
|
||||
- Add distance thresholds: >3km skip, 1.5-3km simplified, <1.5km full
|
||||
- Create find_dominant_path_with_lod() wrapper
|
||||
- Update coverage calculation to use LOD
|
||||
- Expected: 8x speedup for Detailed preset (262s -> ~33s)
|
||||
|
||||
Phase 3.1.0: Performance Optimization
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. **Performance:** Detailed preset completes in <90 seconds (target ~33s)
|
||||
2. **No regression:** Standard preset still works, same speed
|
||||
3. **Logging:** Can see LOD level in server output
|
||||
4. **Quality:** Coverage map visually acceptable (no obvious LOD boundary artifacts)
|
||||
|
||||
---
|
||||
|
||||
## Notes for Claude Code
|
||||
|
||||
- The existing codebase has multiple dominant_path functions - find the one actually being used
|
||||
- Check both `coverage_service.py` and `parallel_coverage_service.py`
|
||||
- Worker processes may have their own copy of the function - update those too
|
||||
- If `find_dominant_path_vectorized` doesn't exist as standalone function, look for it in a class
|
||||
- haversine_distance might be in TerrainService or as standalone function - check imports
|
||||
- Building geometry format varies - handle both `[[lon,lat],...]` and `[{lon:...,lat:...},...]`
|
||||
|
||||
---
|
||||
|
||||
*"Not all points are created equal - distant ones deserve less attention"*
|
||||
@@ -0,0 +1,633 @@
|
||||
# RFCP - Iteration 3.2.0: Comprehensive Performance & Bug Fixes
|
||||
|
||||
## Overview
|
||||
|
||||
Major iteration combining performance optimizations, UI fixes, and bug resolutions. This addresses the Detailed preset timeout, stuck progress bar, app close issues, region data problems, and UX improvements.
|
||||
|
||||
**Scope:** Backend optimizations + Frontend fixes + Electron fixes + Data validation
|
||||
|
||||
---
|
||||
|
||||
## Part 1: Performance Optimizations
|
||||
|
||||
### 1.1 Adaptive Resolution (CRITICAL)
|
||||
|
||||
**Problem:** 10km radius with 200m resolution = ~7850 points → timeout
|
||||
|
||||
**Solution:** Distance-based adaptive resolution
|
||||
|
||||
**File:** `backend/app/services/coverage_service.py` (or grid generation code)
|
||||
|
||||
```python
|
||||
def get_adaptive_resolution(base_resolution: float, distance_from_tx: float) -> float:
|
||||
"""
|
||||
Adaptive resolution based on distance from transmitter.
|
||||
|
||||
Close to TX: use user's chosen resolution (details matter)
|
||||
Far from TX: use coarser resolution (macro view sufficient)
|
||||
"""
|
||||
if distance_from_tx < 2000: # < 2km
|
||||
return base_resolution # User's choice (e.g., 200m)
|
||||
elif distance_from_tx < 5000: # 2-5km
|
||||
return max(base_resolution, 300) # At least 300m
|
||||
else: # > 5km
|
||||
return max(base_resolution, 500) # At least 500m
|
||||
|
||||
|
||||
def generate_adaptive_grid(center_lat, center_lon, radius_m, base_resolution):
|
||||
"""
|
||||
Generate grid with adaptive resolution zones.
|
||||
|
||||
Instead of uniform grid, create concentric zones with different resolutions.
|
||||
"""
|
||||
points = []
|
||||
|
||||
# Zone 1: Inner (< 2km) - full resolution
|
||||
inner_points = generate_grid_ring(center_lat, center_lon, 0, 2000, base_resolution)
|
||||
points.extend(inner_points)
|
||||
|
||||
# Zone 2: Middle (2-5km) - medium resolution
|
||||
if radius_m > 2000:
|
||||
medium_res = max(base_resolution, 300)
|
||||
middle_points = generate_grid_ring(center_lat, center_lon, 2000, min(5000, radius_m), medium_res)
|
||||
points.extend(middle_points)
|
||||
|
||||
# Zone 3: Outer (5km+) - coarse resolution
|
||||
if radius_m > 5000:
|
||||
coarse_res = max(base_resolution, 500)
|
||||
outer_points = generate_grid_ring(center_lat, center_lon, 5000, radius_m, coarse_res)
|
||||
points.extend(outer_points)
|
||||
|
||||
return points
|
||||
```
|
||||
|
||||
**Expected result:**
|
||||
- 10km with 200m base: ~7850 → ~2500 points (3x reduction)
|
||||
- Combined with LOD: 10km detailed should complete in ~60s
|
||||
|
||||
### 1.2 Radial Preview Mode (NEW FEATURE)
|
||||
|
||||
**Purpose:** Instant preview using 360 radial spokes instead of full grid
|
||||
|
||||
**File:** `backend/app/services/coverage_service.py`
|
||||
|
||||
```python
|
||||
def calculate_radial_preview(
|
||||
tx_lat: float, tx_lon: float, tx_height: float,
|
||||
radius_m: float, frequency_mhz: float,
|
||||
num_spokes: int = 360, # 1 degree resolution
|
||||
points_per_spoke: int = 50
|
||||
) -> List[dict]:
|
||||
"""
|
||||
Fast radial preview calculation.
|
||||
|
||||
Instead of grid, calculate along 360 radial lines from TX.
|
||||
Much faster because:
|
||||
- Terrain profile can be cached per spoke
|
||||
- No building calculations (terrain only)
|
||||
- Linear interpolation between points
|
||||
"""
|
||||
results = []
|
||||
|
||||
for angle_deg in range(num_spokes):
|
||||
angle_rad = math.radians(angle_deg)
|
||||
|
||||
# Calculate points along this spoke
|
||||
for i in range(points_per_spoke):
|
||||
distance = (i + 1) * (radius_m / points_per_spoke)
|
||||
|
||||
# Calculate point position
|
||||
rx_lat, rx_lon = move_point(tx_lat, tx_lon, distance, angle_deg)
|
||||
|
||||
# Simple terrain-only calculation (no buildings)
|
||||
path_loss = calculate_terrain_path_loss(
|
||||
tx_lat, tx_lon, tx_height,
|
||||
rx_lat, rx_lon, 1.5, # Standard UE height
|
||||
frequency_mhz
|
||||
)
|
||||
|
||||
results.append({
|
||||
'lat': rx_lat,
|
||||
'lon': rx_lon,
|
||||
'rsrp': tx_power_dbm - path_loss,
|
||||
'distance': distance,
|
||||
'angle': angle_deg
|
||||
})
|
||||
|
||||
return results
|
||||
```
|
||||
|
||||
**Add to API:** New endpoint or parameter `?mode=preview`
|
||||
|
||||
---
|
||||
|
||||
## Part 2: Bug Fixes
|
||||
|
||||
### 2.1 Progress Bar Stuck at "Initializing 5%" (CRITICAL)
|
||||
|
||||
**Problem:** Progress never updates past 5%
|
||||
|
||||
**Root Cause:** WebSocket messages not reaching frontend OR React state not updating
|
||||
|
||||
**Debug & Fix Steps:**
|
||||
|
||||
**Step 1: Backend - Verify messages are sent**
|
||||
|
||||
File: `backend/app/api/websocket.py` or `backend/app/services/coverage_service.py`
|
||||
|
||||
```python
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
async def send_progress(websocket, progress: int, status: str):
|
||||
"""Send progress with logging"""
|
||||
message = {"type": "progress", "progress": progress, "status": status}
|
||||
logger.info(f"[WS] Sending progress: {progress}% - {status}")
|
||||
await websocket.send_json(message)
|
||||
await asyncio.sleep(0) # Yield to event loop
|
||||
```
|
||||
|
||||
**Step 2: Frontend - Check WebSocket handling**
|
||||
|
||||
File: `frontend/src/services/websocket.ts` or similar
|
||||
|
||||
```typescript
|
||||
// Add logging
|
||||
socket.onmessage = (event) => {
|
||||
const data = JSON.parse(event.data);
|
||||
console.log('[WS] Received:', data);
|
||||
|
||||
if (data.type === 'progress') {
|
||||
console.log('[WS] Progress update:', data.progress, data.status);
|
||||
// Update store
|
||||
setCoverageProgress(data.progress, data.status);
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**Step 3: Frontend - Check React state update**
|
||||
|
||||
File: `frontend/src/store/coverage.ts` or state management
|
||||
|
||||
```typescript
|
||||
// Ensure state updates trigger re-render
|
||||
setCoverageProgress: (progress: number, status: string) => {
|
||||
console.log('[Store] Setting progress:', progress, status);
|
||||
set({
|
||||
progress: progress, // Must be new value, not mutation
|
||||
progressStatus: status
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**Step 4: Frontend - Check component subscription**
|
||||
|
||||
File: `frontend/src/App.tsx` or progress display component
|
||||
|
||||
```typescript
|
||||
// Ensure component subscribes to store changes
|
||||
const progress = useCoverageStore((state) => state.progress);
|
||||
const progressStatus = useCoverageStore((state) => state.progressStatus);
|
||||
|
||||
useEffect(() => {
|
||||
console.log('[UI] Progress changed:', progress, progressStatus);
|
||||
}, [progress, progressStatus]);
|
||||
```
|
||||
|
||||
### 2.2 App Close Button Broken (Electron)
|
||||
|
||||
**Problem:** Clicking X kills backend but Electron window stays open
|
||||
|
||||
**File:** `desktop/main.js`
|
||||
|
||||
```javascript
|
||||
const { app, BrowserWindow } = require('electron');
|
||||
const { spawn } = require('child_process');
|
||||
|
||||
let mainWindow;
|
||||
let backendProcess;
|
||||
|
||||
function createWindow() {
|
||||
mainWindow = new BrowserWindow({
|
||||
// ... existing config
|
||||
});
|
||||
|
||||
// Handle window close
|
||||
mainWindow.on('close', async (event) => {
|
||||
event.preventDefault(); // Prevent immediate close
|
||||
|
||||
console.log('[Electron] Window closing, cleaning up...');
|
||||
|
||||
// Kill backend process
|
||||
if (backendProcess) {
|
||||
console.log('[Electron] Killing backend process...');
|
||||
backendProcess.kill('SIGTERM');
|
||||
|
||||
// Wait for graceful shutdown
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
|
||||
// Force kill if still running
|
||||
if (!backendProcess.killed) {
|
||||
backendProcess.kill('SIGKILL');
|
||||
}
|
||||
}
|
||||
|
||||
// Now actually close
|
||||
mainWindow.destroy();
|
||||
});
|
||||
|
||||
mainWindow.on('closed', () => {
|
||||
mainWindow = null;
|
||||
});
|
||||
}
|
||||
|
||||
// Ensure app quits when all windows closed
|
||||
app.on('window-all-closed', () => {
|
||||
console.log('[Electron] All windows closed, quitting app');
|
||||
app.quit();
|
||||
});
|
||||
|
||||
// Cleanup on app quit
|
||||
app.on('before-quit', () => {
|
||||
console.log('[Electron] App quitting, final cleanup');
|
||||
if (backendProcess && !backendProcess.killed) {
|
||||
backendProcess.kill('SIGKILL');
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### 2.3 Memory Leak (1328 MB not released)
|
||||
|
||||
**Problem:** Memory not freed after calculation
|
||||
|
||||
**File:** `backend/app/services/parallel_coverage_service.py` or `backend/app/parallel/manager.py`
|
||||
|
||||
```python
|
||||
import gc
|
||||
from multiprocessing import shared_memory
|
||||
|
||||
class SharedMemoryManager:
|
||||
def __init__(self):
|
||||
self._shared_blocks = []
|
||||
|
||||
def create_shared_block(self, name, size):
|
||||
shm = shared_memory.SharedMemory(name=name, create=True, size=size)
|
||||
self._shared_blocks.append(shm)
|
||||
return shm
|
||||
|
||||
def cleanup(self):
|
||||
"""Explicitly cleanup all shared memory blocks"""
|
||||
for shm in self._shared_blocks:
|
||||
try:
|
||||
shm.close()
|
||||
shm.unlink() # Important! Actually frees the memory
|
||||
except Exception as e:
|
||||
logger.warning(f"Error cleaning up shared memory: {e}")
|
||||
self._shared_blocks.clear()
|
||||
|
||||
# Force garbage collection
|
||||
gc.collect()
|
||||
|
||||
|
||||
# In coverage calculation:
|
||||
async def calculate_coverage(...):
|
||||
shm_manager = SharedMemoryManager()
|
||||
try:
|
||||
# ... calculation code
|
||||
pass
|
||||
finally:
|
||||
# ALWAYS cleanup, even on error/timeout
|
||||
shm_manager.cleanup()
|
||||
logger.info("[MEMORY] Shared memory cleaned up")
|
||||
```
|
||||
|
||||
### 2.4 Region Data / Map Cache Issues
|
||||
|
||||
**Problem:** Western Ukraine region hangs or produces "No coverage points found"
|
||||
|
||||
**Diagnosis steps:**
|
||||
|
||||
**Step 1: Check ProgramData folder structure**
|
||||
|
||||
```python
|
||||
# Add diagnostic endpoint or startup check
|
||||
import os
|
||||
|
||||
def diagnose_data_folders():
|
||||
"""Check data folder structure and validity"""
|
||||
|
||||
# Common locations
|
||||
locations = [
|
||||
os.path.expandvars(r'%PROGRAMDATA%\RFCP'),
|
||||
os.path.expandvars(r'%APPDATA%\RFCP'),
|
||||
os.path.expandvars(r'%LOCALAPPDATA%\RFCP'),
|
||||
'./data',
|
||||
'../data'
|
||||
]
|
||||
|
||||
report = {}
|
||||
for loc in locations:
|
||||
if os.path.exists(loc):
|
||||
report[loc] = {
|
||||
'exists': True,
|
||||
'files': os.listdir(loc),
|
||||
'size_mb': sum(
|
||||
os.path.getsize(os.path.join(loc, f))
|
||||
for f in os.listdir(loc)
|
||||
if os.path.isfile(os.path.join(loc, f))
|
||||
) / 1024 / 1024
|
||||
}
|
||||
|
||||
# Check terrain tiles
|
||||
terrain_dir = os.path.join(loc, 'terrain')
|
||||
if os.path.exists(terrain_dir):
|
||||
tiles = [f for f in os.listdir(terrain_dir) if f.endswith('.hgt')]
|
||||
report[loc]['terrain_tiles'] = len(tiles)
|
||||
|
||||
# Check OSM cache
|
||||
osm_dir = os.path.join(loc, 'osm_cache')
|
||||
if os.path.exists(osm_dir):
|
||||
cache_files = os.listdir(osm_dir)
|
||||
report[loc]['osm_cache_files'] = len(cache_files)
|
||||
|
||||
return report
|
||||
```
|
||||
|
||||
**Step 2: Validate terrain tiles**
|
||||
|
||||
```python
|
||||
def validate_terrain_tile(filepath: str) -> dict:
|
||||
"""Check if terrain tile is valid"""
|
||||
import struct
|
||||
|
||||
result = {
|
||||
'path': filepath,
|
||||
'exists': os.path.exists(filepath),
|
||||
'valid': False,
|
||||
'error': None
|
||||
}
|
||||
|
||||
if not result['exists']:
|
||||
result['error'] = 'File not found'
|
||||
return result
|
||||
|
||||
try:
|
||||
size = os.path.getsize(filepath)
|
||||
|
||||
# SRTM1 (1 arc-second): 3601x3601x2 = 25,934,402 bytes
|
||||
# SRTM3 (3 arc-second): 1201x1201x2 = 2,884,802 bytes
|
||||
|
||||
if size == 25934402:
|
||||
result['type'] = 'SRTM1'
|
||||
result['valid'] = True
|
||||
elif size == 2884802:
|
||||
result['type'] = 'SRTM3'
|
||||
result['valid'] = True
|
||||
else:
|
||||
result['error'] = f'Unexpected size: {size} bytes'
|
||||
|
||||
except Exception as e:
|
||||
result['error'] = str(e)
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
**Step 3: Fix OSM cache for regions**
|
||||
|
||||
```python
|
||||
def get_osm_cache_key(bbox: tuple, data_type: str) -> str:
|
||||
"""Generate consistent cache key for OSM data"""
|
||||
# Round to avoid floating point issues
|
||||
lat_min = round(bbox[0], 4)
|
||||
lon_min = round(bbox[1], 4)
|
||||
lat_max = round(bbox[2], 4)
|
||||
lon_max = round(bbox[3], 4)
|
||||
|
||||
return f"{data_type}_{lat_min}_{lon_min}_{lat_max}_{lon_max}.json"
|
||||
|
||||
|
||||
def validate_osm_cache(cache_path: str) -> bool:
|
||||
"""Check if cached OSM data is valid"""
|
||||
try:
|
||||
with open(cache_path, 'r') as f:
|
||||
data = json.load(f)
|
||||
|
||||
# Check structure
|
||||
if not isinstance(data, dict):
|
||||
return False
|
||||
if 'elements' not in data and not isinstance(data, list):
|
||||
return False
|
||||
|
||||
return True
|
||||
except:
|
||||
return False
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 3: UI Fixes
|
||||
|
||||
### 3.1 Calculate Button Position
|
||||
|
||||
**Problem:** Button overlaps with scrollbar
|
||||
|
||||
**File:** `frontend/src/components/CoverageSettings.tsx` or similar
|
||||
|
||||
```tsx
|
||||
// Move Calculate button outside scrollable area
|
||||
// Or add right margin
|
||||
|
||||
<div className="coverage-settings-panel">
|
||||
<div className="scrollable-content">
|
||||
{/* All settings */}
|
||||
</div>
|
||||
|
||||
<div className="fixed-footer" style={{
|
||||
padding: '16px',
|
||||
borderTop: '1px solid var(--border-color)',
|
||||
marginRight: '16px' // Space for scrollbar
|
||||
}}>
|
||||
<button
|
||||
className="calculate-button"
|
||||
onClick={handleCalculate}
|
||||
style={{ width: '100%' }}
|
||||
>
|
||||
Calculate Coverage
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
### 3.2 Site Drag - Move Sectors Together
|
||||
|
||||
**Problem:** Dragging site doesn't move its sectors
|
||||
|
||||
**File:** `frontend/src/components/map/SiteMarker.tsx` or site handling code
|
||||
|
||||
```typescript
|
||||
const handleSiteDrag = (siteId: string, newLat: number, newLon: number) => {
|
||||
const site = getSite(siteId);
|
||||
if (!site) return;
|
||||
|
||||
// Calculate delta
|
||||
const deltaLat = newLat - site.lat;
|
||||
const deltaLon = newLon - site.lon;
|
||||
|
||||
// Update site position
|
||||
updateSite(siteId, { lat: newLat, lon: newLon });
|
||||
|
||||
// Update all sectors of this site
|
||||
const sectors = getSectorsForSite(siteId);
|
||||
sectors.forEach(sector => {
|
||||
updateSector(sector.id, {
|
||||
lat: sector.lat + deltaLat,
|
||||
lon: sector.lon + deltaLon
|
||||
});
|
||||
});
|
||||
};
|
||||
```
|
||||
|
||||
### 3.3 Site Delete - Remove Sectors Together
|
||||
|
||||
**Problem:** Deleting site doesn't remove its sectors
|
||||
|
||||
**File:** `frontend/src/store/sites.ts` or site management
|
||||
|
||||
```typescript
|
||||
const deleteSite = (siteId: string) => {
|
||||
// First, delete all sectors belonging to this site
|
||||
const sectors = get().sectors.filter(s => s.siteId === siteId);
|
||||
sectors.forEach(sector => {
|
||||
deleteSector(sector.id);
|
||||
});
|
||||
|
||||
// Then delete the site
|
||||
set(state => ({
|
||||
sites: state.sites.filter(s => s.id !== siteId)
|
||||
}));
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 4: Testing Checklist
|
||||
|
||||
### Performance Tests
|
||||
- [ ] 5km Standard: < 10 seconds
|
||||
- [ ] 5km Detailed: < 60 seconds
|
||||
- [ ] 10km Standard: < 30 seconds
|
||||
- [ ] 10km Detailed: < 120 seconds (was timeout)
|
||||
- [ ] Radial preview (any radius): < 5 seconds
|
||||
|
||||
### Bug Fix Tests
|
||||
- [ ] Progress bar updates from 5% → 100%
|
||||
- [ ] App closes completely when clicking X
|
||||
- [ ] Memory returns to baseline after calculation
|
||||
- [ ] Western Ukraine region calculates successfully
|
||||
- [ ] All terrain tiles validate correctly
|
||||
- [ ] OSM cache files are valid JSON
|
||||
|
||||
### UI Tests
|
||||
- [ ] Calculate button not overlapping scrollbar
|
||||
- [ ] Dragging site moves all its sectors
|
||||
- [ ] Deleting site removes all its sectors
|
||||
- [ ] No console errors in browser DevTools
|
||||
|
||||
### Data Validation
|
||||
- [ ] Run `diagnose_data_folders()` - check output
|
||||
- [ ] Validate all terrain tiles in cache
|
||||
- [ ] Validate all OSM cache files
|
||||
- [ ] Check for corrupted files and remove them
|
||||
|
||||
---
|
||||
|
||||
## Build & Deploy
|
||||
|
||||
```powershell
|
||||
# Backend
|
||||
cd D:\root\rfcp\backend
|
||||
pip install -e .
|
||||
|
||||
# Frontend
|
||||
cd D:\root\rfcp\frontend
|
||||
npm run build
|
||||
|
||||
# Electron
|
||||
cd D:\root\rfcp\desktop
|
||||
npm run build
|
||||
|
||||
# Full test
|
||||
cd D:\root\rfcp\installer
|
||||
.\test-detailed-quick.bat
|
||||
|
||||
# If works, rebuild installer
|
||||
pyinstaller rfcp-server.spec --clean
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Commit Message
|
||||
|
||||
```
|
||||
feat: Iteration 3.2.0 - Performance & Bug Fixes
|
||||
|
||||
Performance:
|
||||
- Add adaptive resolution (distance-based grid density)
|
||||
- Add radial preview mode (360 spokes, instant feedback)
|
||||
- Expected: 10km Detailed ~60s (was timeout)
|
||||
|
||||
Bug Fixes:
|
||||
- Fix progress bar stuck at 5% (WebSocket + React state)
|
||||
- Fix app close button (Electron lifecycle)
|
||||
- Fix memory leak (SharedMemory cleanup)
|
||||
- Fix region data issues (cache validation)
|
||||
|
||||
UI Improvements:
|
||||
- Move Calculate button (scrollbar overlap)
|
||||
- Site drag moves all sectors
|
||||
- Site delete removes all sectors
|
||||
|
||||
Data Validation:
|
||||
- Add terrain tile validation
|
||||
- Add OSM cache validation
|
||||
- Add diagnostic reporting
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. **10km Detailed completes** without timeout (~60-120s acceptable)
|
||||
2. **Progress bar works** - shows actual progress 5% → 100%
|
||||
3. **App closes cleanly** - no orphan processes
|
||||
4. **Memory released** - returns to baseline after calculation
|
||||
5. **All regions work** - Western Ukraine calculates successfully
|
||||
6. **Site management** - drag/delete affects sectors correctly
|
||||
|
||||
---
|
||||
|
||||
## Priority Order for Implementation
|
||||
|
||||
1. **Adaptive Resolution** - biggest performance impact
|
||||
2. **Progress bar fix** - critical UX issue
|
||||
3. **App close fix** - annoying bug
|
||||
4. **Site drag/delete sectors** - quick win
|
||||
5. **Calculate button position** - quick win
|
||||
6. **Memory leak** - important but complex
|
||||
7. **Region data validation** - diagnostic + fix
|
||||
8. **Radial preview** - nice to have
|
||||
|
||||
---
|
||||
|
||||
## Notes for Claude Code
|
||||
|
||||
- This is a large iteration - take it step by step
|
||||
- Test after each major change
|
||||
- Backend and frontend changes may need coordination
|
||||
- Electron changes require rebuild of desktop app
|
||||
- Data validation can be added as debug endpoint first
|
||||
- If stuck on one issue, move to next and come back
|
||||
|
||||
---
|
||||
|
||||
*"Big iteration, big impact. Let's make RFCP production-ready!"* 🚀
|
||||
261
docs/devlog/installer/RFCP-Iteration-3.2.2-Diagnostic.md
Normal file
261
docs/devlog/installer/RFCP-Iteration-3.2.2-Diagnostic.md
Normal file
@@ -0,0 +1,261 @@
|
||||
# RFCP - Iteration 3.2.2: Dominant Path Performance Diagnostic
|
||||
|
||||
## Overview
|
||||
|
||||
LOD is working (5 buildings instead of 25) but performance is still ~340ms/point.
|
||||
This should be ~15x faster but it's almost the same speed. Need to find the bottleneck.
|
||||
|
||||
**Observed:**
|
||||
```
|
||||
[DOMINANT_PATH_VEC] Point #1: buildings=5, walls=50, dist=2946m
|
||||
338.8ms/point average
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
```
|
||||
5 buildings × 50 walls should be ~20-30ms/point, not 340ms
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Detailed Timing to Dominant Path
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
Add timing breakdown to understand where time is spent:
|
||||
|
||||
```python
|
||||
import time
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def find_dominant_path_with_lod(
|
||||
tx_lat: float, tx_lon: float, tx_height: float,
|
||||
rx_lat: float, rx_lon: float, rx_height: float,
|
||||
frequency_mhz: float,
|
||||
buildings: list,
|
||||
distance_m: float = None,
|
||||
spatial_idx = None, # May be passed in
|
||||
) -> dict:
|
||||
"""Find dominant path with LOD and detailed timing."""
|
||||
|
||||
timings = {}
|
||||
t_total_start = time.perf_counter()
|
||||
|
||||
# 1. Distance calculation
|
||||
t_start = time.perf_counter()
|
||||
if distance_m is None:
|
||||
from app.services.terrain_service import TerrainService
|
||||
distance_m = TerrainService.haversine_distance(tx_lat, tx_lon, rx_lat, rx_lon)
|
||||
timings['distance_calc'] = (time.perf_counter() - t_start) * 1000
|
||||
|
||||
# 2. LOD level determination
|
||||
t_start = time.perf_counter()
|
||||
lod = get_lod_level(distance_m)
|
||||
timings['lod_check'] = (time.perf_counter() - t_start) * 1000
|
||||
|
||||
# 3. Early return for LOD_NONE
|
||||
if lod == LODLevel.NONE:
|
||||
timings['total'] = (time.perf_counter() - t_total_start) * 1000
|
||||
logger.debug(f"[DP_TIMING] LOD_NONE dist={distance_m:.0f}m total={timings['total']:.2f}ms")
|
||||
return {
|
||||
"path_loss_db": 0.0,
|
||||
"lod_level": "none",
|
||||
"buildings_checked": 0,
|
||||
"walls_checked": 0,
|
||||
"skipped": True,
|
||||
"timings": timings
|
||||
}
|
||||
|
||||
# 4. Building filtering for LOD_SIMPLIFIED
|
||||
t_start = time.perf_counter()
|
||||
buildings_to_check = buildings
|
||||
if lod == LODLevel.SIMPLIFIED and buildings:
|
||||
# This filtering might be slow!
|
||||
if len(buildings) > SIMPLIFIED_MAX_BUILDINGS:
|
||||
mid_lat = (tx_lat + rx_lat) / 2
|
||||
mid_lon = (tx_lon + rx_lon) / 2
|
||||
|
||||
buildings_with_dist = []
|
||||
for b in buildings:
|
||||
geom = b.get('geometry', {})
|
||||
coords = geom.get('coordinates', [[]])[0] if isinstance(geom, dict) else b.get('geometry', [[]])
|
||||
|
||||
if coords and len(coords) > 0:
|
||||
if isinstance(coords[0], (list, tuple)):
|
||||
blat = sum(c[1] for c in coords) / len(coords)
|
||||
blon = sum(c[0] for c in coords) / len(coords)
|
||||
else:
|
||||
blat = sum(c.get('lat', c.get('y', 0)) for c in coords) / len(coords)
|
||||
blon = sum(c.get('lon', c.get('x', 0)) for c in coords) / len(coords)
|
||||
|
||||
from app.services.terrain_service import TerrainService
|
||||
dist = TerrainService.haversine_distance(mid_lat, mid_lon, blat, blon)
|
||||
buildings_with_dist.append((dist, b))
|
||||
|
||||
buildings_with_dist.sort(key=lambda x: x[0])
|
||||
buildings_to_check = [b for _, b in buildings_with_dist[:SIMPLIFIED_MAX_BUILDINGS]]
|
||||
timings['building_filter'] = (time.perf_counter() - t_start) * 1000
|
||||
|
||||
# 5. Wall extraction
|
||||
t_start = time.perf_counter()
|
||||
# ... wall extraction code ...
|
||||
timings['wall_extraction'] = (time.perf_counter() - t_start) * 1000
|
||||
|
||||
# 6. Geometry calculations (intersections, reflections)
|
||||
t_start = time.perf_counter()
|
||||
# ... geometry code ...
|
||||
timings['geometry_calc'] = (time.perf_counter() - t_start) * 1000
|
||||
|
||||
# Total
|
||||
timings['total'] = (time.perf_counter() - t_total_start) * 1000
|
||||
|
||||
# Log timing breakdown
|
||||
logger.info(
|
||||
f"[DP_TIMING] LOD={lod.value} dist={distance_m:.0f}m "
|
||||
f"bldgs={len(buildings_to_check)} "
|
||||
f"filter={timings.get('building_filter', 0):.1f}ms "
|
||||
f"walls={timings.get('wall_extraction', 0):.1f}ms "
|
||||
f"geom={timings.get('geometry_calc', 0):.1f}ms "
|
||||
f"total={timings['total']:.1f}ms"
|
||||
)
|
||||
|
||||
result["timings"] = timings
|
||||
return result
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Check if Building Filtering is the Bottleneck
|
||||
|
||||
The LOD_SIMPLIFIED filtering iterates through ALL 15000 buildings to find 5 nearest.
|
||||
This is O(n) for every point!
|
||||
|
||||
**Potential fix - use spatial index:**
|
||||
|
||||
```python
|
||||
# Instead of iterating all buildings:
|
||||
if spatial_idx is not None:
|
||||
# Use spatial index to get nearby buildings quickly
|
||||
nearby = spatial_idx.query_radius(mid_lat, mid_lon, radius=500) # 500m radius
|
||||
buildings_to_check = nearby[:SIMPLIFIED_MAX_BUILDINGS]
|
||||
else:
|
||||
# Fallback to slow method
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Check Coverage Service Integration
|
||||
|
||||
**File:** `backend/app/services/coverage_service.py`
|
||||
|
||||
Find where dominant_path is called and check:
|
||||
1. Is spatial_idx being passed?
|
||||
2. Is building list pre-filtered or full 15000?
|
||||
3. Are buildings being re-processed for each point?
|
||||
|
||||
Look for patterns like:
|
||||
```python
|
||||
# BAD - full list passed to every point
|
||||
for point in points:
|
||||
result = find_dominant_path_with_lod(..., buildings=all_buildings)
|
||||
|
||||
# GOOD - pre-filter by distance to point
|
||||
for point in points:
|
||||
nearby = spatial_idx.query(point)
|
||||
result = find_dominant_path_with_lod(..., buildings=nearby)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Add Summary Statistics
|
||||
|
||||
At the end of coverage calculation, log timing summary:
|
||||
|
||||
```python
|
||||
# In coverage_service.py after all points calculated:
|
||||
if timing_data:
|
||||
avg_filter = sum(t.get('building_filter', 0) for t in timing_data) / len(timing_data)
|
||||
avg_geom = sum(t.get('geometry_calc', 0) for t in timing_data) / len(timing_data)
|
||||
avg_total = sum(t.get('total', 0) for t in timing_data) / len(timing_data)
|
||||
|
||||
logger.info(
|
||||
f"[DP_SUMMARY] {len(timing_data)} points: "
|
||||
f"avg_filter={avg_filter:.1f}ms, avg_geom={avg_geom:.1f}ms, "
|
||||
f"avg_total={avg_total:.1f}ms"
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Quick Win - Skip Filtering for LOD_NONE
|
||||
|
||||
Make sure LOD_NONE returns IMMEDIATELY without touching buildings list:
|
||||
|
||||
```python
|
||||
def find_dominant_path_with_lod(...):
|
||||
# FIRST thing - check LOD
|
||||
if distance_m is None:
|
||||
distance_m = calculate_distance(...)
|
||||
|
||||
lod = get_lod_level(distance_m)
|
||||
|
||||
# IMMEDIATE return for LOD_NONE - don't even look at buildings
|
||||
if lod == LODLevel.NONE:
|
||||
return {"path_loss_db": 0.0, "skipped": True, "lod_level": "none"}
|
||||
|
||||
# Only now process buildings...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Output
|
||||
|
||||
After implementing, logs should show:
|
||||
|
||||
```
|
||||
[DP_TIMING] LOD=none dist=4500m total=0.05ms
|
||||
[DP_TIMING] LOD=simplified dist=2500m bldgs=5 filter=250.0ms walls=2.0ms geom=5.0ms total=258.0ms
|
||||
[DP_TIMING] LOD=full dist=800m bldgs=25 filter=0.0ms walls=5.0ms geom=50.0ms total=56.0ms
|
||||
```
|
||||
|
||||
This will show us exactly where the 340ms is being spent.
|
||||
|
||||
---
|
||||
|
||||
## Suspected Root Cause
|
||||
|
||||
**Building filtering is O(15000) for every point!**
|
||||
|
||||
Even with LOD_SIMPLIFIED, we iterate through 15000 buildings to find 5 nearest.
|
||||
868 points × 15000 buildings = 13 million iterations just for filtering!
|
||||
|
||||
**Fix:** Use spatial index to get nearby buildings in O(log n) instead of O(n).
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
After implementing diagnostics:
|
||||
|
||||
```powershell
|
||||
cd D:\root\rfcp\installer
|
||||
.\test-detailed-quick.bat
|
||||
```
|
||||
|
||||
Check logs for `[DP_TIMING]` and `[DP_SUMMARY]` lines.
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. Logs show timing breakdown for each component
|
||||
2. Identify which step takes most time (filter vs geometry)
|
||||
3. If filter is slow → implement spatial index fix
|
||||
4. If geometry is slow → investigate vectorized calculations
|
||||
|
||||
---
|
||||
|
||||
*"You can't optimize what you can't measure"*
|
||||
@@ -0,0 +1,723 @@
|
||||
# RFCP - Iteration 3.3.0: Performance Architecture Refactor
|
||||
|
||||
## Overview
|
||||
|
||||
Major refactoring based on research into professional RF tools (Signal-Server, SPLAT!, CloudRF SLEIPNIR, Sionna RT).
|
||||
|
||||
**Root cause identified:** Pickle serialization overhead dominates computation time.
|
||||
- DP_TIMING shows: 0.6-0.9ms per point (actual calculation)
|
||||
- Real throughput: 258ms per point
|
||||
- **99% of time is IPC overhead, not calculation!**
|
||||
|
||||
**Target:** Reduce 5km Detailed from timeout (300s) to <30s
|
||||
|
||||
---
|
||||
|
||||
## Part 1: Eliminate Pickle Overhead (CRITICAL)
|
||||
|
||||
### 1.1 Shared Memory for Buildings
|
||||
|
||||
Currently terrain is in shared memory, but **15,000 buildings are pickled for every chunk**.
|
||||
|
||||
**File:** `backend/app/services/parallel_coverage_service.py`
|
||||
|
||||
```python
|
||||
from multiprocessing import shared_memory
|
||||
import numpy as np
|
||||
|
||||
def buildings_to_shared_memory(buildings: list) -> tuple:
|
||||
"""
|
||||
Convert buildings to numpy arrays and store in shared memory.
|
||||
|
||||
Returns: (shm_name, shape, dtype) for reconstruction in workers
|
||||
"""
|
||||
# Extract building data into numpy arrays
|
||||
# For each building we need: lat, lon, height, num_vertices, vertices_flat
|
||||
|
||||
# Simplified: store as structured array
|
||||
building_data = []
|
||||
all_vertices = []
|
||||
vertex_offsets = [0]
|
||||
|
||||
for b in buildings:
|
||||
coords = extract_coords(b)
|
||||
height = b.get('properties', {}).get('height', 10.0)
|
||||
|
||||
building_data.append({
|
||||
'lat': np.mean([c[1] for c in coords]),
|
||||
'lon': np.mean([c[0] for c in coords]),
|
||||
'height': height,
|
||||
'vertex_start': len(all_vertices),
|
||||
'vertex_count': len(coords)
|
||||
})
|
||||
all_vertices.extend(coords)
|
||||
vertex_offsets.append(len(all_vertices))
|
||||
|
||||
# Create numpy arrays
|
||||
buildings_arr = np.array([
|
||||
(b['lat'], b['lon'], b['height'], b['vertex_start'], b['vertex_count'])
|
||||
for b in building_data
|
||||
], dtype=[
|
||||
('lat', 'f8'), ('lon', 'f8'), ('height', 'f4'),
|
||||
('vertex_start', 'i4'), ('vertex_count', 'i2')
|
||||
])
|
||||
|
||||
vertices_arr = np.array(all_vertices, dtype=[('lon', 'f8'), ('lat', 'f8')])
|
||||
|
||||
# Store in shared memory
|
||||
shm_buildings = shared_memory.SharedMemory(
|
||||
create=True,
|
||||
size=buildings_arr.nbytes,
|
||||
name=f"rfcp_buildings_{os.getpid()}"
|
||||
)
|
||||
shm_vertices = shared_memory.SharedMemory(
|
||||
create=True,
|
||||
size=vertices_arr.nbytes,
|
||||
name=f"rfcp_vertices_{os.getpid()}"
|
||||
)
|
||||
|
||||
# Copy data
|
||||
np.ndarray(buildings_arr.shape, dtype=buildings_arr.dtype,
|
||||
buffer=shm_buildings.buf)[:] = buildings_arr
|
||||
np.ndarray(vertices_arr.shape, dtype=vertices_arr.dtype,
|
||||
buffer=shm_vertices.buf)[:] = vertices_arr
|
||||
|
||||
return {
|
||||
'buildings': (shm_buildings.name, buildings_arr.shape, buildings_arr.dtype),
|
||||
'vertices': (shm_vertices.name, vertices_arr.shape, vertices_arr.dtype)
|
||||
}
|
||||
|
||||
|
||||
def buildings_from_shared_memory(shm_info: dict) -> tuple:
|
||||
"""Reconstruct buildings arrays from shared memory in worker."""
|
||||
shm_b = shared_memory.SharedMemory(name=shm_info['buildings'][0])
|
||||
shm_v = shared_memory.SharedMemory(name=shm_info['vertices'][0])
|
||||
|
||||
buildings = np.ndarray(
|
||||
shm_info['buildings'][1],
|
||||
dtype=shm_info['buildings'][2],
|
||||
buffer=shm_b.buf
|
||||
)
|
||||
vertices = np.ndarray(
|
||||
shm_info['vertices'][1],
|
||||
dtype=shm_info['vertices'][2],
|
||||
buffer=shm_v.buf
|
||||
)
|
||||
|
||||
return buildings, vertices, shm_b, shm_v
|
||||
```
|
||||
|
||||
### 1.2 Increase Batch Size
|
||||
|
||||
**Current:** 7 chunks of ~144 points = high IPC overhead per point
|
||||
**Target:** 2-3 chunks of ~300-400 points = amortize IPC cost
|
||||
|
||||
```python
|
||||
# In parallel_coverage_service.py
|
||||
def calculate_optimal_chunk_size(total_points: int, num_workers: int) -> int:
|
||||
"""
|
||||
Calculate chunk size to minimize IPC overhead.
|
||||
|
||||
Rule: computation_time should be 10-100x serialization_time
|
||||
For RF calculations: ~1ms compute, ~50ms serialize
|
||||
So batch at least 500 points to make compute dominate.
|
||||
"""
|
||||
min_chunk = 300 # Minimum to amortize IPC
|
||||
max_chunk = 1000 # Maximum for memory
|
||||
|
||||
ideal_chunks = max(2, num_workers) # At least 2 chunks per worker
|
||||
ideal_size = total_points // ideal_chunks
|
||||
|
||||
return max(min_chunk, min(max_chunk, ideal_size))
|
||||
```
|
||||
|
||||
### 1.3 Pre-build Spatial Index Once
|
||||
|
||||
Currently spatial index may be rebuilt per-chunk. Build once and share reference.
|
||||
|
||||
```python
|
||||
class SharedSpatialIndex:
|
||||
"""Spatial index that can be shared across processes via shared memory."""
|
||||
|
||||
def __init__(self, buildings_shm_info: dict):
|
||||
self.buildings, self.vertices, _, _ = buildings_from_shared_memory(buildings_shm_info)
|
||||
self._build_grid()
|
||||
|
||||
def _build_grid(self):
|
||||
"""Build simple grid-based spatial index."""
|
||||
# Grid cells of ~100m
|
||||
self.cell_size = 0.001 # ~111m in degrees
|
||||
self.grid = defaultdict(list)
|
||||
|
||||
for i, b in enumerate(self.buildings):
|
||||
cell_x = int(b['lon'] / self.cell_size)
|
||||
cell_y = int(b['lat'] / self.cell_size)
|
||||
self.grid[(cell_x, cell_y)].append(i)
|
||||
|
||||
def query_radius(self, lat: float, lon: float, radius_m: float) -> list:
|
||||
"""Get building indices within radius."""
|
||||
radius_deg = radius_m / 111000
|
||||
cells_to_check = int(radius_deg / self.cell_size) + 1
|
||||
|
||||
center_x = int(lon / self.cell_size)
|
||||
center_y = int(lat / self.cell_size)
|
||||
|
||||
result = []
|
||||
for dx in range(-cells_to_check, cells_to_check + 1):
|
||||
for dy in range(-cells_to_check, cells_to_check + 1):
|
||||
result.extend(self.grid.get((center_x + dx, center_y + dy), []))
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 2: Radial Calculation Pattern (Signal-Server style)
|
||||
|
||||
Instead of grid, calculate along radial spokes for faster coverage estimation.
|
||||
|
||||
### 2.1 Radial Engine
|
||||
|
||||
**File:** `backend/app/services/radial_coverage_service.py` (NEW)
|
||||
|
||||
```python
|
||||
"""
|
||||
Radial coverage calculation engine inspired by Signal-Server/SPLAT!
|
||||
|
||||
Instead of calculating every grid point independently:
|
||||
1. Cast rays from TX in all directions (0-360°)
|
||||
2. Sample terrain along each ray (profile)
|
||||
3. Apply propagation model to profile
|
||||
4. Interpolate between rays for final grid
|
||||
|
||||
This is 10-50x faster because:
|
||||
- Terrain profiles are linear (cache-friendly)
|
||||
- No building geometry per-point (use clutter model)
|
||||
- Embarrassingly parallel by azimuth
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import math
|
||||
|
||||
class RadialCoverageEngine:
|
||||
def __init__(self, terrain_service, propagation_model):
|
||||
self.terrain = terrain_service
|
||||
self.model = propagation_model
|
||||
|
||||
def calculate_coverage(
|
||||
self,
|
||||
tx_lat: float, tx_lon: float, tx_height: float,
|
||||
radius_m: float,
|
||||
frequency_mhz: float,
|
||||
tx_power_dbm: float,
|
||||
num_radials: int = 360, # 1° resolution
|
||||
samples_per_radial: int = 100,
|
||||
num_threads: int = 8
|
||||
) -> dict:
|
||||
"""
|
||||
Calculate coverage using radial ray-casting.
|
||||
|
||||
Returns dict with 'radials' (raw data) and 'grid' (interpolated).
|
||||
"""
|
||||
# Pre-load terrain tiles
|
||||
self._preload_terrain(tx_lat, tx_lon, radius_m)
|
||||
|
||||
# Calculate radials in parallel (by azimuth sectors)
|
||||
sector_size = num_radials // num_threads
|
||||
|
||||
with ThreadPoolExecutor(max_workers=num_threads) as executor:
|
||||
futures = []
|
||||
for i in range(num_threads):
|
||||
start_az = i * sector_size
|
||||
end_az = (i + 1) * sector_size if i < num_threads - 1 else num_radials
|
||||
|
||||
futures.append(executor.submit(
|
||||
self._calculate_sector,
|
||||
tx_lat, tx_lon, tx_height,
|
||||
radius_m, frequency_mhz, tx_power_dbm,
|
||||
start_az, end_az, samples_per_radial
|
||||
))
|
||||
|
||||
# Collect results
|
||||
all_radials = []
|
||||
for f in futures:
|
||||
all_radials.extend(f.result())
|
||||
|
||||
return {
|
||||
'radials': all_radials,
|
||||
'center': (tx_lat, tx_lon),
|
||||
'radius': radius_m,
|
||||
'num_radials': num_radials
|
||||
}
|
||||
|
||||
def _calculate_sector(
|
||||
self,
|
||||
tx_lat, tx_lon, tx_height,
|
||||
radius_m, frequency_mhz, tx_power_dbm,
|
||||
start_az, end_az, samples_per_radial
|
||||
) -> list:
|
||||
"""Calculate radials for one azimuth sector."""
|
||||
results = []
|
||||
|
||||
for az in range(start_az, end_az):
|
||||
radial = self._calculate_radial(
|
||||
tx_lat, tx_lon, tx_height,
|
||||
radius_m, frequency_mhz, tx_power_dbm,
|
||||
az, samples_per_radial
|
||||
)
|
||||
results.append(radial)
|
||||
|
||||
return results
|
||||
|
||||
def _calculate_radial(
|
||||
self,
|
||||
tx_lat, tx_lon, tx_height,
|
||||
radius_m, frequency_mhz, tx_power_dbm,
|
||||
azimuth_deg, num_samples
|
||||
) -> dict:
|
||||
"""
|
||||
Calculate signal strength along one radial.
|
||||
|
||||
Uses terrain profile + Longley-Rice style calculation.
|
||||
"""
|
||||
az_rad = math.radians(azimuth_deg)
|
||||
cos_lat = math.cos(math.radians(tx_lat))
|
||||
|
||||
# Sample points along radial
|
||||
distances = np.linspace(100, radius_m, num_samples)
|
||||
|
||||
# Calculate lat/lon for each sample
|
||||
lat_offsets = (distances / 111000) * math.cos(az_rad)
|
||||
lon_offsets = (distances / (111000 * cos_lat)) * math.sin(az_rad)
|
||||
|
||||
lats = tx_lat + lat_offsets
|
||||
lons = tx_lon + lon_offsets
|
||||
|
||||
# Get terrain profile
|
||||
elevations = np.array([
|
||||
self.terrain.get_elevation_sync(lat, lon)
|
||||
for lat, lon in zip(lats, lons)
|
||||
])
|
||||
|
||||
tx_elevation = self.terrain.get_elevation_sync(tx_lat, tx_lon)
|
||||
|
||||
# Calculate path loss for each point
|
||||
rsrp_values = []
|
||||
los_flags = []
|
||||
|
||||
for i, (dist, elev) in enumerate(zip(distances, elevations)):
|
||||
# Simple LOS check using terrain profile up to this point
|
||||
profile = elevations[:i+1]
|
||||
has_los = self._check_los_profile(
|
||||
tx_elevation + tx_height,
|
||||
elev + 1.5, # RX height
|
||||
profile,
|
||||
distances[:i+1]
|
||||
)
|
||||
|
||||
# Path loss (using configured model)
|
||||
path_loss = self.model.calculate_path_loss(
|
||||
frequency_mhz, dist, tx_height, 1.5,
|
||||
has_los=has_los
|
||||
)
|
||||
|
||||
# Add diffraction loss if NLOS
|
||||
if not has_los:
|
||||
diff_loss = self._calculate_diffraction_loss(
|
||||
tx_elevation + tx_height,
|
||||
elev + 1.5,
|
||||
profile,
|
||||
distances[:i+1],
|
||||
frequency_mhz
|
||||
)
|
||||
path_loss += diff_loss
|
||||
|
||||
rsrp = tx_power_dbm - path_loss
|
||||
rsrp_values.append(rsrp)
|
||||
los_flags.append(has_los)
|
||||
|
||||
return {
|
||||
'azimuth': azimuth_deg,
|
||||
'distances': distances.tolist(),
|
||||
'lats': lats.tolist(),
|
||||
'lons': lons.tolist(),
|
||||
'rsrp': rsrp_values,
|
||||
'has_los': los_flags
|
||||
}
|
||||
|
||||
def _check_los_profile(self, tx_h, rx_h, profile, distances) -> bool:
|
||||
"""Check LOS using terrain profile (Fresnel zone clearance)."""
|
||||
if len(profile) < 2:
|
||||
return True
|
||||
|
||||
total_dist = distances[-1]
|
||||
|
||||
# Line from TX to RX
|
||||
for i in range(1, len(profile) - 1):
|
||||
d = distances[i]
|
||||
# Expected height on LOS line
|
||||
expected_h = tx_h + (rx_h - tx_h) * (d / total_dist)
|
||||
# Actual terrain height
|
||||
actual_h = profile[i]
|
||||
|
||||
if actual_h > expected_h - 0.6: # Small clearance margin
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def _calculate_diffraction_loss(self, tx_h, rx_h, profile, distances, freq_mhz) -> float:
|
||||
"""Calculate diffraction loss using Deygout method."""
|
||||
# Find main obstacle
|
||||
max_v = -999
|
||||
max_idx = -1
|
||||
total_dist = distances[-1]
|
||||
wavelength = 300 / freq_mhz # meters
|
||||
|
||||
for i in range(1, len(profile) - 1):
|
||||
d1 = distances[i]
|
||||
d2 = total_dist - d1
|
||||
|
||||
# Height of LOS line at this point
|
||||
los_h = tx_h + (rx_h - tx_h) * (d1 / total_dist)
|
||||
|
||||
# Obstacle height above LOS
|
||||
h = profile[i] - los_h
|
||||
|
||||
if h > 0:
|
||||
# Fresnel parameter
|
||||
v = h * math.sqrt(2 * (d1 + d2) / (wavelength * d1 * d2))
|
||||
if v > max_v:
|
||||
max_v = v
|
||||
max_idx = i
|
||||
|
||||
if max_v < -0.78:
|
||||
return 0.0
|
||||
|
||||
# Knife-edge diffraction loss (ITU-R P.526)
|
||||
if max_v < 0:
|
||||
loss = 6.02 + 9.11 * max_v - 1.27 * max_v * max_v
|
||||
elif max_v < 2.4:
|
||||
loss = 6.02 + 9.11 * max_v + 1.65 * max_v * max_v
|
||||
else:
|
||||
loss = 12.953 + 20 * math.log10(max_v)
|
||||
|
||||
return max(0, loss)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 3: Propagation Model Updates
|
||||
|
||||
### 3.1 Add Longley-Rice ITM Support
|
||||
|
||||
**File:** `backend/app/services/propagation_models/itm_model.py` (NEW)
|
||||
|
||||
```python
|
||||
"""
|
||||
Longley-Rice Irregular Terrain Model (ITM)
|
||||
|
||||
Best for: VHF/UHF terrain-based propagation (20 MHz - 20 GHz)
|
||||
Based on: itmlogic Python package
|
||||
|
||||
Key parameters:
|
||||
- Earth dielectric constant (eps): 4-81 (15 typical for ground)
|
||||
- Ground conductivity (sgm): 0.001-5.0 S/m
|
||||
- Atmospheric refractivity (ens): 250-400 N-units (301 typical)
|
||||
- Climate: 1=Equatorial, 2=Continental Subtropical, etc.
|
||||
"""
|
||||
|
||||
try:
|
||||
from itmlogic import itmlogic_p2p
|
||||
HAS_ITMLOGIC = True
|
||||
except ImportError:
|
||||
HAS_ITMLOGIC = False
|
||||
|
||||
from .base_model import BasePropagationModel, PropagationInput, PropagationResult
|
||||
|
||||
class LongleyRiceModel(BasePropagationModel):
|
||||
"""Longley-Rice ITM propagation model."""
|
||||
|
||||
name = "Longley-Rice-ITM"
|
||||
frequency_range = (20, 20000) # MHz
|
||||
distance_range = (1000, 2000000) # meters
|
||||
|
||||
# Default ITM parameters
|
||||
DEFAULT_PARAMS = {
|
||||
'eps': 15.0, # Earth dielectric constant
|
||||
'sgm': 0.005, # Ground conductivity (S/m)
|
||||
'ens': 301.0, # Atmospheric refractivity (N-units)
|
||||
'pol': 0, # Polarization: 0=horizontal, 1=vertical
|
||||
'mdvar': 12, # Mode of variability
|
||||
'klim': 5, # Climate: 5=Continental Temperate
|
||||
}
|
||||
|
||||
# Ground parameters by type
|
||||
GROUND_PARAMS = {
|
||||
'average': {'eps': 15.0, 'sgm': 0.005},
|
||||
'poor': {'eps': 4.0, 'sgm': 0.001},
|
||||
'good': {'eps': 25.0, 'sgm': 0.020},
|
||||
'fresh_water': {'eps': 81.0, 'sgm': 0.010},
|
||||
'sea_water': {'eps': 81.0, 'sgm': 5.0},
|
||||
'forest': {'eps': 12.0, 'sgm': 0.003},
|
||||
}
|
||||
|
||||
def __init__(self, ground_type: str = 'average', climate: int = 5):
|
||||
if not HAS_ITMLOGIC:
|
||||
raise ImportError("itmlogic package required: pip install itmlogic")
|
||||
|
||||
self.params = self.DEFAULT_PARAMS.copy()
|
||||
if ground_type in self.GROUND_PARAMS:
|
||||
self.params.update(self.GROUND_PARAMS[ground_type])
|
||||
self.params['klim'] = climate
|
||||
|
||||
def calculate(self, input: PropagationInput) -> PropagationResult:
|
||||
"""Calculate path loss using ITM point-to-point mode."""
|
||||
|
||||
# ITM requires terrain profile
|
||||
if not hasattr(input, 'terrain_profile') or input.terrain_profile is None:
|
||||
# Fallback to free-space if no terrain
|
||||
return self._free_space_fallback(input)
|
||||
|
||||
result = itmlogic_p2p(
|
||||
input.terrain_profile, # Elevation samples
|
||||
input.frequency_mhz,
|
||||
input.tx_height_m,
|
||||
input.rx_height_m,
|
||||
self.params['eps'],
|
||||
self.params['sgm'],
|
||||
self.params['ens'],
|
||||
self.params['pol'],
|
||||
self.params['mdvar'],
|
||||
self.params['klim']
|
||||
)
|
||||
|
||||
return PropagationResult(
|
||||
path_loss_db=result['dbloss'],
|
||||
model_name=self.name,
|
||||
details={
|
||||
'mode': result.get('propmode', 'unknown'),
|
||||
'variability': result.get('var', 0),
|
||||
}
|
||||
)
|
||||
|
||||
def _free_space_fallback(self, input: PropagationInput) -> PropagationResult:
|
||||
"""Free-space path loss when no terrain available."""
|
||||
fspl = 20 * np.log10(input.distance_m) + 20 * np.log10(input.frequency_mhz) - 27.55
|
||||
return PropagationResult(
|
||||
path_loss_db=fspl,
|
||||
model_name=f"{self.name} (FSPL fallback)",
|
||||
details={'mode': 'free_space'}
|
||||
)
|
||||
```
|
||||
|
||||
### 3.2 Add VHF/UHF Model Selection
|
||||
|
||||
**File:** `backend/app/services/propagation_models/model_selector.py`
|
||||
|
||||
```python
|
||||
"""
|
||||
Automatic propagation model selection based on frequency and environment.
|
||||
"""
|
||||
|
||||
def select_model_for_frequency(
|
||||
frequency_mhz: float,
|
||||
environment: str = 'urban',
|
||||
has_terrain: bool = True
|
||||
) -> BasePropagationModel:
|
||||
"""
|
||||
Select appropriate propagation model.
|
||||
|
||||
Frequency bands:
|
||||
- VHF: 30-300 MHz (tactical radios, FM broadcast)
|
||||
- UHF: 300-3000 MHz (tactical radios, TV, early cellular)
|
||||
- Cellular: 700-2600 MHz (LTE bands)
|
||||
- mmWave: 24-100 GHz (5G)
|
||||
|
||||
Decision tree:
|
||||
1. VHF/UHF with terrain → Longley-Rice ITM
|
||||
2. Urban cellular → COST-231 Hata
|
||||
3. Suburban/rural cellular → Okumura-Hata
|
||||
4. mmWave → 3GPP 38.901
|
||||
"""
|
||||
|
||||
# VHF (30-300 MHz)
|
||||
if 30 <= frequency_mhz <= 300:
|
||||
if has_terrain:
|
||||
return LongleyRiceModel(ground_type='average', climate=5)
|
||||
else:
|
||||
return FreeSpaceModel() # Fallback
|
||||
|
||||
# UHF (300-1000 MHz)
|
||||
elif 300 < frequency_mhz <= 1000:
|
||||
if has_terrain:
|
||||
return LongleyRiceModel(ground_type='average', climate=5)
|
||||
else:
|
||||
return OkumuraHataModel(environment=environment)
|
||||
|
||||
# Cellular (1000-2600 MHz)
|
||||
elif 1000 < frequency_mhz <= 2600:
|
||||
if environment == 'urban':
|
||||
return Cost231HataModel()
|
||||
else:
|
||||
return OkumuraHataModel(environment=environment)
|
||||
|
||||
# Higher frequencies
|
||||
else:
|
||||
return FreeSpaceModel() # Or implement 3GPP 38.901
|
||||
|
||||
|
||||
# Frequency band constants for UI
|
||||
FREQUENCY_BANDS = {
|
||||
'VHF_LOW': (30, 88, "VHF Low (30-88 MHz) - Military/Public Safety"),
|
||||
'VHF_HIGH': (136, 174, "VHF High (136-174 MHz) - Marine/Aviation"),
|
||||
'UHF_LOW': (400, 512, "UHF (400-512 MHz) - Public Safety/Tactical"),
|
||||
'UHF_TV': (470, 862, "UHF TV (470-862 MHz)"),
|
||||
'LTE_700': (700, 800, "LTE Band 28/20 (700-800 MHz)"),
|
||||
'LTE_900': (880, 960, "LTE Band 8 (900 MHz)"),
|
||||
'LTE_1800': (1710, 1880, "LTE Band 3 (1800 MHz)"),
|
||||
'LTE_2100': (1920, 2170, "LTE Band 1 (2100 MHz)"),
|
||||
'LTE_2600': (2500, 2690, "LTE Band 7 (2600 MHz)"),
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 4: Progress Bar Fix (WebSocket)
|
||||
|
||||
### 4.1 Proper Progress Streaming
|
||||
|
||||
The 5% bug persists because WebSocket messages aren't reaching frontend.
|
||||
|
||||
**Debug approach:**
|
||||
|
||||
```python
|
||||
# In coverage calculation, add explicit progress logging
|
||||
async def calculate_with_progress(self, ...):
|
||||
total_points = len(points)
|
||||
|
||||
for i, chunk_result in enumerate(chunk_results):
|
||||
progress = int((i + 1) / total_chunks * 100)
|
||||
|
||||
# Log to console AND send via WebSocket
|
||||
logger.info(f"[PROGRESS] {progress}% - chunk {i+1}/{total_chunks}")
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback(progress, f"Calculating... {i+1}/{total_chunks}")
|
||||
await asyncio.sleep(0) # Yield to event loop
|
||||
```
|
||||
|
||||
**Frontend fix - check WebSocket subscription:**
|
||||
|
||||
```typescript
|
||||
// In App.tsx or CoverageStore
|
||||
useEffect(() => {
|
||||
const ws = new WebSocket('ws://localhost:8888/ws/coverage');
|
||||
|
||||
ws.onmessage = (event) => {
|
||||
const data = JSON.parse(event.data);
|
||||
console.log('[WS] Received:', data); // DEBUG
|
||||
|
||||
if (data.type === 'progress') {
|
||||
setProgress(data.progress);
|
||||
setProgressStatus(data.status);
|
||||
}
|
||||
};
|
||||
|
||||
ws.onerror = (e) => console.error('[WS] Error:', e);
|
||||
ws.onclose = () => console.log('[WS] Closed');
|
||||
|
||||
return () => ws.close();
|
||||
}, []);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 5: Testing & Validation
|
||||
|
||||
### 5.1 Performance Benchmarks
|
||||
|
||||
After refactoring, expected performance:
|
||||
|
||||
| Scenario | Before | After | Speedup |
|
||||
|----------|--------|-------|---------|
|
||||
| 5km Standard | 5s | 3s | 1.7x |
|
||||
| 5km Detailed | timeout | 25s | 12x |
|
||||
| 10km Standard | 30s | 10s | 3x |
|
||||
| 10km Detailed | timeout | 60s | 5x |
|
||||
|
||||
### 5.2 Test Commands
|
||||
|
||||
```powershell
|
||||
# Quick test
|
||||
cd D:\root\rfcp\installer
|
||||
.\test-detailed-quick.bat
|
||||
|
||||
# Check for [PROGRESS] logs in output
|
||||
# Check for [DP_TIMING] logs
|
||||
|
||||
# Verify shared memory cleanup
|
||||
# Check Task Manager for memory after calculation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. **Shared Memory for Buildings** (biggest impact) - Part 1.1
|
||||
2. **Increase Batch Size** - Part 1.2
|
||||
3. **Progress Bar Debug** - Part 4
|
||||
4. **Radial Engine** (optional, for preview mode) - Part 2
|
||||
5. **Longley-Rice ITM** (for VHF/UHF) - Part 3
|
||||
|
||||
---
|
||||
|
||||
## Dependencies to Add
|
||||
|
||||
```
|
||||
# requirements.txt additions
|
||||
itmlogic>=0.1.0 # Longley-Rice ITM implementation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Commit Message
|
||||
|
||||
```
|
||||
feat: Iteration 3.3.0 - Performance Architecture Refactor
|
||||
|
||||
Performance:
|
||||
- Add shared memory for buildings (eliminate pickle overhead)
|
||||
- Increase batch size to 300-500 points (amortize IPC)
|
||||
- Add radial coverage engine (Signal-Server style)
|
||||
|
||||
Propagation Models:
|
||||
- Add Longley-Rice ITM for VHF/UHF (20 MHz - 20 GHz)
|
||||
- Add automatic model selection by frequency
|
||||
- Add frequency band constants for UI
|
||||
|
||||
Bug Fixes:
|
||||
- Debug and fix WebSocket progress (5% stuck bug)
|
||||
|
||||
Expected: 5km Detailed from timeout → ~25s (12x speedup)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes for Claude Code
|
||||
|
||||
This is a significant refactoring. Approach step by step:
|
||||
|
||||
1. First implement shared memory for buildings
|
||||
2. Test that alone - should see major speedup
|
||||
3. Then increase batch size
|
||||
4. Test again
|
||||
5. Then tackle progress bar
|
||||
6. Radial engine and ITM can be separate iterations if needed
|
||||
|
||||
The key insight: **99% of time is IPC overhead, not calculation**.
|
||||
Fixing pickle serialization is the #1 priority.
|
||||
|
||||
---
|
||||
|
||||
*"Fast per-point means nothing if IPC eats your lunch"* 🍽️
|
||||
821
docs/devlog/installer/RFCP-Phase-2.4-GPU-Elevation.md
Normal file
821
docs/devlog/installer/RFCP-Phase-2.4-GPU-Elevation.md
Normal file
@@ -0,0 +1,821 @@
|
||||
# RFCP Phase 2.4: GPU Acceleration + Elevation Layer
|
||||
|
||||
**Date:** February 1, 2025
|
||||
**Type:** Performance + UI Enhancement
|
||||
**Priority:** HIGH
|
||||
**Depends on:** Phase 2.3 (Performance fixes)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Goals
|
||||
|
||||
1. **Elevation Layer** — візуалізація рельєфу на карті
|
||||
2. **GPU Acceleration** — прискорення розрахунків через CUDA
|
||||
3. **Bug Fixes** — закриття app, timeout handling
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Bug Fixes (CRITICAL — Do First!)
|
||||
|
||||
### Bug 2.4.0a: App Close Still Not Working
|
||||
|
||||
**Symptoms:**
|
||||
- Clicking X closes window but processes stay running
|
||||
- rfcp-server.exe stays in Task Manager
|
||||
- Have to manually kill processes
|
||||
|
||||
**File:** `desktop/main.js`
|
||||
|
||||
**Debug steps:**
|
||||
1. Add console.log at START of killBackend():
|
||||
```javascript
|
||||
function killBackend() {
|
||||
console.log('[KILL] killBackend() called, pid:', backendPid);
|
||||
// ... rest of function
|
||||
}
|
||||
```
|
||||
|
||||
2. Add console.log in close handler:
|
||||
```javascript
|
||||
mainWindow.on('close', (event) => {
|
||||
console.log('[CLOSE] Window close event triggered');
|
||||
killBackend();
|
||||
});
|
||||
```
|
||||
|
||||
3. Check if the issue is:
|
||||
- killBackend() not being called at all
|
||||
- taskkill not working (wrong PID?)
|
||||
- Process spawning children that aren't killed
|
||||
|
||||
**Potential fix:**
|
||||
```javascript
|
||||
function killBackend() {
|
||||
console.log('[KILL] killBackend() called');
|
||||
|
||||
if (!backendPid && !backendProcess) {
|
||||
console.log('[KILL] No backend to kill');
|
||||
return;
|
||||
}
|
||||
|
||||
const pid = backendPid || backendProcess?.pid;
|
||||
console.log('[KILL] Killing PID:', pid);
|
||||
|
||||
if (process.platform === 'win32') {
|
||||
// Force kill entire process tree
|
||||
try {
|
||||
require('child_process').execSync(`taskkill /F /T /PID ${pid}`, {
|
||||
stdio: 'ignore'
|
||||
});
|
||||
console.log('[KILL] taskkill completed');
|
||||
} catch (e) {
|
||||
console.log('[KILL] taskkill error:', e.message);
|
||||
}
|
||||
}
|
||||
|
||||
backendProcess = null;
|
||||
backendPid = null;
|
||||
}
|
||||
```
|
||||
|
||||
4. Add in app quit:
|
||||
```javascript
|
||||
app.on('before-quit', () => {
|
||||
console.log('[QUIT] before-quit event');
|
||||
killBackend();
|
||||
});
|
||||
|
||||
app.on('will-quit', () => {
|
||||
console.log('[QUIT] will-quit event');
|
||||
killBackend();
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Bug 2.4.0b: Calculation Continues After Timeout
|
||||
|
||||
**Symptoms:**
|
||||
- User gets "timeout" error in UI
|
||||
- But backend keeps calculating (CPU stays loaded)
|
||||
- Machine stays slow until manually kill process
|
||||
|
||||
**File:** `backend/app/services/coverage_service.py`
|
||||
|
||||
**Root cause:** asyncio.wait_for() cancels the coroutine but:
|
||||
- ProcessPoolExecutor workers keep running
|
||||
- Ray tasks keep running
|
||||
- No cancellation signal sent
|
||||
|
||||
**Fix in coverage_service.py:**
|
||||
|
||||
```python
|
||||
# Add cancellation flag
|
||||
_calculation_cancelled = False
|
||||
|
||||
async def calculate_coverage(sites, settings):
|
||||
global _calculation_cancelled
|
||||
_calculation_cancelled = False
|
||||
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
_do_calculation(sites, settings),
|
||||
timeout=300 # 5 minutes
|
||||
)
|
||||
return result
|
||||
except asyncio.TimeoutError:
|
||||
_calculation_cancelled = True
|
||||
_cleanup_running_tasks() # NEW
|
||||
raise HTTPException(408, "Calculation timeout")
|
||||
|
||||
def _cleanup_running_tasks():
|
||||
"""Stop any running parallel workers."""
|
||||
global _calculation_cancelled
|
||||
_calculation_cancelled = True
|
||||
|
||||
# If using Ray
|
||||
if RAY_AVAILABLE and ray.is_initialized():
|
||||
# Cancel pending tasks
|
||||
# Ray doesn't have great cancellation, but we can try
|
||||
pass
|
||||
|
||||
# If using ProcessPoolExecutor - it will check flag
|
||||
_clog("Calculation cancelled, cleaning up workers")
|
||||
```
|
||||
|
||||
**In parallel workers, check cancellation:**
|
||||
|
||||
```python
|
||||
def _process_chunk(chunk, ...):
|
||||
results = []
|
||||
for point in chunk:
|
||||
# Check if cancelled
|
||||
if _calculation_cancelled:
|
||||
_clog("Worker detected cancellation, stopping")
|
||||
break
|
||||
|
||||
result = _calculate_point_sync(point, ...)
|
||||
results.append(result)
|
||||
|
||||
return results
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Part A: Elevation Layer
|
||||
|
||||
### A.1: Backend API
|
||||
|
||||
**New file:** `backend/app/api/routes/terrain.py`
|
||||
|
||||
```python
|
||||
from fastapi import APIRouter, Query
|
||||
from typing import List
|
||||
from app.services.terrain_service import terrain_service
|
||||
|
||||
router = APIRouter(prefix="/api/terrain", tags=["terrain"])
|
||||
|
||||
@router.get("/elevation-grid")
|
||||
async def get_elevation_grid(
|
||||
min_lat: float = Query(..., description="South boundary"),
|
||||
max_lat: float = Query(..., description="North boundary"),
|
||||
min_lon: float = Query(..., description="West boundary"),
|
||||
max_lon: float = Query(..., description="East boundary"),
|
||||
resolution: int = Query(100, description="Grid resolution in meters")
|
||||
) -> dict:
|
||||
"""
|
||||
Get elevation grid for a bounding box.
|
||||
Returns a 2D array of elevations for rendering terrain layer.
|
||||
"""
|
||||
# Calculate grid dimensions
|
||||
lat_range = max_lat - min_lat
|
||||
lon_range = max_lon - min_lon
|
||||
|
||||
# Approximate meters per degree
|
||||
meters_per_lat = 111000
|
||||
meters_per_lon = 111000 * cos(radians((min_lat + max_lat) / 2))
|
||||
|
||||
# Grid size
|
||||
rows = int((lat_range * meters_per_lat) / resolution)
|
||||
cols = int((lon_range * meters_per_lon) / resolution)
|
||||
|
||||
# Cap to reasonable size
|
||||
rows = min(rows, 200)
|
||||
cols = min(cols, 200)
|
||||
|
||||
# Build elevation grid
|
||||
elevations = []
|
||||
lat_step = lat_range / rows
|
||||
lon_step = lon_range / cols
|
||||
|
||||
for i in range(rows):
|
||||
row = []
|
||||
lat = max_lat - (i + 0.5) * lat_step # Start from north
|
||||
for j in range(cols):
|
||||
lon = min_lon + (j + 0.5) * lon_step
|
||||
elev = terrain_service.get_elevation_sync(lat, lon)
|
||||
row.append(elev)
|
||||
elevations.append(row)
|
||||
|
||||
# Get min/max for color scaling
|
||||
flat = [e for row in elevations for e in row]
|
||||
|
||||
return {
|
||||
"elevations": elevations,
|
||||
"rows": rows,
|
||||
"cols": cols,
|
||||
"min_elevation": min(flat),
|
||||
"max_elevation": max(flat),
|
||||
"bbox": {
|
||||
"min_lat": min_lat,
|
||||
"max_lat": max_lat,
|
||||
"min_lon": min_lon,
|
||||
"max_lon": max_lon
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Register in main.py:**
|
||||
```python
|
||||
from app.api.routes import terrain
|
||||
app.include_router(terrain.router)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### A.2: Frontend Component
|
||||
|
||||
**New file:** `frontend/src/components/ElevationLayer.tsx`
|
||||
|
||||
```tsx
|
||||
import { useEffect, useRef } from 'react';
|
||||
import { useMap } from 'react-leaflet';
|
||||
import L from 'leaflet';
|
||||
|
||||
interface ElevationLayerProps {
|
||||
enabled: boolean;
|
||||
opacity: number;
|
||||
bbox: {
|
||||
minLat: number;
|
||||
maxLat: number;
|
||||
minLon: number;
|
||||
maxLon: number;
|
||||
} | null;
|
||||
}
|
||||
|
||||
// Color scale: blue (low) → green → yellow → brown (high)
|
||||
const ELEVATION_COLORS = [
|
||||
{ threshold: 0, color: [33, 102, 172] }, // #2166ac deep blue
|
||||
{ threshold: 100, color: [103, 169, 207] }, // #67a9cf light blue
|
||||
{ threshold: 150, color: [145, 207, 96] }, // #91cf60 green
|
||||
{ threshold: 200, color: [254, 224, 139] }, // #fee08b yellow
|
||||
{ threshold: 250, color: [252, 141, 89] }, // #fc8d59 orange
|
||||
{ threshold: 300, color: [215, 48, 39] }, // #d73027 red
|
||||
{ threshold: 400, color: [165, 0, 38] }, // #a50026 dark red
|
||||
];
|
||||
|
||||
function getColorForElevation(elevation: number): [number, number, number] {
|
||||
for (let i = ELEVATION_COLORS.length - 1; i >= 0; i--) {
|
||||
if (elevation >= ELEVATION_COLORS[i].threshold) {
|
||||
if (i === ELEVATION_COLORS.length - 1) {
|
||||
return ELEVATION_COLORS[i].color as [number, number, number];
|
||||
}
|
||||
// Interpolate between this and next color
|
||||
const low = ELEVATION_COLORS[i];
|
||||
const high = ELEVATION_COLORS[i + 1];
|
||||
const t = (elevation - low.threshold) / (high.threshold - low.threshold);
|
||||
return [
|
||||
Math.round(low.color[0] + t * (high.color[0] - low.color[0])),
|
||||
Math.round(low.color[1] + t * (high.color[1] - low.color[1])),
|
||||
Math.round(low.color[2] + t * (high.color[2] - low.color[2])),
|
||||
];
|
||||
}
|
||||
}
|
||||
return ELEVATION_COLORS[0].color as [number, number, number];
|
||||
}
|
||||
|
||||
export function ElevationLayer({ enabled, opacity, bbox }: ElevationLayerProps) {
|
||||
const map = useMap();
|
||||
const canvasRef = useRef<HTMLCanvasElement | null>(null);
|
||||
const overlayRef = useRef<L.ImageOverlay | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled || !bbox) {
|
||||
// Remove overlay if disabled
|
||||
if (overlayRef.current) {
|
||||
map.removeLayer(overlayRef.current);
|
||||
overlayRef.current = null;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Fetch elevation data
|
||||
const fetchElevation = async () => {
|
||||
const params = new URLSearchParams({
|
||||
min_lat: bbox.minLat.toString(),
|
||||
max_lat: bbox.maxLat.toString(),
|
||||
min_lon: bbox.minLon.toString(),
|
||||
max_lon: bbox.maxLon.toString(),
|
||||
resolution: '100',
|
||||
});
|
||||
|
||||
const response = await fetch(`/api/terrain/elevation-grid?${params}`);
|
||||
const data = await response.json();
|
||||
|
||||
// Create canvas
|
||||
const canvas = document.createElement('canvas');
|
||||
canvas.width = data.cols;
|
||||
canvas.height = data.rows;
|
||||
const ctx = canvas.getContext('2d')!;
|
||||
const imageData = ctx.createImageData(data.cols, data.rows);
|
||||
|
||||
// Fill pixel data
|
||||
for (let i = 0; i < data.rows; i++) {
|
||||
for (let j = 0; j < data.cols; j++) {
|
||||
const elevation = data.elevations[i][j];
|
||||
const color = getColorForElevation(elevation);
|
||||
const idx = (i * data.cols + j) * 4;
|
||||
imageData.data[idx] = color[0]; // R
|
||||
imageData.data[idx + 1] = color[1]; // G
|
||||
imageData.data[idx + 2] = color[2]; // B
|
||||
imageData.data[idx + 3] = 255; // A
|
||||
}
|
||||
}
|
||||
|
||||
ctx.putImageData(imageData, 0, 0);
|
||||
|
||||
// Create overlay
|
||||
const bounds = L.latLngBounds(
|
||||
[bbox.minLat, bbox.minLon],
|
||||
[bbox.maxLat, bbox.maxLon]
|
||||
);
|
||||
|
||||
if (overlayRef.current) {
|
||||
map.removeLayer(overlayRef.current);
|
||||
}
|
||||
|
||||
overlayRef.current = L.imageOverlay(canvas.toDataURL(), bounds, {
|
||||
opacity: opacity,
|
||||
interactive: false,
|
||||
});
|
||||
|
||||
overlayRef.current.addTo(map);
|
||||
};
|
||||
|
||||
fetchElevation();
|
||||
|
||||
return () => {
|
||||
if (overlayRef.current) {
|
||||
map.removeLayer(overlayRef.current);
|
||||
}
|
||||
};
|
||||
}, [enabled, opacity, bbox, map]);
|
||||
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### A.3: Layer Controls UI
|
||||
|
||||
**Update:** `frontend/src/App.tsx` or create `LayerControls.tsx`
|
||||
|
||||
```tsx
|
||||
// Add to state
|
||||
const [showElevation, setShowElevation] = useState(false);
|
||||
const [elevationOpacity, setElevationOpacity] = useState(0.5);
|
||||
|
||||
// Add to UI (in settings panel or toolbar)
|
||||
<div className="layer-controls">
|
||||
<h4>Map Layers</h4>
|
||||
|
||||
<label className="layer-toggle">
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={showElevation}
|
||||
onChange={(e) => setShowElevation(e.target.checked)}
|
||||
/>
|
||||
Show Elevation
|
||||
</label>
|
||||
|
||||
{showElevation && (
|
||||
<div className="elevation-opacity">
|
||||
<label>Opacity: {Math.round(elevationOpacity * 100)}%</label>
|
||||
<input
|
||||
type="range"
|
||||
min="0.2"
|
||||
max="1"
|
||||
step="0.1"
|
||||
value={elevationOpacity}
|
||||
onChange={(e) => setElevationOpacity(parseFloat(e.target.value))}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Elevation legend */}
|
||||
{showElevation && (
|
||||
<div className="elevation-legend">
|
||||
<div className="legend-item">
|
||||
<span className="color-box" style={{background: '#2166ac'}}></span>
|
||||
<100m
|
||||
</div>
|
||||
<div className="legend-item">
|
||||
<span className="color-box" style={{background: '#91cf60'}}></span>
|
||||
150-200m
|
||||
</div>
|
||||
<div className="legend-item">
|
||||
<span className="color-box" style={{background: '#fee08b'}}></span>
|
||||
200-250m
|
||||
</div>
|
||||
<div className="legend-item">
|
||||
<span className="color-box" style={{background: '#d73027'}}></span>
|
||||
>300m
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
// In Map component
|
||||
<ElevationLayer
|
||||
enabled={showElevation}
|
||||
opacity={elevationOpacity}
|
||||
bbox={mapBounds} // Current map view bounds
|
||||
/>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Part B: GPU Acceleration
|
||||
|
||||
### B.1: GPU Service
|
||||
|
||||
**New file:** `backend/app/services/gpu_service.py`
|
||||
|
||||
```python
|
||||
"""
|
||||
GPU acceleration for coverage calculations using CuPy.
|
||||
Falls back to NumPy if CUDA not available.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
from typing import Tuple, Optional
|
||||
import os
|
||||
|
||||
# Try to import CuPy
|
||||
GPU_AVAILABLE = False
|
||||
GPU_INFO = None
|
||||
|
||||
try:
|
||||
import cupy as cp
|
||||
|
||||
# Check if CUDA actually works
|
||||
try:
|
||||
cp.cuda.runtime.getDeviceCount()
|
||||
GPU_AVAILABLE = True
|
||||
|
||||
# Get GPU info
|
||||
props = cp.cuda.runtime.getDeviceProperties(0)
|
||||
GPU_INFO = {
|
||||
'name': props['name'].decode() if isinstance(props['name'], bytes) else props['name'],
|
||||
'memory_mb': props['totalGlobalMem'] // (1024 * 1024),
|
||||
'cuda_version': cp.cuda.runtime.runtimeGetVersion(),
|
||||
}
|
||||
print(f"[GPU] CUDA available: {GPU_INFO['name']} ({GPU_INFO['memory_mb']} MB)")
|
||||
except Exception as e:
|
||||
print(f"[GPU] CUDA device check failed: {e}")
|
||||
|
||||
except ImportError:
|
||||
print("[GPU] CuPy not installed, using CPU only")
|
||||
|
||||
|
||||
def get_array_module():
|
||||
"""Get the appropriate array module (cupy or numpy)."""
|
||||
if GPU_AVAILABLE:
|
||||
return cp
|
||||
return np
|
||||
|
||||
|
||||
def to_gpu(array: np.ndarray) -> 'cp.ndarray | np.ndarray':
|
||||
"""Move array to GPU if available."""
|
||||
if GPU_AVAILABLE:
|
||||
return cp.asarray(array)
|
||||
return array
|
||||
|
||||
|
||||
def to_cpu(array) -> np.ndarray:
|
||||
"""Move array back to CPU."""
|
||||
if GPU_AVAILABLE and hasattr(array, 'get'):
|
||||
return array.get()
|
||||
return np.asarray(array)
|
||||
|
||||
|
||||
class GPUService:
|
||||
"""GPU-accelerated calculations for coverage planning."""
|
||||
|
||||
def __init__(self):
|
||||
self.enabled = GPU_AVAILABLE
|
||||
self.info = GPU_INFO
|
||||
|
||||
def calculate_distances_batch(
|
||||
self,
|
||||
site_lat: float,
|
||||
site_lon: float,
|
||||
point_lats: np.ndarray,
|
||||
point_lons: np.ndarray,
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Calculate Haversine distances from site to all points.
|
||||
Vectorized for GPU acceleration.
|
||||
|
||||
Args:
|
||||
site_lat, site_lon: Site coordinates (degrees)
|
||||
point_lats, point_lons: Arrays of point coordinates (degrees)
|
||||
|
||||
Returns:
|
||||
Array of distances in meters
|
||||
"""
|
||||
xp = get_array_module()
|
||||
|
||||
# Move to GPU if available
|
||||
lats = to_gpu(point_lats)
|
||||
lons = to_gpu(point_lons)
|
||||
|
||||
# Convert to radians
|
||||
lat1 = xp.radians(site_lat)
|
||||
lon1 = xp.radians(site_lon)
|
||||
lat2 = xp.radians(lats)
|
||||
lon2 = xp.radians(lons)
|
||||
|
||||
# Haversine formula (vectorized)
|
||||
dlat = lat2 - lat1
|
||||
dlon = lon2 - lon1
|
||||
|
||||
a = xp.sin(dlat / 2) ** 2 + xp.cos(lat1) * xp.cos(lat2) * xp.sin(dlon / 2) ** 2
|
||||
c = 2 * xp.arcsin(xp.sqrt(a))
|
||||
|
||||
R = 6371000 # Earth radius in meters
|
||||
distances = R * c
|
||||
|
||||
return to_cpu(distances)
|
||||
|
||||
def calculate_free_space_path_loss_batch(
|
||||
self,
|
||||
distances: np.ndarray,
|
||||
frequency_mhz: float,
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Calculate Free Space Path Loss for all distances.
|
||||
|
||||
FSPL = 20*log10(d) + 20*log10(f) + 20*log10(4π/c)
|
||||
= 20*log10(d_km) + 20*log10(f_mhz) + 32.45
|
||||
"""
|
||||
xp = get_array_module()
|
||||
d = to_gpu(distances)
|
||||
|
||||
# Avoid log(0)
|
||||
d_km = xp.maximum(d / 1000.0, 0.001)
|
||||
|
||||
fspl = 20 * xp.log10(d_km) + 20 * xp.log10(frequency_mhz) + 32.45
|
||||
|
||||
return to_cpu(fspl)
|
||||
|
||||
def calculate_okumura_hata_batch(
|
||||
self,
|
||||
distances: np.ndarray,
|
||||
frequency_mhz: float,
|
||||
tx_height: float,
|
||||
rx_height: float = 1.5,
|
||||
environment: str = 'urban',
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Calculate Okumura-Hata path loss for all distances.
|
||||
Vectorized for GPU acceleration.
|
||||
"""
|
||||
xp = get_array_module()
|
||||
d = to_gpu(distances)
|
||||
|
||||
# Avoid log(0)
|
||||
d_km = xp.maximum(d / 1000.0, 0.001)
|
||||
|
||||
f = frequency_mhz
|
||||
hb = tx_height
|
||||
hm = rx_height
|
||||
|
||||
# Mobile antenna height correction (urban)
|
||||
if f <= 200:
|
||||
a_hm = 8.29 * (xp.log10(1.54 * hm)) ** 2 - 1.1
|
||||
elif f >= 400:
|
||||
a_hm = 3.2 * (xp.log10(11.75 * hm)) ** 2 - 4.97
|
||||
else:
|
||||
a_hm = (1.1 * xp.log10(f) - 0.7) * hm - (1.56 * xp.log10(f) - 0.8)
|
||||
|
||||
# Base formula
|
||||
L = (69.55 + 26.16 * xp.log10(f)
|
||||
- 13.82 * xp.log10(hb)
|
||||
- a_hm
|
||||
+ (44.9 - 6.55 * xp.log10(hb)) * xp.log10(d_km))
|
||||
|
||||
# Environment corrections
|
||||
if environment == 'suburban':
|
||||
L = L - 2 * (xp.log10(f / 28)) ** 2 - 5.4
|
||||
elif environment == 'rural':
|
||||
L = L - 4.78 * (xp.log10(f)) ** 2 + 18.33 * xp.log10(f) - 40.94
|
||||
|
||||
return to_cpu(L)
|
||||
|
||||
def calculate_rsrp_batch(
|
||||
self,
|
||||
distances: np.ndarray,
|
||||
tx_power_dbm: float,
|
||||
antenna_gain_dbi: float,
|
||||
frequency_mhz: float,
|
||||
tx_height: float,
|
||||
environment: str = 'urban',
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Calculate RSRP for all points (basic, without terrain/buildings).
|
||||
"""
|
||||
path_loss = self.calculate_okumura_hata_batch(
|
||||
distances, frequency_mhz, tx_height,
|
||||
environment=environment
|
||||
)
|
||||
|
||||
rsrp = tx_power_dbm + antenna_gain_dbi - path_loss
|
||||
|
||||
return rsrp
|
||||
|
||||
|
||||
# Singleton instance
|
||||
gpu_service = GPUService()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### B.2: Integration with Coverage Service
|
||||
|
||||
**Update:** `backend/app/services/coverage_service.py`
|
||||
|
||||
```python
|
||||
from app.services.gpu_service import gpu_service, GPU_AVAILABLE
|
||||
|
||||
# In calculate_coverage, before point loop:
|
||||
|
||||
async def calculate_coverage(sites, settings):
|
||||
# ... existing Phase 1 & 2 code ...
|
||||
|
||||
# Phase 2.5: Pre-calculate with GPU if available
|
||||
if GPU_AVAILABLE and len(grid) > 100:
|
||||
_clog(f"Using GPU acceleration for {len(grid)} points")
|
||||
|
||||
# Prepare arrays
|
||||
point_lats = np.array([p[0] for p in grid])
|
||||
point_lons = np.array([p[1] for p in grid])
|
||||
|
||||
# Calculate all distances at once (GPU)
|
||||
all_distances = gpu_service.calculate_distances_batch(
|
||||
site.lat, site.lon, point_lats, point_lons
|
||||
)
|
||||
|
||||
# Calculate all basic path losses at once (GPU)
|
||||
all_path_losses = gpu_service.calculate_okumura_hata_batch(
|
||||
all_distances,
|
||||
site.frequency,
|
||||
site.height,
|
||||
environment='urban' if settings.use_buildings else 'rural'
|
||||
)
|
||||
|
||||
# Store for use in point loop
|
||||
precomputed = {
|
||||
'distances': all_distances,
|
||||
'path_losses': all_path_losses,
|
||||
}
|
||||
_clog(f"GPU pre-calculation done: {len(grid)} distances + path losses")
|
||||
else:
|
||||
precomputed = None
|
||||
|
||||
# Phase 3: Point loop (uses precomputed if available)
|
||||
# ... modify _calculate_point_sync to accept precomputed values ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### B.3: System Info Update
|
||||
|
||||
**Update:** `backend/app/api/routes/system.py`
|
||||
|
||||
```python
|
||||
from app.services.gpu_service import GPU_AVAILABLE, GPU_INFO
|
||||
|
||||
@router.get("/api/system/info")
|
||||
async def get_system_info():
|
||||
return {
|
||||
"cpu_cores": mp.cpu_count(),
|
||||
"parallel_workers": min(mp.cpu_count() - 2, 14),
|
||||
"parallel_backend": "ray" if RAY_AVAILABLE else "process_pool" if mp.cpu_count() > 1 else "sequential",
|
||||
"ray_available": RAY_AVAILABLE,
|
||||
"gpu": GPU_INFO, # Now includes name, memory, cuda_version
|
||||
"gpu_available": GPU_AVAILABLE,
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### B.4: Requirements
|
||||
|
||||
**Update:** `backend/requirements.txt`
|
||||
|
||||
```
|
||||
# ... existing requirements ...
|
||||
|
||||
# GPU acceleration (optional)
|
||||
# Install with: pip install cupy-cuda12x
|
||||
# Or for CUDA 11.x: pip install cupy-cuda11x
|
||||
# cupy-cuda12x>=12.0.0
|
||||
```
|
||||
|
||||
**Note:** CuPy is optional. Code falls back to NumPy if not installed.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Files to Create/Modify
|
||||
|
||||
**New files:**
|
||||
- `backend/app/api/routes/terrain.py`
|
||||
- `backend/app/services/gpu_service.py`
|
||||
- `frontend/src/components/ElevationLayer.tsx`
|
||||
|
||||
**Modified files:**
|
||||
- `backend/app/main.py` — register terrain router
|
||||
- `backend/app/services/coverage_service.py` — GPU integration, cancellation
|
||||
- `backend/app/api/routes/system.py` — GPU info
|
||||
- `backend/requirements.txt` — cupy optional
|
||||
- `desktop/main.js` — fix app close (debug + fix)
|
||||
- `frontend/src/App.tsx` — elevation layer toggle
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Test Elevation Layer:
|
||||
```bash
|
||||
# Start app
|
||||
./rfcp-debug.bat
|
||||
|
||||
# In browser console or via curl:
|
||||
curl "http://localhost:8888/api/terrain/elevation-grid?min_lat=48.5&max_lat=48.7&min_lon=36.0&max_lon=36.2&resolution=100"
|
||||
|
||||
# Should return JSON with elevations array
|
||||
```
|
||||
|
||||
### Test GPU:
|
||||
```bash
|
||||
# Check system info
|
||||
curl http://localhost:8888/api/system/info
|
||||
|
||||
# Should show:
|
||||
# "gpu_available": true,
|
||||
# "gpu": {"name": "NVIDIA GeForce RTX 4060", "memory_mb": 8192, ...}
|
||||
```
|
||||
|
||||
### Test App Close:
|
||||
```
|
||||
1. Start app via RFCP.exe (not debug bat)
|
||||
2. Click X to close
|
||||
3. Check Task Manager - rfcp-server.exe should NOT be running
|
||||
4. If still running - check console logs for [KILL] messages
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Success Criteria
|
||||
|
||||
- [ ] Elevation layer toggleable on map
|
||||
- [ ] Elevation colors match terrain (verify with known locations)
|
||||
- [ ] GPU detected and shown in system info (if NVIDIA card present)
|
||||
- [ ] Fast preset 2x faster with GPU
|
||||
- [ ] App closes completely when clicking X
|
||||
- [ ] No orphan processes after timeout
|
||||
- [ ] All existing presets still work
|
||||
|
||||
---
|
||||
|
||||
## 📈 Expected Performance
|
||||
|
||||
| Operation | CPU (NumPy) | GPU (CuPy) | Speedup |
|
||||
|-----------|-------------|------------|---------|
|
||||
| 10k distances | 5ms | 0.1ms | 50x |
|
||||
| 10k path losses | 10ms | 0.2ms | 50x |
|
||||
| Full calculation* | 10s | 3s | 3x |
|
||||
|
||||
*Full calculation limited by CPU-bound terrain/building checks
|
||||
|
||||
---
|
||||
|
||||
## 🔜 Next Phase
|
||||
|
||||
Phase 2.5: Advanced Visualization
|
||||
- LOS ray visualization (show blocked paths)
|
||||
- 3D terrain view
|
||||
- Antenna pattern visualization
|
||||
- Multi-site interference view
|
||||
549
docs/devlog/installer/RFCP-Phase-2.4.1-Critical-Fixes.md
Normal file
549
docs/devlog/installer/RFCP-Phase-2.4.1-Critical-Fixes.md
Normal file
@@ -0,0 +1,549 @@
|
||||
# RFCP Phase 2.4.1: Critical Fixes
|
||||
|
||||
**Date:** February 1, 2025
|
||||
**Type:** Bug Fixes + Performance
|
||||
**Priority:** CRITICAL
|
||||
**Depends on:** Phase 2.4
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Goals
|
||||
|
||||
1. Fix memory leak — worker processes not terminating
|
||||
2. Fix app close — хрестик не вбиває backend
|
||||
3. Optimize dominant path — 600 buildings занадто багато
|
||||
4. Fix GPU detection in PyInstaller build
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Bug 2.4.1a: Memory Leak — Workers Not Terminated
|
||||
|
||||
**Symptoms:**
|
||||
- After timeout (300s), worker processes stay running
|
||||
- Task Manager shows 8× rfcp-server.exe using 7.8 GB RAM
|
||||
- CPU stays loaded after calculation cancelled
|
||||
- Only manual kill or reboot frees resources
|
||||
|
||||
**Root cause:**
|
||||
ProcessPoolExecutor workers ignore cancellation token — they're separate processes that don't share memory with main process.
|
||||
|
||||
**File:** `backend/app/services/parallel_coverage_service.py`
|
||||
|
||||
**Fix:**
|
||||
|
||||
```python
|
||||
import psutil
|
||||
import os
|
||||
|
||||
def _kill_worker_processes():
|
||||
"""Kill all child processes of current process."""
|
||||
current = psutil.Process(os.getpid())
|
||||
children = current.children(recursive=True)
|
||||
|
||||
for child in children:
|
||||
try:
|
||||
child.terminate()
|
||||
except psutil.NoSuchProcess:
|
||||
pass
|
||||
|
||||
# Wait briefly, then force kill survivors
|
||||
gone, alive = psutil.wait_procs(children, timeout=3)
|
||||
for p in alive:
|
||||
try:
|
||||
p.kill()
|
||||
except psutil.NoSuchProcess:
|
||||
pass
|
||||
|
||||
return len(children)
|
||||
|
||||
|
||||
async def _calculate_with_process_pool(..., cancel_token=None):
|
||||
"""ProcessPool calculation with proper cleanup."""
|
||||
pool = None
|
||||
try:
|
||||
pool = ProcessPoolExecutor(max_workers=num_workers)
|
||||
futures = [pool.submit(_process_chunk, chunk, ...) for chunk in chunks]
|
||||
|
||||
for future in as_completed(futures):
|
||||
if cancel_token and cancel_token.is_cancelled():
|
||||
_clog("Cancellation detected — terminating pool")
|
||||
break
|
||||
|
||||
result = future.result(timeout=1)
|
||||
results.extend(result)
|
||||
|
||||
except Exception as e:
|
||||
_clog(f"ProcessPool error: {e}")
|
||||
|
||||
finally:
|
||||
# CRITICAL: Always cleanup
|
||||
if pool:
|
||||
pool.shutdown(wait=False, cancel_futures=True)
|
||||
|
||||
# Kill any orphaned workers
|
||||
killed = _kill_worker_processes()
|
||||
if killed > 0:
|
||||
_clog(f"Killed {killed} orphaned worker processes")
|
||||
|
||||
return results
|
||||
```
|
||||
|
||||
**Also add cleanup on timeout in coverage.py:**
|
||||
|
||||
```python
|
||||
@router.post("/calculate")
|
||||
async def calculate_coverage(request: CoverageRequest):
|
||||
cancel_token = CancellationToken()
|
||||
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
coverage_service.calculate_coverage(
|
||||
sites=request.sites,
|
||||
settings=request.settings,
|
||||
cancel_token=cancel_token
|
||||
),
|
||||
timeout=300
|
||||
)
|
||||
return result
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
cancel_token.cancel()
|
||||
|
||||
# Force cleanup
|
||||
from app.services.parallel_coverage_service import _kill_worker_processes
|
||||
killed = _kill_worker_processes()
|
||||
|
||||
raise HTTPException(
|
||||
status_code=408,
|
||||
detail=f"Calculation timeout (5 min). Cleaned up {killed} workers."
|
||||
)
|
||||
```
|
||||
|
||||
**Add psutil to requirements.txt:**
|
||||
```
|
||||
psutil>=5.9.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Bug 2.4.1b: App Close Not Working
|
||||
|
||||
**Symptoms:**
|
||||
- Clicking X closes window but processes stay
|
||||
- Multiple rfcp-server.exe in Task Manager
|
||||
- Have to manually End Task
|
||||
|
||||
**Root cause:**
|
||||
- Electron's killBackend() uses PID that may be wrong
|
||||
- Child processes (workers) not killed
|
||||
- taskkill may fail silently
|
||||
|
||||
**File:** `desktop/main.js`
|
||||
|
||||
**Fix — nuclear option (kill ALL rfcp-server.exe):**
|
||||
|
||||
```javascript
|
||||
const { execSync } = require('child_process');
|
||||
|
||||
function killAllBackendProcesses() {
|
||||
console.log('[KILL] Killing all rfcp-server processes...');
|
||||
|
||||
if (process.platform === 'win32') {
|
||||
try {
|
||||
// Kill ALL rfcp-server.exe processes
|
||||
execSync('taskkill /F /IM rfcp-server.exe /T', {
|
||||
stdio: 'ignore',
|
||||
timeout: 5000
|
||||
});
|
||||
console.log('[KILL] taskkill completed');
|
||||
} catch (e) {
|
||||
// Error means no processes found — that's OK
|
||||
console.log('[KILL] No rfcp-server processes found or already killed');
|
||||
}
|
||||
} else {
|
||||
// Unix: pkill
|
||||
try {
|
||||
execSync('pkill -9 -f rfcp-server', {
|
||||
stdio: 'ignore',
|
||||
timeout: 5000
|
||||
});
|
||||
} catch (e) {
|
||||
console.log('[KILL] No rfcp-server processes found');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Replace killBackend() calls with killAllBackendProcesses()
|
||||
|
||||
// In close handler:
|
||||
mainWindow.on('close', (event) => {
|
||||
console.log('[CLOSE] Window close event');
|
||||
killAllBackendProcesses();
|
||||
});
|
||||
|
||||
// In app quit handlers:
|
||||
app.on('before-quit', () => {
|
||||
console.log('[QUIT] before-quit');
|
||||
killAllBackendProcesses();
|
||||
});
|
||||
|
||||
app.on('will-quit', () => {
|
||||
console.log('[QUIT] will-quit');
|
||||
killAllBackendProcesses();
|
||||
});
|
||||
|
||||
// Last resort:
|
||||
process.on('exit', () => {
|
||||
console.log('[EXIT] process.exit');
|
||||
killAllBackendProcesses();
|
||||
});
|
||||
|
||||
// Also add SIGINT/SIGTERM handlers:
|
||||
process.on('SIGINT', () => {
|
||||
console.log('[SIGNAL] SIGINT received');
|
||||
killAllBackendProcesses();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
process.on('SIGTERM', () => {
|
||||
console.log('[SIGNAL] SIGTERM received');
|
||||
killAllBackendProcesses();
|
||||
process.exit(0);
|
||||
});
|
||||
```
|
||||
|
||||
**Also add cleanup endpoint to backend:**
|
||||
|
||||
```python
|
||||
# backend/app/api/routes/system.py
|
||||
|
||||
@router.post("/shutdown")
|
||||
async def shutdown():
|
||||
"""Graceful shutdown endpoint."""
|
||||
from app.services.parallel_coverage_service import _kill_worker_processes
|
||||
|
||||
killed = _kill_worker_processes()
|
||||
|
||||
# Schedule shutdown
|
||||
import asyncio
|
||||
asyncio.get_event_loop().call_later(0.5, lambda: os._exit(0))
|
||||
|
||||
return {"status": "shutting down", "workers_killed": killed}
|
||||
```
|
||||
|
||||
**Call shutdown from Electron before killing:**
|
||||
|
||||
```javascript
|
||||
async function gracefulShutdown() {
|
||||
console.log('[SHUTDOWN] Requesting graceful shutdown...');
|
||||
|
||||
try {
|
||||
await fetch('http://127.0.0.1:8888/api/system/shutdown', {
|
||||
method: 'POST',
|
||||
timeout: 2000
|
||||
});
|
||||
console.log('[SHUTDOWN] Backend acknowledged');
|
||||
} catch (e) {
|
||||
console.log('[SHUTDOWN] Backend did not respond, force killing');
|
||||
}
|
||||
|
||||
// Wait a moment, then force kill
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
killAllBackendProcesses();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🟡 Bug 2.4.1c: Dominant Path Too Slow
|
||||
|
||||
**Symptoms:**
|
||||
- Detailed preset: 351ms/point (should be <50ms)
|
||||
- Log shows: `line_bldgs=646, refl_bldgs=302`
|
||||
- 600+ buildings checked per point = too slow
|
||||
|
||||
**Root cause:**
|
||||
Spatial index returns all buildings on path, no distance limit.
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
**Fix — limit buildings by distance:**
|
||||
|
||||
```python
|
||||
# At the start of find_dominant_paths() or equivalent:
|
||||
|
||||
MAX_BUILDINGS_FOR_REFLECTION = 100
|
||||
MAX_DISTANCE_FROM_PATH = 500 # meters
|
||||
|
||||
def _filter_buildings_by_distance(buildings, tx_point, rx_point, max_count=100, max_distance=500):
|
||||
"""
|
||||
Filter buildings to only those close to the TX-RX path.
|
||||
Sort by distance to path midpoint, take top N.
|
||||
"""
|
||||
if len(buildings) <= max_count:
|
||||
return buildings
|
||||
|
||||
# Calculate midpoint
|
||||
mid_lat = (tx_point[0] + rx_point[0]) / 2
|
||||
mid_lon = (tx_point[1] + rx_point[1]) / 2
|
||||
|
||||
# Calculate distance to midpoint for each building
|
||||
def distance_to_midpoint(building):
|
||||
blat = building.get('centroid_lat', building.get('lat', mid_lat))
|
||||
blon = building.get('centroid_lon', building.get('lon', mid_lon))
|
||||
# Simple Euclidean approximation (fast)
|
||||
dlat = (blat - mid_lat) * 111000
|
||||
dlon = (blon - mid_lon) * 111000 * 0.7 # rough cos correction
|
||||
return dlat*dlat + dlon*dlon # squared distance, no sqrt needed
|
||||
|
||||
# Sort by distance
|
||||
buildings_with_dist = [(b, distance_to_midpoint(b)) for b in buildings]
|
||||
buildings_with_dist.sort(key=lambda x: x[1])
|
||||
|
||||
# Filter by max distance (squared)
|
||||
max_dist_sq = max_distance * max_distance
|
||||
filtered = [b for b, d in buildings_with_dist if d <= max_dist_sq]
|
||||
|
||||
# Take top N
|
||||
return filtered[:max_count]
|
||||
|
||||
|
||||
# In the main calculation:
|
||||
def calculate_dominant_path(tx, rx, buildings, spatial_idx, ...):
|
||||
# Get buildings from spatial index
|
||||
line_buildings = spatial_idx.query_line(tx.lat, tx.lon, rx.lat, rx.lon)
|
||||
|
||||
# FILTER to reduce count
|
||||
line_buildings = _filter_buildings_by_distance(
|
||||
line_buildings,
|
||||
(tx.lat, tx.lon),
|
||||
(rx.lat, rx.lon),
|
||||
max_count=MAX_BUILDINGS_FOR_REFLECTION,
|
||||
max_distance=MAX_DISTANCE_FROM_PATH
|
||||
)
|
||||
|
||||
# Same for reflection buildings
|
||||
refl_buildings = spatial_idx.query_point(mid_lat, mid_lon, buffer_cells=5)
|
||||
refl_buildings = _filter_buildings_by_distance(
|
||||
refl_buildings,
|
||||
(tx.lat, tx.lon),
|
||||
(rx.lat, rx.lon),
|
||||
max_count=MAX_BUILDINGS_FOR_REFLECTION,
|
||||
max_distance=MAX_DISTANCE_FROM_PATH
|
||||
)
|
||||
|
||||
_clog(f"Filtered: {len(line_buildings)} line, {len(refl_buildings)} refl (from {original_count})")
|
||||
|
||||
# Continue with calculation...
|
||||
```
|
||||
|
||||
**Expected improvement:**
|
||||
- Before: 600-700 buildings → 351ms/point
|
||||
- After: 100 buildings → ~50ms/point
|
||||
- **7x speedup** for Detailed preset
|
||||
|
||||
---
|
||||
|
||||
## 🟡 Bug 2.4.1d: GPU Not Detected in PyInstaller
|
||||
|
||||
**Symptoms:**
|
||||
```json
|
||||
"gpu": {"available": false, "name": null, "memory_mb": null}
|
||||
```
|
||||
- RTX 4060 present but not detected
|
||||
- CuPy not bundled in exe
|
||||
|
||||
**Root cause:**
|
||||
CuPy has complex CUDA dependencies that PyInstaller doesn't auto-detect.
|
||||
|
||||
**Option A: Document manual CuPy install (RECOMMENDED for now)**
|
||||
|
||||
```python
|
||||
# backend/app/services/gpu_service.py
|
||||
|
||||
# At module level, add clear message:
|
||||
GPU_INSTALL_INSTRUCTIONS = """
|
||||
GPU acceleration requires manual CuPy installation:
|
||||
|
||||
1. Check your CUDA version:
|
||||
nvidia-smi
|
||||
|
||||
2. Install matching CuPy:
|
||||
# For CUDA 12.x:
|
||||
pip install cupy-cuda12x
|
||||
|
||||
# For CUDA 11.x:
|
||||
pip install cupy-cuda11x
|
||||
|
||||
3. Restart RFCP
|
||||
|
||||
Note: GPU is optional. CPU calculations work fine.
|
||||
"""
|
||||
|
||||
try:
|
||||
import cupy as cp
|
||||
# ... detection code ...
|
||||
except ImportError:
|
||||
print("[GPU] CuPy not installed — using CPU/NumPy")
|
||||
print("[GPU] To enable GPU acceleration:")
|
||||
print("[GPU] pip install cupy-cuda12x")
|
||||
```
|
||||
|
||||
**Option B: Add CuPy to PyInstaller (complex, large file size)**
|
||||
|
||||
If we want CuPy bundled, add to `installer/rfcp-server.spec`:
|
||||
|
||||
```python
|
||||
hiddenimports=[
|
||||
# ... existing ...
|
||||
'cupy',
|
||||
'cupy._core',
|
||||
'cupy._core._kernel',
|
||||
'cupy.cuda',
|
||||
'cupy.cuda.runtime',
|
||||
'cupy.cuda.driver',
|
||||
# Many more cupy submodules...
|
||||
],
|
||||
|
||||
# Also need to include CUDA DLLs
|
||||
binaries=[
|
||||
# This gets complicated — need CUDA toolkit DLLs
|
||||
],
|
||||
```
|
||||
|
||||
**Recommendation:** Start with Option A (manual install), add Option B later if needed. GPU is nice-to-have, not critical.
|
||||
|
||||
---
|
||||
|
||||
## 🟢 Feature 2.4.1e: Elevation Layer Toggle
|
||||
|
||||
**Current state:**
|
||||
- Elevation layer exists but always visible as green overlay
|
||||
- No toggle in UI
|
||||
|
||||
**File:** `frontend/src/App.tsx` or settings panel
|
||||
|
||||
**Fix:**
|
||||
|
||||
```tsx
|
||||
// In settings/layer controls section:
|
||||
|
||||
<div className="layer-section">
|
||||
<h4>Map Layers</h4>
|
||||
|
||||
<label className="toggle-row">
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={showElevation}
|
||||
onChange={(e) => setShowElevation(e.target.checked)}
|
||||
/>
|
||||
<span>Elevation Colors</span>
|
||||
</label>
|
||||
|
||||
{showElevation && (
|
||||
<div className="slider-row">
|
||||
<label>Opacity</label>
|
||||
<input
|
||||
type="range"
|
||||
min="0.2"
|
||||
max="0.8"
|
||||
step="0.1"
|
||||
value={elevationOpacity}
|
||||
onChange={(e) => setElevationOpacity(parseFloat(e.target.value))}
|
||||
/>
|
||||
<span>{Math.round(elevationOpacity * 100)}%</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
```
|
||||
|
||||
**In Map.tsx:**
|
||||
|
||||
```tsx
|
||||
{showElevation && (
|
||||
<ElevationLayer
|
||||
opacity={elevationOpacity}
|
||||
bounds={mapBounds}
|
||||
/>
|
||||
)}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `backend/app/services/parallel_coverage_service.py` | Add `_kill_worker_processes()`, cleanup in finally block |
|
||||
| `backend/app/api/routes/coverage.py` | Force cleanup on timeout |
|
||||
| `backend/app/api/routes/system.py` | Add `/shutdown` endpoint |
|
||||
| `backend/requirements.txt` | Add `psutil>=5.9.0` |
|
||||
| `desktop/main.js` | Replace `killBackend()` with `killAllBackendProcesses()`, add graceful shutdown |
|
||||
| `backend/app/services/dominant_path_service.py` | Add `_filter_buildings_by_distance()`, limit to 100 buildings |
|
||||
| `backend/app/services/gpu_service.py` | Add install instructions in log |
|
||||
| `frontend/src/App.tsx` | Add elevation toggle if missing |
|
||||
| `frontend/src/components/map/Map.tsx` | Conditional elevation layer |
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Test 1: Memory Cleanup
|
||||
```bash
|
||||
# Start app
|
||||
# Run Detailed preset (will timeout)
|
||||
# Check Task Manager — should be only 1 rfcp-server.exe after timeout
|
||||
# RAM should return to normal (~200MB)
|
||||
```
|
||||
|
||||
### Test 2: App Close
|
||||
```bash
|
||||
# Start RFCP via installer (not debug)
|
||||
# Click X
|
||||
# Check Task Manager — NO rfcp-server.exe should remain
|
||||
```
|
||||
|
||||
### Test 3: Dominant Path Speed
|
||||
```bash
|
||||
# Run test-coverage.bat
|
||||
# Detailed preset should complete in <120s (was 300s timeout)
|
||||
# Log should show "Filtered: 100 line, 100 refl"
|
||||
```
|
||||
|
||||
### Test 4: Elevation Toggle
|
||||
```bash
|
||||
# Open app
|
||||
# Find elevation checkbox in settings
|
||||
# Toggle on/off — layer should appear/disappear
|
||||
# Adjust opacity — should change transparency
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Success Criteria
|
||||
|
||||
- [ ] After timeout: only 1 rfcp-server.exe, RAM < 500MB
|
||||
- [ ] After close (X): 0 rfcp-server.exe processes
|
||||
- [ ] Detailed preset: completes in <120s (not timeout)
|
||||
- [ ] Detailed preset: ~50ms/point (not 350ms)
|
||||
- [ ] Elevation layer: toggleable on/off
|
||||
- [ ] GPU message: clear install instructions in console
|
||||
|
||||
---
|
||||
|
||||
## 📈 Expected Performance After Fixes
|
||||
|
||||
| Preset | Before | After | Improvement |
|
||||
|--------|--------|-------|-------------|
|
||||
| Fast | 0.03s | 0.03s | — |
|
||||
| Standard | 37s | 35s | 1.1x |
|
||||
| Detailed | 300s (timeout) | ~90s | 3x+ |
|
||||
|
||||
---
|
||||
|
||||
## 🔜 Next Steps
|
||||
|
||||
After 2.4.1 fixes:
|
||||
- Phase 2.5: Fun facts loading screen
|
||||
- Phase 2.5: LOS visualization (rays showing blocked paths)
|
||||
- Phase 2.6: Multi-site support improvements
|
||||
495
docs/devlog/installer/RFCP-Phase-2.4.2-Final-Fixes.md
Normal file
495
docs/devlog/installer/RFCP-Phase-2.4.2-Final-Fixes.md
Normal file
@@ -0,0 +1,495 @@
|
||||
# RFCP Phase 2.4.2: Final Critical Fixes
|
||||
|
||||
**Date:** February 1, 2025
|
||||
**Type:** Bug Fixes
|
||||
**Priority:** CRITICAL
|
||||
**Depends on:** Phase 2.4.1
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Summary
|
||||
|
||||
Phase 2.4.1 частково спрацював, але залишились проблеми:
|
||||
- Memory leak — workers не вбиваються (psutil не працює для grandchildren)
|
||||
- Dominant path — все ще timeout (фільтр можливо не застосовується)
|
||||
- Elevation — все зелене (немає локального контрасту)
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Bug 2.4.2a: Memory Leak — Nuclear Kill
|
||||
|
||||
**Problem:**
|
||||
psutil.Process.children() не бачить grandchildren (worker subprocess → python subprocess).
|
||||
Після cleanup все ще 8× rfcp-server.exe, 8GB RAM.
|
||||
|
||||
**File:** `backend/app/services/parallel_coverage_service.py`
|
||||
|
||||
**Current code doesn't work:**
|
||||
```python
|
||||
current = psutil.Process(os.getpid())
|
||||
children = current.children(recursive=True)
|
||||
for child in children:
|
||||
child.terminate()
|
||||
```
|
||||
|
||||
**Fix — kill by process NAME, not by PID tree:**
|
||||
|
||||
```python
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
def _kill_worker_processes():
|
||||
"""
|
||||
Nuclear option: kill ALL rfcp-server processes except main.
|
||||
This handles grandchildren that psutil can't see.
|
||||
"""
|
||||
my_pid = os.getpid()
|
||||
killed_count = 0
|
||||
|
||||
if sys.platform == 'win32':
|
||||
# Windows: use tasklist to find all rfcp-server.exe, kill all except self
|
||||
try:
|
||||
# Get list of all rfcp-server PIDs
|
||||
result = subprocess.run(
|
||||
['tasklist', '/FI', 'IMAGENAME eq rfcp-server.exe', '/FO', 'CSV', '/NH'],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
|
||||
for line in result.stdout.strip().split('\n'):
|
||||
if 'rfcp-server.exe' in line:
|
||||
# Parse PID from CSV: "rfcp-server.exe","1234",...
|
||||
parts = line.split(',')
|
||||
if len(parts) >= 2:
|
||||
pid_str = parts[1].strip().strip('"')
|
||||
try:
|
||||
pid = int(pid_str)
|
||||
if pid != my_pid:
|
||||
subprocess.run(
|
||||
['taskkill', '/F', '/PID', str(pid)],
|
||||
capture_output=True, timeout=5
|
||||
)
|
||||
killed_count += 1
|
||||
_clog(f"Killed worker PID {pid}")
|
||||
except (ValueError, subprocess.TimeoutExpired):
|
||||
pass
|
||||
except Exception as e:
|
||||
_clog(f"Kill workers error: {e}")
|
||||
# Fallback: kill ALL rfcp-server except hope main survives
|
||||
try:
|
||||
subprocess.run(
|
||||
['taskkill', '/F', '/IM', 'rfcp-server.exe', '/T'],
|
||||
capture_output=True, timeout=5
|
||||
)
|
||||
except:
|
||||
pass
|
||||
else:
|
||||
# Unix: use pgrep/pkill
|
||||
try:
|
||||
result = subprocess.run(
|
||||
['pgrep', '-f', 'rfcp-server'],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
for pid_str in result.stdout.strip().split('\n'):
|
||||
if pid_str:
|
||||
try:
|
||||
pid = int(pid_str)
|
||||
if pid != my_pid:
|
||||
os.kill(pid, 9) # SIGKILL
|
||||
killed_count += 1
|
||||
_clog(f"Killed worker PID {pid}")
|
||||
except (ValueError, ProcessLookupError, PermissionError):
|
||||
pass
|
||||
except Exception as e:
|
||||
_clog(f"Kill workers error: {e}")
|
||||
|
||||
return killed_count
|
||||
```
|
||||
|
||||
**Also update ProcessPoolExecutor to use spawn context explicitly:**
|
||||
|
||||
```python
|
||||
import multiprocessing as mp
|
||||
|
||||
def _calculate_with_process_pool(...):
|
||||
# Use spawn to ensure clean worker processes
|
||||
ctx = mp.get_context('spawn')
|
||||
pool = None
|
||||
|
||||
try:
|
||||
pool = ProcessPoolExecutor(
|
||||
max_workers=num_workers,
|
||||
mp_context=ctx
|
||||
)
|
||||
# ... rest of code ...
|
||||
finally:
|
||||
if pool:
|
||||
pool.shutdown(wait=False, cancel_futures=True)
|
||||
|
||||
# Give pool time to cleanup
|
||||
import time
|
||||
time.sleep(0.5)
|
||||
|
||||
# Then force kill any survivors
|
||||
killed = _kill_worker_processes()
|
||||
if killed > 0:
|
||||
_clog(f"Force killed {killed} orphaned workers")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Bug 2.4.2b: Dominant Path — Add Logging + Reduce Limits
|
||||
|
||||
**Problem:**
|
||||
Detailed preset still timeouts. Unknown if filter is being applied.
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
**Add diagnostic logging to verify filter works:**
|
||||
|
||||
```python
|
||||
# At the TOP of the file, add constants (if not already there):
|
||||
MAX_BUILDINGS_FOR_LINE = 50 # Reduced from 100
|
||||
MAX_BUILDINGS_FOR_REFLECTION = 30 # Reduced from 100
|
||||
MAX_DISTANCE_FROM_PATH = 300 # Reduced from 500m
|
||||
|
||||
def _filter_buildings_by_distance(buildings, tx_point, rx_point, max_count, max_distance):
|
||||
"""Filter buildings to nearest N within max_distance of path."""
|
||||
|
||||
original_count = len(buildings)
|
||||
|
||||
if original_count <= max_count:
|
||||
_log(f"[FILTER] {original_count} buildings, no filter needed")
|
||||
return buildings
|
||||
|
||||
# Calculate midpoint
|
||||
mid_lat = (tx_point[0] + rx_point[0]) / 2
|
||||
mid_lon = (tx_point[1] + rx_point[1]) / 2
|
||||
|
||||
# Calculate squared distance to midpoint (no sqrt for speed)
|
||||
def dist_sq(b):
|
||||
blat = b.get('centroid_lat') or b.get('lat', mid_lat)
|
||||
blon = b.get('centroid_lon') or b.get('lon', mid_lon)
|
||||
dlat = (blat - mid_lat) * 111000
|
||||
dlon = (blon - mid_lon) * 111000 * 0.65 # cos(50°) ≈ 0.65
|
||||
return dlat*dlat + dlon*dlon
|
||||
|
||||
# Sort by distance
|
||||
buildings_sorted = sorted(buildings, key=dist_sq)
|
||||
|
||||
# Filter by max distance
|
||||
max_dist_sq = max_distance * max_distance
|
||||
filtered = [b for b in buildings_sorted if dist_sq(b) <= max_dist_sq]
|
||||
|
||||
# Take top N
|
||||
result = filtered[:max_count]
|
||||
|
||||
_log(f"[FILTER] {original_count} → {len(result)} buildings (max_count={max_count}, max_dist={max_distance}m)")
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
**Verify the filter is CALLED in the main function:**
|
||||
|
||||
```python
|
||||
def find_dominant_path_sync(tx, rx, buildings, vegetation, spatial_idx, frequency, ...):
|
||||
"""Find dominant propagation path."""
|
||||
|
||||
# Get buildings from spatial index
|
||||
line_buildings_raw = spatial_idx.query_line(tx['lat'], tx['lon'], rx['lat'], rx['lon'])
|
||||
|
||||
# FILTER - this MUST be called
|
||||
line_buildings = _filter_buildings_by_distance(
|
||||
line_buildings_raw,
|
||||
(tx['lat'], tx['lon']),
|
||||
(rx['lat'], rx['lon']),
|
||||
max_count=MAX_BUILDINGS_FOR_LINE,
|
||||
max_distance=MAX_DISTANCE_FROM_PATH
|
||||
)
|
||||
|
||||
# Same for reflection candidates
|
||||
mid_lat = (tx['lat'] + rx['lat']) / 2
|
||||
mid_lon = (tx['lon'] + rx['lon']) / 2
|
||||
refl_buildings_raw = spatial_idx.query_point(mid_lat, mid_lon, buffer_cells=3)
|
||||
|
||||
refl_buildings = _filter_buildings_by_distance(
|
||||
refl_buildings_raw,
|
||||
(tx['lat'], tx['lon']),
|
||||
(rx['lat'], rx['lon']),
|
||||
max_count=MAX_BUILDINGS_FOR_REFLECTION,
|
||||
max_distance=MAX_DISTANCE_FROM_PATH
|
||||
)
|
||||
|
||||
# Update diagnostic log to show FILTERED counts
|
||||
if _point_counter[0] <= 3:
|
||||
print(f"[DOMINANT_PATH] Point #{_point_counter[0]}: "
|
||||
f"line_bldgs={len(line_buildings)} (was {len(line_buildings_raw)}), "
|
||||
f"refl_bldgs={len(refl_buildings)} (was {len(refl_buildings_raw)})")
|
||||
|
||||
# ... rest of function ...
|
||||
```
|
||||
|
||||
**If still too slow — option to DISABLE reflections:**
|
||||
|
||||
```python
|
||||
# Quick fix: skip reflection calculation entirely
|
||||
ENABLE_REFLECTIONS = False # Set to True when performance is fixed
|
||||
|
||||
def find_dominant_path_sync(...):
|
||||
# Direct path
|
||||
direct_loss = calculate_direct_path(...)
|
||||
|
||||
if not ENABLE_REFLECTIONS:
|
||||
return direct_loss
|
||||
|
||||
# Reflection paths (slow)
|
||||
reflection_loss = calculate_reflections(...)
|
||||
|
||||
return min(direct_loss, reflection_loss)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🟡 Bug 2.4.2c: Elevation Layer — Local Min/Max Contrast
|
||||
|
||||
**Problem:**
|
||||
All green because using absolute thresholds (100m, 150m, 200m...) but local terrain varies only 150-200m.
|
||||
|
||||
**File:** `frontend/src/components/map/ElevationLayer.tsx`
|
||||
|
||||
**Fix — use RELATIVE coloring based on local min/max:**
|
||||
|
||||
```tsx
|
||||
// Color palette (keep these)
|
||||
const COLORS = {
|
||||
DEEP_BLUE: [33, 102, 172], // Lowest
|
||||
LIGHT_BLUE: [103, 169, 207],
|
||||
GREEN: [145, 207, 96],
|
||||
YELLOW: [254, 224, 139],
|
||||
ORANGE: [252, 141, 89],
|
||||
BROWN: [215, 48, 39], // Highest
|
||||
};
|
||||
|
||||
// Interpolate between two colors
|
||||
function interpolateColor(
|
||||
color1: number[],
|
||||
color2: number[],
|
||||
t: number
|
||||
): [number, number, number] {
|
||||
return [
|
||||
Math.round(color1[0] + (color2[0] - color1[0]) * t),
|
||||
Math.round(color1[1] + (color2[1] - color1[1]) * t),
|
||||
Math.round(color1[2] + (color2[2] - color1[2]) * t),
|
||||
];
|
||||
}
|
||||
|
||||
// NEW: Get color based on NORMALIZED elevation (0-1)
|
||||
function getColorForNormalizedElevation(normalized: number): [number, number, number] {
|
||||
// Clamp to 0-1
|
||||
const n = Math.max(0, Math.min(1, normalized));
|
||||
|
||||
if (n < 0.2) {
|
||||
// 0-20%: deep blue → light blue
|
||||
return interpolateColor(COLORS.DEEP_BLUE, COLORS.LIGHT_BLUE, n / 0.2);
|
||||
} else if (n < 0.4) {
|
||||
// 20-40%: light blue → green
|
||||
return interpolateColor(COLORS.LIGHT_BLUE, COLORS.GREEN, (n - 0.2) / 0.2);
|
||||
} else if (n < 0.6) {
|
||||
// 40-60%: green → yellow
|
||||
return interpolateColor(COLORS.GREEN, COLORS.YELLOW, (n - 0.4) / 0.2);
|
||||
} else if (n < 0.8) {
|
||||
// 60-80%: yellow → orange
|
||||
return interpolateColor(COLORS.YELLOW, COLORS.ORANGE, (n - 0.6) / 0.2);
|
||||
} else {
|
||||
// 80-100%: orange → brown
|
||||
return interpolateColor(COLORS.ORANGE, COLORS.BROWN, (n - 0.8) / 0.2);
|
||||
}
|
||||
}
|
||||
|
||||
// In the main render function, USE local min/max:
|
||||
useEffect(() => {
|
||||
// ... fetch elevation data ...
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
// Get LOCAL min/max from the actual data
|
||||
const minElev = data.min_elevation; // e.g., 152
|
||||
const maxElev = data.max_elevation; // e.g., 198
|
||||
const elevRange = maxElev - minElev || 1; // Avoid division by zero
|
||||
|
||||
console.log(`[Elevation] Local range: ${minElev}m - ${maxElev}m (${elevRange}m difference)`);
|
||||
|
||||
// Fill pixel data with NORMALIZED colors
|
||||
for (let i = 0; i < data.rows; i++) {
|
||||
for (let j = 0; j < data.cols; j++) {
|
||||
const elevation = data.elevations[i][j];
|
||||
|
||||
// Normalize to 0-1 based on LOCAL range
|
||||
const normalized = (elevation - minElev) / elevRange;
|
||||
|
||||
const color = getColorForNormalizedElevation(normalized);
|
||||
|
||||
const idx = (i * data.cols + j) * 4;
|
||||
imageData.data[idx] = color[0]; // R
|
||||
imageData.data[idx + 1] = color[1]; // G
|
||||
imageData.data[idx + 2] = color[2]; // B
|
||||
imageData.data[idx + 3] = 255; // A (full opacity, layer opacity handled separately)
|
||||
}
|
||||
}
|
||||
|
||||
// ... rest of canvas/overlay code ...
|
||||
}, [enabled, opacity, bbox, map]);
|
||||
```
|
||||
|
||||
**Also add elevation legend showing LOCAL range:**
|
||||
|
||||
```tsx
|
||||
// In the parent component (App.tsx or Map.tsx), show legend:
|
||||
{showElevation && elevationRange && (
|
||||
<div className="elevation-legend">
|
||||
<div className="legend-title">Elevation</div>
|
||||
<div className="legend-gradient"></div>
|
||||
<div className="legend-labels">
|
||||
<span>{elevationRange.min}m</span>
|
||||
<span>{elevationRange.max}m</span>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
// CSS for gradient:
|
||||
.legend-gradient {
|
||||
height: 10px;
|
||||
background: linear-gradient(to right,
|
||||
#2166ac, /* deep blue - low */
|
||||
#67a9cf, /* light blue */
|
||||
#91cf60, /* green */
|
||||
#fee08b, /* yellow */
|
||||
#fc8d59, /* orange */
|
||||
#d73027 /* brown - high */
|
||||
);
|
||||
border-radius: 2px;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🟢 Enhancement 2.4.2d: GPU Install Message
|
||||
|
||||
**File:** `backend/app/services/gpu_service.py`
|
||||
|
||||
**Add clear install instructions on startup:**
|
||||
|
||||
```python
|
||||
# At module init:
|
||||
|
||||
GPU_AVAILABLE = False
|
||||
GPU_INFO = None
|
||||
|
||||
try:
|
||||
import cupy as cp
|
||||
|
||||
# Check CUDA
|
||||
device_count = cp.cuda.runtime.getDeviceCount()
|
||||
if device_count > 0:
|
||||
GPU_AVAILABLE = True
|
||||
props = cp.cuda.runtime.getDeviceProperties(0)
|
||||
GPU_INFO = {
|
||||
'name': props['name'].decode() if isinstance(props['name'], bytes) else str(props['name']),
|
||||
'memory_mb': props['totalGlobalMem'] // (1024 * 1024),
|
||||
'cuda_version': cp.cuda.runtime.runtimeGetVersion(),
|
||||
}
|
||||
print(f"[GPU] ✓ CUDA available: {GPU_INFO['name']} ({GPU_INFO['memory_mb']} MB)")
|
||||
else:
|
||||
print("[GPU] ✗ No CUDA devices found")
|
||||
|
||||
except ImportError:
|
||||
print("[GPU] ✗ CuPy not installed — using CPU/NumPy")
|
||||
print("[GPU] To enable GPU acceleration, install CuPy:")
|
||||
print("[GPU] ")
|
||||
print("[GPU] For CUDA 12.x: pip install cupy-cuda12x")
|
||||
print("[GPU] For CUDA 11.x: pip install cupy-cuda11x")
|
||||
print("[GPU] ")
|
||||
print("[GPU] Check CUDA version: nvidia-smi")
|
||||
|
||||
except Exception as e:
|
||||
print(f"[GPU] ✗ CuPy error: {e}")
|
||||
print("[GPU] GPU acceleration disabled")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `backend/app/services/parallel_coverage_service.py` | Rewrite `_kill_worker_processes()` to kill by name; add spawn context |
|
||||
| `backend/app/services/dominant_path_service.py` | Add detailed filter logging; reduce limits to 50/30; add ENABLE_REFLECTIONS flag |
|
||||
| `frontend/src/components/map/ElevationLayer.tsx` | Use local min/max for color normalization |
|
||||
| `backend/app/services/gpu_service.py` | Add clear install instructions |
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Test 1: Memory Cleanup
|
||||
```bash
|
||||
# Run Detailed preset (will timeout)
|
||||
# Watch Task Manager during and after
|
||||
# After timeout message:
|
||||
# - Should see only 1 rfcp-server.exe
|
||||
# - RAM should drop from 8GB to <500MB
|
||||
```
|
||||
|
||||
### Test 2: Dominant Path Logging
|
||||
```bash
|
||||
# Run debug mode, watch console
|
||||
# Should see:
|
||||
# [FILTER] 646 → 50 buildings (max_count=50, max_dist=300m)
|
||||
# [DOMINANT_PATH] Point #1: line_bldgs=50 (was 646), refl_bldgs=30 (was 302)
|
||||
```
|
||||
|
||||
### Test 3: Elevation Contrast
|
||||
```bash
|
||||
# Open app
|
||||
# Enable elevation layer
|
||||
# Should see color variation:
|
||||
# - Blue in valleys
|
||||
# - Green/yellow on slopes
|
||||
# - Brown/orange on hills
|
||||
# Console should show: "[Elevation] Local range: 152m - 198m"
|
||||
```
|
||||
|
||||
### Test 4: GPU Message
|
||||
```bash
|
||||
# Start server, check console
|
||||
# Should see clear message about CuPy install
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Success Criteria
|
||||
|
||||
- [ ] After timeout: only 1 rfcp-server.exe, RAM < 500MB
|
||||
- [ ] Dominant path logs show filtered counts (50 buildings, not 600)
|
||||
- [ ] Detailed preset completes in <120s OR logs explain why still slow
|
||||
- [ ] Elevation layer shows visible terrain contrast
|
||||
- [ ] GPU install instructions visible in console
|
||||
|
||||
---
|
||||
|
||||
## 📈 Expected Results After Fixes
|
||||
|
||||
| Metric | Before | After |
|
||||
|--------|--------|-------|
|
||||
| Workers after timeout | 8 (7.8GB) | 1 (<500MB) |
|
||||
| Buildings per point | 600+ | 50 |
|
||||
| Detailed time | 300s timeout | ~60-90s |
|
||||
| Elevation | All green | Color gradient |
|
||||
|
||||
---
|
||||
|
||||
## 🔜 Next Phase
|
||||
|
||||
After 2.4.2:
|
||||
- Phase 2.5: Loading screen with fun facts
|
||||
- Phase 2.5: Better error messages in UI
|
||||
- Phase 2.6: Export coverage to GeoJSON/KML
|
||||
627
docs/devlog/installer/RFCP-Phase-2.5.0-NumPy-Vectorization.md
Normal file
627
docs/devlog/installer/RFCP-Phase-2.5.0-NumPy-Vectorization.md
Normal file
@@ -0,0 +1,627 @@
|
||||
# RFCP Phase 2.5.0: NumPy Vectorization
|
||||
|
||||
**Date:** February 1, 2025
|
||||
**Type:** Performance Optimization
|
||||
**Priority:** HIGH
|
||||
**Goal:** 10-50x speedup for Detailed preset via NumPy vectorization
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Problem
|
||||
|
||||
Detailed preset: **346ms/point** — way too slow, causes 5 min timeout.
|
||||
|
||||
Root cause: Python loops in dominant_path_service.py
|
||||
```python
|
||||
for building in buildings: # 50 iterations
|
||||
for reflector in reflectors: # 30 iterations
|
||||
# Math operations...
|
||||
```
|
||||
|
||||
**1500 Python loop iterations** with function calls = slow.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Solution: NumPy Vectorization
|
||||
|
||||
Replace Python loops with NumPy batch operations:
|
||||
- **Before:** 1500 function calls, 1500 loop iterations
|
||||
- **After:** ~10 function calls, 0 Python loops
|
||||
- **Expected speedup:** 10-50x
|
||||
|
||||
---
|
||||
|
||||
## 📁 Files to Create/Modify
|
||||
|
||||
### NEW: `backend/app/services/geometry_vectorized.py`
|
||||
|
||||
Core vectorized geometry functions:
|
||||
|
||||
```python
|
||||
"""
|
||||
Vectorized geometry operations using NumPy.
|
||||
All functions operate on arrays, not single values.
|
||||
"""
|
||||
import numpy as np
|
||||
from typing import Tuple, Optional
|
||||
|
||||
# Earth radius in meters
|
||||
EARTH_RADIUS = 6371000
|
||||
|
||||
|
||||
def haversine_batch(
|
||||
lat1: float, lon1: float,
|
||||
lats2: np.ndarray, lons2: np.ndarray
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Calculate distances from one point to many points.
|
||||
|
||||
Args:
|
||||
lat1, lon1: Single origin point (degrees)
|
||||
lats2, lons2: Arrays of destination points (degrees), shape (N,)
|
||||
|
||||
Returns:
|
||||
distances: Array of distances in meters, shape (N,)
|
||||
"""
|
||||
lat1_rad = np.radians(lat1)
|
||||
lon1_rad = np.radians(lon1)
|
||||
lats2_rad = np.radians(lats2)
|
||||
lons2_rad = np.radians(lons2)
|
||||
|
||||
dlat = lats2_rad - lat1_rad
|
||||
dlon = lons2_rad - lon1_rad
|
||||
|
||||
a = np.sin(dlat / 2) ** 2 + np.cos(lat1_rad) * np.cos(lats2_rad) * np.sin(dlon / 2) ** 2
|
||||
c = 2 * np.arcsin(np.sqrt(a))
|
||||
|
||||
return EARTH_RADIUS * c
|
||||
|
||||
|
||||
def haversine_matrix(
|
||||
lats1: np.ndarray, lons1: np.ndarray,
|
||||
lats2: np.ndarray, lons2: np.ndarray
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Calculate distances between all pairs of points (M×N matrix).
|
||||
|
||||
Args:
|
||||
lats1, lons1: First set of points, shape (M,)
|
||||
lats2, lons2: Second set of points, shape (N,)
|
||||
|
||||
Returns:
|
||||
distances: Matrix of distances, shape (M, N)
|
||||
"""
|
||||
# Reshape for broadcasting: (M, 1) and (1, N)
|
||||
lats1_rad = np.radians(lats1[:, np.newaxis])
|
||||
lons1_rad = np.radians(lons1[:, np.newaxis])
|
||||
lats2_rad = np.radians(lats2[np.newaxis, :])
|
||||
lons2_rad = np.radians(lons2[np.newaxis, :])
|
||||
|
||||
dlat = lats2_rad - lats1_rad
|
||||
dlon = lons2_rad - lons1_rad
|
||||
|
||||
a = np.sin(dlat / 2) ** 2 + np.cos(lats1_rad) * np.cos(lats2_rad) * np.sin(dlon / 2) ** 2
|
||||
c = 2 * np.arcsin(np.sqrt(a))
|
||||
|
||||
return EARTH_RADIUS * c
|
||||
|
||||
|
||||
def points_to_local_coords(
|
||||
ref_lat: float, ref_lon: float,
|
||||
lats: np.ndarray, lons: np.ndarray
|
||||
) -> Tuple[np.ndarray, np.ndarray]:
|
||||
"""
|
||||
Convert lat/lon to local X/Y coordinates (meters from reference).
|
||||
Uses simple equirectangular projection (good for small areas).
|
||||
|
||||
Args:
|
||||
ref_lat, ref_lon: Reference point (degrees)
|
||||
lats, lons: Points to convert, shape (N,)
|
||||
|
||||
Returns:
|
||||
x, y: Local coordinates in meters, shape (N,)
|
||||
"""
|
||||
cos_lat = np.cos(np.radians(ref_lat))
|
||||
|
||||
x = (lons - ref_lon) * 111320 * cos_lat
|
||||
y = (lats - ref_lat) * 110540
|
||||
|
||||
return x, y
|
||||
|
||||
|
||||
def line_segments_intersect_batch(
|
||||
p1: np.ndarray, p2: np.ndarray,
|
||||
segments_start: np.ndarray, segments_end: np.ndarray
|
||||
) -> Tuple[np.ndarray, np.ndarray]:
|
||||
"""
|
||||
Check if line segment p1→p2 intersects with multiple segments.
|
||||
Uses vectorized cross-product method.
|
||||
|
||||
Args:
|
||||
p1: Start point (2,) — [x, y]
|
||||
p2: End point (2,) — [x, y]
|
||||
segments_start: Segment start points (N, 2)
|
||||
segments_end: Segment end points (N, 2)
|
||||
|
||||
Returns:
|
||||
intersects: Boolean array (N,) — True if intersects
|
||||
t_values: Parameter values (N,) — where along p1→p2 intersection occurs
|
||||
"""
|
||||
d = p2 - p1 # Direction of main line
|
||||
|
||||
# Segment directions
|
||||
seg_d = segments_end - segments_start # (N, 2)
|
||||
|
||||
# Cross product for parallel check
|
||||
cross = d[0] * seg_d[:, 1] - d[1] * seg_d[:, 0] # (N,)
|
||||
|
||||
# Avoid division by zero
|
||||
parallel_mask = np.abs(cross) < 1e-10
|
||||
cross_safe = np.where(parallel_mask, 1.0, cross)
|
||||
|
||||
# Vector from segment start to p1
|
||||
dp = p1 - segments_start # (N, 2)
|
||||
|
||||
# Calculate t (parameter on main line) and u (parameter on segments)
|
||||
t = (dp[:, 0] * seg_d[:, 1] - dp[:, 1] * seg_d[:, 0]) / cross_safe
|
||||
u = (dp[:, 0] * d[1] - dp[:, 1] * d[0]) / cross_safe
|
||||
|
||||
# Intersection if 0 <= t <= 1 and 0 <= u <= 1
|
||||
intersects = ~parallel_mask & (t >= 0) & (t <= 1) & (u >= 0) & (u <= 1)
|
||||
|
||||
return intersects, t
|
||||
|
||||
|
||||
def line_intersects_polygons_batch(
|
||||
p1: np.ndarray, p2: np.ndarray,
|
||||
polygons_x: np.ndarray, polygons_y: np.ndarray,
|
||||
polygon_lengths: np.ndarray
|
||||
) -> Tuple[np.ndarray, np.ndarray]:
|
||||
"""
|
||||
Check if line p1→p2 intersects multiple polygons.
|
||||
|
||||
Args:
|
||||
p1: Start point (2,) — [x, y]
|
||||
p2: End point (2,) — [x, y]
|
||||
polygons_x: Flattened polygon X coords (total_vertices,)
|
||||
polygons_y: Flattened polygon Y coords (total_vertices,)
|
||||
polygon_lengths: Number of vertices per polygon (num_polygons,)
|
||||
|
||||
Returns:
|
||||
intersects: Boolean array (num_polygons,)
|
||||
min_distances: Distance to first intersection (num_polygons,)
|
||||
"""
|
||||
num_polygons = len(polygon_lengths)
|
||||
intersects = np.zeros(num_polygons, dtype=bool)
|
||||
min_t = np.ones(num_polygons) * np.inf
|
||||
|
||||
# Process each polygon
|
||||
idx = 0
|
||||
for i, length in enumerate(polygon_lengths):
|
||||
if length < 3:
|
||||
idx += length
|
||||
continue
|
||||
|
||||
# Get polygon vertices
|
||||
px = polygons_x[idx:idx + length]
|
||||
py = polygons_y[idx:idx + length]
|
||||
|
||||
# Create edge segments (including closing edge)
|
||||
starts = np.stack([px, py], axis=1) # (length, 2)
|
||||
ends = np.stack([np.roll(px, -1), np.roll(py, -1)], axis=1) # (length, 2)
|
||||
|
||||
# Check intersections with all edges
|
||||
edge_intersects, t_vals = line_segments_intersect_batch(p1, p2, starts, ends)
|
||||
|
||||
if np.any(edge_intersects):
|
||||
intersects[i] = True
|
||||
min_t[i] = np.min(t_vals[edge_intersects])
|
||||
|
||||
idx += length
|
||||
|
||||
# Convert t to distance
|
||||
line_length = np.linalg.norm(p2 - p1)
|
||||
min_distances = min_t * line_length
|
||||
|
||||
return intersects, min_distances
|
||||
|
||||
|
||||
def calculate_reflection_points_batch(
|
||||
tx: np.ndarray,
|
||||
rx: np.ndarray,
|
||||
wall_starts: np.ndarray,
|
||||
wall_ends: np.ndarray
|
||||
) -> Tuple[np.ndarray, np.ndarray]:
|
||||
"""
|
||||
Calculate reflection points on multiple walls.
|
||||
Uses mirror image method.
|
||||
|
||||
Args:
|
||||
tx: Transmitter position (2,) — [x, y]
|
||||
rx: Receiver position (2,) — [x, y]
|
||||
wall_starts: Wall start points (N, 2)
|
||||
wall_ends: Wall end points (N, 2)
|
||||
|
||||
Returns:
|
||||
reflection_points: Reflection point on each wall (N, 2)
|
||||
valid: Boolean mask for valid reflections (N,)
|
||||
"""
|
||||
# Wall vectors and normals
|
||||
wall_vec = wall_ends - wall_starts # (N, 2)
|
||||
wall_length = np.linalg.norm(wall_vec, axis=1, keepdims=True)
|
||||
wall_unit = wall_vec / np.maximum(wall_length, 1e-10) # (N, 2)
|
||||
|
||||
# Normal vectors (perpendicular to wall)
|
||||
normals = np.stack([-wall_unit[:, 1], wall_unit[:, 0]], axis=1) # (N, 2)
|
||||
|
||||
# Mirror TX across each wall
|
||||
tx_to_wall = tx - wall_starts # (N, 2)
|
||||
tx_dist_to_wall = np.sum(tx_to_wall * normals, axis=1, keepdims=True) # (N, 1)
|
||||
tx_mirror = tx - 2 * tx_dist_to_wall * normals # (N, 2)
|
||||
|
||||
# Find intersection of rx→tx_mirror with wall
|
||||
# Parametric: wall_start + t * wall_vec = rx + s * (tx_mirror - rx)
|
||||
rx_to_mirror = tx_mirror - rx # (N, 2)
|
||||
|
||||
# Solve for t (position along wall)
|
||||
cross_denom = (rx_to_mirror[:, 0] * wall_vec[:, 1] -
|
||||
rx_to_mirror[:, 1] * wall_vec[:, 0])
|
||||
|
||||
# Avoid division by zero
|
||||
valid_denom = np.abs(cross_denom) > 1e-10
|
||||
cross_denom_safe = np.where(valid_denom, cross_denom, 1.0)
|
||||
|
||||
rx_to_start = wall_starts - rx # (N, 2)
|
||||
t = (rx_to_start[:, 0] * rx_to_mirror[:, 1] -
|
||||
rx_to_start[:, 1] * rx_to_mirror[:, 0]) / cross_denom_safe
|
||||
|
||||
# Reflection point
|
||||
reflection_points = wall_starts + t[:, np.newaxis] * wall_vec
|
||||
|
||||
# Valid if t in [0, 1] and TX is on correct side of wall
|
||||
valid = valid_denom & (t >= 0) & (t <= 1) & (tx_dist_to_wall[:, 0] > 0)
|
||||
|
||||
return reflection_points, valid
|
||||
|
||||
|
||||
def find_best_reflection_path_vectorized(
|
||||
tx: np.ndarray,
|
||||
rx: np.ndarray,
|
||||
building_walls_start: np.ndarray,
|
||||
building_walls_end: np.ndarray,
|
||||
wall_to_building: np.ndarray,
|
||||
obstacle_polygons_x: np.ndarray,
|
||||
obstacle_polygons_y: np.ndarray,
|
||||
obstacle_lengths: np.ndarray,
|
||||
max_candidates: int = 50
|
||||
) -> Tuple[Optional[np.ndarray], float, float]:
|
||||
"""
|
||||
Find best single-reflection path using vectorized operations.
|
||||
|
||||
Args:
|
||||
tx: Transmitter [x, y]
|
||||
rx: Receiver [x, y]
|
||||
building_walls_start: All wall start points (W, 2)
|
||||
building_walls_end: All wall end points (W, 2)
|
||||
wall_to_building: Mapping wall index → building index (W,)
|
||||
obstacle_*: Obstacle polygons for LOS checks
|
||||
max_candidates: Max reflection candidates to evaluate
|
||||
|
||||
Returns:
|
||||
best_reflection_point: [x, y] or None
|
||||
best_path_length: Total path length
|
||||
best_reflection_loss: dB loss from reflection
|
||||
"""
|
||||
num_walls = len(building_walls_start)
|
||||
|
||||
if num_walls == 0:
|
||||
return None, np.inf, 0.0
|
||||
|
||||
# Step 1: Calculate all reflection points at once
|
||||
refl_points, valid = calculate_reflection_points_batch(
|
||||
tx, rx, building_walls_start, building_walls_end
|
||||
)
|
||||
|
||||
if not np.any(valid):
|
||||
return None, np.inf, 0.0
|
||||
|
||||
# Step 2: Calculate path lengths for valid reflections
|
||||
valid_indices = np.where(valid)[0]
|
||||
valid_refl = refl_points[valid] # (V, 2)
|
||||
|
||||
# TX → reflection distances
|
||||
tx_to_refl = np.linalg.norm(valid_refl - tx, axis=1) # (V,)
|
||||
|
||||
# Reflection → RX distances
|
||||
refl_to_rx = np.linalg.norm(rx - valid_refl, axis=1) # (V,)
|
||||
|
||||
# Total path lengths
|
||||
path_lengths = tx_to_refl + refl_to_rx # (V,)
|
||||
|
||||
# Step 3: Sort by path length, take top candidates
|
||||
if len(valid_indices) > max_candidates:
|
||||
top_indices = np.argpartition(path_lengths, max_candidates)[:max_candidates]
|
||||
valid_indices = valid_indices[top_indices]
|
||||
valid_refl = valid_refl[top_indices]
|
||||
path_lengths = path_lengths[top_indices]
|
||||
tx_to_refl = tx_to_refl[top_indices]
|
||||
refl_to_rx = refl_to_rx[top_indices]
|
||||
|
||||
# Step 4: Check LOS for each candidate (this is still a loop, but limited)
|
||||
best_idx = -1
|
||||
best_length = np.inf
|
||||
|
||||
for i, (refl_pt, length) in enumerate(zip(valid_refl, path_lengths)):
|
||||
# Skip if already longer than best
|
||||
if length >= best_length:
|
||||
continue
|
||||
|
||||
# Check TX → reflection LOS
|
||||
intersects1, _ = line_intersects_polygons_batch(
|
||||
tx, refl_pt, obstacle_polygons_x, obstacle_polygons_y, obstacle_lengths
|
||||
)
|
||||
|
||||
if np.any(intersects1):
|
||||
continue
|
||||
|
||||
# Check reflection → RX LOS
|
||||
intersects2, _ = line_intersects_polygons_batch(
|
||||
refl_pt, rx, obstacle_polygons_x, obstacle_polygons_y, obstacle_lengths
|
||||
)
|
||||
|
||||
if np.any(intersects2):
|
||||
continue
|
||||
|
||||
# Valid path found
|
||||
best_idx = i
|
||||
best_length = length
|
||||
|
||||
if best_idx < 0:
|
||||
return None, np.inf, 0.0
|
||||
|
||||
best_point = valid_refl[best_idx]
|
||||
|
||||
# Reflection loss (simplified: 3-10 dB depending on angle)
|
||||
# More grazing angle = more loss
|
||||
direct_dist = np.linalg.norm(rx - tx)
|
||||
path_ratio = best_length / max(direct_dist, 1.0)
|
||||
reflection_loss = 3.0 + 7.0 * min(1.0, (path_ratio - 1.0) * 2)
|
||||
|
||||
return best_point, best_length, reflection_loss
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### MODIFY: `backend/app/services/dominant_path_service.py`
|
||||
|
||||
Replace loop-based calculations with vectorized versions:
|
||||
|
||||
```python
|
||||
# Add imports at top:
|
||||
from .geometry_vectorized import (
|
||||
haversine_batch,
|
||||
points_to_local_coords,
|
||||
line_intersects_polygons_batch,
|
||||
find_best_reflection_path_vectorized
|
||||
)
|
||||
|
||||
# Add helper to convert buildings to numpy arrays:
|
||||
def _buildings_to_arrays(buildings: list, ref_lat: float, ref_lon: float):
|
||||
"""Convert building list to numpy arrays for vectorized ops."""
|
||||
if not buildings:
|
||||
return None, None, None, None, None
|
||||
|
||||
# Extract centroids
|
||||
lats = np.array([b.get('centroid_lat', b.get('lat', 0)) for b in buildings])
|
||||
lons = np.array([b.get('centroid_lon', b.get('lon', 0)) for b in buildings])
|
||||
|
||||
# Convert to local coords
|
||||
x, y = points_to_local_coords(ref_lat, ref_lon, lats, lons)
|
||||
|
||||
# Extract all walls (polygon edges)
|
||||
all_walls_start = []
|
||||
all_walls_end = []
|
||||
wall_to_building = []
|
||||
|
||||
# Flatten polygons for intersection tests
|
||||
all_poly_x = []
|
||||
all_poly_y = []
|
||||
poly_lengths = []
|
||||
|
||||
for i, b in enumerate(buildings):
|
||||
coords = b.get('geometry', {}).get('coordinates', [[]])[0]
|
||||
if len(coords) < 3:
|
||||
poly_lengths.append(0)
|
||||
continue
|
||||
|
||||
# Convert polygon to local coords
|
||||
poly_lats = np.array([c[1] for c in coords])
|
||||
poly_lons = np.array([c[0] for c in coords])
|
||||
px, py = points_to_local_coords(ref_lat, ref_lon, poly_lats, poly_lons)
|
||||
|
||||
all_poly_x.extend(px)
|
||||
all_poly_y.extend(py)
|
||||
poly_lengths.append(len(coords))
|
||||
|
||||
# Extract walls
|
||||
for j in range(len(coords) - 1):
|
||||
all_walls_start.append([px[j], py[j]])
|
||||
all_walls_end.append([px[j+1], py[j+1]])
|
||||
wall_to_building.append(i)
|
||||
|
||||
return (
|
||||
np.array(all_walls_start) if all_walls_start else np.zeros((0, 2)),
|
||||
np.array(all_walls_end) if all_walls_end else np.zeros((0, 2)),
|
||||
np.array(wall_to_building) if wall_to_building else np.zeros(0, dtype=int),
|
||||
np.array(all_poly_x),
|
||||
np.array(all_poly_y),
|
||||
np.array(poly_lengths)
|
||||
)
|
||||
|
||||
|
||||
# Update main function to use vectorized operations:
|
||||
def find_dominant_path_vectorized(
|
||||
tx_lat: float, tx_lon: float,
|
||||
rx_lat: float, rx_lon: float,
|
||||
buildings: list,
|
||||
frequency_mhz: float = 1800
|
||||
) -> dict:
|
||||
"""
|
||||
Find dominant propagation path using vectorized NumPy operations.
|
||||
|
||||
Returns dict with:
|
||||
- has_los: bool
|
||||
- path_type: 'direct' | 'reflection' | 'diffraction'
|
||||
- total_loss: float (dB)
|
||||
- reflection_point: [lat, lon] or None
|
||||
"""
|
||||
# Reference point for local coords (midpoint)
|
||||
ref_lat = (tx_lat + rx_lat) / 2
|
||||
ref_lon = (tx_lon + rx_lon) / 2
|
||||
|
||||
# Convert TX/RX to local coords
|
||||
tx_x, tx_y = points_to_local_coords(ref_lat, ref_lon,
|
||||
np.array([tx_lat]), np.array([tx_lon]))
|
||||
rx_x, rx_y = points_to_local_coords(ref_lat, ref_lon,
|
||||
np.array([rx_lat]), np.array([rx_lon]))
|
||||
tx = np.array([tx_x[0], tx_y[0]])
|
||||
rx = np.array([rx_x[0], rx_y[0]])
|
||||
|
||||
# Convert buildings to arrays
|
||||
(walls_start, walls_end, wall_to_bldg,
|
||||
poly_x, poly_y, poly_lengths) = _buildings_to_arrays(buildings, ref_lat, ref_lon)
|
||||
|
||||
direct_dist = np.linalg.norm(rx - tx)
|
||||
|
||||
# Step 1: Check direct LOS
|
||||
if len(poly_lengths) == 0:
|
||||
# No buildings, direct LOS
|
||||
return {
|
||||
'has_los': True,
|
||||
'path_type': 'direct',
|
||||
'total_loss': 0.0,
|
||||
'reflection_point': None,
|
||||
'path_length': direct_dist
|
||||
}
|
||||
|
||||
intersects, _ = line_intersects_polygons_batch(tx, rx, poly_x, poly_y, poly_lengths)
|
||||
|
||||
if not np.any(intersects):
|
||||
# Direct LOS exists
|
||||
return {
|
||||
'has_los': True,
|
||||
'path_type': 'direct',
|
||||
'total_loss': 0.0,
|
||||
'reflection_point': None,
|
||||
'path_length': direct_dist
|
||||
}
|
||||
|
||||
# Step 2: Find best reflection path
|
||||
refl_point, refl_length, refl_loss = find_best_reflection_path_vectorized(
|
||||
tx, rx, walls_start, walls_end, wall_to_bldg,
|
||||
poly_x, poly_y, poly_lengths,
|
||||
max_candidates=50
|
||||
)
|
||||
|
||||
if refl_point is not None:
|
||||
# Convert reflection point back to lat/lon
|
||||
refl_lat = ref_lat + refl_point[1] / 110540
|
||||
refl_lon = ref_lon + refl_point[0] / (111320 * np.cos(np.radians(ref_lat)))
|
||||
|
||||
return {
|
||||
'has_los': False,
|
||||
'path_type': 'reflection',
|
||||
'total_loss': refl_loss,
|
||||
'reflection_point': [refl_lat, refl_lon],
|
||||
'path_length': refl_length
|
||||
}
|
||||
|
||||
# Step 3: Fallback to diffraction (simplified)
|
||||
# Count blocking buildings for diffraction loss estimate
|
||||
num_blocking = np.sum(intersects)
|
||||
diffraction_loss = 10 + 5 * min(num_blocking, 5) # 10-35 dB
|
||||
|
||||
return {
|
||||
'has_los': False,
|
||||
'path_type': 'diffraction',
|
||||
'total_loss': diffraction_loss,
|
||||
'reflection_point': None,
|
||||
'path_length': direct_dist # Approximate
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### MODIFY: `backend/app/services/coverage_service.py`
|
||||
|
||||
Update `_calculate_point_sync()` to use vectorized dominant path:
|
||||
|
||||
```python
|
||||
# In _calculate_point_sync(), replace dominant_path call:
|
||||
|
||||
if use_dominant_path and buildings:
|
||||
from .dominant_path_service import find_dominant_path_vectorized
|
||||
|
||||
dominant = find_dominant_path_vectorized(
|
||||
tx_lat=site['lat'],
|
||||
tx_lon=site['lon'],
|
||||
rx_lat=point_lat,
|
||||
rx_lon=point_lon,
|
||||
buildings=buildings,
|
||||
frequency_mhz=site.get('frequency', 1800)
|
||||
)
|
||||
|
||||
if dominant['path_type'] == 'reflection':
|
||||
reflection_gain = max(0, 10 - dominant['total_loss']) # Convert loss to gain
|
||||
building_loss = 0 # Reflection path avoids buildings
|
||||
elif dominant['path_type'] == 'diffraction':
|
||||
building_loss = dominant['total_loss']
|
||||
reflection_gain = 0
|
||||
else:
|
||||
# Direct LOS
|
||||
building_loss = 0
|
||||
reflection_gain = 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
```bash
|
||||
# Run test script
|
||||
.\test-coverage.bat
|
||||
|
||||
# Expected results:
|
||||
# Fast: ~0.03s (unchanged)
|
||||
# Standard: ~35-40s (unchanged)
|
||||
# Detailed: ~30-60s (was 300s timeout!) ← 5-10x faster
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Success Criteria
|
||||
|
||||
| Metric | Before | After |
|
||||
|--------|--------|-------|
|
||||
| Detailed time | 300s (timeout) | <90s |
|
||||
| Detailed ms/point | 346ms | <50ms |
|
||||
| Memory peak | ~7GB | ~3-4GB |
|
||||
| Accuracy | Baseline | Similar ±2dB |
|
||||
|
||||
---
|
||||
|
||||
## 📝 Notes
|
||||
|
||||
1. **LOS checks still have a small loop** — but limited to top 50 candidates sorted by path length
|
||||
2. **Building→array conversion** happens once per calculation, not per point
|
||||
3. **Local coordinate system** avoids expensive lat/lon math in inner loops
|
||||
4. **Reflection model simplified** — uses path ratio for loss estimate
|
||||
|
||||
---
|
||||
|
||||
## 🔜 Future Optimizations
|
||||
|
||||
If still slow after vectorization:
|
||||
- Cache building arrays (don't reconvert every point)
|
||||
- Use scipy.spatial.cKDTree for spatial queries
|
||||
- GPU acceleration with CuPy (drop-in NumPy replacement)
|
||||
577
docs/devlog/installer/RFCP-Phase-2.5.1-Performance-AppClose.md
Normal file
577
docs/devlog/installer/RFCP-Phase-2.5.1-Performance-AppClose.md
Normal file
@@ -0,0 +1,577 @@
|
||||
# RFCP Phase 2.5.1: Performance Tuning + App Close Fix
|
||||
|
||||
**Date:** February 1, 2025
|
||||
**Type:** Performance + Bug Fix
|
||||
**Priority:** HIGH
|
||||
**Depends on:** Phase 2.5.0
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Summary
|
||||
|
||||
Phase 2.5.0 vectorization працює, але bottleneck залишився:
|
||||
- `line_intersects_polygons_batch()` має внутрішній loop по polygons
|
||||
- 351-458 walls перевіряються для кожної точки
|
||||
- Результат: 329ms/point (майже без змін)
|
||||
|
||||
Також: App close ДОСІ не працює — хрестик не закриває backend.
|
||||
|
||||
---
|
||||
|
||||
## 📊 Current Performance Analysis
|
||||
|
||||
```
|
||||
[DOMINANT_PATH_VEC] Point #1: buildings=50, walls=351
|
||||
[DOMINANT_PATH_VEC] Point #2: buildings=50, walls=439
|
||||
[DOMINANT_PATH_VEC] Point #3: buildings=50, walls=458
|
||||
|
||||
Result: 329ms/point — NOT improved!
|
||||
```
|
||||
|
||||
**Root cause:** LOS check loop inside vectorized function:
|
||||
```python
|
||||
# geometry_vectorized.py line ~85
|
||||
for i, length in enumerate(polygon_lengths): # ← THIS LOOP
|
||||
# Check each polygon...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Fix 2.5.1a: Limit Walls for Reflection Check
|
||||
|
||||
**File:** `backend/app/services/geometry_vectorized.py`
|
||||
|
||||
**In `find_best_reflection_path_vectorized()`:**
|
||||
|
||||
```python
|
||||
def find_best_reflection_path_vectorized(
|
||||
tx: np.ndarray,
|
||||
rx: np.ndarray,
|
||||
building_walls_start: np.ndarray,
|
||||
building_walls_end: np.ndarray,
|
||||
wall_to_building: np.ndarray,
|
||||
obstacle_polygons_x: np.ndarray,
|
||||
obstacle_polygons_y: np.ndarray,
|
||||
obstacle_lengths: np.ndarray,
|
||||
max_candidates: int = 50,
|
||||
max_walls: int = 100, # ← ADD THIS
|
||||
max_los_checks: int = 10 # ← ADD THIS
|
||||
) -> Tuple[Optional[np.ndarray], float, float]:
|
||||
"""
|
||||
Find best single-reflection path using vectorized operations.
|
||||
|
||||
OPTIMIZATION: Limit walls and LOS checks for speed.
|
||||
- max_walls: Only consider closest N walls for reflection
|
||||
- max_los_checks: Only verify LOS for top N shortest paths
|
||||
"""
|
||||
num_walls = len(building_walls_start)
|
||||
|
||||
if num_walls == 0:
|
||||
return None, np.inf, 0.0
|
||||
|
||||
# === OPTIMIZATION 1: Limit walls by distance to path midpoint ===
|
||||
if num_walls > max_walls:
|
||||
# Calculate midpoint of TX-RX path
|
||||
midpoint = (tx + rx) / 2
|
||||
|
||||
# Calculate wall midpoints
|
||||
wall_midpoints = (building_walls_start + building_walls_end) / 2
|
||||
|
||||
# Distance from each wall to path midpoint
|
||||
wall_distances = np.linalg.norm(wall_midpoints - midpoint, axis=1)
|
||||
|
||||
# Take closest walls
|
||||
closest_indices = np.argpartition(wall_distances, max_walls)[:max_walls]
|
||||
|
||||
building_walls_start = building_walls_start[closest_indices]
|
||||
building_walls_end = building_walls_end[closest_indices]
|
||||
wall_to_building = wall_to_building[closest_indices]
|
||||
|
||||
# Log reduction (first 3 points only)
|
||||
# print(f"[WALLS] Reduced {num_walls} → {max_walls} walls")
|
||||
|
||||
# Step 1: Calculate all reflection points at once
|
||||
refl_points, valid = calculate_reflection_points_batch(
|
||||
tx, rx, building_walls_start, building_walls_end
|
||||
)
|
||||
|
||||
if not np.any(valid):
|
||||
return None, np.inf, 0.0
|
||||
|
||||
# Step 2: Calculate path lengths for valid reflections
|
||||
valid_indices = np.where(valid)[0]
|
||||
valid_refl = refl_points[valid]
|
||||
|
||||
tx_to_refl = np.linalg.norm(valid_refl - tx, axis=1)
|
||||
refl_to_rx = np.linalg.norm(rx - valid_refl, axis=1)
|
||||
path_lengths = tx_to_refl + refl_to_rx
|
||||
|
||||
# === OPTIMIZATION 2: Sort by path length, check LOS only for top N ===
|
||||
# Sort indices by path length (shortest first)
|
||||
sorted_order = np.argsort(path_lengths)
|
||||
|
||||
# Limit to max_los_checks
|
||||
check_count = min(len(sorted_order), max_los_checks)
|
||||
|
||||
# Step 3: Check LOS only for top candidates
|
||||
best_idx = -1
|
||||
best_length = np.inf
|
||||
|
||||
for i in range(check_count):
|
||||
idx = sorted_order[i]
|
||||
refl_pt = valid_refl[idx]
|
||||
length = path_lengths[idx]
|
||||
|
||||
# Skip if already longer than best found
|
||||
if length >= best_length:
|
||||
continue
|
||||
|
||||
# Check TX → reflection LOS
|
||||
intersects1, _ = line_intersects_polygons_batch(
|
||||
tx, refl_pt, obstacle_polygons_x, obstacle_polygons_y, obstacle_lengths
|
||||
)
|
||||
|
||||
if np.any(intersects1):
|
||||
continue
|
||||
|
||||
# Check reflection → RX LOS
|
||||
intersects2, _ = line_intersects_polygons_batch(
|
||||
refl_pt, rx, obstacle_polygons_x, obstacle_polygons_y, obstacle_lengths
|
||||
)
|
||||
|
||||
if np.any(intersects2):
|
||||
continue
|
||||
|
||||
# Valid path found!
|
||||
best_idx = idx
|
||||
best_length = length
|
||||
break # ← EARLY EXIT: First valid shortest path is best
|
||||
|
||||
if best_idx < 0:
|
||||
return None, np.inf, 0.0
|
||||
|
||||
best_point = valid_refl[best_idx]
|
||||
|
||||
# Reflection loss calculation
|
||||
direct_dist = np.linalg.norm(rx - tx)
|
||||
path_ratio = best_length / max(direct_dist, 1.0)
|
||||
reflection_loss = 3.0 + 7.0 * min(1.0, (path_ratio - 1.0) * 2)
|
||||
|
||||
return best_point, best_length, reflection_loss
|
||||
```
|
||||
|
||||
**Key optimizations:**
|
||||
1. **max_walls=100**: Only consider 100 closest walls (was 351-458)
|
||||
2. **max_los_checks=10**: Only verify LOS for 10 shortest paths (was 50)
|
||||
3. **Early exit**: Stop when first valid path found (sorted by length)
|
||||
|
||||
**Expected speedup:** 5-10x for reflection calculation
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Fix 2.5.1b: Simplify LOS Check for Obstacles
|
||||
|
||||
**File:** `backend/app/services/geometry_vectorized.py`
|
||||
|
||||
**Optimize `line_intersects_polygons_batch()` — limit polygons:**
|
||||
|
||||
```python
|
||||
def line_intersects_polygons_batch(
|
||||
p1: np.ndarray, p2: np.ndarray,
|
||||
polygons_x: np.ndarray, polygons_y: np.ndarray,
|
||||
polygon_lengths: np.ndarray,
|
||||
max_polygons: int = 30 # ← ADD LIMIT
|
||||
) -> Tuple[np.ndarray, np.ndarray]:
|
||||
"""
|
||||
Check if line p1→p2 intersects multiple polygons.
|
||||
|
||||
OPTIMIZATION: Only check nearest polygons to the line.
|
||||
"""
|
||||
num_polygons = len(polygon_lengths)
|
||||
|
||||
if num_polygons == 0:
|
||||
return np.array([], dtype=bool), np.array([])
|
||||
|
||||
# === OPTIMIZATION: Filter to nearby polygons first ===
|
||||
if num_polygons > max_polygons:
|
||||
# Quick filter: bounding box check
|
||||
line_min_x = min(p1[0], p2[0]) - 50 # 50m buffer
|
||||
line_max_x = max(p1[0], p2[0]) + 50
|
||||
line_min_y = min(p1[1], p2[1]) - 50
|
||||
line_max_y = max(p1[1], p2[1]) + 50
|
||||
|
||||
# Calculate polygon centroids quickly
|
||||
# This is approximate but fast
|
||||
idx = 0
|
||||
nearby_mask = np.zeros(num_polygons, dtype=bool)
|
||||
|
||||
for i, length in enumerate(polygon_lengths):
|
||||
if length < 3:
|
||||
idx += length
|
||||
continue
|
||||
|
||||
# Polygon centroid (approximate: first vertex)
|
||||
px = polygons_x[idx]
|
||||
py = polygons_y[idx]
|
||||
|
||||
# Check if centroid is near the line bounding box
|
||||
if (line_min_x <= px <= line_max_x and
|
||||
line_min_y <= py <= line_max_y):
|
||||
nearby_mask[i] = True
|
||||
|
||||
idx += length
|
||||
|
||||
# If still too many, take first max_polygons that are nearby
|
||||
nearby_indices = np.where(nearby_mask)[0]
|
||||
if len(nearby_indices) > max_polygons:
|
||||
nearby_indices = nearby_indices[:max_polygons]
|
||||
nearby_mask = np.zeros(num_polygons, dtype=bool)
|
||||
nearby_mask[nearby_indices] = True
|
||||
else:
|
||||
nearby_mask = np.ones(num_polygons, dtype=bool)
|
||||
|
||||
# Now check only nearby polygons
|
||||
intersects = np.zeros(num_polygons, dtype=bool)
|
||||
min_t = np.ones(num_polygons) * np.inf
|
||||
|
||||
idx = 0
|
||||
for i, length in enumerate(polygon_lengths):
|
||||
if length < 3 or not nearby_mask[i]:
|
||||
idx += length
|
||||
continue
|
||||
|
||||
# Get polygon vertices
|
||||
px = polygons_x[idx:idx + length]
|
||||
py = polygons_y[idx:idx + length]
|
||||
|
||||
# Create edge segments
|
||||
starts = np.stack([px, py], axis=1)
|
||||
ends = np.stack([np.roll(px, -1), np.roll(py, -1)], axis=1)
|
||||
|
||||
# Check intersections
|
||||
edge_intersects, t_vals = line_segments_intersect_batch(p1, p2, starts, ends)
|
||||
|
||||
if np.any(edge_intersects):
|
||||
intersects[i] = True
|
||||
min_t[i] = np.min(t_vals[edge_intersects])
|
||||
|
||||
idx += length
|
||||
|
||||
line_length = np.linalg.norm(p2 - p1)
|
||||
min_distances = min_t * line_length
|
||||
|
||||
return intersects, min_distances
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Fix 2.5.1c: Update Constants in dominant_path_service.py
|
||||
|
||||
**File:** `backend/app/services/dominant_path_service.py`
|
||||
|
||||
```python
|
||||
# At top of file, update constants:
|
||||
MAX_BUILDINGS_FOR_LINE = 30 # was 50
|
||||
MAX_BUILDINGS_FOR_REFLECTION = 20 # was 30
|
||||
MAX_DISTANCE_FROM_PATH = 200 # was 300m
|
||||
|
||||
# In find_dominant_paths_vectorized(), add parameters:
|
||||
result = find_best_reflection_path_vectorized(
|
||||
tx_local, rx_local,
|
||||
walls_start, walls_end, wall_to_bldg,
|
||||
poly_x, poly_y, poly_lengths,
|
||||
max_candidates=30, # was 50
|
||||
max_walls=100, # NEW: limit walls
|
||||
max_los_checks=10 # NEW: limit LOS checks
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Fix 2.5.1d: App Close — AGGRESSIVE FIX
|
||||
|
||||
**The X button STILL doesn't close the app!**
|
||||
|
||||
**File:** `desktop/main.js`
|
||||
|
||||
**Problem analysis:**
|
||||
- `killBackend()` is called but processes survive
|
||||
- `killAllBackendProcesses()` uses `execSync` but maybe fails silently
|
||||
- Electron might quit before kill completes
|
||||
|
||||
**AGGRESSIVE FIX — Multiple kill strategies:**
|
||||
|
||||
```javascript
|
||||
const { execSync, spawn } = require('child_process');
|
||||
const path = require('path');
|
||||
|
||||
let backendProcess = null;
|
||||
let isQuitting = false;
|
||||
|
||||
// Track all spawned PIDs
|
||||
let knownPids = new Set();
|
||||
|
||||
function killAllRfcpProcesses() {
|
||||
console.log('[KILL] === Starting aggressive kill ===');
|
||||
|
||||
if (process.platform === 'win32') {
|
||||
// Strategy 1: Kill by image name (most reliable)
|
||||
try {
|
||||
console.log('[KILL] Strategy 1: taskkill /F /IM');
|
||||
execSync('taskkill /F /IM rfcp-server.exe', {
|
||||
stdio: 'pipe',
|
||||
timeout: 5000,
|
||||
windowsHide: true
|
||||
});
|
||||
console.log('[KILL] Strategy 1: SUCCESS');
|
||||
} catch (e) {
|
||||
console.log('[KILL] Strategy 1: No processes or already killed');
|
||||
}
|
||||
|
||||
// Strategy 2: Kill by tree if we have PID
|
||||
if (backendProcess && backendProcess.pid) {
|
||||
try {
|
||||
console.log(`[KILL] Strategy 2: taskkill /F /T /PID ${backendProcess.pid}`);
|
||||
execSync(`taskkill /F /T /PID ${backendProcess.pid}`, {
|
||||
stdio: 'pipe',
|
||||
timeout: 5000,
|
||||
windowsHide: true
|
||||
});
|
||||
console.log('[KILL] Strategy 2: SUCCESS');
|
||||
} catch (e) {
|
||||
console.log('[KILL] Strategy 2: PID not found');
|
||||
}
|
||||
}
|
||||
|
||||
// Strategy 3: PowerShell kill (backup)
|
||||
try {
|
||||
console.log('[KILL] Strategy 3: PowerShell Stop-Process');
|
||||
execSync('powershell -Command "Get-Process rfcp-server -ErrorAction SilentlyContinue | Stop-Process -Force"', {
|
||||
stdio: 'pipe',
|
||||
timeout: 5000,
|
||||
windowsHide: true
|
||||
});
|
||||
console.log('[KILL] Strategy 3: SUCCESS');
|
||||
} catch (e) {
|
||||
console.log('[KILL] Strategy 3: PowerShell failed or no processes');
|
||||
}
|
||||
|
||||
// Strategy 4: WMIC kill (legacy but works)
|
||||
try {
|
||||
console.log('[KILL] Strategy 4: WMIC process delete');
|
||||
execSync('wmic process where "name=\'rfcp-server.exe\'" delete', {
|
||||
stdio: 'pipe',
|
||||
timeout: 5000,
|
||||
windowsHide: true
|
||||
});
|
||||
console.log('[KILL] Strategy 4: SUCCESS');
|
||||
} catch (e) {
|
||||
console.log('[KILL] Strategy 4: WMIC failed');
|
||||
}
|
||||
|
||||
} else {
|
||||
// Unix
|
||||
try {
|
||||
execSync('pkill -9 -f rfcp-server', { stdio: 'pipe', timeout: 5000 });
|
||||
} catch (e) {
|
||||
// Ignore
|
||||
}
|
||||
}
|
||||
|
||||
console.log('[KILL] === Kill sequence complete ===');
|
||||
}
|
||||
|
||||
// Call backend shutdown endpoint before killing
|
||||
async function gracefulShutdown() {
|
||||
console.log('[SHUTDOWN] Starting graceful shutdown...');
|
||||
|
||||
// Step 1: Try graceful API shutdown
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), 2000);
|
||||
|
||||
await fetch('http://127.0.0.1:8888/api/system/shutdown', {
|
||||
method: 'POST',
|
||||
signal: controller.signal
|
||||
});
|
||||
|
||||
clearTimeout(timeout);
|
||||
console.log('[SHUTDOWN] Backend acknowledged shutdown');
|
||||
|
||||
// Wait for backend to cleanup
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
|
||||
} catch (e) {
|
||||
console.log('[SHUTDOWN] Backend did not respond:', e.message);
|
||||
}
|
||||
|
||||
// Step 2: Force kill everything
|
||||
killAllRfcpProcesses();
|
||||
|
||||
// Step 3: Wait and verify
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
|
||||
console.log('[SHUTDOWN] Shutdown complete');
|
||||
}
|
||||
|
||||
// === EVENT HANDLERS ===
|
||||
|
||||
// When window X is clicked
|
||||
mainWindow.on('close', async (event) => {
|
||||
console.log('[EVENT] Window close clicked');
|
||||
|
||||
if (!isQuitting) {
|
||||
event.preventDefault(); // Prevent immediate close
|
||||
isQuitting = true;
|
||||
|
||||
await gracefulShutdown();
|
||||
|
||||
// Now actually quit
|
||||
app.quit();
|
||||
}
|
||||
});
|
||||
|
||||
// When all windows closed
|
||||
app.on('window-all-closed', () => {
|
||||
console.log('[EVENT] All windows closed');
|
||||
killAllRfcpProcesses(); // Extra safety
|
||||
|
||||
if (process.platform !== 'darwin') {
|
||||
app.quit();
|
||||
}
|
||||
});
|
||||
|
||||
// Before quit
|
||||
app.on('before-quit', async (event) => {
|
||||
console.log('[EVENT] Before quit');
|
||||
|
||||
if (!isQuitting) {
|
||||
event.preventDefault();
|
||||
isQuitting = true;
|
||||
await gracefulShutdown();
|
||||
app.quit();
|
||||
}
|
||||
});
|
||||
|
||||
// Will quit (sync only!)
|
||||
app.on('will-quit', () => {
|
||||
console.log('[EVENT] Will quit - final sync kill');
|
||||
killAllRfcpProcesses();
|
||||
});
|
||||
|
||||
// Process exit
|
||||
process.on('exit', () => {
|
||||
console.log('[EVENT] Process exit');
|
||||
killAllRfcpProcesses();
|
||||
});
|
||||
|
||||
// Handle Ctrl+C in dev mode
|
||||
process.on('SIGINT', () => {
|
||||
console.log('[SIGNAL] SIGINT');
|
||||
killAllRfcpProcesses();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
process.on('SIGTERM', () => {
|
||||
console.log('[SIGNAL] SIGTERM');
|
||||
killAllRfcpProcesses();
|
||||
process.exit(0);
|
||||
});
|
||||
```
|
||||
|
||||
**Key changes:**
|
||||
1. **4 kill strategies** on Windows (taskkill, PID tree, PowerShell, WMIC)
|
||||
2. **`isQuitting` flag** prevents multiple shutdown attempts
|
||||
3. **`event.preventDefault()`** on close to allow async shutdown
|
||||
4. **Graceful shutdown first** via API, then force kill
|
||||
5. **Every event handler** calls killAllRfcpProcesses()
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Fix 2.5.1e: Update Test Scripts for Win11
|
||||
|
||||
**File:** All .bat scripts
|
||||
|
||||
**Replace `wmic` with PowerShell (wmic deprecated in Win11):**
|
||||
|
||||
```batch
|
||||
:: OLD (wmic - deprecated):
|
||||
for /f "tokens=2 delims==" %%a in ('wmic OS get FreePhysicalMemory /value') do ...
|
||||
|
||||
:: NEW (PowerShell):
|
||||
for /f %%a in ('powershell -Command "(Get-CimInstance Win32_OperatingSystem).FreePhysicalMemory"') do set /a FREE_RAM=%%a/1024
|
||||
|
||||
:: OLD (CPU):
|
||||
for /f "tokens=2 delims==" %%a in ('wmic cpu get LoadPercentage /value') do ...
|
||||
|
||||
:: NEW (PowerShell):
|
||||
for /f %%a in ('powershell -Command "(Get-CimInstance Win32_Processor).LoadPercentage"') do set CPU=%%a
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `backend/app/services/geometry_vectorized.py` | Add max_walls, max_los_checks limits; optimize LOS check |
|
||||
| `backend/app/services/dominant_path_service.py` | Update constants, pass new parameters |
|
||||
| `desktop/main.js` | Aggressive multi-strategy kill; async shutdown with prevention |
|
||||
| `installer/*.bat` | Replace wmic with PowerShell commands |
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Test 1: Performance
|
||||
```bash
|
||||
# Run Detailed preset
|
||||
# Expected: < 90 seconds (was 292s)
|
||||
# Expected: < 100ms/point (was 329ms)
|
||||
```
|
||||
|
||||
### Test 2: App Close
|
||||
```bash
|
||||
# Start RFCP.exe
|
||||
# Click X
|
||||
# Check Task Manager — should be 0 rfcp-server.exe
|
||||
# Check console for [KILL] logs showing all strategies
|
||||
```
|
||||
|
||||
### Test 3: Accuracy Spot Check
|
||||
```bash
|
||||
# Run Standard preset, note coverage shape
|
||||
# Run Detailed preset, compare coverage shape
|
||||
# Should be similar (within visual inspection)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Success Criteria
|
||||
|
||||
- [ ] Detailed preset completes in < 90 seconds
|
||||
- [ ] ms/point < 100ms (was 329ms)
|
||||
- [ ] App close works on first X click
|
||||
- [ ] No rfcp-server.exe in Task Manager after close
|
||||
- [ ] Test scripts work on Windows 11 (no wmic errors)
|
||||
|
||||
---
|
||||
|
||||
## 📈 Expected Performance
|
||||
|
||||
| Metric | Before 2.5.1 | After 2.5.1 |
|
||||
|--------|--------------|-------------|
|
||||
| Walls checked | 351-458 | 100 max |
|
||||
| LOS checks | 50 | 10 max |
|
||||
| ms/point | 329ms | ~50-80ms |
|
||||
| Detailed time | 292s | ~60-90s |
|
||||
| App close | ❌ broken | ✅ works |
|
||||
|
||||
---
|
||||
|
||||
## 🔜 After 2.5.1
|
||||
|
||||
If performance is acceptable:
|
||||
- Phase 2.6: Fun facts loading screen
|
||||
- Phase 2.6: Export to GeoJSON/KML
|
||||
- Phase 2.7: Multi-site interference visualization
|
||||
1718
docs/devlog/installer/RFCP-Phase-3.0-Architecture-Refactor.md
Normal file
1718
docs/devlog/installer/RFCP-Phase-3.0-Architecture-Refactor.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,191 @@
|
||||
# RF Coverage Planning Software: Performance Optimization and Propagation Models
|
||||
|
||||
**The performance gap between fast per-point calculations (~1ms) and slow overall throughput (~258ms/point) is caused by pickle serialization overhead in Python multiprocessing**, which dominates actual compute time when processing small batches. The solution involves batching 1000+ points per IPC round-trip, using shared memory for terrain data, and leveraging GPU acceleration for workloads exceeding 10,000 points—achieving 10-50x speedups. Modern RF coverage tools like Signal-Server, SPLAT!, and Sionna RT demonstrate that combining radial segment parallelization, multi-resolution terrain tiling, and appropriate propagation model selection (Longley-Rice ITM for terrain-based VHF/UHF, COST-231 Hata for cellular) enables efficient large-area calculations while maintaining accuracy within 6-10 dB standard deviation.
|
||||
|
||||
---
|
||||
|
||||
## The multiprocessing bottleneck: why per-point speed deceives
|
||||
|
||||
The dramatic discrepancy between fast individual point calculations and slow aggregate throughput stems from a classic Python multiprocessing anti-pattern where **inter-process communication overhead dominates computation time**. When each worker processes a single point or small batch, the system spends more time serializing and deserializing data than performing actual RF calculations.
|
||||
|
||||
Python's multiprocessing uses pickle for IPC by default, requiring objects to be serialized twice per task (sending to worker and returning results). For RF calculations involving terrain data, DEM arrays, and GIS features, this serialization cost becomes catastrophic. Research shows that pickling a **40 MB dictionary four times per task can cause a 600% slowdown**. The situation worsens because spawning a subprocess takes approximately 50ms (50,000µs) compared to ~100µs for a thread—making process pool initialization per-request extremely expensive.
|
||||
|
||||
The solution architecture requires three fundamental changes. First, batch operations must amortize serialization costs by processing **1,000-10,000 points per IPC round-trip** rather than individual points. Second, shared memory (`multiprocessing.shared_memory` or `numpy.memmap`) should hold terrain data to eliminate pickle overhead entirely. Third, process pools must be pre-initialized at application startup rather than per-request:
|
||||
|
||||
```python
|
||||
# Anti-pattern: Single-point processing (slow)
|
||||
with Pool() as pool:
|
||||
results = pool.map(calculate_point, points) # Each point pickled separately
|
||||
|
||||
# Optimal pattern: Batch processing with shared memory
|
||||
from multiprocessing import shared_memory
|
||||
shm = shared_memory.SharedMemory(create=True, size=terrain_data.nbytes)
|
||||
chunk_size = 1000 # Process 1000 points per IPC round-trip
|
||||
batches = [points[i:i+chunk_size] for i in range(0, len(points), chunk_size)]
|
||||
```
|
||||
|
||||
The target metric is ensuring computation time exceeds serialization time by **10-100x**. For a 1ms per-point calculation, this means batching at least 100-1000 points to make serialization overhead negligible.
|
||||
|
||||
---
|
||||
|
||||
## Open-source RF tools reveal proven optimization architectures
|
||||
|
||||
**Signal-Server**, the C++14 multi-threaded engine that powered CloudRF from 2012-2016, demonstrates the foundational architecture for RF coverage calculations. Its primary improvement over the original SPLAT! was multi-threading through radial segment parallelization—splitting the circular coverage area so multiple threads process different azimuth ranges simultaneously. The implementation uses POSIX threads with configurable segment counts (must be even and greater than 4), processing up to 32 terrain tiles simultaneously with support for gzip/bzip2 compressed tiles for faster I/O.
|
||||
|
||||
Signal-Server supports 12 propagation models through a simple command-line parameter: ITM (Longley-Rice), line-of-sight, Hata, ECC33, SUI, COST-Hata, free-space, ITWOM, Ericsson, Plane Earth, Egli, and Soil models. The terrain tiling system uses SDF format converted from SRTM HGT files, supporting resolutions of 300/600/1200/3600 pixels per tile with automatic multi-tile loading based on calculation bounds.
|
||||
|
||||
**SPLAT!** (Signal Propagation, Loss, And Terrain), the foundational tool started in 1997, uses a radial ray-casting algorithm that projects rays from the transmitter in all azimuths (0-360°), samples terrain elevation along each path, and applies Longley-Rice ITM calculations to the terrain profile. Its Longley-Rice integration handles three prediction ranges (line-of-sight, diffraction, scatter) with terrain irregularity parameter Δh(d) computed from terrain samples. Key parameters include earth dielectric constant (5-80), ground conductivity (0.001-5.0 S/m), atmospheric refractivity (250-400 N-units), and climate zone selection.
|
||||
|
||||
**Sionna RT by NVIDIA** represents the state-of-the-art in GPU-accelerated RF simulation, using differentiable ray tracing built on TensorFlow, Mitsuba 3, and Dr.Jit. Its key innovation enables gradient computation through channel impulse responses with respect to material properties, antenna patterns, and transmitter/receiver positions—making it suitable for ML-integrated optimization. The path solver supports both Shooting and Bouncing Rays (SBR) and the Image Method, handling direct LOS paths, reflections, diffractions, and scattering patterns. Memory efficiency improvements in version 1.0 support scenes with 3D building models from OpenStreetMap, while configurable path loss thresholds and angular separation control enable scalable computation.
|
||||
|
||||
**CloudRF's SLEIPNIR engine** (replacing Signal-Server in 2019) achieves up to **10x faster** performance through multi-resolution modeling that seamlessly merges different resolution data sources, dual CPU/GPU engines (**78% speedup** with GPU for clutter calculations), and 1m LiDAR resolution support with global 10m land cover integration.
|
||||
|
||||
---
|
||||
|
||||
## VHF and UHF propagation models differ fundamentally from cellular bands
|
||||
|
||||
The **Longley-Rice Irregular Terrain Model (ITM)** serves as the most comprehensive model for terrain-based VHF/UHF propagation, predicting median attenuation over irregular terrain for frequencies from 20 MHz to 20 GHz across distances of 1-2000 km. The model handles five propagation mechanisms: free-space loss, terrain diffraction (multiple knife-edge), ground reflection, atmospheric refraction (4/3 Earth radius approximation), and tropospheric scatter beyond the horizon. Statistical variables include time, location, and situation variability ranging from 0.01 to 0.99, with typical accuracy of ±6-10 dB standard deviation for point-to-point mode.
|
||||
|
||||
Critical ITM parameters require careful selection based on environment:
|
||||
|
||||
| Ground Type | Permittivity | Conductivity (S/m) |
|
||||
|------------|--------------|-------------------|
|
||||
| Average Ground | 15 | 0.005 |
|
||||
| Poor Ground | 4 | 0.001 |
|
||||
| Good Ground | 25 | 0.020 |
|
||||
| Fresh Water | 81 | 0.010 |
|
||||
| Sea Water | 81 | 5.0 |
|
||||
|
||||
**ITU-R P.1546** provides empirical field-strength curves for 30 MHz to 4 GHz based on extensive Northern Hemisphere measurements, covering distances of 1-1000 km with time percentages of 1%, 10%, and 50%. The model uses reference frequencies of 100, 600, and 2000 MHz with interpolation for other frequencies, applying corrections for terrain clearance angle, receiving antenna height, clutter losses, and mixed land/sea paths.
|
||||
|
||||
For UHF and cellular bands, the **Okumura-Hata model** (150-1500 MHz, 1-20 km distance) and its **COST-231 extension** (1500-2000 MHz) provide rapid empirical calculations with 6-8 dB standard deviation in urban environments. The urban path loss formula is:
|
||||
|
||||
```
|
||||
L_urban = 69.55 + 26.16*log10(f) - 13.82*log10(h_b) - a(h_m)
|
||||
+ (44.9 - 6.55*log10(h_b))*log10(d)
|
||||
```
|
||||
|
||||
Where `a(h_m)` is the mobile antenna correction factor varying by city size and frequency. Suburban and rural corrections reduce urban loss by 2*(log10(f/28))² + 5.4 dB and 4.78*(log10(f))² - 18.33*log10(f) + 40.94 dB respectively.
|
||||
|
||||
The key propagation differences across frequency bands are dramatic: **VHF wavelengths (1-10m) enable strong diffraction around obstacles but poor building penetration**, while **UHF (0.1-1m wavelength) provides better building penetration but weaker terrain following**. Cellular frequencies (1800+ MHz) have the highest free-space loss baseline, weakest diffraction, and moderate building penetration. Vegetation penetration follows the opposite pattern—VHF penetrates foliage better than higher frequencies where specific attenuation increases significantly.
|
||||
|
||||
---
|
||||
|
||||
## Terrain diffraction models handle mountainous areas differently
|
||||
|
||||
The **single knife-edge diffraction model** (ITU-R P.526) calculates the Fresnel parameter v and corresponding loss:
|
||||
|
||||
```python
|
||||
v = h * sqrt(2 * (d1 + d2) / (wavelength * d1 * d2))
|
||||
# For v > -0.78:
|
||||
if v < 0: loss = 6.02 + 9.11*v - 1.27*v²
|
||||
elif v < 2.4: loss = 6.02 + 9.11*v + 1.65*v²
|
||||
else: loss = 12.953 + 20*log10(v)
|
||||
```
|
||||
|
||||
For multiple obstacles, the **Deygout method** finds the main obstacle (highest Fresnel parameter v between transmitter and receiver), calculates its diffraction loss, then recursively finds secondary obstacles on each side. It provides better accuracy for **widely spaced obstacles** (2-4 ridges) but tends to overestimate for closely spaced obstacles. The **Epstein-Peterson method** calculates diffraction loss sequentially from transmitter to receiver, providing better accuracy for **closely spaced obstacles** but underestimating for widely separated ones.
|
||||
|
||||
The **Bullington equivalent single edge** method replaces all obstacles with one equivalent knife edge, providing the simplest and fastest calculation but often underestimating loss (too optimistic)—useful only for initial estimates. Professional tools like CloudRF implement **Delta-Bullington** as the default for its balance of accuracy and speed, with configurable options including Huygens (basic), sequential multi-obstacle, and Deygout 94 with combining factor.
|
||||
|
||||
---
|
||||
|
||||
## GPU acceleration delivers 10-50x speedups for appropriate workloads
|
||||
|
||||
The RF calculations benefiting most from GPU acceleration are embarrassingly parallel operations: **ray tracing** (10-100x+ speedup with NVIDIA OptiX), **FFT operations** (cuFFT highly optimized), **viewshed/LOS calculations** (CloudRF reports **50x faster** than CPU), and **batch path loss calculations** for many points. Matrix operations in propagation models benefit from cuBLAS, while terrain correlation matrices and large array operations see significant acceleration.
|
||||
|
||||
**CuPy** provides a drop-in NumPy replacement for NVIDIA GPUs with 10-100x speedups for large arrays (>100,000 elements):
|
||||
|
||||
```python
|
||||
import cupy as cp
|
||||
terrain_gpu = cp.asarray(terrain_data)
|
||||
distances = cp.sqrt(cp.sum((points_gpu - tx_position)**2, axis=1))
|
||||
path_loss = 20 * cp.log10(distances) + 20 * cp.log10(frequency_mhz) - 27.55
|
||||
results = path_loss.get() # Transfer back to CPU
|
||||
```
|
||||
|
||||
**Numba CUDA** enables writing custom GPU kernels in Python for complex propagation models requiring control flow:
|
||||
|
||||
```python
|
||||
from numba import cuda
|
||||
import math
|
||||
|
||||
@cuda.jit
|
||||
def free_space_path_loss_kernel(distances, frequency, output):
|
||||
idx = cuda.grid(1)
|
||||
if idx < distances.shape[0]:
|
||||
output[idx] = 20 * math.log10(distances[idx]) + 20 * math.log10(frequency) - 27.55
|
||||
```
|
||||
|
||||
Minimum problem sizes for GPU benefit are: **10,000+ elements** for array operations, **1,024+ points** for FFT, **512x512+** for matrix multiply, and **5,000+ points** for path loss calculations. Memory transfer overhead (PCIe 3.0: ~8 GB/s practical) means the critical formula is `GPU_worthwhile = compute_time > (2 × transfer_time)`. For 100MB terrain data, transfer overhead is approximately 5-12ms.
|
||||
|
||||
**AMD ROCm/HIP** provides cross-platform compatibility through CuPy (`pip install cupy-rocm-5-0`), with PyTorch and TensorFlow also offering official ROCm builds. **Intel integrated graphics** support via PyOpenCL achieves 2-10x speedups over CPU (3-6x slower than discrete GPUs), suitable for edge deployments with moderate workloads (10,000-100,000 points).
|
||||
|
||||
---
|
||||
|
||||
## Environment modeling requires frequency-dependent clutter coefficients
|
||||
|
||||
**ITU-R P.1812-6** defines default clutter heights and losses by environment type: dense urban (20-25m height, 15-25 dB loss), urban (15-20m, 10-20 dB), suburban (9-12m, 5-15 dB), rural (0-4m, 0-5 dB), and forest (15-20m, 10-25 dB). The **3GPP TR 38.901** path loss models define specific scenarios: UMa (Urban Macro) with 25m base station height, UMi (Urban Micro Street Canyon) with 10m base station, RMa (Rural Macro), and InF (Indoor Factory) variants.
|
||||
|
||||
For vegetation, **ITU-R P.833-10** specifies excess attenuation using `A_ev = A_m * (1 - exp(-d*γ/A_m))` where specific attenuation γ varies by frequency: **0.06 dB/m at 200 MHz**, **0.20 dB/m at 1 GHz**, and **0.60 dB/m at 5 GHz** for in-leaf conditions. Seasonal variation reduces loss by approximately 20% out-of-leaf for deciduous forests, with **2 dB variation at 900 MHz increasing to 8.5 dB at 1800+ MHz**.
|
||||
|
||||
**Building entry loss** per ITU-R P.2109 distinguishes traditional buildings (median 10-16 dB at 100 MHz to 2 GHz) from thermally-efficient modern buildings with metallized glass and foil insulation (25-32 dB). Material-specific losses from 3GPP TR 38.901 show standard glass at **2.4 dB at 2 GHz**, concrete at **13 dB at 2 GHz increasing to 117 dB at 28 GHz**, and IRR/Low-E glass at **23.6 dB at 2 GHz**.
|
||||
|
||||
---
|
||||
|
||||
## Machine learning and hybrid approaches complement physics-based models
|
||||
|
||||
Current ML approaches for path loss prediction rank by accuracy: **XGBoost/Gradient Boosting** (RMSE: 2.1-3.4 dB, best for small-medium datasets), Neural Network Ensembles (2.5-4.0 dB), Random Forest (3.0-4.5 dB), and Deep Neural Networks (3.0-5.0 dB). Training data requirements scale predictably: <1,000 samples yield RMSE 6-10 dB, 10,000-100,000 samples achieve production-quality RMSE 2-4 dB.
|
||||
|
||||
**Hybrid physics+ML architectures** prove most effective. The ML Correction approach calculates `PL_total = PL_empirical(d, f, h_tx, h_rx) + ΔPL_ML(features)` where ΔPL_ML learns systematic biases. The LOS/NLOS Ensemble uses a classifier to weight separate LOS and NLOS regressors. Physics-Informed Neural Networks add penalty terms that enforce physical constraints like "path loss should increase with distance" and "FSPL provides a lower bound."
|
||||
|
||||
**Pre-computed propagation databases** store path loss values at 20-50 bytes per grid cell, enabling sub-millisecond lookups. For a 10km radius at 30m resolution (~349,000 cells), storage is approximately 7 MB compressed. Interpolation techniques range from fast bilinear (1-2 dB error) to kriging (higher accuracy with uncertainty estimates).
|
||||
|
||||
---
|
||||
|
||||
## Tile-based caching enables responsive coverage map delivery
|
||||
|
||||
The optimal caching architecture uses **XYZ (Slippy Map) tiles** with multi-tier storage: L1 in-memory Redis (sub-millisecond access, ~100GB capacity), L2 disk cache (SQLite/MBTiles format), and L3 cloud storage (S3 for permanent pre-computed tiles). Cache keys should incorporate parameter hashes for instant invalidation when transmitter settings change:
|
||||
|
||||
```python
|
||||
def get_tile_key(z: int, x: int, y: int, params_hash: str) -> str:
|
||||
return f"tile:coverage:{params_hash}:{z}:{x}:{y}"
|
||||
```
|
||||
|
||||
For dynamic coverage, TTL-based expiration (15 minutes to 24 hours) combined with Redis pub/sub channels (`map:update:region:*`) enables targeted geographic invalidation. The hybrid approach pre-computes base zoom levels (z=6-12) for commonly accessed areas while generating higher zoom levels (z>12) on-demand.
|
||||
|
||||
**Level of Detail (LOD) techniques** adapt computation intensity to distance: Tier 1 (0-500m) uses full 3D building geometry with 1m terrain resolution, Tier 2 (500m-2km) uses simplified buildings with 10m terrain, Tier 3 (2-10km) uses clutter heights only with 30m terrain, and Tier 4 (>10km) uses statistical clutter with 90m SRTM terrain. Adaptive grid generation provides higher resolution near the transmitter (10m) transitioning to coarser resolution (100m) at distance, reducing computation while maintaining visual quality where it matters.
|
||||
|
||||
---
|
||||
|
||||
## Recommended architecture for Python/FastAPI RF coverage backend
|
||||
|
||||
The optimal stack combines **FastAPI** (async API gateway with rate limiting), **Celery** (distributed task queue for heavy RF calculations), **Redis** (tile caching and job status), and **CuPy/Numba** (GPU acceleration). Terrain data should use **numpy.memmap** for memory-mapped access to large DEMs with **STRtree spatial indexing** for tile lookups via Shapely.
|
||||
|
||||
For the propagation engine, implement **Longley-Rice ITM** as the primary terrain model (using the `itmlogic` Python package), **COST-231 Hata** for quick urban estimates, and **Deygout diffraction** for multiple terrain obstacles. The model selection logic should consider frequency range (Hata for 150-1500 MHz, COST-231 for 1500-2000 MHz, ITM for terrain-specific), distance (empirical for <20km, ITM for longer paths), and accuracy requirements (ray tracing only for <5km urban scenarios).
|
||||
|
||||
```python
|
||||
class GPURFEngine:
|
||||
def __init__(self, max_points=1_000_000):
|
||||
# Pre-allocate GPU memory at startup
|
||||
self.d_buffer = cp.empty((max_points, 3), dtype=cp.float32)
|
||||
|
||||
async def calculate_coverage(self, points: np.ndarray) -> np.ndarray:
|
||||
if len(points) < 1000:
|
||||
return self._cpu_fallback(points) # Small workloads on CPU
|
||||
# GPU path for large workloads
|
||||
d_points = cp.asarray(points)
|
||||
# ... GPU computation
|
||||
return results.get()
|
||||
```
|
||||
|
||||
Celery configuration should use separate queues for fast (cached), compute (full calculation), and batch operations, with `worker_prefetch_multiplier=1` for heavy tasks and `task_acks_late=True` for reliability. Output formats should include PNG tiles with colormap lookup for web display and Cloud-Optimized GeoTIFF for professional GIS integration.
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
Building efficient RF coverage planning software requires addressing the fundamental mismatch between fast per-point propagation calculations and the overhead of Python's multiprocessing model. **Batch processing (1000+ points per IPC round-trip), shared memory for terrain data, and GPU acceleration for workloads exceeding 10,000 points** provide the foundation for achieving throughput within an order of magnitude of commercial tools.
|
||||
|
||||
The propagation model selection should follow a tiered approach: Longley-Rice ITM for terrain-based VHF/UHF planning with available DEM data, Okumura-Hata/COST-231 for rapid urban cellular estimates, and Deygout diffraction for mountainous terrain with multiple obstacles. Environment modeling through ITU-R P.2108/P.2109/P.833 provides standardized clutter, building entry, and vegetation loss coefficients that maintain accuracy across diverse deployment scenarios.
|
||||
|
||||
The most impactful optimizations in order of implementation priority are: fixing the multiprocessing serialization bottleneck (immediate 100x throughput improvement), implementing tile-based caching with parameter-hash keys (sub-millisecond repeat queries), adding GPU acceleration for large coverage maps (10-50x for >10,000 points), and incorporating LOD techniques (3-10x computation reduction with minimal accuracy impact). This architecture enables a Python/FastAPI backend to compete with commercial tools while maintaining the flexibility for custom propagation models and ML integration.
|
||||
Reference in New Issue
Block a user