rfcp/RFCP-Iteration-3.2.0-Comprehensive-Fixes.md

RFCP - Iteration 3.2.0: Comprehensive Performance & Bug Fixes

Overview

Major iteration combining performance optimizations, UI fixes, and bug resolutions. This addresses the Detailed preset timeout, stuck progress bar, app close issues, region data problems, and UX improvements.

Scope: Backend optimizations + Frontend fixes + Electron fixes + Data validation

---

Part 1: Performance Optimizations

1.1 Adaptive Resolution (CRITICAL)

Problem: 10km radius with 200m resolution = ~7850 points → timeout

Solution: Distance-based adaptive resolution

File: backend/app/services/coverage_service.py (or grid generation code)

def get_adaptive_resolution(base_resolution: float, distance_from_tx: float) -> float:
    """
    Adaptive resolution based on distance from transmitter.
    
    Close to TX: use user's chosen resolution (details matter)
    Far from TX: use coarser resolution (macro view sufficient)
    """
    if distance_from_tx < 2000:  # < 2km
        return base_resolution  # User's choice (e.g., 200m)
    elif distance_from_tx < 5000:  # 2-5km
        return max(base_resolution, 300)  # At least 300m
    else:  # > 5km
        return max(base_resolution, 500)  # At least 500m


def generate_adaptive_grid(center_lat, center_lon, radius_m, base_resolution):
    """
    Generate grid with adaptive resolution zones.
    
    Instead of uniform grid, create concentric zones with different resolutions.
    """
    points = []
    
    # Zone 1: Inner (< 2km) - full resolution
    inner_points = generate_grid_ring(center_lat, center_lon, 0, 2000, base_resolution)
    points.extend(inner_points)
    
    # Zone 2: Middle (2-5km) - medium resolution  
    if radius_m > 2000:
        medium_res = max(base_resolution, 300)
        middle_points = generate_grid_ring(center_lat, center_lon, 2000, min(5000, radius_m), medium_res)
        points.extend(middle_points)
    
    # Zone 3: Outer (5km+) - coarse resolution
    if radius_m > 5000:
        coarse_res = max(base_resolution, 500)
        outer_points = generate_grid_ring(center_lat, center_lon, 5000, radius_m, coarse_res)
        points.extend(outer_points)
    
    return points

Expected result:

• 10km with 200m base: ~7850 → ~2500 points (3x reduction)
• Combined with LOD: 10km detailed should complete in ~60s

1.2 Radial Preview Mode (NEW FEATURE)

Purpose: Instant preview using 360 radial spokes instead of full grid

File: backend/app/services/coverage_service.py

def calculate_radial_preview(
    tx_lat: float, tx_lon: float, tx_height: float,
    radius_m: float, frequency_mhz: float,
    num_spokes: int = 360,  # 1 degree resolution
    points_per_spoke: int = 50
) -> List[dict]:
    """
    Fast radial preview calculation.
    
    Instead of grid, calculate along 360 radial lines from TX.
    Much faster because:
    - Terrain profile can be cached per spoke
    - No building calculations (terrain only)
    - Linear interpolation between points
    """
    results = []
    
    for angle_deg in range(num_spokes):
        angle_rad = math.radians(angle_deg)
        
        # Calculate points along this spoke
        for i in range(points_per_spoke):
            distance = (i + 1) * (radius_m / points_per_spoke)
            
            # Calculate point position
            rx_lat, rx_lon = move_point(tx_lat, tx_lon, distance, angle_deg)
            
            # Simple terrain-only calculation (no buildings)
            path_loss = calculate_terrain_path_loss(
                tx_lat, tx_lon, tx_height,
                rx_lat, rx_lon, 1.5,  # Standard UE height
                frequency_mhz
            )
            
            results.append({
                'lat': rx_lat,
                'lon': rx_lon,
                'rsrp': tx_power_dbm - path_loss,
                'distance': distance,
                'angle': angle_deg
            })
    
    return results

Add to API: New endpoint or parameter ?mode=preview

---

Part 2: Bug Fixes

2.1 Progress Bar Stuck at "Initializing 5%" (CRITICAL)

Problem: Progress never updates past 5%

Root Cause: WebSocket messages not reaching frontend OR React state not updating

Debug & Fix Steps:

Step 1: Backend - Verify messages are sent

File: backend/app/api/websocket.py or backend/app/services/coverage_service.py

import logging
logger = logging.getLogger(__name__)

async def send_progress(websocket, progress: int, status: str):
    """Send progress with logging"""
    message = {"type": "progress", "progress": progress, "status": status}
    logger.info(f"[WS] Sending progress: {progress}% - {status}")
    await websocket.send_json(message)
    await asyncio.sleep(0)  # Yield to event loop

Step 2: Frontend - Check WebSocket handling

File: frontend/src/services/websocket.ts or similar

// Add logging
socket.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log('[WS] Received:', data);
  
  if (data.type === 'progress') {
    console.log('[WS] Progress update:', data.progress, data.status);
    // Update store
    setCoverageProgress(data.progress, data.status);
  }
};

Step 3: Frontend - Check React state update

File: frontend/src/store/coverage.ts or state management

// Ensure state updates trigger re-render
setCoverageProgress: (progress: number, status: string) => {
  console.log('[Store] Setting progress:', progress, status);
  set({ 
    progress: progress,  // Must be new value, not mutation
    progressStatus: status 
  });
}

Step 4: Frontend - Check component subscription

File: frontend/src/App.tsx or progress display component

// Ensure component subscribes to store changes
const progress = useCoverageStore((state) => state.progress);
const progressStatus = useCoverageStore((state) => state.progressStatus);

useEffect(() => {
  console.log('[UI] Progress changed:', progress, progressStatus);
}, [progress, progressStatus]);

2.2 App Close Button Broken (Electron)

Problem: Clicking X kills backend but Electron window stays open

File: desktop/main.js

const { app, BrowserWindow } = require('electron');
const { spawn } = require('child_process');

let mainWindow;
let backendProcess;

function createWindow() {
  mainWindow = new BrowserWindow({
    // ... existing config
  });

  // Handle window close
  mainWindow.on('close', async (event) => {
    event.preventDefault();  // Prevent immediate close
    
    console.log('[Electron] Window closing, cleaning up...');
    
    // Kill backend process
    if (backendProcess) {
      console.log('[Electron] Killing backend process...');
      backendProcess.kill('SIGTERM');
      
      // Wait for graceful shutdown
      await new Promise(resolve => setTimeout(resolve, 1000));
      
      // Force kill if still running
      if (!backendProcess.killed) {
        backendProcess.kill('SIGKILL');
      }
    }
    
    // Now actually close
    mainWindow.destroy();
  });

  mainWindow.on('closed', () => {
    mainWindow = null;
  });
}

// Ensure app quits when all windows closed
app.on('window-all-closed', () => {
  console.log('[Electron] All windows closed, quitting app');
  app.quit();
});

// Cleanup on app quit
app.on('before-quit', () => {
  console.log('[Electron] App quitting, final cleanup');
  if (backendProcess && !backendProcess.killed) {
    backendProcess.kill('SIGKILL');
  }
});

2.3 Memory Leak (1328 MB not released)

Problem: Memory not freed after calculation

File: backend/app/services/parallel_coverage_service.py or backend/app/parallel/manager.py

import gc
from multiprocessing import shared_memory

class SharedMemoryManager:
    def __init__(self):
        self._shared_blocks = []
    
    def create_shared_block(self, name, size):
        shm = shared_memory.SharedMemory(name=name, create=True, size=size)
        self._shared_blocks.append(shm)
        return shm
    
    def cleanup(self):
        """Explicitly cleanup all shared memory blocks"""
        for shm in self._shared_blocks:
            try:
                shm.close()
                shm.unlink()  # Important! Actually frees the memory
            except Exception as e:
                logger.warning(f"Error cleaning up shared memory: {e}")
        self._shared_blocks.clear()
        
        # Force garbage collection
        gc.collect()


# In coverage calculation:
async def calculate_coverage(...):
    shm_manager = SharedMemoryManager()
    try:
        # ... calculation code
        pass
    finally:
        # ALWAYS cleanup, even on error/timeout
        shm_manager.cleanup()
        logger.info("[MEMORY] Shared memory cleaned up")

2.4 Region Data / Map Cache Issues

Problem: Western Ukraine region hangs or produces "No coverage points found"

Diagnosis steps:

Step 1: Check ProgramData folder structure

# Add diagnostic endpoint or startup check
import os

def diagnose_data_folders():
    """Check data folder structure and validity"""
    
    # Common locations
    locations = [
        os.path.expandvars(r'%PROGRAMDATA%\RFCP'),
        os.path.expandvars(r'%APPDATA%\RFCP'),
        os.path.expandvars(r'%LOCALAPPDATA%\RFCP'),
        './data',
        '../data'
    ]
    
    report = {}
    for loc in locations:
        if os.path.exists(loc):
            report[loc] = {
                'exists': True,
                'files': os.listdir(loc),
                'size_mb': sum(
                    os.path.getsize(os.path.join(loc, f)) 
                    for f in os.listdir(loc) 
                    if os.path.isfile(os.path.join(loc, f))
                ) / 1024 / 1024
            }
            
            # Check terrain tiles
            terrain_dir = os.path.join(loc, 'terrain')
            if os.path.exists(terrain_dir):
                tiles = [f for f in os.listdir(terrain_dir) if f.endswith('.hgt')]
                report[loc]['terrain_tiles'] = len(tiles)
                
            # Check OSM cache
            osm_dir = os.path.join(loc, 'osm_cache')
            if os.path.exists(osm_dir):
                cache_files = os.listdir(osm_dir)
                report[loc]['osm_cache_files'] = len(cache_files)
    
    return report

Step 2: Validate terrain tiles

def validate_terrain_tile(filepath: str) -> dict:
    """Check if terrain tile is valid"""
    import struct
    
    result = {
        'path': filepath,
        'exists': os.path.exists(filepath),
        'valid': False,
        'error': None
    }
    
    if not result['exists']:
        result['error'] = 'File not found'
        return result
    
    try:
        size = os.path.getsize(filepath)
        
        # SRTM1 (1 arc-second): 3601x3601x2 = 25,934,402 bytes
        # SRTM3 (3 arc-second): 1201x1201x2 = 2,884,802 bytes
        
        if size == 25934402:
            result['type'] = 'SRTM1'
            result['valid'] = True
        elif size == 2884802:
            result['type'] = 'SRTM3'
            result['valid'] = True
        else:
            result['error'] = f'Unexpected size: {size} bytes'
            
    except Exception as e:
        result['error'] = str(e)
    
    return result

Step 3: Fix OSM cache for regions

def get_osm_cache_key(bbox: tuple, data_type: str) -> str:
    """Generate consistent cache key for OSM data"""
    # Round to avoid floating point issues
    lat_min = round(bbox[0], 4)
    lon_min = round(bbox[1], 4)
    lat_max = round(bbox[2], 4)
    lon_max = round(bbox[3], 4)
    
    return f"{data_type}_{lat_min}_{lon_min}_{lat_max}_{lon_max}.json"


def validate_osm_cache(cache_path: str) -> bool:
    """Check if cached OSM data is valid"""
    try:
        with open(cache_path, 'r') as f:
            data = json.load(f)
        
        # Check structure
        if not isinstance(data, dict):
            return False
        if 'elements' not in data and not isinstance(data, list):
            return False
            
        return True
    except:
        return False

---

Part 3: UI Fixes

3.1 Calculate Button Position

Problem: Button overlaps with scrollbar

File: frontend/src/components/CoverageSettings.tsx or similar

// Move Calculate button outside scrollable area
// Or add right margin

<div className="coverage-settings-panel">
  <div className="scrollable-content">
    {/* All settings */}
  </div>
  
  <div className="fixed-footer" style={{ 
    padding: '16px',
    borderTop: '1px solid var(--border-color)',
    marginRight: '16px'  // Space for scrollbar
  }}>
    <button 
      className="calculate-button"
      onClick={handleCalculate}
      style={{ width: '100%' }}
    >
      Calculate Coverage
    </button>
  </div>
</div>

3.2 Site Drag - Move Sectors Together

Problem: Dragging site doesn't move its sectors

File: frontend/src/components/map/SiteMarker.tsx or site handling code

const handleSiteDrag = (siteId: string, newLat: number, newLon: number) => {
  const site = getSite(siteId);
  if (!site) return;
  
  // Calculate delta
  const deltaLat = newLat - site.lat;
  const deltaLon = newLon - site.lon;
  
  // Update site position
  updateSite(siteId, { lat: newLat, lon: newLon });
  
  // Update all sectors of this site
  const sectors = getSectorsForSite(siteId);
  sectors.forEach(sector => {
    updateSector(sector.id, {
      lat: sector.lat + deltaLat,
      lon: sector.lon + deltaLon
    });
  });
};

3.3 Site Delete - Remove Sectors Together

Problem: Deleting site doesn't remove its sectors

File: frontend/src/store/sites.ts or site management

const deleteSite = (siteId: string) => {
  // First, delete all sectors belonging to this site
  const sectors = get().sectors.filter(s => s.siteId === siteId);
  sectors.forEach(sector => {
    deleteSector(sector.id);
  });
  
  // Then delete the site
  set(state => ({
    sites: state.sites.filter(s => s.id !== siteId)
  }));
};

---

Part 4: Testing Checklist

Performance Tests

• 5km Standard: < 10 seconds
• 5km Detailed: < 60 seconds
• 10km Standard: < 30 seconds
• 10km Detailed: < 120 seconds (was timeout)
• Radial preview (any radius): < 5 seconds

Bug Fix Tests

• Progress bar updates from 5% → 100%
• App closes completely when clicking X
• Memory returns to baseline after calculation
• Western Ukraine region calculates successfully
• All terrain tiles validate correctly
• OSM cache files are valid JSON

UI Tests

• Calculate button not overlapping scrollbar
• Dragging site moves all its sectors
• Deleting site removes all its sectors
• No console errors in browser DevTools

Data Validation

• Run diagnose_data_folders() - check output
• Validate all terrain tiles in cache
• Validate all OSM cache files
• Check for corrupted files and remove them

---

Build & Deploy

# Backend
cd D:\root\rfcp\backend
pip install -e .

# Frontend
cd D:\root\rfcp\frontend
npm run build

# Electron
cd D:\root\rfcp\desktop
npm run build

# Full test
cd D:\root\rfcp\installer
.\test-detailed-quick.bat

# If works, rebuild installer
pyinstaller rfcp-server.spec --clean

---

Commit Message

feat: Iteration 3.2.0 - Performance & Bug Fixes

Performance:
- Add adaptive resolution (distance-based grid density)
- Add radial preview mode (360 spokes, instant feedback)
- Expected: 10km Detailed ~60s (was timeout)

Bug Fixes:
- Fix progress bar stuck at 5% (WebSocket + React state)
- Fix app close button (Electron lifecycle)
- Fix memory leak (SharedMemory cleanup)
- Fix region data issues (cache validation)

UI Improvements:
- Move Calculate button (scrollbar overlap)
- Site drag moves all sectors
- Site delete removes all sectors

Data Validation:
- Add terrain tile validation
- Add OSM cache validation
- Add diagnostic reporting

---

Success Criteria

1. 10km Detailed completes without timeout (~60-120s acceptable)
2. Progress bar works - shows actual progress 5% → 100%
3. App closes cleanly - no orphan processes
4. Memory released - returns to baseline after calculation
5. All regions work - Western Ukraine calculates successfully
6. Site management - drag/delete affects sectors correctly

---

Priority Order for Implementation

1. Adaptive Resolution - biggest performance impact
2. Progress bar fix - critical UX issue
3. App close fix - annoying bug
4. Site drag/delete sectors - quick win
5. Calculate button position - quick win
6. Memory leak - important but complex
7. Region data validation - diagnostic + fix
8. Radial preview - nice to have

---

Notes for Claude Code

• This is a large iteration - take it step by step
• Test after each major change
• Backend and frontend changes may need coordination
• Electron changes require rebuild of desktop app
• Data validation can be added as debug endpoint first
• If stuck on one issue, move to next and come back

---

"Big iteration, big impact. Let's make RFCP production-ready!" 🚀