# RFCP - Iteration 3.2.0: Comprehensive Performance & Bug Fixes

## Overview

Major iteration combining performance optimizations, UI fixes, and bug resolutions. This addresses the Detailed preset timeout, stuck progress bar, app close issues, region data problems, and UX improvements.

**Scope:** Backend optimizations + Frontend fixes + Electron fixes + Data validation

---

## Part 1: Performance Optimizations

### 1.1 Adaptive Resolution (CRITICAL)

**Problem:** 10km radius with 200m resolution = ~7850 points → timeout

**Solution:** Distance-based adaptive resolution

**File:** `backend/app/services/coverage_service.py` (or grid generation code)

```python
def get_adaptive_resolution(base_resolution: float, distance_from_tx: float) -> float:
    """
    Adaptive resolution based on distance from transmitter.
    
    Close to TX: use user's chosen resolution (details matter)
    Far from TX: use coarser resolution (macro view sufficient)
    """
    if distance_from_tx < 2000:  # < 2km
        return base_resolution  # User's choice (e.g., 200m)
    elif distance_from_tx < 5000:  # 2-5km
        return max(base_resolution, 300)  # At least 300m
    else:  # > 5km
        return max(base_resolution, 500)  # At least 500m


def generate_adaptive_grid(center_lat, center_lon, radius_m, base_resolution):
    """
    Generate grid with adaptive resolution zones.
    
    Instead of uniform grid, create concentric zones with different resolutions.
    """
    points = []
    
    # Zone 1: Inner (< 2km) - full resolution
    inner_points = generate_grid_ring(center_lat, center_lon, 0, 2000, base_resolution)
    points.extend(inner_points)
    
    # Zone 2: Middle (2-5km) - medium resolution  
    if radius_m > 2000:
        medium_res = max(base_resolution, 300)
        middle_points = generate_grid_ring(center_lat, center_lon, 2000, min(5000, radius_m), medium_res)
        points.extend(middle_points)
    
    # Zone 3: Outer (5km+) - coarse resolution
    if radius_m > 5000:
        coarse_res = max(base_resolution, 500)
        outer_points = generate_grid_ring(center_lat, center_lon, 5000, radius_m, coarse_res)
        points.extend(outer_points)
    
    return points
```

**Expected result:**
- 10km with 200m base: ~7850 → ~2500 points (3x reduction)
- Combined with LOD: 10km detailed should complete in ~60s

### 1.2 Radial Preview Mode (NEW FEATURE)

**Purpose:** Instant preview using 360 radial spokes instead of full grid

**File:** `backend/app/services/coverage_service.py`

```python
def calculate_radial_preview(
    tx_lat: float, tx_lon: float, tx_height: float,
    radius_m: float, frequency_mhz: float,
    num_spokes: int = 360,  # 1 degree resolution
    points_per_spoke: int = 50
) -> List[dict]:
    """
    Fast radial preview calculation.
    
    Instead of grid, calculate along 360 radial lines from TX.
    Much faster because:
    - Terrain profile can be cached per spoke
    - No building calculations (terrain only)
    - Linear interpolation between points
    """
    results = []
    
    for angle_deg in range(num_spokes):
        angle_rad = math.radians(angle_deg)
        
        # Calculate points along this spoke
        for i in range(points_per_spoke):
            distance = (i + 1) * (radius_m / points_per_spoke)
            
            # Calculate point position
            rx_lat, rx_lon = move_point(tx_lat, tx_lon, distance, angle_deg)
            
            # Simple terrain-only calculation (no buildings)
            path_loss = calculate_terrain_path_loss(
                tx_lat, tx_lon, tx_height,
                rx_lat, rx_lon, 1.5,  # Standard UE height
                frequency_mhz
            )
            
            results.append({
                'lat': rx_lat,
                'lon': rx_lon,
                'rsrp': tx_power_dbm - path_loss,
                'distance': distance,
                'angle': angle_deg
            })
    
    return results
```

**Add to API:** New endpoint or parameter `?mode=preview`

---

## Part 2: Bug Fixes

### 2.1 Progress Bar Stuck at "Initializing 5%" (CRITICAL)

**Problem:** Progress never updates past 5%

**Root Cause:** WebSocket messages not reaching frontend OR React state not updating

**Debug & Fix Steps:**

**Step 1: Backend - Verify messages are sent**

File: `backend/app/api/websocket.py` or `backend/app/services/coverage_service.py`

```python
import logging
logger = logging.getLogger(__name__)

async def send_progress(websocket, progress: int, status: str):
    """Send progress with logging"""
    message = {"type": "progress", "progress": progress, "status": status}
    logger.info(f"[WS] Sending progress: {progress}% - {status}")
    await websocket.send_json(message)
    await asyncio.sleep(0)  # Yield to event loop
```

**Step 2: Frontend - Check WebSocket handling**

File: `frontend/src/services/websocket.ts` or similar

```typescript
// Add logging
socket.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log('[WS] Received:', data);
  
  if (data.type === 'progress') {
    console.log('[WS] Progress update:', data.progress, data.status);
    // Update store
    setCoverageProgress(data.progress, data.status);
  }
};
```

**Step 3: Frontend - Check React state update**

File: `frontend/src/store/coverage.ts` or state management

```typescript
// Ensure state updates trigger re-render
setCoverageProgress: (progress: number, status: string) => {
  console.log('[Store] Setting progress:', progress, status);
  set({ 
    progress: progress,  // Must be new value, not mutation
    progressStatus: status 
  });
}
```

**Step 4: Frontend - Check component subscription**

File: `frontend/src/App.tsx` or progress display component

```typescript
// Ensure component subscribes to store changes
const progress = useCoverageStore((state) => state.progress);
const progressStatus = useCoverageStore((state) => state.progressStatus);

useEffect(() => {
  console.log('[UI] Progress changed:', progress, progressStatus);
}, [progress, progressStatus]);
```

### 2.2 App Close Button Broken (Electron)

**Problem:** Clicking X kills backend but Electron window stays open

**File:** `desktop/main.js`

```javascript
const { app, BrowserWindow } = require('electron');
const { spawn } = require('child_process');

let mainWindow;
let backendProcess;

function createWindow() {
  mainWindow = new BrowserWindow({
    // ... existing config
  });

  // Handle window close
  mainWindow.on('close', async (event) => {
    event.preventDefault();  // Prevent immediate close
    
    console.log('[Electron] Window closing, cleaning up...');
    
    // Kill backend process
    if (backendProcess) {
      console.log('[Electron] Killing backend process...');
      backendProcess.kill('SIGTERM');
      
      // Wait for graceful shutdown
      await new Promise(resolve => setTimeout(resolve, 1000));
      
      // Force kill if still running
      if (!backendProcess.killed) {
        backendProcess.kill('SIGKILL');
      }
    }
    
    // Now actually close
    mainWindow.destroy();
  });

  mainWindow.on('closed', () => {
    mainWindow = null;
  });
}

// Ensure app quits when all windows closed
app.on('window-all-closed', () => {
  console.log('[Electron] All windows closed, quitting app');
  app.quit();
});

// Cleanup on app quit
app.on('before-quit', () => {
  console.log('[Electron] App quitting, final cleanup');
  if (backendProcess && !backendProcess.killed) {
    backendProcess.kill('SIGKILL');
  }
});
```

### 2.3 Memory Leak (1328 MB not released)

**Problem:** Memory not freed after calculation

**File:** `backend/app/services/parallel_coverage_service.py` or `backend/app/parallel/manager.py`

```python
import gc
from multiprocessing import shared_memory

class SharedMemoryManager:
    def __init__(self):
        self._shared_blocks = []
    
    def create_shared_block(self, name, size):
        shm = shared_memory.SharedMemory(name=name, create=True, size=size)
        self._shared_blocks.append(shm)
        return shm
    
    def cleanup(self):
        """Explicitly cleanup all shared memory blocks"""
        for shm in self._shared_blocks:
            try:
                shm.close()
                shm.unlink()  # Important! Actually frees the memory
            except Exception as e:
                logger.warning(f"Error cleaning up shared memory: {e}")
        self._shared_blocks.clear()
        
        # Force garbage collection
        gc.collect()


# In coverage calculation:
async def calculate_coverage(...):
    shm_manager = SharedMemoryManager()
    try:
        # ... calculation code
        pass
    finally:
        # ALWAYS cleanup, even on error/timeout
        shm_manager.cleanup()
        logger.info("[MEMORY] Shared memory cleaned up")
```

### 2.4 Region Data / Map Cache Issues

**Problem:** Western Ukraine region hangs or produces "No coverage points found"

**Diagnosis steps:**

**Step 1: Check ProgramData folder structure**

```python
# Add diagnostic endpoint or startup check
import os

def diagnose_data_folders():
    """Check data folder structure and validity"""
    
    # Common locations
    locations = [
        os.path.expandvars(r'%PROGRAMDATA%\RFCP'),
        os.path.expandvars(r'%APPDATA%\RFCP'),
        os.path.expandvars(r'%LOCALAPPDATA%\RFCP'),
        './data',
        '../data'
    ]
    
    report = {}
    for loc in locations:
        if os.path.exists(loc):
            report[loc] = {
                'exists': True,
                'files': os.listdir(loc),
                'size_mb': sum(
                    os.path.getsize(os.path.join(loc, f)) 
                    for f in os.listdir(loc) 
                    if os.path.isfile(os.path.join(loc, f))
                ) / 1024 / 1024
            }
            
            # Check terrain tiles
            terrain_dir = os.path.join(loc, 'terrain')
            if os.path.exists(terrain_dir):
                tiles = [f for f in os.listdir(terrain_dir) if f.endswith('.hgt')]
                report[loc]['terrain_tiles'] = len(tiles)
                
            # Check OSM cache
            osm_dir = os.path.join(loc, 'osm_cache')
            if os.path.exists(osm_dir):
                cache_files = os.listdir(osm_dir)
                report[loc]['osm_cache_files'] = len(cache_files)
    
    return report
```

**Step 2: Validate terrain tiles**

```python
def validate_terrain_tile(filepath: str) -> dict:
    """Check if terrain tile is valid"""
    import struct
    
    result = {
        'path': filepath,
        'exists': os.path.exists(filepath),
        'valid': False,
        'error': None
    }
    
    if not result['exists']:
        result['error'] = 'File not found'
        return result
    
    try:
        size = os.path.getsize(filepath)
        
        # SRTM1 (1 arc-second): 3601x3601x2 = 25,934,402 bytes
        # SRTM3 (3 arc-second): 1201x1201x2 = 2,884,802 bytes
        
        if size == 25934402:
            result['type'] = 'SRTM1'
            result['valid'] = True
        elif size == 2884802:
            result['type'] = 'SRTM3'
            result['valid'] = True
        else:
            result['error'] = f'Unexpected size: {size} bytes'
            
    except Exception as e:
        result['error'] = str(e)
    
    return result
```

**Step 3: Fix OSM cache for regions**

```python
def get_osm_cache_key(bbox: tuple, data_type: str) -> str:
    """Generate consistent cache key for OSM data"""
    # Round to avoid floating point issues
    lat_min = round(bbox[0], 4)
    lon_min = round(bbox[1], 4)
    lat_max = round(bbox[2], 4)
    lon_max = round(bbox[3], 4)
    
    return f"{data_type}_{lat_min}_{lon_min}_{lat_max}_{lon_max}.json"


def validate_osm_cache(cache_path: str) -> bool:
    """Check if cached OSM data is valid"""
    try:
        with open(cache_path, 'r') as f:
            data = json.load(f)
        
        # Check structure
        if not isinstance(data, dict):
            return False
        if 'elements' not in data and not isinstance(data, list):
            return False
            
        return True
    except:
        return False
```

---

## Part 3: UI Fixes

### 3.1 Calculate Button Position

**Problem:** Button overlaps with scrollbar

**File:** `frontend/src/components/CoverageSettings.tsx` or similar

```tsx
// Move Calculate button outside scrollable area
// Or add right margin

<div className="coverage-settings-panel">
  <div className="scrollable-content">
    {/* All settings */}
  </div>
  
  <div className="fixed-footer" style={{ 
    padding: '16px',
    borderTop: '1px solid var(--border-color)',
    marginRight: '16px'  // Space for scrollbar
  }}>
    <button 
      className="calculate-button"
      onClick={handleCalculate}
      style={{ width: '100%' }}
    >
      Calculate Coverage
    </button>
  </div>
</div>
```

### 3.2 Site Drag - Move Sectors Together

**Problem:** Dragging site doesn't move its sectors

**File:** `frontend/src/components/map/SiteMarker.tsx` or site handling code

```typescript
const handleSiteDrag = (siteId: string, newLat: number, newLon: number) => {
  const site = getSite(siteId);
  if (!site) return;
  
  // Calculate delta
  const deltaLat = newLat - site.lat;
  const deltaLon = newLon - site.lon;
  
  // Update site position
  updateSite(siteId, { lat: newLat, lon: newLon });
  
  // Update all sectors of this site
  const sectors = getSectorsForSite(siteId);
  sectors.forEach(sector => {
    updateSector(sector.id, {
      lat: sector.lat + deltaLat,
      lon: sector.lon + deltaLon
    });
  });
};
```

### 3.3 Site Delete - Remove Sectors Together

**Problem:** Deleting site doesn't remove its sectors

**File:** `frontend/src/store/sites.ts` or site management

```typescript
const deleteSite = (siteId: string) => {
  // First, delete all sectors belonging to this site
  const sectors = get().sectors.filter(s => s.siteId === siteId);
  sectors.forEach(sector => {
    deleteSector(sector.id);
  });
  
  // Then delete the site
  set(state => ({
    sites: state.sites.filter(s => s.id !== siteId)
  }));
};
```

---

## Part 4: Testing Checklist

### Performance Tests
- [ ] 5km Standard: < 10 seconds
- [ ] 5km Detailed: < 60 seconds  
- [ ] 10km Standard: < 30 seconds
- [ ] 10km Detailed: < 120 seconds (was timeout)
- [ ] Radial preview (any radius): < 5 seconds

### Bug Fix Tests
- [ ] Progress bar updates from 5% → 100%
- [ ] App closes completely when clicking X
- [ ] Memory returns to baseline after calculation
- [ ] Western Ukraine region calculates successfully
- [ ] All terrain tiles validate correctly
- [ ] OSM cache files are valid JSON

### UI Tests
- [ ] Calculate button not overlapping scrollbar
- [ ] Dragging site moves all its sectors
- [ ] Deleting site removes all its sectors
- [ ] No console errors in browser DevTools

### Data Validation
- [ ] Run `diagnose_data_folders()` - check output
- [ ] Validate all terrain tiles in cache
- [ ] Validate all OSM cache files
- [ ] Check for corrupted files and remove them

---

## Build & Deploy

```powershell
# Backend
cd D:\root\rfcp\backend
pip install -e .

# Frontend
cd D:\root\rfcp\frontend
npm run build

# Electron
cd D:\root\rfcp\desktop
npm run build

# Full test
cd D:\root\rfcp\installer
.\test-detailed-quick.bat

# If works, rebuild installer
pyinstaller rfcp-server.spec --clean
```

---

## Commit Message

```
feat: Iteration 3.2.0 - Performance & Bug Fixes

Performance:
- Add adaptive resolution (distance-based grid density)
- Add radial preview mode (360 spokes, instant feedback)
- Expected: 10km Detailed ~60s (was timeout)

Bug Fixes:
- Fix progress bar stuck at 5% (WebSocket + React state)
- Fix app close button (Electron lifecycle)
- Fix memory leak (SharedMemory cleanup)
- Fix region data issues (cache validation)

UI Improvements:
- Move Calculate button (scrollbar overlap)
- Site drag moves all sectors
- Site delete removes all sectors

Data Validation:
- Add terrain tile validation
- Add OSM cache validation
- Add diagnostic reporting
```

---

## Success Criteria

1. **10km Detailed completes** without timeout (~60-120s acceptable)
2. **Progress bar works** - shows actual progress 5% → 100%
3. **App closes cleanly** - no orphan processes
4. **Memory released** - returns to baseline after calculation
5. **All regions work** - Western Ukraine calculates successfully
6. **Site management** - drag/delete affects sectors correctly

---

## Priority Order for Implementation

1. **Adaptive Resolution** - biggest performance impact
2. **Progress bar fix** - critical UX issue
3. **App close fix** - annoying bug
4. **Site drag/delete sectors** - quick win
5. **Calculate button position** - quick win
6. **Memory leak** - important but complex
7. **Region data validation** - diagnostic + fix
8. **Radial preview** - nice to have

---

## Notes for Claude Code

- This is a large iteration - take it step by step
- Test after each major change
- Backend and frontend changes may need coordination
- Electron changes require rebuild of desktop app
- Data validation can be added as debug endpoint first
- If stuck on one issue, move to next and come back

---

*"Big iteration, big impact. Let's make RFCP production-ready!"* 🚀