# 🚀 Auto MCAP & SHENRON Integration Architecture

This document describes the automated pipeline for generating, processing, and visualizing high-fidelity radar data within the Fox CARLA Simulation project.

---

## 🏗️ 1. The Three-Stage Pipeline

Every time a simulation is triggered (e.g., via `run.bat`), the system follows a deterministic three-stage process:

### Stage 1: Live Simulation (`src/main.py`)
- **Objective:** Generate raw multi-modal sensor data from the CARLA simulator.
- **Output:** `data/<session>/lidar/*.npy`, `camera/*.png`, and `frames.jsonl`.
- **Key Feature:** LiDAR is saved with **Semantic Tags** and **Radial Velocity** calculations, which are essential inputs for the physics-based radar model.

### Stage 2: Multi-Radar Synthesis (`scripts/generate_shenron.py`)
- **Objective:** Convert LiDAR point clouds into high-fidelity radar signals for multiple hardware profiles.
- **Logic:** Calls `ShenronRadarModel` concurrently for **awrl1432** and **radarbook**.
- **Improved Output:**
    - `awrl1432/` & `radarbook/`: Dedicated synthesis folders per radar type.
    - `{r_type}/metrology/`: Nested diagnostic heatmaps (RD, RA, CFAR).
    - `{r_type}/metrology/metrics.jsonl`: Automated **Telemetry** records (SNR, Noise Floor).
- **Physics Constraint:** All signals follow the pure **1/R⁴ Radar Range Equation** with hardware-specific gain offsets.

### Stage 3: MCAP Conversion (`scripts/data_to_mcap.py`)
- **Objective:** Package all hardware profiles and metrics into a unified Foxglove log.
- **Topic Alignment:** Follows the standardized `test_shenron` namespace:
    - `/radar/native`: Raw CARLA radar.
    - `/radar/awrl1432/...`: TI-style radar data and metrology.
    - `/radar/radarbook/...`: Logic-style radar data and metrology.
- **Visual Enhancement:** Implements **Polar Scan Conversion** for RA maps and publishes **Telemetry** curves for live signal monitoring.

---

## 🛰️ 2. Core Metrology & Telemetry

### A. Signal Quality Metrics (Telemetry)
The simulation now tracks real-time signal health using the `foxglove.Telemetry` schema.
- **Peak SNR (dB):** Tracks the separation between the strongest reflection and the noise floor.
- **Avg Noise Floor:** Monitors ambient thermal and phase noise in the RD spectrum.
- **Active Bins:** Counts detections surviving the CFAR threshold.

### B. Doppler-Slice RA Accumulation
To prevent angular smearing, the Range-Azimuth map is generated by looping over Doppler bins.
1. Each bin is processed via a **Windowed Angle FFT**.
2. **Phase Preservation:** Magnitude is never taken before the FFT. Complex inputs ensure beamforming works.
3. Power is summed **Incoherently** across Doppler bins *after* the FFT.

---

## 📁 3. Data Structure Reference

A successful automated run will produce the following nested structure:

```text
data/
└── scenario_timestamp/
    ├── frames.jsonl          # Metadata & Synchronization
    ├── lidar/                # Semantic LiDAR [N, 7]
    ├── camera/               # Dashcam imagery
    ├── awrl1432/             # TI 1432 Radar PCD & Metrology
    │   └── metrology/
    │       ├── metrics.jsonl # Telemetry logs
    │       ├── rd/           # RD Heatmaps
    │       └── ...
    ├── radarbook/            # RadarBook PCD & Metrology
    │   └── metrology/
    │       └── ...
    └── scenario_timestamp.mcap # Final Unified Log
```

---

## 🧭 4. Interpretation Guide for AI Agents

When evaluating a session, AI agents should look for the following "Success Indicators":

1.  **Hardware Parity:** Compare `/radar/awrl1432` vs `/radar/radarbook`. Differences in range/velocity resolution should match the physical hardware specs.
2.  **Telemetry Health:** In Foxglove, the **Telemetry** panel should show a stable SNR curve. Sudden drops suggest occlusion or sensor saturation.
3.  **Target Alignment:** The "Jet Spectrum" blobs in the RA Sector plot must spatially align with the "White Points" in the 3D LiDAR point cloud.
4.  **V=0 Vertical:** Static geometry in the RD heatmap must form a sharp vertical line at the 0 m/s mark, confirming clutter removal is working.

---
*Documented by Antigravity | Project: Fox CARLA ADAS | 2026-04-09*