# Shenron Radar Integration Guide (Fox Pipeline) This guide details the "Model-Based Development" (MBD) integration of the physics-based Shenron radar simulation into the Fox CARLA ADAS pipeline. --- ## 1. Architecture Overview (MBD Standard) In this approach, the Shenron simulator is treated as a **strictly isolated model block** (black box). Your main pipeline (recording/visualization) never touches the internal math or physics files. Instead, it interacts with a single **Shenron Model API**. **The MBD Interface:** - **Model Location**: `ISOLATE/` - **Public API**: `ShenronRadarModel` (class) - **Inputs**: Canonical Semantic LiDAR array `[N, 7]`. - **Outputs**: Radar Point Cloud `[N, 4]`. --- ## 2. File System Setup (MBD Style) ## 2. File System Setup To isolate the logic, we use the `ISOLATE/` directory. Copy this entire folder into the root of your `Fox/` repository. **Required Directory Structure:** ```text Fox/ ├── ISOLATE/ <-- The Shenron Core │ ├── e2e_agent_sem_lidar2shenron_package/ │ │ ├── shenron/ (Sceneset.py, heatmap_gen_fast.py) │ │ ├── ConfigureRadar.py (Hardware settings) │ │ └── lidar.py (Main conversion logic) │ └── sim_radar_utils/ (FFT and BEV Processing) ├── src/ │ └── sensors.py (Update to Semantic LiDAR) ├── scripts/ │ ├── generate_shenron.py <-- NEW: Batch processor │ └── data_to_mcap.py (Update to include Shenron topic) └── data/ (Target dataset) ``` --- ## 3. Step-by-Step Implementation ### Step 1: Update LiDAR Sensor In `src/sensors.py`, you must change the LiDAR blueprint to the **semantic** version. Shenron requires semantic tags (Road, Vehicle, Building) to apply radar reflection coefficients via Fresnel equations. **File:** [`src/sensors.py`](file:///d:/Work/Repo/C-Shenron/src/sensors.py) ```python # Change this: # 'type': 'sensor.lidar.ray_cast' # To this: 'type': 'sensor.lidar.ray_cast_semantic' ``` ### Step 2: Implement the Batch Processor Create `scripts/generate_shenron.py`. This script acts as a wrapper for the `ISOLATE` package. It iterates through your `data/` directory, looks for `lidar/` folders, and generates corresponding hardware folders (e.g. `awrl1432/`, `radarbook/`). > [!TIP] > Use the **`heatmap_gen_fast.py`** logic within the processor to leverage CUDA/GPU acceleration for the FMCW signal generation. ### Step 3: Update the MCAP Writer Modify `scripts/data_to_mcap.py` to recognize the new hardware directories. **File:** [`scripts/data_to_mcap.py`](file:///d:/Work/Repo/C-Shenron/scripts/data_to_mcap.py) ```python # Loop over hardware profiles for r_type in ['awrl1432', 'radarbook']: shenron_path = session_dir / r_type if shenron_path.exists(): # Write to Foxglove topic: "/radar/{r_type}" ``` --- ## 4. Operational Workflow To generate a dataset with high-accuracy radar, follow this sequence: 1. **Run Simulation**: ```powershell run.bat braking --frames 200 ``` 2. **Generate Shenron Radar** (Post-Simulation): ```powershell python scripts/generate_shenron.py ``` 3. **Convert to MCAP**: ```powershell python scripts/data_to_mcap.py ``` --- ## 5. Troubleshooting & Dependencies ### Hardware Requirements - **GPU**: NVIDIA T1000 with 8GB VRAM is sufficient. The model uses PyTorch tensors to parallelize the FMCW chirp calculations across the GPU cores. - **CPU/RAM**: Standard multi-core CPU; 16GB+ RAM recommended for handling large point cloud buffers. ### Installation Execute these commands in your `carla312` environment: ```powershell pip install torch pip install open3d ``` ### Data Interface Adaptation The most critical "surgical" modification occurs in **`ISOLATE/e2e_agent_sem_lidar2shenron_package/lidar.py`**. Your CARLA 0.9.16 Semantic LiDAR data typically contains 7 columns: `[x, y, z, intensity, cos_inc_angle, object_idx, semantic_tag]` The `map_carla_semantic_lidar_latest` function in `lidar.py` must be updated to index your specific columns: ```python def map_carla_semantic_lidar_latest(carla_sem_lidar_data): # Ensure indices match your [N, 7] array # e.g., if semantic_tag is index 6: carla_sem_lidar_data_crop = carla_sem_lidar_data[:, (0, 1, 2, 6)] ... ``` The script `scripts/generate_shenron.py` will handle loading your `.npy` files and feeding them into this function.