Shenron Radar Integration Guide (Fox Pipeline)

This guide details the "Model-Based Development" (MBD) integration of the physics-based Shenron radar simulation into the Fox CARLA ADAS pipeline.

1. Architecture Overview (MBD Standard)

In this approach, the Shenron simulator is treated as a strictly isolated model block (black box). Your main pipeline (recording/visualization) never touches the internal math or physics files. Instead, it interacts with a single Shenron Model API.

The MBD Interface:

Model Location: ISOLATE/
Public API: ShenronRadarModel (class)
Inputs: Canonical Semantic LiDAR array [N, 7].
Outputs: Radar Point Cloud [N, 4].

2. File System Setup (MBD Style)

2. File System Setup

To isolate the logic, we use the ISOLATE/ directory. Copy this entire folder into the root of your Fox/ repository.

Required Directory Structure:

Fox/
├── ISOLATE/                      <-- The Shenron Core
│   ├── e2e_agent_sem_lidar2shenron_package/
│   │   ├── shenron/              (Sceneset.py, heatmap_gen_fast.py)
│   │   ├── ConfigureRadar.py     (Hardware settings)
│   │   └── lidar.py              (Main conversion logic)
│   └── sim_radar_utils/          (FFT and BEV Processing)
├── src/
│   └── sensors.py                (Update to Semantic LiDAR)
├── scripts/
│   ├── generate_shenron.py       <-- NEW: Batch processor
│   └── data_to_mcap.py           (Update to include Shenron topic)
└── data/                         (Target dataset)

3. Step-by-Step Implementation

Step 1: Update LiDAR Sensor

In src/sensors.py, you must change the LiDAR blueprint to the semantic version. Shenron requires semantic tags (Road, Vehicle, Building) to apply radar reflection coefficients via Fresnel equations.

File: src/sensors.py

# Change this:
# 'type': 'sensor.lidar.ray_cast'
# To this:
'type': 'sensor.lidar.ray_cast_semantic'

Step 2: Implement the Batch Processor

Create scripts/generate_shenron.py. This script acts as a wrapper for the ISOLATE package. It iterates through your data/ directory, looks for lidar/ folders, and generates a corresponding shenron_radar/ folder.

[!TIP] Use the heatmap_gen_fast.py logic within the processor to leverage CUDA/GPU acceleration for the FMCW signal generation.

Step 3: Update the MCAP Writer

Modify scripts/data_to_mcap.py to recognize the new shenron_radar/ directory.

File: scripts/data_to_mcap.py

# Add a check for the shenron folder
shenron_path = session_dir / "shenron_radar"
if shenron_path.exists():
    # Write to Foxglove topic: "/radar/shenron"

4. Operational Workflow

To generate a dataset with high-accuracy radar, follow this sequence:

Run Simulation:
```
run.bat braking --frames 200
```
Generate Shenron Radar (Post-Simulation):
```
python scripts/generate_shenron.py
```
Convert to MCAP:
```
python scripts/data_to_mcap.py
```

5. Troubleshooting & Dependencies

Hardware Requirements

GPU: NVIDIA T1000 with 8GB VRAM is sufficient. The model uses PyTorch tensors to parallelize the FMCW chirp calculations across the GPU cores.
CPU/RAM: Standard multi-core CPU; 16GB+ RAM recommended for handling large point cloud buffers.

Installation

Execute these commands in your carla312 environment:

pip install torch
pip install open3d

Data Interface Adaptation

The most critical "surgical" modification occurs in ISOLATE/e2e_agent_sem_lidar2shenron_package/lidar.py.

Your CARLA 0.9.16 Semantic LiDAR data typically contains 7 columns: [x, y, z, intensity, cos_inc_angle, object_idx, semantic_tag]

The map_carla_semantic_lidar_latest function in lidar.py must be updated to index your specific columns:

def map_carla_semantic_lidar_latest(carla_sem_lidar_data):
    # Ensure indices match your [N, 7] array
    # e.g., if semantic_tag is index 6:
    carla_sem_lidar_data_crop = carla_sem_lidar_data[:, (0, 1, 2, 6)] 
    ...

The script scripts/generate_shenron.py will handle loading your .npy files and feeding them into this function.

4.4 KiB Raw Blame History