# Shenron-Style Radar Integration Plan (CARLA → Foxglove Pipeline)

## Objective

Implement a **C-Shenron-inspired radar synthesis module** within the existing CARLA ADAS pipeline to generate a **high-fidelity radar point cloud and heatmap**, while **retaining CARLA’s native radar** for benchmarking and calibration.

---

# 1. Why Shenron Radar (Core Improvements Over CARLA Radar)

## Limitations of CARLA Radar

* Sparse ray-cast sampling (low spatial density)
* No Radar Cross Section (RCS) modeling
* No intensity / energy modeling
* No angular resolution control
* No beam pattern or antenna simulation
* No realistic clustering or spread
* Outputs discrete points → not field-based representation

---

## Shenron Key Ideas (What Makes It Better)

These are the **core principles you must implement**:

### 1. Dense Geometry-Based Sampling

* Uses **LiDAR / depth data** instead of sparse rays
* Captures full object surfaces → better shape fidelity

---

### 2. Radar Cross Section (RCS) Approximation

* Assign reflectivity based on object type
* Vehicles ≫ pedestrians ≫ static clutter

---

### 3. Energy-Based Modeling (Not Binary Hits)

* Each point contributes **intensity**
* Follows simplified radar equation:

```
I ∝ σ / R⁴
```

---

### 4. Range-Azimuth Grid Representation

* Converts scene into **radar image space**
* Aggregates energy into bins instead of discrete hits

---

### 5. Angular Resolution Simulation

* Models antenna array limitations:

```
Δθ ≈ 2 / N_antennas
```

* Applies angular blur → realistic radar spread

---

### 6. Doppler Field (Not Per-Ray Velocity)

* Computes **radial velocity per cluster/bin**
* Produces realistic motion patterns

---

### 7. Multi-Sensor Fusion Capability

* Supports multiple radar views (F, B, L, R)
* Improves situational awareness

---

### 8. Field Representation Instead of Hit Detection

* CARLA: “Ray hit object”
* Shenron: “Energy returned from direction”

---

# 2. Integration Strategy (Your Pipeline)

## Current Pipeline

```
CARLA → SensorManager → Recorder → Dataset → MCAP → Foxglove
```

---

## Updated Pipeline

```
CARLA
  ↓
SensorManager (Camera + LiDAR + Native Radar)
  ↓
RadarSynthesizer (NEW)
  ↓
Recorder
  ├── /radar_carla      (existing)
  ├── /radar_shenron    (NEW)
  ↓
MCAP → Foxglove
```

---

## Key Design Decision

* DO NOT remove CARLA radar
* Add **parallel synthetic radar**
* Enables:

  * A/B comparison
  * Parameter tuning
  * Real-world calibration

---

# 3. Implementation Plan (Step-by-Step)

---

## STEP 1 — Create RadarSynthesizer Module

**File:**

```
src/radar_synthesizer.py
```

**Interface:**

```python
class RadarSynthesizer:
    def __init__(self, config):
        pass

    def generate(self, lidar_points, ego_state, gt_objects):
        return radar_points
```

---

## STEP 2 — Input Data Sources

From your pipeline:

* LiDAR → `[N, 4] (x, y, z, intensity)`
* Ground Truth → object labels, velocity, bounding boxes
* Ego state → pose + velocity

---

## STEP 3 — Coordinate Transformation

Convert LiDAR points to radar space:

```
R = sqrt(x² + y² + z²)
θ = atan2(y, x)
φ = asin(z / R)
```

---

## STEP 4 — Object Association

For each LiDAR point:

* Assign it to nearest bounding box
* Extract:

  * object class
  * velocity

Output:

```
point → {x, y, z, object_type, velocity}
```

---

## STEP 5 — RCS Modeling

Define:

```python
RCS_MAP = {
    "vehicle": 10.0,
    "walker": 1.0,
    "static": 0.3
}
```

---

## STEP 6 — Intensity Calculation

Apply simplified radar equation:

```
I = σ / R⁴
```

Optional extensions:

* Angle attenuation
* Surface normal weighting

---

## STEP 7 — Range-Azimuth Grid Formation

Discretize:

```python
range_bins = 256
angle_bins = 128
```

Accumulate:

```python
grid[r_bin][theta_bin] += intensity
```

---

## STEP 8 — Angular Resolution Simulation

Define antenna count:

```python
N_antennas = 64
```

Apply blur:

```python
sigma_theta = k / N_antennas
```

Use Gaussian smoothing across angle axis.

---

## STEP 9 — Doppler Modeling

Compute radial velocity:

```
v_r = v_rel · r_hat
```

Assign:

* Per grid cell
* Or per cluster

---

## STEP 10 — Thresholding & Point Cloud Generation

Convert grid back:

```python
if grid[r][θ] > threshold:
    x = r * cos(θ)
    y = r * sin(θ)
    z = 0
```

Attach velocity.

---

## STEP 11 — Recorder Integration

Modify `recorder.py`:

Add new topic:

```
/radar_shenron
```

Format:

```
[N, 4] → x, y, z, velocity
```

---

## STEP 12 — MCAP Integration

Update `data_to_mcap.py`:

Add:

| Topic            | Schema     |
| ---------------- | ---------- |
| `/radar_shenron` | PointCloud |

---

## STEP 13 — Foxglove Visualization

Display:

* `/radar_carla`
* `/radar_shenron`

Side-by-side comparison:

* Density
* Spread
* Stability

---

# 4. Development Phases

---

## Phase 1 — Offline Prototype

* Use saved LiDAR logs
* Generate radar output
* Validate visually

---

## Phase 2 — Online Integration

* Plug into SensorManager
* Generate per frame

---

## Phase 3 — Calibration Mode

* Compare:

  * CARLA radar
  * Shenron radar
* Tune:

  * RCS values
  * noise
  * thresholds

---

## Phase 4 — Real-World Matching (Future)

* Match with real radar logs
* Fit:

  * noise distribution
  * angular spread
  * detection density

---

# 5. Performance Considerations

* Use NumPy vectorization
* Avoid Python loops
* Keep grid size moderate
* Consider async processing if needed

---

# 6. Optional Enhancements

---

## 6.1 Noise Model

```
range_noise ~ N(0, σ_r)
velocity_noise ~ N(0, σ_v)
```

---

## 6.2 Clutter Simulation

* Ground reflections
* Random scatter

---

## 6.3 Occlusion Modeling

* Use LiDAR density
* Drop points behind objects

---

## 6.4 Temporal Persistence

* Retain previous detections
* Apply decay

---

# 7. Key Engineering Insight

> The biggest shift is:

**From:**

* Discrete ray-hit detection

**To:**

* Continuous energy field representation

---

# 8. Expected Outcomes

Compared to CARLA radar:

| Feature         | CARLA   | Shenron-style |
| --------------- | ------- | ------------- |
| Density         | Low     | High          |
| Clustering      | Poor    | Realistic     |
| Angular spread  | None    | Modeled       |
| Physics realism | Low     | Medium        |
| ML usefulness   | Limited | High          |

---

# 9. Final Recommendation

Start with a **minimal MVP**:

1. LiDAR → spherical
2. Add RCS scaling
3. Bin into grid
4. Convert to point cloud

Then iterate.

---

# 10. Next Step

* Implement `radar_synthesizer.py`
* Validate offline
* Integrate into pipeline

---

**End of Document**