Shenron Physics-Based Radar: Debug & Troubleshooting Plan

This document serves as a comprehensive guide for diagnosing and resolving the data-fidelity issues in the C-SHENRON radar integration. While the pipeline is functional, the resulting point clouds currently do not accurately reflect the scene geometry or velocity profiles.

🕵️ 1. Immediate Red Flags (Potential Root Causes)

Through architectural review, the following critical issues have been identified:

⚡ A. The "Zero Velocity" Bug

In the current implementation of lidar.py and generate_shenron.py, the Doppler/Velocity channel is currently hardcoded to 0.

Found in: lidar.py:L164-165 (commented out) and L209 (rt_speed = np.zeros_like(rt_rho)).
Impact: The radar model receives no radial velocity information, resulting in static-only heatmaps and incorrect detector peaks.
Fix Required: Extract the velocity of hit actors in CARLA and pass it into the points[:, 3] channel before synthesis.

📐 B. Semantic LiDAR Column Mismatch

The bit-casting logic assumes a specific 6-column layout: [x, y, z, cos, obj, tag].

The Issue: CARLA 0.9.16 ray_cast_semantic raw data is often:
- [0:x, 1:y, 2:z, 3:CosInclinationAngle, 4:ObjectIndex, 5:Tag]
Found in: lidar.py:L149-168. If we are reading ObjectIndex as Cosines, the reflectivity (loss) calculation will be garbage.
Fix Required: Verify column indices in src/recorder.py and ensure lidar.py maps them correctly.

🔄 C. Coordinate Transformation Logic

In lidar.py:L156, indices are swapped: points[:, [0, 1, 2]] = test[:, [1, 0, 2]].

The Risk: This transformation (X->Y, Y->X) might not align with the theta (azimuth) calculation or the internal coordinate system of the Sceneset occlusion removal logic.
Check: Compare CARLA's forward vector (X) with Shenron's expected forward vector (often Y in legacy codebases).

🛠️ 2. Step-by-Step Debugging Checklist

Phase 1: Data Integrity (LiDAR to Input)

Visualize Raw Semantic LiDAR: Use a script to plot the Saved .npy files from the lidar/ folder. Ensure the "Materials" (Tag field) are distinct (Vehicles vs Road vs Pedestrians).
Verify Bit-Casting: Run scripts/verify_tags.py. Check if the unique tags recovered match the scene (e.g., if there's a car, Tag 10 must exist).

Phase 2: Radar Hardare Sync

Check Config.yaml: Inspect scripts/ISOLATE/sim_radar_utils/config.yaml. Are the samp_rate, chirpT, and gain realistic for an ADAS radar (e.g., 77GHz)?
Validate model_wrapper.py: Ensure the _sync_configs method correctly updates the underlying radar object.

Phase 3: Physics Engine (ISOLATE/shenron)

Occlusion Check: In Sceneset.py, check the removeocclusion_hiddenpoint method. Is it being too aggressive and deleting the ego-vehicle's own target (the car in front)?
Phase Summation: In heatmap_gen_fast.py, ensure the complex exponentiation torch.exp(1j * ...) preserves the phase shift related to target distance (rho) and velocity (speed).

🛰️ 3. Relevant Files to Investigate

scripts/ISOLATE/e2e_agent_sem_lidar2shenron_package/lidar.py: The bridge between recorded data and the physics model. (High Priority)
scripts/generate_shenron.py: The parallel orchestrator. (Medium Priority)
scripts/ISOLATE/e2e_agent_sem_lidar2shenron_package/shenron/heatmap_gen_fast.py: The core signal generator. (High Priority)
scripts/ISOLATE/model_wrapper.py: The API layer and config sync unit. (Medium Priority)

💡 4. Context for the Next Session

We have successfully built a pipeline that:

Records Semantic LiDAR from CARLA.
Bit-casts raw bytes into material tags.
Feed those tags into a physics-based radar simulator.
Serializes the results into MCAP (Rich PCD: X, Y, Z, V, Mag).

The Goal of the next session is to find why the "Rich" data doesn't look like the actual scene and fix the Doppler/Visibility fidelity.

4.0 KiB Raw Blame History