CARLA ? C-Shenron based Simualtor for Sensor data generation.

12 KiB

Raw Blame History

Project Context — Fox CARLA ADAS Simulation Pipeline

This document is the primary reference for AI agents and developers navigating this codebase. It covers project purpose, architecture, file-by-file roles, data flows, and extension patterns.

Project Purpose

A modular, scenario-driven simulation framework built on CARLA 0.9.16.

End-to-end pipeline:

CARLA Simulator → Multi-Sensor Capture → Dataset (PNG / NPY / JSONL)
               → MCAP Conversion → Foxglove Visualization

Primary goal: Demonstrate structured ADAS driving scenarios with synchronized, multi-modal sensor data that can be visualized and analysed in Foxglove Studio.

Repository Layout

Fox/
├── run.bat                  ← One-click launcher (activates carla312 conda env)
├── config.py                ← All tuneable constants (FPS, sensor params, scenario defaults)
├── data_to_mcap.py          ← Converts recorded dataset folders → .mcap files
├── data_inspector.py        ← Utility to inspect / debug recorded datasets
│
├── src/
│   ├── main.py              ← Orchestrator — scenario-agnostic entry point
│   ├── sensors.py           ← SensorManager (camera, radar, lidar setup + sync queues)
│   ├── recorder.py          ← Recorder (writes PNG / NPY / JSONL per frame)
│   └── scenario_loader.py   ← Dynamic scenario loader via importlib
│
├── scenarios/
│   ├── __init__.py          ← Package marker
│   ├── base.py              ← ScenarioBase abstract class (the plugin contract)
│   ├── braking.py           ← Lead vehicle hard braking scenario
│   ├── cutin.py             ← Adjacent lane cut-in scenario
│   └── obstacle.py          ← Static obstacle (traffic cone) scenario
│
├── data/                    ← Auto-created; one subfolder per recording session
│   └── <scenario>_YYYYMMDD_HHMMSS/
│       ├── camera/          ← frame_XXXXXX.png
│       ├── radar/           ← frame_XXXXXX.npy  (shape: [N, 4] — depth/az/alt/vel)
│       ├── lidar/           ← frame_XXXXXX.npy  (shape: [N, 4] — x/y/z/intensity)
│       └── frames.jsonl     ← One JSON record per frame (metadata + scenario info)
│
└── intel/
    └── context.md           ← This file

Key Files — Detailed Reference

`config.py`

Single source of truth for all constants. Every module imports from here via import config.

Key	Default	Purpose
`FPS`	20	Simulation tick rate
`DELTA_SECONDS`	0.05	CARLA fixed delta (= 1/FPS)
`CAM_WIDTH/HEIGHT/FOV`	1280×720, 90°	Camera resolution & field of view
`RADAR_HORIZONTAL_FOV`	120°	Radar angular sweep
`RADAR_RANGE`	100 m	Radar max range
`LIDAR_CHANNELS`	32	LiDAR beam count
`LIDAR_POINTS_PER_SECOND`	100 000	LiDAR density
`MAX_FRAMES`	200	Default run length
`DEFAULT_SCENARIO`	`"braking"`	Used when no `--scenario` flag provided
`SCENARIO_LEAD_DISTANCE`	25 m	Lead vehicle spawn distance (braking)
`SCENARIO_BRAKE_FRAME`	80	Frame at which lead brakes
`SCENARIO_CUTIN_DISTANCE`	15 m	NPC spawn distance (cutin)
`SCENARIO_CUTIN_FRAME`	60	Frame at which NPC changes lane
`SCENARIO_OBSTACLE_DISTANCE`	30 m	Cone spawn distance (obstacle)
`SCENARIO_OBSTACLE_PROP`	`static.prop.trafficcone01`	CARLA blueprint name

`src/main.py` — Orchestrator

Responsibilities: CARLA connection, ego spawn, sensor init, scenario load, main loop, shutdown. Does NOT contain any scenario-specific logic.

CLI:

python src/main.py --scenario braking
python src/main.py --scenario cutin --frames 120
python src/main.py --list-scenarios

Execution flow:

Parse args → load_scenario(name) → returns ScenarioBase instance
Connect CARLA (localhost:2000), enable sync mode at DELTA_SECONDS
Clear existing actors, spawn ego (vehicle.tesla.model3)
SensorManager.spawn_sensors() → attach camera / radar / lidar to ego
Recorder(scenario_name=scenario.name) → creates data/<scenario>_<ts>/
scenario.setup(world, ego, traffic_manager)
Main loop per frame: world.tick() → sensor_manager.get_data() → recorder.save(..., extra_meta=scenario.get_scenario_metadata()) → scenario.step(frame, ego)
finally: scenario.cleanup() → sensor_manager.destroy() → restore async mode

Key invariant: main.py never imports a scenario module by name. The CARLA import itself is deferred until after --list-scenarios early-exit so the dry-run works without a running CARLA server.

`src/sensors.py` — SensorManager

Manages three sensors attached to the ego vehicle. All sensors write into queue.Queue objects. get_data() blocks until one item from each queue is available, then asserts frame alignment.

Sensor	CARLA Blueprint	Attachment Point	Output
Camera	`sensor.camera.rgb`	x=1.5, z=2.4	BGRA image → `camera_queue`
Radar	`sensor.other.radar`	x=2.0, z=1.0	Detection list → `radar_queue`
LiDAR	`sensor.lidar.ray_cast`	x=0.0, z=2.5	Point buffer → `lidar_queue`

get_data() asserts cam.frame == radar.frame == lidar.frame — any mismatch raises immediately.

`src/recorder.py` — Recorder

Writes one frame's worth of data to disk each tick.

Output per frame:

camera/frame_XXXXXX.png — BGR image via OpenCV
radar/frame_XXXXXX.npy — float64 array [N, 4]: depth, azimuth, altitude, velocity
lidar/frame_XXXXXX.npy — float32 array [N, 4]: x, y, z, intensity
One line appended to frames.jsonl:

{
  "frame_id": 82,
  "timestamp": 1234.56,
  "ego_pose": {"x": 12.3, "y": 4.5, "z": 0.0, "yaw": -91.2},
  "ground_truth": [
    {
      "id": 123,
      "class": "vehicle",
      "type": "vehicle.tesla.model3",
      "transform": {"x": 10.5, "y": 2.1, "z": 0.5, "yaw": 90.0, ...},
      "velocity": {"vx": 5.0, "vy": 0.0, "vz": 0.0, "speed": 5.0},
      "acceleration": {"ax": 0.1, "ay": 0.0, "az": 0.0},
      "bounding_box": {"l": 4.5, "w": 2.0, "h": 1.5},
      "relative": {
        "range": 15.2,
        "azimuth": -2.5,
        "closing_velocity": 1.2
      }
    }
  ],
  "scenario": "braking",
  "brake_frame": 80
}

ADAS Relative Metrics:

range: Euclidean distance (m).
azimuth: Angle in ego-forward frame (degrees).
closing_velocity: Rate of approach (m/s). Positive means getting closer.

Scope: Now tracks both vehicle.* and walker.* (pedestrians).

extra_meta pattern: save() accepts extra_meta: dict which is merged into the record. Scenarios use get_scenario_metadata() to supply this — no recorder changes needed per scenario.

`src/scenario_loader.py` — Dynamic Loader

Uses importlib to load scenarios.<name> at runtime. Inspects the module for a concrete ScenarioBase subclass and returns an instance.

from scenario_loader import load_scenario, list_scenarios

scenario = load_scenario("braking")       # → BrakingScenario()
names    = list_scenarios()               # → ['braking', 'cutin', 'obstacle']

list_scenarios() uses pkgutil.iter_modules on the scenarios/ package — auto-discovers new files with no configuration changes.

`scenarios/base.py` — ScenarioBase (Abstract)

The plugin contract all scenarios must fulfil.

Required abstracts:      name (property), setup(), step(), cleanup()
Optional overrides:      on_ego_spawned(), get_scenario_metadata()
Shared helpers:          _destroy_actors(), _get_waypoint_ahead(distance, lane_offset)
Protected state:         self._world, self._ego, self._tm, self._actors (list)

_get_waypoint_ahead(distance, lane_offset):

lane_offset=0 → same lane as ego
lane_offset=1 → right lane
lane_offset=-1 → left lane
Returns carla.Waypoint or None

All NPC placement must go through this helper — never use hardcoded world coordinates.

Implemented Scenarios

File	Class	Trigger	Effect
`braking.py`	`BrakingScenario`	Frame 80	Lead vehicle (25 m ahead) applies emergency stop
`cutin.py`	`CutInScenario`	Frame 60	NPC in left lane (15 m ahead) forced into ego lane
`obstacle.py`	`ObstacleScenario`	None (static)	Traffic cone placed 30 m ahead on ego lane

All trigger frames and distances are driven by config.py constants.

`data_to_mcap.py` — MCAP Converter

Scans data/ for subfolders containing frames.jsonl and converts each to a .mcap file. Output is written as data/<session>/<session>.mcap (skips if already exists).

Foxglove topics produced:

Topic	Schema	Content
`/camera`	`foxglove.CompressedImage`	Base64-encoded PNG
`/lidar`	`foxglove.PointCloud`	X/Y/Z float32, Y-axis flipped for ROS convention
`/radar`	`foxglove.PointCloud`	Spherical → Cartesian conversion, Y-flipped
`/ego_pose`	`foxglove.Pose`	Position + quaternion from yaw angle

Coordinate system note: CARLA uses left-handed coords (Y increases right). The converter negates Y and yaw to match ROS/Foxglove right-handed convention.

Run:

python data_to_mcap.py

Processes all unprocessed session folders in data/ automatically.

Full Pipeline — End-to-End

1. CARLA server running (CarlaUE4.exe)
2. run.bat braking          → data/braking_<ts>/ (PNG + NPY + JSONL)
3. run.bat cutin            → data/cutin_<ts>/
4. run.bat obstacle         → data/obstacle_<ts>/
5. python data_to_mcap.py   → data/*/<session>.mcap
6. Open .mcap in Foxglove Studio

How to Add a New Scenario

Create scenarios/my_scenario.py
Subclass ScenarioBase, implement the four required members
Append any spawned actors to self._actors so _destroy_actors() handles cleanup
Use self._get_waypoint_ahead(d) for all NPC placement
Add config constants in config.py (optional but recommended)
Run python src/main.py --list-scenarios — it appears automatically

from scenarios.base import ScenarioBase

class MyScenario(ScenarioBase):

    @property
    def name(self): return "my_scenario"

    def setup(self, world, ego_vehicle, traffic_manager):
        self._world, self._ego, self._tm = world, ego_vehicle, traffic_manager
        wp = self._get_waypoint_ahead(20)
        npc = world.try_spawn_actor(bp, wp.transform)
        if npc:
            self._actors.append(npc)

    def step(self, frame, ego_vehicle):
        if frame == 100:
            pass  # trigger event

    def cleanup(self):
        self._destroy_actors()

    def get_scenario_metadata(self):
        return {"scenario": self.name}

Environment & Dependencies

Item	Value
CARLA version	0.9.16
Python environment	conda env `carla312` (miniconda)
Activation	`run.bat` calls `activate.bat carla312` automatically
Key Python packages	`carla`, `numpy`, `opencv-python` (`cv2`), `mcap`
CARLA server address	`localhost:2000` (hardcoded in `main.py`)
Traffic Manager port	`8000` (hardcoded in `main.py`)

Known Limitations & Future Work

Area	Status	Notes
MCAP encoding	JSON (functional)	Should migrate to Protobuf/typed schemas for performance
Intersection scenario	Not implemented	Needs `waypoint.is_junction` awareness
Scenario parameterization	Config-based	Future: CLI `--param key=val` support
Foxglove layouts	Manual	Future: `.json` layout presets per scenario
Multi-ego support	Not implemented	Single ego vehicle assumed throughout
Radar Foxglove schema	Re-uses PointCloud	Correct but non-semantic; dedicated radar schema planned

Last updated: 2026-03-27 | Pipeline version: modular scenario-driven (post-refactor)

12 KiB Raw Blame History