# Project Context — Fox CARLA ADAS Simulation Pipeline

> This document is the primary reference for AI agents and developers navigating this codebase.
> It covers project purpose, architecture, file-by-file roles, data flows, and extension patterns.

---

## Project Purpose

A modular, scenario-driven simulation framework built on CARLA 0.9.16.

**End-to-end pipeline:**
```
CARLA Simulator → Multi-Sensor Capture → Dataset (PNG / NPY / JSONL)
               → MCAP Conversion → Foxglove Visualization
```

**Primary goal:** Demonstrate structured ADAS driving scenarios with synchronized,
multi-modal sensor data that can be visualized and analysed in Foxglove Studio.

---

## Repository Layout

```
Fox/
├── run.bat                  ← One-click launcher (activates carla312 conda env)
├── config.py                ← All tuneable constants (FPS, sensor params, scenario defaults)
├── data_to_mcap.py          ← Converts recorded dataset folders → .mcap files
├── data_inspector.py        ← Utility to inspect / debug recorded datasets
│
├── src/
│   ├── main.py              ← Orchestrator — scenario-agnostic entry point
│   ├── sensors.py           ← SensorManager (camera, radar, lidar setup + sync queues)
│   ├── recorder.py          ← Recorder (writes PNG / NPY / JSONL per frame)
│   └── scenario_loader.py   ← Dynamic scenario loader via importlib
│
├── scenarios/
│   ├── __init__.py          ← Package marker
│   ├── base.py              ← ScenarioBase abstract class (the plugin contract)
│   ├── braking.py           ← Lead vehicle hard braking scenario
│   ├── cutin.py             ← Adjacent lane cut-in scenario
│   └── obstacle.py          ← Static obstacle (traffic cone) scenario
│
├── data/                    ← Auto-created; one subfolder per recording session
│   └── <scenario>_YYYYMMDD_HHMMSS/
│       ├── camera/          ← frame_XXXXXX.png
│       ├── radar/           ← frame_XXXXXX.npy  (shape: [N, 4] — depth/az/alt/vel)
│       ├── lidar/           ← frame_XXXXXX.npy  (shape: [N, 4] — x/y/z/intensity)
│       └── frames.jsonl     ← One JSON record per frame (metadata + scenario info)
│
└── intel/
    └── context.md           ← This file
```

---

## Key Files — Detailed Reference

### `config.py`
Single source of truth for all constants. Every module imports from here via `import config`.

| Key | Default | Purpose |
|---|---|---|
| `FPS` | 20 | Simulation tick rate |
| `DELTA_SECONDS` | 0.05 | CARLA fixed delta (= 1/FPS) |
| `CAM_WIDTH/HEIGHT/FOV` | 1280×720, 90° | Camera resolution & field of view |
| `RADAR_HORIZONTAL_FOV` | 120° | Radar angular sweep |
| `RADAR_RANGE` | 100 m | Radar max range |
| `LIDAR_CHANNELS` | 32 | LiDAR beam count |
| `LIDAR_POINTS_PER_SECOND` | 100 000 | LiDAR density |
| `MAX_FRAMES` | 200 | Default run length |
| `DEFAULT_SCENARIO` | `"braking"` | Used when no `--scenario` flag provided |
| `SCENARIO_LEAD_DISTANCE` | 25 m | Lead vehicle spawn distance (braking) |
| `SCENARIO_BRAKE_FRAME` | 80 | Frame at which lead brakes |
| `SCENARIO_CUTIN_DISTANCE` | 15 m | NPC spawn distance (cutin) |
| `SCENARIO_CUTIN_FRAME` | 60 | Frame at which NPC changes lane |
| `SCENARIO_OBSTACLE_DISTANCE` | 30 m | Cone spawn distance (obstacle) |
| `SCENARIO_OBSTACLE_PROP` | `static.prop.trafficcone01` | CARLA blueprint name |

---

### `src/main.py` — Orchestrator
**Responsibilities:** CARLA connection, ego spawn, sensor init, scenario load, main loop, shutdown.
**Does NOT contain any scenario-specific logic.**

**CLI:**
```
python src/main.py --scenario braking
python src/main.py --scenario cutin --frames 120
python src/main.py --list-scenarios
```

**Execution flow:**
1. Parse args → `load_scenario(name)` → returns `ScenarioBase` instance
2. Connect CARLA (`localhost:2000`), enable sync mode at `DELTA_SECONDS`
3. Clear existing actors, spawn ego (`vehicle.tesla.model3`)
4. `SensorManager.spawn_sensors()` → attach camera / radar / lidar to ego
5. `Recorder(scenario_name=scenario.name)` → creates `data/<scenario>_<ts>/`
6. `scenario.setup(world, ego, traffic_manager)`
7. **Main loop** per frame: `world.tick()` → `sensor_manager.get_data()` →
   `recorder.save(..., extra_meta=scenario.get_scenario_metadata())` → `scenario.step(frame, ego)`
8. `finally`: `scenario.cleanup()` → `sensor_manager.destroy()` → restore async mode

> **Key invariant:** `main.py` never imports a scenario module by name.
> The CARLA import itself is deferred until after `--list-scenarios` early-exit so the dry-run
> works without a running CARLA server.

---

### `src/sensors.py` — SensorManager
Manages three sensors attached to the ego vehicle. All sensors write into `queue.Queue` objects.
`get_data()` blocks until one item from each queue is available, then asserts frame alignment.

| Sensor | CARLA Blueprint | Attachment Point | Output |
|---|---|---|---|
| Camera | `sensor.camera.rgb` | x=1.5, z=2.4 | BGRA image → `camera_queue` |
| Radar | `sensor.other.radar` | x=2.0, z=1.0 | Detection list → `radar_queue` |
| LiDAR | `sensor.lidar.ray_cast` | x=0.0, z=2.5 | Point buffer → `lidar_queue` |

> `get_data()` asserts `cam.frame == radar.frame == lidar.frame` — any mismatch raises immediately.

---

### `src/recorder.py` — Recorder
Writes one frame's worth of data to disk each tick.

**Output per frame:**
- `camera/frame_XXXXXX.png` — BGR image via OpenCV
- `radar/frame_XXXXXX.npy` — `float64` array `[N, 4]`: depth, azimuth, altitude, velocity
- `lidar/frame_XXXXXX.npy` — `float32` array `[N, 4]`: x, y, z, intensity
- One line appended to `frames.jsonl`:

```json
{
  "frame_id": 82,
  "timestamp": 1234.56,
  "scenario": "braking",
  "brake_frame": 80,
  "braked": true,
  "ego_pose": {"x": 12.3, "y": 4.5, "z": 0.0, "yaw": -91.2},
  "camera": "frame_000082.png",
  "radar":  "frame_000082.npy",
  "lidar":  "frame_000082.npy",
  "ground_truth": [{"id": 42, "x": ..., "speed": 8.3}]
}
```

**`extra_meta` pattern:** `save()` accepts `extra_meta: dict` which is merged into the record.
Scenarios use `get_scenario_metadata()` to supply this — no recorder changes needed per scenario.

---

### `src/scenario_loader.py` — Dynamic Loader
Uses `importlib` to load `scenarios.<name>` at runtime. Inspects the module for a concrete
`ScenarioBase` subclass and returns an instance.

```python
from scenario_loader import load_scenario, list_scenarios

scenario = load_scenario("braking")       # → BrakingScenario()
names    = list_scenarios()               # → ['braking', 'cutin', 'obstacle']
```

`list_scenarios()` uses `pkgutil.iter_modules` on the `scenarios/` package — auto-discovers
new files with no configuration changes.

---

### `scenarios/base.py` — ScenarioBase (Abstract)
The **plugin contract** all scenarios must fulfil.

```
Required abstracts:      name (property), setup(), step(), cleanup()
Optional overrides:      on_ego_spawned(), get_scenario_metadata()
Shared helpers:          _destroy_actors(), _get_waypoint_ahead(distance, lane_offset)
Protected state:         self._world, self._ego, self._tm, self._actors (list)
```

`_get_waypoint_ahead(distance, lane_offset)`:
- `lane_offset=0`  → same lane as ego
- `lane_offset=1`  → right lane
- `lane_offset=-1` → left lane
- Returns `carla.Waypoint` or `None`

**All NPC placement must go through this helper** — never use hardcoded world coordinates.

---

### Implemented Scenarios

| File | Class | Trigger | Effect |
|---|---|---|---|
| `braking.py` | `BrakingScenario` | Frame 80 | Lead vehicle (25 m ahead) applies emergency stop |
| `cutin.py` | `CutInScenario` | Frame 60 | NPC in left lane (15 m ahead) forced into ego lane |
| `obstacle.py` | `ObstacleScenario` | None (static) | Traffic cone placed 30 m ahead on ego lane |

All trigger frames and distances are driven by `config.py` constants.

---

### `data_to_mcap.py` — MCAP Converter
Scans `data/` for subfolders containing `frames.jsonl` and converts each to a `.mcap` file.
Output is written as `data/<session>/<session>.mcap` (skips if already exists).

**Foxglove topics produced:**

| Topic | Schema | Content |
|---|---|---|
| `/camera` | `foxglove.CompressedImage` | Base64-encoded PNG |
| `/lidar` | `foxglove.PointCloud` | X/Y/Z float32, Y-axis flipped for ROS convention |
| `/radar` | `foxglove.PointCloud` | Spherical → Cartesian conversion, Y-flipped |
| `/ego_pose` | `foxglove.Pose` | Position + quaternion from yaw angle |

> **Coordinate system note:** CARLA uses left-handed coords (Y increases right).
> The converter negates Y and yaw to match ROS/Foxglove right-handed convention.

**Run:**
```
python data_to_mcap.py
```
Processes all unprocessed session folders in `data/` automatically.

---

## Full Pipeline — End-to-End

```
1. CARLA server running (CarlaUE4.exe)
2. run.bat braking          → data/braking_<ts>/ (PNG + NPY + JSONL)
3. run.bat cutin            → data/cutin_<ts>/
4. run.bat obstacle         → data/obstacle_<ts>/
5. python data_to_mcap.py   → data/*/<session>.mcap
6. Open .mcap in Foxglove Studio
```

---

## How to Add a New Scenario

1. Create `scenarios/my_scenario.py`
2. Subclass `ScenarioBase`, implement the four required members
3. Append any spawned actors to `self._actors` so `_destroy_actors()` handles cleanup
4. Use `self._get_waypoint_ahead(d)` for all NPC placement
5. Add config constants in `config.py` (optional but recommended)
6. Run `python src/main.py --list-scenarios` — it appears automatically

```python
from scenarios.base import ScenarioBase

class MyScenario(ScenarioBase):

    @property
    def name(self): return "my_scenario"

    def setup(self, world, ego_vehicle, traffic_manager):
        self._world, self._ego, self._tm = world, ego_vehicle, traffic_manager
        wp = self._get_waypoint_ahead(20)
        npc = world.try_spawn_actor(bp, wp.transform)
        if npc:
            self._actors.append(npc)

    def step(self, frame, ego_vehicle):
        if frame == 100:
            pass  # trigger event

    def cleanup(self):
        self._destroy_actors()

    def get_scenario_metadata(self):
        return {"scenario": self.name}
```

---

## Environment & Dependencies

| Item | Value |
|---|---|
| CARLA version | 0.9.16 |
| Python environment | conda env `carla312` (miniconda) |
| Activation | `run.bat` calls `activate.bat carla312` automatically |
| Key Python packages | `carla`, `numpy`, `opencv-python` (`cv2`), `mcap` |
| CARLA server address | `localhost:2000` (hardcoded in `main.py`) |
| Traffic Manager port | `8000` (hardcoded in `main.py`) |

---

## Known Limitations & Future Work

| Area | Status | Notes |
|---|---|---|
| MCAP encoding | JSON (functional) | Should migrate to Protobuf/typed schemas for performance |
| Intersection scenario | Not implemented | Needs `waypoint.is_junction` awareness |
| Scenario parameterization | Config-based | Future: CLI `--param key=val` support |
| Foxglove layouts | Manual | Future: `.json` layout presets per scenario |
| Multi-ego support | Not implemented | Single ego vehicle assumed throughout |
| Radar Foxglove schema | Re-uses PointCloud | Correct but non-semantic; dedicated radar schema planned |

---

*Last updated: 2026-03-27 | Pipeline version: modular scenario-driven (post-refactor)*