Fox CARLA ADAS Simulation Project: Agent Intelligence (gemini.md)

This document is the consolidated source of truth for AI agents working on the Fox CARLA Simulation project. It combines high-level architecture, component-level details, and best practices.

🏗️ 1. Project Overview & Architecture

Objective: A modular, scenario-driven simulation framework built on CARLA 0.9.16 for demonstrating structured ADAS driving scenarios with multi-modal sensor data.

The Pipeline: CARLA Simulator → Multi-Sensor Capture → Dataset (PNG/NPY/JSONL) → MCAP Conversion → Foxglove Visualization

Repository Layout

dashboard.bat: One-click launcher for the Flask GUI orchestrator.
run.bat: One-click runner for the simulation (activates carla312 conda env).
config.py: Global simulation constants (FPS, sensor specs, ego model).
src/main.py: Scenario-agnostic orchestrator and entry point.
src/sensors.py: SensorManager — Handles camera, radar, and lidar sync.
src/recorder.py: Recorder — Serializes raw frames to disk asynchronously.
scenarios/: Contains scenario implementations (e.g., braking.py, cutin.py, showcase.py).
scripts/: Utility scripts for data processing (e.g., data_to_mcap.py).
dashboard/: Flask backend and static web assets for the GUI.
intel/: Detailed deep-dive documentation for specific components.

🛠️ 2. Component Reference

`src/main.py` (The Orchestrator)

Role: Handles CARLA connection, ego spawning, sensor initialization, and main simulation loop.
Rules: Never import a scenario by name. Uses scenario_loader.py to dynamically load classes.
Workflow: Parse CLI args → Initialize ScenarioBase subclass → Tick world once to settle physics → scenario.setup() → Loop world.tick(), recorder.save(), scenario.step() → scenario.cleanup().

`src/sensors.py` (SensorManager)

Manages Camera (RGB), Radar, and LiDAR.
Uses synchronized queues to ensure frame alignment.
Coordinate System: Left-handed (CARLA Default).

`scenarios/base.py` (The Plugin Contract)

All scenarios must inherit from ScenarioBase.
Required methods: setup(), step(), cleanup(), and name property.
Support for CLI parameter injection via apply_parameters(args.params).

🛰️ 3. Data & Visualization

Data Recording (`src/recorder.py`)

Saves images as .png, radar/lidar as .npy, and metadata as .jsonl.
Relative ADAS Metrics: Calculates range, azimuth, and closing velocity for all actors.

MCAP Conversion (`scripts/data_to_mcap.py`)

Converts raw dataset folders into Foxglove-compatible .mcap files.
Coordinate Conversion: Negates Y and Yaw to switch from CARLA left-handed to ROS/Foxglove right-handed convention.
Radar Math: Converts spherical (Depth/Az/Alt) to Cartesian (XYZ) for PointCloud representation.

🖥️ 4. GUI Dashboard Architecture

Backend: Flask (app.py) running on port 5000.
Streaming: Uses Server-Sent Events (SSE) to stream stdout to the browser.
Execution: Spawns run.bat as a subprocess.
Crucial Rule: Always set PYTHONUNBUFFERED=1 to prevent log delays in the GUI console.

🧠 5. Scenario Design & Best Practices

The "Deterministic Pattern"

Use manual coordinate-based control and set_target_velocity instead of Traffic Manager for repeatable maneuvers.
Spawn-and-Move: Spawn ego at a safe point, then set_transform to the exact start location.
Z-Axis Lift: Always spawn NPCs with a +0.5m offset to avoid ground-mesh collision errors ("Spot occupied").

Common Pitfalls

Stale Physics: Always world.tick() once after spawning the Ego but before starting scenario logic.
Progress Bar Clash: Use pbar.write() for logs if a tqdm progress bar is active.
Asynchronous I/O: Use ThreadPoolExecutor for disk writes (images) to prevent simulation frame drops.

🚀 6. Execution Commands

Run Scenario: ./run.bat <scenario_name> --frames <N> --params "KEY=VAL" --weather <Preset>
List Scenarios: ./run.bat --list-scenarios
Manual Python Run: python src/main.py --scenario <name>

Generated by Antigravity AI | Last Updated: 2026-04-01 | Refer to .cursorrules for start-of-turn instructions.

4.3 KiB Raw Blame History