diff --git a/.agent/instructions.md b/.agent/instructions.md new file mode 100644 index 0000000..e7456f0 --- /dev/null +++ b/.agent/instructions.md @@ -0,0 +1,10 @@ +# AI Agent Instructions for Fox CARLA SIM + +**MANDATORY FIRST STEP:** Always read `d:\CARLA\CARLA_0.9.16\PythonAPI\Fox\gemini.md` in the root folder before starting any task to understand current project status, architecture, and best practices. + +## Core Rules +1. **Simulation Logic**: All scenario-specific code belongs in `scenarios/`. Use `src/` for scenario-agnostic orchestrators. +2. **Environment**: Use PowerShell as the shell. Always activate `carla312` conda environment (automatic via `run.bat`). +3. **Determinism**: Use manual coordinate-based control and `set_target_velocity` for repeatable scenarios. +4. **Safety Defaults**: Mandatory +0.5m Z-offset for actor spawning to avoid ground collisions. +5. **Asynchronous I/O**: Use `ThreadPoolExecutor` for disk writes (images/data) to maintain simulation FPS. diff --git a/.cursorrules b/.cursorrules index 501840e..f273851 100644 --- a/.cursorrules +++ b/.cursorrules @@ -1,6 +1,7 @@ # AI Rules for BATL_CARLA_SIM This document contains rules and instructions for AI agents working on this project. +**ALWAYS read `gemini.md` in the root folder before starting any task to understand current project status and architecture.** ## Simulation Pipeline - This project uses CARLA 0.9.16 and the Foxglove integration. diff --git a/gemini.md b/gemini.md new file mode 100644 index 0000000..25a0968 --- /dev/null +++ b/gemini.md @@ -0,0 +1,91 @@ +# Fox CARLA ADAS Simulation Project: Agent Intelligence (gemini.md) + +This document is the consolidated source of truth for AI agents working on the Fox CARLA Simulation project. It combines high-level architecture, component-level details, and best practices. + +--- + +## 🏗️ 1. Project Overview & Architecture + +**Objective:** A modular, scenario-driven simulation framework built on CARLA 0.9.16 for demonstrating structured ADAS driving scenarios with multi-modal sensor data. + +**The Pipeline:** +`CARLA Simulator → Multi-Sensor Capture → Dataset (PNG/NPY/JSONL) → MCAP Conversion → Foxglove Visualization` + +### Repository Layout +- `dashboard.bat`: One-click launcher for the Flask GUI orchestrator. +- `run.bat`: One-click runner for the simulation (activates `carla312` conda env). +- `config.py`: Global simulation constants (FPS, sensor specs, ego model). +- `src/main.py`: Scenario-agnostic orchestrator and entry point. +- `src/sensors.py`: `SensorManager` — Handles camera, radar, and lidar sync. +- `src/recorder.py`: `Recorder` — Serializes raw frames to disk asynchronously. +- `scenarios/`: Contains scenario implementations (e.g., `braking.py`, `cutin.py`, `showcase.py`). +- `scripts/`: Utility scripts for data processing (e.g., `data_to_mcap.py`). +- `dashboard/`: Flask backend and static web assets for the GUI. +- `intel/`: Detailed deep-dive documentation for specific components. + +--- + +## 🛠️ 2. Component Reference + +### `src/main.py` (The Orchestrator) +- **Role:** Handles CARLA connection, ego spawning, sensor initialization, and main simulation loop. +- **Rules:** Never import a scenario by name. Uses `scenario_loader.py` to dynamically load classes. +- **Workflow:** Parse CLI args → Initialize `ScenarioBase` subclass → Tick world once to settle physics → `scenario.setup()` → Loop `world.tick()`, `recorder.save()`, `scenario.step()` → `scenario.cleanup()`. + +### `src/sensors.py` (SensorManager) +- Manages Camera (RGB), Radar, and LiDAR. +- Uses synchronized queues to ensure frame alignment. +- Coordinate System: Left-handed (CARLA Default). + +### `scenarios/base.py` (The Plugin Contract) +- All scenarios must inherit from `ScenarioBase`. +- Required methods: `setup()`, `step()`, `cleanup()`, and `name` property. +- Support for CLI parameter injection via `apply_parameters(args.params)`. + +--- + +## 🛰️ 3. Data & Visualization + +### Data Recording (`src/recorder.py`) +- Saves images as `.png`, radar/lidar as `.npy`, and metadata as `.jsonl`. +- **Relative ADAS Metrics:** Calculates range, azimuth, and closing velocity for all actors. + +### MCAP Conversion (`scripts/data_to_mcap.py`) +- Converts raw dataset folders into Foxglove-compatible `.mcap` files. +- **Coordinate Conversion:** Negates Y and Yaw to switch from CARLA left-handed to ROS/Foxglove right-handed convention. +- **Radar Math:** Converts spherical (Depth/Az/Alt) to Cartesian (XYZ) for PointCloud representation. + +--- + +## 🖥️ 4. GUI Dashboard Architecture + +- **Backend:** Flask (`app.py`) running on port 5000. +- **Streaming:** Uses **Server-Sent Events (SSE)** to stream `stdout` to the browser. +- **Execution:** Spawns `run.bat` as a subprocess. +- **Crucial Rule:** Always set `PYTHONUNBUFFERED=1` to prevent log delays in the GUI console. + +--- + +## 🧠 5. Scenario Design & Best Practices + +### The "Deterministic Pattern" +- Use manual coordinate-based control and `set_target_velocity` instead of Traffic Manager for repeatable maneuvers. +- **Spawn-and-Move:** Spawn ego at a safe point, then `set_transform` to the exact start location. +- **Z-Axis Lift:** Always spawn NPCs with a **+0.5m offset** to avoid ground-mesh collision errors ("Spot occupied"). + +### Common Pitfalls +- **Stale Physics:** Always `world.tick()` once after spawning the Ego but before starting scenario logic. +- **Progress Bar Clash:** Use `pbar.write()` for logs if a `tqdm` progress bar is active. +- **Asynchronous I/O:** Use `ThreadPoolExecutor` for disk writes (images) to prevent simulation frame drops. + +--- + +## 🚀 6. Execution Commands + +- **Run Scenario:** `./run.bat --frames --params "KEY=VAL" --weather ` +- **List Scenarios:** `./run.bat --list-scenarios` +- **Manual Python Run:** `python src/main.py --scenario ` + +--- + +*Generated by Antigravity AI | Last Updated: 2026-04-01 | Refer to .cursorrules for start-of-turn instructions.*