Browse Source
feat(dashboard): dashboard optimization and repository memory initialization
feat(dashboard): dashboard optimization and repository memory initialization
- Optimized dashboard polling (3s) and suppressed noisy terminal status logs. - Implemented collapsible 'Configuration Parameters' card with localStorage persistence. - Overhauled dashboard theme for high-contrast accessibility and element visibility. - Initialized intel/CHRONICLES.md to document project saga and technical breakthroughs. - Established intel/memory_update.md protocol for automated repository memory updates.1843_integration
7 changed files with 331 additions and 56 deletions
-
8dashboard/app.py
-
16dashboard/static/app.js
-
76dashboard/static/style.css
-
89dashboard/templates/index.html
-
2gemini.md
-
151intel/CHRONICLES.md
-
45intel/memory_update.md
@ -0,0 +1,151 @@ |
|||
# 📜 Fox Project Chronicles: The Technical Saga |
|||
|
|||
This document is the definitive record of the Fox CARLA ADAS Simulation project. It bridges high-level git history with the deep technical engineering decisions that shaped the repository. Use this as a guide to understand **not just what changed, but why.** |
|||
|
|||
--- |
|||
|
|||
## 🏛️ 1. Project Manifesto: The Vision |
|||
|
|||
The Fox project was born from a need for **deterministic, high-fidelity ADAS validation.** While standard CARLA simulations provide a starting point, they often lack the physical realism required for modern radar sensor fusion. |
|||
|
|||
**Our Core Pillars:** |
|||
1. **Physical Integrity**: Radar data must follow the laws of physics ($1/R^4$), not just look "busy." |
|||
2. **Deterministic Repeatability**: A scenario must execute with millisecond precision every time. |
|||
3. **Observability**: Developers must have "Radar Lab" visibility into the signal processing chain. |
|||
|
|||
--- |
|||
|
|||
## 🛰️ 2. The Shenron Iteration Log (The Long Road to Fidelity) |
|||
|
|||
The "Shenron" radar engine evolved through 26 critical iterations. This table tracks its journey from a broken port to a physically consistent testbench. |
|||
|
|||
| Iteration | Focus | The "Why" (Decision & Rationale) | Technical Result | |
|||
| :--- | :--- | :--- | :--- | |
|||
| **01-02** | **Stability** | Initial ports from legacy code were unstable. We restored the **AWGN Thermal Noise Floor** because the CFAR detector needs a non-zero background variance to avoid infinite false positives. | **STABLE BASELINE** | |
|||
| **03** | **RCS Recovery** | Surfaced a critical math error in reflection logic. By removing a redundant cosine, we recovered **46% of lost signal energy**, preventing vehicles from "vanishing" at slight angles. | **+46% SNR** | |
|||
| **04-05** | **Vel. & Scale**| Dynamic targets showed 0 m/s because bitfields were mismatched. We introduced `np.view(np.uint32)` for correct metadata unpacking. Fixed scaled axes to prevent "Range Stretching." | **CORRECT GEOMETRY** | |
|||
| **07-09** | **Tag Alignment**| Aligned with CARLA 0.9.16 standards. **Decision:** We prioritized vehicle detection (Tag 14) over generic clutter filtering to ensure ADAS targets were always visible. | **0.9.16 SYNC** | |
|||
| **11-13** | **Z-Filtering** | **Decision:** Implemented a floor-clipping mask (Z > -2.2m). This "Golden Mix" preserves vehicle tires/lips while completely eliminating road-surface multipath ghosts. | **CLEAN METRICS** | |
|||
| **16** | **Target Boost** | Replaced dynamic $1/N$ with a fixed `DENSITY_REF`. **Rationale:** Preventing distant buildings from "dimming" near targets. This preserved context-independent target power. | **+234% SNR BOOST** | |
|||
| **18** | **Phase Lock** | Switched from average Doppler to **Doppler-Slice** Angle FFT. **Decision:** Preserving complex phase is the only way to achieve sharp angular resolution and prevent "blobbing." | **SHARP AZIMUTH** | |
|||
| **20-22** | **Symmetry** | Resolved a 15° boresight tilt. **Rationale:** Standard FFT indexing was asymmetric; we shifted to perfect index linear spaces to ensure the 120° FOV fan was centered. | **ZERO-TILT SYMMETRY** | |
|||
| **26** | **Milestone R4** | **THE BREAKTHROUGH:** Replaced all artificial normalization with the **$1/R^4$ Radar Range Equation**. **Decision:** Physical correctness is the only way to build trust in ADAS validation. | **PURE PHYSICS** | |
|||
|
|||
--- |
|||
|
|||
## 🪦 3. The Bug Graveyard: Lessons from the Abyss |
|||
|
|||
These four bugs represented the most significant blockers in the project's history. Here is how they were tackled. |
|||
|
|||
### 🐛 Bug #1: The "Cos(Cos)" Reflection Error (`Sceneset.py`) |
|||
- **Situation:** Surfaces (especially car hoods) were appearing significantly darker than expected, regardless of material. |
|||
- **Investigation:** Discovered that the `specularpoints()` method was receiving a pre-calculated cosine from the LiDAR engine, but then applying `np.cos()` to it *again*. |
|||
- **The "Why":** A 0-degree incident angle (cos = 1.0) was being computed as `cos(1.0) = 0.54`. |
|||
- **Decision:** Removed the second cosine call. This was a legacy artifact from a different coordinate system port. |
|||
- **Result:** Immediate 46% magnitude recovery and stable tracking during turns. |
|||
|
|||
### 🐛 Bug #2: Metadata Bit-View Packing (`src/recorder.py`) |
|||
- **Situation:** Radar synthesis was working, but all actors showed exactly `0.0` radial velocity. |
|||
- **Investigation:** CARLA 0.9.16 stores Actor IDs and Semantic Tags as `uint32` values, but they are "packed" into a `float32` bitstream to maintain a uniform array. Standard indexing read them as floats, resulting in garbage. |
|||
- **Decision:** Use **`np.view(np.uint32)`** to reinterpret the bits without value conversion. |
|||
- **Result:** Restored full dynamic context to the simulation, enabling velocity-aware ADAS testing. |
|||
|
|||
### 🐛 Bug #3: The Stale Physics Race Condition (`scenarios/base.py`) |
|||
- **Situation:** NPCs were frequently spawning at the center of the map instead of in front of the Ego vehicle. |
|||
- **Investigation:** In Synchronous Mode, the simulator needs at least one `world.tick()` to settle physics. Accessing `get_location()` of a newly spawned Ego vehicle returned `0,0,0` because the frame hadn't processed yet. |
|||
- **Decision:** Forced an explicit **Settling Tick** in the orchestrator before any scenario logic computes coordinates. |
|||
- **Result:** 100% spawning reliability. |
|||
|
|||
### 🐛 Bug #4: Early Dimensional Collapse (`radar_processor.py`) |
|||
- **Situation:** Range-Azimuth plots showed targets as wide, blurry smears rather than sharp points. |
|||
- **Investigation:** The signal processing chain was collapsing the Doppler dimension (summing magnitudes) *before* performing the Angle-FFT. This discarded the relative phase differences between chirps. |
|||
- **Decision:** Re-architected the pipeline to perform Angle-FFT on individual Doppler slices, preserving coherence. |
|||
- **Result:** Angular resolution improved by ~300%, matching hardware-spec performance. |
|||
|
|||
--- |
|||
|
|||
## 🏔️ 4. Major Architectural Milestones |
|||
|
|||
### ⚓ The 1/R⁴ Physical Baseline |
|||
Prior to Week 3, we used artificial scaling to keep the signal "visible." |
|||
**The Shift:** We realized that ADAS algorithms depend on the **physical contrast** between a car and a building. We implemented the $1/R^4$ law: |
|||
- **Calculation:** $P_{rec} = \frac{P_{tx} \cdot G^2 \cdot \lambda^2 \cdot \sigma}{(4\pi)^3 \cdot R^4}$ |
|||
- **Outcome:** The simulation now correctly models how energy drops over distance, allowing for realistic sensitivity-threshold testing. |
|||
|
|||
### 🖥️ GPU Resource Control (Idle Mode) |
|||
The Shenron engine is heavy. Running it alongside a 30 FPS CARLA simulation causes thermal throttling. |
|||
**The Fix:** We implemented **"Idle Mode."** |
|||
- **Decision:** When the dashboard is not actively running a scenario or is processing a recorded dataset, we cap CARLA's `fixed_delta_seconds` to a crawl (near zero GPU usage). |
|||
- **Outcome:** Frees 95% of VRAM and CUDA cores for the high-fidelity Shenron signal synthesis. |
|||
|
|||
### 🛰️ The Metrology Breakthrough: Heatmap Synthesis |
|||
Prior to Week 3, the radar output was a "Black Box." |
|||
**The Sprint:** We spent two days re-engineering the signal chain in `heatmap_gen_fast.py` and `radar_processor.py`. |
|||
- **The Problem:** We were using **Coherent Integration** (complex averaging) across Doppler bins, which caused massive phase cancellation and "Horizontal Banding." Target vehicles were vanishing during maneuvers. |
|||
- **The Solution:** Implemented **Doppler-Slice Synthesis**. We now perform Angle-FFT on each Doppler bin independently to preserve phase, followed by **Incoherent Power Summation**. |
|||
- **Result:** Transitioned from spatially blurred energy fields to sharp, physically consistent RA/RD heatmaps. This unlocked the ability to see "Radar Blue" Jet-spectrum blobs that spatially align with LiDAR ground truth. |
|||
|
|||
--- |
|||
|
|||
## 📜 5. Project Evolution (Weekly Context) |
|||
|
|||
### 📅 Week 1: Genesis & Foundation (March 27 – March 29, 2026) |
|||
- **Objective:** Establish the core ADAS framework and deterministic simulation logic. |
|||
- **Key Results:** |
|||
- Initialization of the modular `ScenarioBase` architecture. |
|||
- Implementation of the `ThreadPoolExecutor` for asynchronous data capture. |
|||
- Deployment of the **Showcase Suite** (Deterministic coordinate control). |
|||
|
|||
### 📅 Week 2: Dashboard & The Shenron Awakening (March 30 – April 5, 2026) |
|||
- **Objective:** Orchestration and high-fidelity sensor integration. |
|||
- **Key Results:** |
|||
- Deployment of the **Flask Dashboard** GUI. |
|||
- Surgical isolation of the **Shenron Radar Simulator** (ISOLATE). |
|||
- Milestone fixes for zero-velocity tracking and RCS magnitude. |
|||
|
|||
### 📅 Week 3: Physics & Optimization (April 6 – April 12, 2026) |
|||
- **Objective:** Physical consistency and resource control. |
|||
- **Key Results:** |
|||
- **Milestone R4:** $1/R^4$ Power Law implementation. |
|||
- **Symmetry Lock:** Zero-tilt boresight calibration. |
|||
- **Idle Mode:** Synchronous GPU throttling for multi-process efficiency. |
|||
|
|||
--- |
|||
|
|||
## 📜 6. Detailed Daily Chronology (Git Absolute) |
|||
|
|||
### **April 13 (Today)** |
|||
- **Task:** Observability & UI Optimization. |
|||
- **Action:** Established the `CHRONICLES.md` (Technical Saga) and `memory_update.md` (Agent Protocol). |
|||
- **Optimization:** Addressed terminal log "flooding" by suppressing `/api/simulator/status` Werkzeug logs and reducing frontend polling from 1s to 3s. |
|||
- **UI Enhancement:** Implemented a collapsible **Configuration Parameters** card. |
|||
- **Accessibility Fix:** Performed a full theme overhaul to improve readability. Boosted panel contrast, increased label font-weight, and transformed the collapse toggle into a prominent, circular blue-accented action button. |
|||
|
|||
### **April 10** |
|||
- **Task:** Process Efficiency & Lifecycle. |
|||
- **Action:** Implemented **Idle Mode** (Synchronous Throttling) and graceful shutdown logic. |
|||
|
|||
### **April 8-9** |
|||
- **Task:** Physics Baseline & Visualization. |
|||
- **Action:** Locked in the **1/R⁴ Power Law** and **Zero-Tilt Symmetry**. |
|||
- **Change:** Removed artificial normalizations in `radar_processor.py`. |
|||
|
|||
### **April 6-7** |
|||
- **Task:** Metrology Suite. |
|||
- **Action:** Added real-time RD/RA heatmaps. Integrated **Gaussian Vertical Damping** (Iter 14a). |
|||
|
|||
### **April 1-3** |
|||
- **Task:** Integration & Calibration. |
|||
- **Action:** Merged **C-SHENRON** into the main pipeline. Fixed the bit-view packing bug and the `cos(cos)` error. |
|||
|
|||
### **March 31** |
|||
- **Task:** Orchestration. |
|||
- **Action:** Modernized the simulation pipeline with centralized scripts and the initial GUI. |
|||
|
|||
### **March 27 (Initial Commit)** |
|||
- **Task:** Project Inception. |
|||
- **Action:** Initialized the Fox CARLA ADAS repository. Implemented the asynchronous recorder and the first deterministic showcase scenarios. |
|||
|
|||
--- |
|||
|
|||
*Generated by Antigravity AI | Fox CARLA ADAS Simulation | 2026-04-13* |
|||
@ -0,0 +1,45 @@ |
|||
# 🧠 Agent Memory Update Protocol |
|||
|
|||
This document instructs future AI agents on how to maintain the repository's documentation and "memory" files. Follow these steps after every significant feature implementation, technical breakthrough, or calibration milestone. |
|||
|
|||
--- |
|||
|
|||
## 📂 Target Files |
|||
|
|||
| File | Purpose | Update Context | |
|||
| :--- | :--- | :--- | |
|||
| `intel/CHRONICLES.md` | The Project Saga | Git milestones, technical "Why" & "How," bug post-mortems. | |
|||
| `gemini.md` | Source of Truth | Architecture changes, new components, active file rules. | |
|||
| `intel/radar/Shenron_debug.md` | Calibration Log | Radar iterations (01-XX), specific physics fixes. | |
|||
| `intel/scenarios/*.md` | Scenario Guides | Decision rationale for specific testing maneuvers. | |
|||
|
|||
--- |
|||
|
|||
## 🔄 Update Procedure |
|||
|
|||
### Step 1: Analyze Evolution |
|||
Run the following to understand what changed: |
|||
```powershell |
|||
git log --pretty=format:"%ad | %s | %b" --date=short -n 10 |
|||
``` |
|||
|
|||
### Step 2: Extract Rationale |
|||
Don't just log *what* happened. Documentation must explain: |
|||
- **The Problem**: What was breaking or inaccurate? |
|||
- **The Decision**: Why was this specific technical path taken over others? |
|||
- **The Physics/Math**: What formulas or logical constraints were applied? |
|||
|
|||
### Step 3: Atomic Updates |
|||
- **Chronicles**: Append new daily entries to Section 6. If a major milestone was reached, create a dedicated Deep-Dive in Section 4. |
|||
- **Iteration Log**: If working on the Shenron radar, update the Iteration Table (01-XX) with the latest physics impacts. |
|||
- **Gemini**: Update the "Repository Layout" if new directories or scripts are added. |
|||
|
|||
--- |
|||
|
|||
## 🎨 Style Guide |
|||
- **Premium Aesthetic**: Use GitHub alerts (Note, Important, Tip) to highlight key insights. |
|||
- **Technical Rigor**: Use LaTeX math for physics formulas. |
|||
- **No Fluff**: Keep descriptions concise but info-dense. |
|||
|
|||
--- |
|||
*Generated by Antigravity AI | Repository Memory System | 2026-04-13* |
|||
Write
Preview
Loading…
Cancel
Save
Reference in new issue