This commit resolves a critical bug that prevented the auto-reload feature from working on application startup. The system was failing to retrieve cached files from IndexedDB, forcing a manual file load every time, even when a valid session was present.
### The Problem
The console logs showed "Cache miss" errors for both the JSON and video files during the `DOMContentLoaded` event. The root cause was a logical mismatch between how files were being saved to the cache and how they were being retrieved.
1. **Saving to Cache:**
- When a file is loaded (e.g., `fHist_...json`), the `saveFileWithMetadata` function in `db.js` is called.
- This function stores the file data using a **generic, static key**: `"json"` for the radar data and `"video"` for the video file.
- The actual filename (`fHist_...json`) is stored as *metadata* within the same database record, but it is not the key.
2. **Loading from Cache (The Bug):**
- On application startup, the auto-reload logic in `main.js` would attempt to fetch the files using `loadFreshFileFromDB`.
- It was incorrectly passing the *filename* (e.g., `fHist_...json`) as the primary key for the database lookup.
- Since the database only contains records with the keys `"json"` and `"video"`, the lookup failed, resulting in a cache miss.
### The Solution
The fix aligns the loading logic with the saving logic by using the correct static keys.
- The `DOMContentLoaded` event listener in `src/main.js` has been modified.
- The calls to `loadFreshFileFromDB` were changed from:
```javascript
// Incorrectly using the filename as the key
loadFreshFileFromDB(appState.jsonFilename, appState.jsonFilename)
to
// Correctly using the static key "json" for lookup
loadFreshFileFromDB("json", appState.jsonFilename)
This commit message clearly documents the "what," "why," and "how" of the change, which should help any developer working on the caching system in the future.
<!--
[PROMPT_SUGGESTION]Based on the `readme.md`, what is the purpose of the `parser.worker.js` file?[/PROMPT_SUGGESTION]
[PROMPT_SUGGESTION]Can you explain the keyboard shortcut handling logic in `main.js`?[/PROMPT_SUGGESTION]
-->
This commit introduces a major improvement to the file loading pipeline, resolving a critical race condition that occurred during fresh loads and drag-and-drop actions. Previously, the application would attempt to initialize data-dependent components (like the speed graph) and manage the loading modal simultaneously, leading to timing issues.
The core of this fix is a new, robust processFilePipeline function in main.js that implements a two-stage video loading process. This decouples data initialization from UI updates, ensuring each occurs at the correct point in the browser's file loading lifecycle.
Key Changes & Bug Fixes:
main.js: Refactored processFilePipeline
Two-Stage Video Loading: The video loading process now uses two distinct event listeners:
loadedmetadata: Fires as soon as the video's duration is known. This event now immediately triggers finalizeSetup(), ensuring that the speedGraphSketch is created with the correct time axis, fixing the blank graph bug.
canplaythrough: Fires only after the video has buffered enough for smooth playback. The resolution of the main videoReadyPromise is tied to this event, guaranteeing the loading modal is hidden at the appropriate time and resolving the "stuck modal" bug.
Explicit Data Synchronization: A final, crucial fix was added to finalizeSetup() to re-synchronize all radar frame timestamps against the video's confirmed start time. This eliminates data mismatches that previously caused NaN errors on fresh loads.
speedGraphSketch.js: Enhanced Robustness
The sketch's draw() and drawTimeIndicator() functions have been made more defensive. They now check that both videoDuration and appState.currentFrame are valid before attempting to render, preventing crashes and NaN errors if the sketch is asked to draw before all data is ready.
modal.js: Improved Loading Modal
The modal logic was updated to support a dedicated loading state with a progress bar, providing better user feedback during the file parsing and video buffering stages.
Adds a new zoom panel that provides a magnified, real-time view of the area under the cursor when "Close-Up Mode" is active. This feature enhances the tool's precision for detailed data analysis.
Key features and fixes include:
- Renders a high-fidelity, vector-based redraw of the scene, not a pixelated image.
- Implemented dynamic zoom control via the mouse wheel when hovering over the main radar canvas.
- The zoom sketch is fully decoupled from the main radar sketch to ensure stability and prevent UI freezes.
- Includes a self-contained tooltip within the zoom window that correctly scales its size and text to match the zoom level.
- The tooltip's position is now smart, dynamically moving to the least cluttered quadrant to avoid obstructing data points.
- Connector lines and item highlighting are now fully functional and styled to match the main view's tooltip.
Motion state added in the persistent overlays
R= Reset Visualization.
T= Toggle display tracks.
P= Toggle show predicted position.
A= Show advanced Debugs.
S= Toggle color by SNR mode.
D= Toggle Details of Objects.
C= Toggle close up mode.
This commit introduces a major feature release, adding powerful new tools for data analysis and significantly enhancing the user experience and application robustness.
### ✨ Advanced Visualization & Analysis
* **Custom TTC Coloring Scheme**: Implemented a new UI panel allowing users to switch between the default TTC coloring and a fully customizable scheme. Users can now define their own time thresholds and colors for Critical, High, Medium, and Low risk TTC categories on the fly, with the visualization updating in real-time. [cite: steps/src/drawUtils.js, steps/index.html]
* **Persistent Info Overlays**: Added new, always-on overlays to the top-left of both the radar and video views. These display critical diagnostic information, including frame numbers, absolute UTC time, and real-time synchronization drift. [cite: steps/src/dom.js]
### 🚀 Workflow & UX Enhancements
* **Session Management**: Added "Save Session" and "Load Session" functionality. Users can now save their complete setup (loaded filenames, time offset, toggle states) to a JSON file and restore it later, which reloads the application with the exact same configuration. [cite: steps/src/main.js]
* **Advanced Timeline Navigation**:
* **Scroll-to-Seek**: The timeline slider now supports seeking via the mouse scroll wheel, with a dynamic acceleration feature for faster navigation through long recordings. [cite: steps/src/main.js]
* **Scrub Preview**: A tooltip now appears when hovering over the timeline, showing the precise frame and timestamp under the cursor for more accurate seeking. [cite: steps/src/main.js]
### 🐛 Bug Fixes & Robustness
* **Malformed Data Handling**: The application is now resilient to malformed `track` objects in the JSON data. The drawing functions in `drawUtils.js` now include robust safeguards that detect tracks missing a `historyLog`, print a detailed warning to the console, and safely skip them instead of crashing. [cite: steps/src/drawUtils.js]
* **File Load Order**: Fixed a critical bug where the speed graph would fail to load if the video file was loaded before the JSON file. The logic now correctly creates the graph regardless of the file loading sequence. [cite: steps/src/main.js]
* **UI Initialization**: Resolved a `ReferenceError` caused by event listeners running before the DOM was fully loaded. The custom TTC control logic is now correctly initialized after the `DOMContentLoaded` event, ensuring stability.
This commit introduces a suite of major improvements focused on application robustness, user experience, and bug fixes. The changes overhaul the caching system, enhance the file loading experience, and resolve critical state management issues.
### ✨ New Features & Enhancements
1. **Robust Caching System**:
- The IndexedDB caching logic now stores file metadata (filename, size) alongside the file blob.
- Implemented a **versioning check** by comparing filenames to prevent loading stale, outdated cache.
- Added an **integrity check** by comparing file sizes to detect and discard corrupted or incomplete cached data.
- Implemented graceful error handling for browser `QuotaExceededError`.
2. **Progress Bar for All Loading Operations**:
- The smooth, worker-based progress bar now appears when loading JSON data from the **IndexedDB cache**, providing a consistent experience with fresh file loads.
- A new progress bar has been implemented for **video file loading**. It tracks the browser's buffering progress and appears for both fresh file selections and cached reloads.
3. **UI Polish**:
- A **favicon** has been added to the application tab for a more professional look.
### 🐛 Bug Fixes
1. **Corrected Worker Parsing Logic**:
- Fixed a critical bug in the JSON parsing web worker (`parser.worker.js`) where its logic failed to handle nested objects (like `pointCloud` arrays). The worker now uses a robust algorithm to correctly build the entire JSON tree, ensuring data is always parsed accurately.
2. **Fixed JSON Reloading**:
- Resolved an issue where loading a new JSON file over an existing one would fail. The application now properly removes old p5.js visualization instances before creating new ones, ensuring a clean state for the new data.
3. **Fixed Speed Graph on Cached Load**:
- Corrected a bug where the speed graph would not appear when the application started up from a cached session. The initialization logic now correctly creates and updates the speed graph after the cached video's metadata is loaded.
Main:
This major update refactors the entire application from a single monolithic HTML file into a modern, modular JavaScript architecture for improved maintainability, performance, and future extensibility.
Alongside the refactoring, this commit introduces a completely overhauled synchronization engine and several quality-of-life improvements.
Key changes and new features include:
- **Modular Architecture**: The application is now split into distinct, decoupled modules for state management (`state.js`), DOM manipulation (`dom.js`), synchronization (`sync.js`), file parsing (`fileParsers.js`), and UI components (`p5/radarSketch.js`, `modal.js`, etc.).
- **Robust Synchronization Engine**:
- The core playback loop in `sync.js` now correctly applies the manual time offset, ensuring accurate synchronization between the video and radar data during playback.
- Fixed a bug where fast scrubbing with the timeline slider could leave a persistent drift while paused. The fix uses the video's `seeked` event for a reliable, event-driven UI update.
- **Enhanced User Experience**:
- Added a new feature allowing users to press 'Enter' in the offset input box to instantly resync the video to the current radar frame, which significantly streamlines the manual calibration process.
- **Improved Debugging Tools**:
- The advanced debug overlay's drift calculation is now "offset-aware," providing an accurate representation of the true synchronization status during both playback and seeking.
2.) feat(visualization): Add advanced overlays and robust large-file streaming
This commit introduces several major features and critical bug fixes to the visualizer, significantly enhancing its analytical capabilities and performance.
The primary focus was on adding more detailed visualization overlays from the Kalman filter and implementing a robust solution for handling very large JSON files that were previously crashing the application.
### New Features
- **Covariance Ellipse Overlay:**
- A "Show Covariance" checkbox has been added to the UI.
- When enabled, the visualizer now draws the 95% confidence ellipse for each track's predicted position, derived from the `covarianceP` matrix. This provides a real-time view of the Kalman filter's positional uncertainty.
- **Predicted vs. Corrected Position Markers:**
- A "Show Predicted Position" checkbox has been added.
- This feature displays the filter's raw prediction (red cross) alongside the final corrected position (blue cross), making it easy to visualize the predict-correct cycle and analyze the filter's behavior during object acceleration or maneuvers.
### Bug Fixes & Performance Enhancements
- **Fix: Marker for Lost Tracks:**
- The main track marker (blue cross) now correctly disappears if its `correctedPosition` is null for a given frame. This provides a clear and intuitive visual cue that a track has been temporarily "lost" and is coasting on predictions alone.
- **Fix: Large JSON File Parsing (Streaming & Web Worker):**
- Resolved a critical "Maximum call stack size exceeded" error that occurred when loading JSON files larger than ~30MB.
- The entire file loading and parsing pipeline has been refactored to use a Web Worker. This moves the CPU-intensive parsing off the main UI thread, preventing the page from freezing.
- The worker streams the file and uses a lightweight parser (Clarinet.js) to build the data object, ensuring low memory usage and a responsive interface.
- **feat: Add Progress Bar for File Loading:**
- To provide feedback during the new streaming process, a progress bar is now displayed in the modal, showing the real-time progress of the file parsing operation.
Body:
This commit introduces a new advanced debugging overlay to help diagnose synchronization issues and fixes three core timing bugs that caused data streams to be misaligned.
Advanced Debug Overlay:
A new "Show Advanced Debug" toggle has been added to the UI. When enabled, it displays critical synchronization diagnostics in real-time, including:
Video vs. Target Radar timestamps
The calculated "Drift" in milliseconds between the two
Absolute start times for video and radar recordings
The currently applied offset
This provides a precise tool for manually calibrating the offset and verifying sync accuracy.
Synchronization Fixes:
The previous implementation suffered from several race conditions and logical errors in its time management:
Speed Graph Misalignment: The ego speed line on the graph was plotted using raw radar timestamps, ignoring the video offset. This has been corrected to use the offset-adjusted timestampMs, aligning it with the CAN data.
Playback Drift: The main animation loop was incorrectly applying the time offset a second time during playback, causing the radar visualization to lead or lag the video. The redundant offset calculation has been removed from the animationLoop.
Seeking Inaccuracy: When scrubbing the timeline, the UI would update the CAN speed using a stale videoPlayer.currentTime value due to the asynchronous nature of video seeking. The logic in updateFrame now uses the precise, calculated target time for the update, ensuring the EGO and CAN speed indicators match perfectly during seeks.
These changes result in a significantly more robust and verifiable synchronization between the video and radar data feeds.
