2
3 - **Modularization:** Extracted monolithic file handling from `main.js` into a new dedicated `src/fileLoader.js`
module.
4 - **Incremental Loading:** Implemented logic to handle JSON and Video files independently, allowing users to replac
individual files without breaking the application state.
5 - **Race Condition Fixes:** Updated `src/db.js` to return Promises and await database initialization, resolving
startup crashes.
6 - **Sync Logic Correction:** Reordered the loading pipeline to ensure file metadata (and thus timestamps) are
extracted *before* parsing visualization data. This fixes `NaN` errors in the speed graph and ensures correct video
synchronization.
7 - **Optimization:** Added a `fromCache` flag to skip redundant database writes when reloading a previous session.
8 - **Testing:** Added `tests/fileLoader.test.js` to verify the new loading pipeline.
" Here is the detailed breakdown of why that solution worked.
The Root Cause: Initialization Order
The error ellipse() ... received NaN occurred because the speedGraphSketch was trying to plot a point based on time
(x-axis), but the time value for the current frame was invalid (NaN or undefined).
Here is the chain of events that caused this:
1. The Dependency Chain:
* To draw the graph, the sketch needs frame.timestampMs.
* To calculate frame.timestampMs, the JSON Parser (parseVisualizationJson) needs appState.videoStartDate.
* To get appState.videoStartDate, the helper function calculateAndSetOffset() needs to know the Video Filename.
2. The Bug (Before Refactor):
When you dragged both files in, the code executed sequentially like this:
* Step A (JSON Block): The code detected the JSON file. It set appState.jsonFilename and immediately called
calculateAndSetOffset().
* CRITICAL FAILURE: At this exact moment, the code had not yet processed the Video file. It didn't know the
video filename.
* Result: calculateAndSetOffset couldn't find a video timestamp, so appState.videoStartDate remained null.
* Step B (Parsing): The JSON parser ran. Because videoStartDate was null, it skipped the math that calculates
frame.timestampMs. All your data frames were created with timestampMs: undefined.
* Step C (Video Block): The code moved to the Video file. It set appState.videoFilename and called
calculateAndSetOffset() again.
* Now it calculated the correct date, but it was too late. The JSON data had already been parsed and baked
without timestamps.
3. The Crash:
When the visualization started, the Speed Graph looked at the current frame, found timestampMs was missing, tried
to divide it by 1000, got NaN, and crashed the ellipse() drawing function.
How the Solution Fixed It
By restructuring processFilePipeline, we enforced the correct dependency order:
1. Pre-Processing (The Fix):
We moved the "registration" of both filenames to the very top of the function.
1 // 1. Register JSON file (if present)
2 // 2. Register Video file (if present)
2. Calculation:
We call calculateAndSetOffset() once, immediately after registration.
* Since both filenames are now known, it successfully extracts the timestamp from the video filename and sets
appState.videoStartDate.
3. Processing:
We then start the JSON parser.
* It receives the valid appState.videoStartDate.
* It successfully calculates frame.timestampMs for every frame.
In short: We ensured that the "ingredients" (filenames and dates) were fully prepared before we started "cooking"
(parsing) the data"
This commit extracts the keyboard shortcut handling logic from the main application entry point (`main.js`) into a new, dedicated module (`src/keyboard.js`).
Previously, all keydown event listeners were defined directly in `main.js`, mixing UI initialization with user input handling. This refactoring improves code organization and modularity by separating concerns, making the system easier to maintain and debug.
Key changes include:
- A new `src/keyboard.js` file was created to house all keyboard-related functionality.
- The `keydown` event listener and its handler function (`handleKeyDown`) were moved from `main.js` into the new module.
- An `initKeyboardShortcuts` function is now exported from `keyboard.js` and called from `main.js` during application startup.
- All dependencies required by the shortcut logic (from `dom.js`, `state.js`, `sync.js`, etc.) are now correctly imported into `keyboard.js`.
This is a pure refactoring effort and introduces no changes to application behavior.
- Refactor(main.js): The manual offset input listener contained duplicated logic for pausing and re-syncing. This has been removed and replaced with `forceResyncWithOffset` imported from `sync.js`.
- Fix(dom.js): The drift calculation in `updateDebugOverlay` and `updatePersistentOverlays` was using static `timestampMs` combined with the dynamic offset, leading to incorrect values after a manual adjustment. Updated to use `currentRadarFrame.videoSyncedTime` for accurate, architecture-aligned drift reporting.
The previous synchronization logic relied on either naive relative time (causing drift) or expensive runtime date parsing (causing lag). This commit implements a hybrid "Baked Offset" architecture.
Changes:
- src/utils.js: Added `precomputeRadarVideoSync` to inject `videoSyncedTime` into every radar frame. Ported robust timestamp parsers from the modularize branch.
- src/main.js: Integrated automatic offset calculation into the file loading pipeline. If filenames contain timestamps, the offset is now auto-calculated.
- src/sync.js: Simplified the `videoFrameCallback` loop. It now treats the video as the "Master Clock" and looks up the pre-computed synced time, ensuring 60fps performance during playback and scrubbing.
- UI: Manual offset adjustments now trigger a "re-bake" of the entire timeline instantly.
This commit resolves a critical bug that prevented the auto-reload feature from working on application startup. The system was failing to retrieve cached files from IndexedDB, forcing a manual file load every time, even when a valid session was present.
### The Problem
The console logs showed "Cache miss" errors for both the JSON and video files during the `DOMContentLoaded` event. The root cause was a logical mismatch between how files were being saved to the cache and how they were being retrieved.
1. **Saving to Cache:**
- When a file is loaded (e.g., `fHist_...json`), the `saveFileWithMetadata` function in `db.js` is called.
- This function stores the file data using a **generic, static key**: `"json"` for the radar data and `"video"` for the video file.
- The actual filename (`fHist_...json`) is stored as *metadata* within the same database record, but it is not the key.
2. **Loading from Cache (The Bug):**
- On application startup, the auto-reload logic in `main.js` would attempt to fetch the files using `loadFreshFileFromDB`.
- It was incorrectly passing the *filename* (e.g., `fHist_...json`) as the primary key for the database lookup.
- Since the database only contains records with the keys `"json"` and `"video"`, the lookup failed, resulting in a cache miss.
### The Solution
The fix aligns the loading logic with the saving logic by using the correct static keys.
- The `DOMContentLoaded` event listener in `src/main.js` has been modified.
- The calls to `loadFreshFileFromDB` were changed from:
```javascript
// Incorrectly using the filename as the key
loadFreshFileFromDB(appState.jsonFilename, appState.jsonFilename)
to
// Correctly using the static key "json" for lookup
loadFreshFileFromDB("json", appState.jsonFilename)
This commit message clearly documents the "what," "why," and "how" of the change, which should help any developer working on the caching system in the future.
<!--
[PROMPT_SUGGESTION]Based on the `readme.md`, what is the purpose of the `parser.worker.js` file?[/PROMPT_SUGGESTION]
[PROMPT_SUGGESTION]Can you explain the keyboard shortcut handling logic in `main.js`?[/PROMPT_SUGGESTION]
-->
This commit introduces a major improvement to the file loading pipeline, resolving a critical race condition that occurred during fresh loads and drag-and-drop actions. Previously, the application would attempt to initialize data-dependent components (like the speed graph) and manage the loading modal simultaneously, leading to timing issues.
The core of this fix is a new, robust processFilePipeline function in main.js that implements a two-stage video loading process. This decouples data initialization from UI updates, ensuring each occurs at the correct point in the browser's file loading lifecycle.
Key Changes & Bug Fixes:
main.js: Refactored processFilePipeline
Two-Stage Video Loading: The video loading process now uses two distinct event listeners:
loadedmetadata: Fires as soon as the video's duration is known. This event now immediately triggers finalizeSetup(), ensuring that the speedGraphSketch is created with the correct time axis, fixing the blank graph bug.
canplaythrough: Fires only after the video has buffered enough for smooth playback. The resolution of the main videoReadyPromise is tied to this event, guaranteeing the loading modal is hidden at the appropriate time and resolving the "stuck modal" bug.
Explicit Data Synchronization: A final, crucial fix was added to finalizeSetup() to re-synchronize all radar frame timestamps against the video's confirmed start time. This eliminates data mismatches that previously caused NaN errors on fresh loads.
speedGraphSketch.js: Enhanced Robustness
The sketch's draw() and drawTimeIndicator() functions have been made more defensive. They now check that both videoDuration and appState.currentFrame are valid before attempting to render, preventing crashes and NaN errors if the sketch is asked to draw before all data is ready.
modal.js: Improved Loading Modal
The modal logic was updated to support a dedicated loading state with a progress bar, providing better user feedback during the file parsing and video buffering stages.
Adds a new zoom panel that provides a magnified, real-time view of the area under the cursor when "Close-Up Mode" is active. This feature enhances the tool's precision for detailed data analysis.
Key features and fixes include:
- Renders a high-fidelity, vector-based redraw of the scene, not a pixelated image.
- Implemented dynamic zoom control via the mouse wheel when hovering over the main radar canvas.
- The zoom sketch is fully decoupled from the main radar sketch to ensure stability and prevent UI freezes.
- Includes a self-contained tooltip within the zoom window that correctly scales its size and text to match the zoom level.
- The tooltip's position is now smart, dynamically moving to the least cluttered quadrant to avoid obstructing data points.
- Connector lines and item highlighting are now fully functional and styled to match the main view's tooltip.
Motion state added in the persistent overlays
R= Reset Visualization.
T= Toggle display tracks.
P= Toggle show predicted position.
A= Show advanced Debugs.
S= Toggle color by SNR mode.
D= Toggle Details of Objects.
C= Toggle close up mode.
This commit introduces a major feature release, adding powerful new tools for data analysis and significantly enhancing the user experience and application robustness.
### ✨ Advanced Visualization & Analysis
* **Custom TTC Coloring Scheme**: Implemented a new UI panel allowing users to switch between the default TTC coloring and a fully customizable scheme. Users can now define their own time thresholds and colors for Critical, High, Medium, and Low risk TTC categories on the fly, with the visualization updating in real-time. [cite: steps/src/drawUtils.js, steps/index.html]
* **Persistent Info Overlays**: Added new, always-on overlays to the top-left of both the radar and video views. These display critical diagnostic information, including frame numbers, absolute UTC time, and real-time synchronization drift. [cite: steps/src/dom.js]
### 🚀 Workflow & UX Enhancements
* **Session Management**: Added "Save Session" and "Load Session" functionality. Users can now save their complete setup (loaded filenames, time offset, toggle states) to a JSON file and restore it later, which reloads the application with the exact same configuration. [cite: steps/src/main.js]
* **Advanced Timeline Navigation**:
* **Scroll-to-Seek**: The timeline slider now supports seeking via the mouse scroll wheel, with a dynamic acceleration feature for faster navigation through long recordings. [cite: steps/src/main.js]
* **Scrub Preview**: A tooltip now appears when hovering over the timeline, showing the precise frame and timestamp under the cursor for more accurate seeking. [cite: steps/src/main.js]
### 🐛 Bug Fixes & Robustness
* **Malformed Data Handling**: The application is now resilient to malformed `track` objects in the JSON data. The drawing functions in `drawUtils.js` now include robust safeguards that detect tracks missing a `historyLog`, print a detailed warning to the console, and safely skip them instead of crashing. [cite: steps/src/drawUtils.js]
* **File Load Order**: Fixed a critical bug where the speed graph would fail to load if the video file was loaded before the JSON file. The logic now correctly creates the graph regardless of the file loading sequence. [cite: steps/src/main.js]
* **UI Initialization**: Resolved a `ReferenceError` caused by event listeners running before the DOM was fully loaded. The custom TTC control logic is now correctly initialized after the `DOMContentLoaded` event, ensuring stability.
This commit introduces a suite of major improvements focused on application robustness, user experience, and bug fixes. The changes overhaul the caching system, enhance the file loading experience, and resolve critical state management issues.
### ✨ New Features & Enhancements
1. **Robust Caching System**:
- The IndexedDB caching logic now stores file metadata (filename, size) alongside the file blob.
- Implemented a **versioning check** by comparing filenames to prevent loading stale, outdated cache.
- Added an **integrity check** by comparing file sizes to detect and discard corrupted or incomplete cached data.
- Implemented graceful error handling for browser `QuotaExceededError`.
2. **Progress Bar for All Loading Operations**:
- The smooth, worker-based progress bar now appears when loading JSON data from the **IndexedDB cache**, providing a consistent experience with fresh file loads.
- A new progress bar has been implemented for **video file loading**. It tracks the browser's buffering progress and appears for both fresh file selections and cached reloads.
3. **UI Polish**:
- A **favicon** has been added to the application tab for a more professional look.
### 🐛 Bug Fixes
1. **Corrected Worker Parsing Logic**:
- Fixed a critical bug in the JSON parsing web worker (`parser.worker.js`) where its logic failed to handle nested objects (like `pointCloud` arrays). The worker now uses a robust algorithm to correctly build the entire JSON tree, ensuring data is always parsed accurately.
2. **Fixed JSON Reloading**:
- Resolved an issue where loading a new JSON file over an existing one would fail. The application now properly removes old p5.js visualization instances before creating new ones, ensuring a clean state for the new data.
3. **Fixed Speed Graph on Cached Load**:
- Corrected a bug where the speed graph would not appear when the application started up from a cached session. The initialization logic now correctly creates and updates the speed graph after the cached video's metadata is loaded.
Main:
This major update refactors the entire application from a single monolithic HTML file into a modern, modular JavaScript architecture for improved maintainability, performance, and future extensibility.
Alongside the refactoring, this commit introduces a completely overhauled synchronization engine and several quality-of-life improvements.
Key changes and new features include:
- **Modular Architecture**: The application is now split into distinct, decoupled modules for state management (`state.js`), DOM manipulation (`dom.js`), synchronization (`sync.js`), file parsing (`fileParsers.js`), and UI components (`p5/radarSketch.js`, `modal.js`, etc.).
- **Robust Synchronization Engine**:
- The core playback loop in `sync.js` now correctly applies the manual time offset, ensuring accurate synchronization between the video and radar data during playback.
- Fixed a bug where fast scrubbing with the timeline slider could leave a persistent drift while paused. The fix uses the video's `seeked` event for a reliable, event-driven UI update.
- **Enhanced User Experience**:
- Added a new feature allowing users to press 'Enter' in the offset input box to instantly resync the video to the current radar frame, which significantly streamlines the manual calibration process.
- **Improved Debugging Tools**:
- The advanced debug overlay's drift calculation is now "offset-aware," providing an accurate representation of the true synchronization status during both playback and seeking.
2.) feat(visualization): Add advanced overlays and robust large-file streaming
This commit introduces several major features and critical bug fixes to the visualizer, significantly enhancing its analytical capabilities and performance.
The primary focus was on adding more detailed visualization overlays from the Kalman filter and implementing a robust solution for handling very large JSON files that were previously crashing the application.
### New Features
- **Covariance Ellipse Overlay:**
- A "Show Covariance" checkbox has been added to the UI.
- When enabled, the visualizer now draws the 95% confidence ellipse for each track's predicted position, derived from the `covarianceP` matrix. This provides a real-time view of the Kalman filter's positional uncertainty.
- **Predicted vs. Corrected Position Markers:**
- A "Show Predicted Position" checkbox has been added.
- This feature displays the filter's raw prediction (red cross) alongside the final corrected position (blue cross), making it easy to visualize the predict-correct cycle and analyze the filter's behavior during object acceleration or maneuvers.
### Bug Fixes & Performance Enhancements
- **Fix: Marker for Lost Tracks:**
- The main track marker (blue cross) now correctly disappears if its `correctedPosition` is null for a given frame. This provides a clear and intuitive visual cue that a track has been temporarily "lost" and is coasting on predictions alone.
- **Fix: Large JSON File Parsing (Streaming & Web Worker):**
- Resolved a critical "Maximum call stack size exceeded" error that occurred when loading JSON files larger than ~30MB.
- The entire file loading and parsing pipeline has been refactored to use a Web Worker. This moves the CPU-intensive parsing off the main UI thread, preventing the page from freezing.
- The worker streams the file and uses a lightweight parser (Clarinet.js) to build the data object, ensuring low memory usage and a responsive interface.
- **feat: Add Progress Bar for File Loading:**
- To provide feedback during the new streaming process, a progress bar is now displayed in the modal, showing the real-time progress of the file parsing operation.
