MIT-AVT's Ryder: A Robust Data Collection System for Large-Scale Semi-Autonomous Driving Research

This podcast transcript details the 'Ryder' data collection device, a core component of the MIT Autonomous Vehicle Technology (AVT) study. The primary goal of Ryder is to gather vast amounts of naturalistic driving data to understand human behavior in semi-autonomous and autonomous vehicles. The system is designed for extreme reliability, operating month after month across multiple vehicles and diverse weather conditions. It integrates various sensor streams, including three cameras, an IMU, GPS, and raw CAN messages from the vehicle, all managed by a custom Linux-based single board computer and stored on a solid-state drive.\n\nThe technical challenges addressed by Ryder are significant, particularly concerning data volume and synchronization. The system handles hundreds of thousands of miles of driving data, equating to petabytes uncompressed, which necessitates onboard H.264 video compression performed directly by the cameras to reduce the load on the single board computer and manage storage. A crucial aspect is the precise synchronization of all sensor streams, achieved with a real-time clock providing two parts per million accuracy, ensuring that video frames, GPS signals, and other data are perfectly aligned for subsequent analysis. Intelligent power management and recording activation, triggered by CAN bus activity, allow the system to autonomously start and stop recording without driver intervention.\n\nThe data pipeline involves physical hard drive swaps from the Ryder boxes, followed by remote copying to a central server. Here, the data undergoes consistency checks, initial cleaning to remove corrupt or anomalous entries, and rigorous synchronization. This meticulous preparation is essential for feeding clean, perfectly aligned multi-sensor datasets into computer vision and deep learning algorithms, which are used for tasks like driver behavior analysis and neural network model training. The system's robustness was tested against extreme conditions, such as high temperatures, and includes mechanisms to restart subsystems or shut down recording if critical sensors fail, prioritizing data integrity.\n\nFuture improvements for Ryder include transitioning to more powerful single-board computers like the Jetson TX2, which offers greater compute power for real-time processing and the integration of additional sensors. A key future direction is the development of onboard algorithms to intelligently filter and prune data, discarding "boring" driving segments and selectively recording only "interesting" events or edge cases relevant to driver behavior and AI model training. This shift from collecting all data to collecting only salient data is driven by the study's core principle: the need for rich internal environment data—such as driver's face, glance, cognitive load, and body pose—to design safe and effective autonomous vehicles that account for human supervision.

the MIT autonomous vehicle technology study is all about collecting large amounts of naturalistic driving data

all of that has to be collected reliably synchronized and post processed once we offload the data

one of the essential elements for that was to do compression onboard so these are Logitech c920 webcam they can do up to 1080p at 30 frames a second the major reason why we went with these is because they do onboard h.264 compression of the video

creating a system that would intelligently and autonomously turn off and on to start and end recording was also a key aspect to this device

synchronization was the highest priority from the very beginning of writers design we have a real-time clock onboard writer that allows us down to two parts per million of accuracy and time stamping

the important hard work behind the magic that deep learning computer vision unlocks is the synchronization the cleaning of the messy data

most of driving is quite boring nothing interesting in terms of understanding driver behavior or training computer vision models for edge cases

we truly believe that autonomy autonomous vehicles require an understanding of how human supervisors of those systems behave how we can keep them attentive keep their glance on the road keep them as effective efficient supervisors of those systems

we need to have rich sensor information about the internal environment what the driver is doing everything about their face the glance all the cognitive load and body pose everything about their activity

we're talking about an incredible amount of data all compressed with h.264 that's close to 300 terabytes worth of data