Deep Learning's Inevitable Role in Autonomous Driving: Evolution from Engineering to Learning

The discussion centers on the role of deep learning in solving the autonomous driving problem, with Yann LeCun asserting that it is an "obviously part of the solution" and indispensable for future systems. He posits that the development of AI-like systems generally progresses through phases: initially hand-built, then a hybrid of learning and extensive engineering, and finally, increasing reliance on learning. Autonomous driving is currently in the hybrid phase, but the long-term trajectory will see deep learning become the dominant paradigm. LeCun distinguishes between current approaches, exemplified by Waymo, which achieve a "decent level of autonomy" by heavily constraining the operational environment (e.g., specific geographic areas with good weather, wide roads). These systems also rely on expensive, sophisticated sensors like LIDAR and extensive pre-mapping of the entire world, reducing the perception task to identifying only moving objects and dynamic changes. This contrasts with the envisioned future where learning systems handle a broader range of complexities without such heavy engineering and environmental limitations. The current state of autonomous driving, while impressive in controlled settings, highlights the limitations of heavily engineered solutions in achieving generalized autonomy. The "over-engineering" of cars with costly sensors and detailed 3D mapping, while effective for fleet operations in specific zones, is not scalable or cost-effective for consumer vehicles or unconstrained environments. The practical insight is that while engineering provides initial breakthroughs and handles corner cases, it ultimately gives way to more robust, learning-based systems for true generalization. The evolution of autonomous driving is framed within a broader historical context of AI development, drawing parallels with fields like character recognition, speech recognition, computer vision, and natural language processing. In each of these domains, initial reliance on engineered rules and limited learning gradually shifted towards deep learning as the primary driver of progress. This historical pattern suggests that autonomous driving will follow suit, eventually leveraging combinations of supervised learning and model-based reinforcement learning to achieve full, unconstrained autonomy, signifying a fundamental shift in how complex real-world AI problems are tackled.

Elon Musk is confident that large-scale data and deep learning can solve the autonomous driving problem

I mean I don't think we'll ever have a set driving system or at least not in the foreseeable future that does not use deep learning

in the history of sort of engineering particularly is sort of sort of a I like systems is generally your first phase where everything is built by hand

there's a phase where this a little bit of learning is used but there's a lot of engineering that's involved in kind of you know taking care of corner cases

as technology progresses we end up relying more and more on learning

the same is going to happen with with the time is driving that currently the the the methods that are closest to providing some level of autonomy... is where you constrain the world

you completely over engineer the car with tons of lidars and sophisticated sensors that are too expensive for consumer cars

the long term solution is gonna rely more and more on learning and possibly using a combination of supervised learning and model-based reinforcement or something like that