Self-Driving Cars: State of the Art, Challenges, and the Human Experience (2019)

This podcast episode provides a comprehensive overview of the autonomous vehicle (AV) landscape in 2019, emphasizing the core mission of improving mobility, increasing efficiency, and critically, saving lives, given the sobering statistic of a car crash fatality every 23 seconds globally. The host reviews significant milestones from 2018, including Waymo's 10 million autonomous miles and Tesla Autopilot's 1 billion semi-autonomous miles, highlighting the role of computer vision and neural networks in these advancements. However, the discussion also confronts the realities of AV fatalities, such as the Uber pedestrian crash and a Tesla Autopilot driver fatality, and the disproportionate public and media response that sets an extremely high bar for AV performance.

A key distinction is drawn between the optimistic predictions from automakers and the more cautious outlooks from leading engineers like Carl Iagnemma and Gill Pratt, who stress the enduring necessity of a "human in the loop" or teleoperation. The host further simplifies the levels of autonomy into "human-centered autonomy" (human responsible) and "full autonomy" (car responsible), defining the latter as a system that can guarantee safety without human intervention, even in critical situations. This segment underscores the profound difficulty of achieving true full autonomy, which goes beyond mere technical capability to encompass legal and experiential aspects.

The episode delves into various deployment strategies for both fully autonomous and semi-autonomous vehicles, ranging from last-mile delivery and highway trucking platoons to constrained urban routes and closed communities. It also explores "out-there" ideas like connected vehicle infrastructure for traffic optimization, underground tunnel systems, and even flying cars, exemplified by Uber Elevate. The host critically argues that widespread adoption of AVs in the near term will not primarily be driven by safety, speed, or cost, as current AVs are often slower and more cautious.

Instead, the host posits that the ultimate driver for AV adoption will be a superior "human experience"—one that is engaging, personalized, and "fun as hell," integrating natural language communication and seamless control transfer. This perspective challenges the common engineering-first approach, suggesting that the human experience should be a co-equal design consideration from the outset, rather than an afterthought. The discussion concludes by revisiting the historical context of the DARPA Challenges, questioning the underlying beliefs that driving is an "easy" task and humans are "bad" drivers, and highlighting the complex, subtle nonverbal communication and ethical dilemmas that current algorithms struggle to address.

"Every 23 seconds somebody in the world dies in a car, auto crash. It should be a sobering, it is for me, thing that I think about every single day."

"That is probably the largest deployment of neural networks in the world that has a direct impact on a human life, that's able to decide, that's able to make life critical decisions many times a second over and over."

"the bar is much higher on every level in terms of performance. So in order to success, as I'll argue, in order to design successful autonomous vehicles those vehicles will have to take risks."

"But when you just look at the numbers, Tesla Autopilot's three times safer than manually driven vehicles. But that's not the right way to look at it."

"removing the human from consideration is really, really far away." (Gill Pratt)

"my guess is that in probably 10 years it will be very unusual for cars to be built that are not fully autonomous." (Elon Musk, 2017)

"autonomy will only take over if, not take over but be adopted by human beings if it creates a better human experience."

"I believe you first have to make it fun as hell to be in the car and then solve the autonomous vehicle problem jointly."

"Full autonomy means the car is responsible both on the legal side, the experience side and the algorithm side."

"The subtle vehicle-to-vehicle, vehicles-to-pedestrian nonverbal communication that happens here in a dramatic sense but really happens in the subtle sense millions of times every single day in Boston."

"The fact that by us accelerating we might make that pedestrian stop, it's something that we have to incorporate into algorithms and we don't today."

"from the human factors, psychology side, there's been over 70 years of research showing that humans are not able to monitor, maintain vigilance, monitoring a system."