Learning to Drive: Perception for Autonomous Cars

Every year, 1.2 million people die in automobile accidents and up to 50 million are injured [1]. Many of these deaths are due to driver error and other preventable causes. Autonomous or highly aware cars have the potential to positively impact tens of millions of people. Building an autonomous car is not easy. Although the absolute number of traf- fic fatalities is tragically large, the failure rate of human driving is actually very small. A human driver makes a fatal mistake once in about 88 million miles [2]. As a co-founding member of the Stanford Racing Team, we have built several relevant prototypes of autonomous cars. These include Stanley, the winner of the 2005 DARPA Grand Challenge and Junior, the car that took second place in the 2007 Urban Challenge. These prototypes demonstrate that autonomous vehicles can be successful in challenging environments. Nevertheless, reliable, cost-effective perception under uncertainty is a major challenge to the deployment of robotic cars in practice. This dissertation presents selected perception technologies for autonomous driving in the context of Stanford’s autonomous cars. We consider speed selection in response to terrain conditions, smooth road finding, improved visual feature optimization, and cost effective car detection. Our work does not rely on manual engineering or even supervised machine learning. Rather, the car learns on its own, training itself without human teaching or labeling. We show this “self-supervised” learning often meets or exceeds traditional methods. Furthermore, we feel self-supervised learning is the only approach with the potential to provide the very low failure rates necessary to improve on human driving performance.