PERCEPTION WITH DEEP NEURAL NETWORKS FOR AUTONOMOUS VEHICLES

Abstract

Autonomous driving is generally accepted to be the next disruptive technology that could bring enormous positive social and economic impact to freight transportation and public transportation with potentially higher productivity, flexibility, and safety. The perception module is a crucial component of an autonomous vehicle. It is responsible for processing multimodal data collected by the sensors equipped in the car and building an environment model to localize and provide information for the planning module to navigate the world safely. The high confidence in the future of autonomous driving technology substantially benefits from the recent great advancement in artificial intelligence, in particular deep learning. The focus of this thesis is to improve the perceptual ability of the autonomous vehicle by advancing the state-of-the-art deep learning architecture design in three crucial perception tasks, including vision-based localization, 2D semantic segmentation, and 3D point cloud processing.