THE SYSTEM ARCHITECTURE AND METHODOLOGY FOR LIDAR AND CAMERA FUSION

Abstract

Object detection for collision avoidance is a fundamental technique for the future autonomous vehicle. Over the past years, the passive vision-based, or active radar-based and fused radar plus vision based object detection systems had a rapid growth. Recently, the LiDAR and vision based fusion system has started to attract the research communities because LiDAR has an excellent resolution compared to radar. It provides the 3D geometry information of the environment. Currently, the LiDAR and vision based fusion system used the traditional hand-crafted features such as LBP, Harris, HoG, edge features or optical flow by combining with Support Vector Machine (SVM) or Hidden Markov model (HMM) to detect the objects from the images while use LiDAR to verify the results. On the other hand, the deep learning based methods offer superior performance to the traditional hand-crafted features algorithms on the ImageNet Large Scale Visual Recognition Challenge. However, the deep learning based method still exists the problems in classical methods such as generalization or loss accuracy under uncontrolled environments. To support future autonomous driving with more safety, we proposed a system with LiDAR and deep neural networks for supporting object detection in road safety application. The proposed system used the light beams to guarantee the driving safety areas. It also uses LiDAR and convoluted features to enhance the object detection. Furthermore, the proposed system has the ability to update the neural networks from learning unknown objects, which are proposed from LiDAR with manual labeling.