EFFICIENT ARCHITECTURE DESIGN FOR DEEP NEURAL NETWORKS

Abstract

Neural network architecture design is always one of the most attractive and important research topics in the era of deep learning. Early neural network design [112, 49, 71] mostly relies on standard convolutions and pooling techniques. Despite of achieving good performance, their learnable parameters and computational cost are often huge and hence restrict their applications in real world applications, such as on the mobile devices. The focus of this thesis is on the design, deployment and evaluation of efficient deep learning models (such as the deep neural networks). a comprehensive design framework of deep learning neural networks are presented with methods of evaluation on real-world images and tasks. We believe the novel concepts and technologies presented in the thesis lay a solid foundation for many promising directions such as using AutoML for drag discovery and building universal deep learning models for all tasks.