DEEP NEURAL NETWORK APPROXIMATION VIA FUNCTION COMPOSITIONS

Abstract

Deep neural networks have made significant impacts in many fields of computer science and engineering, especially for large-scale and high-dimensional learning problems. This thesis focuses on the approximation theory of deep neural networks. We provide (nearly optimal) approximation error estimates in terms of the width and depth when constructing ReLU networks, via the idea of function compositions, to uniformly approximate polynomials, continuous functions, and smooth functions on a hypercube. The optimality of the approximation error estimates is discussed via connecting the approximation property to VC-dimension. Finally, we introduce a new class of networks built with either Floor (the floor function) or ReLU as the activation function in each neuron, which provides a much better approximation error than that of ReLU networks.