PRUNING NEURAL NETWORKS USING DETERMINANTAL POINT PROCESSES

Abstract

Neural network pruning aims at lowering the complexity of a query to a neural network which is already trained. In this scope, various strategies can be imagined in order to find a sub network, which performs nearly as optimally as the main one, without having to relearn a second time. In this master thesis, we tackle the issue of neural network pruning via determinantal point processes (DPP). The goal of this work is to provide some theoretical guarantees on the empirical work from Mariet and Sra from 2016. To find theoretical guarantees, we first give a general approach on DPPs. Then, we analyse neural network pruning via the lens of the coresets in the most general setting. Finally, we generalize an approach coming from statistical mechanics to better understand the generalization error in a neural network pruned with a DPP in the noiseless teacher/student framework.