EFFICIENT MACHINE LEARNING METHODS FOR BIG DATA ANALYTICS

Abstract

In this thesis, we study methods to accelerate the optimization of existing sparse machine learning models, so that they are applicable to problems with a huge amount of samples or features. In particular, we consider three models that are widely used in real world applications: support vector machines (SVM), tree-structured group sparsity and sparse Gaussian process regression. We first present a strictly safe sample screening method for early discarding of inactive samples prior to solving the SVM, so that the problem size of the SVM can be substantially reduced. Then, we develop a dynamic feature screening method for tree-structured group sparsity regularization, to progressively eliminate inactive features during its optimization procedure. Lastly, we present an approximate decomposition method to accelerate the sparse Gaussian process regression, wherein several much smaller subproblems are solved instead of the original one. In summary, we develop various efficient methods to enable prevalent sparse machine models to be applicable to problems with a huge amount of samples or features.