Please use this identifier to cite or link to this item: http://scholarbank.nus.edu.sg/handle/10635/138162
Title: FAST RATE ANALYSIS OF STOCHASTIC OPTIMIZATION IN STATISTICAL ESTIMATION
Authors: QU CHAO
Keywords: machine learning, stochastic optimization, statistical estimation, big data, nonconvex, high dimensional statistics
Issue Date: 3-Aug-2017
Citation: QU CHAO (2017-08-03). FAST RATE ANALYSIS OF STOCHASTIC OPTIMIZATION IN STATISTICAL ESTIMATION. ScholarBank@NUS Repository.
Abstract: The interplay of optimization and machine learning becomes an important part of modern artificial intelligence. On one hand, a lot of machine learning problem can be formulated into the optimization problem, and then in the training phase, parameters in these models are tuned using optimization algorithm. Optimization approaches have enjoyed prominence in machine learning because of their wide applicability and attractive theoretical properties. On the other hand, the increasing complexity of machine learning model and big data era push modern optimization algorithms to a higher level. Our study focuses on the stochastic first order method for the high dimensional statistics model. In particular, we investigate the convex stochastic optimization problem and propose the modified regret to relax the strong convexity assumption in some well-known algorithms. We then extend this to solve the large scale robust optimization problem. At last, we study the finite-sum problem, which covers several important formulations such as Lasso, group Lasso, logistic regression, and some non-convex models such as linear regression with SCAD regularization. We show that three variance reduced randomized first order methods enjoy the fast linear convergence even in the non-convex setting under the assumption of restricted strong convexity.
URI: http://scholarbank.nus.edu.sg/handle/10635/138162
Appears in Collections:Ph.D Theses (Open)

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
QuC.pdf2.95 MBAdobe PDF

OPEN

NoneView/Download

Page view(s)

109
checked on Dec 6, 2018

Download(s)

164
checked on Dec 6, 2018

Google ScholarTM

Check


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.