Please use this identifier to cite or link to this item:
|Title:||EXPLOITING GRADIENT INFORMATION FOR MODERN MACHINE LEARNING PROBLEMS||Authors:||CHEN YIZHOU||ORCID iD:||orcid.org/0000-0002-3628-7555||Keywords:||gradient, Bayesian deep learning, meta-learning, active learning||Issue Date:||10-Jan-2022||Citation:||CHEN YIZHOU (2022-01-10). EXPLOITING GRADIENT INFORMATION FOR MODERN MACHINE LEARNING PROBLEMS. ScholarBank@NUS Repository.||Abstract:||Many deep learning achievements are attributed to the back-propagation (BP) algorithm, which exploits gradient information of the deep neural network (DNN) models: BP efficiently computes the gradient of the loss function with respect to the weights of a DNN for a batch of examples, and such gradient can be used by stochastic gradient descent to perform learning / optimization of the DNN model. Despite recent advances in deep learning like DNN training, there are still important scenarios where we can also use gradient to tackle optimization difficulty. In a broader aspect of deep learning rather than DNN training, a significant challenge faced by ML practitioners is thus whether we can design efficient algorithms to use the model gradient in the training / optimization in various deep learning scenarios. This thesis identifies four important scenarios and, for each of them, proposes a novel algorithm to utilize the gradient information for effective optimization that is both theoretically grounded and practically effective.||URI:||https://scholarbank.nus.edu.sg/handle/10635/226224|
|Appears in Collections:||Ph.D Theses (Open)|
Show full item record
Files in This Item:
|Thesis_Finalized_ChenYizhou.pdf||10.16 MB||Adobe PDF|
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.