REPRESENTATION LEARNING WITH DOMAIN-SPECIFIC BAYESIAN PRIORS

Abstract

How data should be represented so as to maximize the performance of a machine learning algorithm hinges on the priors that act as inductive bias in the algorithm to give rise to the representations desired. While learning with these generic priors leads to fairly good representations, in this thesis, we aim to answer the following question: should domain-specific priors be incorporated when accomplishing certain tasks of interest and how?

To answer it, we resort to the No Free Lunch Theorem and aim to incorporate into machine learning algorithms the priors that encode our belief about the representations or the tasks. With different domain-specific priors injected, we show that several characteristics of data representations can be achieved, respectively. Our research suggests that prior design should be task-driven and that domain-specific priors should be incorporated as proper inductive bias in accomplishing certain tasks of interest.