ALGORITHMIC INDUCTIVE BIASES FOR GRAPH REPRESENTATION LEARNING

Abstract

Deep learning has witnessed phenomenal growth in the last decade. An important factor in the success of deep learning is the ability to encode structure in the neural network that can provide useful inductive bias in learning. However, introducing useful structures in the neural network can often be non-trivial and may need significant domain knowledge. In this thesis, we propose to use the knowledge of well-known algorithms to provide the inductive bias needed for effective learning. Algorithms usually contain simple and iteratively repeated update steps which can help in capturing inherent structure of the data as well as in sharing of parameters. We focus on graph structured data and demonstrate that Algorithmic Inductive Biases are a good tool to improve the representations learnt by Graph Neural Networks (GNNs). Our models enhance the representations learnt on graphs by improving generalization to higher-order structures and increasing the expressive power of GNNs.