A COMPACT NETWORK MODEL FOR LEARNING IN DISTRIBUTION SPACE

Abstract

Despite the superior performance of deep learning methods in various tasks, challenges remain in the area of regression on function spaces. In this thesis, we address the problem of regression where the data are distributions. However, neural networks are not designed for distribution inputs - the networks are unable to encode function inputs compactly as each node encodes just a real value. To that end, we propose our distribution regression network (DRN) which encodes an entire function in each network node. We conducted comprehensive experiments to test DRN against other methods and show that DRN uses at least two times fewer training data and has better accuracies even with increasing data sampling noise and task difficulty. As an application, we use DRN in a defense method against adversarial attacks in convolutional neural networks by integrating with existing transformation-based defenses.