Learning to learn: Automatic adaptation of learning bias

Abstract

Traditionally, large areas of research in machine learning have concentrated on pattern recognition and its application to many diversified problems both within the realm of AI as well as outside of it. Over several decades of intensified research, an array of learning methodologies have been proposed, accompanied by attempts to evaluate these methods, with respect to one another on small sets of real world problems. Unfortunately, little emphasis was placed on the problem of learning bias - common to all learning algorithms - and a major culprit in preventing the construction of a universal pattern recognizer. State of the art learning algorithms exploit some inherent bias when performing pattern recognition on yet unseen patterns. Automatically adapting this learning bias - dependent on the type of pattern classification problems seen over time - is largely lacking. In this paper, weaknesses of the traditional one-shot learning environments are pointed out and the move towards a learning method displaying the ability to learn about learning is undertaken. Trans-dimensional learning is introduced as a means to automatically adjust learning bias and empirical evidence is provided showing that in some instances learning the whole can be simpler than learning a part of it.