DEEP NEURAL NETWORKS

Abstract

Since the late 2000?s, a new area of research in Machine Learning has emerged, known as Deep Learning. Based among others on observations in neuroscience such as the structure of the visual system in the brain, it is believed that, in order

to achieve the AI-dream of building truly intelligent agents, one needs to build models with deep architectures. One class of such models is the class of deep neural networks. Until recently, the idea to train deep neural networks had not shown much success. Among other reasons, it is often mentioned that computers used to be too slow and labeled datasets used to be too small. In this report, we will more particularly emphasize the breakthrough that happened in 2006, when unsupervised learning was shown to help a lot when training deep artificial neural networks