BRAIN-SCIENCE INSPIRED COMPUTATIONAL MODELS FOR VISION TASKS

Abstract

Attention mechanism enables fast processing of visual information for humans to make accurate reactions to the rapidly changing environment. Inspired by the neural behavior, computational models have achieved tremendous progress in understanding visual mechanisms and related subjects. In this thesis, we explore various computational models and datasets for a better understanding of subjects related to human visual systems. In the first project, we built up a large visual attention dataset SALICON based on a mouse-tracking paradigm that mimics human visual behavior. This dataset serves as a benchmark for computational models of top-down or bottom-up visual attentions. We then present two computational models, one is based on human cognition on artificially synthesized stimuli, and the other is based on a generative multimodal model that jointly learns visual representations and language representations. These models contribute to a better understanding of human cognition on artificial and multimodal stimuli.