Affect Analysis in Video

Abstract

In this thesis, we first propose a computational framework to build the representation and model from the affective video content to the categorical emotional states, while developing a computational measure for the intensity of categorical emotional states. Specifically, a sparse vector representation is proposed in this computational framework. The intensity of emotion can be represented by the values computed from the sparse vector. Then, the modeling of affective content video addresses the problem of obtaining the representative sparse vectors based on the low-level features extracted from video.

Then, we propose an approach that employs affective analysis to automatically create video presentations from home videos. Our novel method adaptively creates presentations for family, acquaintances and outsiders based on three properties: emotional tone, local main character and global main character.

Besides the adaptive presentation of home videos, this thesis also exploits the affective analysis (facial expression cue), eye gaze data and previous emotional states to develop an online multimodal approach for estimating the subtle facial expression.

Furthermore, this thesis also utilizes the affective analysis to propose a novel approach to share home photos based on the aesthetic, affective and social features.