Please use this identifier to cite or link to this item: https://doi.org/10.1109/ISM.2011.57
Title: Affective video summarization and story board generation using pupillary dilation and eye gaze
Authors: Katti, H. 
Yadati, K.
Kankanhalli, M. 
Tat-Seng, C. 
Keywords: affective summarization
Pupillary dilation
video summarization
Issue Date: 2011
Source: Katti, H.,Yadati, K.,Kankanhalli, M.,Tat-Seng, C. (2011). Affective video summarization and story board generation using pupillary dilation and eye gaze. Proceedings - 2011 IEEE InternationalSymposium on Multimedia, ISM 2011 : 319-326. ScholarBank@NUS Repository. https://doi.org/10.1109/ISM.2011.57
Abstract: We propose a semi-automated, eye-gaze based method for affective analysis of videos. Pupillary Dilation (PD) is introduced as a valuable behavioural signal for assessment of subject arousal and engagement. We use PD information for computationally inexpensive, arousal based composition of video summaries and descriptive story-boards. Video summarization and story-board generation is done offline, subsequent to a subject viewing the video. The method also includes novel eye-gaze analysis and fusion with content based features to discover affective segments of videos and Regions of interest (ROIs) contained therein. Effectiveness of the framework is evaluated using experiments over a diverse set of clips, significant pool of subjects and comparison with a fully automated state-of-art affective video summarization algorithm. Acquisition and analysis of PD information is demonstrated and used as a proxy for human visual attention and arousal based video summarization and story-board generation. An important contribution is to demonstrate usefulness of PD information in identifying affective video segments with abstract semantics or affective elements of discourse and story-telling, that are likely to be missed by automated methods. Another contribution is the use of eye-fixations in the close temporal proximity of PD based events for key frame extraction and subsequent story board generation. We also show how PD based video summarization can to generate either a personalized video summary or to represent a consensus over affective preferences of a larger group or community. © 2011 IEEE.
Source Title: Proceedings - 2011 IEEE InternationalSymposium on Multimedia, ISM 2011
URI: http://scholarbank.nus.edu.sg/handle/10635/41354
ISBN: 9780769545899
DOI: 10.1109/ISM.2011.57
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

16
checked on Dec 13, 2017

Page view(s)

68
checked on Dec 9, 2017

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.