Sensor-rich video exploration on a map interface

Abstract

Result presentations from searches into video repositories is still a challenging problem. Current systems usually display a ranked list that shows the first frame of each video. Users then explore the videos one-by-one. In our recent work we have investigated the fusion of captured video with a continuous stream of sensor meta-data. These so-called sensor-rich videos can conveniently be captured with today's smartphones. Importantly, the recorded sensor-data streams enable processing and result presentation in novel and useful ways. In this demonstration we present a system that provides an integrated solution to present videos based on keyframe extraction and interactive, map-based browsing. As a key feature, the system automatically computes popular places based on the collective information from all the available videos. For each video it then extracts keyframes and renders them at their proper location on the map synchronously with the video playback. All the processing is performed in real-time, which allows for an interactive exploration of all the videos in a geographic area. Copyright 2011 ACM.