Zoomable video playback on mobile devices by selective decoding

Abstract

Modern mobile devices support multi-touch gestures that allow users to naturally zoom into and pan around Web pages, photos, and videos. When users zoom into a video, only part of the region in the video frames are displayed. Ideally, only the regions that the user is viewing are decoded, reducing the computation time (hence increasing the playback frame rate) and power consumption. We call this selective decoding. We have implemented a system consisting of an offline analyzer and a mobile video player that implements selective decoding in MPEG-4 Part 2 Simple Profile codec. The analyzer traces various dependency relationships among macroblocks of a given video and produces a meta-data file. The mobile video player supports zoom and pan gestures, and uses the meta-data to trace the macroblocks that are needed to decode the RoI. The player uses a modified decoding process to decode macroblocks selectively based on the trace. Our experiments show that selective decoding can improve playback frame rate by up to 193.3% and reduce energy consumption by up to 64.5%. © 2012 Springer-Verlag.