Up-Fusion: An evolving multimedia decision fusion method

Abstract

The amount of multimedia data available on the Internet has increased exponentially in the past few decades and is likely to keep on increasing. Given that a multimedia system has multiple information sources, fusion methods are critical for its analysis and understanding. However, most of the traditional fusion methods are static with respect to time. To address this, in recent years, several evolving fusion methods have been proposed. However, they can only be used in limited scenarios. For example, the context aware fusion methods need the context information to update the fusion model, but the context may not always be available in many applications. In this paper, a new evolving fusion method is proposed based on the online portfolio selection theory. The proposed method takes the correlation among different information sources into account, and evolves the fusion model when new multimedia data is added. It can deal with either crisp or soft decisions without requiring additional context information. Extensive experiments on concept detection task over TRECVID dataset have been conducted, and promising results have been obtained. Copyright 2011 ACM.