Document dependent fusion in multimodal music retrieval

Abstract

In this paper, we propose a novel multimodal fusion frame-work, document dependent fusion (DDF), which derives the optimal combination strategy for each individual document in the fusion process. For each document, we derive a document weight vector by estimating the descriptive abilities of its different modalities. The document weight vector also enables our framework to be easily integrated with existing multimodal fusion schemes, and achieve a better combination strategy for each document given a query. Experiments are conducted on a 17174-song music database to compare the retrieval accuracy of traditional query independent fusion and query dependent fusion approaches, and that obtained after integrating DDF with them. Experimental results indicate that DDF can significantly improve the retrieval performance of current fusion approaches. Copyright 2011 ACM.