Please use this identifier to cite or link to this item: https://doi.org/10.1109/TMM.2013.2280437
Title: Query-document-dependent fusion: A case study of multimodal music retrieval
Authors: Li, Z.
Zhang, B.
Yu, Y.
Shen, J.
Wang, Y. 
Keywords: Information retrieval
Multimodal
Query-document-dependent fusion
Issue Date: 2013
Citation: Li, Z., Zhang, B., Yu, Y., Shen, J., Wang, Y. (2013). Query-document-dependent fusion: A case study of multimodal music retrieval. IEEE Transactions on Multimedia 15 (8) : 1830-1842. ScholarBank@NUS Repository. https://doi.org/10.1109/TMM.2013.2280437
Abstract: In recent years, multimodal fusion has emerged as a promising technology for effective multimedia retrieval. Developing the optimal fusion strategy for different modalities (e.g., content, metadata) has been the subject of intensive research. Given a query, existing methods derive a unified fusion strategy for all documents with the underlying assumption that the relative significance of a modality remains the same across all documents. However, this assumption is often invalid. We thus propose a general multimodal fusion framework, query-document-dependent fusion (QDDF), which derives the optimal fusion strategy for each query-document pair via intelligent content analysis of both queries and documents. By investigating multimodal fusion strategies adaptive to both queries and documents, we demonstrate that existing multimodal fusion approaches are special cases of QDDF and propose two QDDF approaches to derive fusion strategies. The dual-phase QDDF explicitly derives and fuses query- and document-dependent weights, and the regression-based QDDF determines the fusion weight for a query-document pair via a regression model derived from training data. To evaluate the proposed approaches, comprehensive experiments have been conducted using a multimedia data set with around 17 K full songs and over 236 K social queries. Results indicate that the regression-based QDDF is superior in handling single-dimension queries. In comparison, the dual-phase QDDF outperforms existing approaches for most query types. We found that document-dependent weights are instrumental in enhancing multimedia fusion performance. In addition, efficiency analysis demonstrates the scalability of QDDF over large data sets. © 2013 IEEE.
Source Title: IEEE Transactions on Multimedia
URI: http://scholarbank.nus.edu.sg/handle/10635/77906
ISSN: 15209210
DOI: 10.1109/TMM.2013.2280437
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

2
checked on Jun 15, 2018

WEB OF SCIENCETM
Citations

1
checked on May 28, 2018

Page view(s)

32
checked on May 11, 2018

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.