Please use this identifier to cite or link to this item:
Title: Adaptive multimodal fusion based similarity measures in music information retrieval
Keywords: adaptive, multimodal, fusion, similarity measure, music, information retrieval
Issue Date: 12-Aug-2010
Citation: ZHANG BINGJUN (2010-08-12). Adaptive multimodal fusion based similarity measures in music information retrieval. ScholarBank@NUS Repository.
Abstract: In the field of music information retrieval (MIR), one fundamental research problem is the measuring of the similarity between music documents. Based on a viable similarity measure, MIR systems can be made more effective to help users retrieve relevant music information. Music documents are inherently multi-faceted. They contain not only multiple sources of information, e.g., textual metadata, audio content, video content, images, etc. but also multiple aspects of information, e.g., genre, mood, rhythm, etc. Fusing the multiple modalities effectively and efficiently is essential in discovering good similarity measures. In this thesis, I propose and investigate a comprehensive adaptive multimodal fusion framework to construct more effective similarity measures for MIR applications. The basic philosophy is that music documents with different content require different fusion strategies to combine their multiple modalities. Besides, the same multiple documents in different contexts need adaptive fusion strategies to derive effective similarity measures in certain multimedia tasks. Based on the above philosophy, I proposed a multi-faceted music search engine that allows users to customize their most preferred music aspects in a search operviation so that the similarity measure underlying the search engine is adapted to the users? instant information needs. This adaptive multimodal fusion based similarity measure allows more relevant music items to be retrieved. On this multi-faceted music search engine, a query-dependent fusion approach was also proposed to improve the adaptiveness of the music similarity measure to different user queries. Revealed in the experimental results, the proposed adaptive fusion approach improved the search effectiveness by combining the multiple music aspects with customized fusion strategies for different user queries. We also investigated state-of-the-art fusion techniques in audio-visual violin transcription task and built a prototype system for violin tutoring in a home environment based on the audio-visual fusion techniques. Future plans are proposed to investigate the adaptive fusion approaches in semantic music similarity measures so that a more user-friendly music search engine can be made possible.
Appears in Collections:Ph.D Theses (Open)

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
ZhangBJ.pdf1.33 MBAdobe PDF



Page view(s)

checked on Apr 25, 2019


checked on Apr 25, 2019

Google ScholarTM


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.