Please use this identifier to cite or link to this item:
Title: Harvesting visual concepts for image search with complex queries
Authors: Nie, L.
Yan, S. 
Wang, M.
Hong, R.
Chua, T.-S. 
Keywords: complex query
image search
news visualization
photo-based qa
Issue Date: 2012
Citation: Nie, L.,Yan, S.,Wang, M.,Hong, R.,Chua, T.-S. (2012). Harvesting visual concepts for image search with complex queries. MM 2012 - Proceedings of the 20th ACM International Conference on Multimedia : 59-68. ScholarBank@NUS Repository.
Abstract: The use of image reranking to boost retrieval performance has been found to be successful for simple queries. It is, however, less effective for complex queries due to the widened semantic gap. This paper presents a scheme to enhance web image reranking for complex queries by fully exploring the information from simple visual concepts. Given a complex query, our scheme first detects the noun-phrase based visual concepts and crawls their top ranked images from popular image search engines. Next, it constructs a heterogeneous probabilistic network to model the relatedness between the complex query and each of its crawled images. The network seamlessly integrates three layers of relationships, i.e., the semantic-level, cross-modality level as well as visual-level. These mutually reinforced layers are established among the complex query and its involved visual concepts, by harnessing the contents of images and their associated textual cues. Based on the derived relevance scores, a new ranking list is generated. Extensive evaluations on a real-world dataset demonstrate that our model is able to characterize the complex queries well and achieve promising performance as compared to the state-of-the-art methods. Based on the proposed scheme, we introduce two applications: photo-based question answering and textual news visualization. Comprehensive experiments well validate the proposed scheme. © 2012 ACM.
Source Title: MM 2012 - Proceedings of the 20th ACM International Conference on Multimedia
ISBN: 9781450310895
DOI: 10.1145/2393347.2393363
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.


checked on Feb 13, 2019

Page view(s)

checked on Dec 29, 2018

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.