Attribute-augmented semantic hierarchy: Towards bridging semantic gap and intention gap in image retrieval

Abstract

This paper presents a novel Attribute-augmented Seman- Tic Hierarchy (A2SH) and demonstrates its effectiveness in bridging both the semantic and intention gaps in Content- based Image Retrieval (CBIR). A 2SH organizes the seman- Tic concepts into multiple semantic levels and augments each concept with a set of related attributes, which describe the multiple facets of the concept and act as the intermedi- Ate bridge connecting the concept and low-level visual con- Tent. A hierarchical semantic similarity function is learnt to characterize the semantic similarities among images for re- Trieval. To better capture user search intent, a hybrid feed- back mechanism is developed, which collects hybrid feed- backs on attributes and images. These feedbacks are then used to refine the search results based on A2SH. We de- velop a content-based image retrieval system based on the proposed A 2SH. We conduct extensive experiments on a large-scale data set of over one million Web images. Exper- imental results show that the proposed A2SH can charac- Terize the semantic affinities among images accurately and can shape user search intent precisely and quickly, leading to more accurate search results as compared to state-of-the-art CBIR solutions. Copyright © 2013 ACM.