RELEVANCE FEEDBACK TECHNIQUES FOR CONTENT-BASED IMAGE RETRIEVAL

Abstract

When the user has to specify the individual attributes of the features to be used for retrieval, the query formulation process may not be transparent to him. In particular, without detailed knowledge of the collection make-up and retrieval environment, most users find it difficult to formulate queries that are well designed for retrieval purposes. Another problem is that if the query representation used is low level, the initial retrieval may not be effective. There are various ways in which retrieval effectiveness may be improved. One effective approach is to employ relevance feedback (RF) which has been found to be extremely effective in information retrieval (IR) systems. ln a typical RF process, the user's relevance judgements of retrieved images are used to refine the query in order to improve subsequent retrieval effectiveness. The aim of RF is to make use of the user's relevance judgements to better classify images into the relevant and irrelevant sets. Hence, it can be viewed as a classification problem and thus, techniques developed in the field of machine learning may be employed for RF. A RF approach for image retrieval that makes use of techniques developed in the fields of IR and machine learning has been developed in this research. Based on the set of relevant images identified by the user, the approach first extracts a significant set of features that characterises the relevant image set. It then uses this set of features to modify the original query to make it more similar to the relevant images. A decision tree is also built to model knowledge from the retrieved and relevant image sets to better predict the relevance of new images. The modified query and decision tree are used jointly to retrieve a new set of relevant images in subsequent retrievals. The proposed RF approach has been applied to the images' text descriptions and colour histograms. In an integrated content-based image retrieval system, different features tend to have different degrees of importance for different queries. A multiple features RF technique has also been developed that makes use of user's relevance judgements to estimate the importance of the different features used. In many image retrieval systems, the user is forced to use only one representative sample image as the query image even when he knows of several sample images. This is not very satisfactory because information regarding the relevant image set found in the other sample images is ignored during retrieval. A technique has been developed which induces a representative query from a set of sample images and the induced query is used to perform retrieval instead. This query induction technique has also been applied to the images' text descriptions and colour histograms. A system with a Web-based graphical user interface has been implemented to demonstrate and test the various techniques developed with an image collection of 12,019 images. Test results obtained for all the techniques are very encouraging, showing the effectiveness of the proposed techniques.