APPLYING RELEVANCE FEEDBACK TO A PHOTO ARCHIVAL SYSTEM

Abstract

In recent years, the proliferation of picture data and concomitant advances in the image-handling capabilities of computers have motivated research into image retrieval systems. These trends have exerted increasing pressure to develop software information systems which can provide effective management of image archives: applications which involve a large number of images and/or a wide subject domain. Efficient retrieval of pictorial data from such collections is of particular concern. Given the current state of image analysis and understanding, picture archival systems being developed today generally consist of hybrid systems combining image and text, where picture access is provided through the accompanying alphanumeric data. We can distinguish several different approaches to developing picture archival systems. Commercial PC file systems attempt to extend image management systems by adding simple text keywords to the image records [Raskin87]. Most other systems take advantage of available expertise in management of alphanumeric data, i.e. database management systems (DBMS) or information retrieval (IR) systems, and extend these systems to include image data [Shet90, Assm86, Bender88]. Additionally, there have been some attempts to classify and extract the relevant feature information from the images using image processing techniques and incorporate this data into a DBMS [Kunii74, Chang89). Finally, research efforts into object-oriented DBMS have focused on providing a comprehensive and consistent approach to multimedia data, including picture data [Klas90, Chris86J. The Photo Archival System (PAS) was developed at Institute of Systems Science (ISS) at the National University of Singapore (NUS) to provide a general purpose solution to the problem of image archiving, especially for applications whose domain or unpredictable access patterns make retrieval based solely on image analysis or fixed attributes infeasible [AlH91b]. PAS provides free-text access .to images based on probabilistic best-match search of short image descriptions, ~ which ranks returned images in order of similarity to the query. This master's project addresses the problems associated with text-based retrieval from image archival systems, using PAS as a testbed. Effective system performance is critically dependent on the quality of the image descriptions and the match between the user and indexer vocabulary. In practice, the quality and correspondence of these descriptions are highly variable, given the inherent time costs and linguistic ambiguities associated with annotating pictures with text. To solve these problems, we can utilize previous work from free-text document retrieval systems (DRS) which use relevance feedback from the initial query to refine the query and/or document representations [Salton90, Salton75]. These techniques were optimized for PAS to exploit the unique semantic characteristics of image data and account for the special problems associated with text annotation of pictures. Visually oriented relevance feedback and query modification was implemented using direct manipulation of icons. An algorithm designed for dynamic image description modification based on relevance feedback was proposed, implemented, and experimentally tested. Experimental results showed significant performance improvements with the modified database overall, although slight performance degradations for queries with disjoint answer sets indicate that further study is warranted to monitor long-term effects.