Framework for synthesizing semantic-level indices

Abstract

Extraction of the syntactic features is a well-defined problem thereby lending them to be exclusively employed in most of the content-based retrieval systems. However, semantic-level indices are more appealing to user as they are closer to the user's personal space. Most of the work done at semantic level is confined to a limited domain as the features developed and employed therein apply satisfactorily only to that particular domain. Scaling up such systems would inevitably result in large numbers of features. Currently, there exists a lacuna in the availability of a framework that can effectively integrate these features and furnish semantic level indices. The objective of this paper is to highlight some of the issues in the design of such a framework and to report on the status of its development. In our framework, construction of a high-level index is achieved through the synthesis of its large set of elemental features. From the large collection of these features, an image/video class is characterized by selecting automatically only a few principal features. By properly mapping the constrained multi-dimensional feature space constituted by these principal features, with the semantics of the data, it is feasible to construct high level indices. The problem remains, however, to automatically identify the principal or meaningful subset of features. This is done through the medium of Bayesian Network that discerns the data into cliques by training with pre-classified data. The Bayesian Network associates each clique of data points in the multi-dimensional feature space to one of the classes during training that can later be used for evaluating the most probable class to which that partition of feature space belongs. This framework neither requires normalization of different features or the aid of an expert knowledge base. The framework enables a stronger coupling between the feature extraction and meaningful high-level indices and yet the coupling is sufficiently domain independent, as shown by the experiments. The experiments were conducted over real video consisting of seven diverse classes and the results show its superiority over some of the standard classification tools.