Semi-supervised annotation of brushwork in paintings domain using serial combinations of multiple experts

Abstract

Many recent studies perform annotation of paintings based on brushwork. They model the brushwork indirectly as part of annotation of high-level artistic concepts such as artist name using low-level texture features and supervised inference methods. In this paper, we develop a framework for explicit annotation of paintings with brushwork classes. Brushwork classes serve as meta-level semantic concepts for artist names, paintings styles and periods of art and facilitate the incorporation of domain-specific ontologies. In particular, we employ the serial multi-expert framework with semi-supervised clustering methods to perform the annotation of brushwork patterns. Serial combination of multiple experts facilitates step-wise refinement of decisions based on the preferences of individual experts. Each individual expert performs focused subtasks using relevant feature set, which decreases the 'curse of dimensionality' and noise in the feature space. Each expert focuses on the annotation of the currently available samples from its unlabeled pool using semi-supervised agglomerative clustering. This approach is more appropriate as compared to the traditional classification methods since each brushwork class includes a variety of patterns and cannot be represented as a single distribution in the feature space. The experts exploit the distribution of unlabelled patterns and further minimize the annotation error. The multi-expert semi-supervised framework out-performs the conventional methods in annotation of patterns with brushwork classes. This framework will further be adopted to facilitate ontology-based annotation with higher-level semantic concepts such as the artist names, painting styles and periods of art. Copyright 2006 ACM.