Multi-criteria-based active learning for named entity recognition

Abstract

In this thesis, we propose a multi-criteria-based active learning approach and effectively apply it to the named entity recognition task. Active learning targets to minimize the human annotation efforts to learn a model with the same performance level as supervised learning by selecting the most useful examples for labeling. To maximize the contribution of the selected examples, we consider the multiple criteria including informativeness, representativeness and diversity and propose some measurements to quantify them respectively in the SVM-based named entity recognition. More comprehensively, we effectively incorporate all the criteria using two active learning strategies, both of which result in less labeling cost than the single-criterion-based method. The best results show that the labeling cost can be reduced by 95% in the newswire domain and 86% in the biomedical domain without degrading the performance of the named entity recognizer. To our best knowledge, this is not only the first work to incorporate the multiple criteria in active learning but also the first work to study active learning for named entity recognition. Furthermore, since the above measurements and active learning strategies are quite general, they can also be easily adapted to other natural language processing tasks.