SELF-SUPERVISED MODELING FOR OPEN-DOMAIN DIALOGUE EVALUATION

Abstract

Mainstream open-domain dialogue (ODD) systems, designed for natural conversations on a wide array of topics, employ deep generative models. Evaluating these models' output, however, is an extremely complex task. The current gold standard is human evaluation, but its high cost, time-intensiveness, lack of scalability, and irreproducibility often necessitate the use of automatic evaluation as an alternative. In this thesis, we tackle three important challenges in the field of automatic ODD evaluation: (1) multi-dimensional evaluation, (2) understanding and modeling multi-turn interaction, and (3) domain generalization, which collectively contribute to the ultimate goal of creating reliable, adaptable, and holistic automatic dialogue evaluation metrics. Our proposed automatic metrics represent a significant step forward in our ability to evaluate dialogue systems and guide their development with the establishment of new state-of-the-art correlations with human evaluation at both turn and dialogue levels.