Frown more, talk more: Effects of facial expressions in establishing conversational rapport with virtual agents

Abstract

How can conversational agents be better designed to build rapport with human beings? Related work on creating rapport through conversational agents has largely focused on nonverbal contingent envelope feedback. There is relatively little known about how forms of emotional feedback play a role in building rapport between agents and humans. This paper describes a study in which people told stories to an agent that provided emotional feedback in the form of facial expressions. Rapport was measured through the length of the stories, the fluency of their speech, and the user's own subjective experience. Surprisingly, results indicated that inappropriate emotional feedback increased story length, which was the opposite of previous studies on envelope feedback that had shorter stories in unresponsive conditions. This paper explains the factors particular to emotional feedback that could cause this difference. © 2012 Springer-Verlag Berlin Heidelberg.