From text question-answering to multimedia QA on web-scale media resources

Abstract

With the proliferation of text and multimedia information, users are now able to find answers to almost any questions on the Web. Meanwhile, they are also bewildered by the huge amount of information routinely presented to them. Question-answering (QA) is a natural direction to address this information over-loading problem. The aim of QA is to return precise answers to users' questions. Text-based QA research has been carried out for the past 15 years with good success especially for answering fact-based questions. The aim of this paper is to extend the text-based QA research to multimedia QA to tackle a range of factoid, definition and "how-to" QA in a common framework. The system will be designed to find multimedia answers from Web-scale media resources such as Flicker and YouTube. This paper describes the architecture and our recent research on various types of multimedia QA for a range of applications. The paper also discusses directions for future research. Copyright 2009 ACM.