Please use this identifier to cite or link to this item: https://doi.org/10.1145/2388676.2388793
Title: Speak-As-You-Swipe (SAYS): A multimodal interface combining speech and gesture keyboard synchronously for continuous mobile text entry
Authors: Sim, K.C. 
Keywords: Gesture keyboard
Mobile text input
Multimodal interface
Voice input
Issue Date: 2012
Source: Sim, K.C. (2012). Speak-As-You-Swipe (SAYS): A multimodal interface combining speech and gesture keyboard synchronously for continuous mobile text entry. ICMI'12 - Proceedings of the ACM International Conference on Multimodal Interaction : 555-560. ScholarBank@NUS Repository. https://doi.org/10.1145/2388676.2388793
Abstract: Modern mobile devices, such as the smartphones and tablets, are becoming increasingly popular amongst users of all ages. Text entry is one of the most important modes of interaction between human and their mobile devices. Although typing on a touchscreen display using a soft keyboard remains the most common text input method for many users, the process can be frustratingly slow, especially on smartphones with a much smaller screen. Voice input offers an attractive alternative that completely eliminates the need for typing. However, voice input relies on automatic speech recognition technology whose performance degrades significantly in noisy environment or for non-native users. This paper presents Speak-As-You-Swipe (SAYS), a novel multimodal interface that enables efficient continuous text entry on mobile devices. SAYS integrates a gesture keyboard with speech recognition to improve the efficiency and accuracy of text entry. The swipe gesture and voice inputs provide complementary information that can be very effective in disambiguating confusions in word predictions. The word prediction hypotheses from a gesture keyboard are directly incorporated into the speech recognition process so that the SAYS interface can handle continuous input. Experimental results show that for a 20k vocabulary, the proposed SAYS interface can achieve prediction accuracy of 96.4% in clean condition and about 94.0% in noisy environment, compared to 92.2% using a gesture keyboard alone. Copyright 2012 ACM.
Source Title: ICMI'12 - Proceedings of the ACM International Conference on Multimodal Interaction
URI: http://scholarbank.nus.edu.sg/handle/10635/42028
ISBN: 9781450314671
DOI: 10.1145/2388676.2388793
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

3
checked on Dec 13, 2017

Page view(s)

35
checked on Dec 16, 2017

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.