Multisensory musical entertainment systems

Abstract

Bubble Bumble and Magic Music Desk (MMD) are two unique musical entertainment systems that provide users with multisensory experiences in creating and playing back music in a tangible 3D and physical environment. In the Bubble Bumble demonstration, users collaborate to create music in a chaotic, nonlinear manner by physically capturing and bursting augmented virtual bubbles, which contain short musical segments of instrumental performance floating in the air, and dropping them down onto a virtual playback timeline, which resembles the traditional music staff. In MMD, users manipulate the virtual visual metaphor of musical resources in the physical environment by natural, intuitive interactions of hands and speech commands. Users' feedback reveals that both systems provide feelings of participation, providing multiperceptional experiences of entertaining realism, interaction, and immersion; and that the systems encourage cooperation and prompt multicultural interaction between users speaking different languages.