VIM: Vision for interactive music

Abstract

Traditionally, people were either producers of entertainment media, or else consumers of them. Today's digital entertainment, however, provides for a new dimension: that of interactivity. Instead of passive enjoyment, consumers can now control some elements of the media that were previously solely determined by the producer. This interactivity appears to enhance enjoyment. In this paper, we present a vision-based, interactive music playback system which allows anyone, even untrained musicians, to conduct music. The goal is to allow the user to dynamically influence how music is played back, much like what a real conductor would do. The tempo and volume of the music playback are controlled by the user's movements. In addition, our system projects colorful patterns that respond to the user, making the interaction truly multimedia. © 2007 IEEE.