Interactive theater experience with 3D live captured actors and spatial sound

Abstract

This chapter introduces a new reality interactive theater that combines the 3D Live human capturing system, spatial sound, augmented reality, human-oriented interaction, and ambient intelligence technologies. Actors/Dancers at different places remotely are captured by the 3D Live system and transmitted to the theater place in real time and rendered in 3D form and real size, thus they can play/dance with the local actors/dancers and virtual characters at the same place in real time. Audience interaction is allowed and handled by an Ambient Intelligence (AI) agent to generate new actions for virtual characters and send meaningful interactions to real actors. The spatial sound system provides the real 3D sound and at the same time generates special sound effects such as controlling the sound direction that only allows people in a certain direction to heard it. All these features combined together bring a totally new theater experience to both actors/dancers and audiences, and extend the grammar of the traditional theater. © 2010 Springer-Verlag London Limited.