Visual analysis of fingering for pedagogical violin transcription

Abstract

Automatic music transcription, in spite of decades of research, remains a challenging research problem. The traditional audio-only approach has yet to achieve a satisfactory performance for any computer-aided pedagogical system. Inspired by the high correlation between violin playing techniques (fingering, bowing) and the played acoustic notes, this paper presents a first attempt in visual analysis of violin fingering to compensate for the difficulties in audio-only music transcription. This is achieved by a robust multiple finger tracking algorithm and a string detection method that extract press, release, and fingertip position from the fingering video and automatically translate the fingering information into the played acoustic note, i.e., onset, offset, and pitches. Experimental results reveal high correctness in multiple finger tracking and string detection, thus paving the way for an improved audio-visual violin transcription system. Copyright 2007 ACM.