Automatic camera calibration of broadcast tennis video with applications to 3D virtual content insertion and ball detection and tracking

Abstract

This paper presents an original algorithm to automatically acquire accurate camera calibration from broadcast tennis video (BTV) as well as demonstrates two of its many applications. Accurate camera calibration from BTV is challenging because the frame-data of BTV is often heavily distorted and full of errors, resulting in wildly fluctuating camera parameters. To meet this challenge, we propose a frame grouping technique, which is based on the observation that many frames in BTV possess the same camera viewpoint. Leveraging on this fact, our algorithm groups frames according to the camera viewpoints. We then perform a group-wise data analysis to obtain a more stable estimate of the camera parameters. Recognizing the fact that some of these parameters do vary somewhat even if they have similar camera viewpoint, we further employ a Hough-like search to tune such parameters, minimizing the reprojection disparity. This two-tiered process gains stability in the estimates of the camera parameters, and yet ensures good match between the model and the reprojected camera view via the tuning step. To demonstrate the utility of such stable calibration, we apply the camera matrix acquired to two applications: (a) 3D virtual content insertion; and (b) tennis-ball detection and tracking. The experimental results show that our algorithm is able to acquire accurate camera matrix and the two applications have very good performances. © 2008 Elsevier Inc. All rights reserved.