Tracking High Speed Skater by Using Multiple Model

Color-based tracking is a very popular technique for object tracking. However, under varying environmental conditions in video sequences, the existing tracking methods tend to be unreliable for tracking high speed objects. It is possible under such a complex environment that the targets may disappear totally or partially due to occlusion. To over this problem, a novel approach of multiple-color model is proposed in this paper. Different colors extracted from the skater’s head and body are tracked with Kalman filter, respectively. A confidence measure representing the reliability of each color tracking result is presented to guide possible re-detection for continuous tracking. Experimental results show that the proposed approach is very efficient and effective in tracking high-speed skater in real short track sequences.


Introduction
The most common method to track objects is to detect them using background subtraction of an established correspondence frame by frame (Wren et al., 1997;Lou et al., 2002).Despite its popularity, it can only be applied to imagery acquired by "stationary camera".An alternative approach to background subtraction is using simple geometric models to transform the object from frame to frame.Comaniciu et al. (Comaniciu et al., 2003) use the mean-shift approach to compute the translation of a circular region.The object appearance is modeled by weighed histograms.
Tracking of the complete object can be achieved by employing the active contours, which were introduced by Kass et al (Kass et al., 1988).In (Mansouri, 2002), Nonrigid object motion can be modeled in terms of combining optical flow with active contours, Mansouri uses a probabilistic form of the brightness constraints, where the color is defined by a Gaussian distribution.Optical flow derived from the brightness constraint is computed.However, the brightness constraint requires very small variation in the intensities and it is not suitable for images with high dynamic intensity ranges.
Tracking with multiple cue fusion becomes more and more popular.In (Ozyildiz et al., 2002), an autobinomial Gibbs Markov random field is used for modeling the texture and a 2D Gaussian distribution is used for modeling the color.This allows a probabilistic fusion of the texture and color cues.Similarly, in (Yilmaz et al., 2004), visual features (color, texture) and the object shape are considered.Color and texture models are generated for the object and the background regions.A shape prior is used for recovering occluded object parts during the occlusion.
There are many computer vision systems for many sports domains.Pingali et al. (Pingali et al, 1998) develop a real time tracking technology for enhancing the broadcast of tennis matches from stationary cameras.Recently, Yan et al. (Yan et al, 2006) propose a data association algorithm to track a tennis ball in low-quantity tennis video sequences.Pers et al. use two stationary cameras mounted directly above the court and propose a new approach for modeling the radial image distortion more accurately.The tracking algorithm combined with color feature and the template is exploited to track the player, using color feature is to avoid a drift caused by the template tracking.Their systems are applied to many sports domains including handball (Pers et al, 2002) and basketball (Jug et al, 2003).But there are two limitations in their works: first, the cameras must be placed above the playing court, which is a rigorous condition for regular league or championship matches.Second, how to handle occlusion in tracking process seems not to be solved.
The goal of our study is to track high-speed nonrigid skater under a complex environment, where background drastically changes and successive occlusion often occurs and the colors of similar clothing are very close, etc.Many above methods using ideal model or some constraint are not suitable.
In this paper, a novel approach for tracking high-speed skater is proposed.Color features are firstly extracted from skater's head and body.Then, Kalman filter is utilized to estimate the search region and to evaluate each color, respectively.Afterwards, these evaluations are integrated to determine which part of the object needs to re-detect.The detection result is regarded as the initial data of tracking module.To enhance the robustness of tracking, a confidence measure representing the reliability of each color tracking result is presented to guide possible re-detection for continuous tracking.Moreover, the lost tolerance of object is introduced in order to avoid stopping tracking exceptionally due to totally occlusion.The approach is robust and can perform real-time skater detection and tracking.

Proposed Approach
Our computer vision system aims to automatically track the movements of skaters on a large-scale complex and dynamic rink.Our goal is to exploit it not only in daily training but also in competitions.We used a single panning camera, which was mounted at the top auditorium of the stadium as close as possible to the center in order to reduce the projection error.Due to little texture information on the rink, unlike (Intille & Bobick, 1994;Okuma, 2003), zooming was abandoned, it can make recording the high-speed target more difficult and enlarge the error of lens distortion.Though the camera center moves by a small amount, due to an offset from the camera's optical center, the approximation of pure rotation is indeed sufficient such as proven in (Hayman & Murray, 2003).

The Framework of Our Algorithm
The flow chart of the proposed algorithm shown in Figure 1 consists of three major processing modules: 1) Object detection based on colors.
3) Object re-detection determined by confidence measure.

Color Space Selection
An important aspect of any color-based tracking system is to choose a color space that is relatively invariant to minor illuminant change.The two most popular color spaces robust to minor illuminant changes are HSV and normalized RGB.(Terrillon & Akamatsu, 2000) and (Bradski, 1988) proved that normalized RGB color model is more sensitive to lighting changes since the saturation influenced by lighting is not separated out of that model.Hence the best color space is HSV which has been applied to many color tracking problems (Bradski, 1988;Sigal et al., 2004).HSV space separates out hue and saturation from intensity, and we create color models by 2D histograms from the hue and saturation channel in HSV space.

Kalman filter prediction
The Kalman filter is one of the most popular estimation techniques in motion prediction because it provides an optimal method for linear dynamic systems with white Gaussian noise.In general, the Kalman filter describes a system with a system state model and a measurement model as in Eqs. ( 1) and ( 2): (1) (2) The system state at the time is linearly associated with the state at the time , and there is also a linear relationship between the measurement and the system state .The random variables and represent the state and measurement noise, respectively.A is the state transition matrix that relates the state at frame k to the state at frame k+1, and H is called the observation matrix that relates the state to the measurement.In our case, the system state S consists of the following components: (3) The state transition matrix A, the observation matrix H and the measurement M are given by:

Object Confidence Measure
It is frequently the case in our tracking problem that an actual measurement on a target is not reliable due to occlusion and complex environment.When an actual measurement is heavily corrupted, the measurement error accumulates through the feedback cycle and the reliable estimation can not be obtained, which results in tracking failure.Therefore, we should get an optimal detection result as possible as we can.A confidence measure (CM) representing the reliability of each color tracking result is given by the following formula: where and represent the detected area of head and body at time , respectively.and represent the velocity of head and body at time , respectively.and are weighted factors, is a prior threshold of Consequently, the unreliable part determined by CM will be re-detected by the reliable part according to the correlation of initial model.
In Figure 2, the competitor skating through the curve is exactly tracked despite successive occlusion and size variation.
Figure 3 shows good tracking result under a complex environment, where the color of advertisement baffle-board is similar with that of skater's clothing (body).
The experiment results also demonstrate the robustness of our method.Once one of multiple color features is tracked exactly (or not corrupt) even none of them is tracked within limit frames, our tracking system can still work continuously.The unreliable color feature determined by confidence measure (CM) is re-detected according to the correlation of the initial model, which improves the tracking accuracy in every frame to avoid accumulated error.
The trajectory of a skater in an individual race of 500 meters is illustrated in Fig. 4, different lines denote the trajectory of the skater when he/she skates along different loop.More information and statistic of the competitions are available such as the trajectory and velocity, which can be further processed and analyzed by the sports experts.
Our approach saves more computational time than that in (Ozyildiz et al., 2002).The program has been implemented by using Visual C++ on a PIV 3.0GHz machine and it takes only 28ms to process averagely on a true color image with 720 576 resolution and can achieve real-time processing.

Conclusions
A novel approach for tracking high-speed skater is proposed.Color features are firstly extracted from skater's head and body.The combination of them is efficient and effective for tracking skater under a complex environment or successive occlusion.A confidence measure representing the reliability of each color tracking result is presented to guide possible re-detection for continuous tracking.Moreover, the lost tolerance of object is introduced in order to avoid stopping tracking exceptionally due to totally occlusion.The approach is robust and can achieve real-time skater detection and tracking.
It will fail under some special conditions such as all color features lose successively or the light is too dark so as to the color feature in HSV space is unstable and not exactly detected due to non-removable singularity (Cheng et al., 2001).It will be investigated in the future.
Figure 1.Flow chart of the proposed algorithm