Target Tracking Based on Mean Shift and KALMAN Filter with Kernel Histogram Filtering

Visual object tracking is required in many tasks such as video compression, surveillance, automated video analysis, etc. mean shift algorithm is one of popular methods to this task and has some advantages comparing to other tracking methods. This method would not be appropriate in the case of large target appearance changes and occlusion; therefore target model update could actually improve this method. KALMAN filter is a suitable approach to handle model update. We performed mean shift algorithm with model update ability for tracking in this paper and achieve good results


Introduction
Visual object tracking is required in many tasks such as video compression, surveillance, automated video analysis, etc. Object tracking should deal with some challenges like occlusion, target appearance changes, cluttering.Mean shift algorithm is an efficient tracking method which has been employed in the literature (Wang and Yagi Wang,2009;Comaniciu and Meer, 2003;Collins, 2004;Xin and Song, 2006;Sen, Wei, and Sen, 2009).This algorithm needs less computational complexity compared with methods performed in the whole image.In the traditional mean shift algorithm, the target model is unchanged during the tracking and belongs to the first frame, so this algorithm is not robust in the case of large changes of appearance, size and direction of the target.In order to compensate the target size changes, Comaniciu (Comaniciu and Meer, 2003) changes the Kernel size about 10% and selected the kernel with maximum similarity value and (Collins, 2003) used scale space.Peng et al. (Peng N.,Yang J.& Liu Z. ,2005) integrated adaptive KALMAN filter into mean shift algorithm to update the target model to cope with target appearance changes.Target model update in each frame makes the tracker sensitive to occlusion and noise.So model update should be performed with some cautions.Nguyen (Nguyen, Worring and van den Boomagaard, 2001) used filter residual to test model update and (Peng, Yang and Liu, 2005) used filter residual to test the update model hypothesis in the form of t distribution.This paper is organized as follows.The mean shift algorithm is reviewed in section.KALMAN filter is considered in section.Model update is coming in section.Results are in section.

Mean shift algorithm
Mean shift algorithm is a nonparametric mode seeking method introduced by Fukanaga (Fukunaga and Hostetler, 1975) and considered by Cheng (Cheng, 1995).Comaniciu (Comaniciu and Meer, 2003) incorporated mean shift algorithm into tracking.This method optimizes the similarity measure between target model and candidate model in the consecutive frames.
To characterize the target, first a feature space should be defined.This feature space could be selected as the color space of the target.Among the color spaces, RGB has widely been used.

Object representation
The object model is represented by its probability density function (pdf) in the feature space.In order to decrease computational cost imposed by real-time processing, m-bin histograms are so desirable.The object model is called target model and candidate model in reference frame and subsequent frames respectively.is the kernel which Gaussian and Epanechnikov ones are more popular and defined as follows: Epanechnikov kernel: (2) Gaussian kernel: (3) Candidate model is defined as: (4) As it could be seen, the only difference between the target and candidate models is h, the scale of kernel.
Mean shift algorithm optimizes the similarity between target and candidate models.Bhattacharyya coefficient is a common similarity measurement and is defined as: (5) This coefficient could be considered as the cosine of the angle between the unit vectors and . Bhattacharyya coefficient optimization leads to mean shit algorithm as the following equation (see Comaniciu D., 2003 for more details).( 6) Where d is the displacement vector of kernel center, and the weight is defined as: (7) If the Epanechnikov kernel is used, mean shift equation is simplified to: (8)

KALMAN filter
KALMAN filter was first introduced by KALMAN (Kalman, 1960) to estimate the state of a system.This filter has been used to estimate the position of target in many literature (Comaniciu., 2003;Kalman, 1960;Bai, 2010;Zhu, 2002;Zhao Qiao and Men, 2009).KALMAN filter consists of two stages called prediction step and correction step.The state of system is predicted using state equation in the prediction step and is corrected using measurement equation in the correction step.
The State and measurement equations of a linear KALMAN filter are respectively as follows: (9) Where and are the state vector of current and previous time respectively.and are the system and measurement noise.Performance of KALMAN filter depends on two key parts: the reasonable linear model and the initial parameters including the noise statistical properties and the initial state variance (Peng ,Yang and Liu , 2005).Adaptive KALMAN filter is used in (Peng.,Yang, and Liu, 2005)  KALMAN filter residual is the difference between predicted state and measurement as: (13) The variance of filtering residuals over last L frame is calculated as: Then measurement noise variance is assumed constant but variance of state noise is estimated as: (15) Which is not constant, therefore the used KALMAN filter is adaptive.The initial value for variances of state noise, measurement noise and prediction error are calculated (Peng ,Yang and Liu, 2005) Where N is the number of samples and s is the unbiased estimation of standard deviation.
If the null hypothesis is accepted, model update is performed; else the hypothesis would be accepted and previous target model would be held.The following conditional probability assigns the critical region. (20) The critical region is defined as: (21) Where is set to and the critical value is determined from t distribution table.We have made some modifications to mean shift-KALMAN filter method of (Peng ,Yang and Liu, 2005).If the large appearance changes of target are not due to the occlusion, the target model should be updated.We supposed at most 6 frames for occlusion duration, and if the number of model update hypothesis rejections ( ) exceeds this number, model update procedure is performed.

Results
We have performed mean shift algorithm with model update capability to two image sequence here.In the first experiment the head of a tennis playerhas been tracked (Figure .2). Since there is no occlusion in this sequence model update procedure could be omitted.To construct the target and candidate models 16 quantization levels is considered; therefore the histogram size is be 16 16 16.We have used mean shift algorithm with variable kernel size in the first experiment.As could be seen, the result is satisfying.Bhattacharyya coefficient is demonstrated in Fig. 3 for more considerations.
In the next experiment we want to investigate the operation of model update in the mean shift algorithm.Model update comes in useful especially in target occlusion cases.To prove this claim, tracking a person who severely occluded by another person is demonstrated in the third and forth figures.
Figure 4 shows the tracking result of a person using mean shift algorithm without target model update.As it could be seen, the tracking is completely challenging because the target appearance severely changes and occlusion has occurred in some frames.Therefore as it is expected, target is totally lost in the middle frames of Fig. 4. In the next experiment, mean shift algorithm with model update capability is performed to track the person and the result is shown in Fig. 5.As it could be seen, tracking is satisfying and the man has been fully tracked even in the case of occlusion.The tracked path of person is shown in Fig. 6.
Only the histogram bins with none zero values are participated in the model update process and the last three frames is used for hypothesis testing i.e.L=3.We used significance level equal to 0.1 and is assumed equal to 1.3 in the hypothesis testing stage because 21 nonzero value of target model is used (t distribution table).
Bhattacharyya coefficient of tracking of the figures 4 and 5 is demonstrated in Fig. 7. Histogram of KALMAN filter residuals of a frame is brought in Fig. 8.As it is representing, the normal assumption which was made in the last section is met.Fig. 9 shows the t value of each frame and .As it is expected, the t value exceeds in some frames in which large appearance changes and occlusion occurrs and model update is not performed in these frames.To consider the KALMAN filter operation, variance of state noise and averaged variance of prediction error are shown in Fig. 10.Variance of state noise suddenly raises in the frames with large appearance change and occlusion to cope the KALMAN filter with them.
Fig 1 illustrates the histogram construction of a 3 3 gray scale image.Target model is defined as: (1) Where is the normalized coordinates of pixels, is the dirac function.C is normalization constant to keep condition.
to update the target model.State and error variance prediction equations are as follows: (10) Where is the target model and is the variance of prediction error.and i are the histogram bins and frame index respectively.KALMAN gain is calculated as: (11) And finally state and error variance correction equations are: (12) It is important to tune a KALMAN filter with appropriate initial value of prediction error variance and state and measurement noises.used KALMAN residual to do this.

Figure 4 .Figure 7 .Figure 10
Figure 4. Tracking of a person using mean shift without model update in frames 1,14,26,40,61,96,115,124 (Nguyen ,Worring and van den Boomagaard, 2001) model update and some considerations should be taken to avoid that.Nguyen(Nguyen ,Worring and van den Boomagaard, 2001)used filter residual to avoid over update model.
(Peng ,Yang and Liu, 2005)used t distribution of residual population to test the hypothesis of model update.Let suppose filter residuals be the samples from totality T with normal distribution , since the is unknown t distribution is used as follows: (19