Human Identification by Gait Using Time Delay Neural Networks

This paper proposed human identification method by gait. Human gait is a type of biometric features and related to the physiological and behavioral features of a human. In this paper, a feature vector of gait motion parameters is extracted from each frame using image segmentation methods, and categorized into different categories. Two of these categories were used to form the gait motion trajectories; Category one: Gait angle velocity: angle velocity hip, angle velocity knee, angle velocity thigh and angle velocity shank. Category two: Gait angle acceleration: angle acceleration hip, angle acceleration knee, angle acceleration thigh and angle acceleration shank for each image sequence. Finally, the TDNN method with different training algorithms is used for recognition purpose. This experiment is done on our own database. This research developed a method which achieves a higher recognition rate in the training set 100% and in the testing set 83%. Also, category one establishes gait motion features to be used in human gait identification applications using different training algorithms, While category two achieved a higher recognition rate by trainrb algorithm.


Introduction
Biometric systems for human identification on distance are increasing demand on it's various applications.Human identification becomes a more important task.Identification through biometric is the best recognition process because it connected with persons not with personal data through different machines.There are several biometric techniques which can be used for human identification like fingerprints, face recognition, iris scans, hand scans, and speech patterns which have been widely used for authentication (Sanjeev Sharma, Ritu Tiwari, Anupam Shukla, & Vikas Singh, 2011;Jinyan Chen & Jiansheng Liu, 2014).However, these biometrical features have the many disadvantages such as, these features cannot be taken from a relatively long distance.In addition, user's cooperation is required to get superior results by those biometrical features.
As a new biometric method, gait based human identification overcomes the above-mentioned drawback to achieving goals and recognize persons on distance according to their walk way.It has a number of advantages as compared to the other biometric characteristics.Such as, being unremarkable.In comparison with other biometrics, which require careful and close contact with the sensor, this unremarkable nature makes it suitable for surveillance and security applications (Sanjeev Sharma et al., 2011;Jinyan Chen & Jiansheng Liu, 2014).In addition to the advantages of this, gait is can not be hidden.In many personal identification scenarios, the face may be hidden and the hand is obscured.These properties of advantages make gait recognition an attractive technology (Tee Connie, Michael Kahn Ong Goh, & Andrew Beng Jin Teoh, 2011).Also, this method is noninvasive and effective from a distance.Thus, the gait appears as a solution because the gait is visible even from a great distance (Sanjeev Sharma et al., 2011;Jinyan Chen & Jiansheng Liu, 2014;Tee Connie et al., 2011).
Gait-based human identification was firstly studied in the medical area.Doctors examined the human gait to determine out whether patients had abnormal gait problems (Jinyan Chen & Jiansheng Liu, 2014).Later researchers found that just like a fingerprint and iris, almost everyone has his distinctive walking style (Sanjeev Sharma et al., 2011;Jinyan Chen & Jiansheng Liu, 2014;Jang-Hee Yoo & Doosung Hwang, 2008).Thus, it was believed that the gait could also be utilized as a vital feature to distinguish the person (Jinyan Chen & Jiansheng Liu, 2014;Jang-Hee Yoo & Doosung Hwang, 2008;Tee Connie et al., 2011).This is an interesting property which recognition a persons, especially for identifying human on distance.Thus, the gait appears as a unique biometric feature which allow possible tracking of people's identities (Sanjeev Sharma et al., 2011;Jinyan Chen &Jiansheng Liu, 2014;Tee Connie et al., 2011;Jang-Hee Yoo & Doosung Hwang, 2008;Navneet Kaur & Samandeep Singh, 2014).
The model-based technique explicitly as human body models based on body motion parts.Matching Model is commonly executed along each figure to assess dynamic features.In (Sharma et al, 2011) the authors introduced a novel method is the gait recognition which based on silhouette is detected.Those features are extracted using image processing.At last neural network is used for recognition purpose.(Hee Yoo & Doosung Hwang, 2014) proposed a novel method is the gait motion as rhythmic and periodic motion and a 2D stick figure which extracted from gait silhouette using motion information with topological analysis guided by anatomical knowledge.A sequential set of 2D stick figures is used to present the gait signature that is a primitive data for the feature extraction based on motion parameters.A back-propagation neural network algorithm is applied to recognize humans by their gait patterns.In those experiments; higher gait recognition performances have been achieved.In (Mridul Ghosh & Debotosh Bhattacharjee, 2014) the authors devoplement a method with different features, that effectively characterizes the gait pattern.A statistical approach has been applied to recognize individuals based on the feature vectors, each of size 23 for each video frame.It has been found that the recognition result of their approach is promising in comparison with other most last methods.(Joshi et al, 2014) In their research, binary silhouette is detected from each frame, features from each frame are extracted using image processing operation, BPNN and SVM technique is used for recognition purpose.(Jure Cove & Peter Peer, 2014) described a new skeleton model based gait recognition system, focusing on modeling gait dynamics.Furthermore, they inspect the problem of walking speed diverging.They used in OU-ISIR gait dataset to prove the state of the art performance of the proposed methods.
On the contrary, the appearance-based technique is more simple.These methods generally apply some statistical theories to distinguish the entire motion pattern (Connie et al., 2011).In (Jinyan Chen & Jiansheng Liu, 2014) the authors proposed a new human gait recognition method named as average gait differential image (AGDI).It is created by the collection of the silhouettes difference between adjacent frames.(2DPCA) is used to extract features.Experiments on CASIA dataset show that better identification, verification performance and less memory storage consumption.(Connie et al., 2011) described a new method based on the contour width information of the silhouette.Using this statistical shape information, they capture the compact structural and dynamic features of the walking pattern.As the extracted contour width feature is large in size, Fisher Discriminant Analysis is used to reduce the dimension of the feature set.(G.Venkata Narasimhulu & S. A. K. Jilani, 2012)introduced a simple background generation algorithm to subtract the moving figures accurately and to obtain binary human silhouettes.Secondly, these silhouettes are described with Fourier descriptors and converted into associated one dimension signals.And therefore ICA is applied to get the independent components of the signals.For reducing the computational cost, fast and robust fixed-point algorithm for calculating ICs is adopted and a criterion how to select ICs is put forward.(Arai et al, 2014) produced a method that extracted features of the wavelet coefficients using silhouette images which produce better classification performance reach 97.63%.This work depends on my results in previous research in (Eman Fares Al Mashagba, Feras Fares Al Mashagba, &Mohammad Othman Nassar, 2014), special signature like a hip, knee and ankle regions are extracted of individuals from each video image are used as a feature vector of human being that represents a pattern in a brief way and there are mostly well examine to identify of human gait.Its locations are specified with reference to zero locations.The previous research emphasized that the motion segmentation algorithm is depending on color and geometric information.The three colored regions are detected and marked as separate components.Ideally, they extracted an image with four components: the outlier, the hip, the knee, and the ankle.The Gaussian Mixture Model used as a segmentation procedure to subtract the background from the moving body parts of the acquired image frames, which described in (Al Mashagba et al., 2014).A number of techniques which described in (Al Mashagba et al., 2014) used to extract trajectory-based kinematic features are calculated from gait sequences to obtain from an RGB video camera.Different categories of features are taking as training and testing set.As a recognizing method Time delay, the neural network is used to identify human beings by their gait.
The following sections organized as follows.Section 2, the proposed method to motion region extraction and feature extraction process is explained.Section 3, explains the gait recognition system., in section 4, experimental result is presented, finally, we conclude in section 5.

The Proposed Method
The biometric recognition system includes the preparation tasks.In biometric gait recognition system the database collected in video type, frame extraction produced.The gait features of a person can be recognized easily when extracted from the side view.We concerned about recognition the normal gait; this is because the majority of the people has a normal walk.Gait contains the physiological or behavioral features of a human beings.Main preparation tasks in the human identification system are region extraction, feature extraction, and recognition process.The region of the frame can be extracted using the image segmentation method (Al Mashagba et al., 2014).These processes are described as follows as in my previous study in (Al Mashagba et al., 2014):

Motion Region Extraction
The individual motion region must be detected before getting the gait features.Background subtraction is the relatively simple and new approach to finding a region of the image.
The whole process of region extraction is described in (Al Mashagba et al., 2014).The three colored regions are detected and marked as separate components.Ideally, extracted four regions from each frame: the outlier, the hip, the knee, and ankle.An example of gait detection  (b) Extracted Region.

Feature Extraction
According to my previous study in (Al Mashagba et al., 2014) for a given video sequence, absolute and relative angles are both calculated using Matlab.The x and y coordinates of the hip joint used in order to calculate the thigh angle segment.And also the knee and ankle coordinates were used to calculate the shank angle segment.
For each frame low limb joint and segment, partly extracted kinematic features to form the gait motion trajectories that divide into two categories:-

Category one: Gait angle velocity:
[angle velocity hip, angle velocity knee, angle velocity thigh and angle velocity shank] for each image sequence.

Category two: Gait angle acceleration:
[angle acceleration hip, angle acceleration knee, angle acceleration thigh and angle acceleration shank] for each image sequence.

Gait Recognition
We give a brief description to Time Delay Neural Network (TDNN) which used for human identification:

Neural Networks
There are many types of neural networks with different structures have been designed, but all are described by the transfer functions which used in processing elements (neurons), the way of training given and the architectural design.Neural networks are composed of a single layer or multiple layer neurons.Multilayer perceptron (MLP) is the best model to eliminate the difficult problems of the single-layer perceptron by the adding more hidden layers (Bhavna Sharma & K. Venugopalan, 2014; Masahide Sugiyama, Hidehumi Sazoait, & Alexander H. Waibel).We will use the TDNN in the process of classifying the human gait patterns.The previous research showed that the TDNNs are very successful in learning spatio-temporal patterns and thus we are going to use them in this research."TDNN is greatly used for different types of pattern recognition in various platforms (ex.Turbo C, C++, python, MATLAB etc)" (IRubya Shaharin, Uzzal Kumar Prodhan, & Md.Mijanur Rahman, 2014).MLP and TDNN exactly have the similar architectural characteristics; but has an additional own feature is known as a tapped delay line in its input neuron and hidden neuron (Shaharin et al., 2014).The basic TDNN architecture is shown in Fig. 2. (Shaharin et al., 2014)."The primary feature of TDNNs is the time-delayed inputs to the nodes.Each time delay is connected to the node via its own weight and represents input values in past instances in time.In Fig. 2. We see a TDNN unit with 2 inputs.Not only is the current value of each input fed into the network (In1 (t) and In2 (t)), but the past two and future two values are also fed in (In1 (t-2), In1 (t-1), In1 (t+1), In1 (t+2) etc.).The purpose of this is to force the TDNN to generalize not only on the current input, but also across past and future inputs" (Shaharin et al., 2014).
Figure 2. TDNN Architecture The nature of the networks used in human identification by gait is as follows:

Training Algorithms
The training procedure is same to the static network.The performance of the algorithms on a dynamic network."Firstapply regular forward and backward move to all-time series as if they were separate events.This returns different error derivatives for corresponding time series connection rather than changing the weights on time series separately.However, essentially update each weight on corresponding connections by the same value, namely average of all corresponding Time delayed weight changes" (Bhavna Sharma & K. Venugopalan, 2014; Sugiyama et al.;Shaharin et al., 2014;Frauke Günther & Stefan Fritsch, 2010).

Gradient Descent Algorithms
"These are the most popular training algorithms that implement basic gradient descent algorithm and update weights and biases in the direction of the negative gradient of the performance function" (Bhavna Sharma & K. Venugopalan, 2014;Sugiyama et al.;Shaharin et al., 2014;Frauke Günther & Stefan Fritsch, 2010).
Gradient Descent backpropagation algorithm (traingd) "is a gradient descent local search procedure.It measures the output error, calculates the gradient of the error by adjusting the weights in the descending gradient direction" (Bhavna Sharma & K. Venugopalan, 2014;Sugiyama et al.;Shaharin et al., 2014;Frauke Günther & Stefan Fritsch, 2010).

Gradient Descent with Momentum (Traingdm) Algorithm
"Is the steepest descent with the momentum that allows a network to respond to the local gradient as well as recent trends in the error surface.It acts like a low pass filter that means with the momentum the network ignores small features in the error surface.A network can get stuck into a shallow local minimum, but with the momentum it slides through such local minimum" (Bhavna Sharma & K. Venugopalan, 2014;Sugiyama et al.;Shaharin et al., 2014;Frauke Günther & Stefan Fritsch, 2010).

Resilence Backpropagation (Trainrp)
"Training algorithm eliminates the effects of the magnitudes of the partial derivatives.In this sign of the derivative is used to determine the direction of the weight update and the magnitude of the derivative have no effect on the weight update.The size of the weight change is determined by a separate update value.The update value for each weight and bias is increased by a factor whenever the derivative of the performance function with respect to that weight has the same sign for two successive iterations.The update value is decreased by a factor whenever the derivative with the respect that weight changes sign from the previous iteration.If the derivative is zero, then the update value remains the same.Whenever the weights are oscillating weight change will be reduced"(Bhavna Sharma & K. Venugopalan, 2014;Shaharin et al., 2014).

Conjugate Gradient Algorithms
"The basic gradient descent algorithm adjusts the weights in the negative of the gradient, the direction in which the performance function is decreasing most rapidly.This does not necessarily produce the fastest convergence.In the conjugate gradient algorithms, a search is performed along conjugate directions, which produces generally faster convergence than steepest descent directions.The conjugate gradient algorithms require only a little more storage than the other algorithms.Therefore, these algorithms are good for networks with a large number of weights" (Bhavna Sharma & K. Venugopalan, 2014;Sugiyama et al.;Shaharin et al., 2014).

Scaled Conjugate Gradient (Trainscg)
"Does not require line search at each iteration step like other conjugate training functions.The step size scaling mechanism is used which avoids a time-consuming line search per learning iteration.This mechanism makes the algorithm faster than any other second order algorithms.The trainscg function requires more iteration to converge than the other conjugate gradient algorithms, but the number of computations in each iteration is significantly reduced because no line search is performed" (Bhavna Sharma & K. Venugopalan, 2014;Sugiyama et al.;Shaharin et al., 2014).

Conjugate Gradient Backpropagation with Fletcher-Reeves Updates (Traincgf)
"Is the ratio of the norm squared of the current gradient to the norm squared of the previous gradient.The conjugate gradient algorithms are usually much faster than other algorithms, but the result depends on the problem" (Bhavna Sharma & K. Venugopalan, 2014;Sugiyama et al.;Shaharin et al., 2014).

Conjugate Gradient Backpropagation with Polak-Riebre Updates (Traincgp)
"Is the ratio of the inner product of the previous change in the gradient with the current gradient to the norm squared of the previous gradient.The storage requirements for Polak-Ribiére (four vectors) are slightly larger than for Fletcher-Reeves" (Bhavna Sharma & K. Venugopalan, 2014;Sugiyama et al.;Shaharin et al., 2014).
The determination of an appropriate architecture depends deeply on the experience of the person controlling experiment using the TDNN in the research.
Features vector which used in the recognition gait process encoded and placed in the input nodes of the network.Scaling these input values so that they be located in the same range, this can help in time-consuming and get high performance of training methods.

Experimental Results
In this research, We collect our own samples of video for the subject and angular kinematic features examine to recognition human gait.TDNN with different training function in our experiments and demonstrate the performance result of the human identification system.
Each video in the experiment has a range from 50-180 frames.We collected 10 video samples from each person; MATLAB Toolbox was used to implement the human identification development process.For each frame low limb joint and segment part extracted kinematic features to form the gait motion trajectories that divide into two categories based on human identification purposes as follows:-

Category one: Gait angle velocity:
[angle velocity hip, angle velocity knee, angle velocity thigh and angle velocity shank] for each image sequence.

Category two: Gait angle acceleration:
[angle acceleration hip, angle acceleration knee, angle acceleration thigh and angle acceleration shank] for each image sequence.
We eliminate all frames which not view all joint from samples.For each image in sequences range from 30 to 75 frames.The feature vector of all frame forms features category.Features vector category entry to TDNN to recognize persons from their gait.Take 70% of samples for training and 30% of samples for testing of all persons.
During this research, a Hyperbolic tangent sigmoid transfer function 'tansig' function was used in the hidden layer and defined in equation 1 taken from (Frauke Günther & Stefan Fritsch, 2010;G. Venkata Narasimhulu & S. A. K. Jilani, 2012) as followes: Linear transfer function pure linear is used in the output layer.To evaluate each algorithm for the time delay neural network, we create 4 layers TDNN with 4 input neurons and 20 hidden neurons with the delay 0 to 3 for the first two layers and 0 to 5 for another layer.The Results Based on Training Algorithm and Time Consuming.
According to the table, we can recognize human by their gait using different training algorithms which can be sorted depends on the time consuming and features category which has the highest recognition rate in training and testing set which given in trainrb and traincgf.However, the recognition rate in the training set in traincgf, trainrb and trainbfg is equivalent.This implies that we can know each subject using its gait features from each others with promising recognition rate.
Based on features category, the method was developed to achieve a higher recognition rate in the training set 100%, and in the testing set 83%.Also, category one establishes gait motion features to be used in human gait identification applications using different training algorithms.Furthermore, category two achieved a higher recognition rate by trainrb algorithm.

Conclusion
This paper described an automated gait recognition system using TDNN according to the most suitable training algorithms to recognize the human gait, five categories with 4 features in each are considered as gait features.By measuring each category separately, only two important categories for classifying the gait were selected:.Gait angle velocity, and Gait angle acceleration.Then, the TDNN algorithm is applied to different training algorithms in each category.The best training algorithm with higher recognition rates shown in the above table in both categories.
Based on features category, the method was developed to achieve a higher recognition rate in the training set 100% and in the testing set 83%.Also, category one establishes gait motion features to be used in human gait identification applications using different training algorithms.Furthermore, category two achieved a higher recognition rate by trainrb algorithm.

Table 1 .
Performance results using different training Algorithm with 200 epochs of two categories is shown in table 1. Training Rate and Testing Rate For The Two Categories.