Improved Liu Estimators for the Poisson Regression Model

A new shrinkage estimator for the Poisson model is introduced in this paper. This method is a generalization of the Liu (1993) estimator originally developed for the linear regression model and will be generalized here to be used instead of the classical maximum likelihood (ML) method in the presence of multicollinearity since the mean squared error (MSE) of ML becomes inflated in that situation. Furthermore, this paper derives the optimal value of the shrinkage parameter and based on this value some methods of how the shrinkage parameter should be estimated are suggested. Using Monte Carlo simulation where the MSE and mean absolute error (MAE) are calculated it is shown that when the Liu estimator is applied with these proposed estimators of the shrinkage parameter it always outperforms the ML.


Introduction
In the field of economics, health, social and physical sciences, the dependent variable often comes in the form of a nonnegative integers or counts.In that situation one often apply the Poisson regression model which is usually estimated by maximum likelihood (ML) where the solution to a non-linear equation is found by applying iterative weighted least square (IWLS).This method has been shown in Mansson and Shukur (2011) to be sensitive to multicollinearity and it becomes difficult to make a valid statistical inference since the mean squared error (MSE) becomes inflated.In Mansson and Shukur (2011), a ridge regression estimator (RRE) was presented for logistic regression which was a generalization of that proposed for linear regression by Hoerl and Kennard (1970).It has been shown that the RRE outperformed the ML.
The RRE is effective but as Liu (1993) pointed out it has the disadvantage that the estimated parameters are complicated non-linear functions of the ridge parameter k.Therefore, in this paper another shrinkage estimator for the Poisson model will be proposed which is a generalization of the method proposed for linear regression by Liu (1993).The advantage of this method is that the estimators are a linear function of the shrinkage parameter d.For this reason, this shrinkage estimator has become more popular during recent years (see for examples, Akdeneiz & Kaciranlar, 1995;Kaciranlar, 2003;Alheety & Kibria, 2009 among others).
The purpose of this paper is to solve the problem of an inflated MSE of the ML estimator by applying a Liu estimator.Furthermore, we derive the optimal value of the shrinkage parameter and based on this value we suggest some methods of how the shrinkage parameter should be estimated.In a Monte Carlo study we evaluate the performance of the ML and the Liu estimator applied with the suggested estimators of the shrinkage parameter.The performance criteria used in the simulation study is the MSE and mean absolute error (MAE).In our simulation, factors including the degree of correlation, the sample size and the number of explanatory variables are varied.In this application, the effect of the usage of cars and trucks on the number of killed pedestrians in different counties in Sweden is investigated.This paper is organized as follows: In Section 2, the statistical methodology is described.In Section 3, the design of the Monte Carlo experiment is presented and the result from the simulation study is discussed.An application is presented in Section 4. Finally, a brief summary and conclusions is given in section 5.

Poisson Regression
The Poisson regression model is a benchmark model when the dependent variable (y i ) comes in the form of counts data and distributed as Po (µ i ), where µ i = exp (x i β), x i is the ith row of X which is a n × (p + 1) data matrix with p explanatory variables and β is a (p + 1) × 1 vector of coefficients.The log likelihood of this model may be written as: The most common method to maximize the likelihood function is to apply the IWLS algorithm: where and ẑ is a vector where the ith element equals ẑi = log ( μi ) + y i − μi μi .The MSE of this estimator equals: where λ j is the jth eigenvalue of the X ′ WX matrix.When the explanatory variables are highly correlated the weighted matrix of cross-products, X ′ WX, is ill-conditioned which leads to instability and high variance of the ML estimator.In that situation it is very hard to interpret the estimated parameters since the vector of estimated coefficients is on average too long.By noting that the IWLS algorithm approximately minimizes the weighted sum of squared error (WSSE) one may apply a generalization of the Liu (1993) estimator for linear regression instead: For this estimator we have replaced the matrix of cross-products used in the Liu (1993) estimator with the weighted matrix of cross-products and the ordinary least square estimator (OLS) of β with the ML estimator.The MSE of the Liu estimator equals: where α 2 j is defined as the jth element of γβ and γ is the eigenvector defined such that X ′ WX = γ ′ Λγ, where Λ equals diag ( λ j ) .In order to show that there exist a value of d bounded between zero and one so that MS E , we start by taking the first derivative of equation ( 5) with respect to d: and then by inserting the value one in equation ( 6) we get: which is greater than zero since λ j > 0. Hence, there exists a value of d that lies between zero and one so that MS E . Furthermore, the optimal value of any individual parameter d j can be found by setting equation ( 6) to zero and solve for d j .Then it may be shown that corresponds to the optimal value of the shrinkage parameter.Hence, the optimal value of d j is negative when α 2 j is less than one and positive when it is greater than one.However, just as in Liu (1993) the shrinkage parameter will be limited to take on values only between zero and one.

Estimating the Shrinkage Parameter
The value of d may only take on values between zero and one and there does not exist a definite rule of how to estimate it.However, in this paper some methods will be proposed that are based on the work for linear ridge regression by for instance Hoerl and Kennard (1970), Kibria (2003) and Khalaf and Shukur (2005).As in those papers, the shrinkage parameter, d j , will be estimated by a single value d.The first estimator is the following: where we define α2 max and λmax to be the maximum element of α2 j and X ′ ŴX, respectively.Replacing the values of the unknown parameters with the maximum value of the unbiased estimators is an idea taken from Hoerl and Kennard (1970).However, for the Liu estimator another maximum operator is also used that will ensure that the estimated value of the shrinkage parameter is not negative.Furthermore, the following estimators will be used: Using the average value and the median is very common when estimating the shrinkage parameter for ridge parameter and the D2 and D3 estimators has direct counterparts in equation ( 13) and (15) in Kibria (2003).Using other quantiles such as the maximum value, was successfully applied in Khalaf and Shukur (2005) and the idea behind the D4 and D5 estimator are taken from those papers.

Judging the Performance of the Estimators
To investigate the performance of the Liu and the ML methods, we calculate the MSE using the following equation: and the MAE as: where β is the estimator of β obtained from either ML or Liu and R equals 2000 which corresponds to the number of replicates used in the Monte Carlo simulation.

The Monte Carlo Simulation
This section consists of a brief description of how the data is generated together with a discussion of our findings.

The Design of the Experiment
The dependent variable of the Poisson regression model is generated using pseudo-random numbers from the Po (µ i ) distribution, where Following Gibbons (1981), the parameter values in equation ( 11) are chosen so that ∑ p j=1 β 2 j = 1 and To be able to generate data with different degrees of correlation we use the following formula to obtain the regressors: where z i j are pseudo-random numbers generated using the standard normal distribution and ρ 2 represents the degree of correlation (see, Kibria, 2003;Muniz & Kibria, 2009 among others).In the design of the experiment three different values of ρ 2 corresponding to 0.85, 0.95 and 0.99 are considered.To reduce eventual start-up value effects we discard the first 200 observations.
In the design of the experiment the factors n and p are also varied.Since the ML estimators are consistent, increasing the sample size is assumed to lower MSE and MAE while p is assumed to increases the instability of X ′ WX and lead to an increase of both measures of performance.We use sample sizes corresponding to 15, 20, 30, 50 and 100 degrees of freedoms (df=n-p) and number of regressors p equals to 2 and 4.

Results Discussion
The estimated MSE and MAE for p=2 and 4 are presented in Tables 1 and 2 respectively.It is evident from these tables that the degree of correlation and the number of explanatory variables inflate both the MSE and MAE while increasing the sample size leads to a decrease of both measures of performance.We can also see that the MSE increases more when considering the MSE instead of the MAE criteria.Hence, the gain of applying Liu is larger in terms of MSE than MAE.Furthermore, when looking at both measures of performance we can see that the estimator D5 is always either the shrinkage parameter that minimizes the MSE and MAE or it is very close to the shrinkage parameter that minimizes these loss functions.

Conclusions
In this paper, the shrinkage estimator developed by Liu (1993) for the linear regression model has been extended for the Poisson regression model.This estimator is proposed in order to reduce the inflation of the variance of the ML estimator caused by multicollinearity.The Liu and the ML estimators are evaluated by means of Monte Carlo simulations.Both MSE and MAE are used as a performance criteria and factors including the degree of correlation, the sample size and the number of explanatory variables are varied.Both measures of performance show that the proposed Liu estimators are better than ML in the sense of smaller MSE and MAE.We also observed that the estimator D5 is often the shrinkage parameter that minimizes the estimates MSE and MAE.