A Comparison of Correlation Coe ﬃ cients via a Three-Step Bootstrap Approach

In this paper we compare ten correlation coe ﬃ cients using a three-step bootstrap approach (TSB). A three-step bootstrap is applied to determine the optimal repetitions, B , to estimate the standard error of the statistic with certain degree of accuracy. The coe ﬃ cients in question are Pearson product moment ( r ), Spearman’s rho ( ρ ), Kendall’s tau ( τ ) , Spearman’s Footrule ( F t ), Symmetric Footrule ( C ), the Greatest deviation ( R g ), the Top - Down ( r T ), Weighted Kendall’s tau ( τ w ), Blest ( ν ), and Symmetric Blest’s coe ﬃ cient ( ν ∗ ). We consider a standard error criterion for our comparisons. However, since the rank correlation coe ﬃ cients su ﬀ er from the tied problem that results from the bootstrap technique, we use existing modiﬁed formulae for some rank correlation coe ﬃ cients, otherwise, the randomization tied-treatment is applied.


Introduction
One may be interested in the relationship between two factors or two variables and would wish to represent this relationship by a number or even using a statistical technique to make an inference.This number is called a correlation coefficient.The most common and well known correlation coefficient is the Pearson moment product coefficient.Some people may use this coefficient immediately ignoring the bivariate normality assumption of the data.Others may use a nonparametric rank correlation coefficient such as Spearman rho or Kendall tau for the same purpose.However the Pearson coefficient examines different aspects compared to Spearman and Kendall.Pearson coefficient considers the linearity of the relationship whereas Spearman and Kendall study the monotonicity of this relationship.
In some circumstances, we may have data with some outliers, in which case using the Greatest deviation correlation coefficient would be more suitable due to its robustness property against outliers.However, in other situations, assigning more emphasis on the top of the observations is required.For this purpose, several nonparametric coefficients are suggested, namely: the Top-Down, Weighted Kendall's tau, Blest, and Symmetric Blest's coefficient.
Usually the variance of the correlation coefficients is derived under the assumption that the null hypothesis that there is no correlation is true.Otherwise, it is intractable to calculate the variance without this assumption.Surely one would prefer using the correlation coefficient which produces small variation, and, consequently less standard error.Borkowf (2000) presented a new nonparametric method for estimating the variance of Spearman's rho by calculating ρ from a two-way contingency table with categories defined by the bivariate ranks.His method is a computer method depending on the data at hand like the bootstrap and jackknife methods.There are claims that his method is more accurate than the bootstrap and jackknife methods.However, it is more complicated and the differences in accuracy are small in comparison with other methods.
In this paper, we will apply the bootstrap method.Yet, the common issue is how many replications, B, should run to get the most accurate results required.One such approach is introduced by Andrews and Buchinsky (2000;2001;2002) which is called a three-step approach.We will use this approach to determine the optimal number of replications for different degrees of accuracy.In addition we will use it for comparison, precisely, to estimate the standard error of the estimators, in our case the correlation coefficients, without imposing the null hypothesis.However, by using the bootstrap technique, the rank correlation coefficients suffer from tied problem.Therefore, we will use the existing adjusted formula for Spearman and Kendall coefficients (Hollander & Wolfe, 1999); otherwise, the randomization tied-treatment is applied (Gibbons, 1971).
Overviews of the correlation coefficients that are used in comparisons are given in Section 2. In Section 3, we will introduce some bootstrap notation and motivations.Then, in Section 4, we will summarize the three-step approach for estimating the standard error.The application of this approach on the correlation coefficients is introduced via the example in Section 5. Finally, conclusions are reported in the last section.
In Table 1 we summarize these coefficients which are involved in our comparisons.In the second and third columns their formulae are stated, yet the fourth column contains the adjusted formulae for Spearman and Kendall coefficients for when ties occur.For others coefficients where no adjusted formula is found in current literature, we will use the randomization tool (Gibbons, 1971) to deal with tied problem .The justification behind using the randomization method is that it behaves randomly as the bootstrap does.However, the most important property is that the randomization methods do not affect the null distribution of the rank correlation coefficient, so we do not need to adapt the null distribution for these coefficients.

Bootstrap Motivation and Notation
The bootstrap technique was first introduced by Efron in 1979 (Efron & Tibshirani, 1994).It is a computer intensive method of statistical analysis that uses resampling to calculate standard errors, confidence intervals and significance tests.There are various applications of the bootstrap techniques in life sciences such as medical, social, and business science.They are applicable as a parametric or semi-parametric or nonparametric technique.In this paper we will consider the nonparametric bootstrap since the correlation coefficients (except Pearson) in question are nonparametric rank correlation coefficients.
Actually, the most common problem in bootstrap literature is choosing the optimal number of repetitions, B. By choosing different, small values it can result in different answers.By choosing extremely large values it gives a more accurate result, however more costly.Andrews and Buchinsky (2000;2001;2002) introduced a three-step approach to determine the number of repetitions B with pre-fixed degrees of accuracy which is applied for many different bootstrap problems, such as estimating the standard error, confidence interval, and p-value for classical statistical techniques.
The aim is to achieve the desired level of accuracy by choosing B. Accuracy is measured by the percentage deviation of the bootstrap standard error estimate, confidence interval length, test's critical value, test's p-value, or bias-corrected estimate based on B bootstrap simulations from the corresponding ideal bootstrap quantities for which B = ∞.A bound of the relevant percentage deviation, pdb, is specified such as the actual percentage deviation is less than this bound with a specified probability, (1 − δ) tends to one.That is, for given (pdb, δ), the optimal number of repetitions, B * , satisfies where λ is a quantity of interest (the standard error in our case), λ∞ is an "ideal" bootstrap estimate, and λB is the bootstrap approximation of λ∞ based on B bootstrap repetitions.Also, P * here represent the probability with respect to the randomness in the bootstrap samples.
In the rest of this section we will present the notation which we will use in the bootstrap framework.Our observed data is a sample of size n: X = (X 1 , ..., X n ) , where X i = (x i , y i ); i = 1, 2, ..., n.Let X * = (X * 1 , ..., X * n ) be a bootstrap sample of size n based on the original sample X.When the original sample X is comprised of independent and identically distributed (iid) or independent and nonidentically distributed (inid) random variables, the bootstrap sample X * often is an iid sample of size n drawn from some distribution F. In this paper, F is the empirical distribution due to that we use the nonparametric bootstrap.
Let θ = θ(X) be an estimator of a parameter θ 0 based on the sample X.One of the application is to bootstrap standard error estimates for a scalar estimator θ, in our case θ is a correlation coefficient.The quantities (se, ŝe ∞ , ŝe B ) in this case are the standard error, the "ideal" bootstrap standard error estimator, and the bootstrap standard error estimator based on B bootstrap repetitions, respectively, where E * represent the expectation with respect to the randomness in the bootstrap samples.These quantities are given as The three-step method, introduced by Andrews and Buchinsky (2000), depends on the estimation of the coefficient of excess kurtosis, γ 2 , of the bootstrap distribution of the parameter estimator, where By using the three-step method we aim to choose a value of B to achieve a desired accuracy of ŝe B for estimating ŝe ∞ using pre-specified values of (pdb, δ).The method involves the following steps: Step(1): Suppose γ 2 = 0, and a preliminary value of B, denoted B 0 where and int(a) be the smallest integer greater than or equal to a, z α represents the α quantile of the standard normal distribution.

Example
Let us consider the data from a study designed to ascertain the relative importance of the various factors contributing to tuna quality and to find objective methods for determining quality parameters and consumer preference (Hollander & Wolfe, 1999).Table 2 gives values of the Hunter L measure of lightness (X), along with panel scores (Y) for nine lots of canned tuna (also see Figure 1).
The preliminary and the final values of the number of the replications for estimating the standard errors of our correlation coefficients are given in Tables 3 and 4. We will categorize our comparison in three blocks; a parametric correlation coefficient (Pearson), nonparametric rank correlation coefficients (Spearman, Kendall, Spearman's Footrule, Symmetric Footrule and the Greatest deviation coefficient).Finally, weighted nonparametric rank correlation coefficients (Top-Down, Weighted Kendall, Blest's coefficient and Symmetric Blest's coefficient).
For example, let us focus on the situation when pdb = 5% and δ = 0.01, as in Table 5.Here the preliminary value of B 0 (for all correlation coefficients) is 1327 which is quite small as a starting point for any simulation, then the adjusted number of replications, B 1 , is given in the third column in Table 5, which varies from one correlation coefficient to another.For example, for the Pearson correlation coefficient, this number, B 1 is more than double the preliminary value (increased by 177%).However, for Footrule B 1 is just increased by 2.4% (from 1327 to 1527).For Kendall and Weighted Kendall, the number of replications, B 1 , increased almost the same (between 38.9% and 39.41%).Among the nonparametric correlation coefficients, Spearman has the highest increase in the number of replications, increased by 72.57%.
According to the differences between the observed (calculated value from the data) and the bootstrap simulation estimate, we can see that for Kendall correlation coefficient, the difference between the actual value of the statistic (observed value) and the bootstrap estimate is small.However, the largest difference between the observed and the estimated values is for Weighted Kendall since we choose m = 5 which means we ignore about a half of the data but this difference gap will reduce and is equal to that of Kendall at m = n = 9.Also, the differences are quite similar for Pearson, Spearman, Footrule, and Greatest, which are all between 0.0312 and 0.0520.
With respect to the standard error estimate, we found that the Pearson correlation coefficient has the smallest standard error, however, the Weighted Kendall has the largest value of standard error.In fact it is twice as much as the standard error of Pearson correlation coefficient.It is for the same reason that Weighted Kendall ignores some of the data (in our case 4 pairs of observations are ignored).In block 2, we found that the Greatest correlation coefficient has the smallest standard error which is expected since our data has some outliers (see Figure 1).However, Spearman and Footrule have large standard errors within this block.
In block 3, when our interest is focused on the initial (top) data, we noticed that the Symmetric Blest's coefficient has a smaller standard error than its asymmetric version and is also smaller than the other weighted correlation coefficients.However, as we mentioned above, Weighted Kendall has the largest standard error.Generally, the standard errors for correlation coefficients in block 3 are larger than those in block 1 and 2.

Conclusion
To conclude, one should use the Pearson correlation coefficient if the data meets the normality assumption, otherwise, the Greatest deviation performs well especially when the data has outliers.However, when we want emphasis on the initial (top) data, the Symmetric Blest's coefficient has lowest standard error amongst other weighted correlation coefficients.Salama I. A. and Quade D. (2002) where g denotes the number of tied p groups, t i is the size of tied p group i, h is the number of tied q groups, and u j is the size of tied q group j (Hollander & Wolfe, 1999).

Kendall's Tau
where sgn(a) = 1 if a > 0, where g denotes the number of tied X groups, sgn(a) = 0 if a = 0 and t i is the size of tied X group i, h is the number sgn(a) = −1 if a < 0.
of tied Y groups, and u j is the size of the tied Y group j (Hollander & Wolfe, 1999).

Figure 1 .
Figure 1.A simple scatter plot for Tuna Lightness and Quality data