How does the elimination of group mean-differences affect factor score determinacy?

The present study investigates to what degree the common variance of the factor score predictor with the original factor, i.e., the determinacy coefficient or the validity of the factor score predictor, depends on the mean-difference between groups. When mean-differences between groups in the factor score predictor are eliminated by means of covariance analysis, regression, or group specific norms, this may reduce the covariance of the factor score predictor with the common factor. It is shown that in a one-factor model with the same group mean-difference on all observed variables, the common factor cannot be distinguished from a common factor representing the group mean-difference. It is also shown that for common factor loadings equal or larger than .60, the elimination of a d = .50 mean-difference between two groups in the factor score predictor leads to only small decreases of the determinacy coefficient. A compensation-factor k is proposed allowing for the estimation of the number of additional observed variables necessary to recover the size of the determinacy coefficient before elimination of a group mean-difference. It turns out that for factor loadings equal or larger than .60 only a few additional items are needed in order to recover the initial determinacy coefficient after the elimination of moderate or large group mean-differences.


Factor Score Predictors and Group Mean-Differences
Factor score predictors are computed in order to provide individual scores for a factor resulting from factor analysis (DiStefano, Zhu, & Mindrila, 2009).These scores can be used in applied settings, e.g., job-selection.The validity of factor score predictors depends on their correlation with the corresponding factors, which is usually termed determinacy coefficient (Gorsuch, 1983;Grice, 2001).For orthogonal factors and block-diagonal patterns of factor loadings, the reliability of the regression factor score predictor also corresponds to the determinacy coefficient (Beauducel, Harms, & Hilger, 2016).However, under more general conditions, the correlation of the factor score predictor with the corresponding factor is an indicator of validity.
Several group mean-differences may co-occur with the individual differences represented by a common factor.Some group mean-differences may be estimated a priori in the context of multiple-group factor analysis, which leads to the issue of measurement invariance of factors across groups.Measurement invariance refers to the similarity of model parameters across groups, an assumption required for the interpretation of group differences on the factors (Vandenberg, & Lance, 2000).New methods that may allow for the improvement of measurement invariance have meanwhile been proposed (Asparouhov, & Muthé n, 2023).However, not all relevant group mean-differences may be considered right from the start when factor analysis is performed.A reason for this could be that sample size leads to a limitation of the number of groups that can be considered in multiple-group factor analysis.Another reason could be that a grouping of individuals is obtained afterwards (e.g., by means of latent class analysis).A third reason could be that group mean-differences that appear of irrelevant size in the observed variables, can be of relevant size in the factor score predictors.Group mean-differences that are not represented by a model parameter for the means, may affect the covariances of observed variables, and may be present in the factor score predictor.In these cases, an elimination of group mean-differences may be performed a posteriori by means of regression analysis, covariance analysis, or by means of group-specific test norms or test standardization (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014).The elimination of the group mean-differences may affect the validity of the factor score predictor, i.e., the factor score determinacy.Therefore, the present study investigates the effect of eliminating group mean-differences on the resulting determinacy coefficient.

Specifications of the Study
In principle, different types of factor score predictors, different numbers of groups, different factor models, and different score distributions might be of interest in this context.However, in order to provide initial results that could be the starting point for further investigations, the present study is based on only two groups and on the determinacy coefficient computed for the best-linear factor score predictor (Krijnen, Wansbeek, & Ten Berge, 1996), which is sometimes termed regression factor score predictor.Moreover, the study is based on one factor models with equal factor loadings.These simplifications allow for a direct investigation of effects of the size of group mean-differences in terms of different effect sizes d (Cohen, Cohen, West, & Aiken, 2003) and the number of variables on the determinacy coefficient.
First, a population factor model for the description of the effect of group mean-differences on factors and model parameters is defined.On this basis, the effect of factor loadings and the number of variables on the determinacy coefficient is investigated.Benchmarks for factor loadings combined with a given number of observed variables resulting in reasonable determinacy coefficients are provided.Using these benchmarks, the effect of group mean-differences and determinacy coefficients is analyzed.It is proposed to calculate the number of additional items necessary to compensate for reduced determinacy resulting from eliminating group mean-differences.Finally, theoretical and practical implications of the results are discussed.

Definitions
In the population of individuals, the common factor model with an additional factor representing the means between two groups can be defined as where x is a standardized random vector of p observed variables, ξ is a random vector of q common factors, with ' () Λ is the p  q matrix of common factor loadings.Moreover, g is a random vector representing the means of two groups, with ' ( ) 1 E  gg for q = 1 (for q > 1 we get a q  q identity matrix), ' () E  gξ0 , and g Λ is the p  1 matrix of loadings of the observed variables on the factor representing the standardized group mean-difference d (e.g., Cohen et al., 2003).For equal group size ' ( ) 1 E  gg and ( ) 0 E  g g corresponds to an unweighted effect-coded indicator variable.For two groups, unweighted effect-coding means that the resulting indicator variable takes the value +1 for one group and -1 for the other group (Cohen et al., 2003).The loadings g Λ represent the correlation of the observed variables with g.Therefore, the effect size d is converted into the correlation r according to Ruscio (2008, Table 1).For groups of equal size, the conversion of d into a correlation r is , so that the loadings are given by , where 1 is a p  1 unit-vector.For unequal group sizes g Λ can be computed by , where 1 p is the base rate (i.e., the number of individuals in a group, divided by the number of individuals in the total finite population) of group 1 and 2 p is the base rate of group 2. For unequal group size, the group membership is coded into the effect-coded variable θ , so that  εg 0 , and Ψ is a p  p diagonal, positive definite, unique factor loading matrix with p unique factor loadings.This implies that is a correlation matrix.
For q = 1 and when d is the same for all observed variables, it is impossible to disentangle ξ and g, so that only a single common factor g ξ combining ξ and g can empirically be identified when the mean-difference between the groups occurs consistently across all observed variables.A model based on a single detectable common factor g ξ comprising equal group mean-differences on the observed variables and individual differences can be written as The determinacy coefficient is the correlation of a factor score predictor with the corresponding factor, i.e., Σ Λ Φ , the weights for the best-linear factor score predictor (Grice, 2001), for w in Equation 4yields (5) for q = 1.

Effect of Factor Loadings and p on the Determinacy Coefficient
For q = 1 and all loadings ξg Λ being equal, it follows from 1   ξg w Σ Λ Φ that all weights w are equal so that one can write w  w1 , where w is a scalar with w ≠ 0 and 1 is a p  1 unit-vector, so that When all weights are equal and greater zero and all loadings are equal and greater zero, and q = 1, it is possible to write .As for q = 1 and equal factor loadings the weights cancel out, Equation 6 can be written as and because 2 1 x   , the common variance of the scores with the corresponding factor is

Benchmarks of Factor Loadings for Reasonable Determinacy
Factor score predictors should at least have 50% of common variance with the corresponding factor.Entering However, 80% or 90% of common variance of the factor score predictor with the corresponding factor will be a more reasonable benchmark.Entering Ρ can be computed by For 90% of common variance of the factor score predictor with the factor and p = 4 a loading size of .69 is required, which is attainable in several areas (see Figure 1).

Loading Benchmark, Group Differences and Determinacy
In the following, the effect of eliminating group differences in the factor score predictor on the determinacy coefficient is considered for q = 1 and equal loading sizes.Eliminating group mean-differences implies that Ρ is presented for different levels of  ξ .Whereas the effect of eliminating a mean-difference of d = 1.00 on 2 Ρ is quite substantial for  ξ = .20 and  ξ = .40,it is rather small for  ξ = .60and beyond.However, according to Cohen et al. (2003) d = 0.80 is a large effect, whereas d = .50represents a moderate effect, and d = .20represents a small effect.Eliminating a moderate effect of d = .50may also be considered.As an example, consider the condition p = 10 and  ξ = .60.For this condition and d = 0.50 Equation 14 For p = 10,  ξ = .60,and d = 0.00 Equation 14 yields 2  ξg P 2 = .90P 2 = .80 .50 1 4 .50 10 1 10 .861. .60 Thus, in this condition, eliminating a moderate effect size in the factor score predictor will reduce the common variance of the factor score predictor with the original factor by only 2%.
It might be possible to compensate a decrease of 2 Ρ resulting from eliminating a group mean-difference by means of an increase of p.The increase of p necessary for compensating for the elimination of a group mean-difference can be calculated by means of the compensation-factor k as a multiplier of p.The compensation-factor k can be found by means of equating Equation 14comprising the effects of d with Equation 14for d = 0 and k as a multiplier of p.This yields (18) and after some transformation   Ρ can be compensated by a few additional items (Figure 3, B and C).
When a group mean-difference is eliminated and when it is impossible to compute a multiple-group factor analysis for the respective groups, it might be of interest to compute the reduced factor loading.Equation 13 can be transformed in order to correct the initial factor loadings, when the eliminated group mean-difference d is known.This yields .81 1 1 4 .81 1 1 1.38..60 .81 .60 4 .81

Empirical Example
Data from 6892 German-speaking participants (2343 females, 4549 males; age in years: M = 25.93,SD = 9.79), who completed voluntarily the numerical tasks of the intelligence-structure-test 5 (IST 5;Beauducel, Brocke, & Liepmann, 2024) were used as an example.The tasks were numerical calculation, number series, and signs.Participants worked 10 minutes on each task.Each task comprised 20 items.The number of correct responses on each task (raw data) were entered into principal axis factor analysis (IBM SPSS Version 29.0.0.0).The resulting factor loadings are given in Table 1 (left column).The z-standardized best linear factor score predictor was computed and the corresponding detminacy coefficient was P² = .874.When a (Welch) t-test for gender differences was performed, the mean-difference was significant (t 4843.35= 12.98, p < .001,two-tailed), with a larger mean for males (M = 0.11, SE = 0.02) than for females (M = -0.21,SE = 0.02), with d = 0.32.In the next step, the tasks were residualized for gender and entered into principal axis factor analysis (Table 1, right column), so that the mean difference between males and females was zero for the corresponding z-standardized best linear factor score predictor.The determinacy coefficient of the residualized best linear predictor was P² = .872.

Discussion
The determinacy coefficient describes the common variance of a factor score predictor with the respective factor and is therefore an indicator of the validity of factor score predictors.The present study investigates to what degree the validity of factor score predictor may depend on a given mean-difference between two groups.This question is relevant when mean-differences between groups are eliminated by means of covariance analysis, regression, or group specific norms.
It is shown that in a one-factor model comprising the same group mean-difference on all observed variables, the common factor cannot be distinguished from a common factor representing the group mean-difference.This occurs even when the common factor and the group mean-difference have a zero covariance.Moreover, the effect of eliminating a group mean-difference d on the common variance of the factor score predictor with the factor (determinacy coefficient) is investigated under the condition of equal common factor loadings.It is shown that for common factor loadings equal or larger than .60 the elimination of group mean-difference of a moderate effect size (d = .50)leads to only small decreases of the determinacy coefficient.It is, nevertheless, important to estimate the reduced factor score determinacy when group mean-differences in the factor score predictors are eliminated.In this context, an estimation of the reduced factor loadings can also be of interest.
A compensation-factor k is proposed allowing for the estimation of the number of additional observed variables that are needed to recover the size of the determinacy coefficient before elimination of the group mean-difference.It turns out that for factor loadings equal or larger than .60only a few additional items are needed in order to recover the initial determinacy coefficient after the elimination of moderate or large group mean-differences.
An empirical example based on numerical intelligence tasks demonstrates that a significant mean difference between males and females of about d = 0.30 can be partialled out without any relevant effect on the determinacy coefficient.This implies that group-specific norms that might eliminate relevant group-differences in the factor score predictor do not necessarily reduce the validity of the scores.The empirical study shows that the algebraic results found for equal factor loadings can also be found for unequal factor loadings.
As a limitation of the study, it should be noted that no information on the educational level was available in the empirical data set.Therefore, the mean difference found in these data for numerical intelligence cannot be generalized to the population.Moreover, the investigation of the elimination of group differences on determinacy coefficients may also be performed for a larger number of factors.This would be of special interest because the elimination of group differences may affect the rotational position of the factors and the related pattern of factor inter-correlations.A further limitation of the study is that it is only based on the best-linear (regression) factor score predictor (Krijnen et al., 1996).
An investigation of the elimination of group mean-differences on the determinacy coefficients for other factor score predictors (Beauducel, & Hilger, 2022;Grice, 2001;Krijnen et al., 1996) may also be of interest.Other lines for further research are the investigation of the effect of unbalanced group-size, mean-differences between more than two groups, the effect of categorical observed variables, the effect of non-normal score distributions, and the effect of methods for the estimation of model parameters.
The intended loading size for a given p and 2

Figure 1 .
Figure 1.Loading size and p for 50%, 80%, and 90% common variance of the factor score predictor with the factor It follows from Equation 3 that eliminating group mean-differences reduces to ξ Λ .a baseline for Figure2 A-D, where the relationship of d and p with 2 kp p   might be rounded to the nearest integer, which indicates the number of observed variables that might be added in order to compensate for a decrease of 2 Ρ resulting from eliminating d.If 1 kp p it might be reasonable to add observed variables in order to reach the same 2 Ρ that occurred before elimination of the group mean-difference.As an example, consider a factor score predictor based on p = 5, .Now, d is eliminated by means of separate test-norms, or something similar, so that 2 .When 2 items are added the resulting 2 Ρ will be about .83.So, a compensation for eliminating a mean-difference of about .80 is feasible.The compensation of the elimination of a group mean-difference d  .20 by means of additional items p kp p   is plotted for different levels of 2 Ρ  .70 and different levels of  ξ in Figure 3. For eliminating moderate or large mean-differences on 2

Table 1 .
Factor loadings based on raw data and on data residualized for gender (right column)