Construction of Modified Central Composite Designs for Non-standard Models

The use of loss function in studying the reduction in determinant of information matrix due to missing observations has effectively produced designs that are robust to missing observations. Modified central composite designs are constructed for non-standard models using principles of the loss function or equivalently first compound of (I−H) matrix associated with hat matrix H. Although central composite designs (CCDs) are reasonably robust to model mis-specifications, efficient designs with fewer design points are more economical. By classifying the losses due to missing design points in the CCD portions, where there are multiple losses associated with specified CCD portions, the design points having less influence may be deleted from the full CCD. This leads to a possible increase in design efficiency and offers alternative designs, similar in the structure of CCDs, for non-standard models.


Introduction
Response surface methodology has continued to play vital roles in developing, optimizing and improving processes, particularly where several input variables, 1 , 2 , ⋯ , , potentially influence some performance measure or quality characteristic, , of the process under study. The relationship between the input variables and the response variable is = ( 1 , 2 , ⋯ , ) + (1) where is the random error component assumed to have a normal distribution with zero mean and constant variance such that ( ) = ( 1 , 2 , ⋯ , ) = η In most practical situations, the true response function is unknown but may be approximated using low-degree polynomials in some relatively small region of the independent variable space.
When first-order model is considered adequate, it is assumed that there is little of no indication of the presence of curvature in the response function. In terms of the coded input variables, the first-order model is However, if there is an indication of interaction between the independent variables, the first-order model with a measure of interaction is Model inadequacy exists when there is substantial curvature in the true response function (indicated by the lack of fit of first-order model) and the need for an approximating polynomial of higher order arises.
In terms of the coded input variables, the second-order model is

The Hat Matrix and the Loss Function
The mention of hat matrix takes one's mind to leverage which according to Myung and Kahng (2007) is a basic component of influence in linear regression models. Each diagonal element ℎ of the hat matrix is called leverage and measures the extent to which the fitted regression model ̂ is attracted by the given observation or data point . In essence, the ℎ leverage ℎ quantifies the influence that the observation has on its predicted value ̂. The diagonal elements of the hat matrix are such that 0 ≤ ℎ ≤ 1 and ∑ ℎ =1 = ; where is the number of data points and is the number of model parameters, including the intercept. Aside investigating whether one or more observations excessively influence the estimated values, the hat matrix may also be used to quantify the effect of removing one or more observations from a design. Thus, the hat matrix (through its compounds) is useful in understanding effect of missing observations in a complete data set. Akhtar and Prescott (1986) considered the relative reduction in determinant of information matrix due to missing observations, by quantifying the effect of missing observations using the criterion of loss function. When m observations in the design are missing, the complete data set is altered and the model matrix reduces by m rows. As expected, the information matrix, ′ , for the reduced design will differ from the information matrix, ′ , for the complete design. For mxp matrix , of m missing rows corresponding to the m missing observations, the information matrix may be expressed as is an identity matrix of order .
By rearrangement, Taking determinants of both sides yield In Akhtar (1987), the determinant│[ − ( ′ ) ( ′ ) −1 ]│is called the diagonal element of the ℎ compound of (I− ) where is the hat matrix according to the missing observations and I represents an identity matrix of same dimension as H. Specifically, the loss function measures the relative reduction in the determinant of information matrix associated with the complete design in the presence of missing observations. Ahmad et al. (2010) related │( ′ )│and │( ′ )│using identities on the expansion of a bordered determinant where … is the corresponding diagonal element of the ℎ compound of (I− ) matrix and is the symmetric matrix ( ′ ) −1 ′ of order N. Specifically, the ℎ diagonal element of the first compound of (I− ) matrix is ; the ℎ diagonal element of the second compound of (I− ) matrix is , and so on. It is clear from Akram (2002) and Atken and Rutherford (1964) that the first compound of the matrix (I− ) is the hat matrix itself. Thus, the diagonal elements of the hat matrix in addition to establishing measures of influence of design points also quantify the loss due to a single missing observation.
For m missing observations, the determinant of the complete information matrix may be obtained as The relative loss due to any set of missing observations is defined as This implies For central composite designs, the determinant │( ′ )│ is an increasing function of the axial distance α and is maximized at α = ∞. The values of α specifies the location of the axial point and are usually chosen to satisfy various design conditions such as rotatability and orthogonal blocking. A number of researchers have studied the loss due to a set of missing observations over a range of α value associated with central composite designs and minimaxloss designs that are robust to sets of missing observations have emerged. Some of such results include Akhtar (2001) and Akram (2002). The loss functions (or equivalently, compounds of the (I− ) matrix) have thus been effectively used in studying the relative reduction in determinant of information matrix due to missing observations and has effectively produced designs that are robust to missing observations. Much study on loss function considers the second-order central composite designs differing in the number of control variables and configurations of the factorial, axial and center portions; Akhtar (1987), Akhtar (2001), Akram (2002), Ahmad and Gilmour (2010), Ahmad et al. (2011), Yakubu et al. (2014, Okon and Nsude (2015), Iwundu (2017) Akhtar (1987), Akhtar (2001), Akram (2002), Ahmad and Gilmour (2010), Ahmad et al. (2011), Yakubu et al. (2014, Okon and Nsude (2015), Iwundu (2017).

The Central Composite Design and Another Look at the Hat Matrix
In most practical modeling situations involving several control variables, the central composite designs have been satisfactorily used in optimization phase of experimentation. However, the second-order model which justifies the use of the central composite design is assumed prior to experimentation. After data have been collected, model fitting may reveal that not all model parameters are significant and hence the need to remove insignificant parameters from the model thus resulting in a reduced posterior model.
For a full parameter second-order model, the diagonal elements of the hat matrix reveal a unique property associated with central composite designs. The diagonal elements associated with design points in a particular CCD portion are a constant for all design points in that portion. Specifically, 2 vertex points have constant diagonal element say, ℎ = ; 2 axial points have constant diagonal element say, ℎ = and center points have constant diagonal element say, ℎ = . Where fractions of full factorial designs are utilized, the property remains unchanged. This implies that the design points of a CCD portion have an equal influence in modeling. For a "non-standard" or "mis-specified" model, the associated hat matrix loses the uniqueness of its diagonal elements for the employed central composite design. This clearly shows that the design points of the CCD portion do not anymore have an equal influence in modeling. Adjustments may be made on the design to retain the unique property of its diagonal elements for the posterior "mis-specified" model, either absolutely or approximately. The choice of the words "mis-specified" or "non-standard" is on the premise that the central composite design is essentially constructed for a full parameter model.

Construction of the Proposed Modified Central Composite Design (MCCD)
In the study of robustness against many cases of potential model mis-specification, central composite design is reported to be robust against model mis-specification in comparison to the Box-Behnken design as in Borkowski and Valeroso (1997). Myer et al. (2009) also observed the robustness of central composite design in relation to computer generated optimal design. For the purpose of illustrating the construction of Modified Central Composite Design, non-standard models may be formed using the following Hierarchical approach of Borkowski and Valeroso (1997): (i) If a model contains 2 term, then it must contain the corresponding term. (ii) If a model contains term, then it must contain the corresponding and / or term. The central composite design shall be employed in the construction of optimal design for non-standard models. However adjustments will be made to eliminate points in CCD portions having low impacts on modeling. By classifying the diagonal element in the CCD portions, where there are non-constant diagonal elements (multiple influences or even losses) associated with a specific CCD portion, the design points having less influence may be deleted from the full CCD. This allows an improvement in design's efficiency and offers alternative designs, similar in the structure of CCDs, for non-standard models. It is important to say at this juncture that the presence of no outlying observations is assumed.

Design Efficiency for Modified Central Composite Design on Non-standard Model
The design's properties for the modified central composite design shall be considered by computing measures of design efficiencies. Commonly encountered efficiency measures include D-and G-efficiencies. For each reduced model considered, D-efficiency of the modified central composite design (MCCD) shall be compared with D-efficiency of the full central composite design (CCD). Similarly, G-efficiency of the modified central composite design shall be compared with G-efficiency of the full central composite design. As with its name, D-efficiency is a function of the D-optmality criterion which aims at minimizing the determinant of the variance-covariance matrix. On the other hand, G-efficiency is a function of the G-optmality criterion which as a minimax criterion aims at minimizing the maximum scaled prediction variance. By definition, D-and G-efficiency of a design are respectively D-efficiency = 1 ( where is the design size, is the number of model parameters, is the model matrix and is the maximum scaled prediction variance defined as in Myer et al. (2009). The computations of design efficiencies require that the optimal D and G values must be obtained.

Examples
Illustrations will be presented for design categories: (i) A CCD and a modified CCD in a cuboidal region for a non-standard model. (ii) A CCD and a modified CCD in a spherical region for a non-standard model.
The cuboidal region is typified by the choice of axial distance α = 1.0 and the spherical region is typified by the choice of where is the number of design variables and is the number of factorial points in the design. The design efficiency measures shall be employed in assessing the full CCD and the modified CCD across the set of reduced models. However for completeness of comparisons, information regarding the full model shall also be presented.

Illustrations on Cuboidal Region
Example 1 The illustration considers the respective three-variable prior and posterior models ( 1 , 2 , 3 ) = 0 + 1 1 + 2 2 + 3 3 + 12 1 2 + 13 1 3 + 23 2 3 + 11 1 2 + 22 2 2 + 33 3 2 + and ( 1 , 2 , 3 ) = 0 + 1 1 + 2 2 + 3 3 + 12 1 2 + 13 1 3 + 11 1 2 + defined on cuboidal region with axial distance α = 1.0. The full CCD with one center point is  Vol. 7, No. 5;2018 The hat matrices associated with the designs for both prior and posterior models have been presented in section 3 and are in this illustration referred. With regards to the prior model, the model matrix for the full CCD is of dimension (15x10), where the number 15 represents the design size and the number 10 represents the total model parameters. The determinant value associated with the normalized information matrix is 3.1964x10 -4 . The maximum scaled prediction variance is 11.9583. If the full CCD is defined in relation to the posterior model, the model matrix is the (15x7) matrix given below. The (13x7) associated model matrix is The determinant value of normalized information matrix is 0.0326. The maximum scaled prediction variance is 9.1000. The optimality measures and efficiency values for the full and modified CCDs associated with the prior and posterior models are summarized in Table 1.
The design matrix associated with the 9-parameter posterior model is given as The determinant value of normalized information matrix associated with the complete design for the prior model is 0.0028. The maximum scaled prediction variance is 10.1657. Upon examination of the hat matrix in Appendix B associated with the posterior model, the design points (0, 1, 0, 0), (0, -1, 0, 0), (0, 0, 1, 0) and (0, 0, -1, 0) are deleted from the complete design. This gives a reduction in the size of the design and a resulting 21-point design emerges. Thus, the modified CCD for the posterior model is The (21x9) associated model matrix is The determinant value of normalized information matrix is 0.0035. The maximum scaled prediction variance is 10.8182. The optimality measures and efficiency values for the full and modified CCDs associated with the prior and posterior models are summarized in Table 2.
The design matrix associated with the 13-parameter posterior model is given as The determinant value of normalized information matrix associated with the complete design for the posterior model is 0.0536. The maximum scaled prediction variance is 14.5461. Upon examination of the hat matrix in Appendix F associated with the posterior model, the design points (0, 0, 2, 0), (0, 0, -2, 0), (0, 0, 0, 2) and (0, 0, 0, -2) are deleted from the complete design. This gives a reduction in the size of the design and a resulting 21-point design emerges. Thus, the modified CCD for the posterior model is  Vol. 7, No. 5;2018 The determinant value of normalized information matrix is 0.0984. The maximum scaled prediction variance is 13.3438. The optimality measures and efficiency values for the full and modified CCDs associated with the prior and posterior models are summarized in Table 4.
The design matrix associated with the 13-parameter posterior model is given as The determinant value of normalized information matrix associated with the complete design for the posterior model is 0.0253. The maximum scaled prediction variance is 15.6563. Upon examination of the hat matrix in Appendix H associated with the posterior model, the design points (0, 0, 2, 0), (0, 0, -2, 0), (0, 0, 0, 2) and (0, 0, 0, -2) are deleted from the complete design. This gives a reduction in the size of the design and a resulting 21-point design emerges. Thus, the modified CCD for the posterior model is The determinant value of normalized information matrix is 0.0503. The maximum scaled prediction variance is 14.4229. The optimality measures and efficiency values for the full and modified CCDs associated with the prior and posterior models are summarized in Table 5.

Discussion of Results
The removal of less influential design points in the axial portion of the central composite design generally improves the efficiency of the design. For a three-variable central composite design having one center point and defined for α = 1.0, the D-efficiency of the CCD defined for a complete prior 10-parameter model is not as high as the D-efficiency of the CCD defined for the 7-parameter posterior model. The D-efficiency of the modified central composite design for the posterior model exceeds the D-efficiency of the complete CCD whether it is defined for the prior or posterior model. Also, the G-efficiency associated with the modified central composite design defined for the posterior model exceeds the G-efficiency associated with the complete CCD defined for the posterior model. However, the G-efficiency associated with the complete central composite design defined for the prior model exceeds the G-efficiency associated with the complete and modified CCDs defined for the posterior model. What stands out prominently is that G-efficiency values in all three specifications exceed the D-efficiency values.
For the four-variable central composite design having one center point and defined for α = 1.0, the result is similar in the structure of D-efficiency as in the three-variable case, having one center point and defined for α = 1.0. Specifically, the D-efficiency of the CCD defined for a complete prior 15-parameter model is not as high as the D-efficiency of the CCD defined for the 9-parameter posterior model. The D-efficiency of the modified central composite design for the posterior model exceeds the D-efficiency of the complete CCD whether it is defined for the prior or posterior model. On the other hand however, the complete CCD defined for the posterior model has slightly higher G-efficiency value than the G-efficiency value associated with the modified central composite design for the posterior model. Again, as in the three-variable case, the G-efficiency values associated with the complete and modified CCDs are higher than the corresponding D-efficiency values whether prior or posterior models are in use.
For a three-variable central composite design having four center points and defined for α = 1.7321, the D-efficiency of the complete CCD defined for a complete prior 10-parameter model is not as high as the D-efficiency of the modified CCD defined for the 8-parameter posterior model even though it is higher than the D-efficiency of the complete central composite design defined for the posterior model. The G-efficiency associated with the modified central composite design defined for the posterior model exceeds the G-efficiency associated with the complete CCD defined for the posterior model. However, the G-efficiency associated with the complete central composite design defined for the prior model exceeds the G-efficiency associated with the complete and modified CCDs defined for the posterior model. What again stands out prominently is that G-efficiency values in all three specifications exceed the D-efficiency values.
For the four-variable central composite design having one center point and defined for α = 2.0, both Dand G-efficiency values associated with the modified CCD defined for the posterior 13-parameter posterior model exceed the D-and G-efficiency values associated with the complete CCD defined for either the 15-parameter prior model or the 13-parameter posterior model. G-efficiency values associated with the complete and modified CCDs are higher than the corresponding G-efficiency values when defined for the 13-parameter posterior model. An exception holds for the complete CCD defined on the 15-parameter prior model, where the G-efficiency value is less than the corresponding D-efficiency value.
For the four-variable central composite design having three center points and defined for α = 2.0, the D-efficiency of the modified central composite design for the 13-parameter posterior model exceeds the D-efficiency of the complete CCD whether it is defined for the 15-parameter prior or for the 13-parameter posterior model. Also, the G-efficiency value associated with the modified central composite design for the 13-parameter posterior model is higher than the G-efficiency value associated with the complete CCD defined for the 13-parameter posterior model. In all three cases, the G-efficiency values associated with the complete and modified CCDs are higher than the corresponding D-efficiency values whether prior or posterior models are in use.

Conclusion
Designs for non-standard models are constructed using principles of the loss function. The designs are a modification of central composite designs and are similar in the structure of CCDs. By studying possible losses due to missing design points in the CCD portions, where there are multiple losses associated with specified CCD portions, the design points having less influence may be deleted from the full CCD. This leads to a possible increase in design efficiency and offers alternative designs, similar in the structure of CCDs, for non-standard models.