The Validation Process of the SILL : A Confirmatory Factor Analysis

This study examined the validity of the SILL by performing a confirmatory factor analysis among 914 university students learning English in Korea. The results showed that all the fit indices including chi-square, RMSEA, CFI, and NFI used to test Oxford’s two construct and six construct taxonomy of the SILL provided unacceptable fit to the data. These results might be in part due to the large number of the participants and in part due to the high correlations among the constructs. Implications of these findings were discussed, followed by future study areas to shed further light on the SILL and language learning strategies.


Introduction
Learning strategies are defined as specific behaviors or thought processes used by the learner to facilitate acquisition, storage, or retrieval of information (Weinstein & Mayer, 1986).A plethora of research on language learning strategies has been conducted because learning strategies along with other learner variables such as motivation and anxiety play a crucial role in explaining individual differences in second language (L2) acquisition (Skehan, 1991).
Since the "good language learner" studies which investigated the behaviors and thought processes of good language learners compared with less effective language learners, considerable advances have been made in learning strategy research (Rubin, 1975).The advances of research work on language learning strategies have been in part due to the Strategy Inventory for Language Learning (SILL), one of the most popularly used questionnaires in the domain of L2 acquisition and teaching.
The SILL developed by Oxford (1990) consists of direct and indirect learning strategies depending on the extent to which each strategy item is involved in language learning.Direct strategies include memory strategies for remembering and retrieving vocabulary, cognitive strategies for comprehending and producing text, and compensation strategies for compensating for the lack of knowledge, whereas indirect strategies include metacognitive strategies for manipulating learning processes, affective strategies for regulating affective state, and social strategies for learning with others.
Since then, the SILL has been used worldwide to investigate L2 learners' overall learning strategy use, factors underlying strategy choice, relationship between strategy use and L2 performance, and strategy training (Park, 1997;Green & Oxford, 1995;Griffiths, 2003;Hong-Nam & Leavell, 2006;McMullen, 2009;Nisbet et al., 2005;Nyikos & Oxford, 1993;Riazi & Rahimi, 2005;Wharton, 2000;Yang, 1999).In parallel with the empirical use of the SILL in the aforementioned areas, the psychometric properties of the instrument have been examined with the focus on the reliability and validity (Oxford & Burry-Stock, 1995).
However, the construct validity of the SILL determined by exploratory factor analysis (EFA) has been inconsistent with different factor structures across different learning contexts (Eldib, 2004;Green & Oxford, 1995;Nyikos & Oxford, 1993;Robinson & Midorikawa, 2001;Yang, 1999).For instance, Robson and Midorikawa (2001) found as many as 15 factor structures in the SILL among university students in Japan.Green and Oxford (1995) and Eldib (2004) found nine and eight factor structures in the SILL among university students in Puerto Rico and Kuwait, respectively.However, Yang (1999) found six factor structures in the SILL among university students in Taiwan, whereas Nyikos and Oxford (1993) found only five factors among university students in Korea and the USA, respectively.
Researchers have performed EFA to explore latent factor structures and confirmatory factor analysis (CFA) to test a priori factor structures in the relationships between observed and latent variables.Since EFA has shown limitations defining exact factor structures of the SILL because of different findings across studies, CFA should be performed to understand better the latent constructs of the SILL by examining whether the SILL represents either the two-construct or six-construct classification system, as originally proposed by Oxford (1990).The advantage of CFA is that it can test various models by describing the causal relationships between latent constructs and observed variables while allowing for bidirectional relationships among latent constructs (Bryant & Yarnold, 1995).
Despite the pressing need to perform CFA of the SILL, it is surprising to note that only one published study has performed CFA in an attempt to confirm a priori underlying constructs of the SILL among university students in Taiwan (Hsiao & Oxford, 2002).More surprisingly to learning strategy researchers and language teachers who have used the instrument to measure language learning strategies, the result of this study was that all the fit indices used to test the models did not fit the data appropriately.
Taken together, although the SILL has been used worldwide, the results of EFA and CFA to date have produced the potential validity problems of the instrument.The purpose of this study was to examine whether or not the SILL reflects either the two-construct or six-construct classification system proposed by Oxford (1990) by performing CFA among university students learning English in Korea.

Participants
The participants of this study were 914 Korean EFL university students who took an English conversation course which was taught by native English speakers from the USA, Canada, Britain, and Australia.The participants were from various academic fields such as the humanities (19%), social sciences (29%), natural sciences (13%), engineering (19%), medical sciences (18%), and medicine (2%).These students consisted of 357 male students and 557 female students, with the average age of about 21.At the time of data collection, most of the participants had studied English at least for 10 years in elementary and secondary schools with a focus on four skills.Since learning English is in high demand in Korea, these students were assumed to have invested much time and efforts to learn English both inside and outside the school using various language learning strategies to enhance their English learning.Oxford (1990) developed the SILL to measure language learning strategies for speakers of other languages learning English (Version 7.0, see Appendix).The SILL consists of 50 items in the two dichotomous constructs of direct (29 items) and indirect (21 Items) learning strategies.Direct strategies are subdivided into memory (9 Items), cognitive (14 Items), and compensation (6 Items) strategies, whereas indirect strategies are subdivided into metacognitive (9 Items), affective (6 Items), and social (6 Items) strategies.In administering the SILL, respondents are asked to answer their use of language learning strategies in five-point Likert-scale items from "(1) never or almost never true of me" to "(5) always or almost always true of me."Those who score .35 and higher in the overall average of the SILL are high strategy users, between .25 and .34 are medium strategy users, and .24and below are low strategy users.

Data collection and analysis
The data were collected in intact classes by native English teachers who taught an English conversation course.The native English teachers were recommended by the staff in the Department of Practical English Education because they were committed and cooperative teachers.The author met the teachers individually, explained the purpose of this study, and asked for their cooperation in collecting data.After assuring of their cooperation, the author explained to them in-depth data collection procedures.The teachers briefly explained the nature of this study to the students, asked for their sincere and honest responses to the SILL, and made clear that the data would be used for research purposes only.
When data collection was finished, a research assistant examined all the data, screened out missing data, and coded the data into the SPSS program for analysis.In order to examine whether or not the two-construct or six-construct classification system represents the SILL, the reliability of and correlations among the constructs were investigated by calculating Cronbach's α and Pearson product-moment correlations using SPSS 18.Then, three learning strategy models were proposed as described below and tested by performing a maximum likelihood CFA using AMOS 17.
Model 1: The 50 items of the SILL consist of a single construct that is sufficient to account for language learning strategies.
Model 2: The 50 items of the SILL consist of two constructs the extent to which each item involves in language learning: direct (29 items) and indirect learning strategies (21 items).
Model 3: The 50 items of the SILL consist of six constructs subsumed under direct or indirect learning strategies: memory (9 items), cognitive (14 items), and compensation (6 items) strategies subsumed under direct learning strategies, and metacognitive (9 items), affective (6 items), and social strategies (6 items) subsumed under indirect learning strategies.

Results
The Cronbach's alpha of the constructs of the SILL was investigated in order to measure the internal consistency of the items within each construct, as shown in Table 1.Even though the alpha coefficients of compensation and affective strategies were below an acceptable level of .60, the other strategies were above the cut-off point (Landau & Everitt, 2004).This finding indicates that the items within each construct of the SILL measured in general similar characteristics about language learning strategies.In addition, the correlations among the constructs of the SILL were examined, as shown in Table 2.All the correlations among the constructs of the SILL were significant (p<.01), with the relatively low correlations of compensation and affective strategies with other strategy categories.The significant correlations among the six constructs indicate that six strategy constructs were unique as well as interrelated to each other.
(Insert Table 1 and Table 2) In order to examine whether or not the two-construct or six-construct classification system proposed by Oxford (1990) represents the SILL, three different models were proposed and tested by performing a maximum likelihood CFA.The goodness-of-fit indices used to test the structures of the SILL were chi-square, the root mean square error of approximation (RMSEA), Bentler's Comparative Fit Index (CFI), and Bentler and Bonett's Normed Fit index (NFI).As shown in Table 3, even though the six-construct model improved over the two-construct model which again improved over the one-construct model in the goodness-of-fit, all the fit indices failed to fit the models appropriately.More specifically, all the models showed significant chi-square values, RMSEA values bigger than the cut-off point of .05,CFI and NFI values smaller than the cut-off point of .90(Browne & Cudeck, 1993).This finding indicates that the two-construct or six-construct classification proposed by Oxford (1990) is not sufficient to account for the structures of the SILL.

Discussion and Conclusion
One of the findings of this study was that even though the Cronbach's alpha coefficients of compensation and affective strategies were below an acceptable alpha value of .60, the alpha coefficients of other constructs such as memory, cognitive, metacognitive, social, and total strategies were generally consistent (Hair et al., 1998;Landau & Everitt, 2004).The internal consistency of the SILL lends support to previous studies where the items of the SILL were very consistent, with the Cronbach's alpha coefficients of the total SILL higher than .90 in several studies (Park, 1997;Hsiao & Oxford, 2002;Robson & Midorikawa, 2001;Yang, 1999).This finding proves that the items of the SILL, either in part or in total, measure similar characteristics about language learning strategies.
Another finding of this study was that all the correlations among the constructs were statistically significant, supporting previous studies (Park, 1997;Nisbet et al., 2005).The high correlations among the constructs of the SILL were expected because all the strategy categories were mutually supportive in theory, measuring characteristics about language learning strategies (Oxford, 1990).However, caution should be exercised because when latent constructs are highly correlated to each other, the rationale to classify the SILL into either two or six will be questioned.In addition, the role of language learning strategies in L2 performance will be limited because of multicollinearity in regression analysis, and the goodness-of-fit indices will overreject models to be tested in CFA (Hu & Bentler, 1995;Nisbet et al., 2005).
Finally, the main finding of this study was that all the fit indices used to test the models of the SILL such as chi-square, RMSEA, CFI, and NFI were not appropriate to the data, lending support to a previous study where no models derived from current research on language learning strategies fit the data (Hsiao & Oxford, 2002).This might be due to the larger-than-recommended number (n=200) of the subjects and high correlations among the constructs (Hu & Bentler, 1995;Kline, 2005).Be that as it might, other indices that did not fit the data indicated that the SILL did not represent either the two-construct or six-construct model (Kline, 2005;Schumaker & Lomax, 1996).This finding proves that the structural validity of the two-construct or six-construct model proposed by Oxford (1990) is potentially problematic, and that strategy classification of the SILL should be reinvestigated.Extending this, it deserves to note that current classification systems of language learning strategies have been different among researchers, and that a consensus about strategy classification among researchers should be reached on the base of current L2 acquisition research in order to understand better the nature and role of language learning strategies in L2 acquisition.For instance, Rubin (1981) and Oxford (1990) classified learning strategies into dichotomous direct and indirect strategies, whereas O' Malley and Chamot (1990) classified learning strategies into metacognitive, cognitive, and socio-affective strategies.In addition, Oxford (1990) categorized learning strategies into memory, cognitive, compensation, metacognitive, affective, and social strategies.
The implication of these findings was that the popularly used two-construct or six-construct classification system of the SILL proposed by Oxford (1990) did not fit the data and the classification system should be reinvestigated to understand better the structures of the SILL.In consideration of the fact that the SILL has been one of the most popular questionnaires in the domain of L2 acquisition and teaching, it is a pressing need to continue to reinvestigate the psychometric properties of the instrument including the construct validity.Otherwise, the data collected by the SILL and the results interpreted by analyzing the data will bias the nature of the SILL and mislead studies of language learning strategies in the domain of L2 acquisition and teaching.
Since this study was conducted by Korean EFL university students, generalization of the findings should be made with caution.Accordingly, more studies should be undertaken using participants from different learning contexts to shed further light on whether or not the SILL reflects either the two-construct or six-construct model.In addition, research on strategy classification should be conducted further to understand clearly the nature of language learning strategies, and a consensus should be reached with regard to which classification system best accounts for language learning strategies: two-(direct and indirect strategies), four-(metacognitive, cognitive, social, and affective strategies), or six-classification system (memory, cognitive, compensation, metacognitive, affective, and social strategies).Read the item and choose a response (1 through 5 as below).2. I use new English words in a sentence so I can remember them.
3. I connect the sound of a new English word and an image or picture of the word to help me remember the word.
4. I remember a new English word by making a mental picture of a situation in which the word might be used.
5. I use rhymes to remember new English words.
6.I use flashcards to remember new English words.
7. I physically act out new English words.
8. I review English lessons often.9.I remember new English words or phrases by remembering their location on the page, on the board, or on a street sign.
Part B 10.I say or write new English words several times.
11.I try to talk like native English speakers.
12. I practice the sounds of English.
13.I use the English words I know in different ways.
14.I start conversations in English.
think of relationships between what I already know and new things I learn in English.

Table 1 .
Cronbach's α of the Constructs of the SILL

Table 2 .
Correlations among the Constructs of the SILL

Table 3 .
Goodness-of-Fit Indices for the Models of the FLCAS