Are Learners Aware of Their Strategy Use? Investigating EFL Learners’ Self-Perceived Metacognitive Awareness of Reading Strategies with the MARSI-R

This study investigated EFL learners’ self-perceived metacognitive strategy use with the revised Metacognitive Awareness of Reading Strategies Inventory (MARSI-R). Data collection followed two steps. In the first step, 213 EFL learners responded to the inventory twice (before and after a reading comprehension task). In the second step, 81 out of the 213 participants answered an open survey inquiring learners’ perception and changes of perception of metacognitive reading strategy use during the reading process. Results indicate that learners’ self-perceived strategy use changed significantly after performing the reading task. Learners’ feedbacks in the survey show that learners’ self-perceived strategy use might not reflect the actual use of strategies. This inconsistency between self-perceived reading strategy use and the actual strategy use might mislead learners in their foreign language learning. Further empirical studies are needed to validate the instrument and explore classroom instructions that help learners better understand their actual strategy use during the reading process


Introduction
Being a dynamic process, reading comprehension poses challenges to foreign language learners at different levels. Vocabulary learning, collocation acquisition, various syntactic structures, and inferencing during discourse comprehension, on top of these is the metacognition, monitoring the integration of these activities. Metacognition has long been recognized as an important factor in cognitive activities, pertaining to problem-solving, goal-setting, monitoring and conscious cognitive experiences (Brown, 1978;Flavell, 1979).
Regarding foreign language reading, achieving a high-level proficiency requires flexible selection and application of reading strategies and solving problems during the reading process. Studies investigating the role of metacognitive awareness indicate that it is a critical factor for a reader to be proficient and strategic in either native or foreign language reading (Auerbach, 1997;Carrell, 1991;Sheorey and Mokhtari, 2001;Akhmetova, Imambayeva, and Csapó, 2022). "Poor readers do not possess knowledge of strategies and are often not aware of how or when to apply the knowledge they do have." (Alderson, 2000, p.41). Findings from empirical studies indicate that both native and foreign language learners use reading strategies to help them navigate through the text and mostly metacognition plays a role in setting purposes for reading and self-monitoring (Sheorey and Mokhtari, 2001). This corresponds to the finding that proficient readers are more conscious about how reading strategies are used (Paris, Lipson, and Wixon, 1983) and more proficient language learners possess higher level of metacognitive awareness of reading strategies as compared to less proficient language learners (Han, 2011;Pressley, 1995).
Consensus of metacognition's vital role in reading does not ensure an easy path of tapping into learners' metacognitive functions while reading. In empirical studies, metacognition is mainly recognized as strategies or learners' awareness of strategies (Grabe, 2009, p.53). Regarding reading activities, it is difficult to assess learners' actual strategy use with self-report surveys, questionnaires, or think-aloud methods. Researchers may not get the actual strategy use because learners' judgments of what their abilities and habits and measurements of their performance often do not match (MacNamara, 2011, p.195). This then makes measuring learners' awareness of metacognitive strategies a challenging task.

Literature Review
Despite the daunting task, attempts have been made to design evaluation tools to assess metacognitive awareness and explore how it performs noted functions, such as 'monitoring strategy use' during the reading process. Mokhtari, Dimitrov, and Reichard (2002) and Mokhtari and Reichard (2004) designed a self-report instrument (the metacognitive awareness of reading strategies inventory, MARSI) to examine learners' strategy use, especially when reading academic materials. The MARSI is a three-dimensional model, capturing the dynamic role of metacognition from Global reading strategy (GRS), Problem-solving strategy (PSS), and Support reading strategy (SRS). GRS is a set of items aiming to assess learners' pre-reading strategies, such as setting up the reading purpose, and predicting the key content of the reading material. PSS looks into learners' problem-solving skills while reading, e.g., checking one's understanding when coming across conflicting information, rereading for clearer understanding, etc. SRS refers to learners' capabilities to use support mechanism, such as reference materials, to facilitate their comprehension.
The inventory has been broadly applied to examine its reliability, validity, and its relationship with reading performance. Mixed findings have been reported. Azhar, Awan, and Khalid (2015) examined postgraduate foreign language learners' academic reading and reported that learners with a higher level of awareness of PSS scored higher in the reading comprehension test. Sheikh, Soomro, and Hussain (2019) explored undergraduate students' strategy use and reported the MARSI is a good predictor of learners' reading proficiency. However, Luo, Wu, and Yang (2013) surveyed undergraduate EFL learners reporting that no correlations have been found between the MARSI and learners' reading performance. Regarding the validity of the inventory, Guan, Roehrig, Mason, and Meng (2011) commented that the MARSI might not be appropriate for proficient language learners.
Researchers also investigated the effects of raising learners' metacognitive awareness of reading strategies with the MARSI. Li (2009) found positive correlation between the metacognitive awareness of reading strategies and foreign language academic reading performance. Similarly, Han (2011) reported significant improvement of foreign language reading comprehension after training learners with the MARSI to improve their awareness of metacognitive reading strategies. Thus, providing learners with explicit instruction on metacognitive reading strategies might help improve their academic foreign language reading performance.
The current study is an extension of this inquiry. The study uses a mixed method to probe learners' awareness of their strategy uses. Following research questions are framed: RQ1: What are EFL learners' self-perceived strategy use before and after the reading task? RQ2: Is there a significant difference between EFL learners' self-perceived strategy use before and after the reading task? RQ3: What are EFL learners' reflections on their self-perceived strategy use?

Participants
The study was conducted at a university located in the northeast China. Convenient sampling was used; eight classes of second-year English majors (N=213) participated in the study. Students were informed with the nature of the study and understood that the result would only be used for research purposes. Consents were given for the collection of the data. The age range of the participants was 19-21.

Instrument
The present study used the MARSI-R, which is a revised version the MARSI based on the feedback from the empirical studies with the MARSI (Mokhtari et. al., 2018). The MARSI-R kept three dimensions of reading strategies (GRS, PSS, and SRS) but reduced the items from 30 to 15. Mokhtari et. al. (2018:239) suggest that the MARSI-R is a valid measure for assessing students' metacognitive awareness and perceived use of reading strategies. The validation of the MARSI-R used data from ethnic groups like Caucasian, Hispanic, African American and reported that participants assign the same meanings to the items in the survey.
As the MARSI-R is in English, it was then translated into Chinese by two experienced English teachers and piloted with one class of second year English majors (N=26, age 19-21) to estimate the time needed for the completion of the inventory. Wording was improved to guarantee the clarity and accuracy of the translated version (see Appendix). The reading task contains two passages from the IELTS test with altogether 26 comprehension questions. Participants were given 40 minutes to finish the reading task. Each completion of the MARSI-R took about 5 minutes (including distribution and collection).

Procedures
The first stage was the preparation of the instrument. As it was mentioned in the previous section, the instrument was translated and piloted. Then it was the data collection.
The formal data collection followed two steps and took place in the participants' reading class. The whole process took about 75 minutes, spreading over two sessions of the reading course. Each session was 50 minutes. The first step was about 50 minutes, including 10 minutes for finishing the inventory and 40 minutes for the reading task. In the first step, participants did the MARSI-R first (Trail 1) and immediately performed a reading task, and then participants had a ten-minute break and did the MARSI-R again (Trial 2). In the second step, three out of the eight classes (N=81) were invited to answer two survey questions: "How do you understand the strategies mentioned in the MARSI-R? Are there any changes in your understanding of these strategies before and after the reading task?" Questions were printed on papers and distributed to participants in the last 15 minutes of the reading class. No personal information was required. Participants wrote down their responses in Chinese.
The purpose of the open questions was to explore: 1) Did learners understand the strategies framed in the inventory and what was their understanding regarding reading strategies and how did learners actually use them in the reading task? And 2) would there be any changes in learners understanding of strategies or in their actual usage of strategies before and after the reading task?

Data Coding and Analysis
Data from the MARSI-R was coded and submitted to SPSS 26.0 for analysis. As it was mentioned above, the MARSI-R was done twice by the participants: one before the reading task (report in the following section as Trial 1) and one after the reading task (Trial 2). The reliability analysis was performed separately to obtain the coefficients of Trial 1 and Trial 2 of the MARSI-R. Paired-sample t test was used to examine if there is a significant difference between participants' responses in the Trials 1 and 2 about their self-perceived strategy use. The reading task was marked by two English teachers. Learners' responses to open questions in Step 2 were transcribed and translated by researchers.

Results
Table 1 displays the mean and SD of participants' self-report strategy use from Trials 1 and 2. Table 2 presents the results of paired t-test. Table 3 is the results of paired t-test of each sub-scale. Table 4 reports the reliability coefficients of Trials 1 and 2 of the MARSI-R. Table 5 reports the participants' reading score. M e a n S D N Reading score 21.18 5.52 213

Discussion
RQ1: What are EFL learners' self-perceived strategy use before and after the reading task? To answer this question, descriptive analysis was performed on learners' responses in the Trials 1 and 2 ( Table 1). The reliability of the MARSI-R of the two trials was analysed (Table 4). Descriptive analysis of Trial 1 showed that learners reported high level of metacognitive awareness (> 3.5) of all three sub-scales of reading strategies (GRS 3.67; PSS 3.87; and SRS 3.79). Trial 2 reported high level of PSS (3.69), moderate levels of GRS (3.44) and SRS (3.33). Participants' self-perception of strategy use changed in GRS and SRS. We will discuss this change in RQ 2. Overall, learners reported moderate and high levels of metacognitive awareness of reading strategies before and after the reading task, and this is in line with previous findings (e.g., Luo et. al., 2013;Kusumawardana and Akhiriyah, 2022).
The nature of the current research is to best capture learners' metacognitive awareness of reading strategies. We have noticed that previous studies either place the inventory before the reading task or after the reading task. It was mentioned that "researchers may not get the actual strategy use because learners' judgments of what their abilities and habits and measurements of their performance often do not match" (MacNamara, 2011, p.195). In this case, we asked learners to do the inventory twice and placed the reading task in between. From our data, we see that learners' self-perceived strategy use changed after doing the reading task. It seems that learners thought they were familiar with some strategies, e.g., GRS, but after the reading task some of learners changed their perception and realized that they knew some strategies but could not use them often during the reading activity.
Five items in the sub-category GRS are mainly about how learners plan their reading activity. For example, GRS 1 "Having a purpose in mind when I read." Learners may easily come to a decision of high level of awareness of this item because quite often they would be 'instructed' to do some reading activities. This might leave them an impression that they carry a purpose when reading, but if they were not 'instructed', they may not have a reading purpose. Similarly, the item "Critically analysing and evaluating the information read" received lower levels of metacognitive awareness after the reading task.
We have noticed that learners' responses changed mostly in GRS and SRS. This study was conducted in a foreign language reading course, in which learners were instructed about how to perform reading activities with different types of genres and how to use different types of reading strategies. When participants were asked about the reading strategies, they might recall what they have learned in class. However, when they were engaged in doing a reading task, the strategies were activated and the level of awareness and understanding of strategies and how these strategies should be applied might get changed due to the reading context. Then when participants were asked to do the inventory again immediately after the reading task, their perception of strategies might be different. Thus, the change between Trial 1 and Trial 2 will indicate participants' perception of strategies. Regarding the reading process, at that moment, it might be easy for participants to recall how they used reading strategies or how they understood strategies. Trial 2 of the MARSI-R might help reveal more about how aware learners are about the reading strategies and how they were actually using strategies in accomplishing the reading task. MacNamara (2011) pointed out that learners may not truly understand their abilities and habits; the findings in this study confirm this comment and reveal that the procedure of data collection might be a significant factor influencing the interpretation of the results of the MARSI-R.
Finally, the Cronbach alpha coefficient analysis reported high reliability of the MARSI-R (Trial 1=.738; Trial 2=.883). Mokhtari et.al. (2018) reported Cronbach alpha .850 and Kusumawardana and Akhiriyah (2022) reported .930 respectively. These show that the MARSI-R is a reliable and useful instrument to assess learners' self-perceived strategy use. However, the significant difference between Trial 1 and Trial 2 in this study provides new evidence that when interpreting results, researchers need to consider data collection procedure: is there a reading task implemented, and the sequence of the reading task and the instrument.
RQ2: Is there a significant difference between EFL learners' self-perceived strategy use before-and-after the reading task? Paired t-test was used to investigate if there were any significant differences between Trial 1 and Trial 2 ( Table 2 & Table 3). The analysis results reported a significant difference between Trial 1 and Trial 2 of the MARSI-R (t = 4.725, p <.001). Further analysis of the sub-categories of GRS, PSS, and SRS also reported significant differences between Trial 1 and Trial 2: GRS (t = 4.661, p<.001); PSS (t = 3.412, p <.005); SRS (t = 6.046, p <.001). These show that participants' self-perceived understanding of the strategies changed significantly after performing the reading task. What might be the causes for these significant changes? One possible reason was discussed above that learners may not truly understand their reading abilities and habits, and another possible cause is the single approach of using an inventory to measure learners' awareness of reading strategies (Hannon and Daneman, 2001). MacNamara (2011) also pointed out that that with questionnaires and surveys researchers often fail to get the real picture of how learners actually use strategies. Findings from this study may provide some clues. Learners vary in their metacognition and metalinguistic knowledge. Successful readers demonstrate higher levels of metacognitive knowledge as well as control of their reading; less successful and novice readers show less sophistication in metacognition (Paris, Lipson, and Wixson, 1983). Participants in this study may vary in their English reading proficiency as Table 5 (Reading score) reports an SD of 5.53. In this case, for more advanced language learners, they might have higher levels of awareness of their learning issues, namely strategy use, goal-setting, and self-evaluation. However, for lower-proficient learners, they normally get stuck with their learning issues. Therefore, even though they are provided with clear-listed strategies, feedback they provide may not reflect the real situation. This might also explain why mixed findings have been reported when exploring to what extend the MARSI might predict learners' academic reading performance.
Mixed findings have been reported regarding the relationship between the MARSI and learners' reading performance. Luo et. al. (2013), using the MARSI, reports there is no relationship found between the learners' metacognitive awareness of reading strategies and reading performance. Similarly, Kusumawardana and Akhiriyah (2022) report no correlation between the MARSI-R and reading performance. Guan et. al. (2011) suggest that the MARSI might not be suitable for advanced foreign language learners and call for more theoretical and empirical studies to investigate the measures of assessing metacognitive awareness of reading strategies. Sheikh et. al. (2019) conclude that the MARSI is an effective predictor of learners' reading performance. Azhar et. al. (2015) find that high proficient readers use more problem-solving strategies, proposing that it might be necessary to train learners on certain reading skills to level up their reading proficiency. These studies used different data collection procedures; some placed the reading task prior to the inventory and others vice versa. Findings from the current study indicate that learners' self-perceived strategy use may change after performing the reading task. One recent study exploring the structural validity of the inventory indicated that the MARSI-R is useful to assess learners' metacognitive awareness of reading strategy use, however, the construct of three subscales may not tap into the specific part of strategy use as they are claimed so (Ondé et. al., 2022). Therefore, in empirical studies measuring learners' metacognitive awareness with the MARSI or the MARSI-R, it is necessary to consider the procedure of data collection. This is discussed in detail in RQ3.
RQ3: What are EFL learners' reflections on their self-perceived strategy use? The qualitative investigation by asking three classes of learners (N=81) to respond to two open questions: "How do you understand the strategies mentioned in the MARSI-R? Are there any changes in your understanding of these strategies before and after the reading task?" The responses provided further information about the findings from Trial 1 and Trial 2 and more information about the procedure of data collection for future studies.
Firstly, if the MARSI-R is preformed prior to the reading task, it may not induce the 'true' picture of how elt.ccsenet.org English Language Teaching Vol. 16, No. 7; learners use the strategies during reading comprehension because learners themselves may not be aware of their strategy use. For example, These responses revealed that learners may not have a clear and accurate understanding of the strategies they use and how they use them during the reading tasks. This explains why the empirical studies with the MARSI or the MARSI-R exploring either the construct validity and the relationship between the strategy use and reading performance have yielded mixed findings. Different data collection procedure, though with the same inventory, would induce different perception of strategy use.
Secondly, learners' self-perceived strategy use may be influenced by their proficiency level. For example, Zhang: "I will use familiar strategies, but when I am under pressure or when the reading task is too difficult for me, I will not use these strategies." Fang: "I am not familiar with technology topics, so I find the passage difficult to understand. And when I am short of time, I will not use strategies." These responses revealed that when learners face reading tasks of high difficulty levels or when they work under pressure, it seems that learners have little free cognitive resource to help them monitor their reading process or the employment of different types of reading strategies. Thus, they felt they "did not use the strategies" or they "could not use the strategies they learned." Therefore, when using the MARSI-R to evaluate learners' strategy use, researchers and educators need also consider learners' proficiency level and the context in which learners are evaluated. These reflections explained the findings from the paired t-test: Learners view their strategy use differently prior to and after the reading task. It is evident that learners' strategy use might be influenced by factors like the difficulty level of reading task, learners' emotional state (if they are under pressure or anxious), and the sequence of the reading task and the inventory. Alderson (2000) mentioned that the challenge for assessment specialists is how to develop an assessment format around a text or series of short texts that can be used for a multiple set of tasks to tap reading-strategy uses (Grabe, 2009:370) By implementing the inventory twice (pre and post a reading task), this study provides new perspectives for measuring learners' metacognitive awareness of reading strategies. In data collection, it is necessary to consider the sequence of the inventory and the reading task.

Conclusion
This study used a Trial 1 -the reading task -Trial 2 framework to investigate EFL learners' metacognitive awareness of reading strategies. The results show that the MARSI-R is of high reliability, but the responses may not reflect true status quo of strategy use of learners due to three factors. The first lies in learner themselves. They may not be fully aware of their strategy use during the reading process. Secondly, the sequence of data collection when involving a reading task may affect the result of self-report strategy use. Learners are more aware of their strategy use after the reading task, thus implementing the inventory after the reading task might provide more valid information. Finally, the difficulty level of the reading task might influence the result of the inventory. When learners' cognitive resources, such as memory and attention, are fully occupied by the difficult reading task, they may not be able to apply reading strategies or monitor their reading process. Also, it is evident that implementing the MARSI-R helps learners reflect on their own strategy use. Due to the recognized crucial role of metacognition in the reading process, investigations on the pedagogical implications of the MARSI-R and validation of the MARSI-R with different cultural populations and different foreign language proficiency levels are needed.
This research, however, is not without limitations. More participants should be included in the second stage to explore learners' perceptions of strategy use and causes for the change of learners' perception regarding strategy use. In the future research, the difficulty level of the reading task might be considered as a possible variable, and more empirical studies are needed to validate the inventory.

Funding
This research is partially funded by Xi'an International Studies University (20XWYJGB18) to the second author.

Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.