Measuring Critical Thinking in Science : Systematic Review

The review aims to explore possible methods for critical thinking assessment in science from previous studies. For a long time, critical thinking has been among most talked topics among researchers and academicians, due to its nature in improving one‟s quality of life such as becoming an effective problem solver and logical thinkers. In this study, literature search for related studies was conducted through online databases, The Educational Resource Information Centre (ERIC) dated from the year 2010 till 2017 using keywords such as critical thinking, science, science education and measurement. Only refereed/peer-reviewed journals that fulfilled criteria needed were selected for the study with the findings from web-based service providers, including Sage Journals, Springer, Taylor & Francis, Science Direct, and Wiley Online Library. The findings were analyzed using document analysis technique to answer research questions of this study. This systematic review reveals that critical thinking can be assessed using quantitative or qualitative methods depending on the scope and dimensions of the research. Although there are studies on critical thinking in science, the assessment tools used are instrumented for critical thinking in general setting which focuses in general context. However, when it comes to assessing critical thinking in science secondary school/high school, the findings were limited.


Introduction
Critical thinking is not a new concept, especially in education.There are many reports on incorporation of critical thinking into teaching and learning.For example, researchers focused on instructional strategies for fostering students" critical thinking such as inquiry-based learning (Sriarunrasmeea, Suwannatthachote, & Dachakupt, 2015), collaborative learning (Abubakar & Arshad, 2015), and activity-based cooperative learning (Valdez et al., 2015).
Critical thinking is a mental process, strategies, and representations people use to solve problems, decisions and learn new concepts (Stenrberg, 1986, p. 2).In addition, according to Facione (1990), critical thinking is purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as an explanation of the evidential, conceptual, methodological, criteriological, or the contextual considerations judgment.
Despite broad descriptions of critical thinking (Sternberg, 1986;Lai, 2011), researchers agreed on the specific abilities of critical thinking, that critical thinking is a mode of thinking using various cognitive skills to solve problems, or to reach certain conclusions, with one of well-known critical thinking skills construct is from Facione"s Delphi report.Accordingly, Facione (1990) listed six cognitive skills in the Delphi Report: interpretation, analysis, evaluation, inference, explanation, and self-regulation.Halpern (2007) stated that critical thinking is a higher order thinking skill in a complex manner as it used those cognitive skills such as analysis and synthesis in order to produce a desirable outcome, solving problems and making decisions.Thinking defined by Bruno (1986) as a mental activity by utilizing perceptions, concepts, symbols and image to solve problems and make decisions.In short, critical thinking is a form of mental activity that requires subsets of cognitive skills in order to solve problems and make decisions.approaches namely general, infusion, immersion, and mixed on critical thinking skills.These four approaches are based on Ennis"s (1989) critical thinking typology of four courses; in general approach, critical thinking skills and objectives are the main learning objectives without specific content of subject matter; infusion and immersion are content-focused but when critical thinking is made explicit, it is called as an infusion approach whereas critical thinking in immersion approach is as a by-product of the course; and lastly mixed approach in which general and infusion/immersion approaches are combined.Summary of the approaches is shown in Table 1.According to Abrami et al. (2008), the mixed approach showed the largest effect, whereas moderate effect is found on the general approach and infusion approach, meanwhile immersion approach showed the least effect on students" critical thinking development.These findings showed that instructional strategies do enhance students" critical thinking, and it is important to note that critical thinking should be taught as general courses and at the same time the content is applied in the course to get maximum effect.Also, it is interesting to note that both mixed approach and infusion approach which produced the largest and moderate effect, respectively, are content sensitive, thus attest the importance of knowledge in critical thinking.
Measuring critical thinking skills allows teacher to assess students" performance on a task (Bissell & Lemons, 2006).One way to measure critical thinking skills is by using validated and reliable instrument that focus solely on the cognitive aspects of critical thinking.There are various critical thinking tests; e.g The California Critical Thinking Skills Test (CCTST), Watson-Glaser Critical Thinking Appraisal (WGCTA), and Halpern"s Critical Thinking Assessment (HCTA).The tests measure cognitive skills such as analysis, evaluation of argument, argument analysis, respectively.These tests focus on test taker"s critical thinking skills in general situation, which requires no connection to science concepts.Meanwhile, critical thinking within subject-domain (science) enables a person to think, evaluate, and solve problems in a scientific ways (Santos, 2017), thus suitable assessment that assess students" understanding on the content together with the skills is necessary to maximize the learning outcome.
Over the year, there has been a strong emphasis on designing instructional strategies that can foster students" critical thinking skills, using critical thinking skills assessments comprises of domain-general critical thinking (Chaipichit, Jantharajit, & Chookhampaeng, 2015;Chattuchai, Singseewo, & Suksrinarm, 2015;Alpaslan et al., 2016) or domain-specific critical thinking (Yu, Lin, & Chang, 2017).Therefore, the purpose of this article is to investigate evidence and to provide comprehensive information in the critical thinking skills assessment as in science fields in secondary school/high school.This systematic review was conducted to answer research questions as follows: 1) What are the aims and focus of the previous studies?
2) What are the types of study, data gathering methods and critical thinking skills assessed in the previous studies?

Data Collection and Analysis
A systematic review of the existing literature on critical thinking skills assessments in science was conducted.
The Educational Resource Information Centre (ERIC) was used as the main database to search for suitable published articles in peer-reviewed/refereed journals, with among the web-based service providers were Sage Journals, Springer, Taylor & Francis, Science Direct, and Wiley Online Library.Other relevant studies that can be used as additional information was searched using Google Scholar.The keywords literature search was as follows: critical thinking; assessment; performance; science; science education.During the initial stage, there were no limitations on the related studies which resulted in 1095 papers, so in the next stage, the number of findings had to be reduced, which fulfilled these criteria: (1) the selected paper must be published after 2010 and above to get a recent overview of critical thinking skills assessment, (2) critical thinking skills only, (3) secondary school/high school students, (4) science and science-related context.In this study, concept paper, meta-analysis paper, and systematic review paper were excluded since these papers have no relevance with the study.Summary of the inclusion and exclusion criteria are tabulated as shown in Table 2. Based on these criteria, 69 research-based articles were retrieved.After analysing these articles thoroughly using document analysis technique, only 10 papers met the criteria in answering the research questions of the study.
The flow of the article selection process is shown in Figure 1.The content of the findings was read thoroughly and the information extracted from each study are author/s, aim of the study, subject focus of the study, types of the study, assessment tools and critical thinking skills that is being assessed, and summarized as shown in Table 3.

Results and Discussion
The main aim of this study was to obtain information on critical thinking skills assessment in science and science-related subjects that has been done by previous researchers in the last seven years.

What are the aims and focus of the previous study?
From the summary in Table 2, four studies out of ten focused on science, meanwhile three studies were conducted in chemistry including chemistry laboratory setting.While the rest of the studies were focused on biology (n=1), physics (n=1), and mechanical design for high school students (n=1).
What are the types of study, data gathering methods and critical thinking skills assessed in the previous studies?
As shown in Table 3, assessment of critical thinking skills can be carried out through quantitative or qualitative method.Six studies used qualitative approach; meanwhile four studies were conducted quantitatively.The methods of the studies were based on the assessment tool for measuring critical thinking.For the studies using qualitative approach, the critical thinking components were based on the researcher"s conceptions on critical thinking skills, hence the variety of critical thinking components such as building hypothesis (McDonald, 2012;Katchevich, Hofstein & Mamlok-Naaman, 2013), claim identification and validity of evidence (Terry, 2012), analysis and evaluation (Abubakar & Arshad, 2015;Belova & Eilks, 2015).Meanwhile, for the study conducted by Oliveras, Marquez & Sanmarti (2013), the researchers listed element of science critical reading as follow: purpose, point of view, assumptions, and questions, of the writer, which the respondent was required to deliberate during the session.Despite having a different conception of "usual" critical thinking defined in the Delphi Report (Facione, 1990), those elements can be categorized as one of the cognitive skills of critical thinking; the skill is analysis, since the respondent must ponder the meaning behind the text (analysing the writer"s intention behind the text).To determine the students" critical thinking skills, students" laboratory report were analysed using grading rubric, and finding from interview and observations were also analysed.
For the studies using quantitative approach, the critical thinking assessments were conducted using available critical thinking tests; multiple choice questionnaires as follows: Watson-Glaser Critical Thinking Appraisal (WGCTA) and Cornell Critical Thinking Test Level X (CCTT), to determine effectiveness of intervention on students" critical thinking (Chaipichit, Jantharajit & Chookhampaeng, 2015;Chattuchai, Singseewo & Suksrinarm, 2015).Critical thinking components of WGCTA and CCTT are listed in Table 2. Likewise, Yu, Lin and Chang (2017) developed Mechanical Critical Thinking Scale (MCTS) adapted critical thinking components based on Yeh, Yeh and Hsieh (2000) which measures students" recognition of assumption, induction, deduction, interpretation, and evaluation of arguments in mechanical design for high school students.Meanwhile, Alpaslan et al ( 2016) conducted a study using the Motivated Strategies for Learning Questionnaire, with critical thinking as one of the items assessed in the instrument.Another qualitative study using teacher observations during students" discussion session on natural cosmetic advertisement as reported by Belova and Eilks (2015) showed that the students able to come up with the strategies and types of advertisement; however, when it came to discussing in scientific perspectives, the students had difficulties in analysing and evaluating the advertisements.
Discussing the advertisements in everyday life context and non-science subject might be easier for students because they have "heard" it, whereas scientific perspectives must be accompanied with scientific knowledge as knowledge is a pre-conditional for critical thinking (Bailin et al. 1999).Students must be able to know "what" to analyse and evaluate to be able to come up with sound arguments, analysis and evaluation of claims, evidence and examples.This is because the ability to think critically is depending on in-depth knowledge (content) and performance in various thinking tasks of a particular domain (critical thinking skills) (Davies, 2013).From the summary in Table 3, it can be concluded that there are attempts on assessing critical thinking skill, but they are not enough in the sense that they have not been sufficiently accompanied by valid and reliable measures of critical thinking, as those critical thinking assessments are diverse in terms of dimensions and scope (content-specific or general).Thus, it can be concluded that there is a lack of studies on assessing critical thinking skills in specific science-domains, which is as important as assessing critical thinking skills in general setting.

Limitation
This review focused only on critical thinking skills, in secondary school/high school setting in science.Thus, the acquired data were relatively small.Due to small numbers of the studies and only refereed articles were selected, the findings might be less accurate.Hence, the results of the study can only provide a suggestion rather than a generalized idea about the previous research on critical thinking.

Conclusion
From the literature search, there are growing numbers of critical thinking skills assessments in science and science-related subjects, albeit limited, which shows the need to have a critical thinking skills assessment in a specific-domain.The specific-domain of critical thinking skills is important, especially in science because scientists will always ask questions about the reasons that certain phenomena happened, and can only be achieved when one starts to ask "why".By focusing specific domain critical thinking skills in science, it will help students to analyze and evaluate phenomena using scientific perspective.In addition, there is a need for suitable critical thinking skills assessment, as students" strength and weakness in this particular area can be determined and improved using reliable measurement tools.

Figure 1 .
Figure 1.Flow Chart of the Article Selection Process

Table 2 .
Inclusion and exclusion criteria

Table 3 .
Summary of Findings