Test-taking Strategies of Arab Efl Learners on Multiple Choice Tests

Many studies have focused on the function of learners' strategies in a variety of EFL domains. However, research on test-taking strategies (TTSs) has been limited, even though such strategies might influence test scores and, as a result, test validity. Motivated by this fact and in light of our own experience as EFL test-makers, this article will discuss TTS in light of the related literature in the Arab world intending to show that Arab EFL learners use different TTS strategies to improve their test performance on multiple choice test items. The paper then concludes by showing some interesting strategic behaviours exhibited by Arab EFL learners.


Introduction
Tests have become a prevailing tool for decision-making in our competitive society, and individuals are evaluated according to their achievements on them.Language test results, then, may play a considerable role in the test-takers' lives, helping to determine whether they are admitted to their academic program of choice or are offered the job of their dreams (Cohen, forthcoming, a).As a result, how to perform better on tests has become a major concern for students and teachers.Tests require learners to read directions, instructions, and questions, work independently, and write correct responses (Amer, 1993, p.71).Mastery of these skills is vital for performing well on tests.
In analysing test results, most teachers focus on students' scores.However, it is a fact that students employ certain skills, called "test-taking strategies", when taking tests to improve their performance and receive higher scores.According to Cohen (1998, p.90), since the late 1970s, interest has slowly grown in approaching second language testing from the point of view of the strategies used by test-takers while taking tests.These strategies jeopardise the validity and reliability of tests.This in turn influences the process of decision-making, which is reliant upon testing because test validity requires attention to how the test-takers arrive at their answers.Evidently, when taking tests, a person is tested on two things: his or her knowledge about the subject and his or her knowledge about taking a test.Test-taking strategies address the latter.
Due to the lack of studies focusing specifically on how Arab EFL learners behave and use TTS, this study will attempt to examine several of the primary studies in the Arab context.

What is meant by Test-Taking Strategies?
Cohen (forthcoming, a) defines test-taking strategies (TTS) as the consciously selected processes that test-takers use for addressing language issues and the item-response demands in the test-taking tasks at hand.This definition has two important elements: first, the element of consciousness, a "consciously selected processes"; and second, the element of goal-orientation, "used for dealing with language issues and the item-response demands".Next, we discuss these two elements in turn.

Consciousness
Awareness of TTS is one of the debated issues in the TTS-related literature.As is evident from the above definition of TTS, Cohen adopts the idea of consciousness, and as he once put it: "if the learners cannot identify any strategy associated with it as it is unconscious, then the behavior would be referred to as a common process, not a strategy" (Cohen, 1998).Cited in Phakitwe (2003, p.28), Faerch andKasper (1983) argue that once learners develop some strategies to the point that they become automatic, those strategies may be subconscious.Ellis (1994) argues that if strategies become so automatic such that learners are no longer conscious of employing them, they cannot be accessible for description and lose their significance as strategies.
Based on this argument, we believe that among individuals, the awareness of strategy-use may vary.As test-takers, we recall that we used a number of TTS without being aware that we were following a technique and without recognising that these techniques are called "strategies".This is because the notion of strategies or skills is not properly introduced or taught explicitly in the Saudi context.Therefore, we believe that consciousness regarding strategy-use exists in a hierarchy.The investigations in Al Fraidan (2011) and Addamegh (2003) on TTS in the Saudi context reinforce this.These authors provide examples of how some students processed some vocabulary test items automatically, but others used certain strategies.Supporting this, Phakitwe (2003, p.29) claims that the strategies identified may become processes for some individuals but remain strategies for others.

Goal-orientation
Any behaviour without obvious and intended purpose is meaningless; TTS are no exception.In testing situations, test-takers use different sets of strategies with different intentions.One primary purpose of test-taking strategies is to improve one's performance on exams.In the literature, the goals of TTS have two components: facilitation (strategies that facilitate a given process, which occurs more or less without difficulty), and problem-solving (strategies that play a role when a problem comes into sight).However, neither strategy is guaranteed to be successful.For example, L1 translation can be both facilitating and problem solving, but it does not necessarily lead to success in achieving the correct answer.Al Fraidan (forthcoming) details different reasons for using certain TTS by some EFL Saudi university learners, showing that there are certain goals and reasons for selecting specific strategies.

Cognitive vs. Metacognitive Strategies
An outstanding issue in the TTS literature is the distinction between cognitive and metacognitive strategies.The first noteworthy study to look at this relationship was conducted by Purpura (1999).Cited in Phakitwe (2008, p.239), Purpura (1999) examined the relationship between cognitive and metacognitive strategy-use and language-test performance.The findings showed that cognitive processing was a multidimensional construct consisting of a set of comprehension, memory, and retrieval strategies.Metacognitive strategy use was found to be a one-dimensional construct consisting of a single set of assessment processes (e.g., goal-setting, planning, monitoring, self-evaluating, and self-testing).
According to Phakitwe (2003, p.29), the basic concept of metacognition is the notion of thinking about thinking.For example, we think translation in a reading comprehension test is a cognitive strategy, whereas deciding whether to translate or not (and whether it is successful or not) is a metacognitive strategy.Therefore, metacognition observes, directs, and adjusts translation as a cognitive strategy to achieve the goal of reaching correct answers.To summarise, based on the above discussion, it is evident that metacognition has significant, direct effects on cognitive processes, while cognitive processes have a direct effect on test performance.Therefore, the effect of cognitive strategies on performance is interwoven with, and facilitated by, metacognitive strategies.

Methods for Investigating Strategies
Following the above definition of test-taking strategies, it is apparent that TTSs entail cognitive processes that are not conscious and thus easily available for objective inspection.In the TTS-related literature, verbal reports have become the primary tool for collecting data on what test-takers are doing.Verbal reports provide data reflecting one or more of the following approaches (Cohen, forthcoming, a): 1) Self-reporting: learners' descriptions of what they do, characterised by generalised statements about their TTS.As an example: "On multiple-choice items, WE scan the reading passage for possible surface matches between information in the text and that same information appearing in one of the alternatives."Another example is: "On multiple-choice items WE stop reading distracters when WE find what WE judge to be a correct answer." 2) Self-observation: the inspection of specific, contextualised language behaviour, either introspectively (within 20 seconds of the mental event) or retrospectively.For instance: "What WE just did was to skim through the reading passage for possible surface matches between information in the text and that same information appearing in one of the alternative choices." 3) Self-revelation: "thinking-aloud", stream-of-consciousness disclosure of thought processes while the information is being attended to.For example: "Hmm...WE wonder if the information in one of these alternative choices also appears in the text." As evidence of the above-mentioned approaches of verbal reports and especially the introspective methods, we refer to a study by Nevo (1989), which examined the possibility of obtaining immediate feedback from the respondents on strategy use.The findings show that it was possible to obtain such feedback from subjects just after answering each item when a strategy checklist accompanied the test.A variety of data was obtained through introspective and retrospective written reports.This method of data collection had two important features: first, introspective reporting allowed researchers to access the minds of the respondents within seconds of the task performance and to extract authentic information, which could not be collected by other means.This supports the argument of Cohen (1984) that if respondents are asked to indicate the strategies they use at the end of the test or even after one of its segments, they might have forgotten the processes they went through, the means used to choose an answer, and other underlying considerations.Second, the introspective and retrospective verbal reports made respondents aware of the strategies used during test-taking.The subjects stated that this was the first time that they were aware of what they did when taking a test, and they gained a consciousness of what takes place in their minds while taking reading comprehension tests.
Cohen (under contract) mentions a new trend in the field of TTS research and argues that "while verbal reports continue to play a key role in TTS research, there have been changes in procedures for conducting such verbal reports aimed at improving the reliability and validity of the results."One has been to provide a model for test-takers as to the types of responses that are considered appropriate, rather than to let them respond however they wish, which has often failed to produce sufficient data.Researchers now may ask probing questions to make sure, for instance, that the respondents indicate not just their rationale for selecting "b" as the correct alternative on a multiple-choice question but also their reasons for rejecting "a", "c", and "d".

Advantages/Disadvantages of TTS
Following this review of the literature on test-taking, it is necessary to conclude with a brief discussion about the advantages and disadvantages of TTS.We think that there are two main advantages of using TTS if they indeed are part of the construct.First, the primary advantage of TTS is an improvement in test-scores.A second, but related, advantage is a reduction of exam-related anxiety and a reinforcement of students' confidence about exams.
In addition to the advantages of using TTS, there are also advantages for investigating them.First, once test constructors have knowledge of test-takers' strategies, they can make more informed choices in the construction of test items, including choices that, ideally, will better assess the test-takers' language skills, rather than their cleverness at circumventing an assessment of these skills (Cohen, under contract).This can provide insights into the behaviours used for the sake of passing tests (Cohen, forthcoming, b) and can help teachers to make test items that cannot be responded to easily by means of test-taking strategies aimed at bypassing a more informed means of producing responses.Second, collecting TTS data helps to determine the extent to which performance on a given assessment measure is reflective of L2 knowledge and performance in the area assessed.Moreover, if we investigate TTS and find that respondents did not use construct-relevant strategies, this could be attributed to several reasons:  It may be because the test items are of questionable validity and need revision.If a test-taker has a legitimate reason for answering an item incorrectly, then the item needs to be rewritten.


It may be because the test takers lack knowledge regarding how to respond to such items and need clearer instructions or a better orientation prior to the test.


It may be because the students lack proficiency or have other characteristics that work to their detriment (e.g., socio-cultural background, gender, age, occupational status) and would be better served by more appropriate items.
(Adapted from Cohen (forthcoming, b)) In addition to being advantageous, TTS can be disadvantageous as well.Test-wiseness, or TW (see below), can be a source of test invalidity.Test-takers vary in their knowledge and use of TW principles, and unless this is controlled for, some candidates will have unfair advantages over others.According to Houston (2005, p.2), previous research has found evidence that test-wiseness may be a source of additional variance in test scores and a factor that may lower test validity.As Baker et al. (1966, p.13) claim that test-wise examinees can be expected to obtain higher scores than equally competent examinees who lack test-wiseness.

Conceptualisation of Test-taking Strategies
According to Cohen (forthcoming, a), TTSs include three types of strategies: 1) Learner strategies -the ways that test-takers operationalise the skills of listening, speaking, reading, and writing, and the related skills of vocabulary, grammar, and translation (Cohen, forthcoming, a).For example, in summarising reading passages, a learner strategy may be to distinguish key points from non-key ones.
2) Test-management strategies -strategies for responding meaningfully to test items that are consciously selected to assist in producing a correct answer (Cohen, forthcoming, b).These strategies include logistic issues, such as keeping track of time and determining where to look for answers (Cohen, under contract).For example, a strategy for dealing with multiple-choice items can be going back to the question for clarification, re-reading it, or paraphrasing it to verify or confirm understanding of the question.
3) Test-wiseness strategies (TW) -strategies for using knowledge of test formats and other peripheral information to answer test items without going through the expected linguistic and cognitive processes (Cohen, forthcoming, a).With regard to a reading test, these strategies may involve using the process of elimination rather blindly (i.e., selecting an option without understanding it at all, but out of a vague sense that the other options are unlikely to be correct), selecting an option because it appears to have a word or phrase from the passage in it (possibly a key word), and using clues in other items to answer an item under consideration.
A TW-related study was constructed by Addamegh (2003) to gain insight into test-takers' mental processes when taking an EFL multiple-choice vocabulary test designed for Saudi English major university students.The analysis shown sixty-two different TTSs of six types: managing the test as a whole, reading the stem, handling the gap, examining the alternatives, selecting a response, and implementing strategies after choosing an answer.It was found that approximately 40% of test items testing an unknown lexical item were answered correctly based on non-linguistic strategies that could be called TW strategies.He also found surface matching between the stem and the choices, choosing an answer because it feels English, and other types of TW strategies.The high percentage of TW strategy use in his test indicates the problematic construction of the two tests he used, even though one was developed professionally and the other was carefully developed by a teacher.
Al Fraidan (2011) examined the TTSs of EFL Saudi university level learners majoring in English on two vocabulary tests claimed to be widely used in the Saudi context (a cloze test and a multiple choice gap-filling sentence-based test where there are a pool of alternatives to choose from for all the sentences).He found fewer TW strategies than Addamegh (2003) and concluded the following: 1) The classic multiple choice question is the test item that is most vulnerable to TW 2) Varying test types in a single test can reduce TW.
Out of more than 90 TTS strategies, he found there were few TW strategies used (such as guessing and relying on different knowledge sources) in addition to the strategies that were anticipated by the test maker.
As a final remark regarding TW, guessing can be used differently.Al Fraidan (2011) noted that guessing is not completely blind and is based on some partial knowledge of the answer or the construct.Al-Ghamdwe and Davies (1996) conducted a study to investigate whether guessing strategies can be used effectively in responding to multiple-choice vocabulary tests by Saudi EFL learners at King Faisal Air Academy.Five cadets were asked to provide think-aloud protocols as they worked through eighteen vocabulary items in one-sentence contexts.
Their conclusion was that guessing may be an effective strategy, but individual personality factors appear to control its frequency of use.This may well be related to a general feeling in Saudi society that guessing entails the risk of a loss of face and error.Risk-taking may be affected by some psychological or social discomfort, which might need to be reduced to encourage cadets to take a more active role in test-taking situations.
A related study in the Arab context was conducted by Al-Hamly & Coombe (2005) to investigate whether the practice of answer-changing on multiple-choice questions is beneficial to Gulf Arab students' performance.The authors found that 44% of answer changes were from incorrect to correct, and 37% of changes were from incorrect to incorrect.The lowest percentage was the correct to incorrect category, which accounted for 19% of answer changes.The authors found that the lower the score, the more incorrect to incorrect answer changes were made, which could be due to the possibility that students were guessing or not taking the test seriously.Although students are always influenced by traditional wisdom, such as "go with your first response", the findings suggested encouraging students to change answers judiciously after scrutinising their original answers for more plausible alternatives.

Some Behaviours and Strategies
By looking at the previous studies that directly evaluated TTS in relation to Arab learners, we find some interesting behaviours exhibited by learners: 1) Attempting the test in no particular order (Al Fraidan 2011, Addamegh 2005).One would expect that students would answer the test items in order or in some type of sequence.
2) Demonstrating metacognitive strategies (Al Fraidan 2011, Addamegh 2005), which confirms previous findings that learners use metacognitive strategies (Pakhitwe 2003). 3 ) Changing an answer (Al Fraidan, 2011, Addamegh 2005, Al Hamly & Coombe (2005), which is a frequent behaviour of learners in general, but one would like to know the reasons for this behaviour.Al Fraidan (forthcoming) indicates that an answer change could be made for stylistic reasons or as a result of reading further and discovering the unsuitability of the provided answer (Al Fraidan forthcoming).

Conclusion
To summarise, the goal of this article was to describe data on test-taking strategies emerging from different studies on multiple choice test items by EFL Arab learners.Educators and especially test makers should be aware of recent trends in TTS so they can devise good tests and safeguards against undesirable behaviour such as using TW strategies.For many years, it has been recognised that students employ certain skills, called "test-taking strategies" when taking tests to improve their performance and receive higher scores.According to Cohen (forthcoming, a), concern for TTS involves three types of strategies: learner strategies, test-management strategies, and test-wiseness strategies.The role of TTS should be stressed in the Arab and especially the Saudi context because it appears to be a neglected area, but we have shown here the importance of being aware of TTS in designing valid tests.It is best not to assume that any test-taking strategy is a good or poor choice: the strategy might be successful for one test-taker, but it may not work well for another respondent.To obtain information about the process that test-takers undergo, verbal reports have become the primary tool for collecting data on TTS.Finally, individual differences in TW tend to reduce the validity of test scores because scores reflect test-taking skills in addition to knowledge of the subject matter being tested, thereby reducing the validity of the test.