Effects of Example-Problem Pairs on Students’ Mathematics Achievements: A Mixed-Method Study

The aim of this research is to investigate how the utilisation of example-problem pairs affects the outcomes of mathematics students when compared to conventional teaching methods. Thus, a mixed method embedded design, with a main emphasis on a quasi-experiment with supplemental field notes, was conducted with 64 second intermediate grade school students (eighth grade). Participants were divided into two groups comprising 33 students in the experimental group, and 31 students in the control group. An ACNOVA test revealed that the average scores of achievement of the students taught using the example-problem pairs were higher than the average scores of the students who were taught using conventional teaching methods, with a very large effect size. Moreover, the qualitative findings revealed that the students taught using example-problem pairs were more engaged and took more responsibility for their learning than the students who were taught using conventional teaching methods. In addition, the students who lacked the necessary prerequisite knowledge needed more support than the higher achieving students. The implications of the study were discussed.

This article begins by a discussion about the theoretical framework for learning form example-problem pairs. This is followed by assessing the previous studies relevant to example-problem pairs. A quasi-experiment implemented in this study will be discussed in light of the results of the field notes conducted by the author during the intervention.

Theatrical Framework
When learners are studying the examples they should obtain a basic understanding of the domain principles. This provides a source for later problem solving (Renkl, 2014) and is a very good prerequisite for knowledge transfer (Renkl, 2017). Transfer is accomplished when both the examples and abstract principles are delivered first (they are presented as sources of transfer). They should be encoded in the learner's long-term memory and then a schema containing an abstract principle is probably constructed. Later on, when the learner encounters a problem, their prior knowledge should be activated which prompting them to select relevant examples for mapping the problem to the relevant example. When the learner meets a transfer problem, potentially relevant analogical examples (already encoded in their memory) are activated and selected. The learner then maps the transfer problem to the relevant examples (Holyoak, 2012). Schema abstraction can occur when worked examples and problem solving alternate with each other. Renkl, (2014) reviewed the literature and concluded that schema abstraction can occur by analogical reasoning through two stages. First, an initial schema is probably constructed when several examples are encoded. Second, when the knowledge obtained from studying the examples is applied to a new problem.
In contrast to learning relying on worked examples, learning relying on problem solving only could generate a high cognitive load. Consequently, the vast amount of working memory resources is dedicated to solving the problem, rather than obtaining new knowledge about the domain (Kirschner, Sweller, & Clark, 2006;Sweller, Ayres, & Kalyug, 2011). Some researchers found that limited capacity may lead to relying on surface features rather than structural features (Waltz, Lau, Grewal, & Holyoak, 2000). It forces novices to rely on weak problem-solving strategies such as means-ends analysis, and contribute very little to their learning (Sweller, 1988). Conversely, however, learning via worked examples presents learners with the steps of the worked-out solution. Thus, learners can devote all their available cognitive capacity to studying the given solution and constructing a schema for solving late problems Sweller, van Merriënboer, & Paas, 1998;van Loon-Hillen, Van Gog, & Brand-Gruwel, 2012). In other words, the effectiveness of worked examples occurred when learners fundamentally rely on already-familiar examples to solve problems by using analogical reasoning (AR) (Holyoak, 2005;Holyoak & Cheng, 2011). Therefore, the acquisition of effective problem-solving schemas and knowledge transfer required the reduction of working memory demands (Richland, Stigler, & Holyoak, 2012). In addition to this, the learner should also be self-explaining during learning worked example processes for the construct abstract schemas.
Several researchers found that the effectiveness of worked example depends on the self-explanation activities of the learner (Atkinson, Renkl, & Merrill, 2003;Hausmann & VanLehn, 2008;Renkl, 1997;Renkl, 2002;Rittle-Johnson, Loehr, & Durkin, 2017). For example, Renkl (1997) found that many learners do not self-explain; hence, they do not obtain abstract schemas. These deficits might be due to a lack of prerequisite knowledge or learner passivity (Renkl, 2014). Chi, Bassok, Lewis, Reimann, and Glaser, (1989) found that successful learners explained examples more actively to themselves and studied them longer and tried to make sense of the rationale behind the solution procedures.

Previous Studies
Previously, numerous studies have been conducted to examine the effects of example-problem pairs on student outcomes. Within this scope, the results of many research studies reveal that example-problem pairs instruction is more effective for transfer and learning than instruction using problem solving only (Carroll, 1994;Cooper & Sweller, 1987;Coppens, et al., 2019;Kalyuga et al., 2001;Mwangi & Sweller, 1998;Nievelstein et al., 2013;Rourke & Sweller, 2009;Schwonke et al., 2009;Sweller & Cooper, 1985). For example, the initial research of worked examples in the field of mathematics for intermediate school students conducted by Sweller and Cooper (1985) found that students could solve similar problems (identical in structure to the initial examples) more rapidly and with fewer mathematical errors compared to using conventional problem solving strategies. They asked students to solve a similar problem immediately after the example study because they believed this strategy would allow for more activity and motivation than studying another example. Since then, it was believed that coupling examples with problems is more effective than presenting learners with sets of examples only (Manson & Ayres, 2019;Trafton & Reiser, 1993).
In contrast, few studies have examined the use of problem-example pairs. Some scholars argue that when a learner first experiences discrepancies whilst working through problem solving, they could then become more motivated to study the example and may pay more attention to the steps that they could not solve (e.g. Hausmann, Van de Sande, & VanLehn, 2008;Reisslein, Atkinson, Seeling, & Reisslein, 2006). Some researchers found this strategy to be most beneficial for students with greater prior knowledge, whilst those with less prior knowledge benefited most from example-problem pairs (e.g. Kalyug, Ayres, Chandler, & Sweller, 2003;Paas, 1992;Reisslein et al., 2006). Furthermore, research which compares the effects of worked examples only to problem solving only is sparse; the findings of which determines the worked examples to be more efficient in terms of mental effort investment, as well as being more effective for learning and knowledge transfer (Van Gerven, Paas, Van Merriënboer, & Schmidt, 2002;Van Gog, Paas, & Van Merriënboer, 2006). Van Gog, Kester, and Paas (2011) recently found that there are no differences between using worked examples or example-problem pairs on students' learning outcomes.
In addition, research has shown that instruction reliant on worked examples is most beneficial for learners with less prior knowledge; whereas instruction that relies on problem solving only is most beneficial for students with a higher level of prior knowledge (e.g. Kalyug, Ayres, Chandler, & Sweller, 2003;Paas, 1992;Reisslein et al., ies.ccsenet.org International Education Studies Vol. 14, No. 5;. This can be understood by cognitive load theory. One type of cognitive load is an intrinsic load which does not depend on the nature of tasks only, but also depends on the level of expertise of the learner. When expertise increases, the elements related to problems can be retrieved from the long-term memory and handled in the working memory as a single element, then the intrinsic load is decreased for more experienced learners (Schneider & Shiffrin, 1977).
In sum, most of the available research has been carried out in laboratory settings with university students, and a control condition comprising of problem solving, using a single quantitative method only. In addition, limited research has been conducted in the Arab regions. Thus, the current study assesses the effects of using example-problem pairs for second grade intermediate school students (eighth grade) compared to conventional teaching methods, during a 2-week period in the Saudi context.

Participants and Design
The school is located in an urban district in Hail, a small city which is situated to the north of Saudi Arabia. It was randomly selected from 10 private schools and has only 4 classes for second grade intermediate (eighth grade) students. The eighth grade is an important level and is targeted by international tests, such as the Trends in International Mathematics and Science Study (TIMSS).
The participants consisted of 63 eighth-grade students from four classes of one Saudi intermediate school (age M =13.9 years; SD = 0.31). A mixed method embedded design, consisting of a quasi-experiment design with supplemental field notes was conducted. A quasi-experimental design was applied in order to minimise bias in estimating the difference between the example-problem pairs instruction and conventional instruction. This study was designed with a control group and conducted pre-tests to reduce bias and to avoid certain threats which could invalidate the data (Harris, Mcgregor, Perencevich, Furuno, Zhu, Peterson, & Finkelstein, 2006).
Two classes (n = 33) were randomly assigned as the 'experimental group', and two classes (n= 31) were selected as the 'control group'. One teacher taught both groups. Field notes were taken during the interventions with the purpose of providing a deeper meaning and greater understanding of the phenomenon being studied. The combination of a quantitative and qualitative approach is important to understand the implementation efforts between the actual practice and evidence, and examine the context of the study (Albright et al., 2013).

Topics
'Probability' was used as the study topic. The content was new to the students and included: count outcomes, compound events probability, theoretical and experimental probability, and making prediction with probability. The topic was chosen in order to cope with the textbook order at time of implementing the study. The content of the example-problem pairs instruction was designed by the author who has been an expert in teaching mathematics of the same grade for several years prior to working at the university. All examples and problems were taken from a textbook. The content was given to two experts in the field to check its validity. The feedback received from the experts was included in the final version.
The students in the control group were taught according to the textbook instructions. More specifically, the students were presented with concepts and some examples were explained and, following this, they were required to complete a selection of exercises. The students in the experimental group were presented with one worked example for each lesson, followed by various problems which they were required to solve (ranging from 7 to 10 problems that were similar to the worked example in structure but had different contexts). The students would be requested to pay full attention to teacher when they explained a worked example.
The worked example is expected to be a source of transfer for later problems. Once the students had studied the worked example, they were presented with similar problems. Due to their individual differences, not all students were expected to solve all problems. For example, all the students who succeeded in solving the first problem would be asked to proceed solving the subsequent problems, one by one. However, any student who experienced difficulties in solving the first problem would be regrouped and have the worked example explained to them again, as many times as needed (See examples in Figure 1).

Example of worked example
(This worked example was explained step-by-step by the teacher for students)

Worked example:
A restaurant offers two types of food (meat, fish) with five types of spices. How many food options are possible?
Examples of problems followed the worked example and students were required to solve these problems: Problem 1: Saad has 4 shirts and 5 pants. How many different ways can Saad wear his shirts and pants?
Problem 2: Use a tree diagram to find all the possible outcomes when rolling a fair six-sided dice, numbered from 1 to 6. (1-6) twice. The instruction lasted for two weeks and took place during 8 x 45-minute class sessions, with four sessions per week totalling 6 hours for each group. Both groups were given the same amount of time to control the time factor.

Mathematics Test
The mathematics test was designed by the author and contained 14 questions comprising 5 multiple-choice questions, 8 short answer questions and one drawing test, applied twice -once before the experiment took place (pre-test), and once after the intervention (immediate post-test). The mathematics items were verified for reliability and credibility. After the tests had been prepared they were presented to 3 arbitrators for checking. The arbitrators gave their opinions on the clarity, adequacy and relevance of the content. The opinions of the arbitrators were considered and included in the preparation of the final image of the tests. The reliability by retests was .86; the internal consistency for the sub-scale of the tests was .84. For the examination marks, each item scored either one or zero.
(See examples of the items in Figure 2).

Example of mathematics questions
When rolling a fair six-sided dice, numbered from 1 to 6, what is the probability of getting a 3?
A random study was conducted on people in a shopping centre. It showed that 22 of them prefer to travel with the family by car, 18 prefer to travel by plane, and 4 prefer to travel by bus. How many people who might prefer to travel by plane out of the 500 people who have been studied? Figure 2. Examples of mathematics items

Field Notes
A research diary (field observation notes) was kept throughout the project. The research diary was taken during the implementation of the study. As the researcher, I was situated at the back of classroom to make sure everything was proceeding as planned. My intention was to monitor the implementation of the study and I used a diary to document my observations, particularly what took place during lessons and my observations inside mathematics classrooms. The field observation notes used in this study consisted of two parts: firstly, descriptive, followed by reflective information (Patton, 2015). Therefore, after documenting the factual data obtained from inside the classrooms, the researcher then reflects on the meaning of the observations as initial interpretations. This was outside of the classrooms in order to be more accurate, organised and focused on the research problems. The main focus was on the teachers' performance, particularly in respect of teacher intervention, student practices, and student responses.
Field observation notes were also analysed thematically. When the data [collection] was completed, the six phases suggested by Braun and Clarke (2006) were applied. This started with familiarising the researcher with the data by reading and searching for meaning and making notes about expected codes related to the research questions. The second step was to generate the initial codes manually. The third step was to search for themes. The fourth step was ies.ccsenet.org International Education Studies Vol. 14, No. 5; to review the themes. The fifth step was to define and name the themes.
Three final themes emerged, namely: student engagement and the students' individual differences. The sixth step was to produce a report that provided sufficient evidence for each theme. The data being analysed after 3 sessions to develop themes, and the themes emerged after 3 sessions, and at the end of the implementation, the researcher felt the qualitative was sufficient see an example of observation notes at Appendix A.

Statistical Analysis
The quantitative data was analysed by conducting t-test independent samples to check the difference between the groups in the pre-test scores, and a one-way ANCOVA to compare the effectiveness of the two teaching methods for controlling prior knowledge. Normality and Levene's test checks were performed and the assumptions were met. An Effect Size [Partial Eta Squared (η p 2 ),] was performed and reported. The effect size was classified as Cohen suggested, i.e. small .01, medium, .06, and large .14. All analysis was performed on IBM SPSS v22 and at a 5% (0.05) level of significance.

Procedures
The school was chosen and permission was taken to implement the study in the school. The author designed the content of example-problem pairs instruction and checked its validly by two experts. All examples and problems of the example-problem pairs instruction were taken from the textbook. The mathematics test was designed by the author who checked its validity and its reliability. The teacher was trained by the author for two days (two hours per day) to use the strategy before the intervention. All participants gave their consent and agreed to take part in this study; they were informed they could withdraw at any time without having to provide any reason for their decision to leave the study.
Two groups were randomly selected (experimental and control group). A pre-test was conducted for both groups. The students in the experimental group were trained by the teacher and author's supervision for one session which explained how they would be taught over the coming two weeks. In the experimental group, the strategy of example-problem pairs instruction was applied for two weeks. In the control group, students would be taught using the conventional teaching strategy; i.e. instructions would be explained by their teachers and, following this, they would be given exercises to work through. During the study, the author observed the intervention for both groups and made notes. After two weeks a post-test was conducted for both groups.

Results
The aim of the study is to investigate the effectiveness of example-problem pairs instruction when compared to conventional teaching methods in relation to students' mathematics achievements. The results of the quantitative data will be reported first, followed by the results of the qualitative data.

Mathematics Achievement
The t-test results indicated no significant difference in pre-test scores for the experimental and control groups (t = -1.269). Note. ***p<.001. Table 1 shows the outcomes of a one-way ANCOVA which was conducted to compare the effectiveness of the strategy for controlling prior knowledge. Normality and Levene's test checks were performed and the assumptions were met. There was a significant difference in the post-test results, F (1.62) = 69.14, p < .001. The students in the experimental group had higher post-test scores (M = 4.97, SD = 1.88) than the students in the control group (M = 1.45, SD = 1.34).

Field Observation Notes
Following analysis of the field notes, two categories emerged; namely: student engagement and students' individual differences.

Student Engagement
In the experimental group the teacher always asked the students to pay full attention before fully demonstrating the worked example. He explained that they needed to understand this in order to be able to solve the subsequent problems individually. The students seemed aware of the importance of the worked-example as a prerequisite for solving the problems that followed. The awareness of students in this group was becoming increasingly more actionable over time as they found there was no navigation for solving problems, with no understanding of the worked example. For example, the teacher asked the class if they were ready; he reiterated the importance of having to concentrate on this worked example in order to solve the following problems alone. He stressed that they had to learn from this example and if they had fully understood it, they would be able to solve the problems, which was important.
The majority of students seemed to pay full attention and asked for further explanation as needed. For example, some students asked the teacher to repeat some steps of the solution because they didn't think they understood it. This happened almost every time the teacher explained the worked example. However, some students did not pay any attention, particularly at the beginning of the intervention, but this was fading out over time as they began to realise that they themselves must solve the upcoming problems individually. For example, two students were sitting beside each other taking advantage of the teacher by chatting with each other whilst he was writing on the whiteboard. Later on, those students became more committed. Over time, the students showed more commitment as they had to individually solve problems. Consequently, the students seemed to become more responsible for their own learning.
The students were conducting analogical reasoning from the worked examples used to solve the problems. Some students often spoke to themselves for long periods of time whilst attempting to do conduct analogical reasoning well. This was noticed repeatedly and increasingly over time. This indicates that these students were in the process of self-explaining whilst solving the problems. The majority of time located of classrooms was conducted by engaging students in solving problems.
In contrast to the students in the experimental group, students in the control group seemed to pay less attention to the lessons; some students were able to solve the problems as exercises and some were not. More students in this group seemed to be less committed than in the experimental groups. In addition, the majority of time located of classrooms was conducted by teacher explaining lessons.

Students' Individual Differences
The students in the experimental group worked at their own pace. Some students could solve only 2 problems, some could solve between 3 and 5 problems and some were able to solve more than 5 problems per lesson. More importantly, however, all students were seen to be busy with learning. The teaching strategy seemed to meet the students' individual differences. The teacher identified and gathered the students who were unable to solve the first problem and re-taught them they worked example.
Some students could solve various problems needing only a little help from the teacher. On the other hand, some students repeatedly asked the teacher to check their solutions and he just gave them limited feedback, making statements such as "try again" or "make sure you did this correctly". In some cases, the teacher helped the students by giving them some prior knowledge or skills, such as how to multiply fractions.
Some students were able to solve the first problem easily and continued to solve subsequent problems. Other students needed the worked example to be explained again. However, a tiny number of students needed the teacher to explain the worked example more than twice and were clearly lacking prerequisite knowledge, because the teacher took them back to the prior knowledge and explained it again. However, the majority of students succeeded with this strategy.
In contrast to the students in the experimental group, students in control group were exposed to examples presented by teacher more. More students encountered difficulties whilst attempting to solve problems and the teacher tried to help as much as he could. It seemed more difficult for the teacher to handle this.
In summation, it seems that students in the experimental group showed more responsibility for their learning and were more committed compared to the students in the control group. They conducted more self-explaining and appeared to interact with problems more, as well as working more independently and at their own pace.

Discussion
This study aimed to investigate the effectiveness of example-problem pairs on the mathematics achievement of students in comparison to conventional teaching methods. A mixed method embedded design, with a main ies.ccsenet.org International Education Studies Vol. 14, No. 5; 2021 emphasis on a quasi-experiment with supplemental field notes was conducted to address the research aims in depth. The students were taught using example-problem pairs improved better than students taught using conventional teaching methods in mathematical achievement with a large effect size. It was because of different teaching methods.
Students who were taught using the example-problem pairs had an opportunity to interact with problems for the majority of the allocated time in classrooms. However, the students who were taught using conventional teaching methods were following the teachers' instruction for the majority of the allocated time in classrooms. This led to different student practices. Students who were taught using example-problem pairs had to solve several problems individually which made them more responsible for their own learning. This was in contrast to the students who were taught using conventional teaching methods. This responsibility made them engage more for the majority of the allocated time to solve subsequent problems.
The students who had been taught using example-problem pairs expected to learn new knowledge and initial problem-solving skills about the domain from the worked examples before they were presented with problems for abstracting the knowledge. When a learner studies examples they should obtain a basic understanding of the domain principles which then provides a source for later problem solving (Renkl, 2014). In this stage, the worked examples could make learning more smoothly than conventional instruction. Studying from the worked examples could be easier and more understandable than conventional teaching methods (LeFevre & Dixon, 1986;Recker & Pirolli, 1995;VanLehn, 1986). Hence, students should already have adequate prerequisite knowledge (i.e. a basic understanding of domain principles) to enable them to solve subsequent problems effectively.
In this study, the students who were taught using example-problem pairs had to first learn from a worked example which was explained by the teacher in order to solve the following similar problems (similar in structure but with different contexts). They had to solve the problems alone, with no direct assistance from their teacher. Students who were unable to solve the problems would then have the worked example explained to them again, as many times as needed. Some students were easily able to solve the first problem and then continued to solve the subsequent problems. Other students needed to have the worked example explained again. A small number of students needed the teacher to explain the worked example two or more times and they clearly lacked prerequisite knowledge. This will be discussed later (see field notes).
The majority of students were successful in solving the first problem. The problem which immediately followed the worked example required the students to conduct Analogical Reasoning (AR) by themselves. The effectiveness of the worked examples occurred when the learners fundamentally relied on already-familiar examples to solve problems by using analogical reasoning (AR) (Holyoak, 2005;Holyoak & Cheng, 2011). Therefore, the role of the student is to understand the worked example and then use analogical reasoning between the example and problems. They do not need to search for particular solutions. In this stage, students should be ready for abstracting the domain principles just learned with exposing to problems. Then, they should launch their learning journey through problem solving with lower cognitive load because they should have been equipped with the initial stages of skill acquisition, as previously explained.
The role of problems is to abstract an initial schema that should be initiated from the worked example. Abstraction is promoted when the knowledge obtained from studying the examples is applied to a new problem (Renkl, 2014).
The students worked at their own pace. Some students could solve only two problems, some could solve between three and five problems and some were able to solve more than five problems. The strategy seems to meet students' individual differences (see field notes). Future research needs to investigate how example-problem pairs contribute to the students' individual differences.
Importantly, it was noticed that students who taught using example-problem pairs conducted self-explanation. Some students often spoke to themselves for long periods of time whilst attempting to understand how to solve the problems (see field notes). Chi et al. (1989) found that successful learners explained examples more actively to themselves and studied them longer. They tried to make sense of the rationale of solution procedures. Self-explanation activities during learning are important for learners (Atkinson, Renkl, & Merrill, 2003;Hausmann & Van Lehn, 2008;Renkl, 1997;Renkl, 2002;Rittle-Johnson et al., 2017). The majority of students were actively trying to solve problems. However, a few students were lacking in prerequisite knowledge and they needed extra support to keep up with others (see, field notes). The lack of prerequisite knowledge seems to diminish students from conducting self-explanation. Prior research indicated that students with a lack of prerequisites and passive learner were not able to self-explain (Renkl, 2014).
Alternating example with problems seems to encourage students to conducting self-explanation. This is because students have to active to solve subsequent problems individually (Sweller & Cooper, 1985). Prior research ies.ccsenet.org International Vol. 14, No. 5; indicates that the effectiveness of learning via worked examples is reliant on self-explanation. However, little is known about alternating examples with problems. This needs more investigation in future research.

Conclusion and Limitations
The mixed method embedded design study investigated the effectiveness of example-problem pairs instruction when compared to conventional teaching methods on students' mathematics achievements. Quasi-experimental experiments were conducted with two groups (experimental and control groups), embedded with supplemental field notes. The results showed that when example-problem instruction is used as a teaching method, the students' mathematics achievements' could be improved compared to when they are taught using conventional teaching methods. This is possibly because they have more opportunities to interact with problems than the students who are taught using conventional teaching methods. This contribution is important because it is little known about the comparison effects between example-problem instructions with conventional teaching methods.
Students who were lacking in prerequisite knowledge seemed to benefit less from this strategy and this should most certainly be addressed in future research. This study also paves the way for studying the effect of this strategy on far and near transfer and attitudes, and how this interacts with low and high achievers. The limitations of this study are that the results of can be generated only to similar contexts. This study is limited to male students due to a gender segregation system that is operational in Saudi Arabia. 950: the teacher explains the example on the whitebeard: some students asked about some steps that they did not understand.
10:00: the teacher asked the students to solve problems: the majority of students engaged in solving problems. Some loudly explained this to them and some asked for more explanations.
The teachers answer questions by referring them to the example on the whitebeard.
10:15: some students found difficulties in solving the first problem. The teacher gathered them and explained the example again.
10: 30: some students at the end of the session solved several problems (above 5).
The teacher tries to make students' take responsibility for their own learning.
Some students showed attention and a greater sense of responsibility for their learning.
Students engaged and conducted self-explanation.
Some students needed more support and the teacher helped them to understand the example and how to carry out analogical reasoning.
Some students needed to refocus on the teacher's explanation again, while some students could solve many problems. Figure A1. An example of research diary (field notes) translated

Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).