Effects of Instruction-Supported Learning with Worked Examples in Quantitative Method Training

An experimental field study at a German university was conducted in order to test the effectiveness of an integrated learning environment to improve the acquisition of knowledge about empirical research methods. The integrated learning environment was based on the combination of instruction-oriented and problem-oriented design principles and consisted of twelve worked examples. An elaboration intervention was administered as instructional support. The effectiveness of the learning environment both with and without the elaboration intervention was assessed using knowledge application tasks (near and far transfer), which were applied after the training phase. In addition, student’s self-reports on mindfulness (Salomon & Globerson, 1987) were collected. The training was implemented into the regular curriculum. The participants were advanced students in educational science. Both experimental groups (with elaboration intervention: n = 26; without elaboration intervention n = 27) clearly outperformed the control group (n = 17) in the knowledge application tasks. In order to (successfully) foster transferable applicable knowledge, instructional support provided via the elaboration intervention was in fact necessary. Furthermore, the self-reports of students in the experimental group with elaboration intervention showed higher mindfulness scores than those without it. Our results indicate that the integrated learning environment developed in this study can be implemented to improve the acquisition of knowledge about empirical research methods both effectively and efficiently.


Introduction
Many students of the social sciences experience difficulties in understanding concepts of statistics.Frequently, a deeper understanding of the principles of statistical methods is lacking.As a result, students tend to apply erroneous concepts to the interpretation of statistical results (e.g., Castro Sotos, Vanhoof, van den Noortgate, & Onghena 2007;2009).For instance, several studies have shown students' deficits in their ability to interpret statistical data and empirical results in correlation analyses (Krause, 2007;Krause & Stark 2010).Here, learning without reflecting the content of what is being learned, often accompanied by motivational deficits, has been identified as an antecedent of knowledge gaps (e.g., Stark & Mandl, 2000;Stark, Puhl, & Krause, 2009;Tyroller, 2005).Furthermore, students often acquire inert knowledge (Gruber & Renkl, 2000).Inert knowledge at best is accessible in exam situations, but cannot be applied to solve complex problems in practice.Problems of this type are frequently observed in instruction-oriented learning environments (Gräsel & Mandl, 1999;Reinmann & Mandl, 2006), a teaching method that is common among the traditional teaching of research methods in the social sciences.Students therefore often lack essential competencies in applying their knowledge of quantitative statistics.
The goal of this study was to investigate to what extent the outlined problem can be counteracted by using an innovative integrated learning environment (Reinmann & Mandl, 2006).For this purpose, a learning environment based on the integration of both instruction-oriented and problem-oriented design principles was developed.This learning environment depicts several real findings of an experimental study by Tyroller (2005).
Learning tasks were developed as authentic and relevant worked examples.The worked examples consist of problems regarding the use of statistical methods as well as the interpretation of results and the respective detailed solutions.By means of studying this learning environment, the students' acquisition of applicable knowledge to interpret empirical results was to be developed further.
In order to support the effectiveness of the integrated learning environment, we developed an elaboration intervention to facilitate working with the provided material.The intervention included an advance organizer (Ausubel, 1960) that structures working in advance and a short phase of cognitive modelling (Cognition and Technology Group at Vanderbilt [CGTV], 1992;1993) which explains how to use the advance organizer.
We conducted an experimental field study within the setting of advanced seminars in statistics for the educational sciences; our aim was to systematically examine both the effectiveness of the learning environment and the elaboration intervention.By testing the latter, we intended to contribute to the research regarding the effectiveness of instructional support during learning with complex worked examples (e.g., Atkinson, Derry, Renkl, & Wortham, 2000;Hilbert & Renkl, 2008;Renkl & Atkinson, 2007;Stark, 1999;2000;2001).
When learning with worked examples, the learner is presented with the problem, the problem-solving steps as well as the solution itself.The task for the learner is to elaborate these example solutions (Stark, 1999;2001;Stark, Kopp, & Fischer, 2011) and to comprehend the solution.Working on several isomorph worked examples is supposed to support the acquisition of competencies for problem solving, thus primarily improving applicable knowledge (for taxonomies of knowledge see De Jong & Ferguson-Hessler, 1996).The knowledge base thus acquired can facilitate future transfer tasks (Stark, 1999).
There are several mechanisms involved in the effectiveness of worked examples.Since learners are not required to find the solution by themselves, they are able to focus their attention towards understanding the solution and the problem-solving steps (Renkl, Gruber, Weber, Lerche, & Schweizer, 2003).Due to lower cognitive processing efforts, more cognitive resources are available for the acquisition of problem solving schemata (Paas, Renkl, & Sweller, 2004;Sweller, 2005).This in turn facilitates transfer later on.Within contexts of initial knowledge acquisition, worked examples are thus thought to be superior to conventional problem-solving tasks (q.v.cognitive load theory ;Sweller, 1988).The findings of Paas and van Merrienboër (1994) as well as Ward and Sweller (1990) provide empirical support for this notion.
The ability of learners to self-explain the principles behind the example solutions is another factor that considerably influences the effectiveness of learning with worked examples (self-explanations; Chi, Bassok, Lewis, Reimann, & Glaser, 1989;Chi, De Leeuw, Chiu, & La Vancher, 1994;Renkl, 1997).Self-explanations and example elaborations (Stark, 1999) which support learning effectively are characterized by putting an emphasis on structure, meaning that learners explicitly focus on the common concepts and principles underlying the problem-solving steps within different worked examples.This enables the learners to anticipate the structure of upcoming tasks.Less effective example elaborations are paraphrasing or simply re-reading the worked examples (Stark, 1999).However, despite empirical evidence for learning effectiveness (worked example effect, Clark & Mayer, 2003) worked examples cannot be assumed to be generally superior to other types of learning tasks.Worked examples appear to be especially effective when combined with adequate instructional support (e.g., Atkinson et al., 2000;Hilbert & Renkl, 2008;Renkl, 2002;Renkl, & Atkinson, 2007;Stark, 1999;2000;2001).Without instructional support, the didactic advantages of worked examples cannot fully unfold (Renkl, 1997).

Conceptual Basis of the Integrated Learning Environment
In order to transfer the instruction-oriented teaching method of learning with worked examples from well-structured domains like mathematics to the somewhat ill-structured domain of empirical research methods, an adequate contextualizing of worked examples by means of problem-oriented approaches is essential.Stark, Kopp, and Fischer (2011) showed beneficial effects of embedding worked examples in a complex integrated learning environment on diagnostic competencies of medical students.
The domain of empirical research methods has a similarly complex structure.Although tests and procedures are based on stringent mathematical principles producing precise results, the interpretation of these results is a complex process that invariably involves content-related aspects depending upon the particular research question.
For a successful contextualization of instruction-oriented worked examples the use of integrated learning environments (e.g., Reinmann & Mandl, 2006) therefore seems promising.
Integrated learning environments (Reinmann & Mandl, 2006) combine elements from instruction-oriented and problem-oriented learning approaches in an effort to maximize their respective advantages and to minimize their disadvantages.Learning environments of this type have been shown to support the acquisition of applicable knowledge and to have beneficial effects on learning activity and motivation of the learners (Stark, Gruber, Renkl, & Mandl 2000;Stark, Herzmann, & Krause, 2010).Against this background, the didactic approach of this learning environment puts a strong emphasis on the design principles explained in the next paragraphs.
Instruction-oriented design principles focus on an instructional orientation and therefore on a systematic imparting of knowledge (instructional design approach; e.g., Reinmann & Mandl, 2006).The degree of systematization can be inferred from applying elaboration theory (Reigeluth, 1997).Elaborative sequencing aims at systematically depicting learning contents in different degrees of complexity.Elaboration levels of increasing complexity resulting from this method allow learners to reconstruct and elaborate learning contents step-by-step in multiple levels of differentiation (q.v.Schnotz, 2011).
Another design principle of this systematization is the arrangement of the worked examples in structure-emphasizing example sets (Quilici & Mayer, 1996).An example set of this type is a sequence of worked examples characterized by having an analogue underlying structure.With the progression of the sequence, complexity increases.This progression intends to support the acquisition of applicable knowledge.
Problem-oriented design principles require learning processes to be induced by tasks that are both authentic and relevant for the learner (Reinmann & Mandl, 2006).This is an important prerequisite for the acquisition of applicable knowledge, since learning tasks actually relevant to the learner foster the transfer of knowledge to new problems (ibid.).Therefore, authenticity was an essential part of the didactic approach of the learning environment (CGTV, 1992;1993;Gräsel, 1997;Gräsel & Mandl, 1999;Reusser, 2005).
Another important problem-oriented design principle considered in the learning environment was the presentation of the worked examples in multiple learning contexts and perspectives (cognitive flexibility approach; Spiro, Feltovich, Jacobson, & Coulson, 1991) to avoid the acquisition of inert knowledge (Stark, 2000), and to encourage metacognitive processes during learning, such as mindfulness (Salomon & Globerson, 1987).In this context, Salomon und Globerson (1987) stress the role of mindfulness in successfully acquiring and transferring knowledge to new contexts.The concept of mindfulness denotes "[...] a state of mind that is defined as the volitional, metacognitively guided employment of non-automatic, usually effort demanding processes."(Salomon & Globerson, 1987, p. 625).There is strong empirical evidence for the relevance of such mindful, reflected learning for a deeper processing and understanding of learning contents (e.g., Entwistle & Peterson, 2004;Lan, 1998;Wang & Lin, 2007).

Conceptional Basis of the Elaboration Intervention
The elaboration intervention consisted of an advance organizer (Ausubel, 1960) and a subsequent short phase of cognitive modelling (CGTV, 1992;1993) which gave a brief introduction of how to use the advance organizer.The advance organizer's didactic conception is based on the theory of expository teaching (Ausubel, 1960;1968).The vital aspect of expository teaching is to activate prior knowledge by offering associations on the basis of pre-structuring and systematization of learning contents.Activating prior knowledge facilitates the assimilation of new information into the existent knowledge base or, where appropriate, the adaption of existent knowledge on the basis of new information (accomodation; Ausubel, 1968;Mayer, 1979;Hasselhorn & Gold, 2006).It also facilitates knowledge transfer (Mayer, 1980).According to Ausubel (1968), the advance organizer serves to fill gaps between the learner's prior knowledge and the knowledge needed to initiate the learning process.Several authors have suggested formulating the advance organizer on a concrete rather than abstract level (e.g., Corkill, Glover, & Brunning, 1988).
The didactic conception of the cognitive modelling phase was based on Collins, Brown, and Newman's (1989) cognitive apprenticeship approach; its core was the detailed elaboration of the advance organizer's basic structure.Its intention was to facilitate the acquisition of potentially relevant solution heuristics, and to draw the attention to information relevant to the problem.As a result, the cognitive modelling is supposed to facilitate metacognitive processes.
Working with the elaboration intervention should yield a deeper understanding of the learning contents and support the acquisition of applicable knowledge, especially regarding complex transfer tasks.Furthermore, the elaboration intervention is supposed to enhance mindfulness sensu Salomon and Globerson (1987).

Research Questions (1) To what extent is the acquisition of applicable knowledge facilitated by the learning environment and the elaboration intervention?
We expected positive effects of working with the learning environment both with and without elaboration intervention on the acquisition of applicable knowledge.Both experimental groups were expected to outperform the control group.Furthermore, we expected positive effects of instructional support provided by the elaboration intervention on the acquisition of applicable knowledge.The experimental group with elaboration intervention was expected to outperform the group without elaboration intervention. (

2) To what extent is the learner's mindfulness improved by the elaboration intervention? To what extent are mindfulness and post-test performance correlated?
We expected that the elaboration intervention supports learners in working with the learning environment in a more reflective, conscious and mindful (Salomon & Globerson, 1987) manner.Therefore, the experimental group with elaboration intervention was expected to outperform the other groups.We also expected a positive correlation between mindfulness and post-test performance.

Sample and Design
Participants were 70 students of educational science (61 female) recruited in three advanced seminars on empirical research methods at a German university.On average, students were in their seventh semester.The experimental group with elaboration intervention consisted of 26 (24 female) participants with an average age of 24.9 (SD = 3.70) years.The experimental group without elaboration intervention consisted of 27 (22 female) participants with an average age of 24.2 (SD = 2.89) years.The 17 participants in the control group (15 female) had an average age of 24.2 (SD = 2.89) years.
We compared post-test performance of all three groups within the setting of an experimental field study.The participants were randomly assigned to the three groups.The control group received texts on descriptive and inferential statistics (e.g., Bühner & Ziegler, 2009).

Learning Environment
The learning environment was a text of 20 pages that included an outline of an experimental study (Tyroller, 2005) and twelve worked examples referring to the study.
All worked examples were structured according to the same schema "question -empirical evidenceinterpretation": Following the question, a solution was presented.This solution included all the information necessary to answer the question.At the end of the text, the solution to each question was summarized in a general interpretation section.
The worked examples were sorted by an ascending level of complexity and arranged in a structure-emphasizing example set.For instance, consecutive worked examples were based on the same statistical test (Reigeluth, 1997;Quilici & Mayer, 1996).By relating to an actual, real-life study, the worked examples were embedded in a context that was both authentic and relevant to the participating educational science students, thereby also accommodating motivational goals (CGTV, 1992;1993).We constructed multiple perspectives on a given statistical test by putting the test in different contexts (e.g., mediation-vs.moderation-hypotheses) and by presenting different analytical perspectives (e.g., exploratory and confirmatory hypotheses).
Before starting work on worked examples, participants were given an introductory explanation about the framework, sample, and methods of the Tyroller's (2005) study.The first four worked examples covered topics of descriptive statistics and internal validity.Based on this, the subsequent worked examples were intended to enable participants to develop a better understanding of complex statistical concepts.The following three worked examples dealt with tests of mean differences.Worked example 8 was a conceptual question and covered the operationalization of constructs.The last four worked examples dealt with moderator effects, effect size and the resulting practical consequences.SPSS output tables were provided to explain the results of statistical procedures, enabling participants to check on internal validity, significance of mean differences, analyses of moderator effects, and effect sizes.

Elaboration Intervention
The advance organizer consisted of three text pages and was designed as a structured scheme of twelve guiding questions.Each guiding question was directly related to the respective worked example in the learning environment and was complemented by a concise description.The additional descriptions explained the learning goals of the respective worked examples and emphasized underlying principles.Since the advance organizer was formulated with only a minimum level of abstraction, participants were enabled to associate the learning contents with known concepts and thus to activate prior knowledge.
The instructor explained the advance organizer's use in detail by means of cognitive modelling (CGTV, 1992;1993;Collins et al., 1989).Here, students were shown which procedure was useful when working on the guiding questions.The instructor made clear how the students could activate their own prior knowledge about statistical tests and how they could meet the requirements of the tasks by reading the guiding questions.He interpreted exemplified empirical results of the intervention study upon which the learning environment was based.By doing so, he explicated his own example elaboration (Stark, 1999).Thus, students were made aware of the metacognitive processes involved in finding the solution heuristics (Collins et al., 1989;Reinmann & Mandl, 2006).Students were asked to follow the modelling attentively and to subsequently work on the guiding questions of the advance organizer in the same way.

Dimensions for Testing Internal Validity
In order to test the internal validity of the study, the three groups were compared with respect to cognitive, metacognitive, motivational, and socio-biographical learning prerequisites that were potentially relevant to the learning process.The selection of these criteria was based on several studies in the domain of statistics and research methods on learning with worked examples (Krause, 2007;Krause, Stark, & Mandl, 2004;2009).All rating scales had a 6-step rating (from 1 = does not apply at all to 6 = fully applies).All answers to open questions were rated by two experienced raters which were blind to the experimental conditions (Cohen's  = .84).Disagreement was solved by discussion with the first author.
The test on cognitive learning prerequisites comprised prior knowledge in the domain of correlation, t-Test, and ANOVA.Students had to answer six open questions.The test proved to be sufficiently reliable (Cronbach's α = .76,theoretical maximum score: 39 points).
Metacognitive learning prerequisites were tested with a rating scale that consisted of 26 items and was sufficiently reliable (e.g., When I'm learning, I frequently ask myself the question whether or not I have really understood what I have just read.";Cronbach's α = .90) The rating scales for motivational learning prerequisites had already been tested in previous studies on learning with worked examples in statistics and research methods (Krause, 2007;Tyroller, 2005).The intrinsic motivation regarding statistics and research methods scale consisted of six items (e.g., "I am interested in empirical research methods"; Cronbach's α = .88.).The attitudes towards research methods scale consisted of eleven items (e.g., "Statements that have been made by using empirical research methods are absolutely correct"; Cronbach's α = .70.).
Additionally, participants had to indicate gender, age, and number of semesters.

Dependent Variables
We designed a test for applicable knowledge (De Jong & Ferguson-Hessler, 1996) in order to assess the learning effectiveness of the learning environment and the elaboration intervention.Applicable knowledge was operationalized by using two tasks.
Task 1 tested near transfer of applicable knowledge.The task consisted of interpreting the results of an experimental study and to deduce implications for practice.The empirical results originated from 1-factorial designs.The theoretical maximum score for task 1 was 14 points.
Task 2 measured far transfer of applicable knowledge; it also consisted of interpreting the results of a study and to deduce practical implications.Here, the students' task was to interpret a 2x2-factorial design so that certain aspects of the application of knowledge were much more complex than in task 1.The theoretical maximum score for task 2 was 15 points.
Post-test tasks on applicable knowledge showed no significant correlation (r = .18,p = .13).
Additionally, mindfulness was measured using a three-item scale (e.g., While using the learning environment, I was completely immersed in my work"; Cronbachs α = .71)which was gathered from studies of Krause (2007) and Tyroller (2005).

Procedure
During the first session of the seminar, participants were briefly introduced to the study, and then asked to fill out the test on prior knowledge, the scales relating to internal validity, and the socio-biographical questionnaire.
During the second session of the seminar, the training of the two experimental groups (with and without elaboration intervention) was carried out in two separate rooms.Both groups were instructed by experienced tutors.
Each participant had to work on the learning environment by himself.The time limit was 60 minutes for both experimental groups, a generous lead time that ensured that all the participants were able to finish working on the learning environment.After a short break, participants were asked to fill out the scales on measuring mindfulness, and the two post-test tasks.
Students in the control group worked on the socio-biographical questionnaire, the prior knowledge test, the scales relating to internal validity, and the two post-test tasks as well.Instead of working with the learning environment, students of the control group had to study selected texts on statistical procedures and their interpretation for 60 minutes.Table 1 shows the procedure for all groups and the approximate time provided for each test and the training.

Statistical Analyses
In order to test internal validity, we conducted χ 2 tests and 1-factorial ANOVAs, depending on the data's respective scale level.
Regarding the two post-test tasks, we formulated directional hypotheses which made it possible to test the effects of learning environment and elaboration intervention via 1-factorial ANOVAs with a priori contrasts for three groups (e.g., Bühner & Ziegler, 2009).The effect of the elaboration intervention on mindfulness as reported by the participants was tested by running a 1-factorial ANOVA.
As a measure for effect size in our ANOVAs, we used partial eta square (η p ²), and Cohen's d (Cohen, 1969) for contrasts.

Internal Validity
There were no significant differences for age (F(2,67) = 0.10, p = .83)nor gender (χ 2 (2) = 1.41, p = .49)between the three groups.The high percentage of female participants was in accordance with gender distribution in other, comparable courses of studies in Germany (Statistisches Bundesamt, 2012).The three groups were also comparable regarding cognitive prerequisites for learning; we found no significant differences in topic-related prior knowledge (F(2,67) = 0.24, p = .79).Likewise, there were no differences in metacognitive and motivational learning prerequisites.The scales measuring metacognitive learning prerequisites (F(2,67) = 0.26, p = .77)and intrinsic motivation (F(2,67) = 0.54, p = .58)showed no statistically significant differences between the groups.Therefore, the current study can be considered internally valid regarding these variables.

Effects of Learning Environment and Elaboration Intervention
(1) To what extent is the acquisition of applicable knowledge facilitated by the learning environment and the elaboration intervention?
Means and standard deviations as well as theoretical maxima are listed in table 2. Performance of task 1 (near transfer) was clearly below the theoretical maximum score for all groups.However, we found a significant and large effect of the learning environment (F(2,67) = 4.06, p = .022,η p ² =.89, power = .70).As expected, the a-priori contrasts showed that both the experimental group with and without elaboration intervention outperformed the control group (t(67) = 2.77, p = .007,d = 0.99 and t(67) = 2.22, p = .03,d = 0.76), respectively).The contrast effect was large in both cases.In contradiction to our hypothesis, there was no effect of the elaboration intervention; both experimental groups performed similarly well (t(67) = -0.64,p = .52).
Regarding far transfer of applicable knowledge (task 2), all groups performed below the theoretical maximum score as well.There was a significant, medium effect of the learning environment (F(2,67) = 9.02, p < .001,η p ² =.78, power = .97).As expected, there was a significant effect of the elaboration intervention: participants of the experimental group with elaboration intervention significantly outperformed those without elaboration intervention as well as the control group (t(67) = -3.42,p = .001,d = 0.96 and t(67) = 3.85, p < .001,d = 1.15), respectively).The contrast effect was large in both cases.The experimental group without elaboration intervention and the control group did not differ from each other significantly (t(67) = 0.85, p = .40).Mean scores of participants in both experimental groups were clearly above the theoretical mean of the scale.The effect of the elaboration intervention was significant and large (F(1,51) = 6.86, p = .012,η p ² = .88,power = .73).Participants of the experimental group with elaboration intervention (M = 4.69; SD = 0.47) reported higher mindfulness than did participants of the experimental group without elaboration intervention (M = 4.19; SD = 0.93).
In contradiction to our hypothesis, mindfulness only correlated significantly with the performance regarding far transfer of applicable knowledge (task 1: r = .23,p = .10;task 2: r = .32,p = .018.).The correlation was positive and rather low.

Discussion
The results showed that the tested groups were comparable with regard to the socio-biographical, cognitive, metacognitive, and motivational learning prerequisites.With regard to these variables that might potentially interfere with the learning process, we assume that the current study is internally valid.
As the three seminars were mandatory for the students, there were no self-selection-effects influenced by the students' decision concerning seminar choices.The study represents a full inquiry of the respective cohort.Therefore, differences in performance between groups as well as differences in mindfulness can actually be attributed to the experimental variation.
Learners accomplished the near transfer task even without the elaboration intervention.On average, participants scored half of the achievable points.Thus, regarding the acquisition of the knowledge necessary to solve this task, the integrated learning environment has proven to be successful.
The more challenging far transfer task required a deeper understanding of the presented worked examples.Average scores of the participants were about half of the maximum score for this task as well.In accordance with the results of Krause et al. (2009), Krause (2007) as well as Krause and Stark (2010), the performance on this task was supported by the elaboration intervention.Our results demonstrate once more an issue previously discussed in several studies on problem-oriented (Kirschner, Sweller, & Clark, 2006;Schmidt, Loyens, Van Gog, & Paas, 2007) and example-based learning (Renkl & Atkinson, 2007;Hilbert & Renkl, 2008;Atkinson et al., 2000;Stark, 1999;2000;2001).The issue concerns the relevance of additional instructional support in complex learning.This appears to be especially valid for complex domains with a lower degree of structure.
Considering especially the high difficulty of the far transfer task as well as the relatively short training phase of just 60 minutes, the effects gained by the learning environment and elaboration intervention are considerable.
Similar to the study of Stark et al. (2009), the elaboration intervention had a positive impact on mindfulness, meaning that the participants learned with a higher degree of reflection; the scale indicates a deeper involvement with the presented material.The result that only performance on the far transfer task correlated with mindfulness indicates that mindfully learning for this complex task proved especially helpful.
As a whole, our results emphasize the effectiveness of integrated learning environments regarding the acquisition of applicable knowledge (e.g., Stark et al., 2011;Stark et al., 2010).Results of this nature are rather scarce in several instructional contexts (e.g., Dettermann, 1993).With regard to training in research methods and statistics in educational sciences, such effects are in fact an exception (Stark & Mandl, 2000;2002;Stark, Mandl, Gruber, & Renkl, 2002).
However, there are some limitations of the study.The elaboration intervention was designed as an integrated support intervention.The modelling phase referred directly to the advance organizer.Introducing guiding questions without the simultaneous use of a modelling phase is pedagogically not well justifiable, especially in a field of knowledge that is usually quite problematic for students.For this reason, an isolated test of both components by means of experimental variation was not intended.Because of the known learning effectiveness of worked examples, the learning environment's underlying didactic approach was not experimentally varied as well.As a consequence, the results can only be attributed to the combination of the described didactic methods.
Furthermore, the overall sample size was too small to derive more ambitious statements.To state more general conclusions, the findings regarding near and far transfer have to be replicated within a study with a bigger sample size.The sample should be enlarged regarding participants of other studies of the social sciences applying statistical methods like psychology, and sociology.Although the size of the test-power was adequate for each test, the study is regarded as a pilot-study to develop an instructional basis for future learning environments.
The limitations of the current study also suggest desiderata for future research.The learning processes induced by the integrated combination of instruction-oriented and problem-oriented approaches need further research in order to gain more empirical evidence for optimizing integrated learning environments.In this context, the method of thinking aloud (Ericsson & Simon, 1993) that has been well researched in conventional example-based learning (Renkl, 1997;Stark, 1999) would appear suitable.In addition, the sustainability of the effects gained by the learning approach in general and the elaboration intervention in particular needs to be tested in follow-up studies.
Based on these results, the implemented methodical-didactic approach can be recommended for training in research methods and statistics in educational sciences.The principles of the learning environment are also easily applicable for other domains with a sufficient consensus on the correctness of a given problem solving task.

Table 1 .
Sequence of the tests for both experimental groups

Table 2 .
Post-test performance of the three groups; number of participants, means and standard deviations,