Hybrid Learning: An Effective Resource in University Education?

The organisation of university education in Europe is undergoing profound changes as a consequence of the establishment of the European Higher Education Area (EHEA). This transformation entails methodological changes that are focused on student work. The student is now considered to be an autonomous individual who is able to choose a path of study and capable of self-regulation. These objectives are believed to be achievable with hybrid learning models. The economic cost of including these methods makes it necessary to demonstrate whether the investment can be profitable in terms of improved academic results and increased acceptability among students. We analyse whether the use of two tools by students (assessments and forums) influences their grades and whether there are correlations between performance and the evaluation of the tool by students and between the evaluation and the degree of use. The sample consists of 176 students. We follow an ex post facto methodological design, with descriptive and correlational techniques. We found significant differences in the grades received according to the degree of use of the tools studied. Additionally, we found a correlation between grades and student evaluation.


Introduction
There are authors who believe that Engineering Studies are the most complex at the university level (Peterson & Feisel, 2005;Vázquez-Martínez & Alducin-Ochoa, 2014).We have observed that this is true on a daily basis in our classrooms, witnessing high dropout and academic failure rates (Santiago, Brunner, Haug, Malo & Pietrogiacomo, 2009).In our case, the dropout rates before the introduction of Web Course Tools (WebCT) were between 50 and 60%.This dropout rate can be linked to the broad knowledge that the discipline demands of students.For example, the Materials Science courses in the first year of the Higher Technical School of Construction Engineering include concepts from physics, chemistry and geology.When students begin this course, their knowledge is typically insufficient, above all in chemistry and geology.This deficiency has been shown to be a widespread characteristic of all engineering studies (Tynjälä, Salminen, Sutela, Nuutinen, & Pitkänen, 2005).We also observe that students exhibit an underdeveloped ability to reason, analyse, synthesise and think critically (Holvikivi, 2007;Grasso & Martinelli, 2007), as well as low levels of comprehension (Hougthon, 2002).These deficiencies are the result of the traditional teaching methods used in engineering, which relies on lectures and final summative assessments, and a high student/professor ratio.

Hybrid Learning and University Teaching
Although much has been written on hybrid or blended learning (BL), such type of learning continues to be considered up-to-date to the extent to which it is an educational method that (if well-designed, planned and implemented) can respond to current university demands.BL is also being increasingly used as a model in university teaching (Cooner, 2010;Ellis, Ginns, & Piggott, 2009;Mackenzie & Walsh, 2009), whereby traditional classroom teaching models are complemented with their online versions, enabling access to the best of both worlds.The BL modality requires integrating traditional teaching methods with more modern online activities (Graham, 2013;Macdonald, 2008;Mitchell & Forer, 2010), which are reinforced with classroom teaching, thus enabling students to access content at their own pace within a self-directed learning process (Nuffer & Duke, 2013).through the use of tools found in the different platforms and provides more information on student work (Sitzmann, Kraiger, Stewart, & Wisher, 2006).
The opportunity to create new communication and interaction scenarios (Brindley, Walti, & Blaschke, 2009;Ching & Hsu, 2011) should also be noted.In a traditional educational setting with solely simultaneous communication, information and communications technology (ICT) offers the possibility of asynchronous communication.Asynchronous communication makes communication processes more flexible, increasing the level of interaction between participants (Çakýroðlu, 2014) and offering students a period of reflection during which they can develop their arguments and maximise their creativity (Means, Toyama, Murphy, Bakia & Jones, 2009) before providing the response requested.This period of reflection includes the search for information and its subsequent analysis, structuring and publication.BL's fundamental advantage is that the student can participate from any location and at the time deemed most convenient, while the classroom sessions reinforce the sense of belonging to a learning community (Carr-Chellman, Dyer, & Breman, 2000).Thus, BL increases the benefits of collaborative learning.The acquisition and the creation of knowledge are social construction processes, and we can clearly observe it being strengthened through the available technological means (Cheng, Jordan, Schallert, & The D-Teamc, 2013;McGee & Reis, 2012).In this sense, the interaction between students and teachers in forums is important.All of these advantages are driven by the physical and temporal de-localisation that results from the use of BL educational platforms.
Moreover, in classroom teaching, the information that must be learned by the student is presented by the professor, whereas a BL student has access to a wide range of educational resources and can independently determine which resources to use, which constitutes the delocalization of information in close contexts.One noteworthy aspect is the role that must be played by the student, as the creator of his or her knowledge (Mosca, Ball, Buzza, & Paul, 2010;Neto et al., 2013;Nicol & McFarlane-Dick, 2006) and as the manager of his or her time.
BL offers increased learning possibilities to students (Sussman & Dutter, 2010), above all, as individuals in the independent learning process.However, the teaching materials must be structured to support this individuality.In order to improve the quality of learning through BL, the implementation of this methodology should be based on pedagogical criteria (Papastergiou, 2007).In our case, we followed the criteria established for the Spanish context during the design and implementation of the teaching resources (Cabero & Gisbert, 2005).Subsequently, the course we designed was evaluated by ICT experts from the University of Seville.
Research studies on BL and technical teaching remain in their early stages (Aktas & Omurtag, 2013;Hölb & Welzer, 2010).BL is not used in engineering studies (C.Porumb, S. Porumb, Orza, & Vlaicu, 2013) or architecture (Francis & Shannon, 2013) as commonly as in other subjects.Fortunately, important steps are being taken not only to change the teaching methods and practices, but also to publicise the results regarding ICT.However, the use of BL fundamentally depends on the personal initiative of the professors (Francis & Shanon, 2013).

Online Assessments
A BL system must include resources for the diagnostic, educational and summative assessment of the students (Dziuban, Hartman, & Moskal, 2004).Online assessment tests positively influence academic results (Stull, Varnum, Ducette, Schiller, & Bernacki, 2011) and increase the number of opportunities for enhanced comprehension in large groups (Singh, 2010).Such tests are recommended for helping students improve their awareness of their own level of knowledge and for implementing corrective measures if needed (Marriott & Lau, 2008).
In our case, the types of assessment included preparation for the classroom sessions, a knowledge assessment and a global assessment by unit content.The first type of assessment (preparation for the classroom session) consisted of self-assessments, i.e., tests taken by the student to verify his or her level of knowledge on the content.Because the tests were administered within WebCT, the results were not known by the professors.Self-assessment included true/false and fill-in-the-blank type of tests, with the aim of helping students master the basic concepts of each topic in order to attain a good command of the technical and scientific terminology required for their later application.A recommendation was given to the students to take these tests before each classroom session to maximize their autonomy and to learn self-regulation (McKenzie, Perini, Rohlf, Toukhsati, Conduit, & Sanson, 2013).This strategy has provided good results in large groups (Moravec, Williams, Aguilar-Roca, & O'Dowd, 2010), with these tests being formative and diagnostic in character (Mora, Sancho-Bru, Iserte, & Sánchez, 2012).
The second type of assessment consisted on multiple-choice and matching tests, which increase the level of complexity and required a significant degree of knowledge on the subject.The third type was a simulation of an examination, where the conditions were the same as an actual test: 40 questions that used different test formats (e.g., short answer, matching, multiple-choice and true/false).Both of these types of assessment had an educational function.In addition, at the end of each unit, students had to take an in-class examination.
Educational assessment is important because it informs students about the extent of their knowledge and facilitates timely follow-up and support from the professor (Marriott & Lau, 2008;Russell, Elton, Swinglehurst, & Greenhalgh, 2006;Timmers, Broek, & Berg, 2013).
In all of the types of tests, corrections were automatic.As learning by repetition improves content retention in the long term, the students were allowed to repeat the tests as many times as they wished (Roediger & Butler, 2011).However, before repeating a test, the students had to wait 24 hours so those who had failed the test could review.Although this measure punishes the students who achieve good results, the version of WebCT that we used did not facilitate custom unblocking of the times for the same test.In the end, this practice improves academic results (Roediger & Butler, 2011), although in our case, the students did not usually accept the waiting time.

The Forum
The forum facilitates communication between individuals who are separated by time and space, and it enables the formulation of hypotheses, experimentation and reflection (Lust, Elen, & Clarebout, 2013).Participation in this type of communication resource was found to be among the most important student activities (Parry, 2010), and enabled student knowledge to be evaluated efficiently (Shirky, 2010).The forum promotes active learning through participation and feedback (McGee & Reis, 2012), and from the participation in a forum, the student can construct new knowledge (Nicol & McFarlane-Dick, 2006), as the forum is an asynchronous communication tool that encourages reflection by students and higher-level contributions (Cheng et al., 2013).The implementation of the forum in the BL model helped students to find the answers to doubts that prevented them from continuing the learning process promptly, independent of time and space.For these reasons, the forum has a singular importance in the teaching-learning process.Some authors have suggested that the professors should specify what their participation will be like (Augustsson & Jaldemark, 2014).In our case, the participation philosophy was to maximize the number of responses provided by the students, and for the entire process to be controlled by the professor responsible.The commitment was to adhere to a maximum response time of 24 hours.The support that may be provided by professors is important if optimal learning results are to be obtained (Lo, 2010).Easy access to the professor was regarded as an important factor of student satisfaction (Martínez-Caro & Campuzano-Bolarin, 2011), as a highly-visible professor increases the number of interactions with students (Oliveira, Tinoca, & Pereira, 2011).

Objectives and Hypothesis
The objectives of our research were as follows: 1) To verify the influence of the degree to which the forum and the assessments were used by the students on their academic performance.
2) To demonstrate the relation between the student evaluation of WebCT and student academic performance.
3) To demonstrate the relation between the degree to which the forum and the assessments were used and the platform evaluation.
The null hypotheses were as follows: H1 0 : There are no significant differences in performance among Materials Science students as a function of their degree of use of WebCT.
H2 0 : There is no correlation between the student evaluation of WebCT and academic performance.
H3 0 : There are no significant differences regarding the WebCT evaluation as a function of its degree of use.

Design
The proposed research objectives suggested the use of a cross-sectional methodological design that is ex post facto with variables that were not experimentally manipulated and using descriptive and correlational methods.

Participants
The sample consisted of 176 students enrolled in the Materials Science course in their first year at the Higher

Instruments
The data on WebCT use and student ratings were obtained from WebCT.The items that referred to the evaluation using the PSEW questionnaire were specifically designed.This questionnaire consisted of two parts.The first part contained 48 items to be answered using a Likert scale and that were distributed among four aspects: formal and technical aspects (8 items), content (16 items), assessment tests (12 items) and global evaluation (9 items).
The second part was qualitative.The quantitative results are presented in this article.To determine the PSEW's reliability, we calculated Cronbach's alpha and obtained α=.923, which indicated an excellent level of reliability (George & Mallery, 1995).

Data Analysis
The Statistical Package for the Social Sciences (SPSS) statistical suite software (version 17) was used.For objectives 1 and 3, parametric tests were used when the dependent variable was measured at the interval level and had a normal distribution.Student's T-test was used if the independent variable was dichotomous, and variance analysis (ANOVA) was performed when the independent variable was polytomous.Non-parametric tests were used when the dependent variable did not reach an interval level measurement or did not follow a normal distribution.The Mann-Whitney U test was used if the independent variable was dichotomous, whereas the Kruskal-Wallis test was employed when the independent variable was polytomous.For objective 2, we used Spearman's rank-order correlation.For all the tests, a 95% confidence level (α=.05) was used.

Influence of the Use of Assessments on Performance
To conduct this analysis, we established three types of use based on the number of tests taken: low (number of tests taken between percentile the 1 st and 33 th percentile), intermediate (between the 33 th and 67 th percentile) and high (between the 67 th percentile and the maximum number of tests taken).The Kolmogorov-Smirnov normality test indicated that Units 1 and 3 met the assumption of normality but this it was not met in Unit 2.
Table 1 compiles the mean values of the grades received by students in the three units (as a function of the degree of use of the test) and the average ranges of Unit 2, whereby a non-parametric test was used.The Levene test in ANOVA (Table 2) indicated that the homoscedasticity assumption was satisfied.Thus, we used Snedecor's F of the ANOVA and obtained significant differences in the following: Therefore, we accept the alternative hypothesis: "There are significant differences in the student's performance according to the degree of use of the assessment test in the three content units".

Influence of the Use of the Forum on Academic Performance
To analyse this influence, three usage groups based on the number of messages written on the forum were established: low (a number of messages between the 1 st and 33 rd percentile), intermediate (between 33 rd and 67 th percentile) and high (between the 67 th percentile and the maximum number of messages).
The Kolmogorov-Smirnov normality test indicated that the grades from Units 1 and 2 did not have a normal distribution, but that it was found for the Unit 3 grades.Table 4 compiles the means of the grades obtained in the three units (as a function of forum use) and the average ranges of Units 1 and 2, whereby a non-parametric test was used.The Levene test (Table 5) indicated that the homoscedasticity assumption was satisfied.Thus, we used Snedecor's F of the ANOVA.We found that there were significant differences in the grades from Unit 3 between the mean of the low (M=4.Therefore, we accept the alternative hypothesis: "There are significant differences in student's academic performance according to forum degree of use in the three content units".

Relation between Webct Evaluation and Performance
For this analysis, we used a correlational study.Because the platform evaluation variables were quantitative with an ordinal scale of measurement (because they were drawn from the PSEW questionnaire, which used a Likert scale) and the grade variables were quantitative with interval measurements, the Spearman correlation coefficient was used.In Table 7, we compile the correlation coefficients for the study pairs WebCT evaluation and the mean grade of the three content units.* The correlation is significant at .05 (bilateral).
In the analysis of the results of the Spearman correlation coefficients between the four WebCT evaluation aspects and performance, the following was found: • A statistically significant positive correlation with an intermediate effect between the evaluation of the WebCT aspects and subject grades, r S (N=116)=.322;p=.000, and the evaluation of the assessment aspect of the WebCT and grades, r S (N=116)=.501;p=.000.
• A statistically significant positive correlation with a small effect between the evaluation of the formal and technical aspects of the WebCT and grades, r S (N=116)=.208;p=.025, and between the global evaluation of the WebCT and the grades, r S (N=116)=.215;p=.020.
The correlation coefficients confirm our hypothesis, and thus, the following alternative hypothesis can be confirmed: "There is a correlation between student evaluation of the WebCT and academic performance".

Relation between the Degree of Use of the Platform and Its Evaluation
Here, we tried to determine whether there were significant differences among the means of the evaluations of perception and satisfaction made by the students as a function of their degree of use of the platform.Table 8 compiles the means and average ranges of the evaluations by WebCT aspect as a function of the degree of use of the assessment test.The Kruskal-Wallis test (Table 9) indicated that no significant differences were found in the evaluations of the formal and technical aspects of the platform, X 2 (2, N=115)=4.758,p=.093, among the students from the three degrees of usage of the assessment tests, whereas the following evaluations exhibited differences: Because the test does not confirm the hypothesis in all the evaluation aspects, the null hypothesis ("There are no significant differences in the WebCT evaluation as a function of its degree of use") can be confirmed in the evaluations of the platform's formal and technical aspects.Regarding the content, assessment and global evaluation aspects, the alternative hypothesis can also be confirmed: "There are significant differences in the WebCT evaluation by degree of use of the assessment test".Table 10 compiles the means and average ranges of the evaluations by WebCT aspect as a function of the degree of use of the forum.The Kruskal-Wallis test (Table 11) indicates that no significant differences were found in the assessment evaluations, X 2 (2, N=73)=3.928;p=.140, and the global evaluation, X 2 (2, N=73)=2.066;p=.356, between students of the three forum usage degrees, whereas the following differences were found: Because the test does not confirm our hypothesis in all of the evaluation aspects, the null hypothesis ("There are no significant differences in WebCT usage and satisfaction by degree of forum use") can be confirmed for the evaluations of assessment tests and in the global platform evaluation.The alternative hypothesis "There are significant differences in the WebCT evaluation according to degree of use of the forum" can also be confirmed with respect to the content and the formal and technical aspects.
The results demonstrate that the students who achieved a better academic performance were those who were more active, i.e., those students who used the WebCT resources more frequently and more significantly.Some authors believe that the academic results depend on the degree of the student's interaction with the system (Alducin-Ochoa & Vázquez-Martínez, 2016;Beer, Clark, & Jones, 2010).The students with the most messages on the forum were those who had performed the most assessments.
Similarly, we found a correlation between academic performance and platform evaluation.That is, academic performance could have increased due to the above-mentioned considerations, and we could state that the greater the effort, the better the performance and evaluation.Previous research has shown that when a student has a positive perception of the learning environment, he or she tends to achieve better academic results (Pérez-Marín & Pascual-Nieto, 2012;Webster, Chan, Prosser, & Watkins, 2009).
Regarding the differences analyzed, several items enabled us to interpret the results.For example, for the content aspect, the students believed that the constant use of the platform enabled individual learning, that the diverse options offered facilitated comprehension, and that the content itself was easy to understand.The fact that the students used the assessments and could repeat them as many times as they liked decreased the initial difficulty of the topics over time (Cortizo et al., 2010;Vázquez-Martínez & Alducin-Ochoa, 2014).We should note that this method was very flexible in adapting to individual differences (Limniou & Smith, 2010) because it was based on environments that could host a substantial variety of resources that responded to different needs.
As for the formal and technical aspects, students highly valued the platform's ease of use.However, they criticised the speed of access because the system occasionally failed.Regarding assessments, we noted high evaluations awarded by the students to the following items: the tests provided me with motivating results; when I committed an error, they helped me learn; they helped me discover previously unconsidered aspects; they enabled me to decrease stress when taking the examination; and they enabled me to better organize my work and make it more productive.Student motivation increased (Mora et al., 2012), increasing at the same time opportunities for success (Svanum & Aigner, 2011).In addition, the results demonstrated that planning one's work affected grades, which resulted in an optimisation of study time and results (Domingo-Calabuig & Sentieri-Omarrementeria, 2011).
The professor activity item obtained an evaluation of approximately 5 in the global evaluation aspect of the elements taken into account.The BL modality facilitates better support and performance follow-ups for students (Sitzmann et al., 2006) thanks to the tools available, which resulted in individualised counseling.The BL modality also supported the supervision of resource access, of the timeliness of student test taking, the grades obtained, and whether student mistakes were recurrent.The professor's work in the platforms is fundamental for the achievement of satisfactory learning experiences (Martínez-Caro & Campuzano-Bolarín, 2011).The professors are curriculum facilitators, not mere stewards of the content (Francis & Shanon, 2013).
Whereas the majority of the students were satisfied with the method and their results, others thought that test taking resulted in a work overload, and they developed a reluctant stance toward tests and participation in the forums.Finally, the BL modality enabled students to control their learning process and received constant feedback, which provided them with better opportunities to understand and to broaden their knowledge (Singh, 2010).However, BL in general requires commitment, activity and dedication.
In general terms, we can conclude that the experience was well-received, as shown by the PSEW.Whereas our results were satisfactory and could positively influence the use of educational platforms in specific subject areas such as Materials Science, we should address the question of why a relatively significant number of students did not use, or made little use of, the available resources.This question coincides with the findings of other authors (Lust et al., 2013), who state that only a minority of students used the tools available and that this use significantly affected the knowledge that they acquired.Furthermore, as in our case, other research studies concluded that a large number of students did not take any tests and that large differences could be perceived between the number of attempts and the timeframe of the attempts (McKenzie et al., 2013;Alducin-Ochoa & Vázquez-Martínez, 2016).
We view these findings as important for the fields of architecture and engineering, in which knowledge of Materials Science is required, albeit with differences, for all students.However, there is little research on BL in architecture and engineering as compared with other subjects (Francis & Shannon, 2013).
In future research, we propose replicating this study in successive years to verify whether our findings hold true.
We also wish to focus on the students' predominant strategies, styles and approaches and to establish the learning paths that the students follow, to identify which of these paths are effective and to relate them to other learning styles and approaches.

Conclusions
From the analysis of the results obtained, as well as from the discussion presented, the following conclusions can be drawn: a) The student's academic performance improves as the use of the forum and the self-evaluations found in WebCT increase.
b) The students who evaluated the WebCT resources with higher scores were the ones who obtained better academic results.c) As the degree of use of the forum and the self-evaluations increase, so do the scores given by the students on the content offered in WebCT.
Technical School of Construction Engineering of the University of Seville.Of these participants, 60 (34.09%) were women and 116 (65.91%) were men.According to age, 110 (62.50%) students were between 18 and 20 years of age, 40 (22.73%) between 21 and 25, 17 (9.66%) between 26 and 30 and 9 (5.11%) were over 30 years of age, with a mean of 21.25 years of age.The WebCT Usage Perception and Satisfaction Questionnaire (PSEW) were completed by 116 students.

Table 1 .
Descriptive statistics of grades by the degree of use of the assessment test

Table 2 .
.000, with the post-hoc Tukey honest significant difference (HSD) test confirmed that the significance resulted from the differences between the low and intermediate (p=.001), low and higher (p=.000) and intermediate and high degrees of use (p=.002);•As for the Unit 3 grades, a comparison between the mean of the low (M=4.13;SD=1.720), intermediate (M=4.61;SD=2.034) and high degree of use (M=5.20;SD=1.548),F (2,134)=3.470,p =.034, with the post hoc Tukey HSD test confirmed that the significance came from the differences between the low and high degrees of use (p =.045).ANOVA results for aggregate grades by the degree of use of the assessment test • When analyzing the Unit 1 grades, a comparison between the mean of the low (M=3.33;SD=2.178), intermediate (M=4.71;SD=2.054) and high degree of use of the test (M=5.88;SD=1.631),F(2,167)=22.728;p=

Table 3 .
Kruskal-Wallis test for aggregate grades by the degree of use of the assessment test

Table 4 .
Descriptive statistics of grades by forum degree of use

Table 6 .
45; SD=1.713), intermediate (M=4.52;SD=2.127)andhigh forum degrees of use (M=5.53;SD=1.411),F(2,81)=3.517;p=.034, with the post hoc minimum significant difference (MSD) test confirming that the significance resulted from the differences between the low and high (p=.025)and between the intermediate and high (p=.029)degrees of use.Kruskal-Wallis test for aggregate grades by forum degree of use

Table 7 .
Correlation coefficients between WebCT evaluation and academic performance

Table 8 .
Descriptive evaluations of the WebCT by the degree of use of the assessment test

Table 9 .
Kruskal-Wallis test for WebCT evaluation according to degree of use of assessment tests

Table 10 .
Descriptive statistics of the WebCT evaluation by degree of use of the forum

Table 11 .
Kruskal-Wallis test for WebCT evaluation according to degree of forum use