Simulation-Based Training: From a Traditional Course to Remote Learning-the COVID-19 Effect

The University of Bergamo switched from regular classes to online classes due to the COVID-19 pandemic during March 2020, without leaving to the students the chance to meet their teachers in the traditional setting even once. As such, this context represents a unique opportunity to compare the traditional courses, held in the years before, with remote learning. In this paper, we focus on the lessons learned from switching a project management course that combines traditional lectures with Simulation-Based Training (SBT) to an online course with the same structure, same curriculum and the same teaching team. Lessons learned are based on the opportunity to compare the two methods of teaching and their learning outcomes. Based on the analysis, conclusions about the future of this course and similar courses are presented.


Effective Teaching in the COVID-19 Era
The American Association of Higher Education's "Seven principles for good practice in undergraduate education" (Chickering & Gamson, 1987) focus on key elements that any training program should guarantee to achieve proper knowledge development. These are: 1) Encourage contacts between students and faculty.
2) Develop reciprocity and cooperation among students.
3) Use active learning techniques. 4) Give prompt feedback. 5) Emphasize time on task. 6) Communicate high expectations. 7) Respect diverse talents and ways of learning.
The recent lockdown due to the COVID-19 pandemic has dramatically challenged the opportunity to respect such guidelines. Contacts among people have been impossible limiting also cooperation among students. Providing prompt feedback is more challenging since the synchronous interaction is more complex and not as effective as through a personal and direct communication channel.
An extensive debate on the role of online learning has been refurbished and discussion about pros and cons and, more in general, of the learning effectiveness of such a solution, has become dominant in several fields.
Starting from these premises and from the personal experience of the authors during the development of teaching programs in the lockdown period, this work aims to provide evidence about the effects of online teaching perceived by university students and the potential effects that online platforms can have on Simulation-Based Training. The aim of this paper is, therefore, to present some insights about the transition from a traditional to an online setting while maintaining the same course structure, which includes simulation-based training.

Literature Background
Distance education practice and theory are not new. But distance education was defined by infrequent postal communication between student and teacher. The Internet and the Wide World Web changed it all and they are now the basis of modern online learning. The idea is to simultaneously improve quality and cost-effectiveness of the learning experience (Anderson, 2008). The COVID-19 outbreak made online learning the preferred solution for social distancing. In a recent article, Dhawan (2020) examined the attitudes of students towards distance learning university courses amid COVID-19 and found that online learning is difficult to implement in poor countries where the internet infrastructure is not well developed and many students do not have computers and internet connection.
In order to fill the distance created by the sudden switch to online learning, a flipped classroom methodology may help in stimulating students interaction, even online. Many studies (Tucker, 2012;Herreid and Schiller, 2013) explain the logic of the flipped classroom and how it is implemented in different forms. The influence of the flipped classroom on cooperation, innovation and task orientation was studied by Strayer (2012). All these studies show that traditional teaching by lectures can be improved by adding active learning components such as the flipped classroom methodology. Furthermore, using class time for active learning provides opportunities for greater teacher-to-student mentoring, peer-to-peer collaboration and cross-disciplinary engagement, as shown by Lichvar et al (2016).
In the context of Project Management courses, the subject of this study, active learning is often introduced by adopting simulators. The use of simulators to teach project management is well recorded. For example, Davidovitch and colleagues studied the transfer of learning from one simulated project environment to a different project scenario, the impact of breaks on the learning process and the forgetting that is associated with such breaks and the importance of functional fidelity on the learning process (Davidovitch et al., 2006(Davidovitch et al., , 2008(Davidovitch et al., , 2009). More recently, Perez (2015) studied the resource allocation and resource leveling problems encountered in project planning using a simulator to better understand the problem and its solution, thus contributing to highlight the benefits of simulations.
2) Provide opportunities to examine critically the assumptions and implications that underline various decisions.
3) Expose the nature of problems and possible solution paths. 4) Create an environment for learning that generates discovery learning. 5) Promote skills in communicating, role-taking problem solving, leading, and decision making. 6) Motivation and interest in a subject matter are increased. 7) Evidence is offered for increased retention, energizing the learning process and facilitating the understanding of relationships between knowledge areas within a subject.
8) The focus of the simulation approach is on the process of learning rather than on final results (e.g., preparations for taking a test), and on representing the reality of a situation. 9) With the use of simulation-based training, students can take risks without any unwanted or unfortunate consequences they could face in the real world managing a real project.
Working with a simulator provides the opportunity for reflective observation. Particularly in simulation-based training, the different scenarios and data items in the simulator enable the conceptualization and integration of ideas and models considered throughout the learning process. Moreover, the simulator provides a continuous and dynamic environment for active experimentation and is rapidly establishing itself as a critical factor influencing the transfer of learning (Alessi, 1988).

The Project Management Course at the Department of Management, Information and Production Engineering of the University of Bergamo
The Project and Innovation Management course (from now on referred to as PIM) at the University of Bergamo (Italy) is part of the master degree in Management Engineering. Every year, about 60 students, both Italian and international, are enrolled in this course. The course load is equal to 12 ECTS (European Credit Transfer and Accumulation System) and includes topics related to project management, such as the project life-cycle, project organization, managerial principles, planning and control methodologies, risk management, stakeholder management; and to innovation management, such as taxonomy of innovations and innovation dynamics. In terms of teaching methods, the course is structured with 50% of the total number of teaching hours organized as lectures about the theory, 30% as an application of the theory in case studies and analytical applications, 10% as team activities to be developed typically after the theoretical lectures and 10% as simulation-based training. The mixed structure was thought to address the different needs of the students in terms of learning styles, thus properly engaging them in the course. During the first lecture of the course, all the relevant information about the course structure and timetable, as well as the grading and evaluation are disclosed, to communicate the high expectation, as this is one of the most important courses in terms of workload and pedagogical impact. The different activities adopted during the course development are aimed at reaching specific goals. The application lectures are meant to encourage contacts between students and faculty and to provide them with prompt feedback about their understanding of the topics. The team activities are developed as an application of active learning techniques; in fact, after having provided the students with the theory and having them apply such theory in controlled situations (typically through case studies, as well as by providing additional material), they are asked to apply them in class during team activities. Finally, the simulation-based training is meant to develop reciprocity and cooperation among students, as they are asked to deliver a final assignment working in groups of five people. At the beginning of the course, the students are free to choose whether to follow the course full-time, following the previously-mentioned structure, or part-time, which entails an individual final assignment and the exclusion from the simulation-based training. As a result, only some of the students each year participate in simulation-based training.
The course evaluation is made of three components: a written exam, an oral exam and the evaluation of the simulation-based training. The written exam (40% of the final grade) includes multiple-choice questions, quantitative exercises and small case studies. The oral exam (40% of the final grade) may include questions both on the theoretical and the analytical part.
The grading for the simulation-based training part of the course (20% of the final grade) is based on two components:  Individual grades for simulation exercises performed in class.  Teamwork grading of the final assignment of the course.
The final assignment is organized around the development of a project scenario based on a real or fictitious project, analysis of the scenario, development of a project plan and execution of the plan under uncertain conditions in a simulated environment.
While the course is usually taught in a face-to-face setting (from now on "traditional setting"), due to the global pandemic, the course was taught in an online setting during Spring 2020.

Feedback from Students
We collected feedback from students through a survey submitted at the end of the course. The survey was structured as follows: i) qualitative opinions about the online vs. traditional teaching, ii) qualitative and quantitative opinions about the overall course, and iii) qualitative and quantitative opinions about the simulation-based training (the full questionnaire is provided in Appendix A). The quantitative opinions about the overall course were used to compare the results with the previous year, in which the course was provided in the traditional setting; 31 students responded to the survey in June 2020, while the previous year 40 responses were collected. In order to avoid any desirability bias, we submitted the questionnaire before the final exam and we collected anonymous responses. In terms of gender, the course attendees in 2020 were by the 70% male and 30% female. For what concerns nationality, 78% of the students were Italian, 10% foreign students enrolled in the full master program, 10% Erasmus student in exchange from European countries for just one term; the latter attended the classes from their home countries, thus having anyway the possibility to interact with a different cultural context. The majority of the students attending the courses and taking part in the simulation-based training had limited or no working experience, being them in the first year of their master program. It must be noted that, thanks to the online setting, also few part-time students were able to participate in the lessons and to the simulation-based training in 2020. Although, none of them had previous working experience in project management.

Qualitative Opinions about Online vs. Traditional Teaching
Concerning the preference between online and traditional teaching, half of the responding students claimed to prefer online courses (51.6 % of respondents), while one third claimed to prefer traditional courses (32.3%) and the rest was indifferent (16.1%). This means that the respondents were generally favorable to online teaching. By looking into details of which topics they would like to have in online or traditional settings, as expected, the majority of respondents answered that they prefer to have practical topics in the traditional setting and theoretical ones in the online setting. For example, to the question "What kind of courses do you prefer to have in the traditional setting?", one student replied "The more practical ones, for which it would be easier to ask for clarifications directly to the professor in the room. In the online setting, it would not be easy to communicate on real-time"; this may be due to the perceived distance from the teacher in the online setting. Also, another student stated, "Courses with fewer people attending, where a greater interaction with the professor is useful to deeply understand concepts, exercises and what are the latest news from research", demonstrating that there is some sort of peer pressure which is somehow exacerbated in the online setting. Moreover, it is important to highlight that some of the respondents were part-time students, who are working while studying; all of them declared to prefer the online setting for all the types of courses and topics. For example, one part-time student declared "I think that traditional teaching is better. But I am not able to attend for work reasons". Figure 1 shows the results concerning the students' preferences for online and traditional settings. Among the advantages of the online setting, students mentioned mostly the possibility to get the recording (indicated by 22 respondents over 31), as per one student's words "I can watch and rewatch the lesson when I prepare the exam"; time flexibility and management (stated by respectively 20 and 11 respondents), as one student stated "as a part-time student (full-time job) I love the flexibility given by online courses. This way I can watch the recorded lectures in my spare time"; avoiding the need for transport and travel (6 respondents), for example one student said that one of the advantages of the online setting is "not to have to come to University every day, saving transportation costs and time loss"; following lectures from a comfortable environment (5 respondents), as pointed out in the following statement "following lessons at home is much more comfortable, you don't have any problem of space" and improving notes and understanding (4 respondents), in fact, "taking notes is easier thanks to the "pause" option". From the other side, the main disadvantages were the lack of interaction with the teacher (20 respondents), as one student complained "Professors can't understand if students are confused and maybe change how they are explaining something", and with classmates (17 respondents), generally, students felt as "losing the interactive possibility and the human contact with other people"; technical issues (5 respondents), as stated by one student "the problems related to internet signals and the noise coming from the external environment in some case could damage the quality of the audio"; and the lack of motivation and concentration (4 respondents), in fact, the online setting "requires a lot of self-discipline". In the end, students were asked to indicate whether they thought the University should switch back to a traditional setting after the pandemic; interestingly only the 6% of respondents stated that they would like to come back completely to the traditional teaching methodology, meaning that the transition to the online setting was well managed and generally appreciated by the students. Figure 2 reports the details of the responses. Figure 2. Details of the responses to the question "Do you think that the university should switch back to the teaching methodology used before Covid-19?"

Opinions about the Overall Course
At the end of the semester, the University administers a university-wide feedback system. The questionnaire (see Section 2 in Appendix A) is used for all the courses and the data collected is used to evaluate the quality of the course and the instructors. This allowed us to compare results with the previous year.
The feedback from students that participated in the traditional course is summarized in Table 1, while Table 2 summarizes the feedback from students of the online course. The tables report the formation of the questions, the type of variable, the number of answers, the percentage of answers, respectively lower and higher or equal to 6 (in the Italian system perceived as "Sufficient") and the average of the responses. Interestingly, the average was slightly lower (even if the difference in means was not significant according to the t-tests) in the online setting for all what concerned the interest of the students (namely the ability of the teachers in stimulating interest and their own interest in the topic). Instead, the online setting gave better results in terms of the satisfaction with the lab activities (namely, the simulation-based training), in terms of teachers' availability beyond the lecture and the teachers' clarity. By performing a t-test, these last outcomes were significantly different (at the 99% and the 95% levels) from the means of the previous year.

Opinions about the Simulation-based Training
The feedback collected about the simulation-based training was based on the learning experience and the learning process. Concerning the learning experience, the students were highly satisfied with the knowledge transferred ex-ante (i.e., before the simulation-based lab) and with the effectiveness of the online sessions. Still, some students were not completely sure about the importance of the lab for the overall understanding. Finally, two groups of students can be distinguished concerning the preference of traditional vs. online setting: those who believed that the traditional setting would have performed better, and those who were convinced about the higher effectiveness of the online setting. Figure 3 reports the histograms of the results. Regarding the learning process, we asked students to indicate how much the simulation-based training contributed to their understanding of the different parts of the course (i.e., to the overall objectives of the course). In particular, we were able to assess that the simulation-based training was particularly helpful in gaining a systemic view of the project, while it contributed slightly less to transferring concepts related to the different modules of the course (namely, scheduling, cost planning, risk planning, and resource management). In sum, we can assume that, even in the online setting, the simulation-based training proved to be effective in connecting different topics and enabling the students to comprehend the overall picture. Figure 4 shows the histograms of the answers about the learning process.

Grades of the Students
At last, we were able to compare the average grades (in the Italian system assessed on a scale from 0 to 30, with 32 representing the summa cum laude grade) achieved in the overall course and the final assignment of the simulation-based training during the last 4 years. Among the last 4 years, from 2017 to 2019, both the course and the simulation-based lab were conducted in the traditional setting, while only in 2020 both of them were conducted in the online setting. From the graph reported in Figure 5, it is possible to notice how the results were slightly higher in 2020, both for the overall course (Final grade) and for the final assignment (Assignment evaluation), so we can conclude that students were not penalized by the online setting. The Analysis of Variance (ANOVA) conducted to test the difference among the means of the four years confirmed that the means were not statistically different, Table 3 reports the results of the analysis. The last column shows that the probability of having a value higher than the F statistics is 0.336, meaning that the four groups are not statistically different. Moreover, the χ 2 value with 3 degrees of freedom equals to 30.91 and the p-value is 0.000, meaning that the four groups (corresponding to the four years) do not have statistically different variance.

Conclusions and Recommendations
The analyzed data allows us to draw three main considerations about the use of simulation-based training in online settings.
First of all, simulation-based online training programs can indeed contribute positively to the learning experience. Students provided a specific and explicit assessment of the direct positive effect of this experience, but also the positive indirect effects this can have by helping teachers become more effective and more supportive during the learning process. The possibility to easily share with the teacher the specific issues the student is facing or to ask a very punctual clarification and receiving direct feedback has been extremely fruitful and valuable.
Second, students do not show a unique preference concerning traditional vs. online simulation training. We identify that some students find value in developing online learning. We argue that this is partially due to the novelty of applying an interactive lab through an online platform, but partially also to the learning experience provided. On the other side, some students consider of higher value to have simulation training in a traditional setting. Partially this is due to the contingent situation students faced during the lockdown period, but we argue that this is due also to the specific learning style students prefer. As Kolb (1981) suggests, different learning styles exist and individuals are characterized by personal preference also related to the specific background and the particular discipline considered.
Third, the development of the simulation activity online has not harmed the effectiveness of the learning process. On the contrary, the learning outcome (here measured by the final grades of the students) shows no sign of decrease compared to previous years, claiming that the online process is not less effective than a traditional setting.
Based on the data collected, we highly recommend that universities take a second look at the traditional way courses are taught. Online learning is not new, but for some reason, many universities offer traditional courses or online courses, but very few offer courses that simultaneously use the two teaching approaches. A careful evaluation of existing courses is highly recommended to identify those parts of the course that should be transferred to online teaching while other parts of these courses may be more suitable for traditional teaching. Further research should aim at developing specific recommendations on how to select the best candidates for such an evaluation and how to select the parts of these courses that should be taught online.
The study does not come without limitations, even if we managed to find a good case of an immediate switch from traditional to online setting, the unforeseeable nature of the considered event, the COVID-19 pandemic, did not allow us to collect complete data before the event and to conduct a proper design of the experiment. Although, we believe that, being Bergamo and Italy one of the first hit Western contexts, this is one of the best example enabling to compare the traditional setting with a completely online setting, with the transition managed during an emergency. The case presented in this article does not pretend to be the best way to manage the transition from traditional to online setting, but it is reported as an example of a successful transition, aiming to highlight the positive results that the online setting, and simulation-based training as a way to stimulate interaction and active learning, allows to achieve. o Yes, completely o Yes, but some courses can stay online o Yes, but some courses can be partially maintained online o No, it is better to maintain online classes

Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).