Curriculum Development Based on the Big Picture Assessment of the Mechanical Engineering Program

One of the major concerns of the Engineering Accreditation Council (EAC) is the need for an effective monitoring and evaluation of program outcome domains that can be associated with courses taught under the Mechanical Engineering program. However, an effective monitoring method that can determine the results of each program outcome using Bloom’s Taxonomy has not yet been established for each course. The purpose of this research is to conduct a Big Picture Assessment to achieve Outcome-Based Learning. Big Picture Assessment is a comprehensive monitoring tool of courses with studied program outcome domains. The tool applies the three main domains of Bloom’s Taxonomy, namely, psychomotor, cognitive, and affective, in its monitoring process. Furthermore, the identification of program outcomes for each course is evaluated to meet standards set by the EAC. The results of this study will facilitate continuous improvement on existing courses.


Introduction
The Faculty of Engineering and the Built Environment (FKAB) of the Universiti Kebangsaan Malaysia (UKM) prepare themselves to meet the criteria set by the Enginerring Accreditation Council (EAC), which acts as a recognition body of engineering programs in Malaysia.This undertaking is aimed at fulfilling the aspirations of UKM in becoming a leading research university with an accredited study program.In 2004, all engineering programs in public higher education institutions in Malaysia have begun to implement the Outcome-Based Education curriculum (OBE) (Shahrir et al., 2008).In line with the implementation of OBE into the engineering program, emphasis on the curriculum and the methods of delivery, assessment, and measurement is very important and should always be continued to ensure that the Program Education Objectives and Program Outcomes (PO) can be applied in teaching and learning.One of the requirements seriously considered by the EAC is an effective assessment of PO domains that can be associated with courses taught in engineering programs.(EAC, 2012) The purpose of this paper is to conduct the Big Picture Assessment on the mechanical engineering (KM) program in UKM and include the findings in the EAC self-assesment report.Big Picture Assessment is a comprehensive monitoring tool that can be used to assess all KM courses of study with applied domain POs.The results for each course program are assessed to determine if these programs meet the standards set by the EAC to facilitate improvements when needed.

Methodology
Big Picture Rating was developed by Excel software.Mapping between POs and KM courses taken in each semester was created.The maximum number of POs used was only 9; before 2010, the number was 12. Table 1 shows the details and definition of each PO.Meanwhile, the courses were matched based on the existing list in the FKAB Undergraduate Handbook 2011-2012.Given that the focus was only on the compulsory courses of the KM programme, courses such as electives, engineering fundamentals, and general education were not considered.(Aqilah et al., 2008).Each course offered does have certain course outcomes based on the results of the cognitive domain in Bloom's taxonomy.Initiatives have been taken to update all the results of the course based on the cognitive domain of Bloom's taxonomy for engineering programs (Hairi et al., 2008).The cognitive domain includes knowledge, comprehension, and critical thinking about a certain topic.Traditional education tends to emphasize the skills in this domain.The cognitive domain consists of a number of the learning processes that should be implemented, such as the process of remembering, understanding, applying, analyzing, evaluating, and creating.The ordering of the processes is illustrated in Figure 1.

A5 -Characterizing by Value
A4-Organizing and Conceptualizing  The psychomotor domain includes the processes of perception, set, guided response, mechanism, complex overt reaction, adaptation, and origination, as shown in Figure 3.

Results and Discussion
The matrix between POs and KM courses that reflect the Big Picture Assessment of this program are displayed in one sheet only.However, for this paper, the matrix is divided into two phases, from the 1 st to the 4 th semester, and from the 5 th to the 8 th semester.Figure 4 shows the matrix from the 1 st to the 4 th semester.The majority of the KM courses emphasized on PO1, PO2, PO5, PO6, and PO7.Only a few of courses incorporated PO3.Based on the details of these POs, much emphasis has been made on basic skills in science and engineering, environmental issues, and the application of knowledge.These matters are given more focus in most courses, enabling students to form a strong foundation during their 1 st and 2 nd years before continuing into their 3 rd and 4 th years of study.In terms of Bloom's Taxonomy, all the courses successfully incorporated all three domains.Most courses incorporated C3 to C5 of the cognitive domain.For the psychomotor domain, the majority of courses incorporated P3 and P5.Meanwhile, for the affective domains, many courses included A3 and A4.Thus, several of these courses dealt with these domains on an intermediate level and rarely incorporated the highest levels during the 1 st and 2 nd years of study.For the assessment method adopted, most courses used e (examination), a (assignment), p (project), r (report), and l (lab).Only certain courses included o (oral presentation) as one of their assessment methods.Thus, the majority of courses used more traditional evaluation methods that are commonly used in evaluating basic POs and that are less difficult in assessing their students.Figure 5 shows the courses taken from the 5 th to the 8 th semester (3 rd and 4 th years).Many courses incorporated PO1 to PO8.Thus, 3 rd and 4 th year students were tested on all POs compared with 1 st and 2 nd year students.In terms of the cognitive domain, many courses included C5 and C6.For the psychomotor domain P3, P4, P5, and P7 were used, and, in the affective domain, A3 to A5.Thus, when the students enter their 3 rd and 4 th years, all of the highest levels of all three domains are covered.Furthermore, the results are consistent with previous studies in indicating that more than 90% of curricula in institutes of higher learning emphasize cognitive skills over psychomotor ones (Maizura et al. 2008).Furthermore, during this period, students have to undergo a more challenging evaluation in the form of PBL and case studies.Even an oral presentation is more frequent than in the 1 st and 2 nd years.Thus, students in their final years are exposed to actual problems faced in the engineering industry and become more capable of providing solutions in the various mechanical engineering fields.Figure 5. Big picture for KM from the 5 th to the 8 th semester

Conclusion
The evaluation of the prevailing curriculum used in KM courses by Big Picture Assessment was successfully conducted.The majority of the courses under the mechanical engineering program of UKM incorporated PO1, PO2, PO5, PO6, and PO7 during the 1 st and 2 nd years, whereas the remaining levels were covered in the last 2 years of the program.The students were also tested on the highest levels of the three domains of Bloom's Taxonomy during their 3 rd and 4 th years.However, based on the results, most of the courses tended to use the cognitive domain as the primary evaluation method over the affective and psychomotor domains.Lecturers were also more likely to insert the element of PBL and case studies in the final year of the course.Comparisons with the manufacturing program are proposed for future studies to achieve a more thorough analysis.

Figure
Figure 3. Psychomotor domain

Figure 4 .
Figure 4. Big picture for KM from the 1 st to the 4 th semester

Table 1 .
Details of POs (source: FKAB Undergraduate Handbook 2011-2012) Each filled matrix was evaluated according to Bloom's Taxonomy for each KM course.Introduced in 1956, Bloom's Taxonomy categorizes skills and objectives to be achieved by the students into three major domains, namely, cognitive, affective, and psychomotor PO8Recognition of the need for and an ability to engage in lifelong learning PO9 Knowledgeable in project management, administration, business acumen, and entrepreneurshipIn the matrix assessment, two color indicators, red and yellow, were used.Red represents a high correlation, while yellow means moderate correlation.If no color is indicated, the PO and course have a low or almost no correlation.All assessment criteria were listed in each color matrix, which describe how the POs were evaluated for each course.Table2lists the assessment criteria for evaluating the POs-KM courses matrix.