The Level of Critical and Analytical Thinking Skills among Electrical and Electronics Engineering Students , UKM

The high demand by the industry for graduates capable to critically analyze the causes, content and quality of information and using them effectively to identify and solve engineering problems, has been constantly and incessantly discussed. The inability to think analytically and critically contributes to the increased percentage of unemployed graduates. Additionally, the Malaysian students resort to memorizing and rote learning to find an easy way to get a degree and then find a job. This paper investigates the level of critical and analytical thinking skills among the students in the Department of Electrical, Electronics and Systems Engineering (EESE), Faculty of Engineering and Built Environment (FEBE), UKM. This study was conducted on a group of third year students in Semester 1 2010/2011 using three instruments; the analytical component of MTest model questions by the Malaysian Ministry of Education (MOE) in the selection of prospective students for Teachers College throughout the country, Marbach-Ad and Sokolove's taxonomy (MST) for student questions on a topic discussed in the lecture and the open-ended question posed in the final examination for the microprocessor and microcontroller course. Analysis based on these three techniques provide a rough estimation on the level of analytical and critical thinking skills among students and in this study, it was learned that the critical and analytical thinking skills among these students are at a very moderate level despite their high academic achievement.

Engineering Accreditation Council (EAC) and the Malaysian Qualifications Agency (MQA) stipulates that every engineering student should be able to identify, formulate and solve problems.This ability requires a high level of analytical and critical thinking skills.In 2006, it has been reported that 70% of graduates from public universities, 26% from private higher learning institutions and 34% of those graduated from foreign universities are still jobless (Suresh, 2006).It was also reported that the most frequent factors cited for unemployment of the local graduates are lack of communication and critical thinking skills (Ambigapathy & Aniswal, 2005).There is a need for students to think critically and analytically, in whatever discipline they are in.As stated by Scriven & Paul and Elder (2005), critical thinking should more than just to acquire knowledge or skills, it is to further develop and using the skills analytically.It has become a necessary skill where it improves thinking by creating the habit of reflection and questioning in every aspect of life (King, 1995 andScriven &Paul, 2005).
Fatin (2011) stated that in an interview conducted with 5 senior engineers and lecturers who have been working for more than 15 years, all engineers should be inquisitive, analytical and critical.This is undoubtedly true because all engineers has to deal with models through theoretical concepts, such as physical models of thermodynamic systems, electrical systems, control systems and many others.These models should be applied in a form that can be easily realized in a structured reasoning process.
In addition to addressing and solving technical issues, engineers should also be sentient to environmental, community, social and political issues and at the same time be proficient in economics and management aspects.They need to be able to identify with and examine all the issues that arise, make decisions, be ethical, possess good conduct of work, write report and documented results, and provide professional advice.In solving problems, they need to find the optimal solution taking into account of all aspects.Normally, every problem has constraint, uncertainties and various solutions and there are no formal techniques that can produce the best solution.The importance of analytical and critical thinking skills is evident here, in which engineers are expected to make an assessment by analogy and deduction, and provide justification on the choice made to reduce risk and to avoid failure.Teaching contents and methods should be aligned with the need for critical thinking, creativity, logic and reasoning so that students have the opportunity to develop their maximum potential.
In speech by the President of Institute of Engineers Malaysia (IEM), (Chuah, 2009) states that: "Universities need to provide their students with greater experiential breadth of preparation in a variety of non-technical areas and simultaneously developing in them the foundation they need in a complex evolving discipline.There is a pressing need for broad-based engineers who have the ability to think logically and analytically, and be able to make wise decisions based on fundamental principles of engineering." This paper discusses the study conducted on the third-year student in the EESE department, FEBE, UKM to assess the level of analytical and critical thinking skills using the three techniques; MTest model questions, SQ and open-ended examination questions.It also proposes programs and activities that could be implemented to enhance these identified skills.

Method
This study is to present findings from the qualitative and quantitative phase of a more comprehensive study in instilling critical and analytical thinking among electrical engineering students in UKM.The cohort comprise of third year students enrolled in Microprocessor and Microcomputer course in semester 1 session 2010/2011.For the purpose of evaluating their level of critical and analytical thinking skill, three instruments were used; the MTesT model questions, MST and open-ended questions asked in the final examination.The mixed-mode methods were selected to explore the assessment strategies to enable valid evaluation to take place.The analysis on the results of the MTesT and the open-ended question are compared to the analysis of MST to provide a better insight to the level of critical and analytical thinking skill among these students.

Microprocessor and Microcomputer Course
This course is appropriate for the study because the students need to adopt a deep approach to learning and interact with the content critically in order to integrate both the hardware and the software components in their design.In this course, the students learn about the fundamental concept of the microprocessor, internal architecture, programming model, functions and configuration of the pins.This will be followed by the programming concepts, in which the addressing modes and the instruction sets, machine and assembly languages and programming techniques will be taught in detail.This course also introduces the students to memory addressing techniques and I/O devices.The students will also acquire the knowledge on the operations and applications of serial and parallel peripherals, PPI and USART.Finally, the architecture of 8086 and the migration from 8-bit to 16-bit will be exposed to the students.
The fourteen week course focuses on the following course outcomes: 1) Ability to identify the basic architecture microprocessor and microcomputer systems and the relations between hardware and software.
2) Ability to describe the functions and basic concept of 8-bit and 16-bit microprocessors and understand the hardware and software design aspect.
3) Ability to apply the instruction sets in development of assembly language program and decoding of memory and I/O devices in interfacing circuits.4) Ability to interpret, analyze and troubleshoot assembly language programs.5) Ability to develop assembly language program for specific applications.

6) Ability to design and develop a complete microprocessor-based microcomputer system to solve engineering problems
These outcomes demonstrate the high level of critical and analytical thinking skill that the students should have to make the grade and to effectively prepare them in this industry.

Adapted MTeST Questions
MTeST is the acronym for Malaysian Teacher Selection Test and is used as the initial screening for candidates applying for a teaching program in Malaysia.The analytical component of this test consists of 20 multiple choice questions and the candidates are given 30 minutes to complete the answers.There are 4 choices to select from and this component aims to identify and select potential teacher candidates with analytical reasoning ability and critical thinking skill.
The model questions are based on Malaysian scenario which the students can relate to.

Example:
A teacher must select at least 8 students, Adib, Baha, Chandran, Dhalan, Elango, Farah, Ghani and Hamid to take part in a marching competition.
1) If Adib is selected, Baha should also be selected 2) Chandran and Dhalan should be selected together or otherwise 3) If Elango is selected, Dhalan should not be selected 4) Either Elango or Ghani, but not both should be selected 5) Hamid and Baha should not be selected together if Ghani is selected Which of the following can be the right list for the marching competition?A. Hamid, Ghani, Adib and Farah B. Hamid, Dhalan, Chandran, Farah C. Elango, Adib, Baha, Farah, Hamid D. Ghani, Chandran, Dhalan, Baha, Hamid From the example, it can be observed that by using the typical Malaysian names and scenario for which the students are familiar with, they could easily identify and relate to the situation and can answer the questions comfortably and with ease.

Marbach-Ad and Sokolove's Taxonomy
Marbach-Ad and Sokolave (2000) had carried out a study to investigate the type of written questions ask by their students.Their aim is to train the students to ask questions which are more insightful, deep and that are reflective of their inquisitiveness on the content.There are a number of reasons for the need to examine the type of questions the students asked, mainly, to learn whether inquiry is taking place.It was stated by Woodward (1992) that students' questions conveys the message that the science disciplines are areas where inquiry is a natural component and questions need constantly to be raised.Orr (1999) also mentioned that in an effective science education, students should ask the right questions and in return get the right answers.But what is most important is that by asking the right questions, it investigates deeply into certain theories and concepts, focus on the problem, it assist the students to develop sensitivity on clarity, accuracy and relativity of certain theories and concepts, and be aware of the allegations, evidences, conclusions, assumptions, implications, causes, concepts and interpretative views.All of these require deep, critical and analytical thinking skill.
MST was developed to indicate the different categories of questions the students might ask.Marbach-Ad and Sokolave (2000) claimed that Bloom's taxonomy (Bloom 1984) was used mainly to categorize the lecturer's questions rather than the students' questions.It is not designed to examine the level of how they assessed, relate and digest fundamental theories and concepts.This taxonomy is divided into 8 main categories as shown in Questions for which the answer is a functional or evolutionary explanation.

4
Questions in which the student seeks more information than is available in the textbook 5 Questions resulting from extended thought and synthesis of prior knowledge and information, often preceded by a summary, a paradox, or something puzzling 6 Questions that contain within them the kernel of a research hypothesis In this study, the questions that the students asked are categorize into two classes (category 1a and 3) which distinguish whether they are critical and analytical type of questions or otherwise.
In the methodology, during the introductory lecture in the first week of the semester, a lecturer states the basic concepts of the microprocessor.The students were then asked to write any questions that they might have in order to seek for more concrete information to reinforce their understanding on the concept.This approach is necessary because Malaysian students do not ask questions orally in class.The sample questions were collected, reviewed and analyzed to observe whether the questions can be evaluated as an in-depth critical thinking questions (good), superficial (artificial) as defined by the SQ or non-critical thinking questions (dead).From this assessment, an initial conclusion about the extent of the students' search for knowledge and inquisitiveness as well as their obsessions on the subject discussed in the class can be made.

Open-ended Questions
It was claimed that open-ended questions from the upper levels of Bloom taxonomy enable students to develop their critical thinking skill.Studies have also shown that open-ended questions call for divergent thinking whereby there will be a variety of possible answers.Therefore, to evaluate and at the same time to instill this skill, lecturers should design open-ended questions generously.
In microprocessor and microcomputer course, an open-ended question was asked in the final examination.In this question, a water level control system was illustrated and the students were required to describe the operation of the system.They were then asked to describe and draw their design ideas to interface the water level systems to the microprocessor via sensors and motors and comment on the limitation of their design if some parameters in the system were modified.In order to answer this question, the students will need to critically analyze the situation, apply the relevant knowledge and information, and deliberate their answer to indicate deep investigation into these particular theories and concepts.Although there are various possible answers, the evaluation is performed based on content analysis.Marks are given based on pre-determined criteria and in accordance with the marking scheme.

Results
Using the MST method, the written questions raised by the students were collected and analyzed.It was discovered that there are various types of questions posed by the students.There are a few identical questions.Some of the questions asked were generic and some were more focused to the main concepts of the microprocessor.These questions were then classified into the different MST categories, mainly to indicate whether critical thinking is taking place.
Good critical thinking questions are based from higher-order thinking, they arouse thought, deep, open-ended, require analytical thinking, related to the topics and questions that can generate the next question until a solution is obtained.In category 5 of MST, questions resulting from extended thought and synthesis of prior knowledge and information, often preceded by a summary, a paradox, or uncertainties fulfilled the criteria of critical thinking.Table 2 lists some of the questions that were collected and their categories based on MST analysis.In this study, the interest is mainly focussed on the questions that meet the criteria of critical thinking and classified in category 5. Questions number 1 and 2 was analyzed to be in category 5. Question 1 examined the implications and impact to the performance of a microprocessor if some parameters are changed.In this case the parameters of concerns are the number of bits in the data bus.This student is able to make his/her own hypothesis or assumption by the ability of his analytical thinking, generate new questions, and reflect on the inquiry process.In Question 2, the student have displayed his deep thought by seeking more information to clarify concepts, prepare for discussion, and practice critical thinking skills.
The questions in category 1a, 1b and 4 are examples of superficial questions that are shallow, generic, hovering on the surface and does not explore deep into the crux of the matter.This has demonstrated that, in most cases, these students did not think deeply about the content of the topic being studied.Although these questions are related to the topic of discussion, they are of descriptive type, which is at the most basic level in critical thinking model.
There are also students who asked questions which are too general are not relevant to the technical concept of the microprocessor.These questions are classified into category 4 which do not provoke the mind to generate fresh questions and further inquiries into the topic and therefore, is not a good example of critical thinking questions.These questions are said to be dead questions which show that students do not have the passion, enthusiasm and desire to really learn or think about the topics discussed.In short, these students have an inert mind.Table 3 indicates the result of the analysis performed on the questions asked by the students.From the result, it can be observed that the level of the critical and analytical thinking skill among the students is low.Only 2 from 70 questions asked can be claimed to contain high critical thinking elements and fulfilled the MST criteria.Majority of the questions can be classified as superficial with low level of critical thinking abilities.These questions do not motivate the mind to examine or think deeply and are ill-informed.The remainder 5 questions is more worrying because these type of questions signify that students are not showing interest, not serious and is not giving due attention to the topics taught in the class.
Nevertheless, this observation is only based on one particular cohort.More studies could be carried out for the students in year 1, 2 and 4 to verify this result.
In the second instrument, 48 students sat for the MTesT model questions.The students were asked to answer only 13 questions out of the 20 questions as 7 questions were tested to be invalid.The time given was 20 minutes.
It was analyzed that all of the questions with 4 choices of answers provide one correct answer based on the justifications given.It was also observed that out of the 4 choices, there could be answers that might be correct depending on one's interpretation.This study take into consideration of this condition and categorize the answers into correct, might be correct and wrong.This category further illustrates the level of critical and analytical thinking skill into high, moderate and low.
Figure 1 illustrates the three categories of answers given by the students for each question.For example, in question one, 27, 17 and 3 students provide the correct answer, might be correct answer and wrong answer respectively.It was noted that for questions 1, 4, 7, 8, 9, 12 and 13, the number of students getting the answers correct are higher.On the other hand, for questions 5, 6 and 11, more students are getting the wrong answers.The number of correct answers determines the 3 levels of critical and analytical thinking skill of the students.This is based on the students getting more than 75% correct answers, between 40% to 74.5% correct answers and below 39.5% correct answers.Table 4 illustrates this result.In the final examination at the end of the semester, an open-ended question was posed to explore the students' perceptions of critical thinking.The question is mandatory and consists of 4 parts.The maximum score for the part of interest are 20 marks.Table 5 indicates the students' performance and rank them in the respective level of critical thinking skills.In Figure 2, the result using the three instruments is as shown.It can be observed that most of the students ranked in medium level of critical and analytical thinking skill.While MTeST model and open-ended questions produced similar result with 56% and 50.8% respectively fitting into the medium level, the MST signifies that 90% of the students are within that band.The result that includes the average of the percentage of students belonging to the three levels using all the 3 instruments and using 2 instruments is shown in Table 6.The average of 2 instruments (mainly MTeST and Open-ended questions) is more conclusive since both are quantitative analysis compared to the MST which is of qualitative type.

Conclusion
In this study using MST, MTeST model questions and open-ended question to ascertain the level of critical and analytical skill among third year students in semester 1 session 2009/2010 in EES Department, based on the results signify that about 4% of students could be categorized as high, more than 50% in medium and more than 40% in low level of critical thinking skill.The outcomes suggest that the level of the critical and analytical thinking among this cohort lies between medium to low.The result using MST as the qualitative measuring tools is also made available.For MST result to be valid, more work has to be done to guarantee the percentage of students belonging to the rightful level.This includes the analysis made to correctly categorize the questions asked by the students.The other shortcomings that had been identified is the need to use a suitable instrument instead of MTeST which are found to be less convincing and to design more effective rubrics to evaluate the students' scores accurately.This finding will then be used to design a more efficient teaching and learning activities, for example problem-based learning, cooperative learning that is capable to enhance analytical and critical thinking skills of students.

Figure 1 .
Figure 1.Number of students getting correct, might be correct and wrong answers

Figure 2 .
Figure 2. Comparison on the result based on MST, MTeST and open-ended questions

Table 1 .
Table 1 that indicate the level of deep thinking which takes place.Marbach-Ad and Sokolove's taxonomy for 8 categories Category Items 0 Questions that do not make logical or grammatical sense, or are based on a basic misunderstanding or misconception, or do not fit in any other category.

Table 2 .
Analysis on the students' questions and their categories

Table 3 .
Level of critical and analytical thinking based on MST analysis

Table 4 .
Level of critical and analytical thinking based on MTeST score analysis

Table 5 .
Level of critical and analytical thinking based on open-ended score analysis

Table 6 .
Average of percentage of students in high, medium and low level of critical and analytical thinking skill