Evaluation of Pre-assessment Method on Improving Students Performance in Complex Analysis Course

Engineering mathematics has always been the fundamental and important courses in engineering curriculum. Engineering students are required to understand the fundamental of mathematics and apply this knowledge to solve real world problem. The requirement for engineering mathematics for the different branches of engineering is more or less the same at the first and second year level but tend to be more specific and complicated at the later years. Problem started to occur when students lack the fundamental knowledge of mathematics and unable to grasp the higher level of mathematics. One such course is Complex Analysis, which is one of the compulsory course for the third year electrical engineering students at the University Kebangsaan Malaysia, UKM. The course requires the students to be able to understand, analyse and apply the complex concepts and techniques in solving practical electrical engineering problem. However, previous results for different batch of students revealed that most students taking the course had fundamental problem in understanding the new concept although such concepts were built upon on the fundamental mathematics learned in the first year. Thus the objective of this study is to improve student’s performance by assessing systematically student’s level of understanding on a particular topic in the complex analysis course using the Rasch measurement technique. Students were given pre-midsem test with the combination of different question related to the learning outcomes of the course. The result of the pre-midsem test were then analysed using the Rasch measurement and the correlation level between the performance of each student and question was identified..Rasch measurement was able evaluate the validity of the intended question given and classifying the students according to their level of understanding on the course.. Preliminary findings indicated that majority of the students have problems with contour integral especially the use of the Cauchy-Goursat theorem in solving application problem. With this early identification, the existing method of teaching on the particular topics needs to be adjusted and re-evaluated. From this study, some suggestions were put toward for future improvement in the teaching and learning of the Complex Analysis course.


Introduction
Mathematics is a key element in engineering studies and serves as language of expressing physical, chemical and engineering laws (Sazhin, 1998).The understanding of fundamental concepts and ideas in engineering mathematic is very crucial for mastering engineering discipline.Having strong foundation in mathematics for an engineering student is very important to gauge success in engineering.The objective of teaching mathematics to engineering students is to find the right balance between practical applications of mathematical equations and in-depth understanding of living situation (Sazhin, 1998).In 1995 the Engineering Council has published the results of difficulties in mathematical skills that undergraduate engineers faced in studying mathematics.They are having difficulties in achieving higher grade especially in mathematic courses.This explains the growing challenge of undergraduates being accepted for degree courses with relatively low mathematics qualifications.One of the main reason is the lacking of fundamental knowledge especially in understanding the theories (Sutherland & Pozzi, 1995).The decline of the student performance is further worsen with the lacking in mathematical skills among the students.Skill requires practice and the right practice in turn requires strong determination from oneself together with the right and skilful educator.
A lot of researches were conducted in order to find the reason why student's performance on the mathematical courses was beyond below the expected results.From the general perspectives of education systems, there are many factors due to this problem and one of them is the transition of mathematical knowledge from school level to pre-university and lastly to university level through syllabus of the mathematical courses.The students were taught with different approach in the mathematical course and inconsistently with the syllabus between school, pre-university and university level.Other factor that contributes to this low mathematical achievement among engineering students is the students' attitudes towards mathematics courses.Students tend to have negative attitudes when they are facing with mathematical problems in their studies.Studies by Yushau (2006) have shown that attitude is one of the other reasons why learning mathematics is considered difficult.Internationally, other researchers were concerned about the knowledge and skill level of students studying engineering mathematics.Similar concerns were shared by Gardner and Brodus (1990) who worked with US College students and found that mathematics was the basic culprit in undermining student academic progress.Thus it very important to investigate the factors affecting the quality of mathematical understanding and to assess the validity of the assessment being carried out.
In the effort of improving the performance of engineering mathematics among the students in the University Kebangsaan Malaysia, several high level courses in engineering mathematics were identified One of the them is the Complex Analysis course which is taught only for the Electrical Engineering students.As a high level engineering mathematics course, the content requires student to be able to comprehend complex and delicate concept on mathematics.Complex Analysis is a compulsory 3 credit hour subject for the third year electrical engineering students.The aim of this course is to provide the students on the analysis of Complex function to be applied later in their respective engineering subject.Results from previous semesters indicate that students on average obtained lower grade in the Complex Analysis courses and initial investigation revealed that the students were mostly having problem in the fundamental concept and thus difficulties in applying the concept to solve real life problem.Study by Fadiah, Noorhelyna, Haliza and Zulkilfi (2009) showed that result obtained during pre university level did not represent the result of overall ability of student to grasp the complex ideas in mathematics.One way to identify student's performance is by assessing the student through pre-set of questions containing several topics under observation.The answers from this pre-set question are then analysis for any relationship that exists among them.Several methods are available to evaluate the correlation between the performance of each students and quality of questions posed to students.Rasch measurement is one of the reliable and appropriate methods in assessing student's ability (Rasch, 1960(Rasch, /1980)).Ghulman and Masodi (2009) stated that Rasch measurement is useful with its predictive feature to overcome missing data.Rozeha, Azami and Saifudin (2007) concluded that Rasch Model Person-Item Distribution Map (PIDM) provides meaningful information on the student's learning effectiveness.
Hence, the purpose of this study is to improve student performance by identifying the course content and topics which are considered difficult to the students in the complex analysis course .A pre-mid test exam was administrated and the result was analysed using Rasch measurement.Rasch measurement is capable to analyse whether the questions given are measuring what they are intended to measure.Topics which were categorised as difficult will be given extra attention later during the lesson.Each of the questions was assessed by topic in complex analysis course as shown in Table 1.Individual marks are recorded based on the student's answer.

Step 2: Standardization Method
Each question carries a different total mark and students have to answers all the questions.Since these questions have different total marks for each question, the standardization method were used.The formula for standardization is given in Equation ( 1): (1) where: i: i-th students (i=1,2.....48) j: j-th question (j=1,2....10) ij z : standardized marks for i-th student and j-th question ij x : Marks for i-th student and j-th question Total j x : Total marks for j-th question The student's standardized marks were then analysed using a rating scale in Table 2 where students were rated according to their achievement.The highest rating (scale 5) is the highest marks students obtained.

Conversion of Standardized Mark into Winstep Format
This grade rating is tabulated in Excel format.This numerical analysis is required for evaluation of the students' performance using Rasch software, Winstep.The analysis outputs obtained from the Winstep were then analyzed.

Result
The results from the test were tabulated and run in Winstep; a Rasch analysis software to obtain the logit values.
The common analysis outputs from Winstep are summary statistics of person and item, Person-Item Distribution Map (PIDM), item measure and person measure.Each of summary statistics has its own function in extracting in the information from the collected data.

Summary Statistics: Person Measure
Figure 2 shows a fair person spread of 1.64 logit (spread between maximum measured persons 0.65 to minimum measured person -0.99) with Separation, G=0.92 and fair Reliability of Cronbach-α = 0.56 which is below the acceptable level 0.6.The value of Separation is low which means less variability of person on the trait.
The major finding is the Person Mean, μ Person = -0.29 logit which is lower than the value Mean item = 0.These values show that the students were found to be below the expected performance in answering the questions.Only 18.75% (N=9) of student can be categorized as good student (above Mean item ) while 81.25% (N=38) were categorized poor student (below Mean item ).The students have difficulties in answering the entire questions except for question q1-basic operation of complex number which can be considered as easy question.
Figure 2. Summary statistic: Person measure

Summary Statistics: Item Measure
The Item summary (Fig. 3) gives a good summary with high Reliability = 0.93.The index Separation, G=3.77 indicates that there are 3 groups classifiable from the question which are easy, moderate and difficult.It has item spread of 1.63 logit (spread between maximum measured items 0.67 to minimum measured items -0.96) with SDi= 0.52.There are huge gap on the both end of very difficult and very easy item which needs attention.The PIDM shows a better picture on how the student correlates to the respective questions.It can give a clearer view of the person's ability and relevant item difficulty.A higher ranking indicates that the item was more difficult.The orthogonal arrow in the figure shows the gap between the two items.The wider the gap ( ), the more difficulty the students encountered when attempting to answer the question.q5 and q2 is the largest gap, which means that the q5 and q2 were difficult question.By looking at figure 5, one student has the lowest score (-0.99 logit) who can be categorized as a student with the poorest ability.

Conclusion
Based on this analysis, the overall student's performance in this Complex Analysis course is below expectations where on average student ability level is very low.Almost all of the students have problem answering most of the questions.The summary on the Person Mean, μ Person = -0.29 logit is lower than the value Mean item = 0.For this Complex Analysis course, students have difficulty to understand topic on Contour Integral especially the use of the Cauchy-Goursat theorem.Using misfit validity, there were some good students who cannot answer easy question where the weakest student was found to have the ability level below the minimum of item.The inconsistency has help instructor to reconstruct later a proper exam questions and improved the teaching method for the benefits of the students.
With this early identification, the existing method of teaching on the particular topics needs to be re-evaluated and problematic students should be identified and given more attention.For further work, real midsem will be analysed using Rasch measurement and the results are to be compared with the pre-midsem results.

Figure 1
Figure1shows the process flow in this study which involves 4 different stages.First stage involves the design of Pre-Mid test questions.The Pre-Midtest questions were carefully constructed as to cover the necessary syllabus in the class, The format of the questions was subjective typed questions.The Pre-Midtest questions on Complex Analysis course consist of 10 questions which were administered on 3rd year students in electrical and electronic department (JKEES) academic session 2011/2012.A second stage data was collected for all 48 students who were registered for this course.The third stage is the analysis process which involves with the used Rasch measurement.The analysis process consists of 3 steps: the construction topic of each questions, standardization method and Conversion of Standardized mark into Winstep format.Finally the last stage concludes the analysis output.

Table 1 .
Pre-midtest question and topic

Table 2 .
Rating scale