Quality Assurance in Higher Education Institutions : Exist Survey among Universiti Putra Malaysia Graduating Students

This paper presents the exit survey of graduating students at Universiti Putra Malaysia (UPM). The results gathered from 1,823 final year students of the 2006/07 session indicate that overall, the students’ satisfaction level is moderately high (3.55 ± 0.79). The students’ perception on the attributes of graduates resulting from learning outcomes is also moderately high (3.65 ± 0.66). Although there are no differences in students’ satisfaction level according to gender (t = .582, p > 0.05) and students’ residence (t = .121, p > 0.05), however, it differs according to students’ study programs (F = 35.44, p < 0.01), with Social Science students having a higher satisfaction level (3.80) compared to their counterparts in the Physical Sciences and Engineering (3.48) and Bioscience and Medicine (BSM) programs (3.37). Through this exit survey, together with many other assessment initiatives, the university aspires to provide the highest possible quality in terms of teaching, research and professional services.


Introduction
Quality is always a critical issue especially in higher education institutions (HEIs).It is expected that a quality higher education program could give a positive impact on human capital development of its graduates in terms of knowledge and generic skills (Fink, 2003;Walker, 2006).This down-to-earth expectation becomes increasingly contentious at a time when many HEIs are getting more scrutiny especially for public funding, resulting in generating various and often contrary spheres of knowledge (Biggs, 2003).The issue of quality becomes pertinent as the higher education sector is moving towards the third age era; a transformation from interpretation to generation, to commercialization of knowledge (Chartrand, 2008).The shift provides not only a wider opportunity for diverse people especially in developing countries to have access to higher education, but also invites eclectic and bewildering expectation of the quality of education programs that HEIs offer.
The term quality has been defined in many but contentious ways.Joseph Juran (2003) opined quality as "fitness for use".For Noriaki Kano (1984), quality relates to a product or service that meets customer expectation, possessing both "must-be quality" and "attractive quality".The American Society for Quality, on the other hand, coined that quality refers to the level of customer perception upon which a product or service fulfills customer expectation.Though quality is an elusive and subjective term that defies simple explanation, especially within the framework of HEI due to the changing nature of the term "quality" itself, quality can be defined in terms of value of a product or service rendered as perceived by the customer.These definitions imply that quality improvement in HEIs is a dynamic ongoing process and, to some extent, rely on the students' perception of the value being offered by the HEIs.
The HEIs, being a college or university ought to continuously improve their services based on the stakeholders' value.Stakeholders' perception of the HEIs is vital to improving teaching and learning as well as facilities and services being offered.While some quarters argued that quality of the HEIs should be determined by the academic experts (Lomas, 2007;Watty, 2006), most of the universities and colleges gathered information from their students as they are the ones who received the services rendered (Schneider, Russell & Niederjohn, 1995).Exit survey, for example, has been widely used to gathered information in many HEIs such as Central Queensland University, George Manson University, San Francisco State University, University of Western Sydney, and University of Wisconsin Madison.
In an effort to continuously improve her service quality, an exit survey is carried out by Universiti Putra Malaysia (UPM), one of the Malaysian leading research universities, every year on their final year students.The aim is to assess the quality of the academic programs and services offered as well as the students' satisfaction.The level of satisfaction is measured through their opinion on teaching and learning, management and facilities offered at faculties or colleges of UPM.Besides, the study also assesses students' attributes towards learning outcomes and soft skills elements instilled through the academic studies program offered by the university (Centre for Academic Development, 2007;Ministry of Higher Education, 2006).Through this exit survey, together with many other assessment initiatives, the university aspires to provide the highest possible quality in terms of teaching, research and professional services.

Methodology
This exit survey of graduating students is carried out yearly to assess the quality of the UPM academic programs and services.The survey also appraises the students' satisfaction level towards university services throughout the duration of their study at UPM.The instrument used was devised by a group of researchers with the cooperation of Deputy Deans (Academic) from each of the 16 faculties at UPM and administered by the Centre for Academic Development (CADe).It consists of 97 close ended items and three (3) open ended items.
The instrument is divided into five sections.Section A (13 items) focuses on the respondents' profile and Section B (2 items) on their current employability status.Section C consists of 43 items measuring three UPM service domains; teaching and learning experience, administration, and students' facilities.This section used a five point Likert scale, where the respondents need to state their level of agreement on the satisfaction level for each item using "1 = very low", "2 = low", "3 = moderately low", "4 = high" or "5 = very high".Items in Section D (38 items) assess the achievement of graduates' attribute domain in learning outcomes and soft skills.In this section, students need to identify their achievement in their program's learning outcomes based on eight (8) elements that have been identified by the Ministry of Higher Education (Ministry of Higher Education, 2006), namely knowledge; psychomotor, technical and practical skills; lifelong learning and information management; communication skills; thinking and scientific skills approach; management skills and entrepreneurship; social skills and sense of responsibility; and professionalism, values, attitudes and ethics.Section E consists of open ended questions focusing on respondents' perception of their strengths and weaknesses and also their suggestions on improving the study program and university facilities.
The instrument is distributed in the middle of the second semester to all final year students who were still in the UPM campus through the Deputy Deans (Academic) of the 16 faculties.For students who were on practical training outside the campus, for example students from the Faculty of Education who were undergoing the Teacher Training Program at schools, the instrument was mailed to them together with a stamped envelope direct to the school or institution throughout the country where the students were being placed.
The current study reports the analysis on the responses of 1,823 final year students out of an estimated 2,878 UPM graduating students (63.3%) of the 2006/07 academic session as presented in Table 1.This sample distribution of study program cluster reflects, to some extent, the UPM's final year students' population.
The factor analysis results of the original three UPM service domains employing the 1,823 responses indicate that all 43 items can be fitted into four new categories of students' satisfaction level; human-based, system-based, experience and facilities as illustrated in Table 2.It presents the distribution of the items on four categories of UPM services domain that have been identified using factor analysis.
The overall reliability index of the service domains using Cronbach Alpha is 0.92 with a range of 0.80 to 0.95 for each domain as shown in Table 3.The reliability index for the instrument according to study program cluster shows high alpha values with a range from 0.86 to 0.88 as demonstrated in Table 4.This shows that the instrument of the study is suitable for all UPM graduating students regardless of their study program.

Results and Discussions
This section presents the results of graduating students' satisfaction on the UPM four service domains, namely the human, system, learning experience and facilities.It also identifies the graduating students' attributes of learning outcomes and soft skills instilled through the UPM academic studies program.Besides, it discusses whether there are any differences in students' satisfaction level based on gender, type of accommodation and study program cluster.

Students' Satisfaction Level
Overall, the satisfaction level among final year students towards services provided by UPM is moderately high with a mean value of 3.55 (on a 5 point scale) and a standard deviation of 0.79 as presented in Table 5. Attention needs to be drawn especially to system based services and facilities in order to ensure the highest quality services are provided and giving the highest satisfaction to the UPM stakeholders.

Attributes Achievement of Graduating Students
The result indicates that final year students' perception of the attributes of graduates resulting from learning outcomes determined by the Ministry of Higher Education was moderately high with a mean value of 3.65 and a range from 3.59 to 3.75 for each learning outcome as indicated in Table 6.

Students' Satisfaction Level According to Gender, Study Program Cluster and Residence
The comparison of satisfaction level of graduating students according to gender, study program cluster and students' residence are presented in Table 7.

Conclusion
The aim of this study is to assess the graduating students' satisfaction on UPM academic programs and services as well as to identify whether their satisfaction depends on their gender, study program and residence.The study also appraises the students' attributes towards learning outcomes and the soft skills elements.Based on the results, generally the 2006/07 session graduating students are satisfied with the services provided.They also had moderately high perception on the achievement of learning outcomes and soft skills.
Over the years, this kind of stakeholders' assessments together with other appraisal methods have been used widely to inform HEIs of their quality (Miller, 2007).The HEIs could manage resources strategically and effectively by determining the strengths and weaknesses in each domain of their services.In the case of UPM, the exit survey of graduating students can enlighten us on what aspect of improvement should be taken in order for us to develop human capital as this goal is the most critical element in achieving the Malaysian Vision 2020 (Government of Malaysia, 2006Malaysia, , 2007)).Here, human capital development encompasses a holistic acquisition of knowledge, skills and attitudes, complemented by soft skills and entrepreneurial capabilities (Ministry of Higher Education, 2007).
Among the current initiatives undertaken by UPM, partly as a result of these appraisals is to review its academic curriculum by inculcating learning outcomes and soft skills elements (Bancino & Zevalkink, 2007;Centre for Academic Development, 2007;De La Harpe, Radloff & Wyber, 2000).In parallel, UPM has also taken bold steps towards promoting student-centered teaching as well as encouraging alternative continuous assessment approaches.Together with continuous improvements in other aspects of human, system, experience and facilities based services, these quality assurance initiatives are consistent with the First Goal of UPM's Strategic Plan 2001-2010 which is to produce quality graduates who are competitive and resilient through lifelong learning.Students are generally satisfied with services provided by UPM in the aspect of learning experiences gained during their studies such as in teaching and learning and job related (mean = 3.70), human based services such as lecturers, support staff and academic advisor (mean = 3.63), system based services such as administration of teaching and learning (mean = 3.48), followed by facilities offered (mean = 3.38).Detailed analysis of the items indicates that the campus cafeteria services appear to have the lowest mean value of 3.01 with a standard deviation of 0.97.As a whole, graduating students of the 2006/07 session are satisfied with the services offered by UPM.The highest score was for professionalism, values, attitudes and ethics with a mean value of 3.75, followed by social skills and responsibility with a mean value of 3.71.Communication skills and thinking and scientific approach skills appeared to have the lowest mean value of 3.59 compared to the other learning outcomes.

Table 1 .
Distribution of Respondents According to Program Cluster

Table 5 .
Graduating Students' Satisfaction Level of Services Provided by UPM

Table 6 .
Mean for Graduates' Attributes Achievement

Table 7 .
Comparison of Graduating Students' Satisfaction Level According To Gender, Program Cluster and ResidenceThere are no significant differences in the satisfaction level towards program and services offered by UPM between students' gender (t = .582,p > 0.05) and residence (t = .121,p > 0.05).However, there is a highly significance difference in the satisfaction level between students of different study programs (F = 35.44,p < 0.01).The post hoc Bonferroni analysis shows that students in the Social Science (SS) program have a significantly higher satisfaction level (mean = 3.80) than students in the Physical Sciences and Engineering program (PSE) (mean = 3.48) and Bioscience and Medicine (BSM) program (mean = 3.37).