The Evaluative Impact of Graduate Scholarships : The Case of the University of the West Indies , St . Augustine

The UWI Postgraduate Scholarship has been the most longstanding award offered by the University of the West Indies. However, completion rates of students have been decreasing and the number of students registered has not been increasing significantly. This paper investigates the UWI Postgraduate Scholarships awarded from 2001 to 2007. This evaluation was done in order to ascertain whether this award was serving its intended purpose, that is, promoting research and creating new knowledge. The UWI Postgraduate Scholarship does encourage persons to pursue studies full-time. It is very sound in its structure, management and administration. However, more student support systems need to be put in place and student progress needs to be more carefully monitored so as to increase throughput rates.


Background and History of the Problem
The University of the West Indies (UWI) is one of the leading providers of higher education in the Caribbean region with regard to the quality and academic rigour of its programmes (UWI Strategic Plan 2007-12).The UWI St. Augustine campus offers over 90 postgraduate programmes at the Masters and PhD levels and this number is increasing.The UWI Postgraduate Scholarships are intended to encourage and provide students with an incentive to undertake full-time research.These scholarships are also intended to attract high caliber students, foster research, innovation and development on the campus and by extension, the community and the world at large.

The completion levels and approximate dollar values from 2001 to 2007
The number of awards granted each year varies.The throughput (completion) levels (students who graduated/ were awarded), are presented below.Figure 1   The approximate dollar value will also be presented for the number of scholarships accepted by students, for students who have voluntarily withdrawn, did not re-register, graduated/awarded and those that are still currently registered in their respective degree programmes for scholarships awarded from 2001 to 2007.
The Figure 2  This study has become necessary as the throughput or completion levels of these students have dwindled according to statistical evidence.The statistics also uncovered evidence regarding the attrition rates of graduate research students who receive these awards for the same period.To drive the necessity of this study even further, the University subsequently increased the number of scholarships awarded in the 2007/2008 and 2008/09 academic years.These awards were granted without a prior assessment and evaluation of the status of its existing awards i.e., from 2001 to 2007.The paper will therefore seek to provide an investigation into the efficiency and effectiveness of the UWI Postgraduate Scholarships.

Objectives of this study
The objectives of the study are to investigate the efficiency and effectiveness of the UWI Postgraduate Scholarships from 2001 to 2007 and will: (i) examine the throughput levels of the This study sets forth two (2) hypotheses, as follows: 1.The UWI Postgraduate Scholarship facilitate full-time study for students' research which would improve the throughput levels of graduate research students 2. The students awarded the UWI Postgraduate Scholarship are provided with an incentive to pursue a research degree resulting in commitment and reduced likelihood of dropout as well as regular and consistent feedback from their supervisor.
The figures above represent the increased number of awards and opportunities by the University to research students.Similar to the 2007/08 academic year, the stipend for the 2008/09 and 2009/10 academic years was TT$60,000.00per annum and the awards were granted for two (2) years.Therefore, the University's investment is total of approximately TT$11,160,000.00 for these two (2) academic years.

Significance of the study
This study is presently significant as the University indicated its financial constraints as well as the fact that it would uncover the nature of the University's return on investment from the number of scholarships awarded by the university.The research could therefore be used for planning purposes as well as provide a signal to the management whether intervention is necessary.This intervention is intended to uncover concerns and assist the UWI Postgraduate Scholarship students with the timely completion of their theses resulting in the increased throughput levels and reduced attrition rates.

Literature Review
Williams and Harvey (1985) raised concerns regarding higher education in Trinidad and Tobago.However, they did underscore the fact that the restructuring of UWI is a major indicator of the awareness of the educational leadership in Trinidad and Tobago as well as the rest of the Caribbean region and of the need to make these institutions more meaningful instruments of national development.
Higher education institutions in the Caribbean need to take the lead in promoting the connections between Caribbean society, industry and the knowledge economy which drive the development process.They must act as conduits for new ideas and develop their abilities as organizations that transfer knowledge to students through teaching and to business, government and the wider community through technology transfer, informed policy development, commentary and publication (Prichard 2000).

Quality in Higher Education
The University of the West Indies competes with higher education institutions worldwide i.e., locally, regionally and internationally as (Dew, 2009) explains that quality higher education should incorporate:  the duration of time an organisation is in existence,  its physical structure and space which is intended to provide an environment conducive to learning and  conformance to requirements which highlight specific attributes or characteristics to be achieved.
Quality should mean achieving the fastest rate of innovation and improvement in all aspects of an institution which should add value to the consumer.Therefore, completing a degree should mean some measurable improvement in student learning, social skills, social contacts, writing skills, reading skills, critical thinking, or other attributes that are consistent with the mission of the institution.
Ekong (1998) on the other hand indicates that quality in higher education should be approached via quality assurance which implies that all the policies, systems and processes are directed to ensuring the maintenance and enhancement of the quality of educational provision within the institution.A quality assurance system is the means by which an institution confirms to itself and others that the conditions are in place for students to achieve the standards that the institution has set.(Kvavik et al., 2005) state that if one is going to look for a return on investment in higher education, he/she could not focus solely on efficiency.The focus should be put on effectiveness, customer satisfaction, and accountability.However, there are many factors that could affect financial performance.For instance, these factors include quality of service, operation overheads, demand for products or service, stage in product life cycle, productivity, and customer satisfaction (Kanji et al., 1999).
The customers of Higher Education Institutions (HEIs) are divided into different groups of actors that affect the education process, namely: existing and potential students, employees, employers, government, and industry.
Owing to their different characteristics, they exert certain demands that affect the behaviour of the education system (Kanji et al., 1999).
As such, to achieve high customer satisfaction level, organisations have to continuously improve various aspects of their operation.This could be achieved through leadership by making decisions on objective evidence of what would actually be happening (i.e., management by fact), and by involving all employees in quality improvement activities (i.e., people-based management), leading ultimately to business excellence.He further states that leadership serves as a prime factor in this model that must be transmitted through all the principles and core concepts in order to achieve business excellence.

Best Practices Locally and Internationally
The Best Practice Research Scholarship Programme (BPRS) The Best Practice Research Scholarship programme (BPRS) was one of a series of initiatives in the United Kingdom between 2000 and 2003, to support teachers' continuing professional development.Approximately 1,000 Scholarships of up to £3,000 each were awarded to serving classroom teachers to engage in supported, school-focused research during this period.The aims of the scholarships were threefold, i.e.,  develop their professional knowledge, understanding and confidence;  enhance their professional practice; and  engage and encourage the sharing of effective practice and professional knowledge within the teacher's school and wider educational community.
The BPRS scheme programme was a government initiative entitled 'Teachers: Meeting the Challenge of Change' which officially began in September 2000.This was due to the fact that the profession faced constant change, in which teachers and schools we given the task to create learning institutions for their staff as well as their students.
The BPRS scheme was therefore intended to be a form of officially funded 'action research'.Based on the guidelines, researchers needed to uncover whether the action research was:  situational as this implies a commitment to diagnosing problems in specific contexts and attempting to solve them in those contexts. collaborative and participatory i.e., with practitioners themselves having an active role in the research. self-evaluative because the ultimate aim is to improve practice in some way, the context under study is continuously evaluated, and practice, as well as the research design, constantly modified as the project moves The study involved an evaluation of in-house work by teachers in various areas of interest regarding their students and schools.The projects were evaluated based on three areas, the first being documentary analysis i.e., the year in which information was readily available -2001-2002 (with an acceptance rate of 66 % for the scholarships).The second was case studies in which 100 cases were randomly selected for investigation and the third referred to additional evidence.
The study evaluated the BPRS scheme in terms of the contribution of individual projects to:  teachers' own learning (their continuing professional development) (CPD)  the development of their practice in the specific contexts in which they work The impact of the BPRS scheme on the teachers was a very valuable form of continuing professional development (CPD) as it resulted in enhanced confidence skills and knowledge.In addition, it also impacted on the teachers' practice with regard to their teaching, school, parents and students.The BPRS mentoring programme was also of critical concern as it offered support to students in the form of research training, expertise, dissemination support, pastoral care and detailed tutoring.The mentors assisted in shaping initial ideas, research design, research training, access to expert knowledge and support in dissemination of information.
Other factors that led to the success of the programme were money i.e., the student's stipend which allowed for full-time focus on the work for the entire year as well as a supportive Senior Management Team at one's school.One of the biggest concerns was the need to develop effective criteria to judge the quality of the projects.

Best Practice at the Northern Arizona University
At the Northern Arizona University (NAU), (Butcher et al 2009) proposes an Annual Assessment Fair which has been an integral part of linking faculty, staff, and students together in the assessment process resulting in a campus-wide activity.Faculty, staff, and students from departments throughout the university meet to learn about the results of individual and departmental assessment efforts aimed at improving student success at this University.
The Assessment Fair is an opportunity to build connections in the NAU community.It is an opportunity to observe what others are doing, share accomplishments, and think critically about programmes and services.This event showcases how assessment is being used across campus to improve student learning, gauge student satisfaction, evaluate student services, and otherwise contribute to work at the university.The writers also gave tips for implementing the process as well as ideas to overcome barriers to assessing student learning and development.

Best Practice -Government Assistance for Tuition Expenses (GATE) from the Government of Trinidad and Tobago
The Government of Trinidad and Tobago also contributes to the creation of knowledge economy and support students via their Students pursuing postgraduate programmes at local and regional public Tertiary Level Institutions (TLIs) will complete the GATE Application Form to access funding for 50% of their tuition fees.Students pursuing approved postgraduate programmes at approved local private TLIs will complete the GATE application form to access funding for 50% of their tuition fees to a maximum of $10,000.00; or $5,000.00for approved Distance Learning programmes.

Summary
This section provided a foundation for demonstrating how research universities are pivotal links in and of research as well as the purpose of academic scholarships which are necessary to fully facilitate such research degrees.In addition, this section explained the need to ensure customer satisfaction at all levels and that leadership is at the cornerstone of business excellence in this arena.Previous studies on higher education in Trinidad and Tobago and the Caribbean were also presented as it provided a support system of the work from Caribbean academics.

Methodology
This section explains why both the collection of empirical data (UWI Postgraduate Scholarships students from 2001 to 2009) and a quantitative survey (questionnaires) method were necessary for this research.In addition, the chapter would seek to clarify how the data were acquired for the study.

Purpose of Data Acquisition
The purpose of the collection of the statistical data, sought to uncover the completion levels, retention and attrition rates of the students awarded a UWI Postgraduate Scholarship from 2001 to 2007.The survey method (questionnaire) on the other hand would test whether the UWI Postgraduate Scholarships are serving its intended purpose i.e., increase completion and retention levels and reduce the attrition rates of graduate research students.

Procedures of Data Acquisition
The data collection comprised: (i) Acquiring structured record reviews of the UWI Postgraduate Scholarship students from 2001-2009.
(ii) Self-administered questionnaires via email and face to face

Structured Record Reviews
This information was not available in any University report and had to be sourced from the University Registry files and the student systems to ascertain whether or not the student accepted the scholarship, withdrew, graduated or did not re-register.This information was collected manually due to costs, data availability and convenience.Additionally, specific pieces of information needed to be sourced directly by the researcher from the files and student systems as it would have been otherwise overlooked.Figure 3 depicts the dimensions of the UWI Postgraduate Scholarship i.e., the faculties to which scholarships are awarded, the profile of the student and the institutional factors regarding the immediate stakeholders.The questionnaire was designed to test whether the UWI Postgraduate Scholarships are achieving increased completion and retention of graduate research students while simultaneously reducing attrition rates.In addition, the questionnaire would attempt to ascertain whether the UWI Postgraduate Scholarships provide for better student support i.e., feedback from supervisors, career and competency management as well as regular and consistent feedback from supervisors.

Population and Sample
The population would be derived from the following listings: The survey will be cross-sectional as various categories of students have been included in the study and a single stage sampling procedure will be used as all individuals listed above could be sampled directly.A random sampling method was used to select candidates.

Instrumentation
The questionnaire was designed specifically for this study and would be administered via email and on a face-to-face basis to reduce the time and resources required to achieve the results.The questionnaire would be subsequently collected two (2) to three (3) weeks after distribution both via email and face to face.Likert and categorical scaling were used. www.ccsen

Summary
This sectio the necessi and instrum

Introduct
This sectio and the st Augustine

Presentati
The

Presentation of Data from the Questionnaires to Students
The questionnaire was administered to 200 students and 167 responses were received.
The questionnaire was tested using a small number of persons from each category of student mentioned above (a -d).It was altered based on feedback from a pilot study before its full administration.Some of the respondents indicated that it was not applicable, needed restructuring and lengthy.The questionnaire consisted of nine (9) questions and question nine (9) comprised five (5) parts which formed the emphasis of the study.
The results from questions 1 to 8 are as follows:  Question 1 requested information regarding the student's gender.94 or 56% of the respondents were female while 73 or 44% of the respondents were male.None of the students indicated that they preferred not to answer.or not com  This is also evident in the goals laid out in the UWI Strategic Plan 2007-2012, specifically the Graduate Studies core value.The UWI Postgraduate Scholarships therefore can be viewed as a mechanism by which some of these goals could be achieved.However investigations have revealed that financial awards serve as a significant incentive to promote a research oriented institution but contribute to only piece of the puzzle necessary for a student to complete their research degree in a timely manner.


Consequently, according to the scholarships statistics, the completion levels of the UWI Postgraduate Scholarship students have been regressing since the 2001-2002 academic year resulting in a negative correlation analysis for the students who have graduated.However, 73% of the students indicated that the scholarship provided for more focused full-time research.Notwithstanding that, 14% of the students strongly disagreed and disagreed.An inference therefore could be made that: (i) a financial award/scholarship neither guarantees nor is it all that is required for completion of a research degree (ii) The students who agreed and strongly agreed use the full-time research as a stepping stone to accomplish the bulk of the research before returning to full-time employment after the two (2) or three (3) year period of the award.
It must be noted also that almost half of the UWI Postgraduate Scholarship students, from 2001 to 2007 are still registered to pursue their research degrees after termination of the scholarship.According to the University Regulations for Graduate Diplomas and Degrees, there are time limits for the submission of the thesis.
Therefore, UWI Postgraduate Scholarship students for example who commenced their course of study in September 2001 and are currently registered for a research degree should have completed their course of study in August of 2004 according to the regulations.These students were in excess of five ( 5) years for the submission of their theses.(Kvavik et al., 2005) and (Kanji et al., 1999) stated that, 'if one is going to look for a return on investment in higher education, he/she could not focus solely on efficiency.The focus should be put on effectiveness, customer satisfaction, and accountability'.In addition, other factors include the quality of service, operation overheads, demand for products or service, stage in product life cycle and productivity.The 360 degree survey feedback system could be implemented with the M.Phil./Ph.D. research students at UWI, St. Augustine.The UWI Postgraduate Scholarship programme could be used as the pilot study for this tool.The 360 degree feedback could be adapted as a system or process in which students would receive confidential, anonymous feedback from their supervisor/s, advisory committee (Note 1) members and head of department possibly after their Research Seminars (students are required to complete two (2) two Research Seminars at the M.Phil.level and three (3) at the Ph.D. level).This would be ideal as students present their research work to a panel of academic members of staff which includes their supervisor and advisory committee members.It allows for constructive criticism and feedback from all person's regarding the student's research.

The Students
Students will also have the opportunity to complete a self-assessment on their individual progress.This system would be used to acquire a better understanding of a student's strengths and weaknesses.It would be implemented only for developmental purposes, used to better track research progress of students and facilitate better communication between students, supervisors and administration.In addition, this system is intended to bridge the communication divide amongst student, supervisor and administration.The 360 degree feedback system would provide for a more informed student e.g., regarding the University's regulations, requirements and available resources to assist students with their research.
6.3.2How Will the 360 Degree Feedback System Work?
The responses of the 360 degree feedback survey would be completed after a Research Seminar and would be submitted to the Office of Graduate Studies and Research under confidential cover from all students and supervisors.It would be dealt with in a manner of utmost confidentiality for all parties concerned.Feedback from each member of academic staff at the Research Seminar would be compiled and given to students and supervisors in written communication and a copy retained on the student's file in the University Registry.The University could possibly employ academic members of staff qualified to undertake the evaluation of research projects at the UWI St. Augustine campus.This could be done on an individual basis with the assistance of the student's supervisor.These academic members of staff could also attend the Research Seminars where students are required to present their work and also provide feedback regarding any shortfalls that may exist.This is a useful model but would require a continuous evaluation of all research students not just the students on scholarship.In addition, this model tended to evaluate past students but could be adapted for evaluation of currently registered students.The model however would contribute to an understanding of:  students' own learning (their continuing professional development) (CPD)  the development of their practice in the specific contexts in which they work

Recommendations
The various best practices highlighted earlier would add value to the UWI Postgraduate Scholarship and allow it to serve its intended purpose.However other key factors were considered:  In the 2007/08 academic year, 85 students received scholarships only half of which accepted.The other half were mature students and indicated that a stipend of TT$5,000.00was insufficient especially in the case of mature students with mortgages and families.Therefore, should the university decide to increase the stipend, such concerns should be taken into consideration.A special evaluation of the needs of these students could be thoroughly analyzed via an interview process.


With regard to repayment of an award, this only occurs when a student is discovered to have obtained full-time employment while on scholarship.The student would have to repay the award from the point that he accepted full-time employment.However, little or no policing of such cases occur.Therefore it is highly unlikely that this would be discovered.Notwithstanding that, students are advised in their scholarship award letters that they cannot accept full-time employment while on scholarship and that they can also work on a part-time basis i.e., for a maximum of 12 hours per week.Therefore adopting the guidelines laid out for the GATE programme would be beneficial.


Stringent enforcement of the existing regulations and penalties at all levels.This should include reinforcement at the Departmental level with Research Seminars in which clear guidelines have been formulated for students and supervisors.This document should state when students are required to complete these seminars, what should be presented and the length of the presentation on each occasion (Note 2).


Advisory Committee meetings should be held with all members and the students for at least five (5) occasions per semester with the final meeting for the semester to include the Head of Department.The Departments could assist with this new system to facilitate supervisors, Advisory Committee members and students.This initiative would be useful as it would facilitate more interaction and provide a better understanding of the student's research to the supervisor and Advisory Committee members and Head of Department.This would also result in less wastage of time for the student as well as reduce the duration of time students are registered at the University for these research degrees.


Deadlines dates for the submission of a student's M.Phil.and Ph.D. research thesis should be indicated in the initial offer letter of the degree.


An award ceremony or induction seminar for UWI Postgraduate Scholarship students is crucial.This would not only add value to the award but the student experience as well.Moreover, publicizing the event on television would showcase the University reinforcing its efforts and investment in students and research.
o Such an event should provide a better understanding of what is research and what is required from a research student.This has become necessary as situations occur in which students pursue taught master's courses to the equivalent of the M.Sc.degree and then they either transfer or voluntarily withdraw from the University.The University guidelines for M.Phil.and Ph.D. research students are clear regarding the number of courses they are required to pursue i.e., 6 credits at the M.Phil.and 9 credits of Departmental courses at the Ph.D. level.However, more stringent mechanisms should be put in place to monitor these scholarship students' registration.
o Furthermore, such an event would also bridge the gap in communication by providing pivotal information on procedures and resources available to students for their research e.g., the funding available from the Department and by the Campus Research and Publication Fund Committee.Consequently, this would result in stronger cohesion, better direction and smoother transition of these students within the system.
o This event would also allow for the introduction of these students to the Postgraduate Association (PGA).This association represents all postgraduate students and facilitates workshops that cater for the needs of all PG.Diploma, M.Sc., M.Phil.and Ph.D. students. Nelson et al (2001) suggest that Universities should consider preparing undergraduates for the culture of graduate school.Academics often assume that a student's knowledge of the discipline is all that matters, but advanced awareness of the culture of graduate school is equally important.Indeed, some activities, like undergraduate involvement in research projects, can be counterproductive if a student joins a graduate program that has no tradition of collaborative research with faculty members.Students can harbor false expectations about the nature of graduate training, which only fuel their disenchantment.To avoid such misunderstandings, the university could offer undergraduates the opportunity to spend time shadowing graduate students.

Conclusion
The format of the UWI Postgraduate Scholarships nurtures the research component of the campus and is fundamentally sound in governance, organization, structure, management and administration.However, based on the negative correlation analysis, more vigilance is necessary as student are being retained in the system for an inordinately long time to complete their research degrees resulting in stagnation and a drain on the limited resources available to new students.
The UWI Postgraduate Scholarships as indicated by the responses from the questionnaires are important to students and the University as it is the mechanism which it intends to use to create new knowledge that would impact not only the University but the local community, Caribbean and world at large.Therefore based on the evidence, every effort should be extended to these students to assist in timely completion of their research.
refers to: (i) the number of UWI Postgraduate Scholarships awarded from 2001-2007 (ii) the number of UWI Postgraduate Scholarship students who graduated from 2001-2007.

Figure 1 .
Figure 1.Number of students awarded and graduated who received UWI PG Scholarships

Figure 2 .
Figure 2. Approximate dollar value or sum of money spent on the UWI Postgraduate Scholarships from 2001 to 2007

Figure 3 .
Figure 3. Dimensions of the UWI Postgraduate Scholarship (i) The UWI Postgraduate Scholarship awardees from 2001 to 2009.This population comprised 346 M.Phil.and Ph.D. students who received a scholarship and are currently registered as well as, those who graduated, voluntarily withdrew and did not re-register to continue their course of study.(ii) The registered student listing of M.Phil.and Ph.D. students for the 2009/10 academic year.The population was 721 students as at January 2010.A sample size of 200 persons were used which would generate a Margin of Error of approximately +/-7%.The students are categorized as follows: a. 50 students currently on scholarship and registered b.50 students who were previously on scholarship and currently registered c. 50 students who were previously on scholarship and no longer registered d.50 M.Phil./Ph.D. students who were never on a scholarship and currently registered.

Figure 4 .
Figure 4. Distribution of the UWI PG Scholarships and the number of students that have graduated and the awarded, accepted and registered numbers Figu Figure 6.D Government Assistance for Tuition Expenses (GATE) programme.GATE is managed by the Funding and Grants Administration Division of the Ministry of Science, Technology and Tertiary Education.It allows students to pursue postgraduate level work in tertiary education programmes which lead to the conferment of qualifications with the titles of Postgraduate Certificate, Postgraduate Diploma, Master's Degrees, Doctoral Degrees or Postgraduate professional qualifications.
Question 2 referred to the student's age.48 or 29% of the students were in the 18-24 age group, 49 or 29% were in the 25-34 age group, 37 or 22% were in the 35-44 age group, 28 or 17% were in the 45-54 age group and 5 or 3% represented the 55-64 age group.The arithmetic means of the respondents' ages were also calculated using the class intervals by taking the mid-point of each class interval and multiplying that age by the number in each class.This totaled 5539 which was divided by the number of subjects i.e., 167 resulting in a mean of 33.This figure identified with the second class interval i.e., the 25 -34 age group.Question 3 was concerned with whether the student was registered.130 or 78% of the respondents was registered while 37 or 22% of the students were not registered.This provided for a more up to date opinion on how students presently view the awards and the service at the University for these research degrees.
 Question 4 requested information on the Faculty in which the student was registered.40 or 24% of the respondents originated from the Faculty of Science and Agriculture, 29 or 17% were derived from Engineering, 38 or 23% were drawn from Humanities and Education, 23 or 14% represented Medical Sciences and 37 or 22% emanated from in the Faculty of Social Sciences.
Awarded the UWI Postgraduate Scholarship are Provided with an Incentive to Pursue a Research Degree Resulting in Commitment and Less Likelihood of Dropout As Well As Regular and ConsistentFeedback from Their Supervisor Regular and consistent feedback from student and supervisor resulting in the timely completion of the degree Results indicate that 80% of the responses were positive which means that the award is and has been beneficial to students while 20% of the responses were neutral and did not agree.Regardless of these responses, the fact remains that the University has lost approximately 25% of its investment i.e., the students who were on scholarship that have voluntarily withdrawn and did not register and approximately 50% of the investment is still not yet realized.
6.3.3Implementation of the Best Practice Research Scholarship Programme (BPRS) with the UWI Postgraduate ScholarshipsThis model would require an in-depth evaluation of each student's (currently registered) M.Phil./ Ph.D. research project.The major concern at present is the length of time students are registered for a research degree.Therefore, critical information for example the stage of the student's research (product), the amount of work completed and yet to be completed could be evaluated.In addition, this should involve academic student counseling which should incorporate an examination of the research culture and the student's work ethic at the University.This would provide invaluable information as it would separate the 'sheep from the goats' as well as uncover shortfalls in the student's research.It would also assist students in expediting completion.