An Evaluation of Educational Provision Quality at Graduate Level of Rajabhat Universities : Multi-Level Analysis

The purposes of this research were 1) to analyze the educational provision quality scores at graduate level of Rajabhat universities by programs and to analyze the guideline for improving no quality programs at graduate level of Rajabhat universities using Data Envelopment Analysis (DEA) and 2) to analyze 2 level variables (micro level and macro level) that affected to the educational provision quality at graduate level of Rajabhat universities using multi-level analysis. The DEA and multi-level analysis collected by questionnaires from 1,608 of 5 groups of stakeholders (administrators, faculty members, students, alumni and employers). The estimated educational provision quality scores of 27 programs ranged from 45.52–100.00%. The numbers of educational provision quality programs were 11, 9, 10, 9 and 7 evaluating from administrators, faculty members, students, alumni and employers. Of those improving no quality programs, they were increased outputs/outcomes; percentage of a student who graduates within period of time, numbers of published of faculty members’ researches, numbers of alumni award, numbers of utilization of faculty members’ researches and value-added of students in order to enhance their performance to increase their quality. The result of multi-level analysis indicated that factors explaining the quality of programs were numbers of university campus, administrator academic leadership and Office of the Higher Education Commission monitoring.


Introduction
Country development was the step into the modern society as the learning social.Important tool to be used in developing countries is education, because the education is resource human resource development.If the people obtained the quality educational, the countries would have quality.(Office of the National Education Commission, 2003) Especially in graduate studies has increased the demand for education.The people has high expectations about the education quality that received by the service provider.The quality and standard of graduate level educational institutions need to be developed to meet the needs of society and be accepted.(Rockman, 2004) To evaluate the quality of education of any education level, the indicators is very important because the evaluation need to develop indicators and evaluation standards to be used as a tool for evaluation.These indicators are very useful to the administration and research.Indicators administration used in policy formulation, planning process objectives, the monitoring operations and evaluation operations.Both in terms of quality assurance, responsibility of the task, target detection, the rating and ranking system in order to develop the researches.Indicators provide more validity research results than a single variable or set of variables and providing the assumptions for the study of the causal relationship between the indicators further.(Blank, 1993;Johnstone, 1981;Burstein;DfES, 2002;Brunsson, 2001;Fitz-Gibbon, 2002) Of the importance of the indicators in the evaluation and quality assurance, the researchers interested in research to develop indicators of the quality of graduate level education.The researchers chose to study Rajabhat universities group because graduate of these universities compared to other graduate level education institutions still have problems of education quality such as the quality of a master and doctor degree that graduated , the quality of the thesis, term paper writing and independent study of students (Bangkok Post newspaper dated June 29, 2004).The researcher realized the need for research entitle "Educational Provision Quality at Graduate Level of Rajabhat Universities: Multi-Level Analysis" The results of this research were the information used to evaluate the quality of graduate level education both in program level (micro level) and university level (macro level).The evaluation utilized in development planning, and policy management to provide graduate level education quality in the future.

Population and Sample
The total target population of this study consisted of 5 groups of stakeholders to study educational provision quality at graduate level education of Rajabhat universities: students, faculty members, administrators alumni and employers of 27 programs at graduate level education of Rajabhat universities in Bangkok.The formula to estimates the sample size was obtained from Cochran (1963), Yamane (1973) and Kanjanawasee (2001).The samples acquired by different ways as follows: administrator group selected by purposive sampling of 27 administrators of graduated level programs, students, faculty members, alumni and employers groups selected by simple random sampling of 960, 81, 270 and 270 people respectively, the total of 1,608 people.Data analysis with descriptive statistics, Data Envelopment Analysis and multi-level analysis.

Instrumentation
The research instruments consisted of document data recording and questionnaires.We developed five questionnaires, each one for 5 groups of evaluators, The items in the questionnaire was mainly adapted from Chansangawej's questionnaires (Chansangawej et.al ,2002) by integrating much more evaluator issues from The Office for National Education Standards and Quality Assessment, and related literatures, tried out and validated for reliability, content and construct validity.Data would be analyzed using descriptive statistics, DEA analysis and multi-level analysis.

Data Collection and Data Analysis
The questionnaires were distributed by mail and would be analyzed by using SPSS for windows, Frontier Analysis for analyzed DEA, HLM for analyzed multi-level analysis.Preliminary analysis described distribution of all variables in the model in order to check statistical assumptions and transform the variable if necessary.The DEA was to estimation of quality scores of 27 programs at graduate level of Rajabhat universities for analyzing quality to identify sources, the slack analysis and suggestions for improving non quality.The multi-level analysis was to explain the variation in quality scores measuring by DEA.

Educational Provision Quality at Graduate Level of Rajabhat Universities
The educational provision quality indicators at graduate level of Rajabhat universities in the analysis was from Rungnapa Tangchitcharoenkhul and Nonglak Wiratchai's research in the title of Educational Provision Quality at Graduate Level of Rajabhat Universities (Rungnapa Tangchitcharoenkhul and Nonglak Wiratchai, 2011) synthesized from 81 Thai and foreign related literature and in-depth interviewed 5 group stakeholders (72 persons) and group of experts (7), total of 79 people.The factors and indicators of educational provision quality at graduate level education of Rajabhat universities were important and appropriate for using in assessing the quality at graduate level education of Rajabhat universities, based on the process of selection composed of 1) input variables: 1.1) physical factors: 2 indicators; the availability of facilities and numbers of facilities 1.2) learning factors: 3 indicators; appropriateness and flexibility of the program, number of offered program and numbers of student's class hours per semester 1.3) human factors: 5 indicators; numbers of faculty member per student, percentage of faculty member with doctoral degree, percentage of faculty member with academic positions as professor or associate professor, quality of faculty member, quality of thesis advisor 1.4) financial factors: 4 indicators; the percentage of staff salaries to the whole operation, research budget per faculty members, research budget per students and tuition fee 2) process variables: 8 indicators; program management and teaching, information management, faculty member development, the control of thesis quality, evaluation of teaching and learning, the completion of the program, number of academic/professional committee of the other universities' thesis and numbers of teachers have been trained.3) output variables: 7 indicators; the percentage of a student who graduates within the period of time, average GPA, program satisfaction, value-added of students, the quality of graduates, numbers of published of students' researches, numbers of published of faculty members' researches and 4) outcome variables: 7 indicators; numbers of alumni award, numbers of utilization of students' researches, numbers of utilization of faculty members' researches, relationship with the university, numbers of alumni who returned to study at the same institution, project/activities partnership between the university and alumni and comparison of the graduates with graduates of other institutions The validation of indicators and instruments by in-depth interviewed 7 experts found that the developed indicators measuring educational provision quality indicators at graduate level of Rajabhat universities were appropriateness and feasibility.They were based on the context of graduate level of Rajabhat universities The results of validate the content validity of experts showed that there was a high content validity.(IOC = 0.71-1.00)The validation of indicators and instruments quality by surveyed the opinions of 5 stakeholders groups found that 5 stakeholders groups agreed with appropriateness and utility of them at high level ( = 3.59-4.27,S.D.= 0.21-0.98)and ( = 3.65-4.22,S.D.= 0.21-0.92).The results of examining reliability by tried out questionnaires measuring set of indicators to 90 people showed that set of indicators had high reliability.(Cronbach's alpha coefficient = 0.7225 -0.8864) The analysis results of educational provision quality at graduate level of Rajabhat universities by programs found that the numbers of quality programs at graduate level of Rajabhat universities were 11, 9, 10, 9 and 7 measuring from 5 stakeholders groups (administrators, faculty members, students, alumni and employers).The lowest quality of program evaluating from administrators, faculty, students, alumni and graduates was the doctoral program (CD4 program: 45.52 score) followed by CD3 program with 46.14 score and CD6 program with 49.56 score.Considered on quality of the programs of postgraduate program evaluated from 5 stakeholders groups (average of 83.81, 83.35, 80.67, 75.34, 70.49score, respectively) was higher than quality of the programs of doctoral program.(average of 91.41, 89.59, 86.67, 83.75, 83.22 points, respectively) (Table 1) When considering the scores of educational provision quality at graduate level of Rajabhat universities by 5 evaluator groups (administrators, faculty members, students, alumni and employers) found that highly positive correlated (r=0.694-0.951) which was statistical significant at .01 level.The analysis of correlation of the scores of educational provision quality at graduate level education of Rajabhat universities by 5 evaluator groups did not different.Therefore, the proposed guidelines to improve the educational provision quality at graduate level of Rajabhat universities would analyze from faculty members evaluators.(Table 2) 2) qua_s = scores of educational provision quality evaluated by students qua_e = scores of educational provision quality evaluated by employers qua_al = scores of educational provision quality evaluated by alumni qua_f = scores of educational provision quality evaluated by faculty members qua_ad = scores of educational provision quality evaluated by administrators

Guideline for Improving No Quality Programs at Graduate Level of Rajabhat Universities
The guideline for improving no quality programs at graduate level of Rajabhat universities from Data Envelopment Analysis considered on output/outcome (output/outcome orientation).The analyses found that programs at graduate level of Rajabhat universities must increased types of output/outcome to improve no quality programs at graduate level of Rajabhat universities (Table 3) 1) The 3 programs (CD3 CD4 and CD6) ought to increase the percentage of student who graduates within the period of time (OP1), value-added of students (OP4), numbers of published of students' researches (OP6), numbers of alumni award (OC1) and numbers of alumni who returned to study at the same institution (OC5).
2) The 3 programs (CD1 CD2 and CD5) ought to increase the percentage of student who graduates within the period of time (OP1), numbers of published of students' researches (OP6), numbers of published of faculty members' researches (OP7) and numbers of alumni who returned to study at the same institution (OC5).
3) The 4 programs (CM1 CM11 CM19 and CM20) ought to increase the percentage of student who graduates within the period of time (OP1), numbers of published of faculty members' researches (OP7), numbers of alumni award (OC1) and numbers of utilization of faculty members' researches (OC3) 4) The 7 programs (CM2 CM6 CM8 CM9 CM14 CM17 and CM18) ought to increase the percentage of student who graduates within the period of time (OP1), value-added of students (OP4), numbers of published of students' researches (OP6), numbers of published of faculty members' researches (OP7) and numbers of alumni award (OC1) 5) The 3 programs (CM10 CM15 and CM16) ought to increase the percentage of student who graduates within the period of time (OP1), numbers of published of students' researches (OP6) and numbers of alumni award (OC1) For the percentage of the increasing output/outcome to improve the no quality of programs at graduate level of Rajabhat universities.The Data Envelopment Analysis identified them in 5 groups.(Table 2).
1) The CD3 CD4 and CD6 programs ought to increase output/outcome: the percentage of student who graduates within the period of time (71.3) The CM4 CM11 CM19 and CM20 programs ought to increase output/outcome: the percentage of student who graduates within the period of time (12.89, 45.67, 32.41 and 68.35 percent, respectively), numbers of published of students' researches (23.76, 32.81, 21.78 and 56.21 percent), numbers of alumni award (30.12, 33.96, 12.98 and 43.37 percent) and numbers of utilization of faculty members' researches (32.48, 40.28, 21.96 and 41.29 percent) 4) The CM2 CM6 CM8 CM9 CM14 CM17 and CM18 programs ought to increase output/outcome: the percentage of student who graduates within the period of time (52.19, 80.24, 87.12, 68.90, 78.48, 86.33The analysis was divided into two stages;

Null Model
The analyses of fixed effect and random effect were studied within group variation of programs of educational provision quality at graduate level of Rajabhat universities and between group variation of them.The fixed effect analysis showed that the average scores of quality of programs at graduate level of Rajabhat universities were 95.318 and difference from zero at .05 significantly statistical level.The random effect analysis showed that the average scores of quality of 6 Rajabhat universities were varied between group at .05 significantly statistical level and the variance of the estimation of parameters was 25.276.The analysis showed that there were significantly different of quality of programs at graduate level between Rajabhat universities.(Table 4)

Conclusions
In this paper, DEA is clearly a powerful evaluation tool that mathematically estimates the maximum possible aggregate quality score by integrating the combination relationships of inputs and outputs/outcomes of 27 programs at graduate level of Rajabhat universities.The estimated educational provision quality scores of 27 programs ranged from 45.52-100.00%.The numbers of educational provision quality programs were 11, 9, 10, 9 and 7 evaluating from administrators, faculty members, students, alumni and employers.Of those improving no quality programs, they were increased outputs/outcomes; percentage of a student who graduates within period of time, numbers of published of faculty members' researches, numbers of alumni award, numbers of utilization of faculty members' researches and value-added of students in order to enhance their performance to increase their quality.And multi-level analysis is a powerful evaluation tool that analyze 2 level variables (micro level and macro level) that affected to the educational provision quality at graduate level of Rajabhat universities.The result of multi-level analysis indicated that factors explaining the quality of programs were numbers of university campus, administrator academic leadership and Office of the Higher Education Commission monitoring.

Educational Provision Quality at Graduate Level Education of Rajabhat Universities
The analysis results of educational provision quality indicators at graduate level of Rajabhat universities by programs found that the numbers of quality programs at graduate level of Rajabhat universities were 11, 9, 10, 9 and 7 measuring from 5 stakeholders groups (administrators, faculty members, students, alumni and employers).
When considering the correlation of quality scores among five-group evaluators (administrators, faculty members, students, alumni and the employers) found that they positive correlated statistically significant at .01 level.They showed that the results of analysis of the correlation among five-group evaluators did not difference.The analysis found that the lowest quality of program evaluating from administrators, faculty, students, alumni and graduates was the doctoral program (CD4 program: 45.52 score) followed by CD3 program with 46.14 score and CD6 program with 49.56 score.Considered on quality of the programs of postgraduate program evaluated from 5 stakeholders groups (average of 83.81, 83.35, 80.67, 75.34, 70.49score, respectively) was higher than quality of the programs of doctoral program.(average of 91.41, 89.59, 86.67, 83.75, 83.22 points, respectively) In addition, the DEA analysis results were consistent to the correlation between quality scores and program feature (numbers of university campus and administrator academic leadership).The analysis of the correlation between quality scores of programs and numbers of university campus found that programs with 2-4 campus would not be more quality than less than 2 campus (CD2 CD3 CD4 CD6 CM2 CM4 CM6 CM8 CM9 CM17 and CM20).And the analysis found that quality scores of programs positive correlated with administrator academic leadership.The programs with high administrator academic leadership would be more quality than low administrator academic leadership.The DEA analysis of the no quality programs found that the programs mostly had low administrator academic leadership (2-2.99 score).

The Guideline for Improving No Quality Programs at Graduate Level of Rajabhat Universities
The guideline for improving no quality programs at graduate level of Rajabhat universities (CD3 CD4 CD6 CM2 CM6 CM8 CM9 CM14 CM16 CM17 CM20) would increase more than 50 percent of the percentage of student who graduates within the period of time from the output of the percentage of student who graduates within the period of time specified below average (21.89).The results were consistent with findings brought by Burstein, L. Oakes, J. and Guiton, G. (1992), HESA (2008) andJaruwan Pratumsri (2004).The CD3 CD4 CD6 CM8 CM9 CM17 and CM18 programs would increase more than 50 percent of value-added of students specified below average (2.61) and the results were consistent with findings brought by Kiatsuda Srisuk (2002), HESA (2008) and Schafer, Yen and Rahman (2000).The CD3 CD4 CD6 and CM8 programs would increase more than 50 percent of numbers of published of students' researches specified below average (5.83) and the CM8 CM9 CM17 and CM20 programs would increase more than 50 percent of numbers of published of faculty members' researches specified below average (5.32) that results were consistent with findings brought by Geoffrey D. Doherty ( 2008) HESA (2008) Schafer, Yen and Rahman (2000) and Laughton, D. (2003).The CM2 CM6 CM8 CM9 and CM17 programs would increase more than 50 percent of numbers of alumni award specified below average (3.96) and results were consistent with findings brought by HESA.( 2008), Brunsson, N. (2001), Laughton, D. (2003).The results of the analysis could be inferred that the guideline for improving no quality programs by increased output of educational provision quality at graduate level education.When considering the descriptive statistical analysis could be considered that there were consistency.Because of the no quality programs scores were lower than the average of programs scores, they would increase them to improve quality.

Multi-level Analysis of Educational Provision Quality at Graduate Level Education of Rajabhat Universities
Multi-level analysis results found that program level factors and university level factors affected on educational provision quality at graduate level of Rajabhat universities were administrator academic leadership, numbers of campus and Office of the Higher Education Commission monitoring.They described the variability of the educational provision quality at graduate level of Rajabhat universities of 45.28 and 51.82 percent.Preliminary data showed that the average of academic leadership scores of programs at graduate level was 3.15.Multi-level analyses found that administrator academic leadership positively affected on educational provision quality at graduate level of Rajabhat universities because the administrators oversaw and controlled program management in accordance with the objectives of the programs.If the administrators had academic leadership, the program management would be quality.The results were consistent with findings brought by Geoffrey D. Doherty (2008), HESA (2008), Lynette Louw, Johan K. Bosch and Danie J.L. Venter (2001), Jaruwan Pratumsri (2004), Laughton, D. (2003), Mizikaci, F. (2006) and Schafer, Yen and Rahman (2000) that academic leadership of administrator was the key factor affected on the quality of educational organizations.In addition, multi-level analysis results found that numbers of campus affected on educational provision quality at graduate level of Rajabhat universities.As the appropriate of campus numbers, the expertise in the teaching contents of faculty members of the program and the adequate numbers of faculty member per student were contributed to the quality of educational provision quality.The results were consistent with findings brought by Jaruwan Pratumsri ( 2004), Rungnapa Tangchitcharoenkhul (2005) and Kanchana Torung (2007).They indicated that quality controlling of programs must be controlled appropriate numbers of university campus.In addition, in-depth interviews of experts and analysis of multi-level showed that the monitoring of the Commission on Higher Education and Management was an important factor affecting the quality of educational provision quality.Thus, the monitoring of the Commission on Higher Education was increased ,educational provision quality would be risen accordingly.

Recommendations
Based on the study results, the following recommendations may be suggested: 1).The guideline for improving no quality programs at graduate level of Rajabhat universities would increase outputs/outcomes (percent of the percentage of student who graduates within the period of time, value-added of students, numbers of published of students' researches, numbers of published of faculty members' researches, numbers of alumni award).Therefore, the administrators will process the policy to increase those outputs/outcomes.
2).Multi-level analysis results found that program level factors and university level factors affected on educational provision quality at graduate level of Rajabhat universities were administrator academic leadership, the monitoring of the Commission on Higher Education and Management and numbers of university campus.The administrator academic leadership positive affected on educational provision quality at graduate level of Rajabhat universities, therefore they should encourage the development of the academic leadership of administrators.And the monitoring of the Commission on Higher Education and Management positive affected on educational provision, therefore the Commission on Higher Education and Management should increase monitoring.But the numbers of university campus negative affected on educational provision quality, therefore the administrators should monitor the numbers of university campus to be appropriate in order to control quality.

Suggestions for Further Research
1).The others region groups of Rajabhat universities (north, south, central and northeast regions) should be conducted into the further research that could include all regions of Rajabhat universities.
2).Qualitative research should be conducted into the further research that could answer the research questions more clearly.
3).Multi-level analysis should be increased factors affected on of educational provision quality at graduate level education of Rajabhat universities such as characteristics of program (students' academic and ethics) and external factors (education policy such as the ASEAN Community Agreement.

Table 1 .
The 27 programs at graduate level of Rajabhat universities'quality scores by evaluator groups

Table 2 .
Correlation matrix of educational provision quality' scores at graduate level of Rajabhat universities by evaluator groups period of time(22.67,20.35and30.34percent,respectively),numbers of published of students' researches(36.18,39.76 and 38.57percent), numbers of published of faculty members' researches(12.67,21.73  and 18.42 percent)and numbers of alumni who returned to study at the same institution(19.82,22.75 and 34.87  percent) 23, 54.23 and 75.32 percent, respectively), value-added of students (50.11, 61.48  and 52.94 percent),numbers of published of students' researches(79.26,80.48and78.26percent),numbers of published of faculty members' researches(22.21,24.56 and 31.22 percent), numbers of alumni award (18.12, 9.68 and 15.78 percent) and numbers of alumni who returned to study at the same institution(52.17,32.18and  45.78 percent)2) The CD1 CD2 and CD5 programs ought to increase output/outcome: the percentage of student who graduates within the

percentage of the increasing output/outcome to improve no quality of program at graduate level of Rajabhat universities
Researchers analyzed 2 level variables that affected to the educational provision quality at graduate level of Rajabhat universities: 1) Level 1 model (Level of programs at graduate level of Rajabhat universities) analyzed null model and simple model 2) Level 2 model (Level of Rajabhat universities) analyzed hypothetical model.
3.3.1 Level 1 Model (Level of Programs at Graduate Level of Rajabhat Universities)

Table 4 .
Null model analysis result When programs of educational provision quality at graduate level of Rajabhat universities were dependent variables, the analyses of fixed effect showed that the constant of program level variables affected on educational provision quality at graduate level of Rajabhat universities at .01 significantly statistical level (t = 53.492).The regression coefficient of the administrator academic leadership variable (LEAD) positive affected on educational provision quality at graduate level of Rajabhat universities at .01 significantly statistical level (t = 2.624).The regression coefficient of numbers of university campus (CAMPUS) negative affected on educational provision quality at graduate level of Rajabhat universities at .05 significantly statistical level (t = 2.317).The analyses of random effect showed that the intercept of programs variables were varied between programs at .05 significantly statistical level ( 2 = 13.074).The regression coefficient of administrator academic leadership and numbers of university campus variables were varied between Rajabhat universities at .05 significantly statistical level ( 2 = 12.833 and 11.547, respectively).The administrator academic leadership and numbers of university campus variables together described the variability of the educational provision quality at graduate level of Rajabhat universities of 45.28 percent.(Table5)

Table 5 .
Simple model analysis result The regression coefficient of the numbers of university campus and Office of the Higher Education Commission monitoring variable (MONITOR) positive affected on educational provision quality at graduate level of Rajabhat universities at .01 significantly statistical level (t = 5.947).The regression coefficient of numbers of university campus (CAMPUS) negative affected on educational provision quality at graduate level of Rajabhat universities at .05 significantly statistical level (t = 4.178).The analyses of random effect showed that the intercept of university variables were varied between universities at .05 significantly statistical level ( 2 = 14.783) and variation of parameter estimation was 16.129.When educational provision quality at graduate level of Rajabhat universities were dependent variables, the analyses of fixed effect showed that the correlation between numbers of Office of the Higher Education Commission monitoring variable and regression coefficient of numbers of Office of the Higher Education Commission monitoring affected on educational provision quality at graduate level of Rajabhat universities at .01 significantly statistical level (t = 4.412).The regression coefficient of numbers of university campus negative affected on educational provision quality at graduate level of Rajabhat universities at .05 significantly statistical level (t = 4.256).The analyses of random effect showed that the correlation of numbers of university campus and educational provision quality at graduate level of Rajabhat universities were varied between Rajabhat universities at .05 significantly statistical level ( 2 = 8.128) and variation of parameter estimation was 27.629.When educational provision quality at graduate level of Rajabhat universities were dependent variables, the analyses of fixed effect showed that the correlation between numbers of university campus and regression coefficient of numbers of university campus affected on educational provision quality at graduate level of Rajabhat universities at .05 significantly statistical level (t = 5.782).The regression coefficient of numbers of university campus negative affected on educational provision quality at graduate level of Rajabhat universities at .05 significantly statistical level (t = 4.011).The analyses of random effect showed that the correlation of numbers of university campus and educational provision quality at graduate level of Rajabhat universities were varied between Rajabhat universities at .05 significantly statistical level ( 2 = 7.328) and variation of parameter estimation was 28.457.The numbers of Office of the Higher Education Commission monitoring and numbers of university campus variables together described the variability of the educational provision quality at graduate level of Rajabhat universities of 51.82 percent.(Table6)Table6.Hypothetical model analysis result 3.3.2Level 2 Model (Level of Rajabhat Universities)When educational provision quality at graduate level of Rajabhat universities were dependent variables, the analyses of fixed effect showed that the constant of university level variables had influence on educational provision quality at graduate level of Rajabhat universities at .01 significantly statistical level (t = 28.639).