Combining MCDM Methods and AHP to Improve TTQS : A Case Study of the VETC

This study proposed the use of the benchmarking framework to evaluate the performance of vocational education and training centers (VETC) in using the Taiwan Training Quality System (TTQS) to ensure the advantages and disadvantages of each factor and to confirm the priority of the weights of the criteria and alternative solutions. This study used the Delphi method, Analytic Hierarchy Process Theory (AHP), and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) as the base method and divided the whole process into three stages. In the first stage, the Delphi method identified and integrated the criteria, sub-criteria, and alternatives. In the second stage, AHP was used to ensure that VETC underwent performance evaluation against their competitors through TTQS to observe the relationships of every criteria and sub-criteria, as well as the overall performance indicators. In the third stage, the TOPSIS is used as the basis to evaluate the VETC performance.


Introduction
When the world is facing a rapid change in the information era, industrial structures and professional and technical personnel should also change accordingly and quickly.To enhance the competitiveness and cultivation of new talents, vocational training has become the primary task of the government (Lin et al., 2011).However, the main concern of governments with regard to the promotion of vocational training development should be on improving integrated training resources, actively promoting career development, and assisting jobseekers and the unemployed to find new jobs, change career paths, or learn a second skill to improve their employability skills.These skills can equip them to return to the labor market.
In recent years, the government has realized that the international competitiveness of a country should start from human capital.Therefore, the government established a quality training system to strengthen the international competitiveness of its people.The Taiwan Training Quality System (TTQS) emphasizes the quality of human-based service (Lin et al., 2011).Vocational education and training centers (VETC) import mechanisms through TTQS to verify the quality of their manpower training services and connects them with business objectives to obtain the core value performances of business needs.The Bureau of Employment and Vocational Training (BEVT) has invested greatly in promoting TTQS, proposed training institution score sheets for VETC, and constructed education and training policies combined with TTQS.VETCs are also required to pass the TTQS certification, which is needed by establishments when applying for government grants.Therefore, VETCs should pass TTQS certification.The government can understand the actual needs of the current population through TTQS, and VETC can understand the current policy objectives of the government through TTQS to serve as a basis for applying for government grants.Therefore, exploring the decisive factors of the TTQS on the government and the VETC is quite important.
The model framework proposed in this study will evaluate the performance of VETCs by hierarchy, follow-up with criteria and sub-criteria, and link with alternative solutions that will let VETCs understand the factors or policies that will directly affect their operating performances.Thus, the model is important to VETC and the government.TTQS is a system that measures performance and provides five directions for the evaluation of VETC.Lin (2011) proposed the use of cluster analysis as a tool for data mining, conducting data analysis focusing on TTQS index scores, and producing key performance indicators.Thereafter, the TTQS conducts data comparison and analysis such that all the data will go through feedback to obtain a loop.Lin et al. (2010) proposed the TTQS combined with neural network classification technology to reduce the gap between the anticipated goal and actual plan of the company.Chuang (2013) proposed the TTQS training plans and design to solve managerial problems and convert the results into human capital that will effectively improve training results and human capital.Chen (2013) proposed the combination of knowledge innovation with TTQS to createthe internal training of human resources of a company and strengthen its competitive advantage.Gipps and Stobart (2003) defined education training as providing product and experience.VET is defined as the personal knowledge and skill of a person that can be applied in his or her workplace to meet the goals and job performance requirements of a company (Fletcher, 1994;Griffin, Gillis, & Calvitto, 2007).Therefore, VET is considered the orientation training of job seekers and the unemployed and the on-the-job training of employees.In addition to the areas of education, VET provides employees or newly hired employees vocational preparation and the opportunity to learn the necessary professional knowledge and skills needed on the job.Employees need different phases and levels of training in work adaptation, promotion, or preparation for job transfer.VET helps establish and implement career training and provides vocational guidance, retraining, and job counseling.

Vocational Education and Training
VET refers to the training implemented for employment and the promotion of employees to different levels (e.g., from ordinary staff member to top management executive).Okada (2007) believed that VET is an educational measure that provides an immediate effect by focusing on the activities needed for employment or work.With regard to the subjects, VET can also expand education-related trainings, in addition to adding basic knowledge and skills to the unemployed labor group.Ashton et al. (1999) indicated that VET should provide skills that meet the needs of the market and should train trainees effectively acquire new skills.Thus, VET equips the labor force with productivity and employability.Only VET can meet the needs of the market and achieve the desired efficiency and performance requirements.At the same time, VET also trains workers with skills ranging from general to professional to assist them in their work and improve work efficiency.

Taiwan Training Quality System
The TTQS is an evaluation standard proposed by the BEVT Council of Labor Affairs in Taiwan based on the Service Industry Development Guidelines andInitiative (2004-2008) by Executive Yuan, where the content is about a quality accreditation system for the development of a personnel training service industry (Lin et al., 2011).TTQS is an integration of the ISO 10015 Standard, European Foundation for Quality Management (EFQM), Investors in People Standard, IIP, and other benchmarking projects.With regard to confirming the quality of training in companies, not only can TTQS maintain the training quality and performance of staff members but it can also enhance the ability of national human resources and accumulate training knowledge (Lin et al., 2011;Chen, 2013).
The main concepts of TTQS include developing the human capital of Taiwan, strengthening the competitiveness of human resources, increasing the performance of vocational training, and measuring the goals and value of vocational training programs (Chuang, 2013).TTQS can reflect the training quality of Taiwan's overall training plan execution, organizational implementation of system conditions, and implementation capability such that the quality of training in the country is in line with international standards and on-the-job training quality is improved for staff members.
The basic structure of the training quality scorecard of the TTQS is based on the 5 dimensions of "PDDRO," which is composed of 17 training quality scores (Lin et al., 2011).The 5 dimensions of "PDDRO" are plan, design, do, review, and outcome.The output of each stage is the input of the next stage, and the 5 dimensions create a cycle.This system also allows the training of employees to be in line with the business visions and goals.This system can also identify the gap among functions to assess all training processes and performances.The TTQS basically proposes that training quality must originate from planning and that the training process should be systematic and should be according to the goals and visions of training quality to help organizations correct functional and structural problems.
The TTQS standards for training quality use the "PDDRO" process with 18 scored points from 5 procedures.The PDDRO process is composed of planning, designing, doing, reviewing, and obtaining the outcome.This procedure describes the overall training activities of organizations.In conducting the TTQS, the PDDRO phases conduct on-the-job trainings.Reactions, learning, behavior, and results should be evaluated and analyzed after training to establish training results, continuously improve the important points, and compile internal and external benefits to serve as a reference in formulating the planning, designing, doing, and reviewing of the next stage.Evaluating the advantages and disadvantages of all training program processes and results can help integrate the training management process into business performance through a systematic framework.This approach will allow the training to be in line with the needs and goals of the company.
(5) Review: This stage focuses on the training activities and results of the four stages of the management process, namely, plan, design, do, and outcome, to ensure that they are in line with the original plans and that these plans are effectively implemented to meet the goals.Therefore, we must consider monitoring and correcting abnormal processes, providing evaluation reports, and performing regular comprehensive analysis.We should also create management operating procedures that are appropriate, corrective, and preventive.We should develop a suitable countermeasure in case unexpected incidents occur during the training process, and we should manage the problem to prevent it from recurring to ensure training quality and stability.

Results
According to past literature, vocational training, and education experts in the industry and the academia, some important indicators can be identified from a VETC.This study analyzed the five different criteria.The alternatives are classified into three categories, namely, Center A, Center B, and Center C. The indicator framework was developed by the AHP model.These three VETC are presented as follows: (1) Center A: Center A was established in 2000 and is a government-run VETC.To increase the willingness of the students to study, Center A invested NTD 50 million in software and hardware.The courses offered include training and review classes on labor certification, training for staff currently working in companies, and skills tests.The current strategy of this VETC involves regional characteristics planning and training projects for the development of the regional industry to spread vocational training across regions, align types of trainings withthe manpower needs of the industries and the market, and plan certification trainings and counseling to enhance the skills for labor employment.
(2) Center B: Center B was founded in 1998 and mainly provides pre-employment training, training research and development, business promotion, information software professionals.This center also develops vocational trainings for talents in emerging industries.
(3) Center C: Center C was founded in 1996.It mainly operates on trainings funded by the local government and assists companies in organizing vocational training and investment programs for industry talents.This center mainly provides labor-related trainings and is currently in alliance with employment service centers and large-scale enterprises.Its goal is to complete the training of the students so that they can immediately find a job.The framework of the TTQS is shown in Figure 1.

Figure1. The four perspectives of Taiwan Training Quality System
Many steps are involved in the research process (Figure 2).The first step is to identify whether the multi-criteria can help decision makers make objective and fair decisions.The first step adopted the Delphi method (Murry & Hammons, 2004).Not only is the Delphi method based on expert advice but it can also confirm that each dimension is suitable to the criteria.All structures of future criteria use AHP to calculate criteria (Chang et al., 2007;Lin & Wu, 2007;Michnik & Lo, 2009).Finally, this study used the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method to obtain the ranking results.Each of the following subsections provides a detailed description of each step.
Step 1. Designation of the group of experts for the TTQS The TTQS experts chosen by this study are from the government, academia, and VETC.The requirements for selecting these experts are as follows: (1) the expert must be from a government agency responsible for vocational training services; (2) the expert must be from an academic institution and must have served as a juror in the TTQS Review Committee; (3) the expert must be an administrator of a VETC.Nine experts were selected in this study.
Step 2. Establishing an evaluation model and defining the evaluative criteria Archive experts jointly establish the Delphi model.Finally, we can decide the performance evaluation of VETC as the main goal.The evaluation includes 5 evaluation criteria, 19 sub-criteria, and 3 alternatives.The evaluation criteria and sub-criteria used to assess performance are defined as follows: Plan: This criterion is the connection between the training plan and the visions, goals, and operational objectives of a business.This criterion will also help us understand the operational ability of the training system. 1) Regular comprehensive analysis: periodic assessment, analysis reports, and review on improvements applied 2) Monitoring and solving of problems: all monitoring and corrections of abnormal processes should be recorded to serve as future reference for training improvement.
3) Regular follow-up on learning performance: regular tracking of the employment of students and informing training institutions for problems 4) Regular analysis and compilation of evaluation report: conduct regular review meetings and assessment reports on training results, which should be used as reference for adjustments in the next meeting Outcome: This criterion focuses on the assessment levels of outcomes, complete training, continuous improvement, and internal and external benefits.1) Analysis and updates of all files: record all problems, complaints, and solutions from student feedback to serve as a reference for reviewing future trainings 2) Suggestions for the VETC: after participating in the training, the staff should have an effective positive feedback on work.
3) Complete training result assessment: the completeness of the training result evaluations (response assessment, learning assessment, behavioral assessment, and outcome assessment) Step 3. Determining the weight of criteria by using AHP AHP is a multi-objective decision-making method.According to the multi-criteria decision-making method (MCDM) proposed by Saaty (1980), AHP is involved with a complicated MCDM problem wherein the overall structure is divided into numbers of hierarchical structures, and assesses the importance of decision-making (Saaty, 1980;Wei, Chien, & Wang, 2005).Each criterion was compared, and the overall priority and ranking of each decision case was determined to serve as a choice for decision making.In a hierarchical structure, the overall decision goal will be at the top level, the decision criteria will be at the middle level, and the decision alternatives will be at the bottom of the hierarchical structure.AHP consolidates the advice of experts and systemizes the complex problems on the basis of the different levels of decomposition provided by the different levels.It also adds to the comprehensive assessment with the obtained ideas through quantitative judgments to provide sufficient information to decision makers in choosing appropriate programs and in reducing the risks of wrong decisions (Saaty, 1980).The AHP framework is constructed in the form of a matrix, and a local priority vector can be derived as an estimate of relative importance associated with the components being compared by solving the following equation: where A is the matrix of pair-wise comparison, ω is the eigenvector, and is the largest eigenvalue of A. Saaty (1996) suggested the use of a consistency index (CI) to conduct the test.If the consistency ratio C. R 0.1, then the consistency level of the matrix already exceeds the allowed error range and decision makers should reconsider their decision-making relationship.The CI and CR are calculated by using Equation (2) and Equation (3).

C. I.
(2) where R.I. represents the average consistency index obtained from a large number of simulation runs and varies depending on the order of the matrix.
Step 4. Determining the VETC performance by using TOPSIS The TOPSIS is a multiple-criteria method initially proposed by Chen and Hwang (1992) to identify solutions from a finite set of alternatives.The underlying logic of the TOPSIS proposed by Hwang and Yoon (1981) is to define the positive ideal solution (PIS) and the negative ideal solution (NIS).The optimal solution should be closest to the PIS and farthest from the NIS.The ranking of alternatives in the TOPSIS is based on the relative similarity to the ideal solution, which avoids the situation of having similarities in both the PIS and NIS.
Assuming n enterprises are to be evaluated and each enterprise has m evaluation indicators, we let X = [x ij ]nxm denote the decision matrix, where x ij is the jth indicator value of the ith enterprise.We also let w j denote the weight of the jth indicator.
The normalized decision matrix is determined as follows: where r ij denotes the normalized value of the jth indicator of the ith enterprise.
The weight of the normalized decision matrix is determined as follows: where v ij denotes the weighted normalized value of the jth indicator of the ith enterprise.The weights are always set subjectively by experts or regulators.
The PIS (denoted as A + ) and NIS (denoted as A − ) are defined as follows: , 1, , If the jth indicator is a benefit indicator, then , 1, , and , 1, , .On the contrary, if the jth indicator is a cost indicator, PIS and NIS are the weighted normalized best and worst enterprises, respectively.Experts or regulators can also notably determine the best and the worst quality credit enterprises according to their opinions and needs.
The PIS (denoted as and NIS (denoted as are defined as follows: where denotes the distance between the ith enterprise and the PIS, and denotes the distance between the ith enterprise and the NIS.
The relative closeness to the ideal solution is calculated as follows: Ranking the alternatives according to the relative closeness to the ideal solution shows that the alternatives with high CI are assumed to be more important and should be given higher priority.

Empirical Results
Step 1: Establishing an evaluation framework After reviewing the literature and interviewing experts, they will further evaluate the criteria.Figure 3 shows that an independent relationship between the two solutions was determined by the committee.Furthermore, this study used the Delphi method (Lin & Wu., 2007) to explore the evaluation framework and used the semi-structured questionnaire technique for the interviews.The interviewees are people from the government, the industry, and the academe who are responsible for the TTQS.Related literature on the TTQS and its processes are circulated.
After 7 days of reading and brainstorming, the criteria were established and confirmed.From the 27 criteria, 19 criteria remained after combining and overlapping.Lastly, 5 main indicators remained.The results are shown in Figure 3.The evaluation framework is shown as follows: Figure 2. VETC evaluating performance model Step 2: Determining the weight of the criteria and sub-criteria In the evaluation framework, each evaluator performs pair-wise comparison on every node to provide a single decision.Each rated data must comply with the matrix of criteria in the From the weights of the criteria and sub-criteria, the sample of the nine experts is determined to be consistent with the above-mentioned conditions.Each respondent decided a pair-wise comparison of the decision criteria and sub-criteria, and their relative scores are calculated.Related scores are based from the advice provided by the nine experts collected from the geometric mean, and the order of priority of the criteria is determined based from the above-the formula below: In the evaluation framework, each evaluator performs pair-wise comparison on every node to provide a single decision.Each rated data must comply with the matrix of criteria in the questionnaire.
From the weights of the criteria and sub-criteria, the sample of the nine experts is determined to be consistent with the above-mentioned conditions.Each respondent decided a pair-wise comparison of the decision criteria and sub-criteria, and their relative scores are calculated.Related scores are based from the advice provided by the nine experts collected from the geometric mean, and the order of priority of the criteria is determined based from the above-the formula below formula.
0.465 0.097 0.278 0.066 0.093 The weights of the five assessment criteria are expressed as follows: plan (0.465), design (0.097), do (0.278), review (0.066), and outcome (0.093).In this step, the framework evaluators were asked to compare alternatives of each criterion to create a decision matrix.All the weights evaluated by the nine experts will use the MCDM of Saaty.

Discussion
The purpose of this study was to use the AHP and TOPSIS at the same time in the evaluation of VETC to propose an effective framework.To understand the relationships among the criteria, this study proposed the AHP method to obtain the relative weight and used the TOPSIS to obtain the order of priority of the criteria.In the results of the empirical research, the proposed method was found to effectively determine performance.
The contribution of this study is the integration of the AHP and TOPSIS methods to provide a framework that experts can use in a practical application to confirm an appropriate method for performance evaluation.The implementation of the AHP in this study is a valuable and effective method and can indicate the shortcomings of VETC through the selection of a strategy by an expert.VETC can make improvements on the basis of short comings; this capability will be very helpful when requesting for grants from the government.The integration of the AHP and TOPSIS has been already considered by managers as feedback for their goals.The performance evaluation of the decision-making process proposed by this study is operational and practical.The proposed process can find simple and accurate alternatives to help the government and VETC in identifying key factors.The implementation of an effective method is compulsory for managers and the government.This study integrates the AHP and TOPSIS to evaluate appropriate performance and provide an evaluation concept to VETC.Therefore, this study can promote the use of TTQS evaluation such that general businesses can identify key factors.This study determines that the practical implementation of a system framework can easily extend the decision-making process of other management issues.Furthermore, in this practical and complex management environment, the framework proposed by this study can be considered a type of criterion for the future.
VET can be classified into public training and corporate training.Public training refers to the organized training funded by the government for the general public.VET is a variety of training sessions organized by public vocational training institutes, government organizations, commissioned groups, or those that received subsidy from the government or promoted by the universities.Corporate training is a type of training paid by the company to provide orientation training or on-the-job training for staff members.

( 1 )
Plan: The planning stage focuses on the clear, systematic, and connective planning of education and training.The training programs of the company should have a clear causal relationship with its own vision, mission, strategic goals (including training policies and objectives), and management performance.Furthermore, professional training representatives should effectively and clearly implement the training program according to the management system for a systematic training quality to ensure that the direction of the training is compatible with the strategic goals of a company.(2) Design: This stage considers the design of the training course.Training requires the analysis and application of related functions, the system design of training programs, the participation process of stakeholders, the purchasing and selection process of training for products or services, and the combination of the training and objectives of the company.In other words, the entire staff of the company should participate in designing the training program and should consider the functions of the different levels of workers when designing the courses through a design process of a systematic and free-flowing training program.The purpose is to ensure that the course design conforms to the training objectives to avoid wasting any training resources.(3) Do: This stage emphasizes the actual level of implementation of the training program and the training design.It should also consider the transfer and use of the learning results and the classification and filing of information.Thus, the managers of VETC are required to follow the training plan and course design to evaluate the students, materials, teachers, teaching methods, and teaching environment and facilities.Thereafter, they should effectively retain, classify, and index all the records and information of implementing the training process.Similarly, VTEC should consider establishing appropriate training results to ensure that the skills learned by the staff are useful to their jobs and can enhance the work performances of individuals, departments, and the whole organization.(4) Outcome: This stage emphasizes whether the final result of the training can contribute to the performance of the business and whether the result is highly recognized by the employees of the company.This stage should consider the diversity and completeness of the evaluation of the training result and the cognition and feelings of the top management on the training development and its outcomes.

1)
Clearness of training direction: set clear training goals and corresponding programs 2) System of the training plan: training courses can include a clear core-training category or fields.3) Coherence of goal achievement: link between training plan and organizational performance 4) Training-related functions: training personnel possess relevant function evaluation and should provide specific job descriptions.Design: This criterion focuses on the systematic design of the training program.1) Process design of the training: establish a set of standard operating procedures to implement purchasing or staff training 2) Planning of a training course: focus on using the future goals of the company to plan the training needs 3) Systematic training programs: the needed products for training (teachers, teaching materials, and training outsourcing companies) all have established a clear assessment standard for selection.4) Integration of training and goals: a combination of business objectives, technical requirements, manpower requirements, and training needs to demand on staff skills Do: This criterion emphasizes the implementation of training programs, training records, and a systematic level of management. 1) Implementation of the training content according to the plan: have a complete set of actions for the training program 2) Complete records and information systems: the training unit should possess file management capabilities.3) Appropriate education and training: should meet the educational and training needs of the students 4) The training materials are classified and archived: built an information system for education and training Review: This criterion monitors and corrects abnormal processes during the implementation process of the training program and performs regular analysis and evaluation of the outcomes.
Through an integrated framework, VETC can identify key resources and competitive advantages.The differentiation strategy is the best competitive advantage in the integrated model.It is developed to differentiate the various training programs of other VETC.In addition, vocational training centers should adopt and combine local environmental resources and emphasize the uniqueness of trained personnel.

Table 1 .
Eigenvectors for criteria and sub-criteria

Table 3 .
Weight the normalized decision matrix

Table 4 .
Summary of PIS and NIS

Table 5 .
Final ranking of TTQS firm