Effects of Individual and Work Environment Characteristics on Training Effectiveness: Evidence from Skill Certification System for Automotive Industry in Thailand

Previous research over the past two decades has argued Kirkpatrick’s model ignored the work environment and individual factors influencing training effectiveness. A focus of this study is to investigate four levels of Kirkpatrick’s model with a focus on moderating the influences of individual and work environment characteristic variables, which are learning motivation, self-efficacy, motivation to transfer, and social support. In the present study, we used path analysis to test the hypotheses. The results of this study expand our understanding of the progressive causal relationship of reaction, learning, and behavior to results. In particular, this study confirms the influence of the individual and work environment characteristic on training outcomes and it has implications for enhancing training effectiveness. Although the result of motivation to transfer as a moderating variable has negative effects on the relationship between learning and behavior, social support directly affects behavior change after training and moderates the relationship between learning and behavior. Furthermore, future research on training evaluation should consider the training design variables beyond the training course that may have interfered with the training outcomes.


Introduction
Kirkpatrick's four-level model has been the most widely accepted and used primary organizing framework for training evaluations for over fifty years.The evaluation of training effectiveness in organizations often uses one or more criteria of Kirkpatrick's model because they are simple, clear, and easy to implement, as training evaluators expect.However, Kirkpatrick's evaluation model has been increasingly questioned and criticized.For example, its use as a primary typology for functionalizing evaluation has been challenged (Holton, 1996;Kaufman & Keller, 1994;Phillips & Phillips, 2002;Swanson & Holton, 1999).The noteworthy question in human resource development (HRD) is whether the current conceptualization of evaluation is sufficient for answering the many questions that involve the effectiveness of the organizations' training and development efforts (Preskill, 1997).
In the discussions about the insufficiency of effective training evaluation, there are three limitations in Kirkpatrick's model that have implications for the ability of training evaluators to deliver benefits and, further, to satisfy the interests of organizations.These include the incompleteness of the model and more complicated theoretical development to understand training effectiveness because three assumptions of the Kirkpatrick model are (1) arranged in ascending order, (2) causally linked and (3) positively related.Related to those limitations, research over the past two decades has argued Kirkpatrick's model ignored the work environment and individual factors influencing training effectiveness.For example, some previous studies indicated that a variety of organizational, individual, training design and delivery factors can influence training effectiveness before, during, and after training (Salas & Cannon-Bowers, 2001;Tannenbaum & Yukl, 1992).Thus, without considering the role of trainees' individual and work environment characteristics as influencing training effectiveness, it is not possible to fully understand why training is effective or not.Kirkpatrick's model doesn't explicitly incorporate these factors and, in effect, assumes the examination based on the model is not sufficient for appropriate training evaluation.
In particular, the previous studies examined multiple individual and work environment characteristic variables, such as learning motivation, self-efficacy, motivation to transfer, and social support.They have been analyzed as the determinants of learning and behavior (Cannon-Bowers et al., 1995;Chiaburu & Marinova, 2005;Colquitt, LePine & Noe, 2000;Gist, 1987;Hawley & Barnard, 2005;Holton, 1996;Kanfer & Ackerman, 1989;Lim & Morris, 2006;Nijman et al., 2006;Noe, 1986;Russ-Eft, 2002), independently from reaction-learning and learning-behavior relationships.However, although individual and work environment characteristics including learning motivation, self-efficacy, motivation to transfer, and social support have been demonstrated to have a relationship to training outcomes, few previous studies appear to have examined the combined effects of these four variables as a moderating influence on the relationships of reaction, learning, and behavior.
Based on the arguments above, the main purpose of this study is to investigate four levels of Kirkpatrick's model with a focus on the moderating influences of individual and work environment characteristic variables, which are learning motivation, self-efficacy, motivation to transfer, and social support.All the relationships identified in this study have been proven in the previous studies, though not in a comprehensive manner.To achieve the research purpose, this study focuses on two research questions: Are there progressive causal relationships among training outcomes (reaction, learning, behavior, and results)?Also, how do individual and work environment characteristic variables, including learning motivation, self-efficacy, motivation to transfer, and social support, moderate the relationship between training and its outcomes?

Training Effectiveness: Kirkpatrick's Model
Training effectiveness refers to the extent to which the training objectives or training's goal are achieved.The Kirkpatrick four-level evaluation model has been supported as the primary conceptual framework for evaluating training for several decades.In particular, many researches on training evaluation were often using the typology of four levels to investigate the effectiveness of training.Level 1, reaction, is trainees' feelings about or liking the training program.Level 2 is learning that has taken place during training.Alliger and Janak (1989), defined learning as the "principles, facts, and techniques understood and absorbed by the trainees".This level of evaluation allows trainees to demonstrate their understanding of specific knowledge, skills, and attitude (KSAs) within the learning program.Level 3, behavior change or transfer, refers to the knowledge and skills transferred to the job by trainees.This level attempts to determine whether trainees (who can apply the acquired specific knowledge and/or skills) use their new knowledge and/or skills when returning to the work environment.Behavior change can occur only when one or more of the learning objectives have been performed (Kirkpatrick, 1994).Level 4, results, refers to the final results that occurred because the trainees attended the training program (Kirkpatrick, 1994).These could include the attainment of organizational goals and objectives, such as a reduction in absenteeism and personnel turnover, productivity gains, and cost reductions.
Previous research on training evaluation found that reaction to training may indeed play an important role in understanding training effectiveness.A number of studies have examined the relationships between reactions and learning.Particular facets or dimensions of trainee reactions appear to hold more promise, such that Alliger et al. (1997) distinguish between affective and utility judgments of reactions.The results found that utility judgments have had significant relationships to immediate learning and a combined scale of affective and utility judgment shows a significant relationship to immediate learning.However, most of the studies on training effectiveness hypothesized that the reaction is positively related to learning (Kirkpatrick, 1996;Leach & Liu, 2003;Mathieu, Tannenbaum & Salas, 1992;Tan et al., 2003;Warr, Allan & Birdi, 1999).This study tests the relationship between reaction and learning variables as measures for evaluating training effectiveness.Thus, in this study we hypothesize that: Hypothesis 1: Trainee reaction will be positively related to learning.Liebermann and Hoffmann (2008) found that learning had a direct impact on transfer.According to Expectancy Theory (Vroom, 1964), if learners' individual motives are believed to lead to strengthened performance, they will be more motivated.Therefore more successful learners feel that they can work better through utilizing acquired knowledge on their jobs.In one of Kirkpatrick's more recent publications he argued that, "If training is going to be effective, it is important that trainees react favorably and without learning, no change in behavior will occur" (Kirkpatrick, 1994).Furthermore, Maister (2008) indicated that when more knowledge is learned from training more behavioral change can be found in workplaces.
In addition to learning from training, trainees must retain KSAs instilled during the training program to facilitate the transfer process.Baldwin and Ford (1988) argue that learning retention outcomes are directly associated with the generalization and maintenance of training effects on the job.They argue that in order for training skills to be transferred, they first must be learned and retained.Velada et al. (2007) also found that when trainees retain training content, they are more likely to perceive that they have transferred the training to the work context.Based on the literature review above, we hypothesize that: Hypothesis 2: Learning from training has a positive relationship with behavior.
Fewer previous studies have investigated the relationship between behavior and results compared with those studies on the relationship between reaction and learning and the relationship between learning and behavior.The first reason is there can be a long delay between the improvement in job behavior and desired organizational results.The second reason is there are more variables, both inside and outside the organization, which can influence this relationship (Clement, 1982).The final reason is greater difficulty in evaluating training at the higher levels of Kirkpatrick's model.However, considering Kirkpatrick's original idea that there are causal relationships through all four levels, including between behavior and results, in this study we hypothesize that: Hypothesis 3: Behavior will be positively related to results.

Factors Influencing Training Effectiveness
Training effectiveness is aimed at developing and understanding the process of training with respect to the achievement of training's goal or set of goals.In particular, the emphasis of effectiveness is that what is learned in training is ultimately applied on the job (Bates & Coyne, 2005).Furthermore, effectiveness is concerned more with the inputs into training, including individual characteristic variables (for example, learning motivation, self-efficacy, motivation to transfer) and work environment characteristics such as social support.

Individual Characteristics
The effects of individual characteristics have been studied over the past two decades and integrated in training effectiveness models (Campbell, 1988).The empirical literatures in the field of industrial and organizational psychology have shown the interest in how individual traits influence training proficiency (e.g., Chuang, Liao & Tai, 2005;Colquitt, LePine & Noe, 2000).In particular, individual characteristics, such as self-efficacy, learning motivation, trainability, job attitudes, personal characteristics, and transfer of training conditions for learning, have been found to have an impact on training outcomes.However, other factors have been examined as well.For this study, learning motivation, self-efficacy, and motivation to transfer are the focuses because all of these factors affect training outcomes and no previous studies have combined all of them in training effectiveness models.

Learning Motivation
It is recognized that trainees cannot reap knowledge from training without having learning motivation.Learning motivation or motivation to learn refers to the desire of the trainee to learn the contents of the training program (Noe, 1986).In the Cannon-Bowers model of training effectiveness, motivation to learn is hypothesized to be positively related to knowledge acquisition.Thus learning motivation is important for acquiring fundamental levels of knowledge.Many previous studies have investigated the relationship between learning motivation and learning.For instance, one study found that learning motivation is related to learning and program completion (Ryman & Biesner, 1975).Colquitt et al. (2000) came to the conclusion that trainee learning motivation was significantly related to both declarative knowledge and skill acquisition.Furthermore, researchers have also confirmed motivation to learn has a positive effect on learning outcomes (e.g., Mathieu et al., 1992;Randel, Main, Seymour & Morris, 1992;Traci et al., 2009;Zazanis, Zaccaro & Kilcullen, 2001).Therefore, we hypothesize that: Hypothesis 4: Learning motivation will be positively related to learning.Few of the previous studies have shown that learning motivation has a moderating effect on the relationship between reactions and learning.Mathieu, Tannenbaum, and Salas (1992) found that reactions were important for training effectiveness, but not in and of themselves.The best results were observed when trainees were both motivated to do well and reacted positively to the program.If trainees were not motivated at the start, or if attention was directed solely at making training enjoyable at the expense of developing skills, less than optimal results were obtained.Furthermore, Quinones et al. (1995) also found that when employees received the assignment from the organization to attend training because of their poor performance, they were less motivated and learned less when they felt it was unfair.Thus, learning motivation signals greater learning for trainees who react positively to a program than for those who react less positively.However, the studies on learning motivation lack a consistent framework incorporating a moderating effect on the relationship between reaction and learning.Logically, we assume that learning motivation will more strongly affect learning if trainees react positively to the program.Learning motivation will still positively affect learning among trainees who reacted negatively to the program, but to a lesser degree.Therefore, we hypothesize that: Hypothesis 5: Learning motivation will moderate the relationship between reaction and learning.

Self-Efficacy
Self-efficacy emanates from social learning theory and is defined as one's judgment of his (her) capability to successfully perform target behaviors (Bandura, 1986).In a review of training effectiveness studies, self-efficacy is one of the main determinants of proximal training outcomes (Haccoun & Saks, 1998) and it has been shown to positively correlate with learning and behavior (e.g., Axtell, Maitlis & Yearta, 1997;Cheng, 2000;Chuang, Liao & Tai, 2005;Gist et al., 1991;Guerrero & Sire, 2001;Martocchio & Webster, 1992;Quinones, 1995).Additionally, several meta-analyses confirmed the relationship between self-efficacy and outcomes pertaining to performance (Stajkovic & Luthans, 1998).More specifically, many researches in the training literature have supported that self-efficacy has been positively related with learning from training (e.g., Colquitt, LePine & Noe, 2000;Gist, Schwoerer & Rosen, 1989;Gist, Stevens & Bavetta, 1991;Martocchio, 1994;Simmering & Posey, 2009).Therefore, we hypothesize that: Hypothesis 6: Self-efficacy will be positively related to learning.However, there have been a few studies that have illustrated the role of trainees' self-efficacy in mediating or moderating the relationships between work-related behavior and its antecedents (for example, Gist, 1987;Gist & Mitchell, 1992;Gist et al., 1991;Saks, 1995).Two previous studies found that self-efficacy was a moderating variable for the effect of the training method on training outcomes, including learning.Gist et al. (1989) found that participants with low self-efficacy had greater confidence in their ability to master the software training in the modeling compared with the tutorial conditions.Furthermore, Gist et al. (1991) concluded that post training interventions following basic training in negotiation skills, such as goal-setting or self-management, are positively related to salary performance when participants have low self-efficacy.These studies suggest that the effectiveness of training may depend on the strength of trainees' self-efficacy.These findings did not cover self-efficacy's moderating effect, specifically on the relationship between the reaction to training and consequent training outcomes including learning.However, it is expected that trainees with positive reactions are more likely to be affected in their learning performance by their self-efficacy level.Therefore, our focus here is on self-efficacy as an important moderator variable in the relationship between reactions and learning.We propose the hypothesis that: Hypothesis 7: Self-efficacy will moderate the relationship between reactions and learning.

Motivation to Transfer
In general, motivation is defined as variability in behavior which is not attributable to stable individual traits or strong situational coercion (Quinones, 1995).In a training program, motivation can influence the willingness of a trainee to participate in the training and also affect whether or not a trainee utilizes his (her) learning on the job (Baldwin & Ford, 1988).With regard to training effectiveness, motivation to transfer is the learner's intended efforts to utilize knowledge and skills learned in training setting in a real world work situation (Noe, 1986).In previous empirical studies, such as those by Axtell, Maitlis, and Yearta (1997) and Noe (1986), motivation to transfer is described as the trainee's desire to use the knowledge and skills that have been learned in a training program on the job.Research on the effects of motivation to transfer on actual transfer is limited, but a study by Axtell et al. (1997) found motivation to transfer was a significant predictor of positive transfer that trainees felt they had achieved after participation in the training.Therefore, it is evident that motivation to transfer plays an important role in improving work behavior and training effectiveness.Therefore, we hypothesize that: Hypothesis 8: Motivation to transfer will be positively related to behavior.
Learning and motivation are both essential for training transfer (Gegenfurtner et al., 2009).Without learning, nothing can be transferred from training to the workplace.Without motivation, nothing will be transferred from learning to the workplace.However, the studies on motivation to transfer lack a consistent framework for understanding the moderator variable affecting the transfer process.According to Expectancy Theory (Vroom, 1964), if learners' individual motives are believed to lead to strengthened performance, they will be more motivated.The focus in past research remains unclear in terms of motivation to transfer's moderating effect, specifically the relationship between learning and behavior change and training.Logically, behavior change will be likely to occur among trainees who succeed in learning from training and desire to apply new KSAs to the workplace.Trainees are likely to be motivated to transfer new knowledge and skills to the job when they feel confident about using new knowledge and skills.The result is that trainees perceive that by using the new KSAs, the new knowledge and skills emphasized in the training will help them to solve work problems.Therefore, we hypothesize that: Hypothesis 9: Motivation to transfer will moderate the relationship between learning and behavior.

Work Environment Characteristic
During the past thirty years, many researchers have tried to provide an answer to the question of which factors influence the transfer process (e.g., Baldwin & Ford, 1988;Cheng & Ho, 2001;Holton et al., 2000;Colquitt et al., 2000;Cheng & Hampson, 2008).Work environment characteristics have often been referred to as the transfer climate or factors perceived by trainees to encourage or discourage their use of KSAs learned in training and in the workplace (Cromwell & Kolb, 2004).Clarke (2002) indicated that social support is the most important factor in the transfer climate, which influences the use of training in the workplace.
Social support has been conceptualized in many ways.One focus in the previous studies was on the type of support providers, such as supervisors or peer support.However, there are still gaps in the empirical studies, in particular concerning organizational support.The previous studies confirmed that support from supervisors and peers is the work environment variable that has the most powerful effect on the transfer process (e.g., Awoniyi et al., 2002;Baldwin & Ford, 1988;Bates et al., 2000;Clarke, 2002;Cohen, 1990;Cromwell & Kolb, 2004;Elangovan & Karakowsky, 1999;Gregoire, Propp & Poertner, 1998;Gumuseli & Ergin, 2002;Holton et al., 1997;Huczynski & Lewis, 1980;Quinones et al., 1995;Richman-Hirsch, 2001;Russ-Eft, 2002;Salas & Cannon-Bowers, 2001;Smith-Jentsch, Salas & Brannick, 2001;Taylor, 1992).Although past research has focused more on the importance of supervisor and peer support, an increase in the use of a framework is found in terms of the effects of organizational support on the transfer process (e.g., Bates et al., 2000;Facteau et al., 1995;Holton et al., 1997).According to organizational support theory, if employees do not perceive that their contributions are appreciated, then it is likely that they may not put an effort that would be beneficial to the organization (Eisenberger et al., 1986).In a training context trainees who do not perceive their organization to be supportive of their contributions might not put forth an effort in perceiving the usefulness of adopting a trained method.Thus, organizations provide material and socioemotional benefits, such as the value placed on learning and development, rewards, and materials and supplies allocated for learning and transferring, to employees in exchange for their commitment and work effort on behalf of the organization.Based on the literature review above, we hypothesize that: Hypothesis 10: Social support will be positively related to behavior.
Prior research confirmed that social support, such as that from supervisors and peer support, influences the training transfer process.In other words, we expect that social support may influence the relationship between learning and behavior.For example, Huczynski and Lewis (1980) indicated that supervisor support was an important factor in transferring the knowledge and skills from training to the job.They found that when trainees were back on the job, only 35 percent of them tried to transfer what they had learned from training.Campbell and Cheek (1989) also supported those relationship and they found that the involvement of supervisors have an impact to transfer of training.Without supervisory support trainees may face the difficulty for transferring newly acquired knowledge and skills from training to the workplace.Furthermore, Cromwell and Kolb (2004) found that support from a trainee's peers is influential in the transfer process.Trainees who perceived higher levels of peer support on the workplace lead to apply the newly acquired knowledge and skills from training to the job.However, empirical results are still limited to exhibiting social support as a moderating effect on the relationship between learning and behavior.When researchers argue for the influence of social support on the training transfer process, there should be an investigation not only of the direct effect of the support, but also of its interaction with the learning level.That means the interaction between the extents of learning, high or low in terms of knowledge skills and attitude, and the degree of social support have an effect on behavioral change.This particular aspect has been lacking in the literature.
While also considering the effect of social support on training transfer above, we assume that trainees who learned more from the training program will change their behavior more if they received stronger social support.Social support will still positively affect behavior among trainees who learned less from the training program, but to a lesser degree.Therefore, we hypothesize that: Hypothesis 11: Social support will moderate the relationship between learning and behavior.

Conceptual Framework
The conceptual framework of this study focuses on testing the progressive causal relationships of Kirkpatrick's model and investigating individual and work environment characteristic variables, which are: learning motivation, self-efficacy, motivation to transfer and social support, as moderators of the relationship between training and its outcome.Specific hypotheses for each of the relationships are illustrated in Figure 1.

Participants
In the present study, a skill certification system for the automotive industry in Thailand was one of the sub-programs under the Automotive Human Resource Development Program (AHRDP), and was expected to be very significant because of its potential impact on the whole industry.AHRDP was implemented from 2006 to 2011, as part of the Japanese official development assistance (ODA) program, in cooperation with the Thai government and private sectors in both countries.Specifically for the skill certification system, at the beginning Japanese experts from an automotive assembler, Nissan, supported knowledge transfer to local prospective examiners and trainers.They in turn transferred the acquired skill and know-how to employees in local firms through training and examination.Through 2012, 363 people were certified in 16 subjects that included theoretical and practical sessions.Questionnaires were distributed to all the participants who passed the skill certification exam after training in the sub-programs, while 228 were valid responses yielding a response rate of 62.8%.
Of the 228 study participants, 148 people participated in examiner training and 225 in trainer training, while the remaining 61 people attended ordinary training.A participant could attend multiple levels and study various training subjects.The subjects attended by trainees included electrical maintenance (11.2%), mechanical maintenance (9.5%), pneumatic circuits and apparatus device assembling (8.8%), metal press work/stamping (8.4%), hydraulic system adjustment (8.2%), three courses on die and mold finishing, electronic device assembly, and plastic injection (6.5%), both ferrous casting and sequence control (6.0%), milling with numerical control (5.0%), machining (lathe, milling) (4.7%), lathe with numerical control (4.5%), both mechanical assembly finishing and mechanical drawing by hand (3.0%), and mechanical drawing by CAD (2.2%).Among the sample, 98.7% of the participants were male.Regarding their age, 48.0% of the samples were between 31 and 40 years old and 40.1% were between 21 and 30 years old, whereas 11.9% were above 40 years old.38.9% graduated from university, and 33.3% graduated from vocational schools.55.5% of the respondents worked for an automotive assembler or automotive parts manufacturer, while others were from universities and training intuitions such as vocational colleges.

Measures
Variables in this study, as well as their corresponding sources, are described below.
Reaction.Twenty-seven items adopted from Morgan and Casper (2000) were used to assess trainees' feelings about and like of a training program.Responses were made on a five-point Likert scale, with 1 = very dissatisfied and 5 = very satisfied.Learning.Based on Kirkpatrick's model, learning refers to the knowledge, skills, and attitude acquired by trainees.Learning aims at understanding trainees' comprehension of instruction, principles, ideas, knowledge and skills from training.The learning measure consisted of sixteen items adopted from previous studies (e.g., Kirkpatrick, 2006;Leach & Liu, 2003).
Results refer to the final results that occurred because the trainees attended the program (Kirkpatrick, 1994).These could include the attainment of organizational objectives and individual benefits.The results consisted of eighteen items adopted from previous studies (e.g., Kirkpatrick, 2006;Leach & Liu, 2003, Velada et al., 2007;Xiao, 1996).
Learning motivation or motivation to learn refers to the desire of the trainee to learn the contents of the training program (Noe, 1986).Learning motivation consisted of six items adopted from previous studies (e.g., Holton et al., 2000;Bell & Ford, 2007;Liao & Tai, 2006;Lima, Leeb & Nam, 2007).
Self-efficacy refers to an individual's belief that he or she can perform a specific task (Bandura, 1986).Consistent with Bandura (1986), because self-efficacy measures must be task-specific, no standardized measures were available and an appropriate instrument had to be constructed (Frayne & Geringer, 2000).We constructed the self-efficacy measure specifically for the trainees who recently received KSAs learned from training.Self-efficacy consisted of six items.
Motivation to transfer refers to the learner's intended efforts to utilize knowledge and skills learned in a training setting to the workplace (Noe, 1986).The motivation to learn consisted of six items adopted from Gegenfurtner et al. (2009).
Social support.This study identified social support from supervisors, co-workers, and the organization.Supervisor support has the critical role of providing reinforcement for learning on the job.Co-worker support focuses predominantly on supporting the use of learning on the job.Organizational support focuses on an organization's provision of material goods, such as transportation, money, or physical assistance to employees for supporting the transfer of training in the workplace.The social support measure consisted of twenty-five items adopted from previous studies (e.g., Elwood et al., 2000;Xiao, 1996;Kupritz, 2002).
In this research, all of the measures, excluding reaction responses, were made using a five-point Likert scale, with 1 = disagree strongly and 5 = agree strongly.As a test of reliability, Cronbach's α was adopted to represent internal consistency.Cronbach's α for each scale of the questionnaire is acceptable (reaction: .709,learning: .665,behavior: .647,results: .639,learning motivation: 665, self-efficacy: 626, motivation to transfer: 801, and social support: .663),with all values greater than a threshold of .60.Therefore we conclude that the items are reliably measuring the defined constructs and variables.

Measurement model assessment.
In accordance with Gerbing and Hamilton's (1996) recommendation, we followed a three-stage approach.First, the measurement scales of latent variables were examined using exploratory factory analysis (EFA) in SPSS 19.Some items were eventually eliminated using this process.Then, all remaining items from the eight measures were entered into a confirmatory factor analysis (CFA) by using maximum likelihood (ML) estimation.For testing whether or not individual and work environment characteristics influence training effectiveness, we applied a conventional method of mean centering to all multiplicative interaction variables in order to avoid multicollinearity.Finally, this study used path analysis via AMOS version 21.The criteria were used to evaluate the fit of the models in this study by taking suggestions from Bollen (1989), andHu andBentler (1995) and all the criteria were satisfied.The scale internal structure fit measures abstract is shown in Table 1.

Results
Means, standard deviations and intercorrelations among all measures of latent variables are reported in Table 2.
Prior to assessing the hypothesized model, we assessed the fit of the model.As shown in Table 1, the fit statistics presented indicate that the hypothesized model yields a poor fit to the data.However, in this study the fitness of the overall model is assumed to be appropriate according to RMSEA; its 90 percent confidence interval and values between .05 and .10indicate a reasonable fit, and values below .05indicate a close approximate fit (Kline, 2005).
With respect to our specific research hypotheses, there were hierarchy relationships of reaction, learning, and behavior to results.Hypothesis 1, 2, and 3 were supported.First, trainee reaction was positively related to learning ( = 0.252, p < .001).Second, learning was positively related to behavior ( = 0.244, p < .001).Third, behavior was positively related to results ( = 0.306, p < .001).The results for the hypothesized model are depicted in Figure 2.
This model also showed that learning motivation and self-efficacy were positively related to learning ( = 0.119, p < .05 and 0.301, p < .001,respectively).These results supported Hypothesis 4 and 6.Furthermore, social support was positively related to behavior ( = 0.285, p < .001)but motivation to transfer was not a significant predictor of behavior ( = -0.038,p > .05).Thus, Hypothesis 10 was supported and Hypothesis 8 was not supported.
As Figure 2 illustrates, the moderating effect of learning motivation, self-efficacy, motivation to transfer and social support as moderators in the main analysis model exists in the relationship between training and its outcome.Specifically, the relationship between reaction and learning was not moderated by learning motivation and self-efficacy, as the interaction terms concerned were not statistically significant ( = 0.080, and 0.073 respectively, p > .05).Thus, Hypothesis 5 and 7 were not supported.However, the relationship between learning and behavior was moderated by motivation to transfer and social support ( = -0.216,and 0.223 respectively, p < .001);these results supported Hypothesis 9 and 11.The sign of the coefficient indicated a negative effect from learning and behavior when perceived motivation to transfer was high.The sign of the coefficient depicted the positive effect of the relationships between learning and behavior when perceived social support was high.
Indirect and total effects of variables that were significant are shown in Table 3.All of the indirect and total effects of variables are found to be positively statistically significant, excluding the negative indirect effect of the interaction term of learning and motivation to transfer on the total effect of results (effect = -0.066).In sum, reaction, learning motivation and self-efficacy explain 18.0 per cent of the variance of learning.Taken together, reaction, learning motivation, self-efficacy and learning account for 23.8 per cent of the variance of behavior directly and/or indirectly.All of the variables, including behavior, explain 9.4 per cent of the variance of results (Figure 2).

Discussion
This research provides support for Kirkpatrick's model to expand the hierarchy relationship of reaction, learning, and job behavior to results.The results of this study fully supported previous findings in the literature on training effectiveness (Alliger & Janak, 1989;Alliger et al., 1997;Leach & Liu, 2003;Kirkpatrick, 1996;Tan et al., 2003;Warr, Allan & Birdi, 1999).That is, the reaction was significantly related to learning.This result is consistent with Alliger et al.'s (1997) meta-analysis and supports Kirkpatrick's (1967) original suppositions on the hierarchical nature of the relationship among the four primary training criteria.Reaction and learning from training play a critical role in the process of training evaluation.A positive reaction may influence an individual's willingness to use newly acquired knowledge and to attend future training programs (Tracey et al., 2001) and learning outcomes are found to have a direct positive effect on behavior or transfer outcomes (e.g., Xiao, 1996).However, few previous studies have investigated the relationship between behavior change and results because trainees may not be able to immediately apply their acquired knowledge and skills to the job effectively due to the long delay between the change in job behavior and the desired organizational results.
Additionally, the results showed a significant relationship between (1) self-efficacy and learning, and (2) motivation to learn and learning.These finding are consistent with the literature on self-efficacy and learning motivation.
First, self-efficacy had a positively significant relationship with learning, consistent with the training literature (e.g., Colquitt, LePine & Noe, 2000;Gist, Schwoerer & Rosen, 1989;Gist, Stevens & Bavetta, 1991;Martocchio, 1994;Simmering & Posey, 2009).The results of this study showed that trainees have more confident on their ability to use newly acquired knowledge and skills from training can learn better.Therefore, trainees who believe in their ability to be experts and succeed in training may be more likely to consider and use training as an instrument for improving and developing their performance in the workplace to maximize learning.
Second, learning motivation had a positively significant relationship with learning.This finding was consistent with those of previous studies (Alliger & Janak, 1989;Chuang et al., 2005;Clement, 1982;Liao & Tai, 2006).Thus, it suggested that, for example, when organizations require employees to participate in training programs more effectively, they should enhance learning motivation among trainees to increase their learning.For that purpose, the workload should be too excessive in order that organizations allow time for them to learn the new knowledge and skills from training.However, the finding, with regard to moderating variables, including self-efficacy and learning motivation, didn't moderate the relationship between reaction and learning.
With regard to the effects of the other factors, motivation to transfer was not a significant predictor of behavior.
On the other hand, the moderating effect from motivation to transfer was found to be significantly negative in the relationship between learning and behavior.This result is in contrast to the expectation that trainees who have more motivation to transfer are more likely to change the degree of their transfer if they learned more compared with cases of less learning.A possible explanation for the result is as follows; the case organizations have unfavorable work environments such as the difference in machine and equipment that could not be captured by the related variables in the model.Subsequently, when those with higher motivation to transfer have more learning, they face more difficulties in transfer and they were less active in their behavioral change based on their learning, in comparison of those with lower motivation to transfer.
Another important finding was the significant relationship between social support and behavior.This finding is consistent with previous research on supervisor and peer support, such as that by Bates et al. (2000), Cromwell and Kolb (2004), Facteau et al. (1995) and Holton et al. (1997).According to the literature, social support had a higher direct effect on behavior change, particularly support from supervisors, peers and the organization, as well as feedback, coaching, opportunities to apply, materials and socioemotional benefits.The results of this study confirmed that social support had a direct effect on behavior.Particularly, supervisors are required to reinforce learning on the job such as providing assistance for solving the problems by new knowledge and skills from training, setting criteria and discussion for applying new knowledge and skills on the job.Support from co-worker is expected through supporting the use of learning on the job and sharing work-related information or knowledge to trainees.The organizations should provide the efficient and flexible workspace for teaching knowledge and skills from training to other employees.Moreover, the findings of this study provide further evidence of the significant moderating effect of social support on the relationship between learning and behavior.
The results demonstrate that when trainees are successfully learning with high social support, they experience more behavioral change on the job after training.In this respect, social support from supervisors, peers, and the organization might be more effective for encouraging trainees to implement new knowledge and skills they have learned from training to their jobs.This would lead to more powerful transfer-enhancement in the workplace.

Limitations and Future Research
Although this study has some important results, several limitations should be discussed.First, this study relied on self-assessment measures, which may have caused some common-method variance problems that may inflate observed relationships between variables, given that measuring behavioral changes is much more difficult because, to be effective, evaluations of behavioral change require a systematic appraisal of behavioral change both before and after course completion.Further, where possible, these appraisals should be performed by multiple sources, including the individual receiving the training, the person's supervisor(s), the person's subordinates, and the person's peers.In addition, where feasible, such evaluations should also include a control, or comparison, group that has not received the training (Ban & Faerman, 1990).Second, this study didn't control for a variety of course features and demographic variables that may influence trainees' experiences and evaluation of the training they received, such as their age, gender, income, and hierarchical position.
Finally, although this study is based on a varied sample of companies, trainees, and types of training courses, the extent to which the results can be generalized to other cultural and institutional contexts remains open to question.Thus, future research should seek to examine the extent to which the present results can be reproduced in different countries and should cover a full set of individual controls.Moreover, the study also suggests the need for better integration of training design in Kirkpatrick's model to better understand training effectiveness.

Research Implications
Based on analytical results and interpretations, we can understand the progressive causal relationship of reaction, learning, and job behavior to results.Additionally, we can understand how individual and work environment characteristic variables, including learning motivation, self-efficacy, motivation to transfer and social support, moderate the relationship between training and its outcome.In this study, the success of Kirkpatrick's four-level model may provide some beneficial information that increases the clarity of which training criteria should be selected and how to adequately measure them.
Additionally, the results of this study confirm the influence of the individual and work environment characteristics on the training outcomes and it has implications for enhancing training effectiveness.Based on the results of this study, we found direct relationships between (1) self-efficacy and learning, and (2) motivation to learn and learning.Organizations can improve learning by ensuring that trainees believe that they have the capabilities to successfully learn the new knowledge and skills from training.This can be improved by ( 1) showing trainees that other employees who have received the training have successfully improved their knowledge and skills and (2) providing information for trainees on how the learner can achieve success under the training context.
Although motivation to transfer as a moderating variable has negative effects on the relationship between learning and behavior, we cannot necessarily proscribe that other training programs be designed in this way.As noted in this study, the role of motivation to transfer in this training is important in behavior change.In motivation theories, the context in which one's motivation rises or declines is explained by different variables and conditions (Lim & Johnson, 2002).Consequently, expectancy theory explains motivation as the relationship between rewards and one's effort (Moorhead & Griffin, 1992).Thus, for the long term organizations are required to raise the motivation to transfer by using appropriate encouragement and rewards for learners who apply learning on the job as well, so they will exploit the opportunities for transfer of training more effectively.
Furthermore, the role of social support in directly affecting behavior change after training and moderating the relationship between learning and behavior demonstrates the practical implications from the training.Organizations should emphasize that trainees who achieve learning will have necessary support from supervisors, peers, and the organization to apply their learned knowledge and skills from training to their work.One way they can do this is by creating a climate in which all trainees perceive that they will receive the necessary support and feedback regarding their performance from the organization, supervisors and peers in order to effectively transfer the training.

Conclusions
This study provided insight into individual and work environment characteristic variables, which are learning motivation, self-efficacy, motivation to transfer and social support, as moderators in the relationship between training outcomes using Kirkpatrick's model.The results of this study expand our understanding of the progressive causal relationship of reaction, learning, and behavior to results.In particular, this study highlighted the direct relationship between (1) self-efficacy and learning, and ( 2

Figure
Figure 1.Conceptual framework ) learning motivation and learning.Although the result of motivation to transfer as a moderating variable has negative effects on the relationship between learning and behavior, the role of social support directly affects behavior change after training and moderates the relationship between learning and behavior.The results of this study confirm the influence of the individual and work environment characteristics on training outcomes and it has implications for enhancing training effectiveness.Furthermore, future research on training evaluation should consider the training design variables beyond the training course that may have interfered with the training outcomes.

Table 1 .
Goodness of fit of scale internal structure and structural model

Table 2 .
Means, standard deviations, and correlations among all observed variables(N = 228)