Including Organizational Citizenship Behavior in Performance Evaluations : An Investigation of Employee Reactions

This study examined how employees react to formal evaluations of organizational citizenship behaviors in performance appraisals. Using a sample of 107 business students with relevant job experience, this experimental study found that their reactions to such formal inclusion of organizational citizenship behaviors depended on their task performance. Respondents with high task performance, compared with their low task performance counterparts, reported lower satisfaction, perceived distributive, and perceived procedural justice when organizational citizenship behaviors were weighted heavily.


Introduction
More than ninety percent of large organizations conduct some types of performance evaluations (Walsh, 2003).It is not surprising that research has investigated various evaluation issues, including rating errors and accuracy, rating scales, feedback, impacts on performance, and the role of performance appraisals in selection test validation (Arvey & Murphy, 1998;DeNisi & Pritchard, 2006).One research stream, of interest to practitioners and scholars alike, looks at how employees react to performance evaluations (Keeping & Levy, 2000).Evaluation reactions are considered critical to the acceptance, use, and success or failure of an appraisal system (Cardy & Dobbins, 1994;Giles & Mossholder, 1990;Keeping & Levy, 2000).They are usually seen as better indicators of the appraisal system viability than are psychometric indices (Bernardin & Beatty, 1984) and even as ultimate effectiveness criteria (Cardy & Dobbins, 1994).Many facets of reactions, such as satisfaction with a performance appraisal system and its elements, perceived accuracy, justice, and utility have been examined in previous literature (Keeping & Levy, 2000).This body of research, however, has seldom paid attention to the broadened job performance domains that include not only task performance but organizational citizenship behavior (OCB, Organ, 1988).
OCB comprises of a collection of employee behaviors that fall outside of employees' formal job descriptions, but are important for the success of organizations.Various workplace role behaviors fall under the rubric of OCB, such as helping, sportsmanship, organizational loyalty, organizational compliance, and individual initiative (Podsakoff, Mackenzie, Paine, & Bachrach, 2000).These behaviors, also referred to as contextual performance, contribute to organizational effectiveness by supporting the social-psychological work environment (Borman & Motowidlo, 1993).Many studies have examined OCB in performance evaluations (e.g., Borman, White, & Dorsey, 1995;Lowery & Krilowicz, 1994;MacKenzie, Podsakoff, & Fetter, 1991;Van Scotter & Motowidlo, 1996).They showed that OCB can be distinguished from task performance and explain at least as much variance in managerial evaluations of employees' overall performance as do task behaviors (Podsakoff et al., 2000).Although these findings are important, their sole focus on the distinction of OCB from task performance makes it difficult for organizations to determine whether and how OCB should be formally embraced in performance evaluations.To inform such decisions, this study examines employee reactions to the inclusion of OCB in performance evaluations.

Employee reactions to performance evaluations
Performance evaluations have been likened to the "Achilles' heel" of human resources management.By providing developmental feedback and coupling with administrative decisions (e.g., pay, promotion, or discharge) (Rynes, Gerhart, & Parks, 2005), performance evaluations intend to enhance performance, but turn out to be viewed unfavourably by employees.A survey of more than 1,190 employees reported that only 30 percent of them thought that the process helped them improve their performance (Watson Wyatt World Research Report, 2004).This negative reaction could lie in the fact that both mangers and employees consider performance evaluations as a personally threatening process (Pulakos, 2004), suggesting the necessity of considering employee reactions in performance evaluations.
Various facets of reactions, such as satisfaction, justice, and utility have been considered in the literature.Satisfaction with performance evaluations includes satisfaction with the appraisal review session, with the appraisal system, and with performance ratings (Cawley, Keeping, & Levy, 1998;Keeping & Levy, 2000).Fairness, including distributive, procedural, and interactional types (Colquitt, Conlon, Wesson, Porter, & Ng, 2001), is also an important concept in performance evaluations (Bretz, Milkovich, & Read, 1992).It has been shown that both distributive justice (i.e., the actual ratings one receives) and procedural justice (i.e., the process by which the respective ratings are decided) are important in performance evaluations (Greenberg, 1986).

Citizenship behavior in performance evaluations
Available research of OCB in performance evaluations has centered on the extent to which it can make unique contributions to overall performance judgments.These studies sampled various jobs (Avila, Fern, & Mann, 1988;Conway, 1999;Johnson, 2001), used multiple sources of rating (self, peers, and supervisors, Van Dyne & LePine, 1998) and objective performance measures (e.g., MacKenzie et al., 1991), and employed cross-sectional (MacKenzie, Podsakoff, & Fetter, 1993;Motowidlo & Van Scotter, 1994), longitudinal (Van Dyne & LePine, 1998), and policy capturing designs (Rotundo & Sackett, 2002).Podsakoff et al. (2000) summarized the variance accounted for by task performance and OCB and concluded that, on average, OCB contributes to overall performance evaluations at least as much as does task performance, strongly supported by the latest meta-analytic estimate (Podsakoff, Whiting, Podsakoff, & Blume, 2009).Two robust conclusions can therefore be drawn from these findings: OCB is distinguishable from task performance, and OCB is considered vastly important in managerial evaluations of overall performance.

Employee reactions to including OCB in performance evaluations
These studies of OCB in performance evaluations have a narrow focus and are essentially descriptive.Possible consequences, such as changes in reactions, motivations, and individual and collective performance, of formally recognizing OCB in performance evaluations have been seldom explored.Even though managers do consider OCB (Podsakoff et al., 2000;Podsakoff et al., 2009), the distinction between formal inclusion of OCB in performance evaluation and informally taking OCB into account by managers are important.Formality demands OCB to be part of the performance criteria explicitly when employees' performance is appraised, which sets constraints on each evaluator to consider.But the impacts of formal inclusion of OCB do not seem to be straightforward.
Indeed, there are both advocacy and cautions.The reciprocity perspective supports a formal recognition since exclusion of OCB in performance evaluations may result in an overlook of employee contributions (Bernardin, Hagan, Kane, & Villanova, 1998;Johnson, Holladay, & Quinones, 2009).This can causes perceptions of low satisfaction, unfairness, and decreased motivation to contribute.Bernardin et al. (1998) contended that omission of citizenship behaviors places raters in a compromised position where they might adjust ratings of technical performance to account for citizenship behavior.This increases the likelihood that mangers' personal likes and dislikes influence their ratings of employees' performance (Organ, Podsakoff, & MacKenzie, 2006).
But cautions have been raised about possible difficulties and negative outcomes of such inclusion.Job descriptions, produced by the job analysis process, form the foundation of human resource practices and are essential to meet legal requirements that employment decisions are job relevant.But job analysis usually produces descriptions that prescribe work activities directed at task performance and usually do not include citizenship behaviors (Werner, 2000).Turnipseed and Wilson (2009) reported that only small percentages of employees considered OCB to be part of their formal job descriptions (18 to 26% for different OCB dimensions).As such, evaluating citizenship behaviors formally may face legitimacy challenges.Moreover, for employees to accept the inclusion of OCB in performance evaluations, they should perceive these behaviors to be a required part of their jobs.In favour of such inclusion, studies have shown that on average, OCB is perceived by employees more as part of their job than outside of it (Morrison, 1994;Tepper & Taylor, 2003).However, OCB might differ from task performance in respect to their enforceability.In contrast to task performance, OCB is considered more difficult to be enforced (Organ, 1997).As a result, formally evaluating OCB may face the legitimacy challenge and may not always result in positive employee reactions.Another formal inclusion challenge relates to the practical difficulty.Citizenship behaviors, such as altruism, initiatives, loyalty, and courtesy look much like trait-oriented descriptors (Werner, 2000).Due to their vagueness, subjectivity, and rating error-prone nature (Wiese & Buckley, 1998), trait measures have been heavily criticized and are suggested that they be abandoned.Due to the trait-like image of citizenship behaviors, employees may react negatively to the evaluations of them.
With these competitive perspectives, the net reactions to evaluating citizenship behaviors in formal performance evaluations may lie in how heavily they are weighted.In the only study on this matter, Johnson et al. (2009) found that participants reported the highest level of distributive justice when OCB were weighted 30-50%.The moderate weight assigned to OCB recognizes employee contributions, and at the same time avoid the overwhelming influence of OCB on overall performance ratings, and thus may lead to optimal reactions.Hypothesis 1: Moderate weighting of OCB in performance evaluations result in positive employee reactions; exclusion or very heavy weighting result in negative reactions in terms of satisfaction, distributive justice, and procedural justice.

Task performance as a moderator of the inclusion of OCB on employee reactions
Individual differences may moderate the impacts of including citizenship behaviors in performance evaluations on employee reactions.Job performance, as a multi-dimensional construct, apparently includes task performance.Task performance involves completing tasks usually prescribed in job descriptions that contribute to the technical core of the organization (Borman & Motowidlo, 1993).Current appraisal techniques are usually employed to capture this form of behaviors (Whiting, Podsakoff, & Pierce, 2008).It is not surprising that managers assign considerable importance to task performance when employees' overall performance is evaluated (Rotundo & Sackett, 2002;Podsakoff et al., 2000).Because weights assigned to task performance and citizenship behaviors are competitive in overall performance evaluations, a larger weight assigned to OCB results in a smaller weight being assigned to task performance.Therefore, employees' task performance may affect how they react to the inclusion of OCB in performance evaluations.As citizenship behaviors are distinct from task performance (Borman & Motowidlo, 1993;Organ, 1988), good task performance may not necessarily equate with good citizenship performance.Including citizenship behaviors in formal performance evaluations may result in lower overall performance ratings for those who have high task performance but low citizenship performance, compared to the situation when only task performance is considered.This can result in more negative reactions from employees with higher task performance.Therefore task performance, may moderate the impacts of including citizenship behaviors in formal performance evaluation systems on employee reactions.Specifically, employees with higher task performance react less positively to including OCB in performance evaluation systems.
Hypothesis 2: Employees reactions to the inclusion of citizenship behaviors in performance evaluations depends on their task performance, such that those with higher task performance react more negatively in terms of satisfaction, distributive justice, and procedural justice.

Participants
One hundred and forty students enrolled in 3 rd and 4 th year business courses in a comprehensive university in Canada were solicited to participate in this study.They were informed of voluntary participation and offered a coffee shop gift certificate.Usable surveys were received from 107 students, resulting in a 76% response rate.The survey sample consisted of 45% females and 55% males with an average age of 23.03 (SD = 5.77).The average total work experience for the population was 5.44 years (SD = 5.24).

Procedure
Each participant was presented with a performance scenario, which described the performance of task behaviors and organizational citizenship behaviors of a hypothetical employee.Ratings of the individual's task performance, citizenship performance and overall performance ratings from his/her supervisor were also provided in the performance scenario.In order to mimic a relevant scenario for participants, the researcher asked them to choose from one of three job scenarios that best matched their past work experience.
To develop the job performance scenarios, we took four steps.First, a group of business students were consulted in order to gather information about their relevant job experiences.A second group of business students were then asked whether they had worked on one of the jobs that had been mentioned in the previous group.In this process, we identified three most relevant jobs: fast food restaurant food attendant, retail store salesperson, and restaurant waiter or waitress.Second, to describe task activities included in each scenario, the National Occupational Classification (NOC) was used, which provides a standardized language for describing work performed by Canadians in the labour market.The NOC codes for fast food restaurant attendant, retail store salesperson, and restaurant waiter or waitress are 6641, 6421, and 6453 respectively.Our participants typically had 7-12 months of experience relevant to the job scenarios.Third, to describe citizenship behavior in each performance scenario, the five dimensional definitions (altruism, conscientiousness, courtesy, sportsmanship, and civic virtue) of the most widely used OCB measure were used (Podsakoff, MacKenzie, Moorman, & Fetter, 1990).A sample item is "assisting colleagues on work related matters".Fourth, to determine the hypothetical overall managerial performance rating presented in each performance scenario, a weight of one of the 0%, 25%, 50% and 75% was assigned to citizenship behaviors in each performance scenario to arrive at each overall performance rating (thus, the corresponding weights assigned to task behaviors are 100%, 75%, 50%, and 25% respectively).In doing so, we manipulated the extent to which OCB is formally considered in performance evaluations.
Participants were asked to assume that he or she were the employee described in the scenario and had received performance feedback from their supervisor.They were asked to provide an overall performance rating of their own based on the performance information presented in the scenario.They were then asked to indicate their satisfaction with overall performance ratings, perceived justice of overall performance ratings, and procedural justice.Of the returned 107 surveys, 26.2% of respondents filled out the restaurant waiter/waitress survey, 59.8% filled out retail sales survey, and 14.0% filled out the fast food restaurant attendant survey.Of those 107 returned surveys, the distribution of OCB weights was: OCB 0% weight, 22.4%; OCB 25% weight, 30.8%;OCB 50% weight, 24.3%; and OCB 75% weight, 22.4%.

Manipulation Check
Each participant was asked to rate their own overall performance based on ratings of task performance on job duties and OCB described in each job scenario using the scale: 1-20: very poor, 21-40: poor, 41-60: average, 61-80: good, 81-100: excellent.We checked if the information cues in the performance scenarios actually influenced participants' responses.Regression analysis showed that task performance and OCB ratings explained 30% of the variance in participants' own overall performance ratings (task performance: ß = .51,p < .01;OCB: ß = .32,p < .01;R 2 = .30).This analysis supported that our manipulation of task performance and OCB did influence participants' responses.

Measures
Prevalent measures were adopted.Employee reactions to performance evaluations that include OCB were all assessed on 7-point scales ranging from 1-strongly disagree to 7-strongly agree.Satisfaction with overall performance ratings was measured by four items from Keeping and Levy's (2000) scale.A sample item is "the overall rating was an accurate evaluation of my performance."Four items from Korsgaard and Roberson's (1995) scale were used to measure distributive justice of overall performance ratings.A sample item is "the appraisal of my overall performance was fair".The procedural justice scale developed by Keeping, Makiney, Levy, Moon, and Gillette (1999) was adapted to measure participant's procedural justice of performance criteria and procedural justice of criterion weights.Each was assessed by three items.A sample item is "the criteria used to evaluate my performance are fair" and a sample item of the latter is "the weights assigned to criteria used to evaluate my performance are fair".
Participants were also asked to provide the length of experience working on a job similar to the one described in the job scenario in the questionnaire.The participants' typical experience relevant to the job scenarios was between 7-12 months, suggesting they might have been able to relate to the performance scenario they were asked to imagine.In addition, we asked them to provide total amount of work experience, gender, and age.

Results
Table 1 presents means, standard deviations, reliabilities, and inter-correlations for the study variables.The internal consistency coefficients (s) range from .87 to .95.Multiple regression analysis was employed to examine whether the inclusion of citizenship behaviors in performance evaluations affects employees' perceptions of satisfaction, distributive justice, and procedural justice.Dummy regressors were created for the categorical variables of job types, OCB weights, and gender.As shown in Step 1s of Table 2, contrary to our hypothesis 1, including OCB in performance evaluations did not have significant main effects on employees' reactions of satisfaction with performance ratings, distributive justice, procedural justice-criteria, and procedural justice-weights.
Hierarchal regression analysis was used to test the interaction effects in hypothesis 2: whether employees with high/low levels of task performance react differently to the inclusion of citizenship behaviors in performance evaluations.With each reaction variable as the dependent variable, in Step 2 of Table 2, in addition to variables included in Step 1, the product terms of OCB weight dummy regressors with task performance were entered into the regression equation.Shown in Table 2, the interaction effects of OCB weights and task performance were not statistically significant when the procedural justice-criteria was the dependent variable.But there were significant interaction effects when satisfaction, distributive justice, and procedural justice-weights were considered.Specifically, for both satisfaction and procedural justice-weights, the interaction effects of OCB weights and task performance were statistically significant when OCB was assigned a weight of 75%; for distributive justice, this interaction effect was statistically significant when OCB was assigned a weight of 50% or 75%.These significant interaction effects were plotted in Figure 1.It shows that increasing weights assigned to OCB in performance evaluations reduced respondents' satisfaction with performance ratings, their perceptions of distributive justice, and perceptions of procedural justice-weights more significantly when they have higher levels of task performance.

Discussion
Citizenship behaviors have been shown to be as influential as task activities in how managers evaluate employees' performance and are considered to be an important element in the job performance domain (Borman & Motowidlo, 1993;Organ, 1988).While both citizenship behavior and performance evaluations are important in organizational settings, little research has examined possible impacts of including OCB in formal performance evaluations.This study looked at how various weights assigned to OCB may influence employee reactions and how these reactions are contingent on their levels of task performance.
Our study found no evidence that formally including OCB in performance evaluations has significant main effects on employees' reactions in terms of satisfaction with the performance ratings, distributive justice, and procedural justice of evaluation system (including perceived criteria justice and perceived weight justice).Despite these non-significant main effects, the present study did find that employees react differently to the OCB inclusion depending on their levels of task performance.Specifically, respondents with higher task performance reacted more negatively when higher weights were assigned to citizenship behaviors.They reported significantly lower satisfactions when OCB was assigned a significant weight of 75% and perceived the 50% or 75% weight to be unfair.
These findings have practical implications.They suggest that it can be quite complex to assess citizenship behaviors in formal performance evaluations since individuals can react differently to such inclusion and their differences can not be ignored.Therefore, organizations and managers are advised to avoid assigning heavy weights to OCB in performance evaluation decisions.Moreover, they should consider contextual factors (e.g., employee preference) when identifying appropriate weights to OCB.A local study can be conducted to find out the optimal weights in order to maximize positive employee reactions.Moreover, communication of plausible rationales to employees, particularly to those higher task performers, may aid their acceptance of having OCB in performance evaluations.
The strength of this study lies in its experimental design, which enables an examination of causality.However, possible limitations of this research should be mentioned.This study focused on employees' reactions when OCB was formally incorporated in performance evaluations.Other important consequences, such as changes in employee behavior, group, and organizational performance were not included.These issues can be investigated in future research.In addition, the use of a student sample limits the generalizability of research findings to other contexts.However, our careful choice of jobs and matching scenarios with participants' work experience might have mitigated these concerns.Another potential limitation was our use of scenarios.Students might not have been able to fully connect the scenarios with their own experience.In an attempt to offset this limitation, in addition to choosing student-relevant jobs and matching scenarios with their work experience, a manipulation check was performed.The information cues of task performance and OCB did significantly influence participants' evaluations of their own overall performance.In addition, the sample size of this study is relatively small, which may have resulted in the non-significant main effects.Thus, field studies with larger samples should be conducted

Figure 1 .
Figure 1.The interaction effects of task performance and OCB weightings on employee reactions