Rater Errors in Performance Appraisal

Rater Errors in Performance Appraisal

Errors in performance appraisals: Why they occur and how to mitigate their effects

When selecting the criteria used in performance appraisals, supervisors must often strike a delicate balance between objective and subjective measurements. On one hand, if candidates are solely measured in terms of their numerical productivity for the organization, employees will no doubt protest that this is unfair, given that objective criteria fails to take into consideration the impact of the external market environment and the effects of other workers upon departmental output and efforts. However, subjective performance appraisals can be affected by the unconscious or conscious human biases of the rater. It is critical that performance appraisals are accurate, given that "inaccurate performance appraisals can (a) diminish the link between pay and performance, (b) potentially lead to lawsuits, and (c) inhibit development of associates" (Curtis, Harven & Ravden 2005, p.42).

According to Ami B. Curtis, Richard D. Harvey and Daran Ravden's article "Sources of political distortions in performance appraisals" once upon a time, "it was assumed that rating errors were made unconsciously, thus, researchers concentrated on developing rating instruments and training that would help supervisors avoid these unconscious errors (Arvey & Murphy, 1998). Contrary to this belief, interviews conducted by Longenecker, Sims, and Gioia (1987) uncovered that supervisors are aware of their rating errors and admit to biasing results for political purposes" (Curtis, Harven & Ravden 2005, p.42). Organizational political needs could include, but are not limited to, pressure to promote certain individuals because of their connections, status, or identity, or to advance certain departmental interests. However, when the purpose of pay-for-performance is undercut, the morale of the most competent employees, the workers whom the organization needs the most, will suffer. In contrast, employees who have received unfairly flattering appraisals may become difficult to terminate when the organization desperately needs to fire them, and may even have grounds for a lawsuit, given that their poor efforts have no paper trail. Areas of organizational need that demand further training of employees will also go overlooked if reviews are not critical as well as laudatory.

One contextual factor that can encourage politically-motivated distortions is the purpose of the appraisal (Curtis, Harven & Ravden 2005, p.45). A pro forma review designed to validate an already-budgeted-for raise might fall into such a category, much like a student's grade for an assignment that is designed to be 'extra credit.' Many rating systems, partially because of the fact that it is also easier to give someone a good or average rating than a poor rating that could impact an employee's career, foster the Lake Woebegone effect, where 'everyone is better than average.' Forcing reviewers to give critical as well as positive feedback, regardless of the grade bestowed upon the employee (after all, no one is perfect) is one way to guard against the tendency for reviewers to give overly positive assessments. Additionally, the need to include mixed comments ensures that a paper trail exists if there is a need to fire an employee. The critical feedback, if consistently given and not remedied by the employee, can provide a justification.

One institution that tends to encourage inflated performance ratings is so-called downward accountability, or forcing the superior and the subordinate to meet to discuss the subordinate's appraisal. It can be personally uncomfortable to justify face-to-face a wholly negative rating, so superiors often err on the side of being overly positive. However, by holding managers accountable for the accuracy of their ratings, the downward accountability effect can be thwarted. "Whereas downward accountability has been found to increase leniency, evidence suggests that upward accountability will serve to decrease leniency error…[managers] should be heavily influenced by the thought of having to explain the ratings to the ratee face-to-face (Klimoski & Inks, 1990). The desire to avoid the tension of having to give negative feedback should overpower their desire to be accurate (Blumberg, 1972; Katz & Kahn, 1978; Tessor & Rosen, 1975). If raters are held accountable to their superiors (upward accountability), they should want to rate more accurately to appear competent (Simonson & Nye, 1992), and the desire to avoid tension associated with the ratee should not be present" (Curtis, Harven & Ravden 2005, p.47). Upward accountability makes an honest evaluation in the manger's personal interest, unlike downward accountability, which has the opposite effect (but which can circumvent unfairly negative tendencies).

These principles make a great deal of logical sense: rating an individual is an act of organizational performance, with its own tangible and intangible internal rewards. Having upward and downward accountability can counteract the psychological tendencies that give rise to overly positive or negative reviews. Yet more nefarious, concealed rater goals can also affect results in terms of differences in the goals pursued by various raters. "Raters pursue a number of goals when completing performance appraisals.and these goals may lead them to give ratings that can appear psychometrically suspect. For example, a rater who is lenient…might not be making a judgment error at all but rather might be making a carefully calculated decision that it is better to give all subordinates high ratings than to give low ratings to poor-performing subordinates. Uniformly high ratings might lead to better pay for subordinates, more harmony in the workgroup, better supervisor -- subordinate relations" (Murphy et al., 2004, p.158). Of course, these goals may not be due to the rater's desire but to the superior's directives, implied or explicit, to the rater, such as the fact that the organization wants more stringent performance criteria applied to certain divisions and more lenient criteria applied to others. Regardless, upward accountability will provide little assistance to accuracy in the face of such powerful ulterior motives.

The findings that organizational and personal goals affect ratings is significant when any reform of a rating system is undertaken, given that until now the prevailing assumption was that raters were capable and desirous of giving accurate reviews, but did not based upon a lack of skills and knowledge. Rather, it is more likely that "a rater who intends to use performance appraisal as a means of motivating his or her subordinates will give ratings that are most likely to encourage future performance, not necessarily ratings that accurately reflect past performance. Many of the support systems and interventions in performance appraisal (e.g., training, scale refinement) appear to be based on the questionable assumption that raters are trying their best to provide accurate ratings but that they lack the skills and tools to do the job. It is more likely that raters are capable of giving accurate ratings but are often motivated to give ratings that advance goals such as motivating their subordinates or preserving group harmony" (Murphy et al., 2004, p.158).

Additionally, there is often a contradiction between organizational goals, missions, and aims -- organizations today emphasize teamwork and togetherness in their rhetoric, and often in their actual design of projects, given the dominance of team-based learning and management. However, the pay-for-performance system is highly individualized and competitive. Team-based strategies can make it very difficult to adequately measure employee output and objectively factor this into a performance review. This can increase the use of subjective criteria used in such reviews, as managers may feel there is little difference between employees, as the employees as individuals are unknown quantities.

Performance appraisal errors are also difficult to guard against because of natural psychological tendencies on the part of the reviewer. For example, there is the psychological phenomenon known as 'recency bias,' which reflects the fact that more recent performance tends to seem more salient and important (Staudenmayer, 2002, p. 583). Someone who performed exceedingly well on a recent project, but is an adequate employee most of the year might receive a higher or comparable review to someone who is a strong performer, but lacked a showy 'finale' on a project before the review process began. A related problem is the effects of contrast bias, whereby a very strong performer can make an average performer seem inadequate, or more commonly an adequate performer in an office full of individuals showing subpar performance might appear to be stellar (Staudenmayer, 2002, p. 583).