Peer Ratings

From a meta-analysis of previous research Viswesvaran, Schmidt and Ones (1997) state that peer ratings are one of the most common sources of ratings for appraisal of performance (the other being supervisor ratings). As with subordinate appraisal, peer evaluations provide another unique perspective on the manager's behaviour; one would presume that peers are more aware of the situational constraints surrounding a manager than other external raters, because of the comparative similarity between their roles. Perhaps as a function of this, there is evidence that in comparison to supervisor evaluations, peer ratings are more likely to differentiate effort from performance, and more likely to focus on task-relevant abilities and competencies (Klimoski & London, 1974; Tucker, Kline & Schmitt, 1967). Also, because ratings are usually collected from a number of peers, they are more defensible from a legal standpoint, than ratings by the superior alone (Pollack & Pollack, 1996).

Despite the reported (Viswesvaran, Schmidt & Ones, 1997) widespread use, relatively few studies have been carried out into employees' acceptance of peer appraisals; Cederblom and Lounsbury (1980) were the first to look at user acceptance of an existing programme. They surveyed 174 faculty members on a peer appraisal system that had been in use for six years, and found relatively low acceptance. But subsequent research contradicts this early study. McEvoy and Buller (1987) surveyed employees of a food processing plant, concluding 'employees seem guarded in their endorsement of peer evaluations but are substantially more favourable than suggested by previous research'. They found an overwhelming 84% of employees thought the process was beneficial and should be continued. Wexley and Klimoski (1984) concluded from a review of existing research literature that peer ratings are potentially the most accurate judgements of employee behaviour. To a certain degree this conclusion is supported by research into the psychometric properties of peer appraisals.

Reliability

In a comparison of peer and supervisory ratings of management performance, Viswesvaran, Schmidt and Ones (1997) found from their meta-analysis that when two raters were from the same organisational level (i.e. both peers or both superiors to the manager being appraised) the average interrater reliability estimate was 0.52. This supports previous research which found peer ratings to be of acceptable interrater reliability and temporal stability (DeNisi & Mitchell, 1978; Imada, 1982; Kane & Lawler, 1978; Lewin & Zwany, 1976; Love, 1981; Reilly & Chao, 1982; Schmitt, Gooding, Noe & Kirsch, 1984; Siegal, 1982).

Convergent validity

Peer ratings are often validated by correlating them to supervisor ratings; research generally finds them to be moderately but significantly related. The moderate level of the relationship suggests that peers do indeed have a unique and valuable perspective on co-workers' performance (Pollack & Pollack, 1996). In a meta-analysis, after correcting for measurement error and range restriction, Harris and Schaubroeck (1988) found that the relationship between peer and supervisor evaluation was p = 0.62; between self-evaluations and peer appraisal p = 0.36. They also found that self-peer correlations were lower for managerial/professional employees (r = 0.31) than for blue-collar/ service employees (r = 0.40). Peer supervisor correlations of managerial and blue-collar employees did not differ significantly (r = 0.64 and r = 0.62 respectively).

Research has examined in greater depth the degree of overlap between peer and supervisor ratings of a manager. Viswesvaran, Schmidt and Ones (1997) conducted a meta-analysis of previous research that had looked at convergence of peer and supervisor ratings for different behaviours and rating scales. Using three different models of latent job structure to interpret data presented in their meta-analysis, Viswesvaran, Schmidt and Ones (1997) concluded that rating content does not moderate peer – supervisor convergence.

Predictive validity

When peer ratings are correlated with outcome measures (e.g. promotions, turnover, etc.) research has generally found them to be valid predictors of performance (Pollack & Pollack, 1996). Furthermore, when compared to other kinds of predictors, peer evaluations, along with supervisor ratings, assessment centre ratings and work samples, are among the best predictors of performance.

Bias

Despite the seemingly sound psychometric properties of peer appraisals, it has been suggested that organisations are reluctant to include peers in the appraisal process, because of the various forms of bias ratings are susceptible to. Possibly the most commonly cited reason in the research literature (and most frequent explanation given by organisations) is that of friendship bias. Friendship bias is negatively correlated to user acceptance of peer appraisals – the more managers that see their ratings of their peers influenced by their friendship with them, the less favourable they are to the use of peer appraisal (McEvoy & Buller, 1987; McEvoy, 1990). However, much of this research is based on samples from organisations that do not actually have a peer appraisal programme in operation – in practice it remains to be established whether these fears are justified. It also remains to be established what influence friendship has on the favourability of feedback ratings. This is discussed in greater detail later in this chapter.

Although friendship bias is negatively correlated to user acceptance of peer appraisals, research has indicated that this does not reduce the reliability or validity of such appraisals. Other suggested forms of bias are subgroup effects, reliance on stereotypes in ratings and the possibility of retaliation in subsequent ratings (DeNisi & Mitchell, 1978). However, this could be true of ratings from any rating source; there is no evidence from research showing peer appraisals to be overly susceptible to these forms of bias when used for developmental purposes.

Internal and External Customers

A minority of systems invite ratings of managers from internal or external customers. To date, no research literature has been produced on the psychometric properties of this rating group, if only because sample sizes are so small.

In view of the lack of research, one can only speculate on the value of such ratings. They would be especially relevant to customer-facing roles, and it may be that this aspect of interpersonal relationships will grow in significance. Many organisations are enabling employees to become more autonomous, and with the evolution of technology, practices such as tele-working are increasing. The end result is that employees may actually have very little contact with their immediate superiors, subordinates or peers but an increased level of direct contact with internal/extemal customers. So, as well as being able to comment upon the timeliness and quality of product delivery, customers may also be able to provide insight into the processes underlying the individual's effectiveness.

 
< Prev   CONTENTS   Next >