silverman 1984

PERSONNEL PSYCHOLOGY 1984, 37

REACTION OF EMPLOYEES TO PERFORMANCE APPRAISAL INTERVIEWS AS A FUNCTION OF THEIR PARTICIPATION IN RATING SCALE DEVELOPMENT

STANLEY B. SILVERMAN The University of Akron

KENNETH N. WEXLEY Michigan State University

A substantial amount of research has been conducted on the performance appraisal feedback interview. The purpose of the present field study was to see whether employee involvement in the development of rating scales used in the feedback interview affected ratees perceptions of the interview. Behaviorally anchored rating scales (BARS) were developed for five job families in a midwestem hospital. A participation group consisted of employees from each of the job families who were involved in all phases of BARS construction. A nonparticipation group involved an equal number of employees from each of the job families who were not involved in the BARS development. The results showed that participation in BARS construction led to favorable perceptions regarding the performance appraisal interview process as well as positive outcomes.

A great deal of research on performance feedback has appeared in the literature in recent years (Burke, Weitzel and Weir, 1978; Ilgen, Fisher and Taylor, 1979). Much of this literature has focused on the relationship between performance appraisal interview process characteristics (e.g., opportunity for employee participation, amount of derogation) and outcome measures such as motivation to improve and satisfaction with the appraisal (Burke et al., 1978; Greller, 1975, 1978; Nememff and Wexley, 1979; Wexley, Singh, and Yukl, 1973). Reviews of this literature by Wexley (1979) and Burke et al. (1978) suggest six general characteristics of effective interviews: participation, support, goal setting, discussing problems, minimum of criticisms, and split sessions. Friedman and Cornelius (1976), although they were concerned more with psychometric characteristics as the dependent variable, showed that rater participation in scale construction can lead to greater convergent

Requests for reprints should be addressed to Stanley B. Silverman, Division of Associate Studies, The University of Akron, Akron, Ohio, 44325.

0 1984 Copyright Personnel Psychology, Inc. 703

704 PERSONNEL PSYCHOLOGY

validity, less halo, and lower levels of variance attributable to rating errors.

The purpose of the present study was to investigate the impact of participation in the development of behaviorally anchored rating scales (BARS) (Jacobs, Kafry, and Zedeck, 1980) on ratees reactions to the performance appraisal interview that they received from their managers. Extrapolating from the results reported above and previous findings on the effects of participation in performance appraisal interviews (Kay, Meyer, and French, 1965; Wexley et al., 1973), it was hypothesized that ratee participation in the construction of the instrument would result in more favorable perceptions by them of various process characteristics such as their opportunity to participate, the supportiveness of their managers, their contribution, and the utility of the session. It was also hypothesized that ratee participation in instrument development would generate greater ratee motivation to improve performance and more satisfaction with the appraisal. It was believed that these effects would occur because ratee participation should provide a better understanding of the appraisal process, more trust in the instrument used, and more feelings of ownership and involvement in the entire appraisal process.

Method

BARS Developmen1

BARS were developed in a small hospital for each of the following five job families: clerical (secretaries, typists), non-clerical staff (maintenance, housekeeping), technical and professional (lab technicians, pharmacists), nursing (L.P.N.s, R.N.s), and managementlsupervisory. An approach adapted from Smith and Kendall(l963) incorporating the following four steps was undertaken: 1) generation and definition of performance dimensions; 2) generation of examples of effective, average and ineffective behaviors; 3) simultaneous retranslation and scaling of each item; and 4) final instrument development. Employees who participated in the development of the 7-point scales for their job family were involved in only the first three steps; the employees managers were not involved in any of these steps. The authors then chose the items for the final instruments based upon their mean values and standard deviations of less than 1.5. The clerical and non-clerical BARS each contained four scales, the technical/professional and the managementlsupervisory each contained five scales, and the nursing BARS contained seven scales.

Appraisal Questionnaire Developmenl Based upon a review of the performance appraisal interview literature,

SILVERMAN AND WEXLEY 705

the authors assembled a 34-item questionnaire tapping employee reactions to 10 aspects of the performance appraisal interview (Burke et al., 1978; Greller, 1975, 1978; Wexley et al., 1973). A four-point rating scale was used with each item. The scale ranged from (1) Not at all, I do not feel this way at all, Does not describe it at all, Describes it very poorly to (4) Completely, I feel exactly this way, Describes it ex- tremely well. Shown below are the 10 aspects with their respective alpha coefficients in parentheses followed by an example of an item: Perceiv- ed Utility (.88)-I learned a lot from the appraisal; Anxiety/Tenseness (.75)-The interview was upsetting to me; Derogation (.47)-My manager really did not have enough information about my performance; Supportive Appraisal Behavior (. 80)-My manager praised me for what I had done well; Opportunity to Participate (.59)-My manager gave me the opportunity to state my side of the issues; Degree of Criticism (.52)-My manager was quite critical of my performance; Goal Setting (. 84)-Specific objectives were set; Contribution (.77)-I made sugges- tions about how the job might be done differently; Satisfaction with the Appraisal (.63)-I am satisfied with the interview; and Motivation to Improve-At the end of the interview I really wanted to improve my performance. All aspects contained between three and five items with the exception of Degree of Criticism which contained two items. No alpha coefficient could be calculated for Motivation to Improve because it consisted of only one item. The intercomlations among the 10 variables were found to range from -.43 to .77 with an average correlation, calculated using Fisher z Transformations, of .33. Motivation to Im- prove and Satisfaction with the Appraisal are the performance appraisal interview outcome measures, whereas the remaining eight aspects measured by the questionnaire are the appraisal interview process characteristics.

Sample and Procedure Sixty-five employees from a 120-bed midwestern hospital employ-

ing 240 individuals participated in the study. All 65 employees filled out the appraisal questionnaire immediately following a meeting with their manager to discuss their performance as related to the BARS. Ten managers conducted the performance appraisal interviews with the 65 employees. None of these managers had received any previous formal training in conducting feedback interviews. These managers were told to distribute a copy of the appropriate BARS to each of the 65 employees in order to familiarize themselves with the scales that would be discuss- ed during their performance appraisal interview. Out of the 65 employees, 32 participated in the development of the BARS while 33 did not. The employees in the participation and nonparticipation groups were evenly


distributed across the five job families. Each of the ten managers held appraisal interviews with approximately the same number of employees from the participation group and the nonparticipation group. Performance appraisal interviews were conducted annually at the hospital, and all the interviews involved in the research were conducted approximately ten months after the previous interviews. All interviews were conducted over a two week period that occurred three months after the completion of the BARS. All employees knew that these BARS ratings were done for administrative purposes and would be used in the same fashion as previous ratings. The 65 employees were chosen randomly from within each of the three shifts and each of the five job families throughout the hospital. The 32 employees that participated in the BARS development were chosen in a similar random basis.

Before the project began, the entire hospital was sent a memo explaining that the rating scales were being designed for the staff and that some employees would be selected on a random basis to help provide information for the development of the scales. The 32 employees who participated in the development of the BARS attended all meetings, while the remaining 33 did not participate in any stage of the scale development. The 32 participating employees were asked not to discuss the specifics of these meetings with their managers or any of the hospital employees. Therefore, the managers were not aware of who had participated in these scale development meetings. Even though the employees had been involved in previous annual performance appraisal interviews with their managers, none of the employees had ever participated in scale development prior to this research project.

Results and Discussion To ensure that those individuals who participated in the development

of the BARS showed no difference in their job performance from those who did not participate, an average score was calculated for each individual on their respective set of BARS. T-tests were then performed between the two groups and no significant differences were found. T- tests were also conducted in order to investigate whether ratees who were involved in the BARS development differed in their perceptions of the appraisal interviews from those ratees who were not involved. A stepwise discriminant analysis was then conducted among the variables in order to select the most useful discriminating variables.

Table 1 shows the means, standard deviations, and the resultant t- tests. Those employees who participated in the development of the scales on which they were rated differed in their perceptions of the appraisal interview process characteristics as well as the outcome measures. Specifically, those ratees who participated perceived the appraisal in-


TABLE 1 Means, Standard Deviations and t Values For Ratees

Perceived Differences On Feedback Characteristics And Outcomes Based Upon Participation In Development of BARS

Feedback Characteristic Or Outcome Sample Mean SD t

Utility Participation 3.26 0.53 3.87**

Anxiety /Tenseness Participation 1.35 0.67 0.55 No Participation 2.60 0.75

No Participation 1.26 0.51

No Participation 1.44 0.44 Derogation Participation 1.36 0.52 -0.69

Supportive Appraisal Behavior Participation 3.90 0.22 2.26*

Opportunity to Participate Participation 3.73 0.41 2.92**

Degree of Criticism Participation 1.58 0.80 0.49

Goal Setting Participation 3.24 0.65 2.67**

Contribution Participation 3.20 0.59 2.76**

Satisfaction with Appraisal Participation 3.60 0.87 2.92**

Motivation to







Improve Participation 3.52 0.63 3.77** No Participation 2.75 0.94

*p c .05 **p < .01

terview to be significantly more useful, that their supervisor was more supportive, that they were given more of an opportunity to participate, that goals and objectives were set to a greater extent and that they made more of an impact or contribution to the appraisal interview. In addi- tion, there were significant differences on both of the appraisal interview outcome measures. Those employees who participated in the development of the BARS were more satisfied with the interviews and were more motivated to improve their job performance. On the other hand, no significant differences were found between the two groups in their perceptions of how anxious or tense the interviews made them, how much the manager detracted fmm the interview by being unprepared and arbitrary, and how critical the manager was of their performance.

The stepwise discriminant analysis was then conducted using the ten appraisal interview variables shown in Table 1. As a result of the step-


wise discriminant analysis, two variables were entered into the resulting discriminant function. Motivation to Improve (Wilks lambda = .74, p c .001) was the first variable entered and the other variable entered was Opportunity to Participate (Wilks lambda = .71, p c .OOl). The resulting discriminant function had an eigenvalue of .40, Wilks lambda = .7 1, x2 (2) = 16.95, p < .001. The standardized discriminant function coefficients for Motivation to Improve and Opportunity to Par- ticipate were .85 and .39, respectively.

The fact that motivation to improve performance on the part of the ratee proved to be the most discriminating variable is quite encourag- ing. Involving workers in the development of the rating instrument led workers to respond that they were more motivated to improve their performance. Although no measures were available to determine whether or not there were actual improvements in subsequent job performance, Burke and Wilcox (1969) found that actual improvement in job performance correlated .57 with an individuals motivation to improve performance. One possible reason that those employees who participated in the development of the BARS felt they were given more of an opportunity to participate in the interview by their managers might be because they had more of a sense of ownership (Greller, 1978) in the appraisal process. It seems logical that those employees who actually participated in the development of the BARS would perceive more psychological participation (Wexley et al., 1973) in the appraisal interview.

Specifically, our results suggest that involving workers in the development of BARS can lead to more positive reactions by employees to the performance appraisal interview process as well as subsequent outcome measures. These results are supportive of Landy and Trumbo (1980) who pointed out that one of the advantages of BARS may have nothing to do with the measurement of performance per se, but stem from the high degree of involvement of the workers and supervisors in scale development. This study supports their hypothesis with regards to workers. Those workers who were involved in the BARS development clearly showed more positive reactions to performance appraisal than those who were not involved.

This study also suggests that perhaps similar positive outcomes could be achieved by involving employees in the development of other types of performance appraisal instruments, such as Behavioral Observation Scales (Latham and Wexley, 1977) and Mixed Standard Scales (Blanz and Ghiselli, 1972). Friedman and Cornelius (1976) showed that rater participation in scale construction was able to enhance the psychometric characteristics of two instruments, BARS and graphc rating scales. Future research should examine the generalizability of these results to various appraisal instrument.


In the present study, those employees that participated in the development of the BARS participated in three steps. Employee involvement to this full extent may not be necessary and/or practical to achieve similar results. Future research should be directed toward determining the extent of employee involvement needed to achieve similar positive results. Knowing these results, an issue that needs to be dealt with in the future studies is whether the experimental manipulation influenced only the subordinates perceptions of what transpired during the appraisal interviews or whether it also influenced the managers actual behaviors. Un- fortunately, this issue could not be considered in the present study because of the infeasibility of measuring managerial behavior during the appraisal sessions. This would have necessitated the audiotaping or videotaping of the sessions which is something the hospital administration would not permit. Further understanding could also be obtained by having a 3-group study with a placebo participation group in which the par- ticipants would be involved in a consultative nature on an appraisal- irrelevant issue. An additional experimental condition might have been a nonparticipative training session on the developed BARS.

The study raises the following practical questions that should be in- vestigated in future research: (1) Can the expense of participation in scale development be justified from a cost/benefit viewpoint? (2) Are these gains greater than could be achieved by other indoctrination and training procedures regarding existing appraisal programs? (3) How much variation in ratee participation can be permitted across organizational units without causing feelings of unfairness? (4) Do the effects of ratee participation wear off over time or persist over repeated appraisals?

REFERENCES

Blanz, F . , and Ghiselli, E. E. (1972). The mixed standard scale: A new rating system. PERSONNEL PSYCHOLOGY, 25, 185-199.

Burke, R . J . , Weitzel, W. , and Weir, T . (1978). Characteristics of effective employee performance review and development interviews: Replication and extension. PER-

Burke, R . J . , and Wilcox, D. S . (1969). Characteristics of effective employee performance reviews and developmental interviews. PERSONNEL PSYCHOLOGY, 22,

Friedman, B. A , , and Cornelius, E. T. (1976). Effects of rater participation in scale construction on the psychometric characteristics of two rating scale formats. Journal of Applied Psychology, 61, 210-216.

Greller, M. M. (1975). Subordinate participation and reactions to the appraisal interview. Journal of Applied Psychology, 60, 544- 549.

Greller, M. M. (1978). The nature of subordinate participation in the appraisal interview. Academy of Management Journal, 22, 646- 658.

Ilgen, D. R., Fisher, C. D., andTaylor, M . S. (1979). Consequencesof individual feedback on behavior in organizations. Journal of Applied PsychoZogy, 64 349-371.

SONNEL PSYCHOLOGY, 31, 903-9 19.

29 1-305.

7 10 PERSONNEL PSYCHOLOGY

Jacobs, R., Kafry, D., and Zedeck, S. (1980). Expectations of behaviorally anchored rating scales. PERSONNEL PSYCHOLOGY, 33, 595-640.

Kay, E., Meyer, H. H. , and French, J . R . P. , Jr. (1965). Effects of threat in a performance appraisal interview. Journal of Applied Psychology, 49, 31 1-317.

Landy. F. J . , and Trumbo, D. A . (1980). Tne Psychologyof Work Behavior (rev. ed). Homewood, IL: Dorsey Press.

Latham, G. P., and Wexley, K. N. (1977). Behavioral observation scales for performance appraisal purposes. PERSONNEL PSYCHOLOGY, 30, 255- 268.

Nemeroff, W. F., and Wexley, K. N. (1979). An exploration of the relationship between performance feedback interview characteristics and interview outcomes as perceived by managers and subordinates. Journal of Occupational Psychology, 52, 25-34.

Smith, P. C. , and Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied

Wexley, K. N. (1979). Performance appraisal and feedback. In S. Kerr (Ed.), Organiza- fional Behavior. (pp. 241-259). Columbus, Ohio: Grid Publishing.

Werley, K. N., Singh, J . P., and Yukl, G. A. (1973). Subordinate personality as a moderator of the effects of participation in three types of appraisal interviews. Jour- nal of Applied Psychology, 58, 54-59.

Psychology, 47, 149-155.

silverman 1984

Documents