Performance appraisal methods: 360-degree feedback as a case study

Organizational development experts agree that performance management systems that provide timely information concerning employee performance are an essential requirement for almost any type or size of enterprise today. A number of performance evaluation methods are available for this purpose, with some being better suited to certain working environments than others. One of the more popular approaches to emerge in recent years has been multisource feedback methods such as the 360-degree feedback approach that collects survey information from a wide range of sources including supervisors, subordinates, customers, vendors and suppliers. While the 360-degree feedback approach has produced good results for some companies, others have faltered in its application for a number of reasons. Because organizations typically invest large amounts of resources in such performance evaluation systems, it is important that the information that is produced provides the type of data that is needed for informed management decisions while avoiding the potential pitfalls that accrue to their use. To identify these pitfalls and how they can be avoided in the 360-degree feedback approach, this study provides a review of the relevant juried and scholarly literature followed by a summary of the research, recommendations, ideas for future research and the limitations of the study in the concluding chapter.

Performance Appraisal Methods -- The 360 Degree Feedback Approach as a Case Study

Oh, what a great gift it is to see ourselves as others see us. -- Robert Browning

Chapter 1:

Introduction

Many organizational theorists agree that monitoring employee and supervisor performance is an essential ingredient in any performance management system, and a wide range of approaches have been developed over the years in response to this need. Besides evaluating the contribution that individual performance makes to achieving organizational goals, many performance appraisals are used by many companies to evaluate individual performance for promotion and compensation purposes as well. Therefore, it is important that the techniques that are used for measuring performance are effective and provide the timely metrics that are needed to fairly and objectively analyze individual contributions to accomplishing organizational goals. Carefully managed and applied, performance management techniques can "deliver the goods," but the entire process is fraught with opportunities for mismanagement and the potential for serious adverse outcomes is great. In some cases, the individual supervisor is in the best position to make such a performance evaluation, but a growing number of companies of all types and sizes have recognized the need for more robust analyses of performance than a single report can provide. According to Becton and Schraeder (2004), "Traditional performance appraisal systems have primarily consisted of supervisor evaluations of subordinate performance. In recent years, however, organizations have increasingly turned their attention toward gathering performance feedback from sources other than immediate supervisors" (p. 23). Not surprisingly, these more robust reports of performance provided by multisource evaluations have resulted in a number of positive outcomes, including: (a) better performance information; (b) more reliable ratings than those from a single supervisor; and (c) improved ratee performance following receipt of the feedback (Becton & Schraeder 2004, p. 23). As the term implies, multisource feedback methods, including the 360-degree feedback approach, collect information concerning individual job performance from a number of sources; depending on the organizational setting, these sources can include self- appraisal, ratings by the supervisor, subordinates, customers, vendors, as well as suppliers (Becton & Schraeder 2004). Because some traditional performance evaluation methods are marred by subjectivity when a single rating source -- typically a supervisor -- is used, proponents of the 360-degree feedback approach cite the additional feedback provided as the primary strength of the 360-degree feedback method. For instance, Becton and Schraeder emphasize that, "This feedback from multiple sources and perspectives is perhaps the most compelling strength of 360-degree feedback" (2004, p. 24).

Just as the term multisource refers to the fact that more than one source of evaluation is used, the term 360-degree is more accurately used for evaluation situations that draw on ratings from "all around" the ratee. For example, according to London (2003), "Feedback can come from any number sources. It can be visible in the tasks we do as we see the pace and quality of our work. It can come from reports, such as monthly sales figures. It also can come from other people: supervisors, subordinates, peers, and customers" (67). Therefore, all 360-degree feedback techniques are multisource but all multisource techniques are not necessarily 360-degree feedback. In this regard, London makes this minor but potentially important distinction clear: "Multisource feedback refers to ratings that can come from subordinates, peers, supervisors, internal customers, external customers, or others. When feedback comes from all the locations around a person (boss, subordinates, suppliers, customers) it is also called "360-degree" feedback" (London 2003, 67). The research shows that the 360-degree feedback approach has been implemented by a growing number of companies with mixed results, and it is these mixed results that form the background to the research undertaken herein and which is discussed further below.

Background to Research

Because 360-degree feedback approaches provide more robust and comprehensive results than single-rater approaches, some of the constraints associated with traditional performance appraisal systems are minimized or avoided entirely. In this regard, Zaccaro and Klimoski (2001) note that, "Traditional performance appraisals, particularly when tied to salary increases, are often bloated. Even when appraisals do not overrate performance, they are weak tools for comparing candidates if standards are not precisely defined and raters not specifically trained" (p. 37). When individuals receive traditional evaluations from a single rater that consistently exaggerate achievements and performance over the years, the organization is cheated out of an opportunity to provide the employee with importance feedback that could help to better align performance with organizational goals; the employee is likewise cheated out of the type of timely feedback that may be required to effect these types of changes in people. As Zaccaro and Klimoski point out, "The multiperspective, multirater, or 360-degree feedback approach gathers information from colleagues, supervisors, direct reports, and sometimes customers and suppliers. Because ratings by others are usually compared to self-ratings, individuals can gain considerable insight and motivation to develop and change" (p. 37). Just as traditional, single-rater performance evaluation methods have some drawbacks and constraints, so too does the 360-degree feedback approach, especially if the company uses this information for purposes other than performance improvement. For instance, Zaccaro and Klimoski report that, "Multirater methods are not particularly useful for selection. They can easily devolve into popularity contests, deliver damaging and blunt messages, and be corrupted by people who solicit biased ratings to take unfair advantage of the system. Multirater methods also show low levels of interrater agreement and are subject to considerable halo because raters are usually poorly trained, if at all" (Zaccaro & Klimoski 2001, p. 37).

In reality, any multisource rating scheme will be constrained by these factors, but the 360-degree feedback approach includes ways to overcome these that provide more robust results -- but at a price, which suggests the process is more appropriate for high-value executives with a diverse range of responsibilities than for line workers who may occupy identical positions within the organization. According to Zaccaro and Klimoski, "A live 360-degree assessment can introduce more rigor. This way, an executive coach or other trained, objective evaluator collects performance information by interviewing a broad range of observers" (p. 37). The "trained objective evaluation" serves as the focal point through which all of the evaluations are channeled. As an example, Zaccaro and Klimoski cite General Electric which "uses an accomplishment analysis as a concentrated way to gather performance data for its management development system. A consultant interviews the manager for several hours, meets with the manager's boss for several more hours, and interviews former associates and subordinates. The consultant summarizes the individual's achievements and development plans and makes them part of the manager's file" (2001, p. 39).Irrespective of who is interviewed, what instruments are used, and what performance metrics are assigned, when more than one rater is used, there will be a corresponding increase in the amount of resources needed to support the evaluation method, a consideration that is especially salient for the 360-degree feedback method, making an assessment of its effectiveness for this purpose equally important.

While much of the attention being directed at the 360-degree feedback approach is relatively recent, the approach itself dates back to the mid-20th century although informal or variations on the theme were in use prior to that time (Weiss & Kolberg 2003). In this regard, Weiss and Kolberg report that, "From the 1920s through the 1950s, a variety of psychological tests and surveys were developed for the purpose of selection and placement for college entrance exams for military recruitment and business hiring practices. In virtually all of these cases, the purpose of the assessment was to provide the sponsoring organization with data about the person being tested. Rarely was the client being tested shown the results of the testing" (2003, p. 73). In sharp contrast to this approach, the tenets of the 360-degree feedback approach wherein the results of assessments were provided to the individual to identify strengths and weaknesses were the result of innovations in the public sector's Peace Corps. According to Weiss and Kolberg,

"In the 1960s, a breakthrough in sharing the assessment results came from the Peace Corps when the psychologists who were working with the volunteers used surveys that were geared to expand the volunteer's self-knowledge, under the assumption that expanding self-knowledge would help a volunteer better deal with culture change. This was the first time that this type of assessment was done for the primary benefit of the person being assessed" (2003, p. 73). Therefore, the original intent of the 360-degree feedback method was to provide the individual being evaluated with the type of feedback they needed to formulate an informed opinion concerning what others thought of them and how this information could be used to good effect in the workplace rather than for selection or salary increase purposes. The original focus of the 360-degree feedback approach continues to characterize most of its applications today as well. For instance, Weiss and Kolberg conclude that, "Today, 360-degree feedback has become increasingly popular for managers at all levels of organizations and has become a core tool of professional development" (2003, p. 73).

Indeed, the use of 360-degree feedback is now regarded as an essential element in any leadership development initiative, and its use within organizations is increasing significantly to the extent that it has been described as being one of the most important human resource interventions of the last decade (Storey 2004). This point is also made by Wimer (2004) who advises, "During the past decade, 360-degree feedback has become one of the most popular human resource interventions. The power behind this process is that it's a sometimes rare opportunity for employees to receive honest feedback about how they're perceived by their peers" (37). Although the 360-degree feedback approach is frequently used with executives, it is not restricted to management and the insights gained through such individual evaluations can help virtually anyone become more effective in the workplace. For instance, Wimer notes that in recent years, "Most organizations use it for multisource feedback as part of their management development or performance appraisal processes, or on an ad hoc basis with individuals. The idea is that if employees are armed with better self-awareness, they can make important changes in their work behavior" (Wimer 2004, p. 37).

While the 360-degree feedback approach can be effective in this expanded role, some authorities insist that its use with executives is one of the method's most important functions. According to Green (2002), "Multi-rater or 360-degree feedback is essential for overcoming leader blind spots and enhancing their overall emotional intelligence" (p. 8). In addition, Green reports that the 360-degree feedback approach can also be used to assess the performance of leadership teams. An important point to note is that the weaknesses that are identified through the 360-degree feedback approach are termed "improvement areas" while an individual's strengths are behavioral based and that all survey information from the multiple sources used is collected anonymously (Green 2002).

In some ways, the 360-degree feedback approach relies on the same techniques that are used in other performance appraisal methods such as results-oriented methods (which are discussed further in chapter two below) by making comparisons between what is desired and what is observed. In this regard, Green reports that, "Typically, multi-feedback raters identify leader strengths and developmental gaps by comparing survey statements of what is expected or desired in the leader role, versus their actual perceived behavior in their role. Collectively, the summary feedback is used to identify which behaviors a leader needs to enhance job performance, and which current behaviors should continue as is" (p. 8). It is reasonable to suggest, though, that even the most thoughtful and sensitive feedback will fail to evoke the type of behavioral changes desired unless the entire process is carefully managed and follow up procedures are in place. For instance, Green emphasizes that, "Successful 360-degree feedback is not automatic. The process must be properly positioned, implemented, and followed-up over time" (2002, p. 9). Notwithstanding the increasing attention being paid to the use of multisource evaluation methods such as the 360-degree feedback approach, there remains a paucity of research concerning how effective the technique is in aligning performance with achieving organizational goals (Green 2002), making this study particularly relevant and timely.

Overview of Research

The study used a five-chapter format to achieve the below-stated aims and objectives and to answer the study's guiding research questions. Chapter one of the study introduced the topics under consideration, the background of the research, this overview, the study's aims and objectives, the scope of the study, definitions of key terms used and the study's guiding research questions.

Aims and Objectives

The aims and objectives of the study were to generally assess the effectiveness of the 360 degree appraisal feedback system and to compare and contrast its effectiveness with conventional appraisal systems.

Scope of the Study

The focus of this research will be on the effectiveness of the 360 degree feedback system as it is used in Company XXX.

Terms of Reference

Term

Definition

Performance Appraisal

For the purposes of this study, this term generally refers to the periodic evaluation of employee performance.

Performance Management

This term refers to "a strategic and integrated approach of increasing the effectiveness of organizations by improving the performance of the people who work in them and by developing the capabilities of teams and individual contributors" (Armstrong & Baron 1998, 38).

360 degree feedback

This term refers to a performance appraisal method that collects performance ratings from peers, subordinates, supervisors, and sometimes customers (London 2003).

Key Research Questions

To achieve the above-stated aims and objectives, the study was guided by several research questions as follows:

1. What types of performance appraisal methods are commonly used in organizations today?

2. Should 360 degree feedback be used for performance appraisal or salary increases?

3. How confidential is the process?

4. Is 360-degree feedback given anonymously or face-to-face?

5. Is there a consensus among performance management experts and managers concerning the efficacy of 360-degree feedback?

Chapter 2:

Discussion of Relevant Literature

Chapter Introduction

This chapter provides a review of the relevant literature concerning the importance of performance appraisals generally, and a discussion of multisource or 360 degree feedback as an appraisal technique. An analysis of the need for confidentiality and the selection of appropriate raters for the 360 degree feedback approach are followed by a brief summary of this chapter. A description of the study's methodology immediately follows in chapter three.

Importance of Performance Appraisals

Employees want and need to know how they are doing on the job so they can build on their strengths and address identified weaknesses, but the performance appraisal process is fraught with opportunities for miscommunication, disappointment, disillusionment and poor morale if it is not handled carefully. As a result, the performance appraisal process is one of the most important methods any type of organization, large or small, can use to better align employee efforts with overarching organizational goals. For instance, Sims (2002) emphasizes that, "Strategically, it is hard to imagine a more important organizational system than performance appraisal" (p. 80). As noted above, truly effective performance appraisal systems provide employees with the timely feedback they need to determine when their performance is aligned with the corporate vision and goals; however, even more importantly, effective performance appraisal systems also provide these firms with the information they require in order to gauge their progress towards achieving organizational goals. For instance, Sims adds that, "Within this context, the evaluation of performance is a control mechanism that provides not only feedback to individuals but also an organizational assessment of how things are progressing" (p. 80).

Because performance information is so important to an organization's survival, it is not surprising that a number of techniques have been developed over the years for this purpose. As a result, there are a number of performance appraisal methods besides the 360- degree feedback that are available, including the following:

1. Trait methods: This approach to performance evaluation is based on the use of graphic rating scales which are considered by many organizations as being an inexpensive performance appraisal method because the same rating scale can be used for virtually all positions within an organization.

2. Behavioral Methods. This approach to evaluating employee performance also uses rating scales; however, the method relies on a consensus among supervisors and organizational experts within the company concerning what behaviors can be regarded as being superior, good, average, and poor performance for a particular position. Generally, raters assign a number on a graphic scale to indicate the employee's level of performance.

3. Results-Oriented Methods. This is a straightforward performance appraisal method that uses quantifiable benchmarks and target objectives and then compares how well employees achieved these goals over a designated period of time. This approach is also known as management-by-objective, and this approach is widely regarded as being a highly effective and motivational approach to performance evaluation since employees are able to determine what performance levels are expected of them and how well they are progressing toward achieving their target objectives during a given rating period.

4. Comparative Methods. This performance appraisal method operates by evaluating employees in comparison to other employees in similar positions; in these situations. This performance appraisal method includes both ranking systems (e.g., raters evaluate their employees in terms of best to worst) and forced distributions (e.g., employees are assigned a grade or number that identifies the top performing employees compared to the worst performing employees. This type of appraisal system might be used by companies in the event they are considering downsizing and need to identify their best employees (Kulik, 2004).

A comparison of the advantages and disadvantages of the foregoing performance appraisal methods is provided in Table 1 below.

Table 1

Advantages and Disadvantages of Different Employee Performance Appraisal Methods

Method

Advantages

Disadvantages

Trait Methods:

Graphic rating scales; essay method

Inexpensive: the same rating form can be used for all jobs in the entire organization.

Prone to rater error; highly subjective.

Behavioral Methods:

Behaviorally Anchored Rating Scales (BARS); Behavioral observation scales

Reduces rater error; provides valuable behavioral feedback to employees.

Expensive; unique rating forms must be developed for every job in the entire organization.

Results-Oriented Methods (MBO)

Tailored to individual employee's needs and abilities; excellent for providing developmental feedback.

Not easily used for many administrative decisions, because employees are not evaluated on a common scale.

Comparative Methods:

Ranking

Forced Distribution

Good for administrative decisions.

Eliminates some rater errors, but may be perceived as subjective; not very effective in providing developmental feedback.

Source: Kulik 2004, 103

The foregoing breakdown in Table 1 shows that there are a number of performance appraisal approaches that can be used in a company's performance management system, and the type of approach that is used will directly relate to the needs of the organization. Some companies may prefer to use one approach with some classes of employees (e.g., forced distribution might be appropriate for a company that employs a number of workers in the same type of job) while others might find the straightforward techniques used in results-oriented methods more suitable for their purposes. As noted in the introductory chapter, another performance appraisal method that has gained increased popularity in recent years is the multisource 360-degree feedback, and this approach is discussed further below.

Applications of 360-Degree Feedback

From a strictly pragmatic perspective, because the resources that are available for performance management is limited, there remains the decision concerning how this information should be used. As Delassio (1998) points out, a practical issue is to what application organization should put 360-degree feedback, whether for the purposes of "general self-discovery," or to facilitate the development of targeted behavior changes and other administrative human resource decisions such as promotions and wage increases (or decreases). According to this authority, "The short-term view of providing the most immediate payoff for the organization is that 360-degree feedback should be used for targeted behavior change and administrative decisions. The longer term -- and perhaps more enlightened-view is that it should be used for more general self-discovery. This latter use will probably have a longer term benefit to the corporation, resulting in employees who can function better together because they have a deeper self understanding" (Delassio 1998, p. 1022).

More recently, a study by Atkins and Wood (2003) notes that multisource feedback programs such as 360-degree feedback are rapidly increasing in popularity for both development and evaluation application purposes. Virtually all of the sources reviewed, though, emphasized the need for more research in several areas where gaps were identified, including the effect of rater selection and the effect of 360-degree feedback over the long-term. Finally, a point made by Delassio (1998) concerning the need to use any performance management method in ways that are consistent with their organizational culture should be noted. According to Delassio, "The use of 360-degree feedback for development vs. personnel decisions is not an either/or proposition. Rather I see use of 360-degree feedback for pure development vs. pure personnel decisions as the anchors on a continuum with many possible implementation variations along the continuum" (p. 1023). Before jumping on the "360-degree bandwagon," then, corporate leaders should ensure that it is congruent with their corporate culture and that they understand why they are implementing it in the first place. In this regard, Delassio concludes that, "Organizations need to think about what type of implementation strategy makes the most sense given their culture and their purpose for implementing 360-degree feedback" (p. 1023).

Chapter Summary

This chapter provided a general review of the relevant literature concerning the importance of performance appraisals as well as an assessment of the viability of multisource or 360 degree feedback as an appraisal technique. In addition, the need for confidentiality and the selection of appropriate raters for the 360 degree feedback approach was discussed. A description of the study's methodology is presented in chapter three below.

Chapter 3:

Methodology: Presentation of Methods and Rationale

A review of the available research methodologies indicated that a mixed methodology would be best suited for achieving the study's research purpose and for answering the guiding research questions. The first part of the study's methodology consisted of a review of the relevant literature concerning performance appraisal approaches in general and 360-degree feedback in particular. This part of the methodology is highly congruent with a number of social researchers who cite the need to review secondary sources concerning what is known about a given topic as part of almost any type of research project today. For instance, Fraenkel and Wallen (2001) report that, "Researchers usually dig into the literature to find out what has already been written about the topic they are interested in investigating. Both the opinions of experts in the field and other research studies are of interest. Such reading is referred to as a review of the literature" (48). Likewise, Wood and Ellis (2003) report that the following are important outcomes of a well conducted literature review:

1. It helps describe a topic of interest and refine either research questions or directions in which to look;

2. It presents a clear description and evaluation of the theories and concepts that have informed research into the topic of interest;

3. It clarifies the relationship to previous research and highlights where new research may contribute by identifying research possibilities which have been overlooked so far in the literature;

4. It provides insights into the topic of interest that are both methodological and substantive;

5. It demonstrates powers of critical analysis by, for instance, exposing taken for granted assumptions underpinning previous research and identifying the possibilities of replacing them with alternative assumptions;

6. It justifies any new research through a coherent critique of what has gone before and demonstrates why new research is both timely and important.

The second part of the mixed methodology consisted of developing primary data using a convenience sample-based survey of respondents using a series of yes/no and multiple choice questions as well as a series of Likert-scaled questions concerning the perceived and real performance appraisal and feedback of the selected persons over a period of 2-5 years or before and after the introduction of the 360 degree appraisal feedback system. The Likert-scaled questions were ranged on a continuum from "strongly agree" to "strongly disagree" with no option for no opinion in an effort to compel the respondents to provide an answer to each question one way or the other. Potential participants, all of whom were employed at Company XXX, were recruited via email invitations with a URL hyperlink to the online survey hosted by SurveyMonkey.com (see proforma copy of survey instrument at Appendix A). Of the 55 email invitations sent, 35 respondents (or about 64%) completed the online survey in time for the responses to be included in the data analysis.

Based on the results of this online survey and the review of the literature, a semi-structured interview format was developed and administered to three managers with experience in using the 360-degree feedback approach to gain additional insights concerning their views of the effectiveness of the 360 degree feedback approach compared with other conventional appraisal systems (see semi-structured interview sheet at Appendix B). Face validity of the custom survey instrument was achieved by following the guidance provided by Neuman (2003) wherein a copy of the instrument was reviewed by classmates, friends, co-workers, family members and others to ensure that it appeared to collect the type of data that was desired and that it did not contain any ambiguous questions.

The use of both secondary and primary data in this type of study is also highly congruent with Dennis and Harris (2002) who note, "Secondary data are information that has been collected earlier for a different purpose, but which may still be useful to the research project under consideration. . . . Primary data are information that is being collected for the first time in order to address a specific research problem. This means that it is likely to be directly relevant to the research, unlike secondary data, which may be out of date or collected for a totally different purpose. Ideally, an effective research project should incorporate both primary and secondary data" (39).

The statistical data that resulted from the administration of the online survey was analyzed using SPSS Version 11.0 for Windows (Student Version). The data analysis included the computation of ratios and other analytical methods. Ratio analysis involves the expression of a variable with reference to another and they act as reliable indicators of the wellness and performance of the entity being analyzed. The results of the data analysis are presented in tabular and graphic form and analyzed in a narrative fashion in the discussion section of chapter four below. The results of the semi-structured interviews are presented in tabular form according to the interview question in chapter four, edited only for clarity and to avoid repetition and otherwise reported verbatim.

Chapter 4:

Data Analysis and Discussion

Chapter Introduction

This chapter is divided into three sections; section one of the chapter presents the results of the online survey of respondents concerning their views about the effectiveness of the 360 degree feedback approach compared to other conventional performance appraisal methods and section two presents the results of the semi-structured interviews of managers who have experience with the approach for several years. The third section is used for a discussion and synthesis of the survey and semi-structured interview data.

Section One: Results of the Online Survey

Part 1: Yes/No and Multiple Choice Questions

1.

Please indicate how long you have used the 360-degree feedback approach.

Key:

1

Less than 6 months.

2

More than 6 months but less than a year.

3

More than a year but less than 2 years.

4

More than 2 years but less than 5 years.

5

More than 5 years.

Please indicate how long you have used the 360-degree feedback approach.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

5

14.3

14.3

14.3

2.00

9

25.7

25.7

40.0

3.00

8

22.9

22.9

62.9

4.00

9

25.7

25.7

88.6

5.00

4

11.4

11.4

Total

35

2.

On average, how frequently does your department administer a 360-degree feedback survey?

Key:

1

Monthly

2

Semi-monthly

3

Quarterly

4

Every 6 months

5

Annually

6

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

2

5.7

5.7

5.7

2.00

5

14.3

14.3

20.0

3.00

6

17.1

17.1

37.1

4.00

12

34.3

34.3

71.4

5.00

1

2.9

2.9

74.3

6.00

9

25.7

25.7

Total

35

3.

If applicable, what is the main reason your department continues to use the 360-degree feedback method?

Key:

1

It is mandated by the company's leadership.

2

The quality of the feedback that is provided.

3

It is an important part of our performance appraisal system.

4

It provides superior results compared to other performance evaluation methods.

5

Not applicable.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

20

57.1

57.1

57.1

2.00

4

11.4

11.4

68.6

3.00

3

8.6

8.6

77.1

4.00

6

17.1

17.1

94.3

5.00

2

5.7

5.7

Total

35

4.

If inapplicable, please indicate the primary reason your department has stopped using the 360-degree feedback method:

Key:

1

Its discontinuance was mandated by the company's leadership.

2

Too expensive.

3

Too difficult.

4

Unsatisfactory results.

5

Replaced by another performance evaluation method.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

.00

33

94.3

94.3

94.3

1.00

2

5.7

5.7

Total

35

5.

Do you allow your ratees to select any or all of their raters?

Coding key:

1

Yes

2

No

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

16

45.7

45.7

45.7

2.00

19

54.3

54.3

Total

35

Part 2: Likert-Scaled Questions

Key:

1

Strongly Agree

2

Agree

3

Disagree

4

Strongly Disagree

1.

The 360-degree feedback approach provides personnel with accurate and timely information concerning their work performance.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

17

48.6

48.6

48.6

2.00

11

31.4

31.4

80.0

3.00

3

8.6

8.6

88.6

4.00

4

11.4

11.4

Total

35

2.

There has been a measurable improvement in ratee performance following a 360-degree feedback review.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

20

57.1

57.1

57.1

2.00

6

17.1

17.1

74.3

3.00

6

17.1

17.1

91.4

4.00

3

8.6

8.6

Total

35

3.

There is a direct link between changes in personnel performance and the 360-degree feedback approach.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

12

34.3

34.3

34.3

2.00

11

31.4

31.4

65.7

3.00

8

22.9

22.9

88.6

4.00

4

11.4

11.4

Total

35

4

Compared to other performance evaluation approaches that I have used, the 360-degree feedback approach is superior.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

10

28.6

28.6

28.6

2.00

8

22.9

22.9

51.4

3.00

10

28.6

28.6

80.0

4.00

7

20.0

20.0

Total

35

5.

The 360-degree feedback approach should be used alone.

Frequency

Percent

Valid Percent

Cumulative

Percent

Valid

1.00

6

17.1

17.1

17.1

2.00

5

14.3

14.3

31.4

3.00

9

25.7

25.7

57.1

4.00

15

42.9

42.9

Total

35

6.

The 360-degree feedback approach should be used as part of a more comprehensive performance evaluation process.

Frequency

Percent

Valid Percent

Cumulative

Percent

Valid

1.00

15

42.9

42.9

42.9

2.00

16

45.7

45.7

88.6

3.00

3

8.6

8.6

97.1

4.00

1

2.9

2.9

Total

35

7.

I think the 360-degree feedback approach is just another management fad that will be replaced by something else when top management feels a change of some sort is needed.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

6

17.1

17.1

17.1

2.00

7

20.0

20.0

37.1

3.00

10

28.6

28.6

65.7

4.00

12

34.3

34.3

Total

35

8.

The 360-degree feedback approach is good for morale.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

9

25.7

25.7

25.7

2.00

11

31.4

31.4

57.1

3.00

8

22.9

22.9

80.0

4.00

7

20.0

20.0

Total

35

9.

Anything that helps people better understand how others view them is a valuable tool for personal growth.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

12

34.3

34.3

34.3

2.00

16

45.7

45.7

80.0

3.00

3

8.6

8.6

88.6

4.00

4

11.4

11.4

Total

35

10.

Sensitivity and confidentiality are essential elements in a successful 360-degree feedback program.

Frequency

Percent

Valid Percent

Cumulative Percent

Valid

1.00

26

74.3

74.3

74.3

2.00

9

25.7

25.7

Total

35

Section Two: Results of the Semi-Structured Interviews

Question

Responses

QUESTION #1: Since you began using the 360-degree feedback approach, has your department's productivity, cost-savings, efficiency, communication, internal or external customer satisfaction or other quantifiable metric improved due to the behavior feedback that is provided?

Exec. No. 1: Our customer service has improved across the board, I know that. Other aspects of our performance have been affected by the current economic downturn and it's hard to tell where the impact from one begins and the effect of the other begins. But we have always encouraged personnel development here and many of our staff are actively pursuing higher educational programs or using our company's internal resources to further their careers. All of these things have helped our staff become more professional and competent, but motivation remains the key to success.

Exec. No. 2: We have cut our operational costs by more than 10%, but a lot of this savings was achieved by downsizing rather than efficiency. In fact, we have lost about 5% of our staff recently. Despite these losses in personnel, I would have to say that our overall efficiency seems to have improved after we started using 360 degree feedback, but I can't say the same thing about morale.

Exec. No. 3: I'd have to double check for the exact figures, but I recall that our productivity and efficiency have both improved in recent months. I'm not sure if all of this progress is the result of 360-degree, but it has been accomplished during a period when it has been in place.

QUESTION #2: Have you seen ratees behave differently based on the 360-degree feedback they have received?

Exec. No. 1: Without a doubt. To borrow the terminology, we have seen a 180 degree about-face in the performance of some of our employees after they have received 360-degree feedback. I think a lot of this success is due to the fact that we use a trained interviewer to gather and deliver the survey results. And yes, we make double-sure that everything that is discussed in these sessions is confidential.

Exec. No. 2: The most important thing that has happened to our company recently has been the number of people we have been forced to lay off. It's a sad commentary, but most of our personnel have started paying attention to what is expected of them and making efforts to achieve their goals after their peers were laid off, so it's hard to determine what impact that 360-degree feedback has had on the overall process.

Exec. No. 3: The changes I've seen have been mostly by the people I know who already want to get ahead in this business. For these people, 360-feedback is like the Gospel and I've seen some significant improvements in customer service as a result. On the other hand, though, I've seen others who have continued their day-to-day behaviors like nothing at all has been attempted.

QUESTION #3: Have you noticed your executive leader team behaving differently in anyway since receiving 360-degree feedback? If so, please state how?

Exec. No. 1: I would have to say that it's been mostly business as usual around here for as long as I can remember. Most of the time it seems like we are just putting out fires rather than making any real progress.

Exec. No. 2: Our executive team has experienced some turnover recently that has affected our performance management system. The most recent addition to the leadership team advocates a management-by-objective approach that is counter to our drive to implement 360-degree feedback, but I think the program may remain in place for the time being.

Exec. No. 3: Based on what I've seen, our top management team sometimes operates more along the lines of "do what I say" rather than "do what I do." I don't mean they are doing a bad job or anything like that, it's just that many of our employees see how much they are making and don't understand why they are being downsized from time to time.

QUESTION #4: Are the results of the 360-degree feedback developed in your organization used for performance appraisal or salary increases or just for self-improvement applications?

Exec. No. 1: We try to take advantage of every opportunity to help our people grow and advance in our company. When the results of a 360-degree survey show that people are performing above and beyond, we take that into account during performance reviews for promotion and salary increases.

Exec. No. 2: I know that some other divisions are using 360-degree feedback for HR purposes, but I'm reluctant to do that. After all, the whole purpose of the program is to help people change for the better rather than change specific behaviors specifically for the good of the company.

Exec. No. 3: We try to help everyone who receives 360-degree feedback use this information to its maximum advantage. We have sent line personnel and executives to training opportunities we believed would address the deficiencies identified through the feedback process, but not everyone is willing to expend the effort they need to get everything out of this training that is required. In an effort to take advantage of all training that is provided from outside sources, we have recently implemented a policy that require attendees to share what they learned with their peers and to provide a report to management along these lines.

Section Three: Discussion

Unfortunately, there were few clear-cut answers to the study's guiding research questions but rather an elaboration on themes emerged from the synthesis of the literature review, questionnaire and semi-structured interview results. Although most people want and need to know what others think of them so that they can make appropriate changes in behavior to help them achieve their personal and professional goals, not everyone is receptive to negative feedback that makes them feel victimized or a "loser" simply because they are doing the best job they can, at least from their own perspective. Certainly, this is a natural reaction when people believe they are working hard, or at least as hard as their peers. It is therefore reasonable to suggest that because most people believe that they are already doing the best job that they can, any feedback to the contrary will be met with mixed reactions. Many people will believe no matter how many reports they receive otherwise that they are making a superhuman effort to help their organization survive and grow, while others will likely concede that they could be doing much more. Therefore, the overarching theme that emerged from the study thus far was the need to deliver 360-degree feedback in a sensitive and confidential fashion, preferably by a trained interviewer who was also responsible for collecting the feedback. In this way, a balanced and objective analysis can be developed that will help rather than discourage the ratee. The results of the survey provide some useful insights concerning these needs as well as current trends and views from practitioners in the field, and these results are provided below.

Five (or 14.3%) of the respondents reported using 360-degree feedback less than 6 months, nine (or 25.7%) reported more than 6 months but less than a year, eight (or 22.9%) reported using 360-degree feedback more than a year but less than 2 years, nine (or 25.7%) reported more than 2 years but less than 5 years and four respondents (or 11.4%) reported more than 5 years. Two respondents (or 5.7%) reported using the 360-degree feedback survey every month, five (or 14.3%) reported semi-monthly, six (or 17.1%) reported quarterly, 12 (or 34.3%) reported every 6 months, one (or 2.9%) reported annually and nine (or 25.7%) reported "as needed."

Twenty respondents (or 57.1%) reported that the main reason their department uses the 360-degree feedback approach is that it is mandated, four (or 11.4%) reported that it is because of the quality of the feedback, three (or 8.6%) reported that it is part of the performance appraisal system, six (or 17.1%) cited superior results compared to other performance evaluation methods and two respondents (or 5.7%) indicated this question was no longer applicable; both of these latter respondents reported its discontinuance by management as the reason. Slightly more respondents reported not allowing their employees to select any or all of their raters compared to those who did (16 or 45.7% vs. 19 or 54.3%, respectively).

The results of the Likert-scaled questions also revealed some common themes and trends. For instance, most of the respondents either strongly agree or agreed (17 or 48.6% and 31.4%, respectively) that the 360-degree feedback approach provides personnel with accurate and timely information concerning their work performance, while three (or 8.6%) and four (or 11.4%) disagreed or strongly disagreed, respectively. Almost three-quarters (74.2%) of the respondents strongly agreed (20 or 57.1%) or agreed (6 or 17.1%) that there has been a measurable improvement in ratee performance following a 360-degree feedback review, while six respondents (or 17.1%) disagreed and three (or 8.6%) strongly disagreed. Furthermore, almost two-thirds of the respondents (23 or 65.7%) strongly agreed or agreed (12 or 34.3% and 11 or 31.4%, respectively) that there was a direct link between changes in personnel performance and the 360-degree feedback approach.

Most of the respondents also either strongly agreed (10 or 28.6%) or agreed (8 or 22.9%) that the 360-degree feedback approach was superior to other performance evaluations approaches they have used, but 10 respondents (or 28.6%) disagreed and four respondents (or 20%) strongly disagreed. Just 11 of the respondents strongly agreed or agreed (6 or 17.1% and 5 or 14.3%, respectively) that the 360-degree approach should be used alone, while nine respondents (or 25.7%) disagreed and 15 respondents (or 42.9%) strongly disagreed. By contrast, fully 88.6% (or 31) of the respondents either strongly agreed or agreed (15 or 42.9% and 16 or 45.7%) that the 360-degree feedback approach should be used as part of a more comprehensive performance evaluation process, while just three respondents (or 8.6%) and one respondent (or 2.9%) disagreed or strongly disagreed, respectively.

Interestingly, most of the respondents believed that the 360-degree feedback approach was not just another passing management fad, with just six respondents (or 17.1%) and seven respondents (or 20%) reporting that they thought the 360-degree feedback approach was just another management fad, while 22 respondents (or 62.9%) disagreed (10 or 28.6%) or strongly disagreed (12 or 34.3%). There were some mixed views concerning whether the 360-degree feedback approach is good for morale, though, with nine respondents (or 25.7%) and 11 respondents (or 31.7%) either strongly agreeing or agreeing, respectively, while eight respondents (or 22.9%) disagreed and seven (or 20%) strongly disagreed. Most of the respondents (28 or 80%) strongly agreed or agreed (12 or 34.3% and 16 or 45.7%, respectively) that anything that helps people better understand how others view them was a valuable tool for personal growth, but three respondents (or 8.6%) disagreed and four respondents (or 11.4%) strongly disagreed. Finally, there was consensus concerning the need for sensitivity and confidentiality as essential elements in a successful 360-degree feedback program, with all of the respondents either strongly agreeing (26 or 74.3%) or agreeing (9 or 25.7%).

Chapter 5:

Conclusions, Recommendations, Ideas for Future Research and Limitations of the Study

Conclusions

The 360-degree feedback approach was shown to be a viable performance appraisal technique that is particularly well suited for executive-level performance analysis, but which can also be used for other employees as well as leadership teams. The 360-degree feedback approach, though, was also shown to be more expensive than most of its performance appraisal alternatives because of the larger number of people that are involved, and the method is more appropriate for self-improvement purposes than it is for selection and rewards. Perhaps the overriding strength of the 360-degree feedback approach is its ability to provide individuals in whatever position with valuable insights concerning how other people view them in ways that might not otherwise be possible. An old country adage suggests that, "If enough people call you a horse, you'd better buy a saddle," and the same principle applies to the summary information that is provided to a ratee in the 360-degree feedback approach. Conversely, because this feedback is provided anonymously, some people might resent their inability to respond to criticisms by giving "their side of the story," a feature that is common in most other performance appraisal methods. This constraint will even apply if the ratee is able to "tell his side of the story" to the interviewer since this individual may not be in a position to be responsive to these explanations. Moreover, the right to confront one's accusers is a fundamental constitutional right in the United States and this aspect of the 360-degree feedback approach may simply rub many people the wrong way.

As noted in the introductory chapter, this study was guided by several research questions which are reiterated and answered in sum below.

1. What types of performance appraisal methods are commonly used in organizations today? Besides multisource or 360-degree feedback performance appraisal methods, the research showed that other methods included trait methods (graphic rating scales; essay method); behavioral methods (behaviorally anchored rating scales [BARS]; behavioral observation scales); results-oriented methods (MBO); comparative methods; and, ranking (forced distribution) methods.

2. Should 360 degree feedback be used for performance appraisal or salary increases? There were some mixed views identified in the research concerning this issue, with some authorities suggesting that the 360-degree feedback method is better suited for helping people identify where their strengths and weaknesses are and in determining how they can improve rather than being used for performance appraisal or wage determination purposes. Other authorities, though, emphasized that the costs associated with any performance management method must be offset by taking advantage of all of the information that the method provides and this consideration could easily be applied to the use of the 360-degree feedback approach as well.

3. How confidential is the process? The research was consistent and emphatic concerning the need for the results of the 360-degree feedback evaluation to be delivered in a confidential setting; however, whether the process itself is confidential is dependent on the setting that is selected for this purpose by an organization's management team. Savvy business leaders will ensure that the process respects an individual's privacy rights and is sensitive to instances in which negative feedback must be communicated.

4. Is 360 degree feedback given anonymously or face-to-face? The research showed that the approach that is most commonly used with 360-degree feedback is to solicit survey information from multiple sources which is provided to the ratee anonymously; the synthesis of these multiple sources of feedback, though, is typically provided to the ratee in a face-to-face meeting with a supervisor or preferably, another individual who has been specifically trained for this purpose.

5. Is there a consensus among performance management experts and managers concerning the efficacy of 360 degree feedback? The consensus that emerged from performance management experts and the real-world managers surveyed for this study suggested that the 360-degree feedback method is as effective as the organization makes it.

Recommendations

Managing a 360-degree feedback process can be an effective approach to performance evaluation and providing employees at all levels with valuable information they can use for professional development, but the research was consistent in showing that the method is challenging, and there are real barriers to success that must be taken into account. Therefore, the following recommendations are provided:

1. Provide ratees with the opportunity to participate in the rater selection process, allowing as many choices as the company deems appropriate.

2. Ensure that follow-up procedures are in place to monitor the effectiveness of any professional development initiatives that are undertaken pursuant to the performance summary data provided by the 360-degree feedback approach.

3. Ensure that the summary feedback provided by the 360-degree approach is delivered in a confidential setting by an individual who has been trained in the use of the method.

4. Develop a set of best practices based on success stories using the 360-degree feedback method and do more of these.

5. Establish quantifiable metrics that can be used to identify spikes in reporting data that might be indicative of "popularity contests" or skewed feedback based on the quality of the relationships between ratees and raters.

Ideas for Future Research

One of the gaps identified in the review of the relevant literature concerned the lack of research concerning the potential implications of rater selection and the effect this step has on the effectiveness of the feedback process. Because the information that a company needs to make informed decisions concerning promotion and salary increases may already exists somewhere within the constellation of feedback provided by the 360-degree method, identifying which source of feedback is most suitable for different purposes represents a potential idea for future research. For example, feedback from vendors and customers might be a particularly useful source for determining how effective an individual employee is in customer relations, a feature that may be highly relevant to the employee's job function such as in a call center or customer service department. Other areas that deserve the attention of researchers include a comparison of how effective the 360-degree method is when it is used for self-improvement and career development purposes vs. its use for other applications such as promotion and salary increases. It just makes good sense that co-workers recognize that there is a finite amount of money available for salary increases and promotion opportunities are typically rare; therefore, by evaluating a co-worker poorly, their chances of getting a raise or promotion are increased.

Limitations of the Study

One of the most significant limitations of the study related to the relatively small number of respondents for the survey and semi-structured interview components. Clearly, a larger subject population would provide more robust findings, but the results that emerged from the existing study were deemed to provide some valuable insights into how effective the 360-degree feedback method is regarded by practitioners today, at least in a general fashion.