Scientific background 360 degree feedback
The characteristics, usability and reliability of the 360-degree feedback method
The 360 degree feedback Method and its instruments are an important part of the TMA Method, categorized under the influential factor ‘Competencies and capabilities’. It is directed at generating feedback from a number of people with different perspectives on the candidate’s behavior as well as from the candidate himself. Its purpose is to present a broad view on the candidate’s current work behavior.
The 360 degree feedback Method is predominantly used as a basis for staff assessment systems, coaching interviews or performance interviews. People from the candidates’ work environment provide input for the interviews by assessing their performance.
The use of multiple evaluators
In the 360 degree feedback Method several evaluators are asked to assess the candidate’s behavior. There are numerous assets in using multiple evaluators (multi rating) over using a single evaluator (single rating). Firstly, the possibilities to observe the candidate in different situations increase with the number of evaluators. When assessing staff, a manager will not always be able to observe employees in all aspects of their position. Just think about staff working outside the office, spending most of their work time with customers. Different evaluator groups perform different roles regarding the candidate. By combining the perceptions of people with different perspectives a less biased, more complete view on the candidate is obtained.
A risk might be that by increasing the number of observations one’s chance to include false observations increases as well. Keeping the measuring process in mind this soon becomes evident: in every step the evaluators’ assessments may differ. Evaluators have different opinions on the question what makes a successful candidate – and will therefore look at different behavior. This difference in perception also affects the interpretation of observed behavior. What one evaluator strikes as desirable, another might find objectionable. It is, therefore, important to standardize the assessment process as much as possible by evaluating clearly defined and easily observable behavior.
The remarks above seem to oppose the idea that multi rating results in more reliable assessments than single rating. This can be explained by the fact that an assessment based on two evaluators who make the same observation is more reliable than one based on a single person’s observations. Reliable multi rating decreases the chance of false assessments and reduces the influence of bias. One evaluator’s observations are being checked by others’ – so to speak. Statistics tell us that reliability increases with the number of observers. Apart from this, the criteria on which a candidate is assessed play a role: the smaller their number and the clearer they are defined, the more reliable the assessment. The more evaluators agree on their observations, the more easily they are classified and interpreted.
2.2 The use of self assessment
Apart from being assessed by multiple others, it is typical for the 360 degree feedback Method to have candidates assess themselves. The perception candidates have of their own performance is weighed as well as that of people from their social (work) environment. The 360 degree feedback Method makes it possible to compare candidates’ perception of their skills, abilities and style with the perception of their peers.
3. Reliability and validity of the 360 degree feedback Method
The 360 degree feedback Method can be subdivided into assessments by experts, peers and the candidates themselves. The reliability and validity of those three different kinds of assessments have been studied extensively. Results of those studies will be summarized briefly below for every evaluator group.
If the 360 degree feedback Method is used for an assessment at work, the candidate’s superior is usually one of the evaluators. In selection procedures the expert role is usually performed by a psychologist or consultant.
Assessments by colleagues, or peer assessments, are a useful alternative for assessments by superiors or experts. Both field and controlled studies show that peer assessment is a reliable method. Validity of peer assessments depends on a number of moderating factors. Firstly, it depends on the purpose for which the peer assessment is used. If the assessments are directed at development of the candidate with regards to evaluative objectives, not only the interrater reliability increases; leniency also decreases, and the assessments are more differentiated between both evaluators and dimensions. Secondly, validity depends on the kind of peer assessment. Three forms can be distinguished:
1. Peer nomination: a number of peers is asked to 1. nominate within their group who performs best in certain behavioral aspects. The person who is nominated most often performs best according to the group. This method discriminates effectively between group members who perform exceptionally well in certain aspects and those who do not. It renders no useful information on those group members who are not mentioned at all.
2. Peer ranking: a group of peers is asked to rank all group members from best to least performing on certain behavioral aspects. This kind of peer assessment is the most discriminating one. It renders no absolute information on the individual group members nor does it provide any information on the intervals between the ranks.
Although peer ranking and nomination are more reliable and valid, peer rating is most widely used. In peer rating, group members are assessed independently of each other and of the group. It provides an absolute score as opposed to the relative scores peer ranking and nomination provide. The 360 degree feedback Method used within the TMA Method is based on peer rating.
3. Peer rating: every group member assesses all other group members on a number of behavioral aspects using a specific rating scale. This method is useful for feedback purposes: it provides information on every individual group member, independent of other members. Both its reliability and validity are lower than with the other two methods. The use of scales makes the method susceptible to evaluators’ errors and differences between evaluators are more likely to occur.
4. Self assessment
Personality questionnaires ask candidates to describe themselves on the basis of a number of directed questions. Self assessments are generally regarded as less valid than, for instance, peer or expert assessments. Their reliability is low.
Self assessments are susceptible to various sources of bias, such as a limited score range and leniency. A large number of studies show that self assessments tend to be significantly higher than assessments by peers or experts. On the one hand, those higher scores can be explained by a tendency to create an image of oneself that is as positive as possible – what psychologists call: ego preservation. On the other hand, those higher scores can be explained by people’s tendency to adapt their self-image to other people’s wishes – what psychologists call: impression management. Both mechanisms result in a more positive self-perception than the perception of other people reflects. Studies indicate that the difference in self assessment and the assessment of others decreases as the candidate performs more successfully. In other words: successful people perceive themselves more in accordance with the people around them than less successful people do. An explanation could be that successful people tend to have a more realistic self-perception than less successful people do. Maybe the explanation needs to be sought elsewhere. Successful people tend to be rated higher than less successful people. If the range in ratings by others is larger than the range in self-ratings, the difference between self-perceptions and those of others will decrease towards the scores’ extremes.
Various studies report on the relationship between peer, self and expert assessments. They show that assessments by peers and superiors often correlate, whereas little correlation was evident between these two and self assessment.
A difference in average between ratings has no direct consequences for their correlation, as long as the difference is a constant one. Correlations between self assessments and assessments by peers or experts tend to be low: people have a significantly different view of themselves than others do. This is one of the reasons why organizations are reluctant to use self assessments in making staff decisions. As a result of their limited reliability and accurateness the validity of self assessments is suspected to be low. This suspicion is confirmed by a number of studies indicating that self assessments and relevant objective tests hardly correlate. Self assessments can provide added value in an assessment procedure as the instrument can increase the acceptance of both feedback and the final assessment. The predictive value of self assessments depends on a few conditions. It has to rate, for instance, the same competencies during the same period of time as were rated by the other assessment groups. Apart from that, the ratings need to be related to specific situations and performances, and not refer to general characteristics. The predictive value of a self assessment increases if the candidate is familiar with the concept.
5. Conclusion 360 degree feedback Method
In conclusion, we dare say the 360 degree feedback Method is a pretty useful and reliable instrument for assessing employees. Especially with regards to personal development it provides added value. Although self assessments are intrinsically connected to the 360 degree feedback Method, there are certain arguments against them such as ego pre assessment by others, a great asset of self assessments is the possibility to review the difference between self-perception and perception of others – which can be input for an interview between superior and employee. A condition in order to perform 360 degree feedback properly is that key behavioral indicators on which the candidate can be assessed are clearly defined.