The equestrian sport of dressage is the only Olympic competition in which men and women compete as equals with the outcomes determined by subjective performance evaluations. According to research by Anna Sandberg, published in the August 2018 issue of the Economic Journal, judges of these events show no bias based on gender, but they do favour individuals of their own nationality.
Her study provides unique field evidence that the phenomenon of ‘in-group bias’ can occur even among experienced and carefully monitored professionals. Such bias may have important consequences in many other areas, causing discrimination in hiring and promotion decisions, and affecting judges’ trial verdicts and teachers’ evaluations of students.
During a dressage competition, each athlete performs a series of movements on their horse, and five different judges score the performance. Using data from 1,533 international dressage competitions, the new study compares the scores from judges of different genders and nationalities who observed the same performance by an athlete.
On average, judges are not biased in favour of, or against, riders of their own gender. But judges do favour athletes of their own nationality. For a given performance, an athlete receives on average 0.6 more points from judges of their own nationality than from judges of other nationalities. Further analysis indicates that this national bias affects the final ranking of athletes in roughly a quarter of the investigated competitions.
In addition to favouring athletes of their own nationality, judges favour athletes of the same nationality as the other judges on the judging panel. This additional bias implies that going from zero to one in-group member on a committee can have a large impact on outcomes, as it may affect the verdicts of all committee members.
The sizeable effect of judge nationality is particularly striking given that these judges are trained, experienced and constantly monitored professionals. It seems likely that a bias needs to be strong and resilient to penetrate this expert group of evaluators.
Nevertheless, the size of the bias varies significantly between different types of competitions. In particular, the bias is largest in championships and team events – that is, in competitions in which athletes represent their nation. This variation across types of competitions suggests that the national bias is also malleable and responds to situational factors.
Despite the importance of in-group bias, opportunities to study this issue using naturally occurring data have been rare. The sport of dressage provides a rare opportunity to study the occurrence of in-group bias among professionals making high-stakes decisions.
This study is potentially relevant to policy-makers, managers and organisational designers who are interested in reducing in-group favouritism in evaluation situations.
Many labour market evaluations are similar to dressage competitions in the sense that they take place in a competitive and international mixed-gender setting and involve high stakes. Examples of such evaluations include international recruitment or promotion committees, courts, universities and legislative bodies.
The robust evidence of substantial national bias among dressage judges – an expert group of evaluators facing unusually high levels of accountability – indicates that similar processes are likely to operate in most other evaluation settings in the labour market.
‘Competing Identities: A Field Study of In-group Bias among Professional Evaluators’ by Anna Sandberg is published in the August 2018 issue of the Economic Journal.
