Simple binary questions – ”yes or no”, ”for or against”, ”positive or negative” – are the most effective way to get an accurate reading of public opinion. That is the conclusion of research by Dr Kohei Kawamura, published in the June 2011 issue of the Economic Journal.
His findings are relevant to a wide range of situations – from referendums on complex political issues to how seriously we should take customer reviews on Amazon. Why are binary opinion polls more trustworthy and often more informative than more detailed surveys? Dr Kawamura”s study explains that it is because people who are asked binary questions cannot possibly exaggerate their answers. In a world where everyone is competing to get their message across, there is a strong incentive to express extreme opinions. This means that responses to survey questions more elaborate than ”yes or no” are subject to exaggeration and, as a result, are less credible.
In the case of customer reviews on Amazon, these findings suggest that readers might do well to ignore extreme opinions, one or five stars, particularly if there are a lot of reviews of a product. The overall balance between the total numbers of positive and negative reviews is likely to reflect product quality reasonably well.
How should we read numerous customer reviews on the internet? Why do opinion polls and referendums ask simple ”yes or no” questions even if the issues in question are much more complex?
These are closely related questions about communication that involves many providers of information, a phenomenon that has become increasingly common as the internet has made large-scale information transmission cheaper and quicker.
This study shows that when many people try to get their messages across at the same time, they tend to exaggerate their views to compete for influence and attention. Consequently, as the number of information providers becomes larger, extreme messages prevail and such messages tend to be less and less credible, while moderate messages remain relatively credible.
By construction, binary questions and answers suppress any temptation to exaggerate, and thus emerge as a simple and efficient way to elicit public opinion. Think about the Amazon website, which has star rating from one to five for every product it sells. This research suggests that we should perhaps ”discount” one-star and five-star
reviews because the reviewers are, by nature, inclined to go extreme to influence other potential customers.
When there are many reviewers, each reviewer has only a small influence on potential customers and their temptation to write extreme reviews becomes large. This means we should discount extreme reviews more heavily when there are a larger number of reviews. Having studied the problem, Dr Kawamura pays much more attention to moderate reviews.
His research also suggests that the total numbers of positive (four and five star) and negative (one and two star) reviews can reflect product quality fairly well (in fact, Amazon shows such figures for the ratings of their Marketplace sellers). While the intensity of reviewers” opinions may be misrepresented in the five-star rating system, we can at least trust whether each reviewer is positive or negative, because it does not make sense for
them to say something contrary to their actual views.
This leads to the next question addressed in the study, namely why we observe binary questions so frequently, especially when many people are surveyed. For example, referendums and opinion polls on abortion rights, alcohol policy and immigration standard are often conducted though binary questions, even though those issues are clearly much more complex than binary and there is a wide spectrum of opinions far beyond mere ”yes or
no” to a particular question.
Shouldn”t more detailed answers be invited to find out public opinion? But note that elaborate response is subject to exaggeration and thus cannot be taken at a face value. This study points out that simple binary questions have the important advantage that the responders cannot possibly exaggerate their answer.
Since they have no chance to exaggerate, the outcome – how many people are for or against a statement – is completely trustworthy. The research demonstrates that simple binary opinion polls can indeed be just as informative as more detailed surveys, when many people are asked.
”A Model of Public Consultation: Why is Binary Communication so Common by Kohei Kawamura is published in the June 2011 issue of the Economic Journal.
Kohei Kawamura
University of Edinburgh | kohei.kawamura@ed.ac.uk
