One of the most counter-intuitive but observable social phenomena is that a large enough group of people can make startlingly accurate predictions despite an individual lack of expertise, providing their guesses are made independently and then averaged together. This concept, which has come to be known as the “wisdom of the crowd,” can be more formally described as the gathering of information and making a prediction from a group’s aggregated inputs, even if the individual inputs are noisy. It was first was suggested by Galton (1907), who described a 1906 country fair in Plymouth in which the median guess of (about 800) people, participating in a contest to estimate the weight of an ox, was 1207 pounds, accurate within 1% of the true weight of 1198 pounds. In the era of digital innovation, this concept has been widely used by social information sites, like Wikipedia and Stack Exchange. See also a New York Times Business Bestseller, The Wisdom of Crowds: Why the Many are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations by James Surowiecki (2004).
Although in many cases the idiosyncratic noise associated with each individual judgment gets cancelled over the average, leading to a good answer from either polls or prediction markets, in general there is no guarantee that the crowd wisdom yields a better judgment than an individual does. This is especially true if the majority of the people do not know the right answer (e.g. most of people might not realize that Philadelphia is not the capital of Pennsylvania) or the answer to a question is currently unknown to human beings (e.g. whether extraterrestrial life exists).
Surprisingly, a very creative solution is given in Prelec et al. (2017). They point out that in a survey if, in addition to answering one survey question, a participant is also asked to make a prediction of the distribution of answers to the survey question of the whole participants, then a consistent estimator related to a quantity in the objective world can be obtained, under a suitable set of regularity conditions on the subjective and objective worlds, if the majority of the people do not know the right answer. For example, if a survey question asks participants whether Philadelphia is the capital of Pennsylvania, then the simple majority vote obtained from the answers to the question may be wrong, as most of the participants may give the wrong answer. However, if, in addition to the original question, another question is asked about the respondent’s guess of the percentage of the respondents who correctly answer the first question, then a consistent estimator can be found by combining the answers to both questions, and by giving more weights to the answers by the minority respondents, who answer the first question correctly but think that the majority respondents will get the first question wrong.
Dai et al. (2021) study a related problem of the wisdom of the crowd, namely making predictions or judgments using prediction markets. In particular, they attempt to answer the following question: Can one design a prediction market, such that a consistent estimator can be found using data in this market? The answer is yes, if additional individual level data is available. Indeed, they give two estimators, a market-adjusted estimator based solely on market inputs including individual trading positions and a hybrid estimator based on both market and inputs from one survey question (instead of two questions), such that both estimators are consistent under the BTS setting, but under different regularity conditions. Furthermore, using a real data set of sports betting, which include trading positions of three groups (people betting on the home team, people betting on the away team and the bookmakers), Dai et al. (2021) find empirically that the market-adjusted estimator can significantly improve the accuracy of prediction, if the number of participants is large so that the market has enough liquidity.
References
Dai, M., Jia, Y., & Kou, S. (2021a). The wisdom of the crowd and prediction markets. Journal of Econometrics, 222, 561–578.
Galton, F. (1907). Vox populi. Nature, 75, 450–451.
Prelec, D., Seung, H. S., & McCoy, J. (2017). A solution to the single-question crowd wisdom problem. Nature, 541, 532–535.
Surowiecki, J. (2004). The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. Anchor Books, New York.
