- 07 Nov 2014
- Working Paper Summaries
Do Experts or Collective Intelligence Write with More Bias? Evidence from Encyclopædia Britannica and Wikipedia
Executive Summary — Britannica and Wikipedia are sources that aspire to provide comprehensive information. They both face similar challenges over the length, tone, and factual basis of controversial, unverifiable, and subjective content. Such challenges are pervasive in the production of encyclopedic knowledge about current events and political debates surrounding topics like taxation, health care policies, biographical details of presidential candidates, and the funding of stem-cell research, for example. In this paper the authors begin with a simple observation: Each source resolves these challenges differently in distinct production processes. Britannica, for example, produces its final content after consultation between editors and experts. Wikipedia, on the other hand, relies on the "crowd" for its content, receiving contributions from tens of millions of individual users. Examining 3,918 pairs of articles about US politics that appeared in both outlets, the authors compare bias and slant from the two production models. Results suggest that the allocation of editorial time and user contributions is central to the minimization of differences in bias and slant between the two outlets. Among the managerial implications, community managers can work towards a balanced view if intervention alleviates disputes and generates the right kind of participation. Key concepts include:
- The costs of producing, storing, and distributing knowledge shape different biases and slants in the collective intelligence (Wikipedia) and the expert-based model (Britannica).
- Many of the differences between Wikipedia and Britannica arise because Wikipedia faces insignificant storage, production, and distribution costs. This leads to longer articles with greater coverage of more points of view. The number of revisions of Wikipedia articles results in more neutral point of view. In the best cases, it reduces slant and bias to a negligible difference with an expert-based model.
- As the world moves from reliance on expert-based production of knowledge to collectively-produced intelligence, it is unwise to blindly trust the properties of knowledge produced by the crowd. Their slants and biases are not widely appreciated, nor are the properties of the production model as yet fully understood.
Which source of information contains greater bias and slant-text written by an expert or that constructed via collective intelligence? Do the costs of acquiring, storing, displaying, and revising information shape those differences? We evaluate these questions empirically by examining slanted and biased phrases in content on U.S. political issues from two sources-Encyclopedia Britannica and Wikipedia. Our overall slant measure is less (more) than zero when an article leans towards Democrat (Republican) viewpoints, while bias is the absolute value of the slant. Using a matched sample of pairs of articles from Britannica and Wikipedia, we show that, overall, Wikipedia articles are more slanted towards Democrat than Britannica articles, as well as more biased. Slanted Wikipedia articles tend to become less biased than Britannica articles on the same topic as they become substantially revised, and the bias on a per word basis hardly differs between the sources. These results have implications for the segregation of readers in online sources and the allocation of editorial resources in online sources using collective intelligence.