For more than a century, the long, stately rows of Encyclopædia Britannica have been a fixture on the shelves of many an educated person's home—the smooshed-together diphthong in the first word a symbol of old-world erudition and gravitas. So it was a shock to many when, in 2012, the venerable institution announced it would no longer publish a print version of its multivolume compendium of knowledge.
Though the Britannica would still be available online, the writing on the virtual wall was clear: It had been supplanted by the Internet. And more specifically, by an upstart phenomenon Wikipedia, the free, crowd-sourced encyclopedia that since its inception in 2001 had rapidly become the new go-to source for knowledge.
“Most of the topics of content we are dealing with on a daily basis do not have a verifiable answer”
"It's sad to see the trajectory of Encyclopædia Britannica," says Feng Zhu, an assistant professor in the Technology and Operations Management unit at Harvard Business School, who details the rise and fall of the information giant in a new working paper. "There has been lots of research on the accuracy of Wikipedia, and the results are mixed—some studies show it is just as good as the experts, others show [that] Wikipedia is not accurate at all."
Complicating matters, however, many of the topics that we look up in the Britannica—any encyclopedia—aren't factually cut-and-dried. "Most of the topics of content we are dealing with on a daily basis do not have a verifiable answer," says Zhu. "They can be quite subjective or even controversial."
History, they say, is written by the victors, and can read very differently depending on who is telling the tale. Even modern-day issues such as immigration, gun control, abortion, and foreign policy are open to fervent debate depending on who is doing the opining. Over the years, Britannica has handled this uncertainty by seeking out the most distinguished experts in their fields in an attempt to provide a sober analysis on topics; while Wikipedia has urged its civilian editors to maintain what it calls a neutral point of view (NPOV).
Who Is More Objective?
But is objectivity better achieved by considering one viewpoint or thousands? Along with cowriter Shane Greenstein of Northwestern's Kellogg School of Management, Zhu asks that question in a new paper, Do Experts or Collective Intelligence Write with More Bias? Evidence from Encyclopædia Britannica and Wikipedia.
online encyclopedias.©iStock.com/adventtr
Zhu and Greenstein have long been interested in the question of crowd bias, which itself has been hotly debated by scholars in many fields including psychology and politics over the centuries. Are two heads better than one, or do too many cooks spoil the broth? Does the collective will of the majority lead to democratic consensus or fundamentalist groupthink?
The massive, ongoing natural experiment of Wikipedia offers a unique view into these questions. "The Internet makes it so easy for people to aggregate; some scholars worry that people will self-select into groups with a similar ideology," says Zhu. As a result, the Internet may lead to more biased opinions, which only harden over time as users separate into rival virtual camps.
To test this theory, Zhu and Greenstein took a database of terms developed by University of Chicago economists Matthew Gentzkow and Jesse Shapiro to examine newspaper bias. Gentzkow and Shapiro studied speeches in the 2005 Congressional Record to scientifically identify the top 500 unique phrases used by Democrats (e.g., tax breaks, minimum wage, fuel efficiency) and Republicans (e.g., death tax, border security, war on terror), rating each according to political slant.
Zhu and Greenstein then identified some 4,000 articles that appeared in both Encyclopædia Britannica and Wikipedia, and determined how many of each of these code words were included, in an effort to determine overall bias and direction.
They found that in general, Wikipedia articles were more biased—with 73 percent of them containing code words, compared to just 34 percent in Britannica.
“We can only say [that] Wikipedia is more left. We can't say which is reflecting true reality”
In almost all cases, Wikipedia was more left-leaning than Britannica. Dividing articles into categories, the researchers found, for example, that stories on corporations were 11 percent more slanted toward Democrats, while observing similar leanings on topics such as government (9 percent), education (4 percent), immigration (4 percent), and civil rights (3 percent). Other categories did not have enough data to significantly identify bias.
Of course, those findings don't say which of the two sources is correct in its viewpoint—only how they compare to one another. "We can only say [that] Wikipedia is more left," says Zhu. "We can't say which is reflecting true reality."
What's more, much of Wikipedia's bias seems to be due to the longer article length of the online publication, where word count is less of an issue than the historically printed Britannica. When compared word to word, most (though not all) of Wikipedia's left-leaning proclivities come out in the wash. In other words, for articles of the same length, Wikipedia is as middle-of-the-road as Britannica.
"If you read 100 words of a Wikipedia article, and 100 words of a Britannica [article], you will find no significant difference in bias," says Zhu. "Longer articles are much more likely to include these code words."
Rinsing Out Bias
Perhaps the most interesting finding of Zhu and Greenstein's research is that the more times an article is revised on Wikipedia, the less bias it is likely to show—directly contradicting the theory that ideological groups might self-select over time into increasingly biased camps.
"The data suggests that people are engaging in conversation with each other online, even though they have different points of view," says Zhu. "The crowd does exhibit some wisdom, so to speak, to self-correct bias."
The number of revisions required to start showing this effect, however, is quite large—at least 2,000 edits—and the articles most read by users aren't necessarily those most revised by editors. "To some extent, we are not seeing the scenario where too many cooks spoil the broth, we are mostly seeing an insufficient number of cooks," says Zhu.
If Wikipedia would like to improve its objectivity, Zhu recommends that it encourage editors to revise the most-read stories first, as well as encouraging people with different political leanings to edit the same article.
"Wikipedia can easily do this," he says. "It has all the information about how many times people are reading and editing articles. They could easily direct the attention of editors in order to have the most impact."
Room For Both?
As for Britannica, though its experts may be somewhat vindicated by Zhu and Greenstein's findings overall, the editors are still not found to be more objective than the crowd in articles that are sufficiently revised. If the company would like to stay relevant, Zhu suggests, then perhaps it should focus on niche articles on topics not likely to be adequately covered by Wikipedia editors.
"When it comes to their capabilities, Britannica may be able to do a much better job of marketing itself as the expert on topics that Wikipedia can't cover well, such as obscure diseases where there may not be enough experts who have time to write a Wikipedia article."
Readers, meanwhile, should be conscious of the inherent bias found in Wikipedia, and seek out other sources to corroborate information on articles that lack a large number of revisions over time.
On today's virtual bookshelf, in other words, there may be a place for Wikipedia and Encyclopædia Britannica to sit side by side.