| ||||
Word of mouth undoubtedly impacts sales, but it's difficult to quantify. To better understand the word of mouth phenomenon, David Godes and Dina Mayzlin turned their magnifying glass on thousands of online conversations about TV shows on Usenet discussion forums. In this e-mail interview with HBS Working Knowledge senior editor Martha Lagace, they describe their discoveries.
Lagace: What intrigued you most about word of mouth communication, making you decide to study it more deeply?
| These conversations are essentially public and therefore easy to access at a low cost. |
| — Dina Mayzlin |
Godes: On one hand, we noticed that more and more people were talking about word of mouth. Particularly with the rise of online communities, there seemed to be a lot of interest. There were a slew of new popular press books on the topic, Wall Street Journal articles, etc. Yet, we had a very strong sense that the general phenomenon of word of mouth was something that was not at all well understood. In particular, in comparing it to other, more "traditional" media—broadcasting, print, etc.—there was no consensus about how one should measure it. In fact, we could find very few attempts to do so.
Mayzlin: We felt that if companies were really interested in tracking, understanding, and doing something about word of mouth, then they had to be able to measure it first.
Q: Why did you decide to study word of mouth by looking at online newsgroups about TV shows? What were some advantages and disadvantages to that specific focus?
Mayzlin: It's important to point out that our theory about how to measure word of mouth is separate and distinct from the way we test it. That is, we believe that our measurement approach for word of mouth is equally applicable to offline as it is to online communications. Now, why did we choose online newsgroups? Mainly because these conversations are essentially public and therefore easy to access at a low cost. This was important for us, but more important for companies implementing our measures. We wanted to show that they would work with data that companies could get quite easily themselves.
Godes: We chose TV shows as a context for a couple of reasons. First, it's a category that people tend to talk about so we felt pretty confident that we'd have enough data to work with. Also, this too offered us free data. The weekly Nielsen ratings are published in Broadcasting & Cable magazine so this was easy to get. Of course, the challenge is to show that the method works in other categories as well and we're working on this.
Q: How do you both think about volume and about dispersion across online communities?
Mayzlin: Think about it this way: Imagine you were at Starbucks and you tried some new vanilla apple latte or something, a drink they just began offering. And, it turned out that you liked it and you told your roommate about it. Imagine also that Starbucks magically knew about this conversation. The question is whether they should take this as an indication that the product's sales will grow. One problem with this is that your roommate might already know about it and in fact may have already tried it. This is particularly likely if you and your roommate have similar habits. For example, you might take the same route to work and walk by the same Starbucks. Just measuring the "volume" of conversations about a product—how many times people talk about the new latte—might not capture the whole story.
| ||||
Godes: This is where the dispersion measure comes in. We really want to get a sense for how many "new" people hear about the product. So, we created a measure that essentially tells us whether all of these conversations are happening between the same people—and therefore possibly reflecting past purchases more than predicting future ones—or whether they're resulting in new people hearing about the product for the first time.
Q: Did anything surprise you as you began to delve into your word of mouth research?
Mayzlin: We were surprised to find that the volume of conversations did not really impact the future ratings of TV shows, whereas early on the dispersion measure did significantly impact future ratings.
Q: What might our business readers find most useful about this research?
Godes: Word of mouth can and should be measured just as all of the other key metrics of a company's success are typically measured. Just because it is a difficult phenomenon to get one's hands around doesn't mean that it should be thought of as purely "qualitative." We think that our method for measuring it—in particular, our approach to the consideration and measurement of dispersion—will be very helpful to companies in this regard.
Mayzlin: Managers should also, we hope, find it interesting that these online communities might offer such a rich source of data about interpersonal communications.
Q: What are you working on next?
Godes: We're working hard to improve the approach that we've taken in this paper in a few ways. First, we're considering other measures beyond just volume and dispersion. For example, does the length of the conversation offer us any information? Also, we're hoping to apply the method to more real-world contexts and are thus actively seeking companies interested in this type of research. We're hoping to extend it into different purchase contexts and product/service categories. We're confident that the general approach is quite flexible but are interested in understanding the nuances that these different contexts present.
Using Online Conversations to Study Word of Mouth Communication
by David Godes and Dina Mayzlin
Among the many and varied channels through which a person today may receive information, it is hard to imagine any that carry the credibility, and as a result the importance, of interpersonal communication, or word of mouth. There is little debate as to whether word of mouth matters to the firm. In fact, there is good reason to believe that it has more potential impact than any other communication channel. Katz and Lazarsfeld (1955) showed nearly half a century ago that word of mouth was the single most important source of information for certain household items. More recently, Kotler (2000) cites a study of 7,000 consumers in seven European countries in which 60 percent said they were influenced to buy a new brand by family and friends. Similarly, a 1999 study by Jupiter Communications found that 57% of people visiting a new Web site did so based on a personal recommendation, far higher than any other source of influence (Jupiter Communications, 1999). As these studies suggest, managers are interested in word of mouth for one simple reason: It can have a tremendous impact on a product's sales. The specific mechanism behind this impact can vary. It might affect awareness, in some cases, or preferences in others. Word of mouth might also affect sales by making a particular product or category more salient in the consumer's mind. Whatever the specific mechanism, there is significant empirical evidence, as well as a strong intuitive sense, for the hypothesized link between word of mouth and product sales.
A direct implication of this putative causal relationship is that the firm should manage the word of mouth surrounding their product. To the extent that certain types of word of mouth are associated with higher sales, of course, the firm might invest in the processes necessary to generate that type of word of mouth. To the extent that more word of mouth is better for sales, the firm should do what it can to facilitate the flow of interpersonal communications. Paraphrasing Deming, it's hard to manage something if you don't measure it. However, there are at least three significant challenges associated with measuring word of mouth. First and foremost, how does one even gather the data? Since the information is exchanged in private conversations, direct observation is—or at least has traditionally been—quite difficult. As a result, most marketers and researchers have either relied on consumer recall or have inferred the process of information exchange from aggregate data. One fascinating and important implication of the rise of online communities is that this development makes feasible the observation by marketers of consumer-to-consumer conversations. In this paper we investigate the potential that these conversations present us with an opportunity to measure word of mouth.
Second, even if we could observe the conversations, what aspect of them would we, or should we, measure? Sales are easy to capture quantitatively. How does one measure a conversation, a set of statements between people? Should we count the words? The number of people involved in it? It is a significant challenge to determine which of the possible transformations of a conversation are "meaningful." Currently, the most common approach is to use simple counts. This approach is much like the news-clipping services that monitor how many times a firm's products are "mentioned." For example, Yahoo! Buzz Index keeps track of how many times users query a particular topic on the Yahoo! search engine. We will investigate the "informativeness"—that is, the explanatory power—of this naive measure as well as another measure of word of mouth: the dispersion of conversations across different communities. We hypothesize that conversations that are focused within a more narrow audience are likely to have less of an impact than those that occur across a wider spectrum of communities.
The third challenge comes from the fact that word of mouth is not exogenous. While the mapping from word of mouth to future sales is of great interest to the firm, we must also recognize that word of mouth is at the same time an outcome of past sales. This has implications for the measurement of word of mouth as well as for the interpretation of any measurement. High positive word of mouth today does not necessarily mean higher sales tomorrow. It may, in fact, just mean that we had high sales yesterday. Thus, to truly understand the nature of the link, we need to understand the full dynamic relationship between sales and word of mouth. Further, we must allow for the fact that the role and/or impact of word of mouth may change over a product's life.
As a context for our inquiry, we have chosen new TV shows during the 1999/2000 seasons. Word of mouth appears to be especially important for entertainment goods: A recent Forrester report concludes that approximately 50% of young Net surfers rely on word of mouth recommendations to purchase CDs, movies, Videos/DVDs and games (Forrester Research, 2000). Thus, television shows are natural candidates to use for testing the dynamic nature of word of mouth. In addition, the "purchase" of a TV show is a repeat purchase. This is interesting in this context because the consumer's purchase experience in period t will affect not only her decision to talk about it but also her consumption experience in period t+1. Our source of word of mouth information is drawn from Usenet, a collection of thousands of newsgroups with very diverse topics.
It is our primary objective in this paper to address the above challenges. In so doing, we will evaluate the informativeness of the two measures—counts and dispersion—to the manager. Specifically, we envision a manager attempting to learn from aggregate data the underlying process governing the sales of her product. If she had the opportunity to measure word of mouth, this paper offers unique insight into which aspects of it she should try to measure. Given this focus, we are seeking measures that are practical to implement from both the perspective of hard costs and effort. We make no claim that the measures we investigate here are in any sense optimal. Instead, we hope to show when and if they have explanatory power in the dynamic sales model. Another important objective of the paper is an investigation of the usefulness of online conversations in the study of word of mouth communications. The context we study is characterized by a purchase decision made offline, yet we are measuring word of mouth online. Thus, to the extent that we find that certain measures are informative, we argue that this supports the idea that at least some aspects of online word of mouth are proxies for overall word of mouth (offline as well online). Given the obvious operational advantages of measuring word of mouth online vs. offline, we hope to spur a significant increase in focus on the Web as a laboratory for word of mouth research.