It’s Alive! Business Scholars Turn to Experimental Research
Business researchers are turning increasingly to experiments in the lab and field to unlock the secrets of what motivates CEOs, consumers, and policymakers. Key concepts include:
- Interest in experimental research may have been ignited by prominent behavioral economist Danny Kahneman, who won the Nobel Prize in Economics in 2002, and by journalist Malcolm Gladwell, author of The Tipping Point and Blink.
- Researchers use field and lab experiments to better understand the logic of real-world decisions, which sometimes fly in the face of established economic theory.
- Experimental research can be considered a leg on a three-legged stool that also includes field data and theory.
- While agreeing to be part of a field experiment can be time-consuming for a company, the results can be invaluable—and more cost-effective than paying a market research firm or a consultant.
A large amusement park. A long line at an airport. A children's summer camp in Italy. What do these places have in common? Surprisingly, all are settings for serious research by Harvard Business School faculty.
There's a sea change afoot in the world of business academia. While scholars traditionally have depended on hard corporate data, previous literature, and insular theories, they are increasingly hinging their hypotheses on proactive sociological and psychological experiments—both in the lab and in the field.
"The most exciting change I've seen is the increase in the interest of behavioral research from CEOs, policymakers, and high-stakes decision makers."
"It's always been obvious to social scientists and business scholars that there are lots of things that you can't learn in the laboratory, but it is becoming increasingly obvious that there are some things that you may not be able to learn anywhere else," says Alvin E. Roth, a professor in the Negotiation, Organizations & Markets (NOM) Unit at HBS and a longtime champion of experimental economics.
For example, to track how viewers watch online video ads, researchers take to the lab to employ cutting-edge tools like an infrared eye tracker to measure eye movements and special vision software that analyzes facial expressions to gauge emotional responses.
Experiments in the field are gaining popularity too, covering a wide spectrum of complexity but that can be as simple as a researcher testing whether people will let him cut in line at the airport for a $10 bribe.
These experiments are proving integral in learning not only what makes people tick but also how that matters for business.
"Over the last decade, scholars and managers alike have come to realize the importance of understanding how the human mind works, since it has important consequences for all sorts of behaviors," says Francesca Gino, an associate professor in the NOM Unit. "I think the most exciting change I've seen is the increase in the interest of behavioral research from CEOs, policymakers, and high-stakes decision makers."
One indicator that experimental research is hot: business is booming at Harvard Business School's Computer Lab for Experimental Research (CLER), which manages a pool of paid human participants and provides researchers with dozens of computer stations in a central lab, specialized technology tools, and six small-group rooms.
"The level use of experimental research at HBS has grown beyond what anyone imagined when we first built the CLER lab," says Director of Research Computing Chase Harrison. "We're constantly evaluating how to meet the growing needs of our faculty."
In FY2000, for instance, eight HBS faculty and doctoral students, from two of the School's 10 academic units, used CLER to conduct experiments, notes John Galvin, assistant director of research computing services. In 2010, that number had increased to 25 faculty members, representing seven academic units. Another indication that experimental research is becoming more mainstream is the fact that a number of faculty who are not primarily experimentalists use the CLER, including HBS Dean Nitin Nohria.
The tipping point
To explain this surging interest over the past decade, Associate Professor Michael I. Norton cites two key factors. First, prominent behavioral economist Danny Kahneman won the Nobel Prize in Economics in 2002, illuminating the role of psychology in economic science. Second, journalist Malcolm Gladwell wrote The Tipping Point and Blink, meteoric bestsellers that relied heavily on the results of experimental studies at major universities.
"Those books are basically reviews of social science research papers," says Norton, a social scientist who has been teaching in the Marketing Unit since 2005. "It turned out that corporate managers found the subjects compelling and interesting. So then there was a hunger to find people who were doing that kind of research. And business schools started hiring more and more social psychologists—including me."
Indeed, behavioral experiments are often especially colorful, even fun, piquing the interest of outside observers.
"Traditionally, economics had a very rational view of people, and workers were almost considered like widgets."
For instance, Norton and several other researchers ran an experiment at a large American amusement park, documented in the paper Paying to be Nice: Consistency and Costly Prosocial Behavior. As visitors exited a popular ride, they had the option of purchasing a photo of themselves enjoying the ride. For the experiment, some visitors were told upon purchase that half of the photo proceeds would go to a charity. The results showed that visitors who thought the proceeds went to a charity were more likely to exhibit altruistic behavior by purchasing gifts for others at a subsequent visit to a park gift shop. The point of the experiment for business practice: The research indicates that it's possible to induce selfless behavior in your employees.
In addition to the desire to make research more accessible, there is an increasing realization among scholars that humans are often irrational, and that the logic of real-world decisions therefore sometimes flies in the face of established economic theory. Experiments help to insert some humanity into economic research.
"Traditionally, economics had a very rational view of people, and workers were almost considered like widgets," says Assistant Professor Ian Larkin, a member of the NOM Unit. His experimental research suggests that social comparisons are a bigger workplace motivator than financial incentives, helping to explain why even financially minded people make financially unsound decisions.
In one computer lab experiment, for example, Larkin and colleagues showed that participants would perform better on a word search puzzle if they were told that previous participants had performed especially well. In another experiment, a field study, Larkin showed that salespeople at a software firm seemed to care more about belonging to the company's nominal "president's club" than they cared about significant financial bonuses.
"Human beings are biased, prejudiced, and react to incentives according to their preferences," says HBS Research Fellow Marco Piovesan, whose research on how temptation influences work productivity included field experiments with children tempted by soda and candy at an Italian summer camp. "Understanding where these preferences and their behavior come from is crucial to design effective rules and institutions."
The three-legged stool
As enthusiastic as they are about experimentation, the HBS scholars note that it is not the be-all and end-all of research, but rather a necessary component to a well-rounded study. Roth applauds the fact that experiments are no longer sequestered from other business research. "Experiments used to be used more in stand-alone investigations," he says. "Today they are often part of a line of investigation that includes both experimental and nonexperimental work."
Larkin considers experimental research to be a leg on a three-legged stool that also includes field data and theory. Gino stresses the importance of including both field experiments and lab experiments in a study, if possible.
"Field and lab research can really benefit from one another," Gino says, explaining that one often leads to the other. For instance, in her research she found that increasing the salience of one's own ethical standards at the time of temptation (that is, when one faces the decision to cheat) reduces unethical behavior, a conclusion reached through a combination of lab and field studies with Professor Max H. Bazerman and HBS doctoral student Lisa Shu. The trio supported their hypothesis in a lab setting that involved recalling the Ten Commandments, and then performed a similar ethical field study at an insurance company, where they successfully replicated the findings they saw in the lab. "This was nice, since it gave us confidence that our findings were generalizable," Gino says.
A call to arms
The 7,500-person pool available to participate in CLER lab experiments generally comprises college students, even if the hypothesis is related to the corporate world. There's a good reason for that: Students are more accessible than business professionals. CLER pays participants from $15 to $40 per experiment. A college student could eat for a week on that kind of cash, but for a business practitioner, it's not worth it.
"In some cases, college students are a fine example of human nature," says Thales S. Teixeira, an assistant professor in the Marketing Unit. "But there are other situations, depending on your research, in which you shouldn't use college students. If you want to understand what drives nonethical behavior among accountants, for instance, then you've got to get accountants."
That's where field experiments come in.
"Instead of asking companies to provide data for us to analyze, increasingly we're proposing that we do experiments together," Norton says.
Researchers stress the difference between analyzing existing real-world data and conducting experiments in the real world. While existing company data might explain what happens during any given year, an experiment can explain why it happens.
"Field data will tell you that a mouse went over there," says Teixeira, who often uses CLER's infrared eye tracker to study how people respond to online advertisements. "But without a controlled experiment, you won't know if the mouse moved because it was attracted by cheese, or by another mouse, or because sunlight was there."
While agreeing to be part of a field experiment can be time-consuming for a company, the results can be invaluable—and more cost-effective than paying a market research firm or a consultant.
"I wish more companies would be open to do field experiments," Larkin says. "I think it's a big, untapped way for companies to optimize not just their marketing but also how they design incentives, their employee feedback design, and also the different elements of business."