For business executives trying to decide where exactly in the digital realm to invest their advertising dollars, new research indicates that paid search ads on review sites such as Yelp can be a good way to go—at least for small, lesser-known companies.
Harvard Business School Assistant Professor Michael Luca and Weijia (Daisy) Dai, an assistant professor of economics at Lehigh University, recently studied effectiveness of paid search ads for small businesses by designing a large-scale field experiment that involved nearly 20,000 restaurants and 24 million advertising exposures on Yelp.com.
The results, written up in new working paper Effectiveness of Paid Search Advertising: Experimental Evidence, showed these restaurants enjoyed a significant spike in exposure, in actions such as map requests and calls to restaurants, and most likely a boost in business as well from the ads.
Luca says these paid search ads, which appear on top of a Yelp user’s online search results for, say, “Chinese Food” or “Cheap Dinner,” spotlight restaurants that might otherwise blend inconspicuously into a list of many competitors.
“There are thousands of restaurants in cities like San Francisco and New York, which is great for customers … but also means that even some businesses with solid reputations on Yelp can be hard for people to discover,” Luca says.
Dai adds: “Going into the project, we weren’t sure exactly what to expect, but our results suggest that these ads are effective. If the effect were just on clicks and not on calls and direction requests, I’d be less inclined to think that Yelp ads work for these businesses.”
Ad effectiveness best for small businesses
In recent years, internet advertising has become the fastest-growing marketing channel, accounting for a whopping $60 billion in spending in the United States in 2015, up from $26 billion in 2010. Advertisers are steering the biggest share of their digital ad dollars—about half of all online ad expenditures—toward paid search.
"Smaller businesses have less of an established brand and are more likely to benefit from ads"
Yet even with all that money flowing to ads, executives are doing quite a bit of hand-wringing over whether they actually produce sales. Luca and Dai thought they could help answer that question.
Besides, previous research measuring impact of search ads for bigger name brands had executives questioning whether they are a sound investment. An influential study involving eBay showed that search engine ads—especially on brand keywords—were ineffective for the website. After the eBay study came out, the market saw a dip in large-businesses advertising, Luca says.
“I am a fan of eBay’s decision to produce rigorous, managerially relevant research, and I’d like to see more out there,” he says. “In the case of advertising effectiveness, I thought: What if you take small businesses and give them advertisements for a three-month span, would it be like eBay or would it be different? My hunch was that it would be different.”
An experimental field study was born.
Yelp agrees to run free ads
The genesis of the advertising project came while Luca was having coffee with Geoff Donaker, then COO of Yelp. Yelp, which hosts consumer reviews of all types of businesses, has become a popular online destination for people searching for services, boasting about 163 million unique visitors monthly on its site in 2015.
Luca admits to being a bit of a data nerd, a trait he says he and Donaker share. Luca said to Donaker, “Look, we don’t know whether Yelp ads work or not. Maybe they have a big effect and maybe they don’t. Why don’t we collaborate on an experiment testing their impact?”
That was music to Donaker’s ears. Yelp had run experiments to optimize advertisements and had consultants try to figure out the impact of ads. But in the end, these analyses didn’t answer exactly what they (or potential advertisers) wanted to know, which is: What would happen to their business if restaurants bought an advertising package?
Luca proposed a large-scale experiment in which Yelp would give away several months of advertising spots for free to randomly selected businesses. “Geoff sort of cringed,” Luca recalls, at the thought that he would effectively be giving away the equivalent of millions of dollars worth advertising exposures.
By the end of the conversation, they were excited enough that the idea expanded to a broader vision of doing economic research with a high degree of flexibility in accessing Yelp data and running experiments, with the goal of producing managerially relevant academic research with the freedom to publish it.
Example of a Yelp ad for a restaurant
Luca and Dai turned their focus to restaurants throughout the US that had not actively advertised on Yelp in the year prior to the experiment. They also narrowed their sample down to restaurants that were eligible to purchase search ad packages based on Yelp’s guidelines: They were in good standing with a certain number of trusted reviews, solid ratings, and complete business listing information, including photos.
Starting with a total pool of 18,295 businesses, the researchers randomly assigned paid search ad packages on Yelp to 7,210 restaurants during the period between August and October 2015. In an interesting twist, the restaurants were not told they were essentially enjoying a free ad ride. This allowed the researchers to separate the impact of advertisements from the possibility that restaurants would change their behavior (for example, by messaging users more frequently), when they were advertising.
The researchers monitored views of each restaurant’s Yelp page, requests for directions, calls placed to the restaurant, and visits to its home page. Although Luca and Dai couldn’t draw a direct correlation between advertising and profit generated through those observations, they could get a sense it by also matching the number of restaurant clicks to sales data for the subset of businesses being given the free advertising. The calculation suggested that advertising would produce a positive return on average for restaurants in the experimental sample.
Positive results for restaurants
Luca and Dai were right in their hypothesis. Restaurants included in ads saw an average surge of 24.6 percent in Yelp page views, which estimate is the equivalent to an 8 percent revenue gain.
The ads also created other consumer actions. Map inquiries jumped 18 percent, restaurant calls increased 13 percent, and clicks on the restaurants’ website rose 9 percent. The eateries also saw a 5 percent increase in the number of restaurant reviews, suggesting that the ads prompted more visits.
Luca believes the key difference in ad effectiveness might come down to the size of the brand. eBay may not have experienced as much of a boost from its ads since its brand was widely known long before the ads appeared, whereas highlighting the mere presence of a lesser-known but highly rated mom-and-pop restaurant in an ad might lead to more bustle for the business.
According to Dai: “Smaller businesses have less of an established brand and are more likely to benefit from ads. Making people aware of your name is potentially a powerful way to market if you’re a small business.”
(The research did point out one benefit for big brands, says Luca. “Even in the eBay experiment, they found that their advertising was more effective when they focused on keywords that newer users don’t typically associate with eBay.”)
Ads work—but only for awhile
There was some bad news for businesses in the study: The afterglow of the ads was short-lived.
All advertising effects dropped to zero immediately following the advertising period, suggesting that ads temporarily make people aware of businesses but may not have a long-term residual impact after advertising stops.
“That said,” Dai adds, “it is possible that a customer attracted by the search ads may return to the restaurant later without using Yelp; that would be a long-term benefit to the restaurant, but not captured in our Yelp outcome measures.”
Given the research findings, Luca recommends that business executives assess their online reputations, whether on Yelp, TripAdvisor, or another ratings website, and then think about ways to publicize favorable feedback they receive from consumers out to potential customers searching for their type of business.
“This is not to say every type of advertising works for small businesses,” Luca says. “But looking across the growing set of evidence on the impact of advertisements, it seems like businesses don’t always make the right call about whether and where to advertise. Mistakes seem to go in both directions, with some businesses advertising when they shouldn’t be and others underinvesting in digital ads.”
So, big businesses—like eBay or Gap—should be running experiments to test their own ads. If you’re a small business with a good reputation, getting that information out to people is an important thing to do.
“Having an online presence isn’t enough,” Luca says. “You have to make people aware of it.”