There’s a saying in hospitality: The customer is always right. In fact, customers might be more influential than ever, according to a study of online restaurant reviews.
Yelp, the website where consumers share their service experiences, often amplifies pest problems and other hygiene issues at restaurants, causing consumers to avoid those with bad ratings. Restaurants, in turn, often respond by cleaning up their act, according to an analysis of Yelp reviews, OpenTable reservations, and data from the New York City Department of Health and Mental Hygiene.
"The information contained in online reviews helps consumers choose cleaner restaurants, which is a pretty robust finding."
Harvard Business School Assistant Professor Chiara Farronato and Georgios Zervas, an associate professor at Boston University, used machine learning to pull out so-called hygiene signals in reviews of New York City restaurants. Their work has three major takeaways (no pun intended):
- First, online reviews are more revealing when it comes to issues that consumers directly experience—the presence of pests and food-handling practices—than behind-the-scenes dimensions, such as workers’ hygiene.
- Second, these reviews affect consumer demand, as restaurants are less likely to book up during the weeks following a review that notes poor hygiene.
- Lastly, they found suggestive evidence that restaurants might actually take these influential reviews into account when determining hygiene compliance.
Review websites that put some control in the hands of the consumer are growing in popularity, and not just for dining. Sites like Tripadvisor for travel and Glassdoor for employment publish millions of reviews that hold businesses accountable and boost revenue and bookings for favorite spots. As the COVID-19 pandemic eases but labor shortages persist, these sites offer another way for consumers to navigate a bumpy return to normalcy, particularly in the service industry.
Consumer Reviews and Regulation: Evidence from NYC Restaurants, a recent working paper for the National Bureau of Economic Research, shows that restaurant review websites not only provide a useful guide for consumers, but could help influence the restaurant business and its regulators. After all, Farronato says, monitoring hygiene isn’t merely in the hands of regulators. Consumers can play an influential role when it comes to identifying vermin and other cleanliness issues.
“The information contained in online reviews helps consumers choose cleaner restaurants, which is a pretty robust finding. Restaurants should thus take that effect into account when deciding whether to clean up, and we find some evidence confirming this as well,” Farronato says.
Machine learning for dining out
Regulatory health inspections are nothing new: Most cities maintain grading programs to evaluate restaurants’ hygiene practices, and that information is publicly available to savvy diners.
Yelp is a mainstream, user-friendly way to aggregate this information. As such, the professors devised an algorithm to identify how review text on Yelp is associated with regulatory inspections. They found that food handling violations are predicted by words like “sick” and “nauseous,” while the presence of pests is predicted by words like “cockroaches” and “filthy.”
“The semantic connection gives us some reassurance that consumers reading those reviews could actually infer hygiene information,” they write.
Not surprisingly, customers weren’t in a hurry to visit such businesses. The professors’ machine learning model helped them identify the most negative hygiene reviews in order to compare the probability that the restaurant would be sold out in the two weeks before and after the submission of such reviews. They found that a restaurant was between 0.4 and 0.7 percentage points less likely to be sold out on OpenTable after a poor review, constituting a 1.8 to 3.2 percent reduction in sold-out probability.
"I hope regulators will start seeing all these pieces of digital data as useful in helping them do a better job."
They also compared hygiene compliance across restaurants reviewed on the site. The analysis finds that more visible restaurants tend to violate less on the hygiene issues mentioned in reviews, suggesting that restaurants may indeed take reviews into account when cleaning up.
Farronato believes digital tools could streamline the inspection process, which is costly for cash-strapped cities.
“I hope regulators will start seeing all these pieces of digital data as useful in helping them do a better job in two ways: One is doing what they do more efficiently, and two is maybe doing less, because they can outsource certain things to the crowd,” she says.
Crowdsourcing isn’t perfect
Crowdsourcing has limits, though. It’s important for review sites such as Yelp to post chronological reviews that reflect current conditions, since hygiene can vary wildly based on new management and staff, the research team says.
“Review sites should be careful in making sure that the freshness of reviews is coherent with current status,” Farronato warns. In an industry with razor-thin margins, a few outdated but prominently displayed reviews could sink a restaurant for good. “One or two reviews can destroy their life,” she says.
Plus, reviews aren’t representative of all experiences. A consumer with a grudge or a bad day can submit a scathing review.
"Review sites should be careful in making sure that the freshness of reviews is coherent with current status."
It cuts both ways: Restaurants can manipulate users to write positive reviews by offering discounts—or even by writing reviews themselves. User platforms need to be thoughtful about displaying reviews by trustworthy users.
Farronato and Zervas’ research takes into account whether a restaurant had received recent reviews and reviewers’ history throughout Yelp to ward off some of this bias.
“The analysis of platform incentives to provide current and unbiased signals of quality is a valuable avenue for future research,” they write.
The magic of monitoring
Farronato says this monitoring method could be applied to other industries with comparable review sites, such as employment conditions that are reviewed on Glassdoor.
“Glassdoor is a great forum for workers and employees to discuss the quality of a job at a specific company. I think regulators focusing on labor conditions could tap into that trove of data to understand what works, so maybe we can design regulations better and quickly identify what doesn’t work at which business, go in, and fix it. It’s incredibly powerful,” she says.
In the near term, Farronato hopes that regulators and online review platforms can work together to create cleaner, healthier dining experiences overall. Online reviews have the advantage over traditional regulatory efforts of being cheaper to collect and more frequently updated.
“The outcome that I would like to see is having more of an integrated approach,” she says. “They’re definitely complementary resources to inform and protect consumers from poor-quality service.”
You Might Also Like:
- Company Reviews on Glassdoor: Petty Complaints or Signs of Potential Misconduct?
- How Racial Bias Taints Customer Service: Evidence from 6,000 Hotels
- Identify Great Customers from Their First Purchase
Related reading from the Working Knowledge Archives
The Yelp Factor: Are Consumer Reviews Good for Business?
Feedback or ideas to share? Email the Working Knowledge team at hbswk@hbs.edu.
Image: iStockphoto/nezarettin azmanoğlu