Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcast
  • Managing the Future of Work Podcast
  • About Us
  • Book
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Podcasts
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    How Numbers Talk to People
    10 Jun 2013Research & Ideas

    How Numbers Talk to People

    In their new book Keeping Up with the Quants, Thomas H. Davenport and Jinho Kim offer tools to sharpen quantitative analysis and make better decisions. Read our excerpt.
    LinkedIn
    Email

    The problem with big data is that there is, well, so much of it. Analyzing it is like trying to sip from a firehose.

    Just in time, a new book on the art and science of quantitative analysis arrives this week. In Keeping Up with the Quants: Your Guide to Understanding and Using Analytics, authors Thomas H. Davenport and Jinho Kim offer an introduction to the best methods for consuming data.

    In this excerpt, Davenport and Kim discuss multiple ways to tell a story with data, "the best way to communicate results to nonanalytical people." The authors recommend six formats: The CSI Story, the Eureka Story, the Mad Scientist Story, the Survey Story, the Prediction Story, and the "Here's What Happened" Story. The excerpt focuses on the first two.

    —Sean Silverthorne

    Keeping Up with the Quants: Your Guide to Understanding and Using Analyticsbook excerpt

    Framing The Problem

    From: Keeping Up with the Quants

    By Thomas H. Davenport and Jinho Kim

    THE CSI STORY

    Some quantitative analyses are like police procedural television programs; they attempt to solve a business problem with quantitative analysis. Some operational problem crops up, and data are used to confirm the nature of the issue and find the solution. This situation often does not require deep statistical analysis, just good data and reporting approaches. It is often encountered in online businesses, where customer clickstreams provide plenty of data-often too much-for analysis.

    One expert practitioner of the CSI story approach is Joe Megibow, vice president and general manager of online travel company Expedia's US business. Joe was previously a Web analytics maven-and he still is-but his data-based problem-solving approaches have led to a variety of impressive promotions.

    Many of the Expedia investigations involve understanding the reasons behind lost online sales. One particular CSI story involved lost revenue on hotel payment transactions. Analysis of data suggested that after a customer had selected a hotel, filled in the travel and billing information, then clicked the "Buy Now" button, a percentage of the sales transactions were not being completed successfully.

    Megibow's team investigated the reason for the failures, again using Web metrics data and server log files throughout the process. Apparently, the "Company" field under the customer's name was causing a problem. Some customers interpreted it as the name of the bank that supplied their credit card, and then they also supplied the bank's address in the billing address fields. This caused the transaction to fail with the credit card processor. Simply removing the "Company" field immediately raised profits for Expedia by $12 million. Megibow says that Expedia has explored many of these CSI-like stories, and they almost always yield substantial financial or operational benefits.

    Sometimes the CSI stories do involve deeper quantitative and statistical analysis. One member of Megibow's team was investigating which customer touchpoints were driving online sales transactions. The analyst used the Cox regression model—an approach originally used to determine which patients would die and which would live over certain time periods-of "survival analysis." The analysis discovered that the simpler prior models were not at all correct about what marketing approaches were really leading to a sale. Megibow commented, "We didn't know we were leaving money on the table."

    THE EUREKA STORY

    The Eureka story is similar to the CSI story, except that it typically involves a purposeful approach to a particular problem (as opposed to stumbling over the problem) to examine a major change in an organization's strategy or business model. It tends to be a longer story with a greater degree of analysis over time. Sometimes Eureka stories also involve other analytical story types, just because the results are so important to the organizations pursuing them.

    At Expedia again, for example, one Eureka story involved eliminating change/cancel fees from online hotel, cruise, and car rental reservations. Until 2009, Expedia and its competitors all charged up to $30 for a change or cancellation—above and beyond the penalties the hotel imposed. Expedia and other online bookers' rates were typically much lower than booking directly with a hotel, and customers were willing to tolerate change/cancel fees.

    However, by 2009 it had become apparent that the fees had become a liability. Expedia's rates were closer to those of the hotels' own rates, so the primary appeal of Expedia had become convenience—and change/cancel fees were not convenient. Analysts looked at customer satisfaction rates, and they were particularly low for customers who had to pay the fees. Expedia's call center representatives were authorized to waive the change/cancel fees for only one reason: a death in the customer's family. A look at the number of waivers showed double-digit growth for the past three years. Either there was a death epidemic, or customers had figured out they could get their money back this way.

    Expedia executives realized the market had changed, but change/ cancel fees represented a substantial source of revenue. They wondered if the fees were eliminated, would conversion (completed sale) rates go up? In April of 2009, they announced a temporary waiver of fees for the month (a bit of a mad scientist testing story, described below). Conversion rates immediately rose substantially. Executives felt that they had enough evidence to discontinue the fees, and the rest of the industry followed suit.

    Across town in Seattle lies Zillow, a company that distributes information about residential real estate. Zillow is perhaps best known to quant jocks for its "Zestimates," a proprietary algorithm that generates estimates of home values. But, like Expedia, Zillow's entire culture is based on data and analysis—not surprisingly, since the company was founded by Rich Barton, who also founded Expedia.

    One of Zillow's Eureka stories involved a big decision to change how it made its money from relationships with real estate agents. Zillow began to work with agents in 2008, having previously been focused on consumers. One aspect of its agent-related business model was selling advertising by agents and delivering leads to them. Zillow charged the agents for the leads, but the value per lead was not enough in the view of executives. Chloe Harford, a Zillow executive who heads product management and strategy, was particularly focused on figuring out the right model for increasing lead value and optimizing the pricing of leads.

    Harford, who has a PhD in volcanology, or the study of volcanoes, is capable of some pretty sophisticated mathematical analysis. However, she and her colleagues initially relied on what she calls "napkin math" to explore other ways to generate more leads and price them fairly to agents. In April 2010, Zillow created a new feature—immediately copied by competitors—involving selling advertising to agents. It created many more customer contacts than before, and allowed the consumer to contact the agent directly. Zillow also introduced a sophisticated algorithm for pricing leads to agents that attempts to calculate the economic value of the lead, with an estimate of conversion rates. Competitors also do this to some degree, but probably not to the level of sophistication that Zillow does. The leads and pricing of them are so important that Harford and her colleagues frequently test different approaches of them with some of the Mad Scientist testing approaches described below. In short, Zillow's Eureka stories are intimately tied into its business model and its business success.

    Post A Comment
    In order to be published, comments must be on-topic and civil in tone, with no name calling or personal attacks. Your comment may be edited for clarity and length.
      Trending
        • 31 Jan 2023
        • Research & Ideas

        It’s Not All About Pay: College Grads Want Jobs That ‘Change the World’

        • 25 Jan 2022
        • Research & Ideas

        More Proof That Money Can Buy Happiness (or a Life with Less Stress)

        • 17 Jan 2023
        • In Practice

        8 Trends to Watch in 2023

        • 25 Feb 2019
        • Research & Ideas

        How Gender Stereotypes Kill a Woman’s Self-Confidence

        • 17 May 2017
        • Research & Ideas

        Minorities Who 'Whiten' Job Resumes Get More Interviews

    Find Related Articles
    • Demand and Consumers
    • Strategy
    • Information Management
    • Analysis

    Sign up for our weekly newsletter

    Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
    ǁ
    Campus Map
    Harvard Business School Working Knowledge
    Baker Library | Bloomberg Center
    Soldiers Field
    Boston, MA 02163
    Email: Editor-in-Chief
    →Map & Directions
    →More Contact Information
    • Make a Gift
    • Site Map
    • Jobs
    • Harvard University
    • Trademarks
    • Policies
    • Accessibility
    • Digital Accessibility
    Copyright © President & Fellows of Harvard College