Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcast
  • Managing the Future of Work Podcast
  • About Us
  • Book
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Podcasts
    • Managing the Future of Work Podcast
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    Experimentation and Startup Performance: Evidence from A/B Testing
    06 Sep 2019Working Paper Summaries

    Experimentation and Startup Performance: Evidence from A/B Testing

    by Rembrand Koning, Sharique Hasan, and Aaron Chatterji
    Is experimentation the right strategy for startups? This analysis of the adoption of A/B testing technology by 35,000 global startups provides evidence that a strategy based on repeated experimentation will improve performance over time. However, the benefits of experimentation vary. Experimentation helps younger startups “fail faster,” while older firms may discover new, high-growth products.
    LinkedIn
    Email

    Author Abstract

    Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence of how digital experimentation affects the performance of a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. We find that, despite its prominence in the business press, relatively few firms have adopted A/B testing. However, among those that do, we find increased performance on several critical dimensions, including page views and new product features. Furthermore, A/B testing is positively related to tail outcomes, with younger ventures failing faster and older firms being more likely to scale. Firms with experienced managers also derive more benefits from A/B testing. Our results inform the emerging literature on entrepreneurial strategy and how digitization and data-driven decision-making are shaping strategy.

    Paper Information

    • Full Working Paper Text
    • Working Paper Publication Date: August 2019
    • HBS Working Paper Number: HBS Working Paper #20-018
    • Faculty Unit(s): Strategy
      Trending
        • 13 Aug 2021
        • Research & Ideas

        Managers, Here’s How to Bond with New Hires Remotely

        • 09 Dec 2019
        • Research & Ideas

        Identify Great Customers from Their First Purchase

        • 05 May 2022
        • Research & Ideas

        Why Companies Raise Their Prices: Because They Can

        • 30 Nov 2021
        • Cold Call Podcast

        TikTok: Super App or Supernova?

        • 25 Jan 2016
        • Research & Ideas

        When Negotiating a Price, Never Bid with a Round Number

    Rembrand M. Koning
    Rembrand M. Koning
    Assistant Professor of Business Administration
    Contact
    Send an email
    → More Articles
    Find Related Articles
    • Innovation Strategy

    Sign up for our weekly newsletter

    Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
    ǁ
    Campus Map
    Harvard Business School Working Knowledge
    Baker Library | Bloomberg Center
    Soldiers Field
    Boston, MA 02163
    Email: Editor-in-Chief
    →Map & Directions
    →More Contact Information
    • Make a Gift
    • Site Map
    • Jobs
    • Harvard University
    • Trademarks
    • Policies
    • Digital Accessibility
    Copyright © President & Fellows of Harvard College