Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcasts
  • About Us
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Cold Call Podcast
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    • COVID-19 Business Impact Center
      COVID-19 Business Impact Center
      Experimentation and Startup Performance: Evidence from A/B Testing
      06 Sep 2019Working Paper Summaries

      Experimentation and Startup Performance: Evidence from A/B Testing

      by Rembrand Koning, Sharique Hasan, and Aaron Chatterji
      Is experimentation the right strategy for startups? This analysis of the adoption of A/B testing technology by 35,000 global startups provides evidence that a strategy based on repeated experimentation will improve performance over time. However, the benefits of experimentation vary. Experimentation helps younger startups “fail faster,” while older firms may discover new, high-growth products.
      LinkedIn
      Email

      Author Abstract

      Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence of how digital experimentation affects the performance of a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. We find that, despite its prominence in the business press, relatively few firms have adopted A/B testing. However, among those that do, we find increased performance on several critical dimensions, including page views and new product features. Furthermore, A/B testing is positively related to tail outcomes, with younger ventures failing faster and older firms being more likely to scale. Firms with experienced managers also derive more benefits from A/B testing. Our results inform the emerging literature on entrepreneurial strategy and how digitization and data-driven decision-making are shaping strategy.

      Paper Information

      • Full Working Paper Text
      • Working Paper Publication Date: August 2019
      • HBS Working Paper Number: HBS Working Paper #20-018
      • Faculty Unit(s): Strategy
        Trending
          • 29 Oct 2020
          • Research & Ideas

          The COVID Gender Gap: Why Fewer Women Are Dying

          • 13 Jul 2020
          • Research & Ideas

          Merck CEO Ken Frazier Discusses a COVID Cure, Racism, and Why Leaders Need to Walk the Talk

          • 11 Jan 2021
          • Research & Ideas

          Is A/B Testing Effective? Evidence from 35,000 Startups

          • 17 Feb 2020
          • Sharpening Your Skills

          How Entrepreneurs Can Find the Right Problem to Solve

          • 13 Jan 2021
          • Research & Ideas

          How 'Small C' Change Can Beat Large-Scale Rebuilding

      Rembrand M. Koning
      Rembrand M. Koning
      Assistant Professor of Business Administration
      Contact
      Send an email
      → More Articles
      Find Related Articles
      • Innovation Strategy

      Sign up for our weekly newsletter

      Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
      ǁ
      Campus Map
      Harvard Business School Working Knowledge
      Baker Library | Bloomberg Center
      Soldiers Field
      Boston, MA 02163
      Email: Editor-in-Chief
      →Map & Directions
      →More Contact Information
      • Make a Gift
      • Site Map
      • Jobs
      • Harvard University
      • Trademarks
      • Policies
      • Digital Accessibility
      Copyright © President & Fellows of Harvard College