Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcasts
  • About Us
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Cold Call Podcast
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    • COVID-19 Business Impact Center
      COVID-19 Business Impact Center
      Experimentation and Startup Performance: Evidence from A/B Testing
      06 Sep 2019Working Paper Summaries

      Experimentation and Startup Performance: Evidence from A/B Testing

      by Rembrand Koning, Sharique Hasan, and Aaron Chatterji
      Is experimentation the right strategy for startups? This analysis of the adoption of A/B testing technology by 35,000 global startups provides evidence that a strategy based on repeated experimentation will improve performance over time. However, the benefits of experimentation vary. Experimentation helps younger startups “fail faster,” while older firms may discover new, high-growth products.
      LinkedIn
      Email

      Author Abstract

      Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence of how digital experimentation affects the performance of a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. We find that, despite its prominence in the business press, relatively few firms have adopted A/B testing. However, among those that do, we find increased performance on several critical dimensions, including page views and new product features. Furthermore, A/B testing is positively related to tail outcomes, with younger ventures failing faster and older firms being more likely to scale. Firms with experienced managers also derive more benefits from A/B testing. Our results inform the emerging literature on entrepreneurial strategy and how digitization and data-driven decision-making are shaping strategy.

      Paper Information

      • Full Working Paper Text
      • Working Paper Publication Date: August 2019
      • HBS Working Paper Number: HBS Working Paper #20-018
      • Faculty Unit(s): Strategy
        Trending
          • 08 Mar 2021
          • In Practice

          COVID Killed the Traditional Workplace. What Should Companies Do Now?

          • 25 Feb 2019
          • Research & Ideas

          How Gender Stereotypes Kill a Woman’s Self-Confidence

          • 17 May 2017
          • Research & Ideas

          Minorities Who 'Whiten' Job Resumes Get More Interviews

          • 07 Apr 2021
          • Research & Ideas

          How Teams Work: Lessons from the Pandemic

          • 02 Apr 2021
          • Research & Ideas

          Salary Negotiations: A Catch-22 for Women

      Rembrand M. Koning
      Rembrand M. Koning
      Assistant Professor of Business Administration
      Contact
      Send an email
      → More Articles
      Find Related Articles
      • Innovation Strategy

      Sign up for our weekly newsletter

      Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
      ǁ
      Campus Map
      Harvard Business School Working Knowledge
      Baker Library | Bloomberg Center
      Soldiers Field
      Boston, MA 02163
      Email: Editor-in-Chief
      →Map & Directions
      →More Contact Information
      • Make a Gift
      • Site Map
      • Jobs
      • Harvard University
      • Trademarks
      • Policies
      • Digital Accessibility
      Copyright © President & Fellows of Harvard College