Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcasts
  • About Us
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Cold Call Podcast
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    • COVID-19 Business Impact Center
      COVID-19 Business Impact Center
      The Effects of Hierarchy on Learning and Performance in Business Experimentation
      23 Mar 2020Working Paper Summaries

      The Effects of Hierarchy on Learning and Performance in Business Experimentation

      by Sourobh Ghosh, Stefan H. Thomke, and Hazjier Pourkhalkhali
      Do senior managers help or hurt business experiments? Analyzing a dataset of more than 6,300 experiments on the A/B/n testing platform Optimizely, this study suggests that involving senior executives in experimentation teams can have surprising consequences.
      LinkedIn
      Email

      Author Abstract

      Do senior managers help or hurt business experiments? Despite the widespread adoption of business experiments to guide strategic decision-making, we lack a scholarly understanding of what role senior managers play in firm experimentation. Using proprietary data of live business experiments from the widely-used A/B testing platform, Optimizely, this paper estimates the association of management hierarchy with learning from experiments and their performance outcomes across industries and contexts. Our findings suggest that senior management’s association is mixed. On the one hand, senior managers’ involvement associates with bolder experiments that create more statistically significant learning signals aiding in the exploration of new strategic directions. On the other hand, their involvement associates with less cause-and-effect learning that is instrumental to optimization and performance improvements. Our results contribute to a burgeoning literature on experimentation in strategy, while helping articulate limits that organizational design might place on data-driven decision-making. Furthermore, we describe different experimental learning modes in the formation of strategy, offering important implications for how managers can modulate search and performance outcomes.

      Paper Information

      • Full Working Paper Text
      • Working Paper Publication Date: February 2020
      • HBS Working Paper Number: HBS Working Paper #20-081
      • Faculty Unit(s): Technology and Operations Management
        Trending
          • 29 Oct 2020
          • Research & Ideas

          The COVID Gender Gap: Why Fewer Women Are Dying

          • 13 Jul 2020
          • Research & Ideas

          Merck CEO Ken Frazier Discusses a COVID Cure, Racism, and Why Leaders Need to Walk the Talk

          • 11 Jan 2021
          • Research & Ideas

          Is A/B Testing Effective? Evidence from 35,000 Startups

          • 17 Feb 2020
          • Sharpening Your Skills

          How Entrepreneurs Can Find the Right Problem to Solve

          • 13 Jan 2021
          • Research & Ideas

          How 'Small C' Change Can Beat Large-Scale Rebuilding

      Stefan H. Thomke
      Stefan H. Thomke
      William Barclay Harding Professor of Business Administration
      Chair, General Management Program
      Contact
      Send an email
      → More Articles
      Find Related Articles
      • Innovation and Invention
      • Organizational Design
      • Search Technology
      • Product Development
      • Research and Development

      Sign up for our weekly newsletter

      Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
      ǁ
      Campus Map
      Harvard Business School Working Knowledge
      Baker Library | Bloomberg Center
      Soldiers Field
      Boston, MA 02163
      Email: Editor-in-Chief
      →Map & Directions
      →More Contact Information
      • Make a Gift
      • Site Map
      • Jobs
      • Harvard University
      • Trademarks
      • Policies
      • Digital Accessibility
      Copyright © President & Fellows of Harvard College