Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcast
  • Managing the Future of Work Podcast
  • About Us
  • Book
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Podcasts
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    The Effects of Hierarchy on Learning and Performance in Business Experimentation
    23 Mar 2020Working Paper Summaries

    The Effects of Hierarchy on Learning and Performance in Business Experimentation

    by Sourobh Ghosh, Stefan H. Thomke, and Hazjier Pourkhalkhali
    Do senior managers help or hurt business experiments? Analyzing a dataset of more than 6,300 experiments on the A/B/n testing platform Optimizely, this study suggests that involving senior executives in experimentation teams can have surprising consequences.
    LinkedIn
    Email

    Author Abstract

    Do senior managers help or hurt business experiments? Despite the widespread adoption of business experiments to guide strategic decision-making, we lack a scholarly understanding of what role senior managers play in firm experimentation. Using proprietary data of live business experiments from the widely-used A/B testing platform, Optimizely, this paper estimates the association of management hierarchy with learning from experiments and their performance outcomes across industries and contexts. Our findings suggest that senior management’s association is mixed. On the one hand, senior managers’ involvement associates with bolder experiments that create more statistically significant learning signals aiding in the exploration of new strategic directions. On the other hand, their involvement associates with less cause-and-effect learning that is instrumental to optimization and performance improvements. Our results contribute to a burgeoning literature on experimentation in strategy, while helping articulate limits that organizational design might place on data-driven decision-making. Furthermore, we describe different experimental learning modes in the formation of strategy, offering important implications for how managers can modulate search and performance outcomes.

    Paper Information

    • Full Working Paper Text
    • Working Paper Publication Date: February 2020
    • HBS Working Paper Number: HBS Working Paper #20-081
    • Faculty Unit(s): Technology and Operations Management
      Trending
        • 27 Jan 2023
        • Op-Ed

        Have We Lost Sight of Integrity?

        • 01 Feb 2023
        • What Do You Think?

        Will Hybrid Work Strategies Pull Down Long-Term Performance?

        • 31 Jan 2023
        • Research & Ideas

        It’s Not All About Pay: College Grads Want Jobs That ‘Change the World’

        • 17 Jan 2023
        • In Practice

        8 Trends to Watch in 2023

        • 28 Feb 2018
        • Sharpening Your Skills

        Master the Team Meeting

    Stefan H. Thomke
    Stefan H. Thomke
    William Barclay Harding Professor of Business Administration
    Chair, General Management Program
    Contact
    Send an email
    → More Articles
    Find Related Articles
    • Innovation and Invention
    • Organizational Design
    • Search Technology
    • Product Development
    • Research and Development

    Sign up for our weekly newsletter

    Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
    ǁ
    Campus Map
    Harvard Business School Working Knowledge
    Baker Library | Bloomberg Center
    Soldiers Field
    Boston, MA 02163
    Email: Editor-in-Chief
    →Map & Directions
    →More Contact Information
    • Make a Gift
    • Site Map
    • Jobs
    • Harvard University
    • Trademarks
    • Policies
    • Accessibility
    • Digital Accessibility
    Copyright © President & Fellows of Harvard College