Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcast
  • Managing the Future of Work Podcast
  • About Us
  • Book
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Podcasts
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
  • Browse All
    The Effects of Hierarchy on Learning and Performance in Business Experimentation
    23 Mar 2020Working Paper Summaries

    The Effects of Hierarchy on Learning and Performance in Business Experimentation

    by Sourobh Ghosh, Stefan H. Thomke, and Hazjier Pourkhalkhali
    Do senior managers help or hurt business experiments? Analyzing a dataset of more than 6,300 experiments on the A/B/n testing platform Optimizely, this study suggests that involving senior executives in experimentation teams can have surprising consequences.
    LinkedIn
    Email

    Author Abstract

    Do senior managers help or hurt business experiments? Despite the widespread adoption of business experiments to guide strategic decision-making, we lack a scholarly understanding of what role senior managers play in firm experimentation. Using proprietary data of live business experiments from the widely-used A/B testing platform, Optimizely, this paper estimates the association of management hierarchy with learning from experiments and their performance outcomes across industries and contexts. Our findings suggest that senior management’s association is mixed. On the one hand, senior managers’ involvement associates with bolder experiments that create more statistically significant learning signals aiding in the exploration of new strategic directions. On the other hand, their involvement associates with less cause-and-effect learning that is instrumental to optimization and performance improvements. Our results contribute to a burgeoning literature on experimentation in strategy, while helping articulate limits that organizational design might place on data-driven decision-making. Furthermore, we describe different experimental learning modes in the formation of strategy, offering important implications for how managers can modulate search and performance outcomes.

    Paper Information

    • Full Working Paper Text
    • Working Paper Publication Date: February 2020
    • HBS Working Paper Number: HBS Working Paper #20-081
    • Faculty Unit(s): Technology and Operations Management
      Trending
        • 23 May 2023
        • Research & Ideas

        Face Value: Do Certain Physical Features Help People Get Ahead?

        • 30 May 2023
        • Research & Ideas

        Can AI Predict Whether Shoppers Would Pick Crest or Colgate?

        • 02 May 2023
        • What Do You Think?

        How Should Artificial Intelligence Be Regulated—if at All?

        • 31 May 2023
        • HBS Case

        From Prison Cell to Nike’s C-Suite: The Journey of Larry Miller

        • 25 Jan 2022
        • Research & Ideas

        More Proof That Money Can Buy Happiness (or a Life with Less Stress)

    Stefan H. Thomke
    Stefan H. Thomke
    William Barclay Harding Professor of Business Administration
    Chair, General Management Program
    Contact
    Send an email
    → More Articles
    Find Related Articles
    • Innovation and Invention
    • Organizational Design
    • Search Technology
    • Product Development
    • Research and Development

    Sign up for our weekly newsletter

    Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
    ǁ
    Campus Map
    Harvard Business School Working Knowledge
    Baker Library | Bloomberg Center
    Soldiers Field
    Boston, MA 02163
    Email: Editor-in-Chief
    →Map & Directions
    →More Contact Information
    • Make a Gift
    • Site Map
    • Jobs
    • Harvard University
    • Trademarks
    • Policies
    • Accessibility
    • Digital Accessibility
    Copyright © President & Fellows of Harvard College