Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcast
  • Managing the Future of Work Podcast
  • About Us
  • Book
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Podcasts
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    Design and Analysis of Switchback Experiments
    06 Oct 2020Working Paper Summaries

    Design and Analysis of Switchback Experiments

    by Iavor I. Bojinov, David Simchi-Levi, and Jinglong Zhao
    This paper presents a framework for managers to design and run switchback experiments.
    LinkedIn
    Email

    Author Abstract

    In switchback experiments, a firm sequentially exposes an experimental unit to a random treatment, measures its response, and repeats the procedure for several periods to determine which treatment leads to the best outcome. Although practitioners have widely adopted this experimental design technique, the development of its theoretical properties and the derivation of optimal design procedures have been, to the best of our knowledge, elusive. In this paper, we address these limitations by establishing the necessary results to ensure that practitioners can apply this powerful class of experiments with minimal assumptions. Our main result is the derivation of the optimal design of switchback experiments under a range of different assumptions on the order of carryover effect—that is, the length of time a treatment persists in impacting the outcome. We cast the experimental design problem as a minimax discrete robust optimization problem, identify the worst-case adversarial strategy, establish structural results for the optimal design, and finally solve the problem via a continuous relaxation. For the optimal design, we derive two approaches for performing inference after running the experiment. The first provides exact randomization based p-values and the second uses a finite population central limit theorem to conduct conservative hypothesis tests and build confidence intervals. We further provide theoretical results for our inferential procedures when the order of the carryover effect is misspecified. For firms that possess the capability to run multiple switchback experiments, we also provide a data-driven strategy to identify the likely order of carryover effect. To study the empirical properties of our results, we conduct extensive simulations. We conclude the paper by providing some practical suggestions.

    Paper Information

    • Full Working Paper Text
    • Working Paper Publication Date: September 2020
    • HBS Working Paper Number: HBS Working Paper 21-034
    • Faculty Unit(s): Technology and Operations Management
      Trending
        • 25 Jan 2022
        • Research & Ideas

        More Proof That Money Can Buy Happiness (or a Life with Less Stress)

        • 25 Feb 2019
        • Research & Ideas

        How Gender Stereotypes Kill a Woman’s Self-Confidence

        • 14 Mar 2023
        • In Practice

        What Does the Failure of Silicon Valley Bank Say About the State of Finance?

        • 17 May 2017
        • Research & Ideas

        Minorities Who 'Whiten' Job Resumes Get More Interviews

        • 15 Nov 2022
        • Book

        Stop Ignoring Bad Behavior: 6 Tips for Better Ethics at Work

    Iavor I. Bojinov
    Iavor I. Bojinov
    Assistant Professor of Business Administration
    Richard Hodgson Fellow
    Contact
    Send an email
    → More Articles
    Find Related Articles
    • Research

    Sign up for our weekly newsletter

    Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
    ǁ
    Campus Map
    Harvard Business School Working Knowledge
    Baker Library | Bloomberg Center
    Soldiers Field
    Boston, MA 02163
    Email: Editor-in-Chief
    →Map & Directions
    →More Contact Information
    • Make a Gift
    • Site Map
    • Jobs
    • Harvard University
    • Trademarks
    • Policies
    • Accessibility
    • Digital Accessibility
    Copyright © President & Fellows of Harvard College