Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcasts
  • About Us
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Cold Call Podcast
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    • COVID-19 Business Impact Center
      COVID-19 Business Impact Center
      Design and Analysis of Switchback Experiments
      06 Oct 2020Working Paper Summaries

      Design and Analysis of Switchback Experiments

      by Iavor I. Bojinov, David Simchi-Levi, and Jinglong Zhao
      This paper presents a framework for managers to design and run switchback experiments.
      LinkedIn
      Email

      Author Abstract

      In switchback experiments, a firm sequentially exposes an experimental unit to a random treatment, measures its response, and repeats the procedure for several periods to determine which treatment leads to the best outcome. Although practitioners have widely adopted this experimental design technique, the development of its theoretical properties and the derivation of optimal design procedures have been, to the best of our knowledge, elusive. In this paper, we address these limitations by establishing the necessary results to ensure that practitioners can apply this powerful class of experiments with minimal assumptions. Our main result is the derivation of the optimal design of switchback experiments under a range of different assumptions on the order of carryover effect—that is, the length of time a treatment persists in impacting the outcome. We cast the experimental design problem as a minimax discrete robust optimization problem, identify the worst-case adversarial strategy, establish structural results for the optimal design, and finally solve the problem via a continuous relaxation. For the optimal design, we derive two approaches for performing inference after running the experiment. The first provides exact randomization based p-values and the second uses a finite population central limit theorem to conduct conservative hypothesis tests and build confidence intervals. We further provide theoretical results for our inferential procedures when the order of the carryover effect is misspecified. For firms that possess the capability to run multiple switchback experiments, we also provide a data-driven strategy to identify the likely order of carryover effect. To study the empirical properties of our results, we conduct extensive simulations. We conclude the paper by providing some practical suggestions.

      Paper Information

      • Full Working Paper Text
      • Working Paper Publication Date: September 2020
      • HBS Working Paper Number: HBS Working Paper 21-034
      • Faculty Unit(s): Technology and Operations Management
        Trending
          • 24 Feb 2021
          • Lessons from the Classroom

          What History's Biggest Wars Teach Us About Leading in Peace

          • 25 Feb 2019
          • Research & Ideas

          How Gender Stereotypes Kill a Woman’s Self-Confidence

          • 17 May 2017
          • Research & Ideas

          Minorities Who 'Whiten' Job Resumes Get More Interviews

          • 13 Jul 2020
          • Research & Ideas

          Merck CEO Ken Frazier Discusses a COVID Cure, Racism, and Why Leaders Need to Walk the Talk

          • 17 Aug 2020
          • Research & Ideas

          What the Stockdale Paradox Tells Us About Crisis Leadership

      Iavor I. Bojinov
      Iavor I. Bojinov
      Assistant Professor of Business Administration
      Contact
      Send an email
      → More Articles
      Find Related Articles
      • Research

      Sign up for our weekly newsletter

      Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
      ǁ
      Campus Map
      Harvard Business School Working Knowledge
      Baker Library | Bloomberg Center
      Soldiers Field
      Boston, MA 02163
      Email: Editor-in-Chief
      →Map & Directions
      →More Contact Information
      • Make a Gift
      • Site Map
      • Jobs
      • Harvard University
      • Trademarks
      • Policies
      • Digital Accessibility
      Copyright © President & Fellows of Harvard College