Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcast
  • Managing the Future of Work Podcast
  • About Us
  • Book
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Podcasts
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    Scaling Up Behavioral Science Interventions in Online Education
    01 Jul 2020Working Paper Summaries

    Scaling Up Behavioral Science Interventions in Online Education

    by Rene F. Kizilcec, Justin Reich, Michael Yeomans, Christoph Dann, Emma Brunskill, Glenn Lopez, Selen Turkay, Joseph J. Williams, and Dustin Tingley
    Online courses can lack support structures that are often bundled with traditional higher education. Short pre-course interventions can have short-term benefits, but more innovation throughout the course is needed to have sustained impact on student success.
    LinkedIn
    Email

    Author Abstract

    Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over two-and-a-half years, with ¼ million students, from nearly every country, across 247 online courses offered by Harvard, MIT, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically pre-registering new hypotheses in between waves of data collection, we identify individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We find minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Our findings demonstrate how scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order of magnitude and the promise of iterative scientific investigations to uncover what works where for whom.

    Paper Information

    • Full Working Paper Text
    • Working Paper Publication Date: June 2020
    • HBS Working Paper Number: Proceedings of the National Academy of Sciences of the United States of America
    • Faculty Unit(s): Negotiation, Organizations & Markets
      Trending
        • 17 Jan 2023
        • In Practice

        8 Trends to Watch in 2023

        • 24 Jan 2023
        • Research & Ideas

        Passion at Work Is a Good Thing—But Only If Bosses Know How to Manage It

        • 10 Jan 2023
        • Research & Ideas

        How to Live Happier in 2023: Diversify Your Social Circle

        • 25 Jan 2022
        • Research & Ideas

        More Proof That Money Can Buy Happiness (or a Life with Less Stress)

        • 25 Feb 2019
        • Research & Ideas

        How Gender Stereotypes Kill a Woman’s Self-Confidence

    Find Related Articles
    • Online Technology
    • Education
    • Behavior
    • Performance Improvement
    • Education

    Sign up for our weekly newsletter

    Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
    ǁ
    Campus Map
    Harvard Business School Working Knowledge
    Baker Library | Bloomberg Center
    Soldiers Field
    Boston, MA 02163
    Email: Editor-in-Chief
    →Map & Directions
    →More Contact Information
    • Make a Gift
    • Site Map
    • Jobs
    • Harvard University
    • Trademarks
    • Policies
    • Accessibility
    • Digital Accessibility
    Copyright © President & Fellows of Harvard College