Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Working Knowledge
Business Research for Business Leaders
  • Browse All Articles
  • Popular Articles
  • Cold Call Podcast
  • Managing the Future of Work Podcast
  • About Us
  • Book
  • Leadership
  • Marketing
  • Finance
  • Management
  • Entrepreneurship
  • All Topics...
  • Topics
    • COVID-19
    • Entrepreneurship
    • Finance
    • Gender
    • Globalization
    • Leadership
    • Management
    • Negotiation
    • Social Enterprise
    • Strategy
  • Sections
    • Book
    • Podcasts
    • HBS Case
    • In Practice
    • Lessons from the Classroom
    • Op-Ed
    • Research & Ideas
    • Research Event
    • Sharpening Your Skills
    • What Do You Think?
    • Working Paper Summaries
  • Browse All
    Judgment Aggregation in Creative Production: Evidence from the Movie Industry
    27 Feb 2019Working Paper Summaries

    Judgment Aggregation in Creative Production: Evidence from the Movie Industry

    by Hong Luo, Jeffrey T. Macher, and Michael Wahlen
    Selecting early-stage ideas in creative industries is challenging because consumer taste is hard to predict and the quantity to sift through is large. Using The Black List that ranks scripts annually based on nominations from film executives, this study shows that aggregating expert opinions helps reduce quality uncertainty and can influence high-budget production.
    LinkedIn
    Email

    Author Abstract

    This paper studies judgment aggregation from a large number of industry experts in the selection and development of early-stage ideas. Our context is the movie industry, in which customer appeal is difficult to predict and investment costs are high. The Black List, an annual publication, ranks unproduced scripts based on anonymous nominations from film executives. We find that listed scripts are twice as likely to be released as unlisted scripts and, conditional on release, generate a higher return on investment. Further evidence supports a causal interpretation of these differences in outcomes and suggests that by aggregating expert judgment, the List is able to discern good ideas from poor ones. We also find that scripts from less-experienced writers are more likely to be listed and to rank higher if listed. Yet in terms of release probability, even though being listed has a large and positive effect on less-experienced writers, the discrepancy relative to experienced writers remains large (even for the top-ranked scripts). These results on writer experience suggest a simple, albeit important, determinant of the data-generating process of such aggregations—the visibility of an idea among potential evaluators.

    Paper Information

    • Full Working Paper Text
    • Working Paper Publication Date: January 2019
    • HBS Working Paper Number: HBS Working Paper #19-082
    • Faculty Unit(s): Strategy
      Trending
        • 31 Jan 2023
        • Research & Ideas

        It’s Not All About Pay: College Grads Want Jobs That ‘Change the World’

        • 25 Jan 2022
        • Research & Ideas

        More Proof That Money Can Buy Happiness (or a Life with Less Stress)

        • 17 Jan 2023
        • In Practice

        8 Trends to Watch in 2023

        • 25 Feb 2019
        • Research & Ideas

        How Gender Stereotypes Kill a Woman’s Self-Confidence

        • 27 Jan 2023
        • Op-Ed

        Have We Lost Sight of Integrity?

    Hong Luo
    Hong Luo
    James Dinan and Elizabeth Miller Associate Professor of Business Administration
    Contact
    Send an email
    → More Articles
    Find Related Articles
    • Creativity
    • Decision Choices and Conditions
    • Forecasting and Prediction
    • Film Entertainment
    • Innovation and Management
    • Motion Pictures and Video

    Sign up for our weekly newsletter

    Interested in improving your business? Learn about fresh research and ideas from Harvard Business School faculty.
    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
    ǁ
    Campus Map
    Harvard Business School Working Knowledge
    Baker Library | Bloomberg Center
    Soldiers Field
    Boston, MA 02163
    Email: Editor-in-Chief
    →Map & Directions
    →More Contact Information
    • Make a Gift
    • Site Map
    • Jobs
    • Harvard University
    • Trademarks
    • Policies
    • Accessibility
    • Digital Accessibility
    Copyright © President & Fellows of Harvard College