18 Jul 2013Working Paper Summaries

Cumulative Innovation & Open Disclosure of Intermediate Results: Evidence from a Policy Experiment in Bioinformatics

by Kevin J. Boudreau and Karim Lakhani

The practice of opening intermediate works such as early results, algorithms, materials, data, and techniques—and disclosing and granting access to them for reuse by others—has been observed in many areas of innovation. In this paper, Kevin J. Boudreau and Karim Lakhani devise an experimental approach in order to investigate effects of an open regime on a challenging problem in bioinformatics that was amenable to cumulative innovation. The authors compared outcomes in this open regime with those in a closed regime in which no solutions were disclosed until the end of the experiment. Results suggest important trade-offs related to incentives, participation, and learning. For example, freer disclosures coincided with drops in participation and development activity, consistent with longstanding theories of economic incentives to make investments in innovation. Particularly striking is the magnitude of drops in incentives and participation. Even so, positive effects on learning outweighed the negative effects on incentives. Overall, the study contributes to analysis of the effect of supporting institutions on cumulative innovation. It also raises important questions for policy makers responsible for innovation. Key concepts include:

Open disclosures led to lower participation and lower effort but nonetheless led to higher average problem-solving performance.
Closed secrecy produced higher participation and higher effort, while producing less correlated choices of technical approaches that participants pursued. This resulted in greater individual and collective experimentation and greater dispersion of performance.
The design of policies that enable both investment in innovation and disclosure to others will be increasingly important for economic growth.

Author Abstract

Recent calls for greater openness in our private and public innovation systems have particularly urged for more open disclosure and granting of access to intermediate works-early results, algorithms, materials, data and techniques-with the goals of enhancing overall research and development productivity and enhancing cumulative innovation. To make progress towards understanding implications of such policy changes we devised a large-scale field experiment in which 733 subjects were divided into matched independent subgroups to address a bioinformatics problem under either a regime of open disclosure of intermediate results or, alternatively, one of closed secrecy around intermediate solutions. We observe the cumulative innovation process in each regime with fine-grained measures and are able to derive inferences with a series of cross-sectional comparisons. Open disclosures led to lower participation and lower effort but nonetheless led to higher average problem-solving performance by concentrating these lesser efforts on the most performant technical approaches. Closed secrecy produced higher participation and higher effort, while producing less correlated choices of technical approaches that participants pursued, resulting in greater individual and collective experimentation and greater dispersion of performance. We discuss the implications of such changes to the ongoing theory, evidence, and policy considerations with regards to cumulative innovation.