In a perfect world, scientists share problems and work together on solutions for the good of society. In the real world, however, that's usually not the case. The main obstacles: competition for publication and intellectual property protection.
Is there a model for encouraging large-scale scientific problem solving? Yes, and it comes from an unexpected and unrelated corner of the universe: open source software development.
That's the view of Karim R. Lakhani, an assistant professor at Harvard Business School with an extensive research background in open source software communities and their innovation and product development strategies. His latest research analyzes how open source norms of transparency, permeable access, and collaboration might work with scientists.
What he and his coauthors discovered: "broadcasting" or introducing problems to outsiders yields effective solutions. Indeed, it was outsiders—those with expertise at the periphery of a problem's field—who were most likely to find answers and do so quickly.
People often think about open source as a special case, as if such openness can only happen in software.
The study and its findings are described in his paper "The Value of Openness in Scientific Problem Solving," coauthored with Lars Bo Jeppesen, Peter A. Lohse, and Jill A. Panetta. It describes how broadcast search was used with 166 distinct scientific problems from the research laboratories of twenty-six firms from ten countries over a four-and-a-half year period. Problems involved everything from biotech to consumer products and agrochemicals.
Thanks to broadcasting, nearly one-third of the previously unsolved problems found successful solutions.
"Innovations happen at the intersection of disciplines. People have talked about that a lot and I think we're providing some systematic evidence now with this study," Lakhani says.
He met recently with HBS Working Knowledge to discuss the work and its implications for firms.
Martha Lagace: Given your research background in open source software communities, how did you become interested in the world of scientific problem solving?
Karim R. Lakhani: Open source collaboration is a very different model for innovation and product development than most firms are used to. I began to wonder where we might see similar patterns occur outside the software domain. In open source communities we see a vast degree of openness in which everybody can participate, but also the practice of broadcasting your work to everybody else. People continually broadcast their problems, others broadcast solutions, and the person with the problem is not always the one with the solution. Oftentimes, somebody else can make sense of both what the problem has been and what people are proposing as solutions, and can come up with a better answer.
I also read a book by Dava Sobel about the longitude prize [Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of his Time]. Finding longitude at sea was one of the toughest economic, scientific, and technological problems up until the eighteenth century. Isaac Newton said the only way to solve the problem was through astronomical methods, but he was proven wrong because someone from rural Yorkshire, England, came up with a clock that could keep time at sea. Nobody had anticipated that that kind of invention was practical.
Most firms and scientists believe that the problems they work on are their most important things.
The longitude prize was set up to allow other people access to the problem and create incentives for them to work on a solution. And when I was thinking systematically about where else I could look, I discovered a company, InnoCentive.com, that took problems in R&D labs and broadcast them to outsiders. So the study was a direct port of both the longitude-prize method but also what I see in open source.
People often think about open source as a special case as if such openness can only happen in software, and this is an attempt to work on generalizing what we see.
Q: What is different about problem solving in the open source and science communities?
A: Open source software developers are very pragmatic and focused on solving problems. Scientists are focused on problems too, but their priority is often publication and that can sometimes come in the way of openness and sharing. The ideals of science are, of course, openness, sharing, and no restrictions on the free flow of knowledge, but in practice that doesn't happen much at all. Some scientists, however, are pushing back and many say they need to rethink how they conduct science.
Q: What are the risks of opening problems to outsiders?
A: For firms, the first order risk is the loss of intellectual property, especially if you think about the fact that most firms and scientists believe that the problems they work on are actually their most important things. If you provide hints to competitors, it will reveal a lot of your strategy.
I think it's a legitimate concern, although practice doesn't prove that out in the sense that even if other people know about the problems you're working on or have seen your solutions, it's very hard to implement those solutions in other settings. Knowledge is actually very sticky. Even if you reveal everything about what's going on, there's tacit knowledge behind a lot of scientific and technological activities.
And the benefit of opening up your problems to outsiders is that in fact you can get novel solutions—quicker solutions than what the firm or R&D lab might develop. It also opens up new domains for the pursuit of knowledge and activities.
But it's still a very counterintuitive way of working.
Q: I found it amazing in your research that outsiders were most likely to find a solution.
A: Yes. The problem may reside in one domain of expertise and the solution may reside in another. I've done interviews with scientists who participated by posting problems for broadcast, and most of these scientists were highly skeptical about this method because they considered themselves to be at the top of their discipline. However, they had never thought about the possibility of scientists in other disciplines looking at their problem, reconceptualizing it, and coming up with a solution that could be off-the-shelf. So when they actually see solutions from this type of method, they're blown away.
Recently, an internal science team at a U.S.-based major biotechnology firm was assigned to develop a method for rapid and simple detection of DNA sequences in unconventional field settings. Their mission was to produce a DNA sequencing test that was cost effective, robust, and operable in extreme field conditions. After several months of effort, the team in consultation with company experts concluded that no viable solution existed for their problem. Since solving the problem was of critical importance for the firm, management decided to break from convention and the disclose the specifics of the problem to a large group of unknown "outside" scientists requesting a solution in return for substantial prize money.
Most people take knowledge and information from their back pockets and transfer it to the problem at hand.
In a four-week period of time, over 574 scientists investigated the problem statement and forty-two of them submitted potential solutions for considerations. The winning solution was proposed by a scientist from Finland who did not work in this field. The solution involved the novel application of an existing methodology to the problem at hand. Besides solving the problem, the solution information opened up a new knowledge domain for future investigations and resulted in a valuable patent for the firm. This case example illustrates how the disclosure and wide dispersion of scientific problem information allows R&D organizations to access a broader range of scientific knowledge, and can help accelerate internal and proprietary research efforts.
Q: What is another example or two from your study of the kinds of problems that were posted?
A: Innovations happen at the intersection of disciplines. People have talked about that a lot and I think we're providing some systematic evidence now with this study. Another example is that a pharmaceutical company got unusual toxicology results for an ongoing drug study. The best toxicologists within the firm had a look at the results and couldn't understand them. Their external academic consultants, also toxicologists, also failed to interpret the results. Then they finally posted it onto InnoCentive. A protein crystallographer looked at the problem and basically gave an off-the-shelf solution. The pharmaceutical company had never viewed the problem as a crystallography problem; they thought it was a toxicology problem. Again, this opened up a whole new domain for the firm to pursue in terms of future studies as to how to think about the types of problems they face.
We see this in many different places. The insight is that what you want to do is open up your problem to other people—not just to serendipity, but in some systematic way.
What we don't know is whether some firms may be large enough by themselves to already have the requisite variety and heterogeneity inside the firm. Could they first start by broadcasting problems inside?
There are always issues around managerial incentives, silos, and so forth, but certainly by the way we see open source communities and InnoCentive work, in fact, by broadcasting a problem you can actually attract a lot of people. And what's also important to note is that the problem solving being done is not "We'll spend five years coming up with a solution." Most people take knowledge and information from their back pockets and transfer it to the problem at hand. In our study the average time spent by successful solvers was two weeks, so that's fairly little in the scheme of things.
Q: What motivated potential solvers to participate?
A: Our findings about motivations are consistent with other distributed innovation communities. The big question in all of these settings is why people would participate given that there's no guaranteed outcome. In the InnoCentive case, there might at least be the promise of a reward: a cash award. Open source has none of that: It's based on sharing.
Q: In open source, the solvers' names become known, right?
A: Yes, so reputation matters, use value matters, and so forth. I have conducted a study about motivations in open source communities. People talked about feeling part of a community and being intellectually challenged. In fact, the strongest correlate between time spent on a project and other factors was how a person assessed the creativity in the project. One of the questions we asked open source people was, "Imagine a time in your life when you felt the most creative, and rate this project's creativity with that experience." The more creative they felt in those projects, the more likely they were to participate and spend time, which makes sense. So fun and the enjoyment of problem solving fit together in open source.
In our study of scientific problem solving, most people were not going to win yet still participated. We saw that, on average, 240 people seriously looked at each problem statement and then, on average, ten submitted solutions. When you look at the ten people who submitted solutions and see that they expended anywhere from thirty-five to seventy-five hours' worth of time, what motivates them is a critical question. We've found that the population was divided into two sets of folks: those motivated by money who wanted to win the challenge, and those who enjoyed the problem-solving experience in itself. They found it to be stimulating and fun, and both of those were strong indicators. Enjoyment and the challenge of learning was the strongest correlate of being a successful solver. But money was also important and it was a significant correlate.
Q: The solutions were not posted. Did you consider the pros and cons of posting them?
A: Absolutely. I think that's one of the bigger issues, because the firms that participated were concerned about IP so they didn't feel they could open up the solution process and expose the solution to others. Some other research has shown that, in fact, if you do open up the solution process you can get anywhere from 10X to 100X improvement in problem-solving performance. The ideal process would be to keep it open and get other people to comment on the solutions and perhaps even refine them more.
There's a growing movement and establishment of nonprofit foundations that are participating in drug discovery efforts. These foundations aren't as concerned about intellectual property as they are about finding a cure or treatment for a disease, and it's likely that they might open up solutions as well.
When you talk with lawyers, most of them say, "Protect, protect, protect, close, close, close," but there are some very innovative licensing schemes and innovative ways by which you can allow others to peek into your process and not give up the entire keys to the kingdom. I think we just need to find innovative licensing ways or legal regimes that allow people to share knowledge without risking the overall intellectual property of the firm. Unfortunately, the models right now are "Close, close, close" or "Open all the way." I definitely think that if they could take a best solution and put it back out there for review and comment, they might actually go down a pathway of improvement that they didn't anticipate.
Q: What research is next for you?
A: My hope is to systematically examine the application of open source principles to settings outside of software and to also track the real performance of these methods to concrete problem-solving situations. I am also interested in studying the intersection of firms and communities and the emergence of hybrid models of organizations that blend and blur firms and communities.
I am planning to run experiments where we take the same problem and broadcast it inside and outside a firm to see what the differences are. Some firms may be large enough to benefit from that. So I'm trying to identify firms that might be willing to take a substantial number of problems and broadcast them inside and outside.
Another study looks at the interaction between intellectual property constraints and collaboration constraints. MathWorks, which makes MATLAB software, has been running a fun "wiki-like" programming contest where anyone can look at anyone else's submission and then resubmit it as their own; and through this competitive collaboration MathWorks has found improvements between 10X and 100X in programming performance. I'll be working with them on some experiments to see what happens to performance when there are constraints on collaboration and the introduction of IP rights.
Last, we've seen open source communities evolve and now open source nonprofit foundations are setting up their own for-profit companies to compete with traditional software companies. A key question for them is "How can you to channel volunteers around the world to work with you and still be a money-making company?" So I'm investigating these hybrid models and trying to understand how firms and communities can jointly benefit from these new models of collaboration. See Karim R. Lakhani's blog: http://spoudaiospaizen.net/