19 Mar 2001  Research & Ideas

The Essentials for Enlightened Experimentation

In the past, the high cost of experimentation has greatly impacted many firms' ability to successfully innovate. But now, new technologies are enabling reinvention of R&D from the ground up. HBS associate professor Stefan Thomke explains.


New technologies such as computer simulations not only make experimentation faster and cheaper, they also enable companies to be more innovative. But achieving that requires a thorough understanding of the link between experimentation and learning. Briefly stated, innovation requires the right R&D systems for performing experiments that will generate the information needed to develop and refine products quickly. The challenges are managerial as well as technical:

Organize for rapid experimentation

  • Examine and, if necessary, revamp entrenched routines, organizational boundaries, and incentives to encourage rapid experimentation.
  • Consider using small development groups that contain key people (designers, test engineers, manufacturing engineers) with all the knowledge required to iterate rapidly.
  • Determine what experiments can be performed in parallel instead of sequentially. Parallel experiments are most effective when time matters most, cost is not an overriding factor, and developers expect to learn little that would guide them in planning the next round of experiments.

Fail early and often, but avoid mistakes

  • Embrace failures that occur early in the development process and advance knowledge significantly.
  • Don't forget the basics of experimentation. Well-designed tests have clear objectives (what do you anticipate learning?) and hypotheses (what do you expect to happen?). Also, mistakes often occur when you don't control variables that could diminish your ability to learn from the experiments. When variability can't be controlled, allow for multiple, repeated trials.

Anticipate and exploit early information

  • Recognize the full value of front-loading: identifying problems upstream, where they are easier and cheaper to solve.
  • Acknowledge the trade-off between cost and fidelity. Experiments of lower fidelity (generally costing less) are best suited in the early exploratory stages of developing a product. High-fidelity experiments (typically more expensive) are best suited later to verify the product.

Combine new and traditional technologies

Do not assume that a new technology will necessarily replace an established one. Usually, new and traditional technologies are best used in concert.

  • Remember that new technologies emerge and evolve continually. Today's new technology might eventually replace its traditional counterpart, but it could then be challenged by tomorrow's new technology.

Harvard Business Review's Executive Summary of"Enlightened Experimentation: The New Imperative for Innovation"

by Stefan Thomke

The high cost of experimentation has long put a damper on companies' attempts to create great new products. But new technologies are making it easier than ever to conduct complex experiments quickly and cheaply. Companies can now take innovation to a whole new level, contends Stefan Thomke, if they're willing to rethink their R&D from the ground up.

Understanding what enlightened experimentation is all about requires an appreciation of the innovation process. All development organizations need a system of experimentation in place to help them decide which ideas to pursue. Of course, the more rapid and efficient the system is, the quicker researchers can find solutions. Many companies today, however, mistakenly view new technologies solely in terms of cost cutting. They overlook the fact that some technologies can introduce entirely new ways of discovering novel concepts and solutions.

Thomke argues that new technologies affect everything, from the development process itself — including the way an R&D organization is structured — to how new knowledge is created. So companies that are trying to be more innovative face both managerial and technical challenges. Drawing on his research in the pharmaceutical, automotive, and software industries, Thomke introduces the following four rules for enlightened experimentation: organize for rapid experimentation; fail early and often, but avoid mistakes; anticipate and exploit early information; and combine new and old technologies.

The article uses real-world examples to explain each rule in detail. It also suggests how this system of experimentation will affect other industries and examines the implications for knowledge workers.

Excerpted from the article "Enlightened Experimentation: The New Imperative for Innovation" in the Harvard Business Review, Febraury 2001.

[ Order the full article ]