The Essentials for Enlightened Experimentation

In the past, the high cost of experimentation has greatly impacted many firms' ability to successfully innovate. But now, new technologies are enabling reinvention of R&D from the ground up. HBS associate professor Stefan Thomke explains.
by Stefan Thomke

New technologies such as computer simulations not only make experimentation faster and cheaper, they also enable companies to be more innovative. But achieving that requires a thorough understanding of the link between experimentation and learning. Briefly stated, innovation requires the right R&D systems for performing experiments that will generate the information needed to develop and refine products quickly. The challenges are managerial as well as technical:

Organize For Rapid Experimentation

  • Examine and, if necessary, revamp entrenched routines, organizational boundaries, and incentives to encourage rapid experimentation.
  • Consider using small development groups that contain key people (designers, test engineers, manufacturing engineers) with all the knowledge required to iterate rapidly.
  • Determine what experiments can be performed in parallel instead of sequentially. Parallel experiments are most effective when time matters most, cost is not an overriding factor, and developers expect to learn little that would guide them in planning the next round of experiments.

Fail Early And Often, But Avoid Mistakes

  • Embrace failures that occur early in the development process and advance knowledge significantly.
  • Don't forget the basics of experimentation. Well-designed tests have clear objectives (what do you anticipate learning?) and hypotheses (what do you expect to happen?). Also, mistakes often occur when you don't control variables that could diminish your ability to learn from the experiments. When variability can't be controlled, allow for multiple, repeated trials.

Anticipate And Exploit Early Information

  • Recognize the full value of front-loading: identifying problems upstream, where they are easier and cheaper to solve.
  • Acknowledge the trade-off between cost and fidelity. Experiments of lower fidelity (generally costing less) are best suited in the early exploratory stages of developing a product. High-fidelity experiments (typically more expensive) are best suited later to verify the product.

Combine New And Traditional Technologies

Do not assume that a new technology will necessarily replace an established one. Usually, new and traditional technologies are best used in concert.

  • Remember that new technologies emerge and evolve continually. Today's new technology might eventually replace its traditional counterpart, but it could then be challenged by tomorrow's new technology.

About the Author

Stefan Thomke is an associate professor at Harvard Business School.