Given the importance of software, the lack of research on the best ways to manage its development is surprising. Many different models have been proposed since the much cited waterfall model emerged more than 30 years ago. Unfortunately, few studies have confirmed empirically the benefits of the newer models. The most widely quoted references report lessons from only a few successful projects.
Now a two-year empirical study, which the author and colleagues Marco Iansiti and Roberto Verganti completed last year, reveals thought-provoking information from the Internet-software industry—an industry in which the need for a responsive development process has never been greater. The researchers analyzed data from 29 completed projects and identified the characteristics most associated with the best outcomes. (See "Four Software-Development Practices That Spell Success.") Successful development was evolutionary in nature. Companies first would release a low-functionality version of a product to selected customers at a very early stage of development. Thereafter work would proceed in an iterative fashion, with the design allowed to evolve in response to the customers' feedback. The approach contrasts with traditional models of software development and their more sequential processes. Although the evolutionary model has been around for several years, this is the first time the connection has been demonstrated between the practices that support the model and the quality of the resulting product.
Research On The Internet-software Industry
Our study of projects in the Internet-software industry asked the question "Does a more evolutionary development process result in better performance?" The study was undertaken in stages. First, the researchers conducted face-to-face interviews with project managers in the industry to understand the types of practices being used. Next, they developed metrics to characterize the type of process adopted in each project. Finally, the metrics were incorporated into a survey that went to a sample of Internet-software companies identified through a review of industry journals. The final sample contained data on 29 projects from 17 companies.
The most remarkable finding was that getting a low-functionality version of the product into customer's hands at the earliest opportunity improves quality dramatically.
To assess the performance of projects in the industry, we examined two outcome measures—one related to the performance of the final product and the other to the productivity achieved in terms of resource consumption (resource productivity). To assess the former, the researchers asked a panel of 14 independent industry experts to rate the comparative quality of each product relative to other products that targeted similar customer needs at the time the act was launched. Product quality was defined as a combination of reliability, technical performance (such as speed) and breadth of functionality. Experts' ratings were gathered using a two-round Delphi (in which information from the first round is given to all experts to help them make their final assessment). To assess the resource productivity of each project, the researchers calculated a measure of the lines of new code developed per person-day and adjusted for differing levels of product complexity. Analysis of the data uncovered four practices critical to success.