Start to Measure Your E-commerce Success

After the dot-com fallout, surviving companies needed to sharpen strategy and analyze metrics much better. Visiting professor Marc J. Epstein shows how to put metrics to work.
by Marc J. Epstein

"The study of e-commerce has unfolded in much the same way that e-commerce thrust itself on the business world—with a great deal of overstatement," says Marc J. Epstein, Visiting Professor and Wyss Visiting Scholar in Social Enterprise at Harvard Business School. "After a long period of treating the topic of e-commerce with awe and confusion, we can now discuss the subject more objectively." What follows is an excerpt from his new book, Implementing E-commerce Strategies: A Guide to Corporate Success after the Dot.Com Bust.

CIOs often suggest that better measures of the payoffs of e-commerce operational and capital investments are necessary to demonstrate the value creation of e-commerce initiatives and to obtain additional resources for critical e-commerce projects. The measures are essential to monitor the key performance drivers (inputs and processes) and assess whether the e-commerce initiative is achieving its stated objectives (outputs) and thus contributing to the long-term success of the corporation (outcomes). Companies often waste resources on e-commerce initiatives or do not invest when they should because they cannot effectively evaluate the potential payoffs of e-commerce investments.

Measuring returns on e-commerce projects can be a daunting challenge. Predicting customer behavior is difficult, because using the Web to do business is still relatively new to many businesses and thus forecasting sales and profits is typically imprecise. There is not much historical data and experience for managers to draw upon when developing or applying metrics, and many economic benefits of e-commerce projects are seen as difficult to measure. Further, the pace of change in e-business and Web-based technologies has been so rapid that precise measurements are often difficult.

Many senior managers have come to believe that further investment in new technology and e-commerce is an imperative that is required to maintain or develop a competitive position. They often make expenditures without completing rigorous analysis. However, today's more stringent economic environment and the widely publicized negative impacts of many e-commerce initiatives has caused many senior managers to question the payoffs of e-commerce investments.

As companies assess the choice of appropriate measures to evaluate e-commerce initiatives, numerous potential issues arise. Since the choices are different for each company, because the strategies, structures, and systems are different, substantial customization is necessary. Senior managers should consider six initial questions that can lead to the development of appropriate measures for e-commerce operations:

  • What measurement systems are currently in place and being utilized within the organization?
  • What are the important criteria to the company and its constituencies and stakeholders?
  • What does the company desire to accomplish with the e-commerce initiative?
  • What is the anticipated timeframe associated with the e-commerce program?
  • Who are the parties involved in implementing the e-commerce project, and who will be affected by the results?
  • What critical processes are associated with the successful execution of the e-commerce project?

To address these questions, it is imperative that companies not only specifically tailor their e-commerce measurement approach, but also utilize multiple measures to fully analyze their situations. Different measurement criteria are important for companies that have different strategies or may be in a different stage of their life cycle or their e-commerce development. The multiple measures will typically include both financial and non-financial measures that are leading and lagging indicators of performance. They may be used in a balanced scorecard, shareholder value-added, or other approach and can be developed specifically for IT or e-commerce or as a part of an overall corporate performance measurement system. Companies can also use a weighted scoring system to evaluate investments related to overall IT, e-commerce, or business strategy.

There are many obstacles to implementing a successful measurement system; whether a lack of focus, a low priority, or just difficulty. It is the responsibility of senior managers to evaluate the e-commerce initiatives and decide on the right measures for their organization and ensure that the measures are captured and responded to properly. To obtain adequate resources for e-commerce and to effectively manage e-commerce initiatives, the payoffs of e-commerce investments must be calculated and integrated into management decision-making systems.

Developing Appropriate Metrics

To closely monitor the cause and effect relationships evidenced in the e-commerce causal linkage model, appropriate metrics must be developed. These metrics must be consistent with and support the objectives and drivers and key success factors already defined. The selected metrics will likely include a combination of input, processes, output, and outcome metrics to effectively measure performance. Senior managers involved in the e-commerce decision- making process should develop metrics appropriate to the strategy and objectives of the e-commerce initiative, the company, and its stakeholders. During the measure selection process, it is useful for the involved individuals to choose just a few measures, to focus those senior managers involved in the e-commerce initiative on the critical performance indicators.

The list of metrics presented here is not meant to be a comprehensive set of e-commerce performance measures. Rather, it is a selection and example of some metrics that may be appropriate. Managers must select those that most closely fit their strategy and adapt or develop others. There is no rule for the right number of metrics to include in a measurement system; however, including too many tends to distract managers from pursuing a focused strategy. Generally, a complete measurement system includes perhaps three to six measures for each element being evaluated and no more than twenty measures in total.

For each key success factor, a specific target should be identified and results should be measured against these targets. These results should be widely communicated among not only senior managers directly involved in the e-commerce initiative, but also other individuals within the organization upon whom the initiative will have an impact. One of the important contributions that e-commerce can make to an organization is an expanded communication and informational capability. Any e-commerce measurement system will have little impact if the results are not fully discussed. Results should be monitored regularly and used to identify areas of weakness, address the plans and systems in place, and establish new initiatives to improve deficiencies.

It is useful for the involved individuals to choose just a few measures.

The measures chosen should be quantifiable, in either absolute or percentage terms, as well as complete and controllable. They should be complete in that the measure sums up in one number the contribution of all elements of performance that matter; for example, profitability is a summary measure of revenue generation and cost control. They should be controllable in that employees in the organization can actually influence improvement in the factor measured.

Some of the metrics shown here are evaluations of overall firm performance. Others are indicators of e-commerce performance that are derived through an aggregation of measures of individual business units and functions. It is important to evaluate the performance of both overall e-commerce performance and the specific aspects of e-commerce that lead to revenue enhancement or cost savings to determine the success of various operations and corrective action that can be taken to make improvements.

The measures should be of use to both senior and middle managers in the business units and functions. Thus, they must be disaggregated so each unit can examine its contribution to the achievement of the company's e-commerce strategy. These analyses ensure that each unit is making a contribution to the e-commerce initiative and improving corporate profitability. Additionally, these metrics can be used to provide a gap analysis that permits managers to determine what other inputs or processes are required to meet the company's e-commerce project objectives.

Different tools and techniques are available to measure the different aspects of e-commerce performance. For example, online surveys and polls are powerful tools to help e-commerce enabled companies to better understand the benefit of Internet usage for increasing revenue or decreasing costs related to their customers, thus providing valuable information regarding opportunities to improve overall profitability. Internally, surveys, focus groups, and other techniques are increasingly being used to measure and monitor employee, personnel, and stakeholder reactions and provide valuable feedback.

Once metrics have been developed, data on these indicators must be collected and statistical analysis, such as multiple regression, should be performed to analyze and test the validity of the customized e-commerce measurement system and causal relationships hypothesized by the company. As companies evaluate the initial measurement system's performance, they will typically add some metrics and drop others because of a lack of evidence of a strong relationship. It is here that a final measurement system emerges, and the focus then shifts to applying the model to support improved decision making.

About the Author

Marc J. Epstein is Visiting Professor and Wyss Visiting Scholar in Social Enterprise at Harvard Business School.