04 Oct 2004  Research & Ideas

Start to Measure Your E-commerce Success

After the dot-com fallout, surviving companies needed to sharpen strategy and analyze metrics much better. Visiting professor Marc J. Epstein shows how to put metrics to work.

 

Editor's Note— "The study of e-commerce has unfolded in much the same way that e-commerce thrust itself on the business world—with a great deal of overstatement," says Marc J. Epstein, Visiting Professor and Wyss Visiting Scholar in Social Enterprise at Harvard Business School. "After a long period of treating the topic of e-commerce with awe and confusion, we can now discuss the subject more objectively." What follows is an excerpt from his new book, Implementing E-commerce Strategies: A Guide to Corporate Success after the Dot.Com Bust.

CIOs often suggest that better measures of the payoffs of e-commerce operational and capital investments are necessary to demonstrate the value creation of e-commerce initiatives and to obtain additional resources for critical e-commerce projects. The measures are essential to monitor the key performance drivers (inputs and processes) and assess whether the e-commerce initiative is achieving its stated objectives (outputs) and thus contributing to the long-term success of the corporation (outcomes). Companies often waste resources on e-commerce initiatives or do not invest when they should because they cannot effectively evaluate the potential payoffs of e-commerce investments.

Measuring returns on e-commerce projects can be a daunting challenge. Predicting customer behavior is difficult, because using the Web to do business is still relatively new to many businesses and thus forecasting sales and profits is typically imprecise. There is not much historical data and experience for managers to draw upon when developing or applying metrics, and many economic benefits of e-commerce projects are seen as difficult to measure. Further, the pace of change in e-business and Web-based technologies has been so rapid that precise measurements are often difficult.

Many senior managers have come to believe that further investment in new technology and e-commerce is an imperative that is required to maintain or develop a competitive position. They often make expenditures without completing rigorous analysis. However, today's more stringent economic environment and the widely publicized negative impacts of many e-commerce initiatives has caused many senior managers to question the payoffs of e-commerce investments.6

As companies assess the choice of appropriate measures to evaluate e-commerce initiatives, numerous potential issues arise. Since the choices are different for each company, because the strategies, structures, and systems are different, substantial customization is necessary. Senior managers should consider six initial questions that can lead to the development of appropriate measures for e-commerce operations:

  • What measurement systems are currently in place and being utilized within the organization?
  • What are the important criteria to the company and its constituencies and stakeholders?
  • What does the company desire to accomplish with the e-commerce initiative?
  • What is the anticipated timeframe associated with the e-commerce program?
  • Who are the parties involved in implementing the e-commerce project, and who will be affected by the results?
  • What critical processes are associated with the successful execution of the e-commerce project?

To address these questions, it is imperative that companies not only specifically tailor their e-commerce measurement approach, but also utilize multiple measures to fully analyze their situations. Different measurement criteria are important for companies that have different strategies or may be in a different stage of their life cycle or their e-commerce development. The multiple measures will typically include both financial and non-financial measures that are leading and lagging indicators of performance. They may be used in a balanced scorecard, shareholder value-added, or other approach and can be developed specifically for IT or e-commerce or as a part of an overall corporate performance measurement system. Companies can also use a weighted scoring system to evaluate investments related to overall IT, e-commerce, or business strategy.7

There are many obstacles to implementing a successful measurement system; whether a lack of focus, a low priority, or just difficulty. It is the responsibility of senior managers to evaluate the e-commerce initiatives and decide on the right measures for their organization and ensure that the measures are captured and responded to properly. To obtain adequate resources for e-commerce and to effectively manage e-commerce initiatives, the payoffs of e-commerce investments must be calculated and integrated into management decision-making systems.

Developing appropriate metrics

To closely monitor the cause and effect relationships evidenced in the e-commerce causal linkage model, appropriate metrics must be developed. These metrics must be consistent with and support the objectives and drivers and key success factors already defined. The selected metrics will likely include a combination of input, processes, output, and outcome metrics to effectively measure performance (Exhibit 6.4). Senior managers involved in the e-commerce decision- making process should develop metrics appropriate to the strategy and objectives of the e-commerce initiative, the company, and its stakeholders. During the measure selection process, it is useful for the involved individuals to choose just a few measures, to focus those senior managers involved in the e-commerce initiative on the critical performance indicators.

The list of metrics presented here is not meant to be a comprehensive set of e-commerce performance measures. Rather, it is a selection and example of some metrics that may be appropriate. Managers must select those that most closely fit their strategy and adapt or develop others. There is no rule for the right number of metrics to include in a measurement system; however, including too many tends to distract managers from pursuing a focused strategy. Generally, a complete measurement system includes perhaps three to six measures for each element being evaluated and no more than twenty measures in total.

For each key success factor, a specific target should be identified and results should be measured against these targets. These results should be widely communicated among not only senior managers directly involved in the e-commerce initiative, but also other individuals within the organization upon whom the initiative will have an impact. One of the important contributions that e-commerce can make to an organization is an expanded communication and informational capability. Any e-commerce measurement system will have little impact if the results are not fully discussed. Results should be monitored regularly and used to identify areas of weakness, address the plans and systems in place, and establish new initiatives to improve deficiencies.

It is useful for the involved individuals to choose just a few measures.

The measures chosen should be quantifiable, in either absolute or percentage terms, as well as complete and controllable. They should be complete in that the measure sums up in one number the contribution of all elements of performance that matter; for example, profitability is a summary measure of revenue generation and cost control. They should be controllable in that employees in the organization can actually influence improvement in the factor measured.8

Some of the metrics shown here are evaluations of overall firm performance. Others are indicators of e-commerce performance that are derived through an aggregation of measures of individual business units and functions. It is important to evaluate the performance of both overall e-commerce performance and the specific aspects of e-commerce that lead to revenue enhancement or cost savings to determine the success of various operations and corrective action that can be taken to make improvements.

The measures should be of use to both senior and middle managers in the business units and functions. Thus, they must be disaggregated so each unit can examine its contribution to the achievement of the company's e-commerce strategy. These analyses ensure that each unit is making a contribution to the e-commerce initiative and improving corporate profitability. Additionally, these metrics can be used to provide a gap analysis that permits managers to determine what other inputs or processes are required to meet the company's e-commerce project objectives.

Different tools and techniques are available to measure the different aspects of e-commerce performance. For example, online surveys and polls are powerful tools to help e-commerce enabled companies to better understand the benefit of Internet usage for increasing revenue or decreasing costs related to their customers, thus providing valuable information regarding opportunities to improve overall profitability. Internally, surveys, focus groups, and other techniques are increasingly being used to measure and monitor employee, personnel, and stakeholder reactions and provide valuable feedback.

Once metrics have been developed, data on these indicators must be collected and statistical analysis, such as multiple regression, should be performed to analyze and test the validity of the customized e-commerce measurement system and causal relationships hypothesized by the company. As companies evaluate the initial measurement system's performance, they will typically add some metrics and drop others because of a lack of evidence of a strong relationship. It is here that a final measurement system emerges, and the focus then shifts to applying the model to support improved decision making.

Footnotes:

6. For more information on the measurement of IT and e-commerce investments, see Devaraj and Rajiv Kohli, The IT Payoff; and M. Khosrow-pour, "Does Corporate America Know How to Measure Payoff in Information Technology Investment?" Document Management-Executive Strategies, January 2000.

7. Bob Violino, "Payback Time for E-Business," Internet Week, May 2000, (4).

8. For more on complete and controllable measures, see Marc J. Epstein and Jean-Francois Manzoni, "Implementing Corporate Strategy: From Tableaux de Bord to Balanced Scorecards," European Management Journal, April 1998, 16(2): 190-203.

Processes Metrics
Corporate Strategy
  • Competitive position within industry
  • Cost, development time, delivery time, quantity, price, and channels of products offered
  • Number, complexity and size of competitors, customers, partners, and suppliers
  • Dollars of resources available
Corporate Structure
  • Number of Strategic Business Units (SBUs)
  • Geographic diversity of production and sales
  • Level of empowerment to SBU and functional managers
Corporate Systems
  • Customer and employee satisfaction and retention rate
  • Product and process quality scores
  • Investment in training
Resources
  • Dollars available for e-commerce investment
  • Skills assessment of existing employees
  • Quality assessment of current technology and processes
Processes Metrics
External Environment
  • Assessment of competitor's e-commerce investments
  • Assessment of customer needs
  • Assessment of supplier needs and capabilities
Leadership
  • Time dedicated to e-commerce
  • Budget percent allocated to e-commerce initiatives
  • Performance percentage linked to e-commerce success
  • Objectives for e-commerce initiatives clearly communicated to senior managers and employees
  • Percent of senior managers who are "e-commerce literate"
Create & Execute Appropriate E-Commerce Strategies
  • Number, cost, price, and perception of products and services offered online
  • Availability and planning for e-commerce security features
  • Perception of online brand
  • Amount and quality of information available on the site
  • Quality levels, delivery options, fulfillment rates, and customer satisfaction of orders placed online
  • Profitability of e-commerce operations
  • Quality of e-commerce Web site
Design & Institute Proper E-Commerce Structure
  • Amount of e-commerce that is outsourced
  • Quality of strategic partnerships formed
  • Change in cost and quality of procurements from suppliers
  • E-commerce integration across business units and functions
  • Number of channel-specific products and services
Develop & Implement Appropriate E-Commerce Systems
  • Number and quality of employee e-commerce skills and knowledge
  • Amount and quality of e-commerce training
  • Percentage of performance measures and rewards aligned and linked to e-commerce activities
  • Amount and quality of customer data acquired through e-commerce systems
  • Time required to fill customer orders and service requests made through e-commerce
  • Level of intradepartmental integration by electronic means

  • Quality of e-commerce sales and delivery performance
Outputs Metrics
Channel Optimization
  • Dollar value of activities completed through e-commerce sites
  • Site traffic (number of visits) and functionality of Web site (click-through rate, stickiness)
  • Amount of Web site downtime
  • Partner and supplier satisfaction levels
  • Improvement in cross-selling
Cost Savings
  • Dollars saved in employee expenses, product and material acquisition, data storage, processing, and communication
  • Dollars saved in new product development and introduction
  • Spending saved on customer acquisition and service (online and non-online)
  • Labor costs per unit sold
Customer Acquisition
  • New customers gained through e-commerce
  • Percentage of customers using e-commerce exclusively
  • Percentage of visitors to e-commerce Web site who are visitors versus those who are buyers (reach)
  • Number of new customers in other channels informed through Web site
  • New unique visitors who convert into customers
    (conversion rate)
Customer Loyalty and Retention
  • Frequency of customer return visits to the e-commerce Web site
  • Average yearly sales per e-commerce customer
  • Customer satisfaction with e-commerce activities
  • Customer shopping cart abandonment rates
  • Percentage of customer attrition
  • Ratio of new visitors to repeat visitors
Value Capture
  • Cost and price of products and services offered to customers
  • Average of prices paid by customers
  • Number of new products and service lines introduced
  • Profitability of E-commerce operations
  • Revenues generated through the initiative (total revenue, e-commerce revenue, revenue per e-commerce customer)
  • Customer profitability
Outcome Metrics
Long-term Corporate Profitability
  • Stock price
  • Income growth
  • Sales growth

About the author

Marc J. Epstein is Visiting Professor and Wyss Visiting Scholar in Social Enterprise at Harvard Business School.

Copyright © 2004 by Marc J. Epstein. Implementing E-commerce Strategies: A Guide to Corporate Success after the Dot.Com Bust. Westport, CT: Praeger Publishers, an imprint of Greenwood Publishing Group, Inc. Reprinted by permission.

[ Buy this book ]