The term captain's log is most known today from the famous beginning segments of the classic TV show Star Trek. The term, however, originated with the first seafaring voyages beyond view of land. While you can determine how far north or south you've progressed using only the stars, you can only measure how far east or west you have traveled with the help of precision timekeeping instruments. In the early days of nautical exploration (and up to the 1700s, when reliable maritime clocks were invented), the captain of a ship traveling across an ocean literally carried a log on board. Several times a day, the captain tied a rope to the log, threw it overboard from the bow of the ship, and counted how long it took to reach the stern. From this timing, the captain calculated a rough speed and then estimated how far the ship had traveled that day. The book he recorded these measurements in took on the name of the instrument used—the captain's log.1
As you can imagine, these measurements were horribly imprecise. In a journey across oceans, the estimate was often wrong by a thousand miles, and countless ships and lives were lost when a reef or another obstruction suddenly appeared days before the captain expected it. Outsourcing measurement in many cases is no more precise in measuring progress toward business goals than the captain's log. Measures, if they even exist, are imprecise and bear little relevance to business outcomes.
How can you measure your progress toward your required business outcomes? The goal is to create a Multisourcing analog of the modern global positioning system (more commonly known as GPS) that can help you with this measurement task. This is the third responsibility of the Multisourcing management office: the creation of standard measurements that illustrate the business value of sourcing actions. These metrics can then be brought together in a Multisourcing dashboard, an integrated measurement technique used to collect, track, and communicate the results of services, and to link these results to business outcomes, thereby illustrating business value. Using the dashboard allows all levels of enterprise management to gauge success according to their own criteria.
The process of measurement (including the continuous improvement that flows out of measurement) is the last piece of Multisourcing, but it is also the beginning. Measurement is an ongoing, cyclical set of activities. The measurement of current services feeds the sourcing strategy with the data needed to make decisions. For managing current providers, then, measurement is your first step, rather than your last.
This would seem to be the simplest and most straightforward part of managing Multisourcing—after all, what is management but measuring progress and adjusting inputs to create desired outputs? Unfortunately, though, we rarely find a client that has mastered the art of measuring its sourcing environment. Measurement processes, when they are even implemented at all, often break down under the following circumstances:
- Many measurement schemes start and end with pricing benchmarks and do not measure quality of service or service level achievement.
- When service levels are measured, many clients measure too much and get lost in the data.
- Service level measurements are often built around what is easy to measure rather than true indicators of service value and performance.
- The measurements do not take into account innovation and continuous improvement.
- The measurements ignore metrics relevant to service consumers.
- The measurements overlook the relationship factors that are a far better indicator of business outcomes.
We'll look in detail at how to set up an effective Multisourcing measurement practice. We'll cover how to determine what to measure and how to create an integrated Multisourcing dashboard that delivers the needed information to all levels of the enterprise and feeds the sourcing strategy process.
The big picture: The governance of evaluation and the design of measurement
The governance of sourcing evaluation . . . encompasses delineating the authority for measurement—how are service providers measured, and how is success and failure determined? One of our key themes, that Multisourcing is built on relationships, means that your governance of measurement needs to specify how to not only measure quantitative performance, but also value measures. In short, evaluation governance must define standards for measuring relationships and develop the approaches and competencies to use measurement to create continuous improvement. The first aspect, though, is the evaluation and measurement process itself.
| The measurement of current services feeds the sourcing strategy with the data needed to make decisions. |
One of the chief failings in many attempts to measure outsourced environments is that they do not take continuous improvement into account; they simply crunch numbers, calculate an incentive or a fee reduction, and move on. Effective sourcing evaluation governance requires defining a four-phase approach that links with several of the comanagement processes. The four phases are define and measure, examine, correct, and guide.
Define and measure
The first phase is the literal process of measurement. In the statements of work you created while selecting, and negotiating with, providers, you should have defined your required service levels and your behavioral expectations of the service providers. Ideally, your contract also includes benchmarking clauses. Now you need to get down to the nitty-gritty—who measures, who benchmarks, who audits, and how often are each of these processes performed? The measurement phase feeds the information required to the equity and the audit and assessment comanagement processes. Measurement takes the guesswork out of management and allows a focus on quantitative measures of performance. This phase is just the beginning, but it is where most governance systems stop.
Examine
The second phase, examine, is not so much about identifying success and failures for the purpose of calculating fee reductions and incentives (although that is part of it), but about identifying the root causes of missed service levels, inadequate relationship processes, or other failures. Who will perform these analyses? How will the results be presented, and to whom? Most often, the provider leads this examination, with users participating in the process. The examine phase supports the audit and assessment, equity, and communication and feedback comanagement processes.
Correct
In this phase, the results of the measure and examine phases are used as the basis for correcting flawed activities and processes. Correction is accomplished by generating, selecting, designing, testing, and implementing improvements. Your process design should include representation from service consumers and service providers as well as any appropriate staff from the Multisourcing management office. The service provider generally documents and implements the corrective action and reports the correction to all participating parties. The correction phase supports the responsibility and integration comanagement processes and provides vital input to future investment decisions.
| Establishing fewer measures that are consistently focused on business impact will yield better results. |
Guide
Following the implementation of corrective actions, the service provider and recipient must work together to continuously guide the activity or process. This helps the parties ensure compliance and maintain the gains achieved through the correction process. The guide phase should include the creation of rules for monitoring service delivery changes, the documentation of results and lessons learned, and the application of lessons to other aspects of service delivery. This phase supports the audit and assessment, communication and feedback, and integration comanagement processes.
Traditional measurement: Service levels, price, and value
Once the four phases of measurement are designed, the next step is to define and begin collecting data on the actual measures for each of the services being measured. Traditionally, these measures have focused on price and service levels (which have most often been completely focused on technical or operational metrics). Because these remain critical components of Multisourcing performance measurement, it is worth spending some time discussing how these measures should be handled.
If you did not define service levels as part of the evaluation and negotiation process, this is, of course, the first place to start. In conjunction with service providers, you will have to define these service levels for any existing relationships that do not have them as yet.
Keep in mind as you begin the measurement process that establishing fewer measures that are consistently focused on business impact will yield better results. Using too many service levels is an all-too-common mistake. Measuring fewer service levels helps you and the service provider focus on key outcomes and therefore improves results. As you review the contracted service levels you have already created or are planning to put in place, keep these rules in mind:
- Having fewer, more meaningful measures gives greater focus and, therefore, greater deal success.
- To be an effective tool to drive service provider behavior, SLAs (service level agreements) must have fee reductions (often called penalties) associated with underperformance.
- Fee reductions must be significant (10 percent to 20 percent of contracted price) to effect behavioral change.
- Focusing fee reductions over fewer measures helps guide the service provider into appropriate behaviors.
Of course, from a business perspective, measuring service levels without reference to the cost of achieving those levels is irrelevant. Start by comparing the cost per service level to your baselines measured during your sourcing strategy process. This is the first step toward gaining insight into business value and the achievement of required outcomes. Have the assumptions and projections of the sourcing business case been achieved? Have costs been reduced? Has value been delivered?
These quantitative comparisons are impossible if you are evaluating an existing service relationship that had neither baselines nor a business case prepared. In these situations, the first priority is to begin measurement immediately. From there, you can start to measure trends and set goals for improvement from these "new" baselines.
This question of value also needs to be put into perspective via external benchmarks. This is why we recommend including a benchmarking clause in your contracts. While it is important to understand the performance of a service provider in relation to baselines, understanding the business value of your sourcing relationships also depends on understanding the market rates for your required service levels. The equity comanagement process depends on this market benchmarking to make determinations of fair compensation and fair value for all the parties involved in the relationship. 