Managers of Maines Paper & Food Service were stumped. Their benchmarking data showed what appeared to be significant performance differences between two company locations. Each site was handling very similar work—in fact, they both were servicing the same client—but figures associated with cost and profitability were dramatically different.
"One unit appeared to be markedly underperforming," recalls William A. Mastrosimone, chief financial officer of the Conklin, NY, company. "We puzzled over it for quite some time."
The puzzle was solved when Mastrosimone and his colleagues uncovered a factor not considered in the benchmarking effort: the age of the two facilities. One had been operating for twenty years, the other for only three—and both units had quite a few staff who had joined the company when their location opened. The older facility had a significant share of twenty-year employees, whose pay "had hit the top of the grid," Mastrosimone says, while the newer location had only a handful of employees with more than a few years' employment. That skewed many unit costs, which took into account personnel expenses.
When the benchmarking effort was recast to include indicators of the value-added benefits of longer-term employees—for example, measures related to productivity and customer service—neither unit was shown to be underperforming relative to the other.
| Resist the understandable urge to settle for a possibly superficial interpretation. |
The lesson Mastrosimone takes from this incident in benchmarking: "Numbers in a vacuum are dangerous." Or another way to look at it: In benchmarking, context is king.
That is but one of the cautionary tales that have emerged from some two decades of corporate benchmarking—one of the business world's most popular and enduring management tools. Benchmarking, a systematic comparison of the processes and practices of two or more companies or two or more units of a company, gauges the performance of an organization or unit relative to a peer. When executed well, benchmarking prominently reveals gaps between the performance of the benchmarker and the performance of a benchmarked "best practices" leader, and often suggests the means by which the benchmarker might close those gaps.
Benchmarkers must be careful, though, to analyze the best practices of others in light of their own culture and circumstances, or they may find that their efforts do more harm than good. They also must precisely determine at the outset of any benchmarking exercise what they are seeking to learn about, why they want to learn it, and what they'll do with the information to make their own processes work better. Insufficient preparation is roughly equivalent to heading out on an auto tour of historic sites without deciding what to see, where to go, or why you're going. In fact, ill-defined benchmarking is nothing more than "industrial tourism," says Cindy Hubert, director of custom solutions for the American Productivity & Quality Center, in Houston. At best, it gains you nothing useful. At worst, it can send you in the wrong direction.
These and other pitfalls can be avoided by observing the best practices of some of benchmarking's leading practitioners and advisers, who point the way to benchmarking success with these guidelines for averting common pitfalls and wrong turns.
A what without a how can do more harm than good
As the experience of Maines Paper & Food Service illustrates, benchmarking isn't really being done—and can't be advantageously used—if the exploration of how others perform stops with statistics. The hows that underlie numbers are essential to a benchmarking effort for several reasons: They can tell you whether you're comparing apples to apples. They can clue you in to methods that you may need to alter to fit your unique culture or operations. And they may persuade you that trying to achieve a best-in-class measure isn't cost-effective for you or that the activity being benchmarked might best be outsourced.
Getting the hows can be a laborious process, easily taking twice as long or more than getting the whats. Resist the understandable urge to settle for a possibly superficial interpretation rather than digging for the underlying explanation of what makes the best better than the rest. If you don't, says Antonio Battaglia, president and CEO of American Beverage Corp., based in Verona, PA, you may find that "you're trying to resolve an apparent discrepancy that may not be a real discrepancy."
Intel corporate purchasing technical assistant Steve Viera says his firm employs front- and back-end measures to dig out explanations. Intel sends its benchmarking surveys with a glossary of terms that helps participating firms provide comparable data. At the back end of the benchmarking process, Viera says, "we peel the onion back with a three- or four-hour teleconference or site visit and ask how they do it. In the explanation, you can sometimes find apparent great results that get completely turned around."
Measure what's needed, not what's easy
Benchmarking the broader and more widely similar measures of corporate performance can fail to give you actionable information, something that is sufficiently detailed for a unit manager to make changes that improve performance. Sure, you could compare the number of products your unit ships per month with the number of computers Dell sells or the number of seats Southwest fills in the same time period, but chances are that this will tell you little or nothing of use in taking your performance to the next level. The sort of measure that can provide such a breakthrough, experts say, is often a relatively precise and perhaps even obscure measure that may be a critical indicator of overall performance.
Although some of the most useful benchmarking data comes from companies with markedly different core businesses than the benchmarker's, many firms struggle with how to undertake comparisons that go outside their industry. In their 2000 Harvard Business Review article "Creative Benchmarking," Dawn Iacobucci and Christie Nordhielm, both of Northwestern's Kellogg Graduate School of Management, suggest an approach that starts from the customer's point of view.
To begin, they write, "list each step of your customer's buying experience, from the initial recognition of need to the final follow-up after the purchase. Next, determine which factors most influence customers' perception of value at each step. Finally, identify companies that excel at each factor—no matter what industry they're in."
The authors continue, "By breaking down the value delivery system into detailed, customer-focused steps, this process helps managers identify relevant companies to study."
No matter what approach you take or which benchmarking measures you study, "you don't need a thousand measures," says Maines' Mastrosimone. "Just find the key measures that serve as critical indicators. We've found that one of these for us is employee turnover in our Customer Touch Departments."
No matter what your benchmarking measures, guard against inadvertently letting a nonbenchmarked metric go bad while you're making great strides in the measure you've benchmarked. At the most obvious level, you can increase production if you lower your quality standards. More subtly, and therefore more dangerously, you might increase a call center's completed contacts per hour at the expense of reps doing some high-margin cross-selling.
"You have to measure the right things and have a balanced set of metrics so you don't push the balloon in at one place and have it pop out at another," says Michael Katzorke, senior vice president of supply chain management for Cessna Aircraft, based in Wichita, KS.
Find the happy medium in frequency
Take the "Goldilocks" approach in setting the frequency with which you benchmark any particular process: not too little nor too often. Benchmarking that's one shot or, in many cases, takes place once annually, is unlikely to gain a continuing commitment to improvement from implementing managers—they may see the effort as a "flavor of the month." On the flip side, benchmarking that's weekly, or in some cases monthly, can turn those managers into slaves of the numbers—so intent on making them better that they're diverted from paying adequate attention to other aspects of their jobs.
Steve Anderson, chairman of the consultancy Acorn Systems, of Wayne, PA, says he generally favors quarterly sampling. "With quarterly, you're talking the language of an operator," he notes, because they are used to working with quarterly reports.
Yes, it's likely that you'll best benchmark different processes at different frequencies, perhaps ranging from as often as weekly to as seldom as once every few years. But experts add that frequency, in general, is increasing, because the speed of business is increasing, and also because the rankings of best-practices companies are changing at an ever-faster pace.
"Times change, and the guy you just benchmarked last year may not be the best to benchmark a given process with today," Katzorke says. "Seeking the best is a dynamic process."
Today's need for speed is even compressing the time expended for discrete benchmarking exercises, from six months for more extensive benchmarking to two or three months.
Eastman Kodak, based in Rochester, NY, calls its shorter efforts "benchmarking lite," says corporate benchmarking manager Shelly Marketell. "The steps are the same as in the traditional approach; what we really are talking about is the rigor and resource you put behind the effort, with the caveat that you get what you pay for," she says.
Recognize you're not the best
Some executives with years of experience in benchmarking will admit to a persistent, almost reflexive, urge to regard the way they do things as uniquely suited to their operations. Even when they're wide open to alternate practices, they say, executives less familiar with the practice of benchmarking will often dismiss findings as "not invented here." Or they'll come up with any number of creative reasons that their operations are unique and therefore not comparable to those of benchmarked units.
Overcome such urges with training in benchmarking, a dose of appropriate humility, and a determination to think outside the box, where the box is "the way things have always been done around here."
"After thirteen years in my first job, I thought I knew supply chain pretty well," says Cessna's Katzorke. "But then I moved to Motorola [a longtime leading benchmarker] and found I could improve much faster if I didn't have to think of all the best ideas myself."
After that, Katzorke says, he better understood some top executives' initial resistance to benchmarking. "How do you get these people to believe they need to change? Take them on a benchmarking tour. Visit ten or eleven companies. Let them see and think for themselves. Somewhere down the road, the light will suddenly go on, and they'll say, 'You know, we could get a lot better.'"
Make benchmarked units an offer they can't refuse
You can't do benchmarking unless others agree to be benchmarked. Why would a world-class outfit want to take time to reveal how it's gotten to the top? There are several common reasons, the first of which is a simple "you scratch my back and I'll scratch yours": Give them the reciprocal lowdown on a process in which you're world-class. When that's not an option, offer to share results of the benchmarking you intend to do; they'll see where they stand in relation to you and the others you benchmark.
Steve Viera explains that Intel routinely offers to share its final report on benchmarking exercises with participating companies. "We explicitly identify their results and ours, with all other participants' data reported without company names." The report also includes Intel's narrative of how data was explained and how Intel's analysts interpret it. Even with all that, Viera says, Intel routinely needs to invite participation of 40 to 60 companies to get 10 to 12 that agree to take part. "That process—getting folks to agree to do it—takes the longest."
| Without the evidence provided by benchmarking, any claim that your unit is among the best is just an opinion. |
| — Steve Viera, Intel |
No matter what you offer as "bait," always approach other companies with a concise explanation of exactly what you're benchmarking, why you're doing it, how you'll use your findings, and when each step will happen. Also, know and strictly obey a benchmarking "code of conduct"; a number of organizations, including the American Productivity & Quality Center, have them.
Describing how Kodak considers the many benchmarking invitations it receives, Marketell says a number of questions are addressed: "Do we have a relationship with [the requesting firm], or would we benefit in starting a relationship? Is the topic of interest to Kodak? Has the survey been designed appropriately? Do they mention the benchmarking code of conduct? Do we know what we're going to get back in return for participation? What other companies have been asked to participate?"
Involve your implementers from the outset
Companies assign responsibility for benchmarking in many different ways—some, such as Kodak, use a central benchmarking office, while other firms make benchmarking one of several major responsibilities of staff jobs at division headquarters. No matter where benchmarking responsibility resides, however, it must always gain the buy-in and involvement—from planning through implementation—of top managers of the activity being benchmarked.
"We've found this to be critical to success," says Kodak's Marketell. "The best way to ensure that the findings are implemented is to have the people who need to implement the findings engaged during the entire benchmarking process."
Implementing managers must also be included in the benchmarking exercise so they won't appear to be getting hit over the head with other firms' data that bests their performance. "You can't use benchmarking metrics to embarrass someone into changing," says Bain & Company director and benchmarking expert Darrell Rigby. "It never works."
Seek benchmarking in every form and forum where it may help you
Benchmarking can be entirely internal to the company—performed among various business units or facilities or divisions—or completely external to the company and (in most such cases) its industry. Most top-performing companies take a two-pronged approach. They pursue their own independent, customized benchmarking efforts while participating in formal or informal coalitions of companies that continually benchmark against one another in a largely unchanging lineup of metrics.
Many trade associations run benchmarking operations for their member firms. They're typically viewed as a neutral gatherer-disseminator that can preserve confidentiality and guard proprietary information.
"It's right up there as one of our top member benefits," says Metals Service Center Institute president M. Robert Weidner of the Chicago-based association's quarterly and annual industry benchmarking report. The chief drawback of such services is their usual lack of underlying detail and explanations of widely varying measures.
"In the metals service center industry you can have a heck of a time defining processes," says Arnold Tenenbaum, retired CEO of Chatham Steel, in Savannah, GA. "We're all stuck with our own internal differences." Weidner acknowledges disparities, saying he suggests that members use the report as "a foundation. It's not meant to be the end-all for how high-performing companies get where they are."
Nonetheless, benchmarking is becoming ever more important in establishing any business unit's contribution to corporate performance.
"Intel managers are challenged to demonstrate their units are a value-add," says Steve Viera. "Without the evidence provided by benchmarking, any claim that your unit is among the best is just an opinion."