19 Nov 2014Working Paper Summaries

The Search for Benchmarks: When Do Crowds Provide Wisdom?

by Charles M.C. Lee, Paul Ma and Charles C.Y. Wang

Finding appropriate economic benchmarks for individual firms is a fundamental issue. Firms, managers, investors, and researchers all need to identify fundamentally similar benchmarks for such tasks as performance evaluation, executive compensation, equity valuation, statistical arbitrage, and portfolio construction. While traditional benchmarking methods rely primarily on industry classification schemes, more recent approaches introduce new dimensions by utilizing novel data sources or fresh data analytic techniques. Some of these approaches suggest we may need to rethink the reliance on traditional industry classification for benchmarking purposes. In this paper, the authors conduct a comprehensive analysis of the state-of-the-art representatives of four broad categories of peer identification schemes nominated by either financial practitioners or recent academic studies as potential solutions to economic benchmarking. The study's results suggest that the class of bench-marking solutions that harnesses the collective wisdom of investors is a promising path for the future. This approach's effectiveness, however, depends on the sophistication of the individuals in the population (the inherent level of collective wisdom attainable through sampling) and the quality of the information environment surrounding the firm, as well as the size of the sample itself. Key concepts include:

Traditional industry classifications are unlikely to capture nuanced or changing economics in firms in an increasingly service- and knowledge-based economy.
Aggregated revealed-choice-based approaches have great potential in resolving long-standing benchmarking problems in accounting and finance. These approaches aggregate individual agents' choices to reveal the collective wisdom of investors with respect to the set of economically-related firms.

Author Abstract

We compare the performance of a comprehensive set of alternative peer identification schemes used in economic benchmarking. Our results show the peer firms identified from aggregation of informed agents' revealed choices in Lee, Ma, and Wang (2014) perform best, followed by peers with the highest overlap in analyst coverage, in explaining cross-sectional variations in base firms' out-of-sample: (a) stock returns, (b) valuation multiples, (c) growth rates, (d) R&D expenditures, (e) leverage, and (f) profitability ratios. Conversely, peer firms identified by Google and Yahoo Finance, as well as product market competitors gleaned from 10-K disclosures, turned in consistently worse performances. We contextualize these results in a simple model that predicts when information aggregation across heterogeneously informed individuals is likely to lead to improvements in dealing with the problem of economic benchmarking.