A blob is taking over asset-manager due diligence: ignore it at your peril. Ronald Surz is president of PPCA, a San Clemente, Calif.-based software firm that provides performance-evaluation and attribution analytics, and a principal of RCG Capital Partners, a Denver, Colo.-based fund-of-hedge-funds manager.

There are many peer-group biases; some can be controlled, some can't. But there's one obscure bias that just won't go away, and it's raising havoc with investment-manager evaluations. It oozes out from compromises that all peer-group providers make, and stealthily prevades evaluators' judgments.

This perilous perpetrator is the classification bias.

Peer-group providers establish rules for classifying managers as large- or small-cap, value or growth, etc., and then populate peer groups with managers that meet these criteria. Classification bias feeds on the lack of similarity among the funds that meet these classification rules, enveloping manager evaluations with scary ratings. It is an amorphous blob of a bias and to rid ourselves of it we have, as they say, to think outside the box.

Most investment professionals know about survivor biases in peer groups -- and most of them make a conscious choice to ignore their effects. But few understand classification bias so the decision to ignore its effects is wholly unintentional. And the problem is that classification bias distorts traditional peer-group rankings and invalidates hedge-fund peer groups.

Classification bias in traditional peer groups

Value investing was in favor before 2007, but value managers lagged their benchmarks woefully: more than 90% of the value managers in Morningstar's value peer groups underperformed their benchmarks in 2006. Were value managers brain-dead?

Then in 2007, value managers redeemed themselves. Most funds in the Morningstar value peer groups outperformed their benchmarks. Was this turnaround a salutary effect of mass brain transplants? We rather think not. The real culprit was classification bias.

The majority of funds in Morningstar's large-cap value peer group aren't large-cap value at all: they're populated by big mid-cap companies with more of a growth orientation than those S&P 500 Value range. As a consequence, the majority of these managers lagged their index in 2006 because their growth exposure put them at a disadvantage. Now that value is out of favor, however, this growth orientation makes them look good again. What goes around comes around.

Classification bias distorts studies of investment manager rankings. One recent study in a growing list of studies that purport to instability in manager skill, this one by Matthew Rice of the Chicago-based investment consulting firm DiMeo Schneider, says that "about 90% of managers with top-quartile results for the 10 years through 31 December 2006 suffered through a below-median stretch of three years or more along the way." But could it be that manager skill isn't actually changing much at all?

The reality is that peer-group universes are revolving around the mangers as opposed to the managers moving within their universes. Styles go in and out of favor but skill persists, although classification bias makes things appear otherwise. Classification bias is causing much of the change in rankings, rather than changes in skill, because this bias has different affects as styles go in and out of favor, as summarized in this contingency table.

Classification bias: Winner and losers When style is Pure Managers Impure Managers

In favor

Win

Lose

Out of favor

Lose

Win

Hedge funds

Classification bias for hedge funds is far worse than it is for traditional managers because hedge funds have far more moving parts, including: Approach long: style, no. of names, geography, derivatives, beta, etc. Approach short Amounts long and short (direction) Leverage Fees

Most hedge-fund managers differ from others in at least one of these moving parts. As a result funds in hedge-fund peer groups don't behave like one another; they're not correlated. Harry Kat of the Cass Business School in London is one of several experts who have documented this fact in tables like this one, which shows the average correlation both within and across hedge fund peer groups. |image2| Kat concludes from this that you get roughly as much diversification by picking funds in the same peer group as you do by using managers in different peer groups.

|image2| Some say that these low correlations are exactly what you should expect for hedge funds -- and a good thing too as it makes for good diversification. This is true. But it doesn't make for good peer group comparisons. We can't have it both ways.

Hedge-fund peer groups make reasonable shopping malls for selecting hedge funds, but they are very poor backdrops for evaluating individual hedge fund performance. Think about the stores in a shopping mall: you'll shoe stores and pet shops and food courts, etc., but you'll be hard pressed to find a real basis for comparing them to one another for the simple reason that they have specific and often quite unique missions.

To evaluate traditional and hedge-fund managers properly we need to remove as many biases as we can, including classification bias. Attempts to cleanse traditional peer groups don't work because we can't make classification bias go away using traditional approaches. As Albert Einstein said: "The significant problems we face today cannot be solved at the same level of thinking we were at when we created them."

Fortunately, there is a solution to the myriad problems with peer groups. It takes the form of Monte Carlo simulations of all the portfolios that a manager could have held, and it's called Portfolio Opportunity Distributions (PODs). -FWR

Purchase reproduction rights to this article.