Peer analysis can provide a cross check for the recommendations we derive from more normative analysis.  The best peer analyses begin with a narrowly defined population of comparable firms.  If the proper care isn’t given to this selection process, the results can prove misleading, as we’ll see in the following example.

The Problem With Too Many Peers

With a hazy recollection of his college statistics course, an analyst may be tempted to improve the “significance” of his analysis by building a larger peer group.  This can backfire.  Most companies have very few comparable peers — a handful at most — so we are quick to question analyses against larger populations.

Here’s an example of what can go wrong.  The question is whether leverage helps or hurts profitability.  (We define leverage as Debt/EBITDA; profitability as EBITDA/Sales.)  We’ll answer the question by conducting a straightforward univariate regression analysis.

We begin by examining the S&P500 firms classified as “Drug manufacturers – major”, which includes BMY, MRK, LLY, JNJ, PFE, and ABT.  We find a positive (albeit weak) relationship between leverage and profitability:

We now repeat for the “Drug wholesale” sector, which includes MHS, MCK, CAH, and ABC:

The relationship is still positive.  However everything goes pear shaped when we merge the populations:

Caveat analyst.

This is an example of Simpson’s paradox, which is one of many ecological fallacies which can plague statistical analysis of aggregates.