I’ve gotten a few questions about the performance analysis noted in a recent New York Times piece about Dimensional Fund Advisors and Vanguard. Here’s the quote:
“What’s more, investors in Vanguard stock mutual funds have had higher actual returns than investors in Dimensional funds. On an asset-weighted basis in the 10 years through Jan. 31, the return received by Vanguard investors was 6.614 percent, annualized, compared with 5.05 percent for Dimensional funds, Morningstar calculates.”
I have some sympathy for financial writers who only have so many words to work with and in particular here where the main thrust of the piece is about mutual fund shareholder activism. However, this form of returns comparison is so high level that it’s effectively devoid of any informational value.
Before getting into the weeds of how best to compare one fund against another (and a couple of important caveats about these types of comparisons), I wanted to illustrate the basic flaw of the approach described above. Imagine we have two fund companies who only have bond mutual funds:
- Fund company A’s bond funds invest entirely in U.S. Treasury bonds.
- Fund company B’s funds invest entirely in high-yield corporate bonds.
I then tell you that over the 10 years ending January 2013 company A’s funds have earned returns of 4.7 percent per year, while company B’s funds have earned 10.4 percent per year. (These happen to be the actual 10-year returns for the Barclays Capital Treasury and High Yield indexes, respectively.)
Hopefully the first thing you would say would be “So what? You’re comparing apples to oranges.” Fund company B’s bond strategy isn’t comparable to fund company A’s. This is effectively the flaw in the New York Times story.
When weighting by assets, Vanguard’s strategies tend to be more market-like, while DFA’s assets under management will have a definitive bias toward small-cap and value. Further, I suspect that DFA has a larger proportion of its assets under management in international stocks compared to Vanguard. As one anecdotal point in this area, Vanguard’s S&P 500 Index strategies had about $256 billion in assets at the end of February, while its international developed markets index strategies had just $13 billion in assets. This is a huge bias toward U.S. stocks and I suspect indicative of what you would find if you had the full picture of Vanguard’s stock fund assets. DFA’s assets would be more balanced between U.S. and international stocks. Ideally, you want to control for these types of differences when analyzing performance differences.
Into the Weeds
There are two ways to do a meaningful performance comparison, both of which involve comparing one fund directly against another instead of one fund family directly against another. The simplest method is to compare the performance of two funds in the same asset class. For example, you could compare the returns of one U.S. large-cap value stock fund to another over the same period of time, or you could compare the returns of one international small-cap value strategy against another over the same period of time.
This methodology allows you to more accurately control for things that matter such as geography, size of companies held and whether the companies held are value or growth companies. In theory, then, any difference in long-term returns between the two funds should be due to cost differences and other more subtle methodological differences between the funds that may lead to one fund outperforming another. Here’s an example of this type of comparison between DFA and Vanguard in the asset classes of U.S. large-cap, U.S. large-cap value, U.S. small-cap, U.S. small-cap value and REITs over the 10 years ending January 2013.
Annualized Returns for the 10 Years Ending January 2013 Per Bloomberg
This analysis shows that DFA has generally outperformed Vanguard in value strategies, while its results are mixed in small-cap asset classes, matched performance in large cap and underperformed a bit in REITs. That differential should be close to zero in the future, since DFA has lowered the expense ratio on its REIT strategy.
The second method, however, is the gold standard for comparing performance of one fund relative to another, but it involves regression analysis, which apparently scares everyone but this guy. This method is the gold standard because it’s generally accepted as the best way to understand why a given fund had the returns it did and how it added value. For example, did one fund outperform another simply because it held smaller stocks or because it was more deeply tilted toward value? Any readers who’re interested in an example of this type of analysis can shoot me an e-mail and I’ll send you an example.
The first caveat is that you’re wasting time if you expect any of these methods to identify whether one fund is superior to another over relatively short periods of time. (By short I mean anything less than five years of data.) For example, if you found that one U.S. large-cap value fund outperformed another by 1 percent per year over three years and then conclude that it’s the superior fund, you’ll find yourself consistently making bad decisions. Financial data is far too noisy to derive major conclusions over short periods of time.
The second caveat is that understanding the context of performance differences is important regardless of whether you’re looking at short or long periods of data. For example, let’s say that I’m comparing two small-cap funds:
- One of the two funds only holds micro-cap stocks.
- The other holds micro-cap stocks but holds slightly larger stocks as well.
Generally, you would expect the micro-cap stock fund to outperform, since extremely small companies tend to outperform slightly larger companies. But if the period you’re using to compare performance is a period where small stocks did poorly, then you’ll likely see the exact opposite result. If you conclude that the micro-cap stock fund is a “bad” fund due to its performance over this period, you’ve made a mistake. Its performance wasn’t poor due to bad management, but rather due to the (uncontrollable) fact that small stocks did poorly over that particular period.
Random Links and Commentary of the Week
We're at the height of the NFL free agency period, and I've enjoyed Bill Barnwell's Grantland pieces on the topic.