Is the S&P 500 still a diversified index?

Key takeaways

• Growing concentration is unusual and has significant implications for portfolio construction, particularly for conservatively managed, risk-conscious portfolios.
• Previous periods of extreme market concentration have proven to be short lived.
• As the largest names in a cap-weighted index soar, they pull the valuation multiple of the overall index higher; investors looking to maintain diversification and control risk should consider decoupling from cap-weighted indexes and look hard at the rest of the market.

What has become of the S&P 500?

Much ink has been spilled of late about extreme market concentration—whether discussing the Magnificent Seven, the hyperscale cloud providers or even just Nvidia. These stocks account for an increasing proportion of market capitalization and have driven the preponderance of market returns. Such concentration is unusual and has significant implications for portfolio construction, particularly for conservatively managed, risk-conscious portfolios.

Going back to 1990, single sectors in the S&P 500 Index have neither routinely exceeded 20% of the index nor been more than 2x the size of the next largest sector. During the dot com" bubble of the late 1990s, the information technology (IT) sector surged to 32% of the S&P 500, but just 12 months later it retreated back to 18%. IT remained a mid-to-high teens weighting in the S&P 500 for the next 15+ years (Exhibit 1).

Today, the IT sector again represents 32% of the S&P 500. This figure understates its true size, however. Over the last six years, S&P Global Ratings has repeatedly removed large names from the sector to prevent it from appearing too big. In 2018, S&P created the communication services sector, placing Google (Alphabet) and Facebook (Meta Platforms) in the new classification. Likewise, in 2023, Visa, Mastercard and PayPal were reclassified from IT to financials. If not for these adjustments, IT would currently stand at greater than 40% of the index (Exhibit 2).

The issue of concentration at the single stock level is even more extreme. Three technology companies—Apple, Microsoft and Nvidia—represent 20.5% of the S&P 500, the equivalent of the bottom 362 stocks. Ten years ago, the top three stocks represented less than 7% of the total index, and two were technology stocks. Twenty years ago, the top three stocks constituted 8.5% of the total index, and only one was a technology stock.

Stock-level concentration in the S&P 500 extends beyond the three largest names. Today, the top 10 stocks constitute 37.5% of the index—the highest level in recent history—and seven of those 10 are technology and internet companies (Exhibit 3).

Cap-weighted benchmarks: Increasingly problematic yardsticks

The S&P 500 has always been thought of as a diversified benchmark, but it is increasingly less so. While there is no obvious bright-line test for diversification, it seems time to ask: Is the S&P 500 still a diversified benchmark?

The market consists of 11 sectors. Would a risk-averse investor put over 40% of their assets into just one of them?

Large-cap growth managers have wrestled with this phenomenon for many years. Indeed, concentration levels are even more extreme in the Russell 1000 Growth Index than in the S&P 500 (Exhibit 4).

Because market-cap-weighted indexes are frequently used as benchmarks for diversified portfolios, increasing concentration creates challenges for active, long-only managers. Long-only managers are primarily judged based on their relative performance. As stocks become bigger and bigger in the index, one must take bigger and bigger positions to keep up. Over the last 18 months, as a handful of the largest names have powered the index higher, most active managers have underperformed. Few managers are overweight Apple, Microsoft and Nvidia, which together have powered 85% of the year-to-date returns of the S&P 500.

While it is no fun to trail a benchmark, the inability of long-only managers to keep up in a surging tape should be of secondary concern. The average core portfolio may have lagged its benchmark over the last 18 months, but its clients have likely made good money. In our view, outperforming the S&P 500 in this environment requires embracing a level of concentration and risk that should be unacceptable for a risk-averse investor. Do we really want diversified investment managers putting 10%+ in each of Apple, Microsoft and Nvidia? Do we really want portfolio managers putting more than 40% of a “diversified” portfolio in the IT sector?

An episode or a condition?

Previous periods of extreme market concentration have proven to be short lived. Portfolio managers wring their hands and pull their hair out, but ultimately the bubble bursts and concentration recedes (Exhibit 5).

The current cycle may end the same way, but this time may also be different. The network effects and scale advantages of the largest technology companies have proven profoundly durable, and their massive investments in artificial intelligence (AI) seem likely to only reinforce and extend their competitive moats.

However, just because a handful of tech companies have grown to dominate their industry does not mean they must dominate our portfolios. There is no logical reason why an investor should own more of a certain stock simply because it represents a large weighting in well-known indexes. Benchmarks were introduced to measure performance, not drive it.

Maintaining diversification reduces risk in concentrated markets

To underscore the value of diversification in moments of extreme concentration, consider the equal-weighted S&P 500 (SPW). The SPW is made up of the same constituents as the S&P 500, only each is allocated a fixed, equal weight. Over longer periods of time, the performance of the S&P 500 and the SPW have been quite similar (Exhibit 6). Indeed, over the last 20 years the S&P 500 has compounded at 10.3% annually, while the SPW has compounded at 10.1%.

From 2004 through November 30, 2022—the date Chat GPT was released—the cap-weighted index rose 9.3% per year while the SPW rose 10.2% per year. Since the debut of Chat GPT, however, a handful of AI frontrunners have driven divergent returns. The S&P 500 has surged approximately 22.0% annualized, while the SPW has compounded at 8.6% per year (Exhibit 7).

The dot-com bubble was the only other period of such stark disparity between the S&P 500 and the SPW. We all know how that ended.

Cap-weighted indexes are generally thought to be less volatile than more diffuse indexes, but this does not hold true when concentrated regimes unwind. In 2000, the cap-weighted S&P 500 declined 9.1% while its equal-weighted twin rose 9.6% (Exhibit 8). In the drawdown of 2022, the S&P 500 declined 18.0% while the SPW was down only 11.5%. Investing in S&P 500 index products today is like putting all of one’s eggs in the same basket; a perfectly reasonable thing to do, so long as nothing goes wrong.

Concentrated markets mask most valuations

As the largest names in a cap-weighted index soar, they pull the valuation multiple of the overall index higher. Indeed, the Price-earnings (P/E) ratio of the S&P 500 is near its highest level of the last 30 years. With that as a headline, the average investor would be forgiven for assuming that the entire market is expensive—but that is not the case! There is tremendous divergence between the valuation of the 10 largest companies in the S&P 500 and the other 490 companies (Exhibit 9).