Not too long ago, before the explosion of countless exchange-traded funds (ETFs) in recent years, the arena of choice for stock-index speculators was the futures markets.

Speculating on the future direction of the major stock indices via futures has a long and distinguished history compared to the new ETF phenomenon. While most of the hyper-popular ETFs like the QQQs universally loved by stock traders today were only introduced in the late 1990s, index futures trading stretches back over two decades.

Way back in 1982, near the very bottom of what would later grow into the greatest bull market in modern history, several individual futures contracts tied to specific stock indices began trading within months of each other. The Kansas City Board of Trade offered futures on the Value Line Index, the New York Futures Exchange introduced futures based on the NYSE Composite Index, and most important of all the Chicago Mercantile Exchange launched the mighty S&P 500 futures contract.

Not surprisingly, the fantastic concept of stock-index futures took the financial world by storm. This important financial innovation allowed hedgers to protect their existing long stock positions from dangerous market volatility. The hedgers could offload their risk to gunslinging speculators, traders fully willing and able to vastly multiply their own risk in the hopes of earning legendary returns.

Only four years after their launch, in 1986 the S&P 500 futures contract was the second most actively traded futures contract on the planet, with volume approaching 20m contracts that year. And for the lion's share of the time since then, futures trading was the only game in town for speculators wanting to deal in entire stock indices rather than individual stocks. To this day, the S&P 500 futures contracts (SPX) are often among the most heavily traded futures contracts on Earth on any given trading day.

As stock-index-futures trading soared in popularity, new tools arose to help speculators make the crucial decisions necessary to time their trades. One of the most widely respected and followed of these tools was the Commodity Futures Trading Commission's (CFTC) Commitments of Traders reports (CoT).

The famous CFTC CoT reports actually predate the dawn of stock-index futures significantly. The CFTC originally started publishing its CoT reports for agricultural commodities in 1962. The reports were designed to help futures hedgers and speculators better understand what other traders were collectively doing in specific futures markets, a very valuable insight to possess.

As all contrarian speculators know, the behavior of all other players is one of the most important variables to consider in timing trades, and the CoT reports offered tantalizing insights into other traders' behavior. When the Age of Index Futures launched in the early 1980s, naturally these brave new index futures were included in the longstanding CoT reports compiled for futures players.

Originally published once a month, today in the early Information Age the CFTC manages to publish the CoT reports weekly. Eventually I really hope that advances in computing power and information technology will enable this excellent data to be made available at the end of each trading day. Today though, the CoT reports are released every Friday afternoon and include data current to the preceding Tuesday.

If you want to dig into these CoT reports yourself, they are freely available at the CFTC's website. If you haven't confronted these beasts before though, be warned that they are very detailed and can be quite cryptic and intimidating for new readers. Each report also contains an incredible amount of raw data. One single spreadsheet we put together at Zeal using only historical CoT data runs almost 40mb in size, or about 28 floppy disks worth of raw data! The CoTs definitely aren't for the faint of heart.

At a basic level, the CoT reports chronicle the existing outstanding futures positions in the S&P 500 and all other futures contracts of all traders required to report their positions to the CFTC. In order to be required to report, futures traders generally have to maintain positions above minimum thresholds or be futures professionals such as clearing members on exchanges or commission merchants.

The CFTC claims that reportable futures positions generally run between 70% to 90% of the total open interest, or currently outstanding futures contracts, in any given market. As such, all these reportable positions included in the CoT reports provide an excellent proxy of what the futures market as a whole for any individual contract looks like.

Of these reportable positions, the CFTC farther breaks them down into commercial and non-commercial traders. Commercial traders are hedgers, meaning they are commercially engaged in a business directly related to the futures contracts they are trading. In general, hedgers use the futures market to minimize their own risks by offloading these risks to willing speculators.

On the other hand, if a given futures trader is not running a business that needs to directly hedge its risks in the futures market, it is classified as a non-commercial trader. The non-commercials are the speculators, the other side of the futures trade. They actively seek to acquire risk in the hopes of winning large rewards by betting right on future market direction. Since being classified as a non-commercial requires relatively large futures positions, the non-commercials are usually considered to be large speculators.

After the commercial hedgers and non-commercial speculators, the remaining outstanding futures contracts are classified as nonreportable. This nonreportable remnant is comprised of the small speculators, folks like you and I who are not elite futures traders and do not maintain positions big enough to interest the CFTC.

The weekly CoT reports detail this breakdown of the balance of open futures positions between commercials, non-commercials, and the nonreportables. In other words, the CoT details the balance of futures contracts held by hedgers, large speculators, and small speculators, both long and short, in any given futures contract including the famous S&P 500 index futures.

Futures trading, unlike stock trading, is a zero-sum game. This means that capital only changes hands, for every $1 won by one trader there is exactly $1 lost by another trader. If you want to buy a futures contract, someone else has to be willing to sell it to you and take the offsetting position. As such, at any given moment the number of long futures contracts exactly equals the number of short futures contracts. This is very different from the equity world, where essentially everyone can win or lose at once and winning does not require an opposing trader to realize an identical loss.

This zero-sum nature of futures is partially what makes it such a challenging cutthroat game. If you are not a superior trader to the traders on the opposite end of your futures trades, you will lose. Unlike stocks, futures don't create or destroy any wealth, they just shuffle it around between speculators and hedgers and other speculators. It is crucial to fully understand this zero-sum foundation of futures trading!

So, in any given futures contract, including the mighty S&P 500 index futures, the total number of long (positive) and short (negative) positions outstanding always equals exactly zero. For every long contract another speculator or hedger went short on the opposite side of that trade. Total longs can never exceed total shorts and vice versa.

The beauty of the CFTC's CoT reports, however, is that they effectively show the distribution of long and short exposure among all three classes of futures traders, the commercial hedgers, the large speculators, and the small speculators. Using these weekly reports, contrarian speculators can immediately see if one class of futures trader is currently net long or net short. While all three classes' total longs and shorts outstanding always equal, the distribution among the three classes constantly changes.

Net long, of course, means that a class of traders happens to own more longs than shorts at the moment, or they are generally bullish on their futures contracts. Net short, the other side of the trade, shows that a class of traders happens to be generally bearish on their futures contracts. The perpetual interaction between the net long and net short positions of all three classes of futures trading can be quite illuminating.

Our first graph this week shows the S&P 500 itself superimposed over the SPX futures CoT data since 1997. The net long or net short positions of each of the three classes of futures traders are graphed on the left axis. The higher the net long or lower the net short numbers, the more bullish or bearish the particular class of futures traders happens to be.

This unique CoT data, while complex and challenging to interpret, grants us valuable insights that are simply unavailable anywhere else into the mindsets of all the professional and amateur futures traders. As such, contrarian speculators involved in index trading, either via futures or options or ETFs, ought to pay attention to the happenings in the weekly CoT reports.

This first wide-angle strategic chart showing both pre-bubble and post-bubble environments helps set the stage for analyzing today's SPX CoTs. I included this chart to help illustrate just how anomalous this amazing post-bubble environment truly is even for the S&P 500 index-futures trading.

Prior to the 2000 bubble top in the S&P 500, the overall balance of long and short SPX contracts stayed relatively tight between the hedgers, large specs, and small specs. From 1997 to early 2000, the hedgers and small specs were generally right, betting with the Great Bull. Meanwhile the large specs, usually seen as the most intelligent and accomplished traders, had trouble accepting the duration of the huge stock-market bubble of historically unprecedented magnitude and they remained net short.

In May 2000 however, about two months after the enormous stock-market bubble topped, the tight balance of futures positions between the three groups suddenly decoupled. Small speculators, just as history prophesied they would be, were seduced by the mania and became hyper-bullish near the very long-term stock-market top and piled into SPX long positions. At the same time, the hedgers sensed grave peril and dramatically upped the "insurance" to protect their equity longs by selling extensive SPX short positions.

Meanwhile the large specs switched from net short to net long mode, buying some of the SPX futures that the hedgers were selling like crazy. Ever since this odd decoupling of the futures balance in 2000, the balance of positions in the SPX landscape has been dramatically altered and incredibly volatile.

What do you do with this information? I don't know, but I felt it was important to alert you to the very different nature of the SPX futures balance after the bubble as compared to before the bubble popped. At the very least, now you can easily discern that the graphs below show historically anomalous SPX futures extremes spawned by the bubble, not normal environments. Just like every other major market, the huge post-bubble trauma and dislocations have also affected the futures world.

Our next graph merely zooms in on this dataset since 2000, to grant us higher resolution of the ongoing Great Bear bust ravaging the S&P 500.

Drilling down into the Great Bear period that has bloodied the stock markets in recent years, we can gain a great understanding of which classes of SPX futures traders have been generally right or wrong in these challenging times. The idea behind this analysis is simple. If the past CoT data suggests that one group of futures traders is perpetually wrong, at least in this Great Bear, as contrarian index speculators today we probably don't want to emulate their current positions!

Of all three CFTC groups, the hedgers have had the most success in their post-bubble trading. As the yellow line above shows, they have been hardcore net short for almost the entire Great Bear since shortly after the bubble burst in early 2000. The only time the hedgers have even bothered going net long is from mid-March to mid-June of this year, when they deftly rode the greatest rally of the entire Great Bear.

Provocatively however, in mid-June the hedgers lost faith in the war rally and threw net short again with a vengeance. This is more evident in the final graph below, so we will discuss this most recent SPX futures activity a little later in this essay.

Many of these hedgers no doubt had long positions themselves in the stock markets, or their clients did, so they were probably selling SPX futures as a hedge to protect themselves from excessive downside exposure. In hindsight this was a very wise decision, as a properly hedged stock investor could have eliminated 90%+ of his losses and maintained capital even through this brutal environment with proper futures hedges deployed.

I also find it fascinating that the hedgers have gradually reduced their net short position as the markets have fallen lower. Interestingly, the reduction of hedger net shorts corresponds rather well with the Valuation Wave mean reversion in the S&P 500, meaning that the hedgers have sold fewer SPX contracts short as the valuation multiples of the S&P 500 gradually meander back down towards Terra Firma.

This is a prudent strategy and makes great sense, being most short when valuations are most extreme and gradually peeling off short positions as valuations mean revert. More than either other class of futures traders, the hedgers have played this Great Bear exceedingly well and today's contrarian index speculators ought to pay close attention to their current net short positions. So far the right trade has been to bet with these astute hedgers!

Like the hedgers, the large specs have also been net short most of the Great Bear, on the right side of the trade. They had a slow start initially in 2000, which is understandable. When bubbles are actually bursting, like in 2000, it is very difficult to distinguish in real-time between a bubble burst and a normal bull-market pullback. I suspect that the large specs were trying to play contrarians in 2000, as they initially remained net long as if the burst was a normal pullback.

Any speculator has to survive quite awhile and accumulate a lot of market wisdom in order to swing multi-million dollar lines of pure futures speculations, so the large speculators aren't dumb. They patiently waited for a standard 20% pullback from the S&P 500 top and when it arrived and failed at SPX 1225 or so they realized the pullback wasn't normal and immediately ended their net long bias. Since early 2001, the large specs have also been relentlessly net short, making the right bet on this ongoing Great Bear.

While the hedgers' and large specs' execution during this brutal bear has been commendable to excellent, the small specs have been utterly slaughtered. Remember that futures is a merciless zero-sum game? The bottom of the food chain, the helpless shrimp that the whales eat, is the small speculators. Sometimes I think that small specs exist solely to feed the big players and make them rich!

For this entire Great Bear, with the trivial exception of a couple recent war-rally blips, the small specs have been overwhelmingly bullish. They have eagerly gobbled up all the SPX contracts that the hedgers and large specs want to sell them, and they have been ripped to shreds because of their folly.

In futures lore the small specs are often considered "dumb money" and "marks" and the graph above makes it easy to see why. Throughout what will probably prove to be the worst bear market in US history, the small specs have stayed overwhelmingly and steadfastly bullish!

If you don't care about your capital and want to throw it away, the quickest way in stock-index-futures land today is to bet with the small speculators. The legions of small speculators are the sheep that blindly follow the herd instinct, always greedy and almost always wrong. These small specs are the "thundering herd" that contrarian speculators need to bet against in order to be successful.

If you find your own index positions mirroring the small specs in SPX-land, chances are you are going to get hurt when the big smart guys eat your lunch in the markets. Sadly, the small specs tend to be most bullish shortly after major interim tops and once again their net long position has soared today right as the war rally appears to be peaking! Our final graph zooms in one last time.

The relative wisdom or folly evident in the three classes' net SPX futures positions is even more apparent over the last 19 months or so. The hedgers and large specs have been right, and the small specs have been incredibly wrong. Which group of traders would you rather emulate in your own index speculations, regardless if you play the SPX futures, options, or the ETFs?

You have to admit that it sure looks like the little guys have nicely set themselves up to be slaughtered, yet again! The hedgers actually threw long for most of the war rally, but suddenly in June near the latest interim S&P 500 top they dramatically shifted back to net short again. So far their collective behavior has proven to be the right decision, as the stock markets have ground sideways and lower ever since.

Conversely the naïve small specs happily devoured all the SPX contracts the hedgers and large specs were selling short in recent weeks. Like most unsophisticated investors and speculators, it seems like the small specs tend to be most bullish at tops when everything looks rosy, which is exactly the wrong time. Conversely the perma-bullish resolve of the small specs is only slightly challenged near interim bottoms, when they tend to wrongly reduce their net long exposure.

In light of the experience in this brutal Great Bear to date, today's index speculators ought to pay much respect to the net positions of the SPX futures hedgers and steer well clear of the net positions held by the SPX futures small specs. The small specs have long been a goldmine of data for contrarian players, as all you have to do is bet against them and there is a high probability you will win! The hedgers have been right, betting against a dramatically overvalued equity market, while the little guys have been chronically wrong for years now.

While this SPX CoT isn't published often enough to be a primary contrarian index speculation indicator, it is very important data for today's index speculators to consider. It offers priceless insights into the mindset and current betting of three important and very different classes of futures traders. And like so many other indicators today, it is strongly suggesting that the interim top for the S&P 500 is in and another significant downleg is rapidly approaching.

We are currently carefully preparing for this highly probable coming downleg in both our Zeal Intelligence and Zeal Speculator newsletters for our dear subscribers, layering on new short-oriented index trades as well as contrarian long gold-stock plays.

Like the SPX commercial hedgers, I believe for a wide array of fundamental, sentimental, and technical reasons that the war rally is finito and short-side plays need to be deployed ahead of the coming downleg. The time to act is today, not after the markets plunge yet again!

Are your investments and speculations ready for the waking Great Bear?