Now Playing: Technical analysis
The Commitments of Traders report is now being referred to as a conspiracy by some writers. Others are claiming the Commercials are wrong, and that the Speculators are correct. Another camp claims the Large Commercials (Hedgers) are never wrong. A quick analysis should assist us in resolving this controversy.
What is the Commitment of Traders Report?
The first Commitments of Traders (COT) report was published for 13 agricultural commodities as of June 30, 1962. At the time, this report was proclaimed as "another step forward in the policy of providing the public with current and basic data on futures market operations." Those original reports were compiled on an end-of-month basis and were published on the 11th or 12th calendar day of the following month.
Over the years, in a continuous effort to better inform the public about futures markets, the Commodity Futures Trading Commission has improved the COT in several ways. The COT report is published more often - switching to mid-month and month-end in 1990, to every 2 weeks in 1992, and to weekly in 2000. The COT report is released more quickly - moving the publication to the 6th business day after the "as of" date (1990) and then to the 3rd business day after the "as of" date (1992). The report includes more information - adding data on the numbers of traders in each category, a crop-year breakout, and concentration ratios (early 1970s) and data on option positions (1995). The report also is more widely available - moving from a subscription-based mailing list to fee-based electronic access (1993) to being freely available on the Commission's internet website (1995).
The COT reports provide a breakdown of each Tuesday's open interest for markets in which 20 or more traders hold positions equal to or above the reporting levels established by the CFTC. The weekly reports for Futures-Only Commitments of Traders and for Futures-and-Options-Combined Commitments of Traders are released every Friday at 3:30 p.m. Eastern time.
Reports are available in both a short and long format. The short report shows open interest separately by reportable and non reportable positions. For reportable positions, additional data are provided for commercial and non-commercial holdings, spreading, changes from the previous report, percents of open interest by category, and numbers of traders. The long report, in addition to the information in the short report, also groups the data by crop year, where appropriate, and shows the concentration of positions held by the largest four and eight traders.
Current and historical Commitments of Traders data are available on the Internet at the Commission's website: http://www.cftc.gov. Also available at that site are historical COT data going back to 1986 for futures-only reports and to 1995 for option-and-futures-combined reports.
We now know that the COT is a government generated report that publicizes the total commodity futures positions held by each of the two main categories (Commercial, and Non-Commercial). This report is issued weekly by the Commodity Futures Trading Commission (CFTC). This allows one to see the changes in their holdings.
It is very questionable that the COT report represents a conspiracy of some kind, given the source, and the continual updates of the information. This would require a massive government conspiracy, covering numerous agencies.
How does one interpret the COT Report?
This part of the study becomes murky. It appears that there are three differing opinions. The basic argument comes down to their belief in a "battle" between the Large Commercials (Hedgers), and the Large Speculators. Each side either believes that their "team" is the superior in terms of success, or that the COT is irrelevant. In the interest of objectivity, I will present all positions.
The Forecasting Methodology
By William L. Jiler
Commodity Research Bureau
Basically, we tried to determine the "forecasting" performance of the major identifiable groups of market participant - Large Hedgers, Large Speculators, and Small Traders. It was logical to assume that the larger and more sophisticated traders should have market insights that would enable them to predict futures price movements, if not infallibly, at least more accurately than the small traders who presumably included the "uninformed public." We also thought it was possible that the sizes of the various market positions, at different times, could well result in a type of self-fulfilling prophecy.
From the statistics in the "Commitments if Traders" report, we were able to approximate the net positions at the end of each month for Large Hedgers, Large Speculators, and Small Traders. We averaged their month-end statistics over a number of years to find out what their normal positions would be at any given time of the year. We then compared each group's actual position with their so-called normal position. Whenever their positions deviated materially from the norm, we took it as a measure of their bullish or bearish attitude on the market.
By studying subsequent price movements, we were able to establish "track records" for each of the groups. As anticipated, we found that Large Hedgers and Large Speculators had the best forecasting records, and the Small Traders the worst, by far. We were somewhat surprised to find that the Large Hedgers were consistently superior to the Large Speculators. However, the predictive results for the Large Speculators varied widely from market to market.
The differences between their current net open interest position and the seasonal norm supply us with a tangible percentage measure of the degree of bullishness or bearishness of each group towards a particular market to a certain extent. From these "net-net" figures, we obtain a configuration of market attitudes of the principal players. From our research and long experience we have drawn up some general guidelines:
The most bullish configuration would show large hedgers heavily net long more than normal, large speculators clearly net long, small traders heavily net short more than seasonal. The shades of bullishness are varied all the way to the most bearish configuration which would have these groups in opposite positions-large hedgers heavily net short, etc. There are two caution flags when analyzing deviations from normal. Be wary of positions that are more than 40% from their long-term average and disregard deviations of less than 5%.
We'd like to present some examples of how we utilized this open interest analysis in our "Technical Comments" section of the CRB Futures Chart Service. In late August of 1983, we turned bearish on sugar when it was over 10¢ a pound. Throughout 1983 and 1984, we advocated a bearish stand even though prices had dropped below 4¢ to 16-year lows. An important reason for our doggedness, in addition to the bearish chart, was our analysis of the "Commitments" report. For over two, years, the Large Hedgers' average net short position was over 20% larger than their previous 6-year average. Small traders, despite tremendous losses, averaged almost 20% higher net long positions throughout the entire debacle.
In August of 1983, Chicago wheat futures soared to new contract highs. The charts were very bullish, which we acknowledged in our "Comments" of August 12, 1983. However, we noted that the latest "Commitments of Traders" report sounded a negative note. Large Hedgers were 36% net short and Small Traders were 24% net long, both way over their 10-year averages at that time. Subsequently, the market topped out and prices trended lower for the next 6 months.
A study of the open interest configuration for corn and soybeans just prior to their spectacular bull move in the summer of 1983 will show how the analysis "did" and "didn't" work. It worked for corn, which showed Large Hedgers with net long positions well above normal and Small Traders net short. This bullish pattern was just the opposite of the soybean open interest. Here, Large Hedgers were heavily net short and Small Traders had a net long position of 20% versus a more normal 10% for June. Yet, both commodities enjoyed similar bull moves. An unforeseen drought that summer probably accounted for the strange results.
While we have shown only some relatively recent examples of this kind of open interest analysis, our experience with the technique goes back over two decades. The performance patterns are fairly consistent. Yet, we have to admit that there were exceptions that proved to be quite dramatic. Therefore, it is important also to utilize other available technical and fundamental tools to arrive at a high probability of success in forecasting prices. The nature of the events that shape price trends of futures contracts should keep even the most proficient of technical and fundamental analysts on their guard and flexible at all times. International developments, weather, and politically-motivated legislation are among the unpredictable forces that can change the direction of the markets in an instant. There is no master key that can unlock all the doors to successful price forecasting. Nevertheless, we believe that the proper interpretation of the Commitments of Traders" reports is valuable and belongs on the analyst's key ring.
Jim Bianco, President of Bianco Research
Looking at things from a different angle (as he is wont to do), Jim Bianco, president of Bianco Research in Barrington, Ill., focused on the Commercial hedger's vs. Large speculators rather than hedgers vs. small traders. Why? Because "small traders" refers specifically to the volume of activity rather than style (hedgers or speculators), Bianco explained. Large speculators are just that, he said, "trend-following technical types" with no position in the underlying asset, in this case the S&P 500 cash. In the latest report, the large speculators were net long 10, 721 contracts -- near-record levels as well.
Historically, "commercial hedgers have been right the vast majority of time and the speculators wrong, " Bianco said plainly. How wrong? Large speculators were net short S&P 500 futures every week except five for the six years ending May 16, while commercial hedgers were net long roughly 95% of the time, or all but about 16 weeks, he reported. (Given that the S&P 500 went from roughly 300 to 1500 in that span and these are "naked shorts, " Bianco wondered how speculators managed to stay in business. He mused that there probably has been a big turnover in their ranks.)
Bianco offered two caveats to analysis of the report:
One, when commercial hedgers get it wrong, it's usually in a "major way" where they miss a major secular change. For example, commercial hedgers were net short crude futures for much of 1998 and 1999 as the commodity plunged to $10 a barrel, but remained so during the initial phase of its recovery to above $30 this year. "Eventually, they got back in synch, but struggled in the beginning."
Two, May 16 is significant because the hedgers and speculators swapped positions (the former getting short, the latter long) in conjunction with a change in the reporting requirements by the CTFC. Previously, trades above 600 contracts were considered "large" vs. small. Since May 16, the threshold has risen to 1000."The CTFC is not going to recalculate [the reports] going back in time so we effectively have six months of history and can't read into this either way, " Bianco said.
The Myth of Commercial Superiority in the Futures Markets - Dan Norcini
Once again we see that many in the gold community are anxiety-laden with dread over the prospects for poor "ol Yeller. As can be expected, the "I never met a gold rally I could not pick a top in" expert advisors are warning of impending disaster for gold bugs. Some of these gold perma-bears, and I hate to sound too piquant, remind me of roaches that emerge as soon as the lights go out in the kitchen. They run hither and thither defiling everything they come in contact with only to scurry back into the cracks and crevices as soon as the lights come back on. "Gold is finished". ""The top is in and we are looking for gold to move back down in earnest". "The nth super-cycle has combined with the xth sine wave that has harmonized with the zth multi-year squiggly whatever to tell me that the golden goose is cooked". And so on and so on and so they pontificate, ad infinitum, ad nauseam!
To buttress their claims, they trot out the war-wearied, old soldier argument that the Commercial category, comprised of the "genius, boy wonders, miracle working, epitome of wisdom and knowledge" traders have amassed a sizeable net short position against the poor, dumb, ignorant, hapless, witless speculators who have moved over to the net long side of the market. Obviously, the "smart" money is betting on declining gold prices while the dim-witted speculators are long and wrong or so the argument goes.
Is this really the case however? Are the commercials the superior traders? Do they always make money? Can the hapless speculators, especially the feeble small traders ever hope to beat the commercials? If the answer to this last question is, "NO", then I submit the futures markets might as well be shut down since no small trader should ever attempt to defy the powerful commercial interests. Happily for us, the answer is not, "NO", but is rather the opposite. The small specs and the large specs are quite capable of regularly taking money out of the market and right out of the commercial category's pocketbooks if they trade smart.
We find no reason to believe that Commitments of Traders Report is manipulated by the CFTC or the FDA. However, the increasing frequency of the COT Reports may be having a deleterious effect on its usefulness. The historical value of the COT Report was based on the monthly cycle. We are now using a weekly report. We may be comparing apples to oranges. A monthly chart is more reliable than a weekly chart for trend determination!
Another problem is the interpretation of a Large Commercial's (Hedgers) activities. They are long the commodity by definition. Their primary concern is to avoid a price break which would reduce the value of their inventories. A rising market is not a major focus of their trading activities. Therefore, it seems likely that the Large Commercials (Hedgers) will be the best guide for predicting bear markets. The very size of their positions preclude trading for quick profits.
The question of proper interpretation of the COT must be addressed. Fortunately, various authors have shared their perspective on the value and use of the COT report.
Jim Bianco found that the Commercials (Hedgers) were almost always right, and the Large speculators were almost always wrong. (Given that the S&P 500 went from roughly 300 to 1500 in that span and these are "naked shorts, " Bianco wondered how speculators managed to stay in business. He mused that there probably has been a big turnover in their ranks).
William L. Jiler "We were somewhat surprised to find that the Large Hedgers were consistently superior to the Large Speculators. However, the predictive results for the Large Speculators varied widely from market to market".The Commodity Research Bureau studies validate the Large Commercial's superiority in market forecasting. However, one would have to use the interpretative rules developed by the CRB.
Dan Norcini finds that the COT is a secondary input to his trading activities Mr. Norcini is adamant that the following the trend is the primary approach, and all other factors are for confirmation only.
Jim Sinclair gave this response to a question about the COT. "I put more attention on the price movement of gold and the MACD 3, 7 & 9 plus Momentum 14 than I would now on COT. Therefore, I make nothing of COT either bullishly or bearishly on gold. Also, do not look at static numbers on anything but rather look for trend. COT is more important on Cotton than it is say for gold."
George Paulos volunteered the following observation; "I watch the COT myself for gold and silver. There seems to be some correlation with intermediate trends, but the other markets don't make sense to me.".
Summation
The readers will have to make their own minds from the evidence presented. My personal opinion is that Mr. Sinclair, Mr. Norcini, and Mr. Paulos have the best of this argument Perhaps it's time to downplay the COT Report. It appears to be needlessly complicating our trading strategies! Most of the time the markets are in a trading range, or trending upward. The COT is the most useful only a small percentage of the time, but we must trade most of the time. Let's rely on tools that work most of the time!
FORGET THE DRAMA! TRADE WITH THE TREND!
