The year 2007 has proven to be one of the most volatile years in recent memory. You'd have to go all the way back to 2003 when the last bear market ended to see a similar level of market volatility as measured by the Volatility Index (VXO).

Volatility is most common at the end of a market decline or correction. It's really a combination of factors: the public, in typical reactionary fashion, gets scared and panics as the end of the correction approaches. They throw everything overboard and completely unload their stock holdings. Making matters worse (for them), they begin heavily buying put options and selling stocks short based on the mistaken belief that things can only get worse. Volatility increases and is a reflection of the amount of fear investors have about the future.

More than anything else, volatility is a measure of the three dominant emotions that investors experience: fear, greed and complacency. It's when the most dominant of these emotions, namely fear, is at a peak that huge spikes in the Volatility Index are seen like the one recently (see chart below). These volatility spikes, while usually short-lived, tend to feed on themselves and create even more fear and panic among retail investors. The fear is further augmented by the massive exposure given to volatility in the mainstream financial press. One could advance the argument that the press has helped create the volatility by its incessant campaign of headline fear which it has waged virtually non-stop this year.

Too many investors have been under the media's spell as these investors have been too scared to take advantage of the two great buying opportunities the market has presented them with this year. The first one was in March and the other in August. This really isn't at all surprising given that most mainstream investors are under the control of the media, psychologically speaking (see Lonny Kocina's book, Media Hypnosis, for a more in-depth discussion of this concept of media mind control). As Don Hays puts it, "The news is the camouflage that drives the herd crazy."

Volatility is also used as a weapon against mainstream investors. It can be used, for instance, to clear small investors out of the way so that big money traders can scoop up shares at bargain prices. Most small investors simply can't afford a prolonged exposure to market volatility and are often forced to sell out to the big money traders, who in turn are more than happy to relieve the small traders of their shares at cheap prices.

From a psychological standpoint, volatility has been shown to increase anxiety and paranoia in most investors. It makes them more likely to let go of a potentially profitable long position at the slightest hint of weakness. Ironically, this helps bolster the stock market's support since an increase in worry has been shown to strengthen the market's "Wall of Worry." It comes as no surprise then that the recent market bottom was been accompanied by a huge increase in bearish sentiment, worry and volatility. It's as if investors have been collectively hit by a big "volatility stick" and are afraid to go anywhere near the stock market.

Carl Swenlin of Decisionpoint.com points out that with the recent change to the "up tick rule" on short selling this summer, there was a corresponding increase in volatility. This may account, at least in part, for the severity of the recent correction. Swenlin also points out that this is a "two-edged sword" that can magnify the severity of a decline by allowing short sellers to sell into it. But it will also tend to produce a greater number of short sellers who can be "squeezed" when the short interest builds too much. To give you an idea just how much the public short selling has increased lately, take a look at the following chart.

One of the biggest moves in the major indices was accompanied by a huge move in the Volatility Index recently. As mentioned earlier, the CBOE Volatility Index (VXO) can be used to measure short-term spikes in fear and complacency in the stock market. A huge rally in the VXO such as the one we saw in the past two weeks normally precedes important correction lows. The latest spike in the VXO, the highest since 2003, is one of many signs that the market has overdone the fear-driven decline and now an important bottom is being made.

To underscore the point I've been trying to make that the most important role of market volatility, let's discuss the correlations between volatility increases and insider buying of stocks during market panics. Other than the obvious benefits that increased volatility confers to insider traders in making short term capital gains, it can also be used as a weapon to scare away the small investors from participating in a market uptrend as we've looked at here. Additional proof of this is the comparison that can be made between the Gambill Oscillator, which measures insider buying, and the Volatility Index.

The Gambill Oscillator tracks corporate insider activity in the Russell 3000 stocks. Along with the VXO, the Gambill Oscillator is showing the highest level of insider buying since the March 2003 bear market low. Since 2002 when this indicator was first created, whenever the oscillator went above 25% (which it did in the latest correction), the stock market was up over 10% in the coming six months and up 22% over the next year. The Gambill Oscillator hit a high reading of 75% during the broad market decline.

Another point worth making is that since 1990, whenever the Volatility Index has been between 30 and 40, as it was during the recent correction, the stock market has been up by an average of nearly 11% over the next six months. Moreover, the market's gains over the following 12 months has been an average 16.2%. This compares with average "normal" market returns of 4.8% and 10.2%, respectively, over the 6-month and 12-month time frames.

Bolstering this bullish stock market outlook based on the insider sales data is a recent report by a respected financial newspaper. According to the Financial Times, total insider buying in the U.S. stock market reached $252 million in August, the highest level since 2003. This compares to a seasonal average of $186 million. At the same time, insider sales have dropped sharply from a four-year monthly average of $4 billion to $2.9 billion.

Volatility can indeed be used as a "big stick" to hit small investors with and send them running for cover. But this stick has two edges and the end result of the recent volatility spike will be positive for stocks.