• 13 hours Is This The World’s First Truly Democratic Stock Exchange?
  • 16 hours India’s Wealthiest Set To Hold $23 Trillion By 2028
  • 19 hours First Quarter Profits Slip For World's Top Oil Companies
  • 21 hours The Yuan May Be China's Biggest Weakness
  • 2 days Hedge Funds Having A Banner Year
  • 2 days Disney Heiress Asks “Is There Such A Thing As Too Much?”
  • 2 days BHP Turns Bullish On EVs
  • 2 days Investors Turn Bullish On America’s Nuclear Decommissioning Business
  • 3 days The $90M Inflatable Rabbit Redefining Modern Art
  • 3 days Huawei’s Fate In The Air
  • 3 days Tesla Slashes Prices Again
  • 3 days The Modern History Of Financial Entropy
  • 4 days Italy’s Central Bank Embraces Sustainable Investing
  • 4 days Trump Lifts Metals Tariffs To Cool Simmering Trade War
  • 4 days Researchers Push To Limit Space Mining
  • 4 days Could China Start Dumping U.S. Treasury Bonds?
  • 5 days Is Winter Coming For HBO?
  • 5 days Rise Of EVs Signals Peak Gasoline
  • 6 days Jeff Bezos Doubles Down On Space Colonization Ambitions
  • 6 days Gold Mining Stocks Stuck In Limbo
Strong U.S. Dollar Weighs On Blue Chip Earnings

Strong U.S. Dollar Weighs On Blue Chip Earnings

Earnings season is well underway,…

How Millennials Are Reshaping Real Estate

How Millennials Are Reshaping Real Estate

The real estate market is…

  1. Home
  2. Markets
  3. Other

Good Models and Bad Models

I have recently begun to spend a fair amount of time explaining the difference between a "good model" and a "bad model;" it seemed to me that this was a reasonable topic to put on the blog.

The difference between a good model and a bad model isn't as obvious as it seems. Many people think that a "good model" is one that makes correct predictions, and a "bad model" is one that makes bad predictions. But that is not the case, and understanding why it isn't the case is important for economists and econometricians. Frankly, I suspect that many economists can't articulate the difference between a good model and a bad model...and that's why we have so many bad models floating around.

The definition is simple. A good model is one which makes good predictions if high-quality inputs are given to the model; a bad model is one in which even the correct inputs doesn't result in good predictions. At the limit, a model that produces predictions that are insensitive to the quality of the inputs - that is, whose predictions are just as accurate no matter what the inputs are - is pure superstition.

For example, a model of the weather that depends on casting chicken bones and rat entrails is a pretty bad model since the arrangement of such articles is not likely to bear upon the likelihood of rain. On the other hand, a model used to forecast the price of oil in five years as a function of the supply and demand of oil in five years is probably an excellent model, even though it isn't likely to be accurate because those are difficult inputs to know. One feature of a good model, then, is that the forecaster's attention should shift to the forecasting of the inputs.

This distinction is relevant to the current state of practical economics because of the enormous difference in the quality of "Keynesian" models (such as the expectations-augmented Phillips curve approach) and of monetarist models. The simplest such monetarist model is shown below. It relates the GDP-adjusted quantity of money to the level of prices.

M2/GDP and GDP Deflator 1913-2013

This chart does not incorporate changes in money velocity (which show up as deviations between the two lines), and yet you can see the quality of the model: if you had known in 1948 the size of the economy in 2008, and the quantity of M2 money there would be in 2008, then you would have had a very accurate prediction of the cumulative rate of inflation over that 60-year period. We can improve further on this model by noting that velocity is not random, but rather is causally related to interest rates. And so we can state the following: if we had known in 2007 that the Fed was going to vastly expand its balance sheet, causing money supply to grow at nearly a 10% rate y/y in mid-2009, but at the same time 5-year interest rates would be forced from 5% to 1.2% in late 2010, then we would have forecast inflation to decline sharply over that period. The chart below shows a forecast of the GDP deflator, based on a simple model of money velocity that was calibrated on 1977-1997 (so that this is all out-of-sample).

GDP Deflator, 4q Average 2006-2016

That's a good model. Now, even solid monetarists didn't forecast that inflation would fall as far as it did - but that's not a failure of the model but a failure of imagination. In 2007, no one suspected that 5-year interest rates would be scraping 1% before long!

Contrariwise, the E-A-Phillips Curve model has a truly disastrous forecasting history. I wrote an article in 2012 in which I highlighted Goldman Sachs' massive miss from such a model, and their attempts to resuscitate it. In that article, I quoted these ivory tower economists as saying:

"Economic principles suggest that core inflation is driven by two main factors. First, actual inflation depends on inflation expectations, which might have both a forward-looking and a backward-looking component. Second, inflation depends on the extent of slack (or spare capacity) in the economy. This is most intuitive in the labor market: high unemployment means that many workers are looking for jobs, which in turn tends to weigh on wages and prices. This relationship between inflation, expectations of inflation and slack is called the "Phillips curve."

You may recognize these two "main factors" as being the two that were thoroughly debunked by the five economists earlier this month, but the article I wrote is worth re-reading because it describes how the economists re-calibrated. Note that the economists were not changing the model inputs, or saying that the forecasted inputs were wrong. The problem was that even with the right inputs, they got the wrong output...and that meant in their minds that the model should be recalibrated.

Core CPI Inflation, Actual versus Predicted

But that's the wrong conclusion. It isn't that a good model gave bad projections; in this case the model is a bad model. Even having the actual data - knowing that the economy had massive slack and there had been sharp declines in inflation expectations - the model completely missed the upturn in inflation that actually happened because that outcome was inconsistent with the model.

It is probably unfair of me to continue to beat on this topic, because the question has been settled. However, I suspect that many economists will continue to resist the conclusion, and will continue to rely on bad, and indeed discredited, models. And that takes the "bad model" issue one step deeper. If the production of bad predictions even given good inputs means the model is bad, then perhaps relying on bad models when better ones are available means the economist is bad?

 


P.S. Don't forget to buy my book! What's Wrong with Money: The Biggest Bubble of All. Thanks!

You can follow me @inflation_guy!

Enduring Investments is a registered investment adviser that specializes in solving inflation-related problems. Fill out the contact form at http://www.EnduringInvestments.com/contact and we will send you our latest Quarterly Inflation Outlook. And if you make sure to put your physical mailing address in the "comment" section of the contact form, we will also send you a copy of Michael Ashton's book "Maestro, My Ass!"

 

Back to homepage

Leave a comment

Leave a comment