• 287 days Will The ECB Continue To Hike Rates?
  • 287 days Forbes: Aramco Remains Largest Company In The Middle East
  • 289 days Caltech Scientists Succesfully Beam Back Solar Power From Space
  • 689 days Could Crypto Overtake Traditional Investment?
  • 694 days Americans Still Quitting Jobs At Record Pace
  • 696 days FinTech Startups Tapping VC Money for ‘Immigrant Banking’
  • 699 days Is The Dollar Too Strong?
  • 699 days Big Tech Disappoints Investors on Earnings Calls
  • 700 days Fear And Celebration On Twitter as Musk Takes The Reins
  • 701 days China Is Quietly Trying To Distance Itself From Russia
  • 702 days Tech and Internet Giants’ Earnings In Focus After Netflix’s Stinker
  • 706 days Crypto Investors Won Big In 2021
  • 706 days The ‘Metaverse’ Economy Could be Worth $13 Trillion By 2030
  • 707 days Food Prices Are Skyrocketing As Putin’s War Persists
  • 709 days Pentagon Resignations Illustrate Our ‘Commercial’ Defense Dilemma
  • 710 days US Banks Shrug off Nearly $15 Billion In Russian Write-Offs
  • 713 days Cannabis Stocks in Holding Pattern Despite Positive Momentum
  • 714 days Is Musk A Bastion Of Free Speech Or Will His Absolutist Stance Backfire?
  • 714 days Two ETFs That Could Hedge Against Extreme Market Volatility
  • 716 days Are NFTs About To Take Over Gaming?
  1. Home
  2. Markets
  3. Other

Good Models and Bad Models

I have recently begun to spend a fair amount of time explaining the difference between a "good model" and a "bad model;" it seemed to me that this was a reasonable topic to put on the blog.

The difference between a good model and a bad model isn't as obvious as it seems. Many people think that a "good model" is one that makes correct predictions, and a "bad model" is one that makes bad predictions. But that is not the case, and understanding why it isn't the case is important for economists and econometricians. Frankly, I suspect that many economists can't articulate the difference between a good model and a bad model...and that's why we have so many bad models floating around.

The definition is simple. A good model is one which makes good predictions if high-quality inputs are given to the model; a bad model is one in which even the correct inputs doesn't result in good predictions. At the limit, a model that produces predictions that are insensitive to the quality of the inputs - that is, whose predictions are just as accurate no matter what the inputs are - is pure superstition.

For example, a model of the weather that depends on casting chicken bones and rat entrails is a pretty bad model since the arrangement of such articles is not likely to bear upon the likelihood of rain. On the other hand, a model used to forecast the price of oil in five years as a function of the supply and demand of oil in five years is probably an excellent model, even though it isn't likely to be accurate because those are difficult inputs to know. One feature of a good model, then, is that the forecaster's attention should shift to the forecasting of the inputs.

This distinction is relevant to the current state of practical economics because of the enormous difference in the quality of "Keynesian" models (such as the expectations-augmented Phillips curve approach) and of monetarist models. The simplest such monetarist model is shown below. It relates the GDP-adjusted quantity of money to the level of prices.

M2/GDP and GDP Deflator 1913-2013

This chart does not incorporate changes in money velocity (which show up as deviations between the two lines), and yet you can see the quality of the model: if you had known in 1948 the size of the economy in 2008, and the quantity of M2 money there would be in 2008, then you would have had a very accurate prediction of the cumulative rate of inflation over that 60-year period. We can improve further on this model by noting that velocity is not random, but rather is causally related to interest rates. And so we can state the following: if we had known in 2007 that the Fed was going to vastly expand its balance sheet, causing money supply to grow at nearly a 10% rate y/y in mid-2009, but at the same time 5-year interest rates would be forced from 5% to 1.2% in late 2010, then we would have forecast inflation to decline sharply over that period. The chart below shows a forecast of the GDP deflator, based on a simple model of money velocity that was calibrated on 1977-1997 (so that this is all out-of-sample).

GDP Deflator, 4q Average 2006-2016

That's a good model. Now, even solid monetarists didn't forecast that inflation would fall as far as it did - but that's not a failure of the model but a failure of imagination. In 2007, no one suspected that 5-year interest rates would be scraping 1% before long!

Contrariwise, the E-A-Phillips Curve model has a truly disastrous forecasting history. I wrote an article in 2012 in which I highlighted Goldman Sachs' massive miss from such a model, and their attempts to resuscitate it. In that article, I quoted these ivory tower economists as saying:

"Economic principles suggest that core inflation is driven by two main factors. First, actual inflation depends on inflation expectations, which might have both a forward-looking and a backward-looking component. Second, inflation depends on the extent of slack (or spare capacity) in the economy. This is most intuitive in the labor market: high unemployment means that many workers are looking for jobs, which in turn tends to weigh on wages and prices. This relationship between inflation, expectations of inflation and slack is called the "Phillips curve."

You may recognize these two "main factors" as being the two that were thoroughly debunked by the five economists earlier this month, but the article I wrote is worth re-reading because it describes how the economists re-calibrated. Note that the economists were not changing the model inputs, or saying that the forecasted inputs were wrong. The problem was that even with the right inputs, they got the wrong output...and that meant in their minds that the model should be recalibrated.

Core CPI Inflation, Actual versus Predicted

But that's the wrong conclusion. It isn't that a good model gave bad projections; in this case the model is a bad model. Even having the actual data - knowing that the economy had massive slack and there had been sharp declines in inflation expectations - the model completely missed the upturn in inflation that actually happened because that outcome was inconsistent with the model.

It is probably unfair of me to continue to beat on this topic, because the question has been settled. However, I suspect that many economists will continue to resist the conclusion, and will continue to rely on bad, and indeed discredited, models. And that takes the "bad model" issue one step deeper. If the production of bad predictions even given good inputs means the model is bad, then perhaps relying on bad models when better ones are available means the economist is bad?

 


P.S. Don't forget to buy my book! What's Wrong with Money: The Biggest Bubble of All. Thanks!

You can follow me @inflation_guy!

Enduring Investments is a registered investment adviser that specializes in solving inflation-related problems. Fill out the contact form at http://www.EnduringInvestments.com/contact and we will send you our latest Quarterly Inflation Outlook. And if you make sure to put your physical mailing address in the "comment" section of the contact form, we will also send you a copy of Michael Ashton's book "Maestro, My Ass!"

 

Back to homepage

Leave a comment

Leave a comment