I had had recent phone and email conversations with senior economists at the New York Fed and Atlanta Fed regarding their GDP forecasts.
The Atlanta Fed calls its model GDPNow and the New York Fed calls its model the FRBNY Staff Nowcast (simply referred to as "Nowcast" below).
The Fed offices are not in competition with each other, although quarter-to-quarter "bragging rights" may be in order. That is my subjective hypothesis, not based on any economist's statement.
Let's take a look at the theory as well as possible flaws in the models.
I know most about the New York Fed model, so let's start there. Some of the discussion is mathematical, but I will explain in easy to understand terms.
Nowcast Model
Nowcast is a "single factor model" based on Kalman Common Filtering Techniques.
The explanation is easier than the math.
- The model starts from a presumed known point such as the last GDP estimate, then takes into consideration new data as it comes in.
- The forecast is derived on the basis of past correlations, continually revised by incoming data. Current data is more important than prior data.
- The difference between the model forecast and incoming new data determines whether or not estimate rises and falls.
Single Factor
The "single factor" is the latent state of the economy, a pervasive source of common fluctuations in all the data that enter the model. Think of it as a business cycle factor that summarizes the state of the system. The modeling approach is based on the insight that business cycle fluctuations are pervasive and hence there is a lot of co-movement in the economy.
The model does not attempt to mimic the BEA. Rather, the model attempts to predict what the BEA will report based on incoming data.
In short, one data point affects estimates of related data points that follow.
Model Explanations and Examples
- The BEA GDP report does not take into consideration any regional manufacturing reports, jobs, or any of the factors in the model. In contrast, and based on past experience, there is an established relationship between regional Fed reports (such as the Philly Fed manufacturing report) and the next GDP report. This relationship goes into the Nowcast model.
- In advance, the model predicts the next result of the Philly Fed report (based on all the previous data points, not just recent Philly Fed reports). For example, a change in the Empire State manufacturing report may change the previous assumption about the Philly Fed manufacturing report.
- One cannot assume a good data report will necessarily cause the model forecast to rise. A recent example of this is construction spending. A rise in construction spending may cause the model estimate for housing starts to go up. If the actual housing starts report is less than the model predicted, the GDP model estimate will drop.
- Thus, a seemingly good report might lead to a lower forecast if the model predicted a better outcome. The converse is true as well. A seemingly bad data report could actually add to the GDP model estimate.
Uncertainty
The BEA states its accuracy in the range of +-1 standard deviation, approximately 1 percentage point in GDP.
My Nowcast economist contact made this statement "We will release the estimates of uncertainty based on the historical errors of the model in real-time."
Patrick Higgins writes ...
Hi Mish,
The first version of GDPNow was completed in 2011q3 and we've been maintaining real-time forecasts of the model since then. The Forecast Errors for the model's final forecasts over the 2011q3 - 2016q1 period are available in the tab "TrackRecord" of the Excel sheet. Over this period, the average absolute forecast error of the annualized real GDP growth rate was 0.61 percentage points and the root mean square error was 0.83 percentage points. An analysis of the forecast errors and a comparison of the model's performance with professional forecasters is available in the May 16, 2016 Macroblog GDPNow and Then and also the May 23, 2016 Macroblog Can Two Wrongs Make a Right? The tab "TrackRecord" has a chart showing an estimate of the historical average absolute forecast error as a function of how far away the advance release data is. The spreadsheet linked above also has historical forecasts of the model if you ever want to do your own analysis. See the tab "ReadMe" for a description of what tabs contain what data.
Nowcast Parameters
That table reflects the Nowcast as of June 3. I highlighted the key changes from the May 27 report.
For the month, I count 35 items in a grouping of 8 general buckets. The two regional manufacturing reports are the Philly Fed and New York Fed regions.
Why just those two regional reports? Why not Dallas, Kansas City, Richmond, etc.? Where does one stop? Why not other variables?
The answer is complexity.
Forecasting GDP with a Dynamic Factor Model
Please consider a snip from Forecasting GDP with a Dynamic Factor Model, an article by the Spanish Ministry of Economy and Finance, on how Spain derives its GDP forecasts.
At the Ministry of Economy and Finance we have developed a dynamic factor model to estimate and forecast the rate of growth of the Spanish economy in the very short term. This model uses a coincident indicator, or estimated common factor, to forecast GDP by means of a transfer function. The model estimates a common factor underlying 31 economic indicators spanning domestic production, the labor market, and domestic trade flows. It enables us to forecast GDP several times a week, providing a virtually real-time complement to the four quarterly GDP official reports issued each year.
With 31 indicators, our model avoids the disadvantages inherent in both larger and smaller models. Models with more than 80 indicators are difficult to interpret, require more work to maintain, and suffer from group effects that can distort the estimation of the common factor. On the other hand, models with fewer than 12 indicators can lack sufficient coverage, omit useful information from excluded indicators, and be more susceptible to the abnormal behavior of a single indicator.
The indicators we use cover a wide range of domestic economic activity, including industrial production, air traffic, cement usage, tourism, energy consumption, transport by railway, housing starts, and employee compensation. We supplement these domestic indicators with external ones, such as imports and exports of goods and services, as well as social security contributors (employment) and other labor-market indicators.
In addition, our model includes qualitative indicators reflecting the level of confidence Spaniards have in their business and household finances. Such indicators are less common in dynamic factor models because they are difficult to integrate with hard data. These so-called soft indicators, however, provide a useful alternative perspective on the economy, and they are promptly available, making them valuable for short-term forecasting.
GDPNow Model
Atlanta Fed economist Patrick Higgins explains GDPNow: A Model for GDP "Nowcasting".
The GDPNow model forecasts GDP growth by aggregating 13 subcomponents that make up GDP with the chain-weighting methodology used by the U.S. Bureau of Economic Analysis.
Using real-time data since the second-half of 2011, GDPNow model forecasts are found to be only slightly inferior to consensus near-term GDP forecasts from Blue Chip Economic Indicators. The forecast error variance of GDP growth for each of the GDPNow model, Blue Chip, and the Federal Reserve staff's Green Book is decomposed as the sum of the forecast error covariances for the contributions to growth of the subcomponents of GDP. The decompositions show that "net exports" and "change in private inventories" are particularly difficult subcomponents to nowcast.
The advantage of GDPNow over Blue Chip is the GDPNow forecast is continual, following key economic reports, not twice a month as with the Blue Chip estimates.
One can download the Full Text of the GDPNow Model but I caution the document might only be understood by a mathematician.
GDPNow Construction Spending
On May 26, I noted GDPNow Bounces to 2.9% Following Durable Goods Report.
There was nothing remarkable in the report at first glance. GDP rose almost entirely due to the durable goods report. However, one day later I reported New York Fed 2nd Quarter "Nowcast" Rises to 2.2% Following Housing, Durable Goods Reports
Housing, durable goods, and revisions added 0.254, 0.099, and 0.074 percentage points to the New York Fed model forecast.
On May 24, there was a Massive Jump in New Home Sales but that did not factor into the GDPNow model at all.
Higgins explained in an email.
Hi Mish,
Both new and existing single-family home sales are ingredients used to estimate brokerage commissions, which comprise 23% of residential investment in nominal terms. Since new-single family home sales are only about 1/10th the size of existing-single family home sales; the latter are generally more important for this subcomponent.
Best regards,
Pat
Is that a big model difference between GDPNow and Nowcast?
What Can Go Wrong?
The answer is plenty. GDP is heavily revised. So are construction spending, durable goods, housing, and jobs reports.
I also question at least one Nowcast model assumption as noted in New York Fed Nowcast Up to 2.4% (I'll Take "The Under"); Modeling Error on Unemployment Rate?
Nowcast Details
Positive and Negative Factors
The biggest positive factors since the May 27 report were personal consumption expenditures, the manufacturing ISM, and the unemployment rate.
The largest negative factor since the May 27 report is construction.
Modeling Error on Unemployment Rate?
I suspect the New York Fed has a small modeling error in the civilian unemployment rate.
In general, it's logical to presume a decline in unemployment likely is a positive factor for growth, but I also propose one needs to look at why the rate declined.
Today, the unemployment rate dropped because close to half a million people dropped out of the labor force.
Moreover, there was a massive jump in involuntary part-time work of 468,000. Neither of those can remotely be considered positive for spending or GDP.
JOLTS
Take a look at the top line in the above Nowcast detail. It's on JOLTS (Job Openings and Labor Turnover). It added 0.04 percentage points to the Nowcast model.
I believe job openings data should have subtracted from the model.
I discussed why in BLS Says Jobs Openings Up; Actually, Openings Falling Fast!
First, let's consider what the BLS says, then let's look at an alternative viewpoint.
BLS: Job Openings vs. Hires
Job Openings - Real Time Macroeconomics
Jon Hartley, Researcher and Policy Analyst for Real Time Macroeconomics sees things this way (anecdotes Mish).
Clearly someone is wrong. Who is it?
If I am correct on both counts, the Nowcast model should have subtracted instead of added for both the unemployment rate and job openings.
GDPNow does not consider either the unemployment rate or job openings, but it does include some related variables.
Higgins writes ...
Hi Mish,
The civilian unemployment rate is not a direct input into the model, but GDPNow does include some variables related to the unemployment rate in its dynamic factor model.
- Civilian Employment: Nonagricultural Industries: 16 yr + (SA, Thous)
- Civilian Employment: Sixteen Years & Over (SA, Thous)
- Civilian Unemployment Rate: Men, 25-54 Years (SA, %)
- Median Duration of Unemployment (SA, Weeks)
But you're right; the civilian unemployment rate is not a direct input into the model.
Pat
Employment and unemployment are related in ways, but unemployment can rise of fall independent of employment.
Unemployment is a factor of labor force participation compared to employment, not employment directly.
A decline in unemployment because people dropped out of the labor force is a lot different than a decline in unemployment because more people are working.
In the last jobs report, the civilian Labor Force declined by 468,000. Household survey employment rose by only 26,000. Last month, household employment declined by 316,000 but the unemployment rate was steady.
If there is another bad jobs report, it's quite likely there is a significant trend change. Regardless, I suggest the Nowcast model needs to incorporate "why" the unemployment rate went up or down.
Second Quarter GDP Estimates
- New York Fed Nowcast June 3: 2.4% New York Fed Nowcast Up to 2.4% (I'll Take "The Under"); Modeling Error on Unemployment Rate?
- Atlanta Fed GDPNow June 14: 2.8% GDPNow Forecast Rises to 2.8% Following Retail Sales Report
- Markit June 3: 0.7% to 0.8% Composite PMI Flirts With Contraction; Markit Chief Economist Estimates GDP 0.7-0.8%
- ISM June 3: 1.6% Non-Manufacturing ISM Much Weaker Than Expected
Turning Points
GDP models are only as good as the data going into them. In general, it's highly likely that small errors even out in the GDPNow and Nowcast models.
However, at economic turning points, models forecasts are likely to be way off. Are we at a turning point?
Here's two likely clues, and something that none of the above models considers.
Percent Change in Temporary Help Services Employment
US Federal Personal Witholding Tax Receipts
For tax survey details, please see US Tax Receipts Signaling Recession?
If we are at an economic turning point, the model forecasts are likely significantly wrong, and downward revisions will follow.
Final Comments
Everything above has been reviewed by the New York and Atlanta Fed economists. They did not comment on my estimation of modeling errors or turnings points, only their models.
I asked each of them one final question: "How far back does back-testing of your model go?"
I received an answer from Pat Higgins at GDPNow: "The forecast errors are constructed by comparing with the BEA's as reported first-release GDP data. Using revised current-vintage data, we constructed forecasts back to 2000q1. These data are described in rows 28-31 of the ReadMe Tab of the Excel spreadsheet GDP Tracking Model Data and the historical forecasts can be found in the linked tabs in column A."
The New York Fed economist was not available to comment on the short notice I gave him.
Back-testing does bring up another issue in regards to modeling. Each recession will be different. We have already seen manufacturing data so bad that in every historical precedent, the economy was long in recession. There are normally yield curve inversions prior to recessions. I am confident we will not see one this time. Might there be a "job gain recession, or one in which minimal jobs are lost?" Sure, why not?
None of this detracts from the respective models. There is no such thing as a perfect model. I hope my comments go into making a better model, and I thank both the Atlanta Fed and New York Fed economists to taking the time to answer my questions.
