Summary Treasury's Forecasting Performance 2008

1. Forecast errors for real GDP growth#

The Treasury's forecasts of real GDP growth and nominal GDP growth are neither persistently too low nor too high over different forecast horizons (one year ahead, two years ahead, etc.) over the past 20 years or so.

Blue line = root mean square error (RMSE), a measure of the average forecast error, disregarding the sign (positive/negative) of the error.
Red line = mean error.
Green area = forecast error confidence interval, i.e. ideally, we would like the red line to stay in the green area to be able to say that forecasts are, on average, neither too low nor too high.

2. Forecast errors for CPI inflation#

There is some evidence that Treasury forecasts of CPI inflation made two or more years ahead of the actual result being known tend to be too low, i.e. longer-term inflation forecasts tend to be too low. This was not the case for CPI inflation forecasts of shorter horizons (current year and one year ahead).

3.Comparison of average forecast errors#

Comparing the average forecasting error of the Treasury's forecasts of the New Zealand economy with that of other forecasting organisations, both within and outside New Zealand, shows that The Treasury's forecasts, at the time they were compiled, were amongst the best available to Ministers (over the 2000 to 2008 period).
(Note that different forecasts may be prepared for different purposes and that forecast performance varies with the period selected).

4.Tax receipts forecast errors#

There is evidence to suggest that half-year update (HYEFU) forecasts of current-year tax receipts tended to be too low (1994 to 2007 inclusive). This was not the case for forecasts of tax revenue and tax receipts at any other horizons.

5.One-year-ahead tax receipts forecast errors#

From 1998 through to 2007, nearly all of the tax forecasts were too low. This may be evidence of “serial correlation” in the forecast errors, i.e. an under-forecast in one year increases the likelihood of the next forecast also being too low. Clearly, this is not a desirable quality for a forecasting model to have. The Treasury is working to eliminate serial correlation from its tax forecasting models (where applicable).

6. Recent tax receipts forecast errors#

All tax forecasts (and GDP growth forecasts) made from the end of 2007 onwards have been too high. This is the result of:

• the severity of the current economic downturn;
• the difficulty of accurately predicting turning points in the economy; and
• switching to “correlation-free” models as mentioned above.

7. Average tax forecast errors across countries 1995 to 2006#

The average forecast errors for the New Zealand Treasury's tax forecasts were similar to those of the UK and Australia, and smaller than Canada's, over a 12 year period ending in 2006.

The Treasury is striving to improve its macroeconomic and tax forecasting accuracy.

Some recent measures include:

• Subjecting Treasury's forecasting processes to external review;
• Use of expert panels to quality assure forecasts;
• Commissioning independent expert opinions;
• Use of general equilibrium modelling and specialised forecasting software;
• Continuing redevelopment of forecasting models;
• Continuing in-depth monitoring; and
• Maintaining experience.