4 Uncertainty

4.1  The need to address uncertainty

Cyclically-adjusted balance estimates have been found to be unreliable in real time, primarily due to the uncertainty surrounding real-time output gap estimates (Hallet et al, 2009). This suggests that the indicator should be used with caution, as one amongst many different indicators of the fiscal position. It also suggests real-time use of a cyclically-adjusted balance indicator demands an analysis of uncertainty.

There are two principal means of addressing uncertainty. One important way is to test sensitivity of results to the key input variables and parameters. Sensitivity analysis is limited in that it cannot attribute probabilities to various outcomes. One way to address this is through the construction of confidence intervals, which are developed next and presented using fan charts.

4.2  Confidence intervals

Confidence intervals for key fiscal variables can be estimated using historical data of forecast errors and revisions (eg, see Congressional Budget Office [2003] and Office for Budget Responsibility [2010]).

Confidence intervals for New Zealand's cyclically-adjusted balance can be constructed using Treasury's historical errors and revisions to real-time estimates and forecasts for government revenues, expenses, nominal GDP growth, output gap, unemployment rate as well as estimates of the uncertainty around the elasticity parameter values. Other assumptions are required: in particular it is assumed that errors are normally distributed with constant cross-sectional correlation and no serial correlation. Errors are adjusted for policy changes which occurred after the forecasts were finalised so that the results attempt to capture the likely range of outcomes conditional on constant policy settings. Annex 2 provides further detail on the data and methodology employed. Given the need to make assumptions about the distributions of future errors and that the limited data available, the numerical results should be seen as indicative rather than robust estimates of uncertainty.

Figure 12 shows the output gap with an 80% confidence interval. This is based on the variance of the official Treasury estimate with respect to the latest estimate. It is therefore limited because (i) there exists no “true” output gap from which to measure error and (ii) the Treasury has changed its estimation method over time, which will be the cause of some portion of the error. The confidence bands should also be viewed as being conditional on the output gap estimation technique used, rather than capturing the uncertainty introduced by the wide range of estimation techniques available (in this case, an MV filter over history and production function for the forecast horizon).

The fan chart for the cyclically-adjusted balance is shown in Figure 13, which is again conditional on using the official Treasury output gap estimation technique. The 80% confidence interval for the one-year-ahead estimate is ±2% of GDP, thus the 2011 deficit (forecast, at Budget 2010, to be 4.0% of GDP) appears to be structural with a high degree of confidence.

As noted, alternative output gap estimation techniques have not been incorporated into this analysis, nor the terms of trade. One approach for further investigation may be to assign subjective probabilities to different possibilities for the key structural parameters (eg, potential output model specification, structural terms of trade) which could assist with making finely balanced policy judgements in real time (ie, a decision-making under uncertainty framework). Another avenue would be to conduct simulations on real-time data using different indicator specifications or drawing on a wider literature, such as on output gap uncertainty (eg, Orphanides and van Norden, 2002).

Figure 12 – Output gap fan chart (using official Treasury output gap measure)

Source:  The Treasury, author's calculations

Note: There are four bands on each side of the mean each representing sequential deciles such that the range of the fan represents the boundary of the 80% confidence interval around the central projection.

Figure 13 – Cyclically-adjusted balance fan chart (using official Treasury output gap measure)

Source:  The Treasury, author's calculations

Note: There are four bands on each side of the mean each representing sequential deciles such that the range of the fan represents the boundary of the 80% confidence interval around the central projection.