An Analysis of Tax Revenue Forecast Errors (WP 07/02)

Abstract#

Print this section

The New Zealand Treasury forecasts tax revenue for the twice-yearly Economic and Fiscal Updates. The accuracy of these forecasts is important for the government’s annual budget decisions as they affect key fiscal aggregates such as the operating balance and debt levels. Good decision-making in this area is important for macroeconomic stability and sustainability, one of the Treasury’s outcomes.

Over the past six years, Treasury tax forecasts, and the macroeconomic forecasts on which they are based, have underestimated the actual outturns. This report presents an analysis of the Treasury’s tax revenue forecast errors, both in aggregate and disaggregated by individual tax type.

The analysis focuses primarily on the annual one-year-ahead Budget forecasts that are typically based on rating up past tax revenues by growth rates in related macroeconomic variables such as GDP. The objective of the analysis is to better determine the major sources of tax revenue forecast error and to identify the potential for methodological improvements.

A review of the Treasury's tax forecasting methods is given and a general class of models proposed that encompasses these methods. Adopting one of the simplest of these as a benchmark, the individual tax revenue forecast errors are first disaggregated into component errors due to forecasting the macroeconomic drivers used as a proxy for the tax base, and a component due to forecasting the tax ratio, or ratio of tax revenue to proxy tax base. The tax ratio is further disaggregated into a component error due to forecasting the tax ratio trend and random error. The latter provides a measure of the best accuracy that can be achieved using the benchmark models adopted.

Among other findings, the report shows that the main source of tax revenue underforecasting is the underforecasting of the macroeconomic variables used as tax-base proxies. The tax ratio forecasts were generally unbiased, but less precisely determined than the macroeconomic forecasts. This and other evidence indicate that better tax ratio forecasts are likely to be achieved, even with the simple benchmark model used here. The benchmark models have merit as competing models that could be investigated further alongside other simple structural time series models in a systematic evaluation using historical data.

Acknowledgements#

Print this section

We gratefully acknowledge the assistance and support received from our colleagues within the New Zealand Treasury, particularly Bob Buckle, John Carran, Katy Henderson, Dean Hyslop, John Janssen, Daniel Lawrence, Kam Szeto, Wayne Tan, and everyone else within the Treasury who has taken an interest in this research.

Disclaimer#

Print this section

This document was commissioned by the New Zealand Treasury.  However, the views, opinions, findings and conclusions or recommendations expressed in it are strictly those of the authors, do not necessarily represent and should not be reported as those of the New Zealand Treasury.  The New Zealand Treasury takes no responsibility for any errors, omissions in, or for the correctness of, the information contained in this Paper.

1  Background#

Print this section

In each of the last three years, the New Zealand Treasury has published a report analysing the performance of its macroeconomic, tax and fiscal forecasts. These reports can be found at http://www.treasury.govt.nz/forecasts/performance/. In particular, Treasury forecasts have persistently underestimated actual tax flows over the past six years. Why this should be and what the sources of error are, form the background for this study where the primary objectives are to better determine the major sources of tax revenue forecast errors and to identify the potential for methodological improvements.

The study builds on and complements Schoefisch (2005) which focussed on the Treasury's general tax forecasting methods and processes rather than specific forecasting models and methodology. Schoefisch (2005) noted that a study (Mühleisen, Danninger, Hauner, Krajnyák, and Sutton, 2005) published by the International Monetary Fund (IMF) showed that the Treasury's tax forecasting performance compared well with the performances of government agencies in other countries over the period 1995 to 2003. It would seem that New Zealand is not unique in terms of persistent underestimation of tax, and we are aware of similar reviews undertaken in Australia, Canada and the United Kingdom that also address these issues. Some of these reviews have not yet been published, but others have, including O'Neill (2005) which reviews Canadian federal fiscal forecasting. In general, the literature on tax forecasting seems to be sparse and largely the preserve of official government agencies or organisations such as the IMF. In addition to those already cited, some relatively recent examples of this literature are Basu, Emmerson and Frayne (2003), who examine United Kingdom corporate tax forecasts by the Institute for Fiscal Studies, London, and Rich, Bram, Haughwout, Orr, Rosen and Sela (2005) who use regional economic indices to forecast tax revenues for New York. Further publications are listed in the references of the publications already mentioned.

The Treasury's tax forecasts are based on rating up past tax revenues by growth rates in related macroeconomic variables such as gross domestic product (GDP) which also need to be forecast. Included in the 2003 and 2004 forecast performance reports was an attempt to disaggregate each one-year-ahead Budget total tax forecasting error into a macroeconomic component and a tax component. Separating out the two sources of forecast error allows one to test the proposition that the tax forecasts might still be too low even in the case of a perfect macroeconomic forecast. For total tax, part of the forecast error was attributed to errors in the forecasts of nominal GDP, with the remaining portion being attributed to the tax forecasting process. In both the 2003 and 2004 analyses, there was insufficient evidence to determine whether the proposition was true.

Nevertheless, these analyses suggested that errors in nominal GDP forecasts were making a considerable contribution to errors in the tax forecasts. However, the Treasury does not explicitly use nominal GDP to forecast total tax. Rather, forecasts of various macroeconomic variables are used to forecast each of the component tax types and the forecasts of the tax types are then aggregated into a total tax forecast.

Schoefisch (2005) noted these earlier attempts to split the tax forecast errors into macroeconomic and tax components and recommended that resources should be directed towards analysing the relative contributions of macroeconomic and tax errors to the total tax error for the various tax types (PAYE, GST, company tax, etc). This is a primary objective of the analysis presented in this report.

A review of the Treasury's tax forecasting processes and methods is given in Section 2 and a general class of models is proposed in Section 3 that encompasses these methods. In Section 4, one of the simplest of these models is adopted as a benchmark model where tax revenue is expressed as a product of a tax ratio and a suitable macroeconomic variable that can be regarded as a proxy for the tax base. Using this model, expressions for suitable forecast error decompositions are also derived. Individual tax revenue forecast errors are first decomposed into component errors due to forecasting the underlying macroeconomic driver used as the tax-base proxy, and a component due to forecasting the tax ratio. The tax ratio is then further disaggregated into a component error due to forecasting the tax ratio trend and random error. The latter provides a measure of the best accuracy that can be achieved using the benchmark models adopted.

Using these decompositions, an analysis was undertaken of historical data from 1995 to 2005 where a discussion of the data and adjustments applied is given in Section 5. Results and discussion of the analysis are given in Section 6 with conclusions presented in Section 7. As noted by Schoefisch (2005), the overriding purpose of this analysis is to enhance the understanding of key deficiencies in forecast performance and provide a base for other research projects designed to improve forecast quality.

2  The Treasury's tax forecasting process#

Print this section

Twice each year, the Treasury produces economic and fiscal forecasts. The first forecast each year is usually prepared for the government's annual Budget and published as the Budget Economic and Fiscal Update (BEFU) around May or June. The second forecast is usually released in December, a week or two before Christmas, and published as the Half-Year Economic and Fiscal Update (HYEFU). Prior to 2005, this was known as the December Economic and Fiscal Update (DEFU). In an election year, there may be another forecast published four to six weeks before the general election, called the Pre-Election Economic and Fiscal Update.

The Treasury's economic forecasts are produced by the macroeconomic forecasting team, which devotes four to six weeks to the task at each forecasting round. The macroeconomic forecasters examine recent economic data, the forecasts produced by other New Zealand economic forecasters and discuss the state of the New Zealand economy with many business people around the country. They then run a variety of forecasting models to produce forecasts of many macroeconomic variables, such as GDP and the Consumer Price Index (CPI) of inflation. These forecasts typically cover the current year and the next four years.

The tax forecasts are prepared more-or-less concurrently with the economic forecasts. In the period under examination, a team of three forecasters and a manager prepared forecasts for each of the 20 or so tax types. A tax type is a category of total tax such as PAYE, which is the income tax paid by salary and wage earners, or GST or company tax. Like the economic forecasts, the tax forecasts cover the current year and the following four years. Unlike the economic forecasts, the tax forecasts are for June years. The macroeconomic variables are forecast either quarterly or in March years.

As each part of the economic forecast is prepared, the tax forecasters prepare forecasts of the relevant tax types. For example, as the labour market forecast is prepared, the tax forecasters prepare PAYE forecasts using variables from the labour market forecast. The economic and tax forecasters then examine the labour market and PAYE forecasts together to ensure that they are consistent with each other and with the total economic and tax forecast.

When forecasts have been prepared for each tax type, they are aggregated into a total tax forecast. Three taxes are not forecast by the Treasury. The Ministry of Transport supplies forecasts for Road User Charges (RUC) and motor vehicle licensing fees (MVF), and the Ministry of Economic Development supplies forecasts of Exhaustible Resource Levies (ERL). Collectively, these taxes account for less than 2% of total tax.

The tax forecasts are prepared in terms of both receipts, the tax actually paid to the collecting agency (usually Inland Revenue), and revenue, the tax that is actually due, regardless of whether or not it has been paid. The remainder of this document focuses on tax revenue, although the reasoning is identical for receipts and the results of the analysis for both revenue and receipts are similar.

2.1  Current forecasting methods#

Print this section

In common with many other official agencies around the world, the New Zealand Treasury uses mainly spreadsheet-based tax forecasting models and procedures comprising the following phases.

Phase 1: Determine the nominal tax revenue for the last available year which is the base year.

Phase 2: Adjust the nominal tax revenue for the base year by removing any known anomalies to establish the true underlying tax position for that year.

Phase 3: Apply the forecast growth rates of relevant macroeconomic variable(s) to forecast tax for 1 to 5 years ahead, applying elasticities if required.

Phase 4: Adjust the tax forecasts for anomalies such as tax policy changes, expected shifts in payment dates or taxpayer behaviour, and include any judgemental forecasting adjustments that may be deemed appropriate.

More detailed descriptions of some of the major tax types follow. These serve to illustrate the general nature of the forecasting methods used and how they are implemented.

2.1.1  Source deductions

This is the largest single tax type. It makes up about a third of the total tax collections, around NZ$18 billion out of a total of NZ$51 billion in the year to June 2005. Approximately 97% of source deductions are pay-as-you-earn (PAYE) deductions on wages, salaries and social assistance benefits, with the other 3% being specified superannuation contribution withholding tax (SSCWT).

The forecasting model used during the 1995-2005 period under examination was a quarterly multiplicative model that starts with a history of collections up to the most recently complete quarter. It projects forward by multiplying the collections base by macroeconomic forecasts of wage, salary and employment growth, but also makes adjustments for the progressivity of the individuals' tax scale (higher rates at higher incomes) and payday weightings (most people are paid on a fortnightly cycle). Two components of the source deductions forecast are prepared by other agencies and combined with the Treasury's forecast to produce the final forecast. These components are the Ministry of Education forecasts of PAYE on teachers' salaries, and the Ministry of Social Development forecasts of PAYE on social assistance benefits, which are both small in relation to total source deductions.

More formally, total source deductions SD_q for quarter q are given by

where G_q denotes general source deductions excluding PAYE on teachers' salaries T_q and PAYE on social assistance benefits B_q. Forecasts of G_q are obtained by rating up past values by macroeconomic growth rates according to the recursion

where E_q denotes total employment and W_q denotes total, payday-weighted wages and salaries. Note the elasticities of 1 on employment and 1.2 on wages and salaries, with the wage and salary elasticity being estimated by empirical research. Using this recursion and associated forecasts of the macroeconomic growth rates, a forecast for G_q is now calculated for each quarter for the next 20 quarters or so. These quarterly forecasts are then accumulated into annual forecasts. Further ad-hoc adjustments may be made to the annual forecasts, such as adjustments for changes in tax policy or judgemental adjustments.

Throughout the remainder of this report we use the simple abbreviation PAYE to denote total source deductions (PAYE plus SSCWT).

2.1.2  Other persons tax

This is tax paid mainly by individuals and trusts on income that is not withheld, or is under-withheld, at source. Typically, this is tax paid by small-business operators and investors. Terminal tax from wage and salary earners also falls into this category. Net other persons tax, or other persons tax less refunds to all individuals, makes up about 8% of the total tax take.

This tax type is notoriously difficult to forecast. Since it is an amalgam of tax on a variety of sources, it is difficult to find a reliable macroeconomic driver to use for the forecasts. In the past, the Treasury has tried various multiplicative models and a micro-simulation model, but with varying degrees of success.

Currently, the Treasury uses a composite approach with some smaller anomalous components forecast separately using simple straight line extrapolation. The latter include revenue from income summaries (end-of-tax-year reconciliations for salary and wage earners), and rebates for charitable donations and child-minding/housekeeping expenses. With these components removed, the adjusted other persons tax is monthly revenue that has resulted from personal income. This is allocated into past tax years based on sampled data from the Inland Revenue Department (IRD), New Zealand's main tax-gathering agency. Then the monthly data are accumulated to give tax revenue and tax receipts totals for the respective tax years. The resulting annual adjusted other persons tax for tax year t is denoted O_t.

Although various macroeconomic variables have been used over the years to forecast O_t, the one that has been at the heart of all of the models considered is a measure of household entrepreneurial income calculated by Statistics New Zealand. In essence, the Treasury forecasting model assumes that forecasts of O_t follow the recursion

where I_t denotes entrepreneurial income, and A_t denotes any judgmental or policy adjustments made. The annual tax-year forecasts of revenue and receipts that result are then spread across budget years using interpolation based on the monthly seasonal patterns observed. Finally, these forecasts are accumulated together with the annual forecasts for income summaries and charitable donation rebates to yield an overall forecast for net other persons tax.

2.1.3  Other tax types

Most of the other tax types use models similar to these. Fringe benefit tax, company tax and GST forecasting models, for example, are based on growth rates of compensation of employees, total operating surplus and nominal domestic consumption respectively. Forecasting models for some of the smaller tax types are even simpler with forecasts of excise duties, for instance, based on trend growth estimates. A different model is adopted for forecasting withholding tax on interest. This uses a regression relationship with compensation of employees, domestic consumption, house prices and interest rates as regressor variables.

3  Towards a model framework#

Print this section

The tax revenue forecasting procedures used by the Treasury suggest a multiplicative model for the monthly, quarterly or annual levels of taxation revenue considered. A simple example is

(1)

where Y_t denotes a particular tax revenue, X_t denotes a macroeconomic predictor such as GDP, and e_t denotes multiplicative error which varies about a mean of unity. If the transformed macroeconomic predictor

 

 can be thought of as a proxy for the relevant tax-base, then α can be interpreted as a mean tax rate. Other multiplicative variables can be included and parameters such as α and β may also be time-dependent. Monthly or quarterly variables may have seasonal variation and all are likely to be affected, to some degree, by longer-term economic cycles.

 

In terms of continuously-compounding growth rates, (1) becomes

(2)    

 

 

where Δ denotes the difference operator (

 

), the parameter β can be interpreted as an elasticity, and the ε_t now correspond to additive errors (possibly stationary) with zero mean. Note that the approximation

 

(3)    

 

 

yields

 

 

 

to a good degree of approximation provided the right-hand side of the above (a simple growth rate) is small. Throughout this report, we mainly consider the more-commonly used compound growth rates of the form ΔlogZ_t rather than their simple growth rate equivalents. The reasons for this are largely technical convenience and a direct link to continuous time growth models, but little is lost in adopting either definition since they differ very little in practice.

Consider (2) and forecasting the tax revenue growth rate ΔlogY_t. If the predictor ΔlogX_t is known and the ε_t are independent, in addition to having zero mean, then the best predictor of ΔlogY_t is given by

(4)    

 

 

or, using the above approximations and simple growth rates,

(5)    

 

 

provided the various growth rates are relatively small. Forecast functions such as these lie at the heart of the Treasury's current tax forecasting methods described in Section 2.1. This suggests that the Treasury's tax forecasting methods could be regarded as optimal predictors for simple models that are suitable variants of (1).

This linkage between a model, such as (1), and its forecast function is not unique since other models can be found that will yield the same forecast function (5). However the simplicity of (1) and its growth rate model (2) make it a suitable starting point for a model framework within which the Treasury’s tax forecasting methods can be embedded. This is the strategy that has been adopted here.

3.1  Alternative models#

Print this section

Schoefisch (2005) notes that a number of the Treasury’s tax revenue forecasting methods assume that the elasticity β in (1) is identically unity. He questions this assumption, noting that β may well depend on the phase of the economic cycle which could impact differentially on the various components of GDP. In addition, the parameter α is also likely to change slowly over time to accommodate structural changes in New Zealand's economy.

Such considerations suggest a more general model of the form

(6)    

 

 

where Y_t, X_t and e_t are as in (1), but now the mean rate α_t and elasticity β_t are assumed to vary over time. In this case, models for the evolution of α_t and β_t are needed in order to use (6) for forecasting. It is also possible that seasonal factors will need to be included in (6) in the case of monthly or quarterly data. The analysis of such a model would directly address many of the recommendations made in Schoefisch (2005).

In terms of growth rates, (6) becomes

(7)    

 

 

where, as in (2), the ε_t correspond to additive errors with zero mean. In practice, the mean rate α_t and elasticity β_t are likely to evolve smoothly over time and so Δlogα_t, Δβ_t will typically be small, or have small variance, relative to the other sources of variation. Such considerations lead to modelling log α_t as a stochastic trend within a suitable structural time series framework. See Harvey (1989) for a full discussion of this general class of models.

Many other variations of (6) are possible using different tax-base proxies where X_t is replaced by geometric combinations of one or more macroeconomic regressors and their lagged values. An example is

 

 

 

which allows the tax-base proxy to be a moving geometric combination of current and past values of the macroeconomic driver X_t. This model can also be framed in terms of growth rates and, in this case, a long-run co-integrating relationship between log Y_t and log X_t may be needed. Alternatively, these macroeconomic drivers could be replaced by lagged values of Y_t or some other tax revenue series. A simple example is

 

 

 

where the α_t are assumed to be evolving smoothly over time and the log e_t are white noise errors. If |β| Y_t reduces to a conventional time series trend plus additive error where the latter is a first-order autoregression. If β = 1, then the tax revenue growth rates ΔlogY_t follow a trend plus error model. This simple model is readily generalised to include other more complex time series models.

In short, the general model (6) provides a flexible modelling framework for forecasting tax revenues and their growth rates, either in terms of suitable macroeconomic drivers and their lags that are proxies for the tax-base, or time series models involving just the tax revenue alone, or a combination of both.

4  Forecast error decompositions#

Print this section

A better understanding of the source and nature of the Treasury's tax revenue forecasting errors is an important prerequisite to building more accurate and robust tax forecasting models. To this end, and as recommended in Schoefisch (2005), we now develop decompositions of the Treasury's past tax revenue forecasting errors into suitable structural components.

The decompositions considered include:

• the disaggregation of total tax revenue forecast errors into their component tax types (Section 4.2);
• the decomposition of individual tax revenue forecast errors into a component due to forecasting the macroeconomic variables that are a proxy for the tax-base used, and a component due to forecasting the ratio of tax revenue to proxy tax-base, or tax ratio (Section 4.3);
• a further decomposition (Section 4.4) of the tax ratio into a trend measuring an underlying mean tax rate and a random error component. The former provides a benchmark against which tax ratio forecasts can be benchmarked and the latter is typically non-informative noise that is predicted only by its mean.

As discussed in Section 2.1, the Treasury's tax revenue forecasts are typically variations of simple multiplicative models that project tax revenue forward at the same rate as the growth forecast of the macroeconomic aggregate that serves as a proxy tax-base. As might be expected, within each tax type no one method has been used consistently and, instead, the Treasury's methods have been refined and modified over time. In addition, any forecasts produced by the methodology described in Section 2.1 are further modified by judgemental factors, both at the individual tax level by the Treasury’s tax forecasting unit, and subsequently at an aggregate level by an internal review panel of senior Treasury staff. As a consequence, the Treasury's tax revenue forecasting models and processes cannot be replicated exactly.

These considerations have led us to consider a simple benchmark model for each tax type that facilitates the decompositions referred to above. Although based on similar tax-base proxies, these models are not the same as the Treasury models, but do have the virtues of transparency, since they have a simple structural interpretation, and consistency over time. The benchmark model provides a structural decomposition of the individual tax revenues against which the Treasury's forecasts can now be assessed.

4.1  Benchmark model#

Print this section

The general structure of taxation suggests the simple model

(8)    

 

 

which is a special case of (6) with unit elasticity β_t = 1. Here the macroeconomic variable X_t is to be regarded as a proxy for the relevant tax-base of the tax concerned, the tax ratio R_t = Y_t/X_t is the observed ratio of tax revenue to proxy tax-base, and the multiplicative errors e_t have unit mean so that α_t can be interpreted as an underlying mean tax rate. A simpler version of this model was used in O'Neill (2005) to decompose tax forecasting errors into suitable components with X_t set at Canada's nominal GDP.

The systematic component α_t is assumed to evolve smoothly over time to accommodate minor policy changes whose effects are phased in gradually over time, and also any discrepancies between the proxy tax-base X_t and the underlying true tax base. To maintain this assumption, it is possible that any abrupt one-off changes will need to be accounted for by prior adjustments made to the data. See Section 5.

With these assumptions and caveats in mind, the multiplicative model (8) can now be transformed into the additive model

(9)    

 

 

where log α_t is a trend and the ε_t = log e_t will be assumed to be stationary white noise, independent of α_t. This simple model for log R_t belongs to the general class of structural time series models discussed in Harvey (1989).

Decompositions of the Treasury's tax forecasting errors can now be undertaken using this simple structural model and its components as a benchmark.

4.2  Decomposition of total tax revenue by tax type#

Print this section

Consider the case where the total tax revenue is denoted by Y(t) and the component revenues by Y_j(t) (j=1,…,m) so that

    

 

 

and the Y_j(t) follow models of the form (8). Given forecasts Ŷ_j(t) of the individual components Y_j(t), a forecast of the aggregate Y(t) is given by

 

 

 

yielding the forecast error decompositions

    

 

 

and, in terms of proportionate errors,

 

 

 

where

 

 measures Y_j(t) as a proportion of the total tax revenue Y(t).

 

Using the approximation (3), the latter decomposition can now be framed in terms of logarithms to yield

(10)    

 

 

where

(11)    

 

 

with the e(Y_j(t)) defined similarly. As before, the quality of the approximation is such that these errors can be interpreted as simple proportionate errors. Note that the forecast errors defined have the opposite sign to those more commonly adopted (logY(t) – log Ÿ(t) for proportionate errors and Y(t) – Ÿ(t) for actual errors). However definition (11) allows for more natural interpretations with positive errors implying over-forecasting and negative errors implying under-forecasting.

In Section 6.1, it is shown that the P_j(t) evolve slowly and vary little over time by comparison to the e(Y_j(t)). In this case, the proportionate forecast error for the total tax revenue Y(t) has mean-squared error given by

(12)    

 

 

where

    

 

 

 

    

 

 

with var[.], cov[.], E[.] denoting variance, covariance and expectation respectively.

The decompositions (10) and (12) provide an appropriate framework for evaluating the relative contributions of the various tax forecasting errors to both individual tax components and their aggregates. Note that these particular decompositions are not dependent on the benchmark model (8).

4.3  Separating out the macroeconomic forecast errors#

Print this section

Consider the benchmark model (8) where, as before, Y_t denotes a particular tax revenue, the macroeconomic variable X_t is a proxy for the tax-base, and R_t is the associated tax ratio. Given forecasts Ÿ_t,

 

 of Y_t, X_t respectively, a natural forecast of R_t is given by

 

(13)    

 

 

so that the three forecasts satisfy the simple relationship

(14)    

 

 

If R_t, X_t are independent or, more generally, if they are conditionally uncorrelated given past data, then the best predictors of Y_t, R_t and X_t will satisfy (14). These and other considerations lead us to assume that (14) holds for the forecasts considered in this report so that R_t can be forecast by the simple predictor (13).

From the multiplicative relationships (8) and (14) we obtain

    

 

 

or, using the notation introduced in (10),

(15)    

 

 

where these quantities are the proportionate forecast errors for each component. Note that (15) additively decomposes the total tax revenue proportionate forecast error e(Y_t) into two component proportionate errors, one due to forecasting the tax ratio R_t and the other the macroeconomic variable X_t used as the tax-base proxy.

Multiplying (15) by Y_t and using the approximation (3) yields the actual forecast error decomposition

    

 

 

which shows the influence of the respective errors in absolute terms.

The mean-squared proportionate forecast error of Y_t is given by

(16)    

 

 

where

    

 

 

    

 

.

 

This decomposition and (15) provide a suitable framework for separating out the forecast errors for the tax ratio R_t from those of the macroeconomic tax-base proxy X_t. They can also be used in conjunction with decompositions (10) and (12) to examine the relative contributions of the various tax-ratio forecasting errors.

4.4  A model-based decomposition of tax forecast errors#

Print this section

Here we consider the tax ratio R_t given by the benchmark model (8). Since α_t has been assumed to be independent of ε_t and e_t = exp(ε_t) has unit mean, the best forecast of R_t will always be the same as the best forecast of α_t. In essence, the benchmark model decomposes R_t into a structural forecastable component α_t and a non-informative noise component e_t. These considerations lead us to assume, in addition to (14), that the forecasts considered in this report satisfy

(17)    

 

 

where

 

 are the forecasts of R_t and α_t respectively.

 

Now (6) and (17) yield the decomposition

    

 

 

or

(18)    

 

 

where these proportionate error components are defined in the same way as before and n_t = –ε_t is non-systematic white noise error. Here the mean-squared proportionate forecast error of R_t is given by

(19)    

 

 

where, to a good approximation,

    

 

 

    

 

 

provided E(ε_t) is close to zero and the forecasts

 

 are closely correlated to the optimal forecasts of R_t.

 

These decompositions can now be used to determine the relative contributions, within tax revenue types, of the proportionate forecast errors of the tax ratio trend α_t and, just as importantly, the nature and size of the non-informative noise components ε_t. They can also be used in conjunction with the decompositions given in the previous sections to better understand the inter-relationships between the various tax types and their error components.

5  Data#

Print this section

Tax forecast data have been taken from the various Budget Update and December/Half-Year Update publications produced by the Treasury since the 1994 Budget. The data were collected from spreadsheets stored on Treasury computers and cover all of the tax types forecast by the Treasury, the Ministry of Transport (road user charges and motor vehicle licensing fees) and the Ministry of Economic Development (energy/exhaustible resource levies).

The Treasury has maintained a database of forecasts of macroeconomic variables ever since the 2001 Budget and this database was used as a primary source. Forecasts of macroeconomic variables prior to the 2001 Budget were collected from macroeconomic and tax forecasting spreadsheets stored on Treasury computers. The data collected were restricted to the primary nominal macroeconomic variables used in the Treasury’s tax forecasting models such as GDP, compensation of employees, consumption, operating surplus and entrepreneurial income.

Up until the end of 1999, the Treasury produced tax and macroeconomic forecasts for the current year and the following three years. From 2000 onwards, this was extended to include a fourth year. Although data across all these forecast horizons were collected, the analysis undertaken focuses on one-year-ahead forecasts, since these are the forecasts against which tax outcomes are measured by the New Zealand Parliament.

Actual tax outcomes are calculated by the Treasury each month and published on the Treasury's website. The macroeconomic data outcomes used in the analysis were the latest available from Statistics New Zealand.

5.1  Data issues#

Print this section

The historical tax and macroeconomic data analysed were, for the most part, final outcomes rather than unrevised estimates. However, the corresponding forecasts were often based on unrevised data available at the time or, in some cases, were prepared in advance of significant policy changes. To correct for these and other such effects, a number of prior adjustments were made to the forecasts. Details are given below.

Policy changes

Tax forecast data have been adjusted for policy changes that affected the final outcomes, but which were not known about at the time the forecast was made. This is to ensure that the forecasts, and the actual tax outcomes they are being compared with, were prepared on the same policy basis.

For example, PAYE forecasts prepared for the 1995 Budget have been adjusted for the personal income tax rate reductions of July 1996 and July 1997 that were announced in December 1995. In the 1997 Budget, the July 1997 personal income tax rate reductions were deferred until July 1998, so forecasts prior to that have also been adjusted for this deferral.

The adjustments used are the actual policy costings that were available at the time the new policy was announced. We have decided to use these adjustments, rather than recalculate the actual effect of the policy change, as these adjustments are the closest we can get to the actual adjustments that the Treasury’s tax forecasters would have made had the new policy been known about at the time of forecasting.

Macroeconomic data imputation

As previously mentioned, some of the earlier forecasts of macroeconomic variables were collected from macroeconomic and tax forecasting spreadsheets.  Although we cannot be absolutely sure that these were the final published macroeconomic forecasts, in all cases the tax forecasts in these spreadsheets matched the final tax forecasts made and so it seems reasonable to assume that these spreadsheets also contained the final macroeconomic forecasts.

Some of the forecasts of macroeconomic variables were stored as growth rates rather than as levels. We have converted these forecast growth rates into forecast levels using base level outcomes that were known at the time, but have no way of knowing how close these reconstructed forecasts are to the actual final macroeconomic forecasts of levels. Nevertheless, any discrepancies introduced are likely to be small.

Data revisions

Macroeconomic data available from Statistics New Zealand are subject to revision. For example, initial estimates of nominal GDP for the year to June 1999 were around $100 billion, whereas the latest estimate is some 5% higher than this at around $105 billion. For the most part, revisions to nominal GDP in the period under examination have been upward, although this is not necessarily so for all of the components of GDP.

The Treasury's macroeconomic forecasts typically apply forecast growth rates to the historical macroeconomic data available at the time, the most recent of which will often be an unrevised or partially-revised estimate. To ensure that the analysis is not unduly influenced by such data revisions, each macroeconomic forecast was multiplied by the ratio of the most recent macroeconomic estimate to the unrevised or partially-revised estimate available at the time of making the forecast. This simple correction factor is based on the assumption that the macroeconomic forecast is, at least approximately, the product of the last available value of the macroeconomic variable and a forecast growth factor.

Adjusting the macroeconomic forecasts by scaling in this way is less than perfect since the forecasts may not be scale-invariant. For example, the composition of nominal GDP has been revised over the intervening period, something that scaling does not necessarily account for. Perhaps the best way to adjust the economic forecasts for subsequent data revisions would be to repeat each forecast using the latest data. This would mean using the same forecasting models, the same forecasters and replicating the judgmental processes that were used at the time. Such a task would be time-consuming and costly and may not lead to better adjustments. Scaling has the advantages of being quick, simple and transparent.

Tax revenues are not subject to revision and so their forecasts did not have to be adjusted. However, the forecasts of the tax ratios R_t were implicitly adjusted since they are based on (13) which is the ratio of the tax revenue forecast to the adjusted forecast of its associated macroeconomic variable or tax-base proxy.

Data mis-alignment

The Crown's accounts are prepared on a Budget year, ie June year, basis. Thus tax revenues are reported as June year totals and tax revenue forecasts are prepared accordingly. However, some macroeconomic variables are forecast solely in March years. In these cases, the March year forecasts and actual outcomes have been used in the analysis and no attempt has been made to correct for any temporal mis-alignment.

6  Analysis#

Print this section

We focus primarily on the one-year-ahead tax revenue forecasts prepared by the Treasury for the annual May or June Budget since it is these forecasts and their associated forecast errors that are subject to the greatest scrutiny. A top-down analysis of the tax revenue forecast errors was undertaken using the decompositions developed in Section 4. The same analysis was also undertaken for the other forecasting horizons with larger errors overall that increase with forecast horizon as expected. However, apart from scale, the results are very similar.

The total tax revenue forecast error was first disaggregated into its component tax types using the decompositions of Section 4.2. In particular, the proportionate error decomposition (10) is given by

(20)    

 

 

where e(Y(t)) is the proportionate forecast error for the total tax revenue and the e(Y_j(t)) are the proportionate forecast errors for the individual tax types. The tax share P_j(t) measures the tax revenue for tax type j as a fraction of the total tax revenue Y(t).

Figure 1 – Tax revenues as a percentage of total tax revenue

PAYE (red), GST (green), corporate tax (blue), net other persons tax (cyan) and other taxes (magenta)

 

/>

Source: The Treasury

Plots of the P_j(t) expressed as percentages are given in Figure 1 where it can be seen that they evolve slowly and smoothly over time as expected. In general, it is evident that the last available value of any tax share P_j(t) should provide an excellent one-year-ahead forecast of P_j(t+1). The averages of the tax shares P_j(t) are given in Table 1 and indicate that PAYE (37%) and GST (26%) are the largest tax revenues with the remaining tax revenues each less than 20% of total tax revenue in the period under examination (June years 1995 through to 2005).

Source: The Treasury

Each individual tax revenue proportionate forecast error e(Y_j(t)) was then further decomposed into a component due to forecasting X_j(t), the macroeconomic variable used as a proxy for the associated tax-base, and a component due to forecasting the tax ratio

 

. Using (15), this decomposition is given by

 

(21)    

 

 

where, as before, the e(.) denote proportionate forecast errors of the components concerned.

Finally, the proportionate forecast error e(R_j(t)) of each individual tax ratio is further decomposed into an error in forecasting the systematic tax ratio trend, and a non-systematic error. This decomposition based on (18) is given by

(22)    

 

 

where e(α_j(t)) is the proportionate forecast error for the tax ratio trend α_j(t) and n_j(t) is the non-systematic random error. The latter provides a measure of the best accuracy that can be achieved using the benchmark models adopted.

Note that the additive nature of the decomposition (20) implies that it is the weighted errors

 

,

 

and

 

,

 

, rather than their unweighted forms, that contribute to the proportionate forecast error for the total tax revenue. Because of their importance, we refer to these as weighted proportionate forecast errors in what follows.

 

Fitting the benchmark model to the tax ratio data R_j(t) entails estimating the unobserved trends α_j(t). Many trend estimates are possible. We have chosen to use the trend estimate proposed by Hodrick and Prescott (1997) for the identification of business cycles, the so-called Hodrick-Prescott filter, which fits a trend to all the available data points. No attempt was made to optimise the smoothing parameter λ of this trend filter and the same value as that given in Hodrick and Prescott (1997) was used (λ =1600). In general, this procedure worked well and gave trend estimates that were smooth and ran through the middle of the data leaving residuals that were, for the most part, well-approximated by uncorrelated random errors with zero mean and common variance (non-systematic white noise).

The Hodrick-Prescott trend filter cannot be used directly for forecasting future values of the unobserved trends α_j(t). However, it and other more general filters such as the Kalman filter, can be underpinned by parametric stochastic trend models that can be used to forecast future values of α_j(t). See Harvey and Jaeger (1993).

In the following sections, these forecast error decompositions are applied and a limited statistical analysis undertaken. Since the data runs from 1995 to 2005 inclusive, only 11 observations are available for any one series and so the statistical results obtained are at best indicative.

6.1  Total tax revenue decomposition#

Print this section

Figure 2 – Forecast errors for total tax revenue and major tax types

Percentage forecast errors (top row), weighted percentage forecast errors (middle row) and actual forecast errors (bottom row) for total tax revenue (black), PAYE (red), GST (green), corporate tax (blue), net other persons tax (cyan) and other taxes (magenta). Time series plots are given on the left and boxplots on the right.

 

Source: The Treasury

The results of applying decomposition (20) are given in Figure 2. For each tax revenue as well as total tax revenue, the percentage forecast errors are plotted as well as the weighted percentage forecast errors and the actual forecast errors. Note that, in this case, the weighted percentage forecast errors are just the actual forecast errors expressed as a percentage of the total tax revenue Y(t). This simple interpretation follows from the relation

    

 

 

where Ÿ_j(t) is the forecast of Y_j(t) and the tax share

 

.

 

The boxplots show the marginal distributions of the various forecast errors with the medians (middle bars of the boxes) located centrally between the quartiles (ends of the boxes) for some, but not all, of the tax revenues indicating symmetric distributions. In particular, corporate tax and other taxes would appear to have negatively skewed distributions. The notches give an approximate 95% confidence interval for the true median of the distribution of forecast errors concerned and so give an indication of whether the forecasts are biased. Adopting this criterion suggests that all tax revenue forecasts are biased downwards (negatively biased forecast errors) with the exception of corporate tax and net other persons tax, although the evidence is marginal in some cases. The whiskers (bars extending from the boxes) indicate the range of the data.

Although individual percentage forecast errors can be quite large (corporate tax and net other persons tax forecast errors are good examples), their effect on the total tax revenue error is moderated by their tax shares P_j(t). The weighted percentage forecast errors reflect the errors that do directly impact on the total tax revenue error. These present quite a different picture and show that, while corporate tax is still clearly the largest source of errors, it is followed by PAYE and GST with the other taxes (net other persons tax in particular) now playing a more minor role. This illustrates the importance of considering the weighted percentage forecast errors.

Apart from scale, the actual forecast errors displayed in both the boxplots and the time series plots differ very little from those of the weighted percentage forecast errors. This suggests that little is lost by focussing on just the weighted percentage forecast errors and the percentage forecast errors. Within any individual tax revenues, it will be sufficient to consider just the percentage forecast errors since the P_j(t) are approximately constant.

The time series plots of the various tax forecast errors indicate that some of them may be serially correlated. The lag one autocorrelations were calculated in each case for the weighted percentage errors for each tax type and these are given in Table 2 together with summary statistics such as the mean forecast error (bias), the forecast error standard deviation and the root mean squared forecast error (RMSE). The square of the latter is the mean squared error (MSE) which is the sum of the forecast error variance and the squared bias. The bias, standard deviation and RMSE values reflect what has already been seen and commented on in the boxplots. The Durbin-Watson test statistics indicated that the weighted percentage forecast errors for total tax revenues, PAYE and corporate tax have significant lag one autocorrelations indicating serial correlation. However the limited number of observations available makes these results marginal.

Source: The Treasury

The contemporaneous correlations between the weighted percentage forecast errors for the individual tax revenues are given in Table 3. These are, on the whole, not significant with the exception of the correlation between the weighted percentage forecast errors for PAYE and other taxes, and possibly PAYE and corporate tax. These associations, if present, may be to do with the macroeconomic variables used as tax-base proxies in each case, or may be related to other causes. These issues should become clearer when the percentage forecast errors for the individual tax revenues are decomposed further.

Table 3 – Contemporaneous correlations for the weighted percentage forecast errors

Source: The Treasury

The lag one cross-correlations of the weighted percentage forecast errors were also calculated and indicated that the forecast errors for GST led those of both PAYE and corporate tax. Again, however, these results are marginal and may have more to do with the macroeconomic variables used.

6.2  Individual tax revenue decompositions#

Print this section

In the following subsections, we apply decompositions (21) and (22) to the percentage forecast errors of each tax revenue including the total tax revenue. Within any individual tax revenue, it is sufficient to consider just the percentage forecast errors, rather than the weighted percentage forecast errors, since the tax shares P_j(t) are approximately constant over time t.

6.2.1  Total tax revenue

Although total tax revenue is not forecast directly, but is constructed indirectly by summing the forecasts of each tax type, it may still be useful to disaggregate the total tax revenue forecast errors with respect to nominal GDP, since the tax-to-GDP ratio is something that is typically focussed on at each forecasting round. Such a model is also discussed in Schoefisch (2005) and O'Neill (2005). This simple model provides a direct forecast against which to benchmark the indirect forecast obtained by aggregation and, ideally, the two forecasts should be combined to give a better forecast overall. See Granger (1989) for a review of the advantages of combining forecasts.

 

 

Figure 3 – Total tax revenue and nominal GDP

The top plots show total tax revenue (solid black) and its forecast (dashed black), scaled GDP (solid red) and its forecast (dashed red), the associated tax ratio (solid blue) and its forecast (dashed blue), and the tax ratio trend (solid green). The remaining time series plots and boxplots show the percentage forecast errors due to forecasting total tax revenue (black), GDP (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

 

 

The results of applying decompositions (21) and (22) are shown in Figure 3 together with total tax revenue and its forecast, GDP and its forecast, the associated tax-to-GDP ratio and its forecast, and the tax ratio trend. Evidently, the forecasts of total tax revenue and GDP have both underestimated the actual outcomes consistently from 2000. Moreover, the tax ratio forecasts appear to be overestimating actual outcomes when the tax ratio trends downwards, and underestimating actual outcomes when it trends upwards.

The boxplots in Figure 3 show that, for decomposition (21), GDP forecast errors are significantly biased downwards and it is these that are contributing to the predominantly negative percentage forecast errors for total tax revenue. The tax ratio percentage forecast errors do not appear to be biased, but have higher volatility (standard deviation) than the GDP percentage forecast errors. For decomposition (22), the boxplots indicate that the percentage forecast errors due to forecasting the tax ratio trend may be positively biased. The statistical measures of bias, standard deviation and RMSE given in Table 4 also support these observations. In particular, the relative sizes of the RMSE values for the tax ratio percentage forecast errors and those of the non-systematic error component suggest that there are forecast gains to be had, even with this simple model.

Total tax revenue Y_t and its components: GDP X_t, associated tax ratio R_t, tax ratio trend α_t and residual e_t

Source: The Treasury

The plots of the time series in Figure 3 suggest that some may be serially correlated. The results of applying decompositions (21) and (22) are shown in Figure 3 together with total tax revenue and its forecast, GDP and its forecast, the associated tax-to-GDP ratio and its forecast, and the tax ratio trend. Evidently, the forecasts of total tax revenue and GDP have both underestimated the actual outcomes consistently from 2000. Moreover, the tax ratio forecasts appear to be overestimating actual outcomes when the tax ratio trends downwards, and underestimating actual outcomes when it trends upwards.

The boxplots in Figure 3 show that, for decomposition (21), GDP forecast errors are significantly biased downwards and it is these that are contributing to the predominantly negative percentage forecast errors for total tax revenue. The tax ratio percentage forecast errors do not appear to be biased, but have higher volatility (standard deviation) than the GDP percentage forecast errors. For decomposition (22), the boxplots indicate that the percentage forecast errors due to forecasting the tax ratio trend may be positively biased. The statistical measures of bias, standard deviation and RMSE given in Table 4 also support these observations. In particular, the relative sizes of the RMSE values for the tax ratio percentage forecast errors and those of the non-systematic error component suggest that there are forecast gains to be had, even with this simple model.

Table 4 gives the lag one autocorrelations of the various percentage forecast errors together with other summary statistics. There appears to be significant lag one autocorrelation in the percentage forecast errors due to forecasting the tax ratio trend, possibly an artefact of the effect mentioned earlier, and a suggestion of significant lag one autocorrelation in the percentage forecast errors for total tax revenue and the residual.

Each forecast error decomposition (21) and (22) led to two additive components whose cross-correlations were not significantly different from zero. A marginal exception was GDP and the tax ratio where the percentage forecast errors of the former appeared to lead the latter by one year, perhaps indicating that lagged GDP might be a better tax-base proxy. The tax ratio trend provides a good fit to the data suggesting that the benchmark model is reasonable.

6.2.2  Source deductions (PAYE)

As before, the simple abbreviation PAYE is used to denote total source deductions, where approximately 97% of total source deductions are PAYE deductions from salaries, wages and social assistance benefits, and the remainder are from employer contributions to registered superannuation schemes. Here the macroeconomic variable used as a tax-base proxy for the PAYE forecast is compensation of employees (COE).

Figure 4 – Source deductions (PAYE) and compensation of employees (COE)

The top plots show PAYE (solid black) and its forecast (dashed black), scaled COE (solid red) and its forecast (dashed red), the associated tax ratio (solid blue) and its forecast (dashed blue), and the tax ratio trend (solid green). The remaining time series plots and boxplots show the percentage forecast errors due to forecasting PAYE (black), COE (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

The results of applying decompositions (21) and (22) are shown in Figure 4 together with PAYE and its forecast, COE and its forecast, the associated tax-to-COE ratio and its forecast, and the tax ratio trend. Forecasts of PAYE and COE have both underestimated actual outcomes over the last 4 years (5 years for COE). The tax ratio trend provides a good fit to the tax ratios with a generally downward trend in the late 1990s, when some personal tax rates were reduced and income thresholds were increased, and a slight increase more recently, with the introduction of the 39% tax rate.

The boxplots in Figure 4 show that, for decomposition (21), COE forecast errors are significantly biased downwards and it is these that are contributing to the same significant bias of the forecast errors for PAYE. The tax ratio percentage forecast errors, although more often negative than positive, do not appear to be significantly biased and are less volatile than the COE percentage forecast errors. For decomposition (22), the boxplots show few signs of bias, but the percentage forecast errors due to forecasting the tax ratio trend are more volatile than those of the non-systematic error component. The personal tax cuts of 1 July 1996 and 1 July 1998 have also introduced additional volatility in these components over the period 1996 through to 1999. These observations are supported by the summary statistics given in Table 5.

PAYE revenue Y_t and its components: compensation of employees X_t, associated tax ratio R_t, tax ratio trend α_t and residual e_t

Source: The Treasury

The lag one autocorrelations of the various components given in Table 5 are, for the most part, not significantly different from zero. The exception is the residual component which has a significant negative correlation, most likely due to the alternating pattern over the 1996-1999 period that resulted from the personal tax cuts of 1 July 1996 and 1 July 1998.

The components of decomposition (21) were not significantly cross-correlated, but there was a significant negative correlation between the components of decomposition (22) at lag zero with the other lags being non-significant. This contemporaneous correlation is likely to be an artefact of the personal tax cuts of 1 July 1996 and 1 July 1998.

6.2.3  Goods and services tax (GST)

GST is a broadly-based value-added tax applied to most goods and services consumed within New Zealand. Forecasting models for GST have ranged from using many macroeconomic drivers to just a few. The main macroeconomic driver used in all the GST forecasts is nominal consumption, so this was the variable used in the benchmark model and the associated forecast error decompositions.

Figure 5 – Goods and services tax (GST) and nominal consumption

The top plots show GST (solid black) and its forecast (dashed black), scaled nominal consumption (solid red) and its forecast (dashed red), the associated tax ratio (solid blue) and its forecast (dashed blue), and the tax ratio trend (solid green). The remaining time series plots and boxplots show the percentage forecast errors due to forecasting GST (black), nominal consumption (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source:The Treasury

The decompositions (21) and (22) are shown in Figure 5 together with GST and its forecast, nominal consumption and its forecast, the associated tax ratio and its forecast, and the tax ratio trend. Forecasts of GST and nominal consumption have been in relatively close agreement with actual outcomes up until 2001, but both have underestimated actual outcomes from 2001. The tax ratio trend provides a good fit to the tax ratios, despite their rapid increase over the 2002-2004 period.

Since GST is a flat-rate tax, we would expect the tax ratio and its trend to be relatively constant and, if nominal consumption were a good proxy for the GST tax base, this constant should be 11.1%. However, as noted above, after maintaining a value around 11% until 2001, the tax ratios have moved from 11.1% in 2002 to 11.6% in 2005. Evidently, nominal consumption is an imperfect proxy for the GST tax base. This is because some components of nominal consumption are not subject to GST, such as housing rentals, and there are items that are subject to GST that do not form part of nominal consumption, such as new dwelling construction costs. The latter is a likely candidate for the up-swing in the tax ratios seen here. Residential investment has experienced something of a boom in New Zealand over the last few years, which has increased the GST take, but has not increased the nominal consumption base as measured by Statistics New Zealand.

The boxplots in Figure 5 show that, for decomposition (21), the forecast errors for nominal consumption are significantly biased downwards and it is these that are contributing to the same significant bias of the GST forecast errors. The tax ratio percentage forecast errors are not biased, but are more volatile than the percentage forecast errors for nominal consumption. For decomposition (22), the boxplots show no signs of bias, but the percentage forecast errors due to forecasting the tax ratio trend are approximately twice as volatile as those of the non-systematic error component, implying that there are forecast gains to be had, even with the simple benchmark model used here. These observations are supported by the summary statistics given in Table 6.

Table 6 – Summary statistics for GST revenue percentage forecast errors

GST revenue Y_t and its components: nominal consumption X_t, associated tax ratio R_t, tax ratio trend α_t and residual e_t

Source: The Treasury

The lag one autocorrelations of the various components shown in Table 6 are not significantly different from zero. In addition, the components of each decomposition (21) and (22) showed no evidence of significant cross-correlation at all lags.

6.2.4  Corporate tax

This is the sum of net company income tax, non-resident withholding tax (NRWT) and foreign dividend withholding payments (FDWP). We include these other two withholding taxes in the definition of corporate tax as companies will typically get a credit towards their income tax for at least some of any NRWT or FDWP paid. A variety of forecasting models has been used over the years to forecast corporate tax. The macroeconomic variable at the heart of all of these is operating surplus and so this was the variable used in the benchmark model and the associated forecast error decompositions.

 

 

Figure 6 – Corporate tax and operating surplus

The top plots show corporate tax (solid black) and its forecast (dashed black), scaled operating surplus (solid red) and its forecast (dashed red), the associated tax ratio (solid blue) and its forecast (dashed blue), and the tax ratio trend (solid green). The remaining time series plots and boxplots show the percentage forecast errors due to forecasting corporate tax (black), operating surplus (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

 

 

The decompositions (21) and (22) are shown in Figure 6 together with corporate tax and its forecast, operating surplus and its forecast, the associated tax ratio and its forecast, and the tax ratio trend. Forecasts of corporate tax have tended to overestimate actual outcomes in the 1995-2000 period and underestimate actual outcomes in the 2001-2005 period. Forecasts of operating surplus have tended to underestimate actual outcomes over the entire 1995-2005 period. The tax ratio trend provides a good fit to the tax ratios, despite their rapid increase over the 2002-2005 period. Note that the tax ratio forecasts appear to be overestimating actual outcomes when the tax ratio trends downwards, and underestimating actual outcomes when it trends upwards.

Company income tax, NRWT and FDWP are all levied at fixed tax rates, and so we might expect the tax ratio and its trend to be relatively constant. The resulting average tax rate can fluctuate owing to the utilisation of tax losses, the claiming of tax credits and the timing of revenue recognition, all of which can vary a great deal from year to year. However, none of these adequately explains the upward trend in the tax ratio and its trend in recent years. As in the case of GST, the tax-base proxy adopted (operating surplus) appears to be less than perfect. One possible explanation might be that investment returns, while taxable, do not form part of the economic measure of operating surplus.

The boxplots in Figure 6 show that, for decomposition (21), the forecast errors for operating surplus are significantly biased downwards and the tax ratio forecast errors are, if anything, biased the other way. As a consequence, corporate tax percentage forecast errors show no evidence of bias. However the volatility of the tax ratio percentage forecast errors is considerably greater than (more than twice) that of the operating surplus percentage forecast errors, and this is the primary source of the considerable volatility present in the corporate tax percentage forecast errors. The decomposition (22) of the tax ratio percentage forecast errors also highlights the inaccuracy of the tax ratio forecasts as forecasts of the tax ratio trend by comparison to the volatility of the non-systematic error component. Better tax ratio forecasts are needed and could be achieved, even with the simple benchmark model used here. These observations are supported by the summary statistics given in Table 7.

Table 7 – Summary statistics for corporate tax revenue percentage forecast errors

Corporate tax revenue Y_t and its components: operating surplus X_t, associated tax ratio R_t, tax ratio trend α_t and residual e_t

Source: The Treasury

Of the lag one autocorrelations given in Table 7, only that for the percentage forecast errors due to forecasting the tax ratio trend α_t is significantly different from zero and this reflects the pattern of the tax ratio forecasts discussed earlier. In addition, the components of each decomposition (21) and (22) showed no evidence of significant cross-correlation at all lags.

6.2.5  Net other persons tax

This is tax on personal income that is not taxed at source. It includes income tax from self-employed people, trusts, clubs, societies and Maori authorities, together with tax on investments that are not already taxed at source. Again, a variety of forecasting models has been used at various times, drawing on a range of macroeconomic variables. The one variable that has been used in all cases is entrepreneurial income and this was used in the benchmark model and the associated forecast error decompositions.

Figure 7 – Net other persons tax and entrepreneurial income

The top plots show net other persons tax (solid black) and its forecast (dashed black), scaled entrepreneurial income (solid red) and its forecast (dashed red), the associated tax ratio (solid blue) and its forecast (dashed blue), and the tax ratio trend (solid green). The remaining time series plots and boxplots show the percentage forecast errors due to forecasting net other persons tax (black), entrepreneurial income (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

The results of the decompositions (21) and (22) are shown in Figure 7 together with net other persons tax and its forecast, entrepreneurial income and its forecast, the associated tax ratio and its forecast, and the tax ratio trend. Forecasts of net other persons tax have tended to overestimate their actual outcomes whereas forecasts of entrepreneurial income have tended to underestimate actual outcomes. The tax ratio trend provides a reasonable fit to the tax ratios, including the period of tax rate reductions in the late 1990s. However, the tax ratio forecasts almost all overestimate actual outcomes.

Highly significant biases are the dominant feature of the boxplots in Figure 7. For decomposition (21), the forecast errors for entrepreneurial income are significantly biased downwards, and those for the tax ratio are significantly biased upwards. These cancel and lead to forecast errors for net other persons tax that are not significantly biased, but do have increased volatility. Note also that the volatility of the tax ratio percentage forecast errors is greater than the volatility of the percentage forecast errors for entrepreneurial income. For decomposition (22), the boxplots show that the significant bias of the tax ratio percentage forecast errors comes from forecasting the tax ratio trend, as expected. These observations are supported by the summary statistics given in Table 8.

Table 8 – Summary statistics for net other persons tax revenue percentage forecast errors

Net other persons tax revenue Y_t and its components: entrepreneurial income X_t, associated tax ratio R_t, tax ratio trend α_t and residual e_t

Source: The Treasury

The lag one autocorrelations given in Table 8 are not significantly different from zero with the marginal exception of the tax ratio percentage forecast errors which showed negative autocorrelation due to the cycling between errors above and below the bias level. If real, it is unclear what this effect might be caused by. The components of each decomposition (21) and (22) also showed little evidence of any significant cross-correlation.

6.2.6  Other taxes

The previous tax types account for more than 80% of the total tax take. The remainder is made up of resident withholding tax (RWT), excise taxes, customs duty and a few smaller taxes, some of which are not forecast by the Treasury. Most of these taxes do not necessarily depend on any particular component of GDP. For example, most of RWT is dependent on interest rates, and excise taxes are dependent on long-run growth trends. While a large part of customs duty is tariffs on imported goods, and therefore has some relationship with nominal goods imports, about half of customs duty is excise duty on imported petrol, which is a volume-based duty that can be very volatile as it is dependent on the arrival of bulk fuel shipments. Nevertheless, the aggregate of these other taxes has been analysed using the benchmark model with nominal GDP as the macroeconomic driver or tax-base proxy. Since the components of other taxes have, at best, a loose association with GDP, the resulting decompositions may be of limited use.

The decompositions (21) and (22) are shown in Figure 8 together with other taxes and its forecast, GDP and its forecast, the associated tax ratio and its forecast, and the tax ratio trend. Forecasts of other taxes and GDP have both underestimated their actual outcomes over the entire 1995-2005 period except for 1998-1999 when the reverse was true. The tax ratio trend provides a reasonable fit to the tax ratios, despite the sharp decrease in the late 1990s.

For decomposition (21), the boxplots in Figure 8 show that the forecast errors for GDP are significantly biased downwards and it is these that are contributing to the same significant bias of the forecast errors for other taxes. The tax ratio percentage forecast errors are not biased, but are more volatile than the percentage forecast errors for GDP. For decomposition (22), the boxplots show no signs of bias and the percentage forecast errors due to forecasting the tax ratio trend are slightly more volatile than those of the non-systematic error component. These observations are supported by the summary statistics given in Table 9.

Table 9 – Summary statistics for other tax revenue percentage forecast errors

Other tax revenue Y_t and its components: GDP X_t, associated tax ratio R_t, tax ratio trend α_t and residual e_t

Source: The Treasury

The lag one autocorrelations in Table 9 are not significantly different from zero, and the components of each decomposition (21) and (22) showed no evidence of significant cross-correlation.

6.3  Other forecast horizons#

Print this section

Similar analyses were undertaken for other forecasting horizons. Apart from the size of the forecast errors which, as expected, increase in magnitude with increasing forecast horizon, the results obtained were very similar to the one-year-ahead analyses given in the previous sections. An example of these results is given in the Appendix where the equivalents of Figures 2-9 are given for the case of a two-year-ahead forecast horizon.

7  Conclusions#

Print this section

This report presents an analysis of the New Zealand Treasury's tax revenue forecast errors, both in aggregate and disaggregated by individual tax type. The primary objective was to better determine the major sources of tax revenue forecast error and to identify any potential for methodological improvements. Using a simple structural model as a benchmark, the individual tax revenue forecast errors were first disaggregated into a component due to forecasting the associated macroeconomic driver used as tax-base proxy, and a component due to forecasting the tax ratio. The tax ratio is further disaggregated into a component error due to forecasting the tax ratio trend and non-systematic random error. The latter provides a measure of the best accuracy that can be achieved using the benchmark models adopted.

In terms of their contribution to total tax revenue, PAYE (37%) and GST (26%) are the largest followed by corporate tax (15%), net other persons tax (8%) and other taxes (14%). These tax shares or weights were used to scale the percentage forecast errors of each disaggregated tax revenue component to determine the contribution of that component to the total percentage forecast error. The weighted percentage forecast errors for the various tax revenues indicated that all the individual tax revenue forecasts, with the exception of corporate tax and net other persons tax, were significantly underestimating actual outcomes. As a consequence, the total tax revenue was also significantly underestimated.

After checking for bias, the volatility (standard deviation) of the individual forecast errors is a measure of the precision of the forecasting methods used. Large volatilities indicate poor precision and highlight the need for better forecasting models and methods. Here, corporate tax stood out as the tax revenue that was least precisely forecast since its weighted percentage forecast errors had a standard deviation that was almost three times the average of the others. Overall forecasting performance is measured by the root mean squared error (RMSE), or square root of the sum of the variance and squared bias. This showed that corporate tax was clearly the least precisely forecast, followed by PAYE, then GST and other taxes, with net other persons tax having the smallest RMSE.

The primary forecast error decomposition (21) splits the percentage forecast error for each tax type into a component due to forecasting the macroeconomic variable used as a tax-base proxy, and a component due to forecasting the tax ratio. In terms of bias, all the macroeconomic variables used for tax-base proxies significantly underestimated their actual outcomes resulting in percentage forecast errors with significant downward biases. However the corresponding tax ratio percentage forecast errors all showed no significant bias with the exception of net other persons tax, for which the percentage forecast errors had a significant compensating upward bias. This shows that the main source of tax revenue underforecasting is almost certainly the underforecasting of the macroeconomic variables used as tax-base proxies, rather than the tax ratios.

Conversely, the macroeconomic variables yielded percentage forecast errors that were generally clustered relatively closely about their mean, or bias, and were less volatile than those for the tax ratios. This suggests that the tax ratio forecasts, while unbiased, are less precisely determined than the macroeconomic forecasts.

The contributions to the total tax revenue percentage forecast error of the various tax-share weighted components of the primary decomposition (21) are shown in Figure 9. These summarise and provide further graphical support for the comments made above.

Figure 9 – Tax-share weighted percentage forecast errors

Tax-share weighted percentage forecast errors due to forecasting the associated macroeconomic driver (top plots) and tax ratio (bottom plots) for total tax revenue (black), PAYE (red), GST (green), corporate tax (blue), net other persons tax (cyan) and other taxes (magenta). Time series plots are given on the left and boxplots on the right.

 

Source: The Treasury

The secondary forecast error decomposition (22) splits the percentage forecast error for each tax ratio into a component due to forecasting the tax ratio trend, and non-systematic random error. This was done using the benchmark model which provided a good fit to the tax ratios considered and, as a result, yielded reasonable decompositions with components that were, in general, not significantly correlated. With the exception of net other persons tax, all tax types yielded percentage forecast errors for the disaggregated components that showed no significant bias. However, the volatility of the error component due to forecasting the tax ratio trend was almost always greater than that of the non-systematic error component (more than twice in the case of GST and corporate tax) indicating that better tax ratio forecasts are needed and could be achieved, even with the simple benchmark model used here.

Some of the percentage forecast error time series show persistence which suggests that they might be serially correlated. However this was not borne out by the Durbin-Watson test which failed to show significant lag one autocorrelations in most cases, even with nominal autocorrelations as large as 0.5. This is partly due to the small sample size of 11 observations and partly because, in this case, even one anomalous pattern of errors in the time series can destroy any patterns seen in the rest of the time series. In cases where there was significant lag one autocorrelation, it could often be explained in other ways. In particular, if random walk predictors are used when there is a trend cycle present in the data (consider the tax ratio time series for total tax revenue and corporate tax for example) then there will be systematic under-forecasting in times of increasing trends and over-forecasting in times of decreasing trends. Adoption of conservative predictors such as these, if sustained over a reasonable period, can lead to eliminate many of these effects

The simple benchmark models adopted are, at best, approximations to the methods the Treasury currently uses or has used in the past. However, judging from the analysis, they appear to have some merit as competing models, if only because of their simplicity. In particular, the benchmark model with a suitable structural parametric model for the trend of the tax ratios could be used for forecasting the tax ratios. Other time series models without macroeconomic drivers could also be used. Simple forecasts such as these would help calibrate the current Treasury forecasts and could be combined appropriately to achieve a better forecast overall. A systematic evaluation of the forecasting accuracy of a selection of the more formal models suggested in Section 3 using historical data would help resolve some of these issues.

Finally, there is the issue of how to best adjust the forecasts of the macroeconomic data from Statistics New Zealand so that they represent forecasts of fully revised data rather than unrevised data. This is an important general issue that extends beyond just tax forecasting and deserves a separate study and analysis.

Appendix: Plots for two-year-ahead forecast errors#

Print this section

Figure 11 – Total tax revenue and nominal GDP: two-year-ahead forecasts

The top plots show total tax revenue (black), scaled GDP (red), tax ratio (blue) and its trend (green) with solid lines denoting actuals and dashed lines two-year-ahead forecasts. The remaining plots show the percentage forecast errors due to forecasting total tax revenue (black), GDP (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

Figure 12 – PAYE and COE: two-year-ahead forecasts

The top plots show PAYE (black), scaled compensation of employees (red), tax ratio (blue) and its trend (green) with solid lines denoting actuals and dashed lines two-year-ahead forecasts. The remaining plots show the percentage forecast errors due to forecasting PAYE (black), compensation of employees (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

Figure 13 – GST and nominal consumption: two-year-ahead forecasts

The top plots show GST (black), scaled nominal consumption (red), tax ratio (blue) and its trend (green) with solid lines denoting actuals and dashed lines two-year-ahead forecasts. The remaining plots show the percentage forecast errors due to forecasting GST (black), nominal consumption (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

Figure 14 – Corporate tax and operating surplus: two-year-ahead forecasts

The top plots show corporate tax (black), scaled operating surplus (red), tax ratio (blue) and its trend (solid green) with solid lines denoting actuals and dashed lines two-year-ahead forecasts. The remaining plots show the percentage forecast errors due to forecasting corporate tax (black), operating surplus (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

Figure 15 – Net other persons tax and entrepreneurial income: two-year-ahead forecasts

The top plots show net other persons tax (black), scaled entrepreneurial income (red), tax ratio (blue) and its trend (green) with solid lines denoting actuals and dashed lines two-year-ahead forecasts. The remaining plots show the percentage forecast errors due to forecasting net other persons tax (black), entrepreneurial income (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

Figure 16 – Other tax revenue and nominal GDP: two-year-ahead forecasts

The top plots show other taxes (black), scaled GDP (red), tax ratio (blue) and its trend (solid green) with solid lines denoting actuals and dashed lines two-year-ahead forecasts. The remaining plots show the percentage forecast errors due to forecasting other taxes (black), GDP (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury

Figure 17 – Tax-share weighted, two-year-ahead, percentage forecast errors

Tax-share weighted two-year-ahead percentage forecast errors due to forecasting the associated macroeconomic driver (top plots) and tax ratio (bottom plots) for total tax revenue (black), PAYE (red), GST (green), corporate tax (blue), net other persons tax (cyan) and other taxes (magenta). Time series plots are given on the left and boxplots on the right.

 

Source: The Treasury

Figure 10 – Two-year-ahead forecast errors for total tax revenue and major tax types

Percentage forecast errors (top row), weighted percentage forecast errors (middle row) and actual forecast errors (bottom row) for total tax revenue (black), PAYE (red), GST (green), corporate tax (blue), net other persons tax (cyan) and other taxes (magenta). Time series plots are given on the left and boxplots on the right.

 

Source: The Treasury

References#

Print this section

Basu, Suman, Carl Emmerson and Christine Frayne (2003) “An examination of the IFS corporation tax forecasting record.” London, The Institute for Fiscal Studies, WP03/21.

Granger, Clive W.J. (1989) “Combining forecasts - twenty years later.” Journal of Forecasting 8: 167-173.

Harvey, Andrew C. (1989) Forecasting, Structural Time Series Models and the Kalman Filter. (Cambridge University Press).

Harvey, Andrew C. and A. Jaeger (1993) “Detrending, stylized facts and the business cycle.” Journal of Applied Econometrics 8(3): 231-247.

Hodrick, Robert J. and Edward C. Prescott (1997) “Post-war U.S. business cycles: an empirical investigation.” Journal of Money, Credit and Banking 29: 1-16.

Mühleisen, Martin, Stephan Danninger, David Hauner, Kornélia Krajnyák and Ben Sutton (2005) “How do Canadian Budget forecasts compare with those of other industrial countries?” Washington, International Monetary Fund, IMF Working Paper 05/66.

O'Neill, Tim (2005) “Review of Canadian Federal Fiscal Forecasting, Processes and Systems.” Department of Finance Canada

Rich, Robert, Jason Bram, Andrew Haughwout, James Orr, Rae Rosen and Rebecca Sela (2005) “Using regional economic indices to forecast tax bases: evidence from New York.” The Review of Economics and Statistics 87: 627-634.

Schoefisch, Ulf D. (2005) “Examination of the New Zealand Treasury's tax forecasting methods and processes.” Wellington, The Treasury, October.

Figure 8 – Other tax revenue and nominal GDP

The top plots show other taxes (solid black) and its forecast (dashed black), scaled GDP (solid red) and its forecast (dashed red), the associated tax ratio (solid blue) and its forecast (dashed blue), and the tax ratio trend (solid green). The remaining time series plots and boxplots show the percentage forecast errors due to forecasting other taxes (black), GDP (red), tax ratio (blue), tax ratio trend (green) and residual error (cyan).

 

Source: The Treasury