Media statement

Data error prompts process improvements

Statistics New Zealand
The Treasury

The Treasury and Statistics New Zealand are improving their quality assurance processes after data miscalculations affected estimates of disposable income, Treasury Chief Economist Girol Karacaoglu and Statistics New Zealand Deputy Government Statistician Vince Galvin said today.

Estimates of household disposable income (income from all sources after deducting income tax and adding tax credits) were miscalculated for 2007/08 and for 2009/10 to 2011/12 in a way that tended to overestimate the disposable incomes of households on low to middle incomes.

The problem is limited to this particular area of work and has no bearing on other Treasury or Statistics New Zealand work streams or data. Policy advice to the government is unchanged, and there has been no impact on benefit payments or taxation for individuals or households.

"The Treasury and Statistics New Zealand hold themselves to very high standards. We are deeply disappointed to have failed to meet those standards with these data calculation errors," said Dr Karacaoglu.

"The miscalculations were human error. But the main problem was that communication protocols between the Treasury and Statistics New Zealand were inadequate for this data and neither agency had an overview of the complete process to ensure the anomalies, once identified, were properly explained. Both agencies are determined to fix that," said Mr Galvin.

"Statistics New Zealand and the Treasury have changed their quality assurance and communications processes to ensure this problem doesn't happen again. These new processes will be in place for the next round of estimates for 2013, which begin in March."

The impact on two major pieces of work was assessed in detail:

  1. In the Ministry of Social Development's Household Incomes Report, the trends and most of the findings remain broadly valid. The main difference is that the revised figures show that the global financial crisis had a greater one-off impact on household incomes in 2009 than the original data showed. This was especially the case for lower-income households with children in private rental accommodation.
  2. Estimates used by the Tax Working Group in 2009 and by the Treasury in Budget 2010 are unchanged within statistical margins of error. The main impact is that corrected estimates show the increase in the number of children in households with less than 50% of median income after the Budget 2010 tax changes is less than previously advised.

"We recognise that issues of estimates of household disposable income are important and we considered carefully what impact the corrected data may have had on the Treasury's policy advice," said Dr Karacaoglu. "We concluded that that advice would not have changed, as the effects of the miscalculations on the data we used are too small to have altered our judgements. We consulted with the chair of the Tax Working Group, Professor Bob Buckle, who agreed that the impacts were not material and would have been most unlikely to have changed the Tax Working Group's findings.

"It is also important to note that there are no 'real world' impacts on New Zealanders from the miscalculations. There is zero effect on the individual or household benefit payments and tax credits people get, or the tax they pay," said Dr Karacaoglu.

The table below summarises the key miscalculation impacts.

Miscalculation Scale Key statistics affected
Accommodation Supplement was double-counted in estimates of disposable income for 2009/10, 2010/11 and 2011/12. $1.2bn

The largest differences are for families with children. For example:

  1. the revised 2011/12 rate is 14% rather than 12% of children (150,000 rather than 125,000) in households with less than 50% of median income
  2. using the 60% of median after housing costs measure, the revised 2011/12 rate is 27% rather than 25% (285,000 rather than 265,000).

The proportion of all households with a high 'outgoings-to-income ratio' (OTI) increases from 26% to 27%, with lower income households even more likely to have high OTIs, increasing from 38% to 41%.

Working for Families payments were overestimated in Treasury estimates of disposable income in 2007/08. $900m In combination these miscalculations had no statistically significant impact on estimates of the number of households on estimated incomes below 50% and 60% of median income.
 

Approximately 11,000 fewer children were in households below 50% of median income after the Budget 2010 tax changes than previously estimated. No statistically significant change to the 60% measure.

Accommodation Supplement payments were underestimated in Treasury estimates of disposable income in 2007/08. $650m

The estimates of household disposable income are jointly produced by Statistics New Zealand and the Treasury, and involve modelling of Statistics New Zealand's Household Economic Survey (HES) figures by Treasury using its Taxwell micro-simulation model. The Taxwell-generated estimates are then added by Statistics New Zealand to the HES customised dataset before being sent on to the Ministry of Social Development and others. Through the Ministry of Social Development's Household Incomes Report, analysis of this data has become the basis of almost all detailed analysis of income distribution in New Zealand.

The Treasury and Statistics New Zealand have conducted two thorough reviews into the miscalculations, covering:

  1. how the errors occurred, including the internal and co-production processes used by Statistics New Zealand and Treasury
  2. the Taxwell dataset and associated documentation

Both of these reviews have been externally peer-reviewed by Deloitte. All of the reviews are available at http://www.treasury.govt.nz/publications/informationreleases/disposableincome.

The Ministry of Social Development has published on its website corrected figures for the most commonly used statistics, together with a short commentary on the impact of the revised data.

For further information:#

  1. on the revised data, call Colin Marshall at Statistics New Zealand, (04) 931 4630
  2. on the data errors, call Bryan McDaniel at the Treasury, (04) 917 6268
  3. on the Household Incomes Report, see www.msd.govt.nz or call Simon King at the Ministry of Social Development, (04) 916 3447

Media contacts:#

Bryan McDaniel | Principal Communications Advisor, The Treasury
Tel: +64 4 917 6268
Mob: +64 21 817 207
Email: [email protected]

 

Colin Marshall | Acting Manager Strategic Communications, Statistics New Zealand
Tel: +64 4 931 4630
Mob: +64 21 285 9191
Email: [email protected]