The Treasury

Global Navigation

Personal tools

Executive summary

Researchers who use survey data have an interest in understanding how well it compares with administrative data. We explore the difference between people's survey responses to the Household Economic Survey (HES) and administratively recorded information on income and benefit receipt using the Integrated Data Infrastructure (IDI).

The motivation for this analysis is the potential to incorporate administrative data into Treasury's tax and welfare model, which is currently based on the Household Economic Survey (HES) data. It has long been recognised that there are some significant limitations with this approach, in particular, a relatively small sample size and the under-reporting of welfare payments. For some research questions, it is debated which data - survey or administrative - are more appropriate. However, in the case of modelling tax and welfare changes, it can be argued, a priori, that administrative data are more appropriate, given that administrative data better reflect the tax revenue the government actually receives and the benefit expenses the government actually incurs.

83% of adults (aged 15+) in HES can be linked to the IRD data within the IDI. Those that are linked - compared to those unlinked - have higher incomes, are more likely to be male, are more likely to report European ethnicity and have a lower level of reported benefit receipt. Because the linked sample differs quite substantially from the unlinked sample, researchers using only the linked subset should consider reweighting the data to better reflect the characteristics of the New Zealand population.

HES reported total comparable incomes (excluding benefit income) are highly correlated (ρ=0.79) with IRD incomes, although HES incomes are 1.5-6% higher (depending on the year) than the administrative measures.

Benefit receipt in HES compares poorly with the administrative measure. Many people report either HES benefits for which there is no administrative record that they received it or do not report receiving a benefit when an administrative record shows that they received it. On average, people are much more likely to under-report a benefit than over-report a benefit, and this is especially true of the Accommodation Supplement.

Our results indicate that incorporating administrative data on benefit receipt is where the largest gains are likely to be made to Treasury's Taxwell model. Following that, incorporating data from IRD on income is the next logical step. As part of making these changes, the survey weights and population definition will likely need to be changed.

Page top