The Treasury

Global Navigation

Personal tools

1  Introduction

Survey data, such as Statistics New Zealand's Household Economic Survey (HES), are frequently used for research and modelling work, but there are a number of reasons why survey responses may be inaccurate. First, respondents may struggle to remember what they received. Second, respondents may struggle to understand the questions, particularly with social transfers with similar target groups like Supported Living Payment and Disability Allowance. Even when respondents understand the questions, they may make approximations, such as rounding earnings to the nearest $10,000 or rounding time periods to the nearest month. In other cases, respondents may feel stigmatised or feel as if their privacy is invaded if they answer truthfully - for example, when reporting unemployment assistance or very large income items. Lastly, but not exhaustively, some respondents may report deliberately misleading information. These issues are not unique to survey data, and some of these and other issues may affect administrative data. Statistical agencies and researchers are aware of such issues and have ways of managing them both ex ante, during the design of the survey and ex post, in a way suitable for the research question. This paper presents comparisons between survey measures and administrative measures, which allows researchers using the HES survey data to better understand their quality.

"The three main objectives of HES are: to contribute to the reweighting of the consumers price index; to supply expenditure statistics for use in estimating gross domestic product, and to provide an indication of the overall living standards of New Zealanders".[1] Similarly, IRD data and MSD data are collected for administration of the tax and welfare system. Any differences between HES data and administrative sources does not imply that either data source isn't fit for purpose. Nevertheless, it is helpful to understand the nature of such differences.

In 2016, HES data was linked to Statistics New Zealand's Integrated Data Infrastructure (IDI). This has made it possible to compare the reported survey data with the administrative records at the individual level. Previously, HES could only be compared to other data sources at an aggregate level, where the measures were calculated using different people. 83.3% of adults in the HES sample aged 15 years or over were linked to the Inland Revenue Department (IRD) register.[2]

Using these data, we perform three strands of analysis. First, we compare the linked HES respondents to the unlinked respondents to understand whether the linked sample is representative. Second, we compare income reported in HES to income recorded by IRD. Third, we compare benefit data reported in HES to that reported by the Ministry for Social Development (MSD). These strands were chosen because the analysis helps inform issues relevant to incorporating administrative data into Taxwell, Treasury's tax and welfare microsimulation model.

The rest of this paper is outlined as follows. Section 2 briefly compares this paper to prior literature. Section 3 describes the data including how the data are linked and the concordance between HES incomes and IDI measures of incomes. Section 4 compares the linked population to the unlinked population. Section 5 presents the comparisons of HES incomes to IRD incomes. Section 6 presents comparisons of HES reported benefits to MSD records of benefit receipt. Section 7 concludes.


Page top