The Treasury

Global Navigation

Personal tools

6  HES benefits vs IRD benefits

Summary

  1. Main benefits are measured poorly in HES. Many people fail to report benefits they received, and many other people report benefits they did not receive. Of people the Ministry of Social Development have paid main benefits to, only 75% report any type of benefit in HES, and of those who report receiving a main benefit in HES, only 88% have a record of receiving one in the MSD data.
  2. On average, under-reporting of benefit receipt is more common than over-reporting. This is true even for people who report benefit receipt in both data sources.
  3. 63% of people who receive Accommodation Supplement from the Ministry of Social Development do notreport receiving Accommodation Supplement in HES.

This section compares self-reported HES data on benefit receipt to administrative benefit records provided by the Ministry for Social Development (MSD) to Inland Revenue.

The HES coverage of the benefit system is, in principle, the same as MSD. MSD benefits are attributed exclusively to individuals, so there is no possibility of accidentally attributing benefits to a company. However, HES respondents may not remember what benefits they have received over the last year, they may make approximations such as rounding time periods or they may confuse different types of benefits, for example, Supported Living Payment and Disability Allowance. Respondents may also feel stigmatised or feel like their privacy is being invaded and decide not to report benefit receipt.

No data set is perfect, and it is possible that there are errors in the administrative data too, including from mismatches or other reasons.[22]

We focus on two comparisons across all main benefits - comparing benefit indicators and comparing the number of days on benefit. Accommodation Supplement, a non-taxable housing subsidy available to both beneficiaries and non-beneficiaries, is also compared in this section. Comparisons by main benefit type are presented in Appendix C.

The benefit system was changed on 15 July 2013. For this reason, we compare HES and MSD benefits under the old system (we performed these comparisons for the new benefit system and found similar patterns). Comparisons of the old benefits use HES surveys 2006/07-2012/13. We exclude the HES 2013/14 year as this year included both benefit systems, and the HES data has known issues with benefit reporting in this year.[23] Accommodation Supplement (AS) wasn't changed as part of the 2013 reform, so the comparison presented for AS are for the full time period.

Table 3 shows that, of the people MSD have paid main benefits to, only 75% report any type of benefit in HES. It also shows that only 88% of those who report receiving a main benefit in HES have a corresponding MSD record. The level of misreporting is too high to be explained by a 1.4% false positive link rate. For example, a 1.4% false positive rate would generate about 540 mismatches, and as a rough back-of-the-envelope calculation, we might expect about 50 in each of the lower left and upper right cells or about 10% and 5% of what we observe.[24] Even if the false positive rate was as high as 10%, this would only account for about 340 in the lower-left cell and 400 in the upper-right cell. Thus, most of the people in lower-left cell and upper-right cell of Table 3 are likely due to survey respondent error. Respondents are both reporting benefits that they do not receive and failing to report benefits that they do receive, though on average, respondents are under-reporting benefit receipt.

Table 3: HES vs MSD any benefit indicator
(2006/07-2012/13)
Any HES benefit indicator  Any MSD benefit indicator
No Yes
No 33,573
(87.2%)
1,110
(2.9%)
Yes 465
(1.2%)
3,339
(8.7%)

Table 4 shows a similar tabulation for Accommodation Supplement (AS), which shows that only 37% of people MSD has paid AS to report it in HES (ie, 63% do not report it). For people who report receiving AS in HES, only 79% have a corresponding MSD record.

< /thead>
Table 4: Accommodation Supplement HES vs
administrative indicator (2006/07-2014/15)
HES AS indicator  MSD AS indicator
No Yes
No 46,662
(87.8%)
3,681
(6.9%)
Yes 594
(1.1%)
2,199
(4.1%)

Figure 4 shows the difference in total days on benefit between MSD and HES across all benefit types. Figure 4 shows the old benefit system, though the conclusions apply to the much smaller sample size using the new benefit system. The skew towards the positive indicates that MSD records more days on benefit than is reported in HES. Simply put, benefits are under-reported on average in HES. This may be due to, among other things, stigma associated with receiving a benefit, confusion between benefits and tax credits or linkage error. The bar on the far left (negative) shows HES recipients who report being on benefit for a full year, yet MSD has no record of paying this person a benefit. This shows, in addition to under-reporting of HES benefits, even the people reporting an HES benefit may not actually be receiving one.

Figure 4: Difference in total days on any benefit between MSD and HES
Figure 4: Difference in total days on any benefit between MSD and HES   .

These results are likely to have less impact on Treasury's Taxwell modelling than might be expected for three reasons. First, Treasury's calibration (reweighting) process explicitly uses the total number of main benefit recipients as a benchmark. Which households receive main benefits will change, but the weights will adjust so that the total number of beneficiaries remains constant. Any changes to aggregate costs would need to come through benefit rates, which are modelled directly and hence are unlikely to dramatically change. Second, non-benefit income is used to determine if an individual is eligible to receive a benefit in Taxwell - only people reporting an HES benefit with the appropriate non-benefit income level will actually receive it. This will fix some of the people who report receiving a benefit in HES when no MSD record exists. Third, AS is modelled on an eligibility basis.[25] This likely means that Taxwell is overestimating the number of people receiving AS. Making the necessary changes to Taxwell to determine the impact of moving to administrative data is beyond the scope of this paper.

Because the survey measures of benefit receipt compare so poorly to the administrative measures, there is a strong case for using administrative data on benefit receipt in place of HES data in Taxwell. This is particularly so in the case of the second-tier benefit Accommodation Supplement.

Notes

  • [22] Benefit data are sourced from the IDI tables named msd_clean.msd_spell and msd_clean.msd_partner
  • [23] See the HES 2012/13 Commentary which notes that "However, there was a 10.0 [percent] fall in the number of people receiving this [government benefits except New Zealand Superannuation or war pensions] income".
  • [24] The rough calculation is as follows. Lower-left cell estimate = false positive rate*Number of people on benefit in HES*proportion in MSD not on benefit = 48. The upper-right cell estimate is false positive rate*Number of people on benefit in MSD data*proportion on HES not on benefit = 56. These estimates assume that the false positive rate is independent of benefit receipt and that who people are mislinked to is also independent of benefit receipt. The number of people on benefit/not on benefit in MSD figures is based on those linked to the HES data. These assumptions are not likely to hold completely, but they provide a useful approximation.
  • [25] It is possible in Taxwell to use reported AS or even no AS. These are only used for testing purposes.
Page top