The Treasury

Global Navigation

Personal tools

2.7  Caveats

The process of matching records is probabilistic and creates some level of error, as there are likely to be some cases where individuals cannot be matched (and appear in the data with less service use than actually occurred), as well as cases where individuals have been wrongly matched (and appear in the data with inaccurate estimates of service use).

The data covers a specific time and cohort, and some care must be taken in generalising results to the experience of more recent cohorts of children. Cohorts born more recently have had a higher likelihood of being notified to CYF, partly because of administrative changes related to family violence events attended by police. This is described in further detail in Treasury Analytical Paper 15/01.

There are also possible biases for those young people who have spent any lengthy period of time outside of New Zealand before the age of 21. The characteristics of these people, including any outcomes achieved, are less likely to be visible in our data, as any contact with government agencies may happen outside of New Zealand. It may look like these people fail to gain qualifications, avoid prison sentences or benefits, and do not access health services, where these things happen out of New Zealand.

To some degree this is controlled for by including an indicator in the modelling when a young person is out of the country for the entire previous year, however there may be some biases introduced that may be better controlled for by including more sophisticated measures of time outside of the country, or by treating this group differently, possibly excluding some from the analysis. There is no single approach that would be better however, and more thinking may be needed on this issue for future work.

Page top