Insights - Understanding the needs of children and youth at risk (AP 17/02)

Abstract#

This paper accompanies the release of an online interactive tool called Insights (see insights.apps.treasury.govt.nz). Insights uses anonymous data from Statistics NZ's Integrated Data Infrastructure (IDI) and presents it in an accessible way, allowing users to visualise integrated government data at a detailed geographical level. This paper summarises the content of the different parts of the Insights tool, describes the approach taken, outlines high level findings, and illustrates how the tool can be used to develop insights at a local level.

Acknowledgements#

The authors are grateful to Sylvia Dixon, who developed the initial youth transitions reporting, and to Danny Wu, Sam Caldwell, Shirley Wu, and Phil Shepherd from Harmonic Analytics Limited for their work on developing the Insights tool. Any remaining errors are the sole responsibility of the authors.

Code Availability#

The code used to produce the statistics used in this report can be accessed at the following GitHub address: https://github.com/Treasury-Analytics-and-Insights/

Disclaimer#

The views, opinions, findings, and conclusions or recommendations expressed in this report are strictly those of the author. They do not necessarily reflect the views of the New Zealand Treasury, Statistics New Zealand or the New Zealand Government. The New Zealand Treasury and the New Zealand Government take no responsibility for any errors or omissions in, or for the correctness of, the information contained in this Analytical Paper.

The results in this report are not official statistics, they have been created for research purposes from the Integrated Data Infrastructure (IDI) managed by Statistics New Zealand.

Access to the anonymised data used in this study was provided by Statistics NZ in accordance with security and confidentiality provisions of the Statistics Act 1975. Only people authorised by the Statistics Act 1975 are allowed to see data about a particular person, household, business or organisation and the results in this paper have been confidentialised to protect these groups from identification.

Careful consideration has been given to the privacy, security and confidentiality issues associated with using administrative and survey data in the IDI. Further detail can be found in the Privacy impact assessment for the Integrated Data Infrastructure available from www.stats.govt.nz.

The results are based in part on tax data supplied by Inland Revenue to Statistics NZ under the Tax Administration Act 1994. This tax data must be used only for statistical purposes, and no individual information may be published or disclosed in any other form, or provided to Inland Revenue for administrative or regulatory purposes.

Any person who has had access to the unit-record data has certified that they have been shown, have read, and have understood section 81 of the Tax Administration Act 1994, which relates to secrecy. Any discussion of data limitations or weaknesses is in the context of using the IDI for statistical purposes, and is not related to the data's ability to support Inland Revenue's core operational requirements.

Executive summary#

This paper accompanies the release of an online interactive tool called Insights (see insights.apps.treasury.govt.nz). Insights uses anonymous data from Statistics NZ's Integrated Data Infrastructure (IDI) and presents it in an accessible way, allowing users to visualise integrated government data at a detailed geographical level. This paper summarises the content of the different parts of the Insights tool, describes the approach taken, outlines high level findings, and illustrates how the tool can be used to develop insights at a local level.

Insights provides evidence to inform policies and services, with the current content focussed on at-risk children and youth. It replaces the Social Investment Insights (SII) tool, which was launched in February 2016, and provides updated information on children and youth at risk of poor outcomes, new content, and new ways to visualise and map the data presented.

The information previously provided by the SII tool on children and youth at risk of poor outcomes has been updated to 2015. The definition of the study population has been refined. Some measures have been improved, and new ways of visualising the results have been developed. The new results presented in Insights are broadly consistent with those presented in the SII tool.

Insights also presents new information on young people's activities and outcomes as they transition to adulthood, such as their rates of participation in employment, education and training. These outcomes can be graphed according to whether a young person was identified as being at-risk at age 15, and can be analysed at a detailed geographical level within broad age groups. Risk measures at age 15 are shown to be predictive of poorer future outcomes through to age 24, while the extent of this varies somewhat across New Zealand.

Finally, Insights includes some new experimental results on the extent to which educational and employment services are accessed by children and youth at risk. Almost all services targeted at improving educational and employment outcomes are disproportionately likely to be accessed by children and youth who are considered to be at risk, consistent with the intent of these services. Nevertheless, the coverage of services varies across New Zealand, with more analysis needed to understand this in the context of specific services.

The range of services currently included in Insights is limited. Future work will seek to include a more complete set of service information such as health and social services.

While the results highlight the power of using integrated administrative data in new and innovative ways, some of the methods are exploratory in nature. Caution should be taken in interpreting the results of any detailed analysis undertaken using Insights. Population coverage errors, linkage errors and biases present mean that detailed results should not be viewed as necessarily being highly accurate.

The Treasury will continue to develop Insights over time, extending the content of the information it contains about children and youth, improving the way data can be visualised, and incorporating new areas of analysis.

1  Introduction#

This paper accompanies the release of an online interactive tool called Insights (see insights.apps.treasury.govt.nz). Insights allows users to visualise integrated government data at a very detailed level. The data used to construct Insights is from the Integrated Data Infrastructure (IDI)[1], a secure database managed by Statistics NZ. All of the data included in Insights has been aggregated and is protected by Statistics NZ confidentiality rules. This means that no individual can be identified from either the data visualisation within the tool, or from the data sitting behind it. The data is completely anonymous.

Insights replaces the Social Investment Insights (SII) tool, which was launched by the Treasury in February 2016. Insights expands SII with updated content on children and youth at risk, new content, and new ways to visualise and map the data presented. The new content describes the outcomes achieved by children and youth at risk as they transition to adulthood, and their use of government services.

• Youth transitions to adulthood - This part of the tool presents information on the activities of youth as they transition to adulthood. It provides a connection between childhood risk factors and future outcomes, both positive (participation in education and employment) and negative (NEET[2] and custodial sentences).
• Use of education and employment services - This part of the tool provides information about education services used by young people aged 6 to 19 and employment services used by young people aged 15 to 24. It shows the extent to which children and youth at risk are accessing these services, both in absolute numbers, and relative to other youth, and how this varies across the country.

This paper summarises the content of the different parts of the Insights tool, describes the approach taken, outlines high level findings, and illustrates how the tool can be used to develop insights at a local level. The paper is intended to provide an introduction to Insights, and examples of how it can be used, but does not provide an exhaustive or comprehensive overview of its functionality or content. The tool is designed to allow for a broad range of approaches to be used to analysis, depending on the interest of the user.

Most visualisations in Insights allow analysis to be filtered by gender and/or ethnicity, however this paper focuses on the results for all children or young people in particular age groups. Insights also contains a large amount of information at a detailed geographical level, but the paper only presents a short introduction to the sub-national analysis that can be undertaken with the tool. The paper shows maps and graphs from the tool, but doesn't present all figures associated with these in numeric form. These are able to be derived from the tool itself and the data behind it.

Many of the maps and graphs in Insights can be downloaded, as can the data underlying them. This functionality will be developed further in future releases.

The rest of this paper is outlined as follows. Section 2 provides an overview of the data underlying the tool. Sections 3 to 5 give background to the different Insights pages, describe the data and the analytical approach taken, present high level findings, and illustrate the way Insights can be used to investigate geographical differences across New Zealand. Section 6 discusses the next steps.

2  Data and caveats#

2.1  Data overview#

The IDI brings together data from across government agencies[3] in a secure and anonymised form, which is then made available to bona fide researchers under strict conditions. The data used in this study comes from a number of sources including: the Ministry of Social Development (benefit receipt, employment services, and CYF care and protection and youth justice), the Ministry of Education (schooling and tertiary participation, and education services), Inland Revenue (employment), and the Department of Corrections (custodial and community sentences)[4]. Datasets are provided to Statistics NZ by agencies, where they are linked at an individual level, before identifying information is removed and they are provided to researchers in the IDI environment.[5]

2.2  Study population#

Because the IDI source data is collected primarily for administrative, rather than statistical purposes, populations can be defined in many different ways. This choice is often dictated by the research needs, building on previous approaches used. These approaches are designed to ensure that as much as possible the study population includes those people who are relevant to the research questions being addressed, while excluding those who are not of interest to the study.

This study adopts a slightly different approach to that used in the SII tool and associated papers (McLeod et al. 2015 and Ball et al. 2016). The SII tool used a range of criteria to identify a study population at the end of December 2013. For example, to be included, a child or young person had to:

• be eligible to live in NZ on a permanent basis
• be living in New Zealand for at least six months during 2013
• have records in the education data (where they are young enough to do so)
• have records in the tax data (where they are aged over 19).

Insights uses an alternative approach, based on the estimated IDI New Zealand resident population. This approach is outlined in Gibb et al. (2016), although we have modified it in a number of ways:

• We use reference years ending in December rather than June. In addition time in the country is calculated using the calendar year, with people included in the ERP if they spent more than 6 months of the year in New Zealand.
• Temporary migrants, such as international students and working holidaymakers, are excluded as we have limited data about their past, they may not stay in NZ long-term, and they are often not eligible for services.
• Custodial sentences are used as an additional 'activity measure' to identify people who are in the population.

2.3  Caveats#

The data presented in Insights was developed by linking administrative information across government agencies. This data will not always be accurate. This may be of particular concern when such data is presented at a very detailed level. In some cases information for one person may be incorrectly linked to information for a different person, while in other cases information may be missing or incorrect.

While the results highlight the power of using integrated administrative data in new and innovative ways, some of the methods are exploratory in nature. Caution should be taken in interpreting the results of any detailed analysis undertaken using Insights. Population coverage errors, linkage errors and biases present mean that detailed results should not be viewed as necessarily being highly accurate.

All data sources have quality issues which need to be considered as the data is used. Some issues are particularly relevant to the data used in this study.

• The scope of the data is limited by the nature and breadth of the information collected in agencies' administrative systems and included in the IDI. For example: administrative data provides only a partial picture of childhood adversity, service use and service costs; and education data does not always capture the date an enrolment ends.
• The methods used to estimate future outcomes and costs are designed to provide a comparative picture of future outcomes and costs for different population subgroups, but they have some significant limitations, as discussed in McLeod et al. (2015) and Ball et al. (2016). These estimates should be viewed as indicative, and not as forecasts of the actual outcomes and costs that will be incurred in the future.
• Geographical location is derived from administrative information held by government agencies. In some cases this data may be inaccurate or out of date, and individuals may be allocated to areas in which they don't reside as a result. The quality of geographical information in the IDI is discussed in detail in Gibb and Das (2015).

3  Children and youth at risk#

3.1  Background#

In 2015 and 2016 Treasury undertook a number of projects aimed at using integrated data to better understand the risk of poor outcomes for children and youth. The approach and findings for youth aged 15 to 24 are outlined in Mcleod et al. (2015), and for children up to age 14 in Ball et al. (2016). The release of the latter paper was accompanied by the launch of the Social Investment Insights online tool, which presented information on both children and youth at risk at a very detailed geographical level. Information was presented through interactive maps for New Zealand areas, down to area unit level, essentially equivalent to a city suburb.

3.2  Approach and data#

Insights replicates the content of SII, but extends the original 2013 data to the 2014 and 2015 years, and allows the data to be visualised in different ways. As discussed above, the population is defined slightly differently. This resulted in a slightly smaller population of children aged 0 to 5 (359,070 instead of 362,832), a slightly larger population of children aged 6 to 14 (517,900 instead of 510,348), a slightly larger population of youth aged 15 to 19 (292,764 instead of 289,539), and an almost identical population of youth aged 20 to 24 (292,314 instead of 292,764).

The calculation of some risk indicators has also been improved through minor refinements. For example, parental education was taken from MSD benefit records in the 2015/16 work. This has now been supplemented by direct educational qualification information for the birth mother, where that information is available.

Overall, the population of children considered to be at risk is estimated to be slightly larger using the new approach. In December 2013, 15.3 percent of children aged 0 to 5 and 14.5 percent of children aged 6 to 14 were estimated to have 2 or more risk indicators. This compared with 14.6 percent and 13.4 percent respectively in the earlier analysis. Consistent with this, young people aged 15 to 19 were also slightly more likely to be considered to be at risk with the new data (14.0 percent, up from 12.6 percent). The number of youth aged 20 to 24 considered to be at risk was the same under both the old and new approaches (9.6 percent).

These approaches represent just one way of identifying populations of children and youth at risk, taking advantage of data held in the IDI. There are many other approaches that could be used, either based on IDI data or on alternative data sources. Further investigation of such approaches is warranted, and could provide more useful results for some applications. Nevertheless, the approaches used in this study are probably of general applicability, and show the way administrative data can be used to inform policies and service provision.

3.3  High-level findings#

Comparisons of children and youth considered to be at risk in different time periods are unlikely to be very meaningful, as the data being collected by agencies, and subsequently incorporated into the IDI, has developed over time. Changes could be due to changes in the underlying data collections. As such, the focus of attention should be primarily on comparisons within the same time period.

The number of children at risk

Overall, the estimated population of children aged 0 to 5 decreased in size by about 2 percent between December 2013 and December 2015, while the population of children aged 6 to 14 increased by a similar relative margin (overall, there were 8,000 fewer 0 to 5-year-olds in 2015, and 11,000 more 6 to 14-year-olds). The proportion of children estimated to be at risk of poor outcomes decreased somewhat across all age groups, from 15.3 percent to 13.3 percent in the case of 0 to 5-year-olds, and from 14.4 percent to 13.8 percent in the case of 6 to 14-year-olds. As discussed above, this shift over time may not be meaningful however. Figure 1 shows the relative size of the different groups of children at risk as at December 2015.

Figure 1: Size of child and youth risk groups, December 2015

As described in Ball et al. (2016), children are identified as being at risk through the use of four risk indicators identified from the linked data. The more of these risk indicators a child has, the more likely they are to have poor outcomes in later life. The risk indicators are:

• having a Child Youth & Family finding of abuse or neglect
• being mostly supported by benefits since birth
• having a parent with a prison or community sentence
• having a mother with no formal qualifications.

We identify three groups of children at risk; those with two or more risk indicators (less than 15 percent of children), those with three or more risk indicators (less than 5 percent of children), and those with all four risk indicators (less than 1 percent of children). The latter risk groups are contained within the earlier groups as illustrated in Figure 1 (ie, those with all 4 risk indicators are also considered to be part of the 2+ and 3+ risk indicator groups).

The number of youth at risk

The estimated population of youth aged 15 to 19 decreased in size by less than a percent between December 2013 and December 2015, while the population of youth aged 20 to 24 increased by less than a percent (overall, there were 900 fewer 15 to 19-year-olds in 2015, and 2,400 more 20 to 24-year-olds). The proportion of youth estimated to be in at least one target population risk group decreased slightly across both age groups, from 14.0 to 13.8 percent of 15 to 19-year-olds, and from 9.6 to 8.3 percent of 20 to 24-year-olds. As discussed earlier, this shift over time does not necessarily indicate there has been a reduction in risk over time.

Figure 1 shows the relative size of the different groups of youth at risk as at December 2015. As described in McLeod et al. (2015) youth are identified as being at risk through a wide variety of indicators that are linked to poor outcomes in early adulthood and later in life.

Combinations of these indicators are used to define ten 'target population' risk groups, five at ages 15 to 19, and five at ages 20 to 24. These target populations are all associated with poor future outcomes, but the extent of that association varies across the different outcome domains. Young people can be in more than one target population, so the total population considered to be at risk is smaller than would be calculated by summing across the target populations.

Projected outcomes and costs

The analysis underpinning the Ball et al. (2016) and McLeod et al. (2015) papers was based on looking at people who were born in 1993, and tracking their experiences and outcomes through to young adulthood. Future outcomes were then projected out even further by linking data for this birth cohort to an earlier birth cohort (specifically the July 1978 to June 1979 birth cohort) and observing their outcomes through to age 35.[6] Because the matching exercise was not repeated as part of the current work, projected outcomes have not changed since the earlier release of SII, and do not change depending on the year selected.

Figure 2 shows projected outcomes for children considered to not be at risk (those with fewer than two risk indicators) with those considered to be at risk (those with two or more risk indicators). Projected outcomes were estimated across all children aged 0 to 14, and as a result do not differ for 0 to 5-year-olds and 6 to 14-year-olds. For children who are not at risk, failing to achieve a level 2 qualification is the most prevalent poor outcome (23 percent), while around 60 percent of those with two or more risk indicators are projected to have future contact with CYF, and around a half to not achieve a level 2 qualification. All outcomes were much less prevalent in the not-at-risk group of children, as is evident when comparing the y-axes of the graphs.

Figure 2: Projected future outcomes for children by number of risk factors

Projected costs are also presented in Insights, and include income support payments, costs associated with serving sentences administered by the Department of Corrections, and costs associated with the services provided by CYF in childhood. They are presented in 2013 dollars. In general projected costs are less than $50,000 per person for children and youth who are not considered to be at risk, while they are at least $180,000 and as much as $410,000 for different risk groups.

Figure 3 shows projected outcomes for youth aged 15 to 24[7] by whether they were in any of the target population risk groups, or in no target population risk group. As with children, youth considered to be at risk had a much higher likelihood of poor outcomes, while those who were not in a target population were predicted to be more likely to have earnings at age 34.

Figure 3: Projected future outcomes for youth by whether in a target population risk group

3.4  Exploring differences across New Zealand#

A key strength of Insights is the ability to look at at-risk populations at a detailed geographical level. Figure 4 shows the variation in the proportion of children and youth who are considered to be at risk by region. The patterns exhibited are fairly consistent across age groups, even though the approach to identifying risk is quite different for youth aged 15 to 24 than children aged 0 to 14.

Regardless of age group, the regions with the highest proportions of children or youth at risk are Northland and Gisborne. Around a quarter of children in these regions had two or more risk indicators in 2015, while around one in six youth were in one or more target population risk groups. Tasman region had the lowest proportion of children with 2 or more risk indicators (less than ten percent), while Tasman, Auckland, Wellington, Canterbury and Otago all had low proportions of 15 to 19-year-old youth identified as being in a target population (all around 11 to 12 percent). Auckland, Wellington, and Otago had the lowest proportion of 20 to 24-year-olds in a target population, with around six percent.

Figure 4: Proportion of children and youth at risk across New Zealand

More detailed analysis shows that there are smaller areas with high proportions of children and youth at risk in every region. This information could inform the targeting of funding and service provision. Although rates of risk are highest in some regions with relatively small populations, the majority of at-risk children and youth live in Auckland, and large numbers live in the regions associated with the main urban centres, as shown in Appendix A, Table 1.

4  Youth transitions to adulthood#

4.1  Background#

This part of the tool shows outcomes for youth as they transition to adulthood. It shows where they are doing well, where they may need additional support, and the parts of the country where the need is greatest.

Youth transitions to adulthood provides information about the activities and outcomes of all young people aged 15 to 24 in New Zealand. It shows the main activities they are undertaking as they transition to adulthood, how many are in employment, and how many are receiving benefits. Results are presented by gender, age and ethnic group and for different risk groups, and are mapped down to the area unit level (the equivalent of a city suburb).

Activities and outcomes are tracked at different ages by looking at the interactions that young people have with government agencies. School and tertiary enrolment information from the Ministry of Education tells us whether a young person is in education, tax data from Inland Revenue tells us whether they are working, and data from the Department of Corrections tells us if they are serving a custodial sentence.

Young people can be engaged in many different activities in a particular year. We prioritise these activities, defining a 'main activity' they were undertaking each month and averaging these across the year. Where someone is mainly overseas in a particular month, that month is excluded from the results.

4.2  Approach and data#

The main activity being undertaken is derived using a prioritised list. If a young person is doing multiple activities, the one that is highest in the list is used. For example, if a young person is in school and in limited employment in a particular month, they are only counted as being at school. The activities reported in Insights, in order of priority, are:

• in custody
• in school
• in tertiary education
• in substantial employment (with industry training)
• in substantial employment (without industry training)
• in limited employment
• short-term NEET
• long-term NEET.

If a person spent more than 15 days overseas in a particular month they are excluded from the results in that month. If a person spent more than 15 days in a particular month serving a custodial sentence, that is considered to be their main activity for the month. Education and employment activities are defined where a person spent at least 1 day of a month enrolled in school or in a tertiary course that equates to more than half of an Equivalent Fulltime Student (EFTS), or they earned at least $10 in the month.

Where a young person had self-employment income in a tax year, it is not possible to determine in which months it was earned. The earnings are distributed across the year, while negative values are treated as zero income.

'Substantial employment' is where a young person earned more than a person working 30 hours at the adult minimum wage would have earned in a month, while 'limited employment' is where they earned less than that amount in a month.

NEET means a young person is 'Not in Employment, Education or Training'. They may or may not be receiving a benefit, but there is no information to indicate they are in formal education, training or employment. In some cases they may be in an unpaid caring role, or some other unpaid role. A young person is considered to be long-term NEET if their main activity is NEET for six months or more at a time.

Separate measures of the time youth spent on benefit and in employment are also reported. These are not prioritised. A young person's main activity in a month may be tertiary education, but if they also have earnings and benefit receipt they will also be counted as being on benefit and in employment.

The number of risk indicators a young person had at age 15 is a useful measure of childhood risk that we know is predictive of poor outcomes later in life (see Section 3). Unlike the target populations also discussed in that section, this measure is not affected by outcomes experienced by young people as they transition to adulthood, as it is measured before those outcomes occur. The youth transitions to adulthood analysis uses this measure as a way of defining risk as people enter young adulthood.

4.3  High-level findings#

Figure 5 presents a view of all young people in 2015, showing the way the transition to adulthood differs for young people who had more than two indicators of childhood risk from those who did not.

There are some obvious differences in the two graphs. While young people who were not considered to be at risk were highly likely to be in tertiary education at age 19 (44 percent), and in substantial employment at age 24 (57 percent), these figures are much lower for those who were considered to be at risk at age 15 (24 and 35 percent respectively). Youth who were considered to be at risk were more likely than those not considered to be at risk to be long-term NEET (35 percent compared to 13 percent), or serving a custodial sentence (2.1 percent compared to 0.2 percent) at age 24.

Figure 5: Main activities of 15 to 24-year-olds during the 2015 calendar year, by number of risk indicators at age 15

Figure 6 shows time in employment or on benefit according to the same risk measure. Care needs to be taken with this comparison as the graphs are on different axes. Nevertheless, it is clear that those young people who were considered to be at risk at age 15 are much more likely to be on benefit and much less likely to be in employment than other young people. While those who had two or more risk indicators spent almost as much time on benefit as in employment at ages 20 to 24, other young people were around six to seven times as likely to be in employment as on benefit.

Figure 6: Proportion of time 15 to 24-year-olds spend on benefit or in employment 2015 calendar year, by number of risk indicators at age 15

4.4  Exploring differences across New Zealand#

Location of 15 to 19-year-old youth who are long-term NEET

As in the risk analysis discussed in Section 3, Insights allows youth outcomes to be compared across different geographic locations within New Zealand down to area unit level. Figure 7 shows how the proportion of time spent long-term NEET in 2015 differs across regions. The region with the highest proportion of time spent as long-term NEET is Northland (9.1 percent), while the lowest is Tasman (3.1 percent). By comparison, the time spent long-term NEET by all 15 to 19-year-olds in New Zealand was 5.9 percent.

There is considerable variation within regions however. Figure 7 also shows the variation in the same measure across ward areas in the Auckland region. Overall 15 to 19-year-olds in Auckland spent 5.6 percent of time as long-term NEET, almost identical to the national average. This varied considerably across Ward areas however, with young people in Manurewa-Papakura ward spending 9.9 percent of the time long-term NEET, compared to just 2.5 percent for those living in Orakei ward.

These rates differ from the NEET rates reported by Statistics NZ from the Household Labour Force Survey (HLFS). NEET, as measured in the HLFS, is based on people’s reporting of their activity over the previous week, and is measured on a quarterly basis. By contrast, our IDI-base measures of NEET are calculated on a monthly basis and averaged across the year.  The HLFS approach is essentially measured at a point-in-time. While it will include both short-term and long-term NEET spells, long-term NEET spells are more likely to be captured with this approach.

The overall 15 to 19-year-old NEET rate as reported in the HLFS for 2015 (averaged across four quarters) was 7.3 percent, whereas we estimate 15 to 19-year-olds spent an average of 5.9 percent of time in a long-term NEET spell in 2015. If we include time spent in short-term NEET spells of less than six months we estimate the NEET rate to be somewhat higher than the HLFS rate, 8.5 percent in 2015. When we break these figures down by region the patterns are very similar however (see Appendix B, Table 2).

Figure 7: Proportion of time spent as long-term NEET in 2015 for youth aged 15 to19 across New Zealand regions and Auckland wards

 

The HLFS sample is not sufficiently large to provide estimates at a detailed geographical level. The IDI is not subject to sample-size issues, and this enables us to present NEET measures for very small areas in the Insights tool. Caution should be taken with these very detailed estimates however, as address data may not always be accurate or up-to-date.

Proportion of 15 to 19-year-old youth who are in education by their level of risk

We can also look at geographical differences in outcomes for different age groups defined according to risk at age 15. Figure 8 shows the amount of time spent in education across New Zealand for youth aged 15 to 19, by whether or not they had two or more risk indicators at age 15.

In all regions youth with fewer risk indicators were more likely to be in education, and generally speaking there was little variation across the country once the number of risk indicators is taken into account. The proportion of time spent in education by youth aged 15 to 19 with fewer than two risk indicators ranged from 38 percent in the West Coast region up to 41 percent in six different regions. There was more variation amongst those young people with two or more risk indicators, with at-risk youth in Marlborough spending 29 percent of their time in education, compared to 35 percent in Wellington. The difference between education participation of at-risk and not-at-risk youth was lowest in the West Coast region (4 percentage points), while it was highest in Marlborough and Canterbury (10 percentage points). In all other regions at-risk youth were 6 to 8 percentage points less likely than not-at-risk youth to be in education.

Figure 8: Proportion of time spent in education in 2015 for youth aged 15 to19 across New Zealand regions by number of risk indicators at age 15

5  Use of education and employment services#

5.1  Background#

This part of the tool shows information about the use of services by children and youth at risk, and how this varies across the country. The tool provides information about a selection of education services used by young people aged 6 to 19 and employment services used by young people aged 15 to 24 in New Zealand. It shows the extent to which children and youth at risk are accessing these services, and maps the overlap between risk groups and service users in different parts of the country.

5.2  Approach and data#

IDI data on the services people access is limited. Employment and education are two areas where a lot of data on government-funded services is available, but not all services are included. In many cases government agencies, such as the Ministry of Education, do not collect information about which individuals access which services. For example, schools may have access to a number of different services to meet the needs of a particular child. Some of this service information is not provided to the Ministry and cannot be presented in Insights. Insights does not present information where services are not funded by government.

Education services

A number of education services are presented in the Insights tool. The data varies in quality and coverage, and for many other services participation is not captured at an individual level.

Participation information for 6 to 14-year-olds is presented for the following services:

• Correspondence School - Students who study by correspondence at Te Aho o Te Kura Pounamu (Correspondence school) instead of attending a physical school.[8]
• Interim Response Fund - A fund that is available to keep students engaged in learning following a significantly challenging behavioural event. It gives funding for a short term response while a more comprehensive plan is developed.[9]
• Reading Recovery - An early intervention for students making limited progress in reading and writing after their first year at school.[10]
• Resource Teachers: Learning and Behaviour - RTLB teachers are funded to work with teachers and schools to find solutions to support students in Years 1-10 with learning and/or behaviour difficulties.[11]
• Special education services - Children can access different special education services. Services include communication, behaviour, and early intervention services, the Ongoing and Reviewable Resourcing Scheme (ORRS)[12], High Health, and where a child attends secondary school over age 19, or primary school over age 14.
• Truancy (Non-Attendance) - a student may be referred to truancy services when they are enrolled at a school but don’t attend classes. This data is only available from 2013.
• Truancy (Non-Enrolment) - a student is referred to truancy services when they leave a school but aren’t re-enrolled in another school within 20 school days.

Data is collected from different systems and is of varying quality. For example, Special Education service participation is identified by a school administrator where a child receives some form of Special Education assistance. In future this will be sourced directly from the Special Education database, which could improve the quality of the data collected. Correspondence school enrolments are of high quality, but the date when a young person leaves school may not be recorded, and as such, it may appear they have attended correspondence school more recently than they actually have.

Participation information for 15 to 19-year-olds is also presented for the following services focussed at secondary and tertiary-level students:

• Youth Guarantee: Fees-Free - provides fees-free tertiary education for students aged 16-19 years who have no or low prior qualification achievement.[13]
• Secondary-tertiary programmes - provide alternative learning options for secondary school students, such as through Trades Academies.
• Gateway - The Gateway service provides year 11+ school students with the opportunity to access learning opportunities in the workplace.
• Alternative education - 15 year old students who cannot settle into the school environment are offered an alternative education outside the school.
• Industry Training - Delivered to people in employment, helping support the development of skills that meet industry needs.
• Secondary-Tertiary Programme - Provide vocational education opportunities for secondary school students, such as through Trades Academies.
• Student Allowance - A weekly payment to help with living expenses while studying full-time. Eligibility is dependent on a person’s income, their parents’ income, their living situation, and whether they have dependants.

While Insights provides some information on the extent to which at-risk youth access education services, it does not provide any information on the effectiveness of these services. Impact evaluations, such as the recent evaluation of the Youth Guarantee policy (see Earle 2016), which looks at both Fees-Free and Secondary-Tertiary programmes, are required to do this.

Employment services

The employment services included in Insights are funded by the Ministry of Social Development, and often delivered by Work and Income. The services data included in the IDI, and reported in Insights is not an exhaustive list of employment assistance available or offered by Work and Income to its clients. This will be improved over time. In most cases information is presented by type of service, instead of for individual services. The service types are:

• Information services - This includes careers guidance advice and seminars.
• Placement and matching services - Services that match people with jobs and help place them into job vacancies.
• Skills training services - Business training and advice, payments for course fees, industry partnerships, job-focussed training, and literacy/numeracy training.
• Wage subsidies - Payments to employers to incentivise them to hire, train, and retain disadvantaged jobseekers, including those with disabilities.
• Other employment services - This includes a range of services, including those that build work confidence, involve people in work in the community or conservation work, grants to help setting up businesses, and job search assistance.
• Youth Service: Youth Payment/Young Parent Payment - Youth Payment is available for young people aged 16 or 17 who are unable to be supported by their parents, while Young Parent Payment is available for 16 to 18 year old parents. Both payments are conditional on participation in Youth Service, a mentoring service for young people.
• Youth Service: NEET or Youth Transition Service - Mentoring service for youth at risk of becoming NEET. Youth Transition Service was superseded by YS: NEET in mid-2012.

A list of the services included under each category is included in Appendix B. As with education services, Insights provides some information on the extent to which at-risk youth access employment services, but does not provide any information on the effectiveness of these services. See de Boer and Ku (forthcoming), McLeod et al (2016), and Dixon et al (2016) for more information on the effectiveness of employment services in New Zealand, as well as a more detailed description of the services.

Extent of service use by at-risk children and youth

At-risk children and youth are more likely to need support to achieve good outcomes. Different services are targeted in different ways to meeting these needs. While some services are offered to a broad group of young people, others are targeted at a specific high-needs group, or are in reaction to a poor outcome occurring (such as non-attendance at school). As such, we would expect the extent of the overlap between risk groups and services to vary by service. The overlap could vary across the country for a number of different reasons, including access issues, differing local area needs, or different choices about the use of alternative services.

5.3  High-level findings#

Use of education services by at-risk children and youth

Figure 9 shows the extent to which children aged 6 to 14 with two or more risk indicators had used various education services in the 5 years to 2015. The size of the orange circle on the left-hand side shows the number of service users for each service, while the blue circle on the right-hand side is indicative of the size of the risk group selected. The overlap between the two circles shows the degree to which young people are both in the selected risk group, and have used that service. This overlap is the main group of interest in this analysis, and can also be expressed as a percentage of the orange circle (the proportion of service users considered to be at risk), or as a percentage of the blue circle (the proportion of the risk group who had accessed the service ie service coverage for the risk group).

Figure 9: Use of selected education services by at-risk 6 to 14-year-olds in the five years to 2015

As we saw in Section 3.3 around 14 percent of all 6 to 14-year-olds were considered to be at risk in 2015. For all education services we can see that service users were much more likely to come from an at-risk background than the general population of children. This varied from 24 percent in the case of Reading recovery and Special education, up to 47 percent for Interim response fund. The most commonly accessed service for at-risk children was Reading recovery, with almost ten thousand 6 to 14-year-olds (or 13 percent of children at risk) having accessed the service in the previous five years.

Figure 10 shows the extent to which youth aged 15-19 who had two or more risk indicators at age 15 used selected education services in the 5 years to 2015. As for the younger group of children, service users were generally much more likely to come from an at-risk background than the general population of young people. This varied from 24 percent in the case of Secondary-Tertiary programmes, up to 65 percent for Alternative Education [14]. Around five thousand youth aged 15-19 had used Alternative Education services in the five years to 2015. This represents 12 percent of all 15 to 19-year-old youth who had two or more risk indicators.

Figure 10: Use of selected education services by at-risk 15 to 19-year-olds in the five years to 2015

Figure 11 shows the extent to which youth aged 15 to 19 years who had two or more risk indicators at age 15 had accessed funding to support tertiary study in the past five years. This funding varies in the extent to which it is targeted at at-risk youth, and this is reflected in the extent of overlap with the population of at-risk youth. Generally speaking, the proportion of service users who were considered to be at risk was lower than for the school-based services discussed above. This is not surprising considering that a far lower proportion of at-risk young people participate in tertiary education, as illustrated in Figure 5.

The main source of tertiary funding used by at-risk youth was Youth Guarantee: Fees Free, which is targeted at young people with very low previous qualification achievement. Almost nine thousand at-risk young people had accessed this funding, around a fifth of all at-risk youth. Overall, 40 percent of youth who had accessed Youth Guarantee: Fees Free had two or more risk indicators at age 15, compared to 17 and 18 percent of youth who had accessed a Student Allowance or the Gateway Service respectively. Only 14 percent of Industry Trainees had two or more risk indicators at age 15, reflecting the low proportion of these young people who were in employment.

Figure 11: Use of employment services by at-risk 15 to 19-year-olds in the five years to 2015

Use of employment services by at-risk youth

Figure 12 shows the extent to which at-risk youth aged 15 to 19 received ‘Youth Service: Youth Payment or Young Parent Payment’ (YP/YPP) or ‘Youth Service: NEET or Youth Transition service (YSNEET/YTS)’. These are mentoring services for young people, and are aimed at supporting people to stay in education, and ultimately, to move into employment. Young people receiving YP/YPP are paid a benefit, while YSNEET/YTS participants are either NEET or considered at risk of becoming NEET, but do not receive income support. Participation in Youth Service is compulsory for recipients of YP or YPP, but YS: NEET and YTS are voluntary services.

As with the education services discussed above, service users are much more likely to have two or more risk indicators than would be expected in the general population (53 percent for YP/YPP and 36 percent for YSNEET/YTS). Overall, almost a quarter of 15 to 19-year-olds with two or more risk factors had accessed YSNEET/YTS in the past five years, while 14 percent had accessed YP/YPP.

Figure 12: Use of employment services by at-risk 20 to 24-year-olds on benefit in the five years to 2015

Figure 13 shows the extent of service use by at-risk 20 to 24-year-olds on benefit. As these services are generally only available to young people receiving a benefit, the analysis is restricted to the population of young people who received a benefit in 2015. While 15 percent of all 20 to 24-year-olds had 2 or more risk indicators at age 15, this was much higher (34 percent) amongst those 20 to 24-year-olds who received a benefit in 2015.

Generally speaking the proportion of 20 to 24 year old beneficiaries accessing employment services who were at risk was in line with the proportion of all beneficiaries who were at risk. Thirty-four percent of beneficiaries who had accessed Placement services or Wage subsidies in the previous five years were considered to be at-risk at age 15, while higher proportions of beneficiaries who had accessed Skills training or Information services were at-risk (42 and 37 percent respectively).

Figure 13: Use of employment services by at-risk 20 to 24-year-olds on benefit in the five years to 2015

5.4  Exploring differences across New Zealand#

The extent of service use in specific areas of New Zealand by at-risk children or youth can be easily contrasted against other areas. This may reveal differences in targeting, choices made at a local or national level about provision, or could indicate areas where alternative services are being used. Care needs to be taken in drawing conclusions from the results, but the analysis could present a useful starting point in understanding service gaps. The services currently included in Insights are quite limited, but this will expand over time. Here we look at one educational intervention for children, Reading Recovery, and an employment intervention for youth, Youth Service: NEET.

Use of Reading Recovery by at-risk children

There are two ways we can look at the extent of service use by at-risk children. The first is the proportion of service users who are considered to be at-risk. This allows us to see the extent to which the service is targeted towards risk in some areas relative to others. The second measure is essentially a measure of service coverage, ie, the percent of the at-risk group who have used the service.

Figure 14 maps the use of Reading Recovery services across New Zealand regions for children with two or more childhood risk indicators. Reading Recovery is a literacy intervention targeted at 6 year old children with literacy difficulties. It provides daily one-on-one teaching for children who are making the slowest progress in literacy learning[13].

Figure 14: Distribution of children aged 6 to 14 with 2+ risk indicators using Reading Recovery service in the past 5 years across New Zealand

The left-hand map shows the extent to which children accessing Reading Recovery are considered to be at-risk. Not surprisingly, those areas with the greatest proportion of Reading Recovery users being at risk, are the same areas with high numbers of at-risk children, as illustrated in Figure 4. The national average is for 24 percent of Reading Recovery service users to have 2+ risk indicators. This varies from 17 percent in Tasman region (where 9% of children have 2+ risk indicators) to 37 percent in Gisborne region (where 24% of children have 2+ risk indicators).

The right-hand map shows the service coverage for at-risk children - the proportion that have accessed the service. Across New Zealand 13 percent of 6 to 14-year-olds with 2 or more risk indicators had accessed the service in the last 5 years. This varies from 11 percent in the Bay of Plenty and Northland, up to 22 percent in the West Coast region. While this analysis indicates that at-risk children in Northland and the Bay of Plenty are less likely to access Reading Recovery than at-risk children in many other regions, it does not tell us why this is the case. It may be that they are more likely to access alternative services for example.

Use of Youth Service: NEET or Youth Transition Service by at-risk youth

The Youth Service: NEET (YS:NEET) and Youth Transition Service (YTS)[14], which it replaced, offer mentoring services to young people aged 16 or 17 who are not on benefit, and considered to be at risk of becoming NEET. Figure 14 shows the overlap between the use of YS:NEET or YTS and those youth identified as being at risk due to having two or more risk indicators at age 15.

We have established that the presence of these risk indicators at ages 0 to 14 is associated with poor educational and employment outcomes in later years, so we would expect this group to be a target population of particular interest for these services, and to be disproportionately likely to access the services.

Figure 15: Distribution of youth aged 15 to 19 with 2+ risk indicators at age 15 using Youth Service: NEET or Youth Transition Service across New Zealand

As illustrated in Figure 12, 36% of YS:NEET or YTS service users had 2 or more risk indicators at age 15. This compares to 14 percent of all 15 to 19-year-olds. The left-hand map of Figure 15 illustrates how this varies across New Zealand. In Auckland only 27 percent of service users had two or more risk indicators, while in Manawatu-Wanganui, Hawkes Bay, Gisborne and Northland 46 to 49 percent of service users were considered to be at-risk. The analysis does not tell us why this variation occurs, but it could be that the underlying populations of at-risk young people are different in different regions, or that the targeting of the service is more effective in some regions than others, possibly due to the services in some regions being more tightly rationed.

The right-hand map of Figure 15 shows the coverage of the service for youth with 2 or more risk indicators at age 15. Comparing this with the left-hand map, it's obvious that having a high proportion of service users considered to be at-risk does not necessarily mean that the coverage is high in that region. While almost half of service users in Northland were considered to be at-risk, the region had one of the lowest proportions of at-risk youth who had accessed the service (19 percent). Gisborne region, on the other hand, had both high proportions of at-risk youth in the service (47 percent) and high coverage for at-risk youth (29 percent). It could be that while the service is well targeted in Northland, limits in service availability may result in coverage still being reasonably low.

6  Next steps#

Insights will be developed over time. This is still a prototype tool, and a number of enhancements are planned in future. These include improvements in the usability of the tool, new ways of visualising data, and improved functionality to download data. Any suggestions for improvements to the tool would be welcomed, and can be sent to [email protected].

Insights will also be extended to include a broader range of services for children and young people, providing a more complete picture of the extent of service use by at-risk young people, and any areas where gaps in service use or access might lie. A particular area of interest is the addition of health service information.

Finally, Insights provides a platform for the delivery of a broader range of IDI information than just relating to the outcomes, risk and services for children and young people. Other domains of research interest will be explored over time, and a wider range of data delivered in an accessible way for the benefit of all New Zealanders.

References#

Ball, C., Crichton, S., Templeton, R., & Tumen, S. (2016). Characteristics of Children at Greater Risk of Poor Outcomes as Adults.Treasury Analytical Paper 16/01.

de Boer, M., & Ku, B. (Forthcoming). Effectiveness of MSD employment assistance, Summary report for 2014/2015 financial year. Ministry of Social Development.

Dixon, S., & Crichton, S. (2016). Evaluation of the Impact of the Youth Service: NEET Programme.Treasury Working Paper 16/08.

Earle, D. (2016). Monitoring the Youth Guarantee policy 2014. Ministry of Education.

Gibb, S., Bycroft, C., & Matheson-Dunning, N. (2016). Identifying the New Zealand resident population in the Integrated Data Infrastructure (IDI).Retrieved from www.stats.govt.nz.

Gibb, S., & Das, S. (2015). Quality of geographic information in the Integrated Data Infrastructure. Retrieved from www.stats.govt.nz.

Mcleod, K., Dixon, S., & Crichton, S. (2016). Evaluation of the Impact of the Youth Service: Youth Payment and Young Parent Payment.Treasury Working Paper 16/07.

Mcleod, K., Templeton, R., Ball, C., Tumen, S., Crichton, S., & Dixon, S. (2015). Using Integrated Administrative Data to Identify Youth Who Are at Risk of Poor Outcomes as Adults.Treasury Analytical Paper 15/02.

Appendices#

Appendix A: At-risk children and youth by region#

* Children aged 0 to 14 are identified as being 'at risk' if they have two or more risk indicators. Youth aged 15 to 24 are identified as being at risk if they meet the criteria for inclusion in one or more of ten target populations.

Appendix B: Comparison of HLFS and IDI NEET rates#

* Source: Statistics New Zealand [17].

Appendix C: Employment services by service type#