The Impact of Tertiary Study on the Labour Market Outcomes of Low-qualified School Leavers: An update (WP 18/03)

Executive Summary#

Print this section

This paper examines the labour market benefits obtained by young people who leave school without gaining NCEA level 2 if they enrol at a tertiary institution while aged 15-22 years. The central research question is whether and to what extent undertaking tertiary education raises the employment rates and earnings of these low-qualified school leavers, compared with not doing any post-school study.

The current paper extends the results provided in an earlier paper (Tumen et al, 2015) by improving the research design and tracking student outcomes for a greater number of years after the end of the tertiary education. The impacts of level 1-2 certificates are estimated separately from the impacts of level 3 certificates in this revised paper. In addition, the current paper provides supplementary results for more recent birth cohorts of youth who followed the same educational path as the original study population.

Study design#

Print this section

The research uses linked administrative data from the Integrated Data Infrastructure (IDI). The data sources in the IDI provide detailed information on the educational and labour market activities, incomes and family characteristics of all young people in the birth year cohorts studied.

Our main study design focuses on young people who were born between July 1990 and June 1992, who left school without completing NCEA level 2 and then studied at a tertiary institution at any time while they were aged 15-22 (more specifically, within the six years following their 11th year at school). The outcomes of these young people can be tracked for a minimum of three years and a maximum of six years after they finished their tertiary study, depending on the timing and duration of their tertiary enrolment.

We estimate the impacts of the tertiary education by comparing the labour market outcomes of the study population youth with those of a matched comparison group of similar school leavers who did not undertake any tertiary education. The matching is done using the method of propensity score matching and uses an extensive set of variables on the young person's characteristics, schooling and employment history, as well as their family background and parental characteristics.

Our main set of results evaluates the impacts of tertiary education on the young people's employment rates, benefit receipt rates and monthly earnings, three years (36 months) after the end of their tertiary study.[1]

In supplementary results, we estimate the impacts of tertiary education up to six years after the end of tertiary study, for the subset of youth whose outcomes can be observed for a longer period. Finally, we also investigate the impacts of tertiary study on the labour market outcomes of later birth cohorts, those born between July 1992 and June 1994, who followed the same educational path as our main study population.

Main impact estimates#

Print this section

For the average low-qualified youth who enrolled, tertiary education had a small positive impact on their subsequent employment rates. The employment rate of the low-qualified school leavers was 5 percentage points higher than that of their matched non-participants, three years after they completed or withdrew from their tertiary programme. There was no significant impact on the proportion who were on a benefit three years later.

The average impact is somewhat misleading, however, because the benefits of tertiary study were confined to the 51% of students who completed a qualification. Non-completers did not significantly improve their likelihood of employment, and they were more likely to be on a benefit three years later (by 4 percentage points). Of the low-qualified school leavers who enrolled in tertiary education and had finished studying by the time they were 22, 26% completed a level 1-2 certificate, 39% completed a level 3 certificate, 34% completed a level 4 certificate or diploma and 1.4% completed a higher qualification.

Completing a level 1-2 certificate was associated with a 5 percentage point increase in the group's employment rate and no change in its benefit receipt rate three years later. Completing a level 3 certificate was associated with an 11 percentage point increase in the group's employment rate and a 6 percentage point reduction in its benefit receipt rate three years later. Completing a level 4 certificate or higher qualification was associated with a 10 percentage point increase in the group's employment rate and an 8 percentage point reduction in its benefit receipt rate three years later.

Higher employment rates led to higher total monthly earnings for the youth who completed a qualification. However, we found no evidence that tertiary qualifications raised their levels of earnings. Conditional on being in work, those who had secured a tertiary qualification had the same average monthly earnings as the matched youth who did not undertake tertiary study.[2]

Impacts of tertiary education after five or six years#

Print this section

The impacts of tertiary study can be tracked over a longer period (five to six years) for the youth who were born in 1990-92, enrolled within the first few years after leaving school and had finished their tertiary study by four years after the end of year 11 at secondary school. These young people were still in their teens when they undertook the tertiary study. They represent 68% of the main study population.

We found limited evidence of changes in the impacts of tertiary study over the extended follow-up period. In general, the positive effect of completing a qualification on the likelihood of being employed was increasing in size over the first two to three years and then stable or slightly declining rather than increasing further through time.

Impacts experienced by later birth cohorts#

Print this section

Turning to the supplementary study population of youth who were born in 1992-94, the impacts of tertiary education assessed at three years after the completion of study were found to be similar in size or slightly smaller than those estimated for youth born in 1990-92.

Variation in impacts#

Print this section

Focusing on the 51% of students who completed a qualification, we find that the size of the employment and benefit rate impacts varied somewhat by gender, ethnicity, the school qualifications of the student, the type of tertiary provider and the subject area of the qualification. The most consistent finding was that larger benefits were gained by students with no school qualifications than by students who had achieved NCEA level 1.

At level 3, certificates in management and commerce; engineering and related technologies; creative arts; food, hospitality and personal services; architecture and building; and mixed-field programmes had the largest positive employment impacts. At level 4, certificates in food, hospitality and personal services; health; engineering and related technologies; and architecture and building had the largest positive employment impacts.

Limitations of the research#

Print this section

The most important limitation of the study is that its impact estimates could be influenced by unmeasured differences in the characteristics of the study and comparison groups. We cannot rule out the possibility that the students who successfully completed a tertiary qualification may have had better or worse employment outcomes and lower or higher benefit receipt rates than non-participants, even if they had not studied, due to differences in characteristics that we do not have information about. It is possible that differences in the outcomes of those who compete a qualification and those who do not reflect other underlying unobserved factors and not the gaining of a qualification.

Conclusion#

Print this section

The results in this paper indicate that tertiary education has the potential to significantly improve the employment rates of youth who leave school without NCEA level 2 but only if they complete a qualification. Level 3 and level 4 qualifications had more substantial impacts on employment rates and benefit receipt rates than level 1-2, suggesting that obtaining a level 3 qualification or higher should be the policy goal for this group of youth.

A high rate of non-completion was identified (with half failing to complete any qualification), and this substantially reduced the average gains associated with the tertiary enrolment of this group. Policies that either encourage more realistic enrolment decisions or improve the completion rates of those who do enrol have the potential to improve the cost-effectiveness of the tertiary education undertaken by low-qualified school leavers.

1  Introduction#

Print this section

Young people who leave secondary school with few qualifications are at greater risk of becoming unemployed or inactive than those who leave with a higher level of attainment. In 2016, the employment rate of 20-24-year-olds who had no formal qualifications was just 52%, 14 percentage points below the employment rate of 20-24-year-olds with school qualifications and 28 percentage points below the employment rate of 20-24-year-olds with post-school qualifications.[3]

The New Zealand Government has set a goal of increasing the proportion of young people who obtain either the NCEA level 2 certificate at school or equivalent post-school qualifications by the age of 18. A level 2 qualification is regarded as the desirable minimum level of educational attainment needed to participate in tertiary education, support entry to the workforce and facilitate full participation in society.[4] Although qualification attainment rates in schools have been rising rapidly, the number of young people who leave school without a level 2 qualification continues to be quite substantial. In 2009, 33% of school leavers had not achieved an NCEA level 2 certificate. In 2016, the equivalent number was 20%.[5]

Many of the programmes that tertiary institutions and private training providers offer at lower qualification levels are open to school leavers who have not completed NCEA level 2. Post-school qualifications at levels 1-3 provide a second chance for these young people to acquire basic qualifications through programmes that are pitched at an academic level similar to upper secondary school but with greater vocational focus.[6]

This study is an update of a previous study that was published by Treasury in 2015 (Tumen et al, 2015). It examines the benefits gained by low-qualified school leavers who enrol in a tertiary programme after leaving school. Specifically, it selects all young people who were born between July 1990 and June 1992 who were enrolled as domestic students at New Zealand schools at the start of 2006 or 2007 (when they were expected to be in year 11), and tracks their subsequent educational pathways and outcomes. The study population comprises the group who left school without completing NCEA level 2, who subsequently enrolled in a tertiary education programme, had completed or withdrawn from tertiary study by the end the sixth calendar year after the year they were enrolled in year 11 at secondary school and did not re-enrol after that time.[7]

The paper addresses the following questions:

• What impact does participating in tertiary education have on the employment, benefit outcomes and earnings of young people who have left school without a level 2 NCEA certificate?
• What impact does completing a tertiary qualification have on the outcomes of these low-qualified school leavers?
• To what extent do these impacts vary by the demographic characteristics of participants, by institution and field of study?

The impacts of the tertiary study are estimated by comparing the outcomes of the poorly qualified school leavers who enrolled with those of a matched comparison group of school leavers who did not enrol. The matching is done using a mixture of exact and propensity score matching. The main impacts of the education on subsequent employment outcomes are measured three years (36 months) after the student finished studying. Three years was the maximum follow-up period for which the outcomes of everyone in the study population can be observed.

The research uses linked longitudinal data from the tax, benefit and education systems that have been incorporated into the Integrated Data Infrastructure (IDI) at Statistics New Zealand. The IDI incorporates a wide range of administrative and survey datasets, linked at the individual level and covering the whole of the New Zealand population. It provides comprehensive information on school leavers who studied at tertiary institutions, including their school leaving date, highest school qualification, the level and field(s) of their tertiary study and the tertiary qualifications they completed. Information is also available on labour market outcome measures including monthly income from wages and salaries, annual income from self-employment and daily income from government income support benefits.

Despite controlling for a large number of observed characteristics, our impact estimates could be influenced by unmeasured differences in the characteristics of the study and comparison groups. We cannot rule out the possibility that the students who completed a tertiary qualification may have had better or worse employment outcomes, even if they had not studied, due to differences in characteristics that we do not have information about.

The current paper extends results provided in Tumen et al (2015) by improving the research design and tracking student outcomes for a greater number of years after the end of the tertiary study spell. It provides separate impact estimates for level 1-2, level 3 and level 4 certificates. In addition, it provides a new set of results for more recent birth cohorts of students who followed the same educational path.

To foreshadow the main results, the main findings show that enrolling in tertiary programmes and completing a qualification was associated with substantial employment rate increases. The 51% of students who completed a tertiary qualification experienced a 9 percentage point increase in their likelihood of being employed three years after completion. These students were also less likely to be receiving income support, but their average monthly earnings were no higher than those of comparable young people who did not undertake any tertiary study.

Students who enrolled in tertiary education but did not complete the programme they enrolled in (49%) did not show any material improvement in their employment rates and were 4 percentage points more likely than matched comparisons to be in receipt of a benefit, three years later.

These impact estimates are almost exactly the same as those obtained in the previous paper at two years after the completion of the tertiary study.

Supplementary results using longer follow-up periods indicate that the employment benefits of tertiary qualifications for low-qualified youth tend to reach their maximum size after two to three years and are stable or declining rather than increasing after that. Supplementary results looking at a younger birth cohort (those born in 1992-94) give impact estimates that are similar to those obtained for the original birth cohort.

2  Background#

Print this section

2.1  Post-school study at levels 1-3 and level 4#

Print this section

Post-school level 1-3 certificates are roughly equivalent in academic level to upper secondary school qualifications, but they have a greater vocational focus and tend to be shorter in duration than a school year. They are intended to prepare people for entry-level employment or to enable them to progress to higher levels of vocational education (Earle, 2010). Enrolment in many level 1-3 certificate programmes is open to people who have not completed NCEA level 2.

People whose highest qualification is a post-secondary level 1-3 certificate are commonly employed in clerical, sales and service or semi-skilled manual occupations. Some level 1-3 certificates provide trade-related skills, but most do not constitute trade qualifications. Post-secondary level 1-3 qualifications also provide a pathway to further study, providing the prerequisites for enrolment at level 4 or higher.

The level 1-3 certificate pathway to higher education is particularly important for young people who have left school without University Entrance or NCEA level 2. They will generally need to take level 1-3 courses first to gain the prerequisites for enrolment in courses at higher levels.

Level 4 certificates provide more advanced employment-related education and generally involve around one year's full-time equivalent study. People whose highest qualification is a level 4 certificate are most likely to work as trades workers or technicians (Earle, 2010).

The majority of post-secondary level 1-4 qualifications are offered by institutes of technology and polytechnics (ITPs), wānanga, and private training establishments (PTEs). A wānanga is a publicly owned tertiary institution that provides education in a Māori cultural context. A PTE is a privately owned educational entity, such as a training operation owned by a company, an English language school or a privately owned design school. There are more than 800 PTEs in New Zealand, and in 2016, 27% of all students who were enrolled in level 1-3 tertiary programmes were enrolled at a PTE.[8]

2.2  Previous research findings#

Print this section

Our literature search focused on the question of whether young people who leave school with few qualifications are able to improve their employment outcomes if they participate in post-school educational programmes that provide second-chance qualifications, roughly equivalent in level to upper secondary school qualifications. We looked for studies with research designs that were more likely to isolate the causal impact of the education, for example, those using difference-in-difference approaches to compare the labour market outcomes of young people who enrolled in post-school study with those of matched non-students before and after the study spell.

International research

We were not able to locate any high-quality studies focused on this specific question but found a number of studies offering relevant insights. McIntosh (2004) is a frequently cited British study that examined the impact of post-school vocational study undertaken by unqualified school leavers. The paper used cross-sectional and longitudinally linked Labour Force Survey data to estimate the labour market benefits gained by individuals who left school in the mid-1990s without qualifications but then completed a vocational qualification by the age of 23-25 years (at any level). The employment rates and wages of this group were compared with those of unqualified school leavers who did not obtain any vocational qualifications. The results showed positive impacts on employment rates. Young people who left school without any qualifications were more likely to be in work when aged 23-25 years if they had achieved vocational qualifications after leaving school. Vocational qualifications had little impact on wages, however.

McIntosh acknowledges several alternative explanations for these results, including selection effects (those who choose to participate in post-school education may have characteristics that made them more likely to be employed) and reverse causation (it is easier to acquire vocational qualifications once in employment). The data and study design do not allow these alternative explanations of the positive employment impacts to be investigated and ruled out.

Stromback (2010) investigates the benefits that early school leavers in Australia gain by completing a vocational qualification, by estimating the impact of the qualification on their weekly earnings and hourly wages when they are aged in their mid-20s. The data source for this study is the Longitudinal Survey of Australian Youth, and the study population is youth who left school at an early age during the mid-1990s. These young people were followed in the survey until 2006, when they were aged in their mid-20s. A propensity score matching approach was used to construct a matching control group of other early school leavers who did not complete a vocational qualification against which the outcomes of the study population or treatment group were compared. Measures of academic ability and socioeconomic characteristics were included in the matching model. The estimates given in the paper represent the average impact of all vocational qualifications completed by this study population, regardless of their level.

Stromback finds no evidence that vocational qualifications have statistically significant effects on the early career earnings of early school leavers. Rather than concluding the qualifications were not useful, he suggests the lack of significant earnings benefits may be due to the relatively short follow-up period. The group who returned to education had gained less work experience by their mid-20s than other early school leavers. Over a longer timeframe, he argues, the effects of the difference in early work experience are likely to fade while the benefits of the education may become more pronounced.

Summarising, the first of these two studies finds evidence of a positive impact of post-school vocational study on employment rates. Neither finds evidence of an impact on wages. Unlike the current paper, neither paper focused specifically on the impacts of participation in courses at the lowest levels of post-secondary education (ie, courses roughly equivalent to upper secondary school in academic demands). Both studies used survey rather than administrative data, and the small number of early-school-leaver observations available prevented the researchers from examining the effects of specific pathways or specific tertiary qualifications.

New Zealand research

The prior New Zealand research on this topic is limited. Earle (2010) used cross-sectional survey data to compare the employment rates and earnings of working-aged adults who had completed level 1-3 certificates with those of adults who had no qualifications. He found that adults whose highest qualification was a level 1-3 certificate had lower employment rates and incomes than adults whose highest qualification was an upper secondary school certificate but higher employment rates and incomes than adults who had no qualifications at all. However, because this study used a cross-sectional rather than a longitudinal study design, it is not able to clearly identify the contribution of the educational attainment to the difference in outcomes.

Crichton and Dixon (2010) analysed the labour market benefits associated with the completion of post-school qualifications by adults aged 25 and over. Using a longitudinal design, the study compared the earnings growth experienced by adults who completed a tertiary qualification at levels 1–6 with the earnings growth experienced by a matched comparison group of adults who did not return to education. It found that those who completed a level 1–3 or a level 4 certificate generally did not improve their earnings during the following three years. However, significant earnings benefits were gained by a minority of students in some particular fields of study. The study design did not include people aged under 25, and therefore the paper does not provide any results for younger students.

Crichton (2013) analysed the labour market benefits associated with the completion of post-school qualifications for those aged 18 and over who had been supported by benefits for at least six months immediately before they started studying. Using a longitudinal design, the study compared the benefits, employment and earnings of beneficiaries who studied with the outcomes experienced by a matched comparison group of beneficiaries who did not study. It found that those aged 18–24 years who completed a qualification at level 1–3 were 7 percentage points more likely to be employed and 4 percentage points less likely to be receiving a benefit five years after starting study than matched comparisons. Larger employment rate increases were experienced by those who completed a qualification at level 4 or higher.

Tumen et al (2015) was the first New Zealand study to use longitudinal data and a matched comparison group design to study the labour market impacts of level 1-3 tertiary programmes undertaken by young, poorly qualified school leavers. This study reported that enrolling in a level 1-3 or level 4 certificate programme had a small positive impact on the employment of low-qualified school leavers, raising their employment rate by 3.4 percentage points on average two years after they ceased studying. However, the benefits of tertiary study were confined to the 44% of students who completed a qualification and were not experienced by non-completers. Students who completed a level 1-3 certificate were 8.5 percentage points more likely to be employed and 6.4 percentage points less likely to be receiving a benefit than their matched comparisons two years after finishing. Slightly larger employment and benefit rate effects were experienced by students who completed a qualification at level 4 or higher. There was no evidence that tertiary study had a significant impact on participants' level of earnings after controlling for their employment status.

3  Methods#

Print this section

3.1  Data sources#

Print this section

The study uses data from Statistics New Zealand's Integrated Data Infrastructure (IDI), which links together many types of administrative data. The IDI provides longitudinal monthly information on individuals’ employment, earnings and receipt of income support payments over the period 1999–2016, their benefit receipts from 1992 onwards and their tertiary enrolments over the period 1997–2016. It provides data on secondary school enrolments, NCEA qualifications completed and detailed NCEA standard level attainment from 2006–2016.

At the time the previous study was carried out (Tumen et al 2015), the data on secondary schooling that were available in the IDI were not as comprehensive as they are now. In the previous study, we used a derived research dataset containing information on the highest qualification and characteristics of the school leavers. In the current study, we draw on more comprehensive sets of data that have been sourced directly from the Ministry of Education's enrolment database and NZQA's qualification database. This means the results are likely to be more accurate.

For each person who participated in tertiary education, the IDI provides information on their enrolment dates, level and field of study, equivalent full-time students (EFTS) associated with the programme of study in the current year and total EFTS associated with the qualification enrolled in. It also provides information on whether the programme of study (ie, the set of courses associated with a qualification) was completed.

Students who formally withdraw from a course in the first few weeks do not attract government funding and therefore are not included in the enrolment data and are not included in the participant study population. We also excluded students from the study population who undertook tertiary programmes that did not offer any formal qualifications. We included both students whose tertiary programmes were funded through the mainstream tertiary funding system and those funded by some specific targeted training funds such as Youth Guarantee[9].

3.2  Study population for the main analysis#

Print this section

We focus on young people who were born between 1 July 1990 and 30 June 1992 who left school without completing NCEA level 2 but studied at a tertiary institution during the following few years. Specifically, we focus on those who started and finished a programme of tertiary study between their date of leaving school and the end of the 6th calendar year after they were enrolled (or should have been enrolled) in year 11 at school.

The July 1990 to June 1992 birth cohorts were selected for study to ensure that information on school enrolment, benefit receipt and employment status would be available for everyone for at least 36 months before the start of the tertiary study spell and at least 36 months after the completion of the study spell.

We identified the study population using the school enrolment records of all young people who were enrolled in New Zealand schools as domestic students at the start of the year when they were likely to be in year 11 and therefore studying towards NCEA level 1. That is 2006 for the July 1990 to June 1991 birth cohort and 2007 for the July 1991 to June 1992 birth cohort.

Figure 1 illustrates the method of selecting the study population in detail, focusing on the 1990-91 birth cohort (ie, half the main study sample). The numbers presented in Figure 1 are also included in Appendix Table 1.

Figure 1: Selection of the study population: illustration using 1990–91 birth cohort

Notes: Counts have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

About 61,200 young people born between 1 July 1990 and 30 June 1991 were enrolled at New Zealand schools as domestic students at the beginning of 2006. We excluded those who did not have a match to the IDI spine and therefore could not be linked to other data sources in the IDI (1,600) and those who attended schools that offered international school qualifications (3,800). The latter were excluded because we lack good data on the qualification achievements of these young people.

Of those who attended schools offering NCEA only, around 21,100 left school without having completed NCEA level 2. We label this group low-qualified school leavers and track their tertiary enrolment patterns in the period until the end of 2016.

Around 11,700 of these low-qualified school leavers enrolled at a tertiary institution in a level 1-4 certificate programme between leaving school and the end of 2012, and another 1,000 enrolled directly in level 5 or higher tertiary programmes. Those admitted directly into higher level 5 or higher programmes are likely to have received recognition of prior learning or to have gained credit for co-requisite study and were not included in our study population. The enrolments that are counted include all Student Achievement Component-funded courses at levels 1-4 and enrolments funded by some specific targeted training funds such as Youth Guarantee.

Of the 11,700 who enrolled in a level 1-4 certificate programme between leaving school and the end of 2012, approximately 6,860 had ceased studying by the end of 2012 (ie, they did not enrol again in the period for which data are available, 2013-16). This subgroup represents our study population. Another 4,800 were enrolled during 2013-16 and were excluded from the study population.

Of the 6,860 low-qualified school leavers who were included in the study population, around 3,400 had not completed a qualification by the time they ceased studying, 2,200 had completed a level 1-3 certificate and 1,200 had completed a level 4 or higher qualification.

An equivalent approach was used to select study population members from the cohort of young people who were born between 1 July 1991 and 30 June 1992, giving a total study sample from both birth years combined of approximately 13,630 (see the first two columns of Appendix Table 1).

Given this study design and the fact that we have data on outcomes until the end of 2016, the minimum follow-up period for the young people in the 1990-91 birth year is four years after the end of their tertiary study spell. The minimum follow-up period for youth in the 1991-92 birth year is three years after the end of the last tertiary study spell. Our main set of estimates focuses on impacts measured at 36 months and uses the combined study population of 13,630 individuals.

As noted above, around 40% of all students who do not achieve NCEA level 2 but enrol in level 1-4 after leaving school were studying when they were aged 23-25 years. This will include some students who progressed from study at level 1-4 to higher-level study. Encouraging students into further study is a key objective of many lower-level courses, and we would expect outcomes to be particularly good for these students. Unfortunately, we are not able to evaluate this with data currently available. Very few students in our study population progressed to higher-level study and gained a qualification at level 5 or higher within six years following their 11th year at school (when they would have been aged 21 or 22 years).

3.3  Supplementary study populations#

Print this section

We created two supplementary study populations:

• Early enrollers: Youth who were born between 1 July 1990 and 30 June 1992 who enrolled and either finished or withdrew from their tertiary study within four years (calendar years) after the end of year 11 at school. This is a subset of the main study population described above, which also picks up youth who studied five or six calendar years after the end of year 11 at school.
• More recent birth cohorts: Youth who were born later, between July 1992 and June 1994, who followed a comparable educational path - that is, they left school without completing NCEA level 2 and subsequently enrolled at a tertiary institution in a level 1-4 certificate programme, finishing or withdrawing within four calendar years after the end of year 11 at school.

All of the study populations include youth who did not actually complete year 11 at school. The early enrollers group enables us to measure impacts over a longer follow-up period than is possible with the main study population. We have data on the outcomes of everyone for a minimum of five years and six years' data for 73%. The more recent birth cohorts group allows us to estimate impacts for a different population of young people who participated in a slightly different set of tertiary programmes and are likely to have experienced somewhat different labour market conditions in the years after leaving school. The minimum follow-up period for this latter group is three years.

Appendix Table 1 gives information on the size of these supplementary study populations. The early enrollers group (9,270 students) represents 68% of the main study population. Approximately 8,090 youth are in the more recent birth cohorts sample.

3.4  Measures of characteristics before tertiary enrolment and outcome measures#

Print this section

Periods of tertiary study

Some students in the study population enrolled in more than one tertiary programme between the date of leaving school and the end of the period allowed for tertiary study in our research design. If that was the case, pre-study outcomes were measured before the start of the first tertiary study programme, and post-study outcomes were measured after the end of their last enrolment period.

Individual and family characteristics prior to tertiary study

We used a variety of administrative data sources to build up an extensive picture of the characteristics of the youth in our study population.

• Gender, birthdate and ethnicity were taken from the Ministry of Education's student records.
• Secondary school enrolment history, credits and qualifications obtained in school and measures of school attendance, stand-downs and suspensions were taken from the Ministry of Education's data sources.
• Characteristics of the last school attended, including its regional location, were taken from the Ministry of Education's data sources.
• Information on a young person's history of benefit receipt (through their parents when they were a child or directly when they were aged 18 and over), their history of contact with Child, Youth and Family (the national child welfare agency) and their history of youth justice referrals was obtained from the Ministry of Social Development's data sources.
• Information on country of birth, type of residence visa held and the young person's history of entries and exits from New Zealand was obtained from the administrative data of the Ministry of Business, Innovation and Employment. Data on entries and exits from New Zealand were used to derive measures of time spent overseas.
• Information on whether the young person had any appearances in a Youth Court, proven charges in an adult court or sentences served was obtained from administrative data compiled by the Ministry of Justice and the Department of Corrections.
• Information on employment and earning was obtained from administrative data compiled by Inland Revenue.

We used birth records to identify the birth mothers and fathers of the youth in our study population. We were able to identify a birth mother whose identity was linked to the IDI for 88% of youth and a birth father whose identity was linked to the IDI for 79% of youth.

Having identified these parent relationships, we were able to construct measures of the proportion of time they were in employment during the past five years, their earnings from employment during the past five years, their income support history during the past five years and their history of convictions and sentences served.

The individual and parental variables used in the matching models are described below in section 3.5.

Outcome measures

Due to the manner in which tax data are collected, the employment and earnings measures in the IDI are available on a calendar month basis only. There are no measures of weekly earnings, hourly earnings or hours of work in the administrative data in the IDI.[10]

In this study, a person is classified as employed in a given calendar month if they received any wage and salary earnings in that month (that was reported through the tax system). An employment rate measures the proportion of people who received any wage and salary earnings in a particular month. Similarly, a person is classified as in receipt of a benefit if they received any income from one of the main income support benefits during the month, and the benefit receipt rate is the proportion of people who received any main benefit income in the specified month. It is possible to be employed and receive benefit income in the same month. They are not mutually exclusive states.

Average earnings were calculated using the IDI data on the total gross wage and salary income earned by each individual in each calendar month. We use two different measures of average earnings. One includes people whose earnings were zero, while the other excludes them. The first measure is used to analyse changes in the total earnings of the entire study population. The second is used to study changes in earnings levels, conditional on being employed. Note that monthly earnings can be affected by changes in hours worked as well as by changes in wage rates. All nominal monthly earnings were adjusted for general price level changes over time and are expressed in December 2016 dollar values.

Information on the main income benefit received by an individual (if any) was used to calculate the rate of benefit receipt in this study. A person who only received a supplementary income payment, such as the Accommodation Supplement, but not a main benefit is not counted as being on a benefit.

Our monthly employment, earnings and benefit receipt measures were further refined by excluding people who were overseas for the whole of the calendar month. Approximately 12% of the main study sample members were overseas 36 months after the end of their study spell, and 13% were overseas 48 months afterwards. Not excluding those who were out of New Zealand would have led to an underestimation of employment and benefit receipt rates.

3.5  Impact estimation methods#

Print this section

The impacts of tertiary study on employment and incomes were estimated by using the method of propensity score matching to construct comparison groups of low-qualified school leavers who are as similar as possible to the individuals in the study population but did not enrol in tertiary study during the study period (2006-16). The outcomes of these comparison group individuals provide the counterfactual against which the outcomes of the students (the treatment group in this study design) are compared. Differences between the benefit and employment rates and earnings of the two groups in the follow-up period provide our estimates of the impact of tertiary study.

More specifically, the method has two parts. First, the characteristics and activities of low-qualified leavers who did not enrol in tertiary education (non-participants) can be measured in each calendar month from January 2006 through to December 2016. We created a vector of monthly records for each non-participant containing information on their personal and family characteristics, measured at each calendar month (see section 3.4.2 for more detail on the specific variables). This generated over 1 million non-participant records. The purpose of creating this pool of non-participant records was to ensure we could match each student in the study population with a group of non-participants whose characteristics and life histories matched those of the student in the reference month - the month when the student first enrolled in a tertiary education programme.

In the second stage, we took a random sample of 120,000 records from the pool of non-participant records and used them along with the study group records to estimate logistic regression equations modelling the probability of starting a tertiary study spell. Separate regression models were estimated for enrolment at institutes of technology and polytechnics (ITPs) and enrolment at private training establishments (PTEs) because this segmentation of the sample led to a better model fit in both cases.

The probability of starting a tertiary study spell was modelled using information on a wide range of variables, including demographic characteristics, secondary school characteristics, attainment at secondary school, the young person's employment and benefit receipt history up to the reference date, time elapsed since leaving school and the young person's child welfare and youth justice history during their childhood. A number of variables about the young person's mother and father were also included, such as the parent's age at the birth of the reference child, their offending history, their employment history during the past five years and their levels of earned income and benefit income on average during the past five years. A full list of the explanatory variables included in the regressions is given in Appendix Table 2.

Using the parameters from each regression model, predicted probabilities of enrolling in a tertiary programme were then calculated for all members of the treatment group and potential comparison group (not just the subsample of 120,000 potential comparisons who were used in the regressions). These predicted probabilities are referred to as propensity scores, following the terminology of Rosenbaum and Rubin (1983).

The third stage of the method was to match each individual in the study population with a group of suitably selected individuals from the non-participant potential control sample. Matches were only made between individuals from the same birth cohort who were observed in the same reference month (determined by the student's month of enrolment), had exactly the same gender, prioritised ethnicity and highest secondary school qualification and had a similar time gap between leaving school and enrolment (or the reference month). Within those constraints, each treatment group individual was matched to up to 20 comparison group individuals with the closest values of the propensity score from the relevant model (ITP or PTE) within a radius of plus or minus 0.03 propensity score points. Fewer than 20 matches were selected if fewer than 20 people met these criteria. Matching with replacement was used, meaning that each comparison group individual could be matched to more than one treatment group individual. About 143,830 non-participant reference month records were selected and used in the main set of impact estimates. On average, this is about 12 records per student.

Each matched comparison record was assigned a weight based on the number of matches made (eg, 0.05 if the individual was one of 20 matches for a particular study sample member). These weights were applied in all subsequent analysis.

We dropped individuals in the study population who could not be matched with one or more non-participants. This reduced the size of the main study population from around 13,630 to around 11,800 and yielded an overall match rate of 87%. The match rate did not vary greatly across most demographic groups or dimensions of educational attainment. However, members of some small population subgroups were less likely to be matched. In the case of Pacific people, for example, the match rate was 73%. The proportion of each subpopulation that was matched to at least one comparison group individual and therefore retained in the final sample for analysis is shown in Appendix Table 3 (with the results for the main study population in the left-hand column).

The three-stage matching method is designed to balance the average characteristics of the treatment and comparison groups. After matching, there were no remaining statistically significant differences in variable means between the treatment and comparison groups for any of the model variables.Although we did not exactly match on every variable, the method ensured that the matched non-participant samples were very similar in terms of their regional profiles, distributions of NCEA credits attained at school, the time between leaving school and enrolling in study (or the same time period for their matched comparisons) and other key variables. In the left-hand column of Appendix Table 4, we compare the profile of the matched study sample with the profile of the matched comparisons, showing that the two groups have almost identical distributions across all measured characteristics.

Exactly the same methods were used to construct matched comparison groups for the two supplementary study populations. Details on the match rates obtained for these supplementary study populations can be found in Appendix Table 3. A comparison of the matched participant and non-participant samples for each supplementary study population can be found in Appendix Table 4. The match rates and extent of balancing are very similar to those achieved for the main study population.

Having selected a matched comparison group for each individual in each study population, impact estimates were calculated as the difference between the study population mean and the (weighted) comparison group mean. Standard errors for each impact estimate were estimated using bootstrapping methods, using 500 replications, with sampling at the individual level prior to propensity estimation.

4  Profile of the study population and its tertiary enrolment patterns#

Print this section

4.1  Introduction#

Print this section

In this section of the paper, we provide summary statistics on the characteristics of the low-qualified students who enrolled in tertiary study, their activities before enrolling in tertiary study, the nature of the tertiary courses they enrolled in and the qualifications they obtained.

To simplify the discussion, we focus on the main study population. Information on the supplementary study populations can be found in the Appendix. In general, the characteristics and schooling histories of the main and supplementary study populations are similar, as are their tertiary education patterns and outcomes.

We use the full study population for this descriptive analysis rather than the 87% who were matched with one or more suitable non-participants. (The latter analytical sample is used to obtain the impact estimates in section 5).

4.2  Characteristics and activities of low-qualified youth before their tertiary study#

Print this section

Tables 1-3 provide summary statistics on the demographic characteristics, family background, schooling history, welfare and justice sector history and employment history of the low-qualified school leavers that make up our main study population. Information on the characteristics of low-qualified school leavers who did not undertake any tertiary study is also shown in the right-hand column.

Males made up 59% of the main study population, reflecting their higher likelihood of leaving school without an NCEA level 2 qualification compared with females. Using a prioritised measure of each person's ethnicity (in which Māori is given first priority, Pacific people second and other ethnicity third priority), 52% of the main study population were classified as NZ European, 33% as Māori and 11% as Pacific people.

Turning to schooling history, about one-third left school during or at the end of year 11, while 44% spent at least some time in year 12 and 25% spent time in year 13. Just over one-third completed NCEA level 1 before leaving school, while the remaining two-thirds left without any formal qualifications.

However, the vast majority gained some credits towards NCEA level 1 while at school, and around half gained some credits towards NCEA level 2. The average number of credits obtained across all subjects and levels was 58.[11]

While 26% attended secondary schools classified by the Ministry of Education to deciles 1-3, indicating they were located in neighbourhoods of low socioeconomic status, almost half (46%) attended schools at deciles 4-7, and the remaining 28% attended schools classified to deciles 8-10.

Notes: Counts have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

The bottom rows of Table 1 provide indicators of disengagement from schooling. Considering all their years of schooling, 11% of the students had at least one suspension recorded in the administrative data, 30% had at least one stand-down recorded and 17% had one or more episodes of truancy recorded.

By comparing the columns for all students and all non-participants in Table 1, we can see that the characteristics and prior attainment of the low-qualified school leavers who enrolled in level 1-3 tertiary study were quite similar to those of low-qualified school leavers who did not undertake any tertiary study. The main difference is that a somewhat higher proportion were female. The youth who enrolled in tertiary programmes had remained in school for a similar length of time as those who did not enrol, and they had very similar levels of school achievement, as measured by their NCEA credits and qualifications. In addition, they had similar rates of suspensions, stand-downs and truancy episodes.

Other childhood history measures are set out in Table 2. Over two-thirds of the low-qualified youth who eventually enrolled in tertiary education programmes were supported by a parent's income support benefit at some point in their childhood, and 32% were supported by a parent's benefit for half or more of their childhood. Just under 29% were the subject of a Child, Youth and Family (CYF) notification, and 14% were the subject of a substantiated finding of abuse or neglect. Just under 10% were referred to CYF's youth justice services at some point in their childhood (because of a misdemeanour). Just over 6% of the students had at least one Youth Court appearance before starting their tertiary study, and 12% had at least one proven charge in the adult court. Just over 5% had served a Department of Corrections sentence (either a community or custodial sentence).

Notes: Counts have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI). The pre-tertiary experiences of the students are measured at the time of their first tertiary enrolment. The pre-tertiary histories of each non-participant can be observed in every month from January 2006 to December 2016. We generated records for each non-participant in each month and took a random sample of 120,000. This is the basis for our non-participant profile. Results are shown for the 18 months before the reference month.

The random sampling on non-participant months results in an analytical sample of non-participants that is older than the participant population. This is clearly demonstrated in Table 3, where 69% of the non-participant sample had left school more than 18 months ago compared to 30% in the participant population. This contributes to many of the differences in offending, employment and benefit history in Tables 2 and 3.

The construction of the matched comparison group (described in section 3.5) results is a companion group that is very well matched on age and the time between leaving school and enrolment (or the reference month).

Notes: Counts have been randomly rounded to base 3. Earnings are in $ Dec 2016 values. Figures have been derived from the Integrated Data Infrastructure (IDI). The prior activities of the students are measured at the time of their first tertiary enrolment. The prior activities of each non-participant can be observed in every month from January 2006 to December 2016. We generated records for each non-participant in each month and took a random sample of 120,000. This is the basis for our non-participant profile. For non-participants, results are shown for the 18 months before the reference month.

The prior employment history and benefit receipt of low-qualified school leavers before they started tertiary study is described in Table 3. About 38% of students enrolled in tertiary education within three months of leaving school, and the majority (70%) enrolled within 18 months of leaving school. About two-thirds undertook some paid employment in the 18 months before enrolling, but only 26% were employed for more than 12 of the previous 18 months. While in paid employment, the vast majority had average monthly earnings below the full-time equivalent of the minimum wage (about $2,200 a month). This indicates that most were working in part-time or part-month (possibly casual) jobs.

The majority spent no or very little time receiving benefit income during the 18 months before enrolling. This is not surprising as people under 18 years of age are generally not eligible for state income support, except in a very limited set of circumstances.

4.3  Tertiary enrolment and attainment#

Print this section

Data on tertiary enrolment patterns and programme completion rates for the main study population are set out in Table 4.

Notes: Counts have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

The average age of first enrolment in a tertiary programme was 17 years. Just under 9% were aged 15 at their first enrolment, 28% were aged 16, 26% were aged 17, 18% were aged 18, 11% were aged 19 and 9% were aged 20-21.

The majority (78%) of students enrolled in a level 1-3 certificate initially, while 22% enrolled in a level 4 programme. The proportion that enrolled for a level 4 qualification is somewhat surprising, given the group's low level of school qualifications. We assume that these youth were required to begin with courses at lower levels before progressing to courses at level 4.

Just under 60% of these low-qualified school leavers enrolled at an ITP, 33% enrolled at a PTE, 5% enrolled at a wānanga and a very small number enrolled at a university or other type of institution. Over the study period, more lower=level courses were being offered by PTEs.

About 25% of the students were enrolled for less than six months, and 60% were enrolled for less than one calendar year. Most of the remaining 40% of students were enrolled for one to two years in total. The distribution of students by the total number of EFTS (equivalent full-time students) completed also suggests that about 60% of the study population studied for at least one full-time equivalent study year.

Just over half (51%) of the students successfully completed their programme and gained a qualification. Completion rates were higher for females than for males, slightly higher for Pacific people and other ethnic groups than for NZ European and Māori and materially higher for school leavers who had completed NCEA level 1 at school (58%) than those with no school qualification (48%). Completion rates were higher for students who first enrolled at older ages. Comparing institutions, completion rates were highest at wānanga but similar in level at ITPs and PTEs (just over 50%). Completion rates also varied across subject fields and were particularly low for students who enrolled in mixed-field programmes at 34%. None of the broad subject fields had an average completion rate higher than 61%.

Table 5 describes the highest qualification completed. For those who completed a qualification, 26% were level 1-2 certificates, 39% level 3 certificates and 34% level 4 certificates. The most common broad fields were food, hospitality and personal services; engineering and related technologies; management and commerce; and agriculture, environmental and related studies.

Notes: Counts have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

Due to slightly different study designs, in which a shorter time period is allowed for the tertiary programme to be started and completed, the qualification achievement rates of the students in the two supplementary study populations were lower at 44% and 46% respectively. Information on the qualifications obtained by the students in these supplementary study populations is given in Appendix Table 6 and Appendix Table 7. The qualification profile of the more recent birth cohorts (born 1 July 1992 to 30 June 1994) is quite similar to that of the 1990-92 birth cohorts, suggesting that enrolment and achievement patterns were fairly consistent over time.

5  Impacts of tertiary study on labour market outcomes#

Print this section

5.1  Introduction#

Print this section

We use two ways to present the estimates of impacts on employment, earnings and benefit receipt rates. First, we present graphs that show the differences between the outcomes of the students and the outcomes of the matched non-participants at each calendar month before and after tertiary study in section 5.2. This provides a dynamic picture of how the outcomes of the treatment group and comparison group evolved before and after the tertiary study. Second, in section 5.3, we present tables of impact estimates with standard errors, focusing on impacts assessed at 36 months after the end of the study spell. This is the maximum follow-up period for which we have data for all individuals in the main study population.

Initially we focus on the results for the main study population. In section 5.4, we turn to the supplementary study populations and look at impacts measured over a longer period of time (up to six years) and experienced by later birth (those born 1 July 1992 to 30 June 1994 rather than 1 July 1990 to 30 June 1992).

5.2  Labour market outcomes before and after studying#

Print this section

A graphical comparison of labour market outcomes before and after the tertiary study spell is presented in Figures 2-5, using the data for the main study population. Each of these figures plots a particular outcome measure (employment rate, benefit rate, monthly earnings or monthly earnings conditional on employment) for the students and matched comparison group members in each calendar month before they started their study spell and each calendar month after the end of their study spell. The outcomes in the months spent studying (or the period from the first to the last study spell, if there was more than one) and the corresponding months for the comparison group individuals are not shown in the graphs.[12]

In the four parts of Figure 2, we show the employment rates of students who completed a level 1-2 certificate, those who completed a level 3 certificate, those who completed a level 4 certificate and those who failed to gain a qualification, together with the employment rates of the corresponding matched comparison groups. Note that monthly employment is based on wage and salary employment and excludes self-employment. Information on income from self-employment is only available annually. Very few young people in our study population or comparison group have income from self-employment (less than 2%).

The employment rates of the students and their comparisons during the 36 months before study are close to each other, indicating that the prior employment histories of the participants and matched non-participants are very similar after matching. Pre-study employment profiles slope upwards with time as increasingly large fractions of the young people in these groups left school and started jobs.

Figure 2: Employment rate

Figure 3: Benefit receipt rate

Figure 4: Average monthly earnings

Figure 5: Average monthly earnings when employed

By 6-12 months after the end of the study spell, the young people who enrolled in tertiary education and completed a qualification had substantially higher employment rates than the matched comparison groups, on average. The size of the difference increases over the two years and then remains fairly stable, although it increases further in the case of students who completed a level 1-2 certificate.

However, there is no difference between the employment rate of the students who did not gain any qualification and that of their matched comparison group.

In Figure 3, we show benefit receipt outcomes for the same subgroups. The pre-study benefit profiles slope upwards with time as increasing numbers of individuals reached the minimum age for benefit receipt (generally 18 years).

The post-study outcomes plotted in Figure 3 show that, after tertiary study, students who completed a qualification at level 3 or higher were less likely to be on a benefit, and this effect was increasing in size over the first three years. There was no reduction in the benefit receipt rates of students whose highest qualification was a level 1-2 certificate, and students who enrolled but did not complete a qualification were more likely to be on a benefit afterwards than the individuals in their matched comparison group.

The earnings and conditional earnings results for the same subgroups are given in Figures 4 and 5. Figure 4 shows that the total monthly earnings of students who completed a qualification (using the data for everyone, regardless of whether or not they were employed) were on average substantially above those of the matched comparison group at the end of the 36-month follow-up period.

After restricting the sample to those in employment, however, we find that the students who completed a qualification actually earned less than their matched comparisons, particularly in the first two years (as shown in Figure 5). This gap in monthly earnings diminishes over time, and by three to four years after the end of the tertiary study, monthly earnings conditional on employment were similar for the participant and non-participant groups.

These graphs provide a good indication that there were substantial differences in outcomes between the students who completed a qualification and the matched comparison samples, but they do not show whether the differences were statistically significant. Estimates of impact size and significance are given in the next section.

5.3  Main impact estimates at 36 months after the end of the tertiary study#

Print this section

Our main estimates of the impact of tertiary study on labour market outcomes are calculated at 36 months after the end of the study spell. These are summarised in Table 6 and Figure 6.

The first section of Table 6 shows the impacts estimated for all students. The second section shows the impacts experienced by the students who did not complete a qualification. The remaining sections show the impacts experienced by students who completed and gained qualifications in total and by the highest level.

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Earnings were converted to December 2016 values. Proportions and averages were calculated excluding those who were overseas 36 months after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

Figure 6: Estimated impacts of tertiary study three years later

Notes: Figures have been derived from the Integrated Data Infrastructure (IDI). The underlying numbers are given in Table 8. Error bars show the 95% confidence interval. If the error bar crosses the vertical axis at zero, the impact estimate is not significantly different from zero.

Sample sizes are given in the first column of Table 6. In the second and third columns, we show the average values of each outcome measure for the students and their matched comparisons. For example, 61.6% of the former students and 56.4% of their matched comparisons were employed at 36 months after the end of the tertiary study spell, while 28.7% of the former students and 29.2% of their matched comparisons were receiving one of the main income support benefits at that time.

The impact estimates are shown in the fourth column of the table. These represent the difference between the study group and comparison group means. Statistically significant impacts are marked with an asterisk. Standard errors are shown in the fifth column. The standard errors were estimated using bootstrapping methods. Relative impacts (showing the impact estimate as a percentage of the comparison group's mean employment rate, benefit rate or earnings) are shown in the final column.

In the following discussion, we focus on the statistically significant results.

All low-qualified school leavers who enrolled in tertiary education

Looking at the top row of Table 6, we can see that the employment rate of all low-qualified school leavers who enrolled in tertiary programmes was 5.3 percentage points (or 9.3%) higher than the employment rate of the matched comparisons three years after finishing or withdrawing from tertiary study. The difference in the employment rate was statistically significant at the 0.95 confidence level. However, the benefit receipt rate of students three years after finishing was not significantly different from that of the matched comparisons.

The average monthly earnings of the students were 6.5% higher than that of their matched comparisons. This earnings measure incorporates income from employment for all students, in all months, including months without paid work and therefore captures the direct effects of differences in employment rates on earnings. An alternative measure of earnings, shown in the following row calculates average monthly earnings for students who had some earnings in the month. On this measure, the students' earnings were 2.5% lower than those of their matched comparisons.

Non-completers

The second section of Table 6 gives our impact estimates for the students who enrolled but failed to complete a qualification (49% of the total). The estimated effects of tertiary study were negligibly small or negative for this group of young people. The estimated impact on the employment rate (1.3 percentage points) is not significant. Non-completers were more likely to be in receipt of a benefit three years after they ceased studying by 4.0 percentage points or 13.3% in relative terms. Their average monthly earnings, conditional on employment, were also lower by 3.8%.

Completers

Our estimates of the impacts experienced by students who completed a qualification are given in the remaining sections of the table. On average, those who completed a qualification were 9.1 percentage points (15.9%) more likely to be employed three years after finishing study and 4.8 percentage points (17.1%) less likely to be on benefit. Their average total monthly earnings were 13.7% higher (capturing the effect of higher employment). Average earnings conditional on being employed were not significantly different from those of the matched comparison group.

About 27% of the students gained a qualification at level 1-2. Completing a level 1-2 certificate was associated with a 5.3 percentage point (8.8%) increase in the employment rate and no significant change in the benefit receipt rate or in total monthly earnings.

39% of the students gained a level 3 certificate. Completing a level 3 certificate was associated with a 10.8 percentage point (19.7%) increase in the employment rate, a 6.1 percentage point (20.4%) decrease in the benefit receipt rate and a 18.8% increase in total monthly earnings.

34% gained a level 4 or higher certificate (but these were nearly all at level 4). Completing a level 4 certificate was associated with a 10.1 percentage point (17.5%) increase in the employment rate, an 8.3 percentage point (30.1%) decrease in the benefit receipt rate and a 14.6% increase in total monthly earnings (but no significant change in average monthly earnings conditional on employment). These results are similar to the impacts associated with level 3 certificates.

It's interesting that there was almost no difference in outcomes by level of qualification completed. People completing level 1-2 qualifications had almost the same employment outcomes as those who completed level 4 qualifications and better outcomes than those who completed level 3 qualifications. The lower impactof level 1-2 on employment rates is mostly due to the comparison group having a higher employment rate than the comparison groups for other levels. Similarly, they were less likely to be on benefit than the comparison group for other levels.

This raises the question of whether level 1-2 qualifications are poorly targeted, with school leavers enticed to undertake them who would have obtained employment without gaining a qualification.

Summary of the main impact estimates

The results from the main study population indicate that tertiary study is beneficial for low-qualified school leavers who complete a level 3 or level 4 qualification but of little value (or even costly) for those who do not finish their courses and less value to those who only gain a level 1-2 certificate.

For those who complete a qualification at level 3 or higher, the improvement in the likelihood of being employed is substantial and accompanied by reductions in the likelihood of receiving income support. There is no evidence of higher earnings, conditional on being in employment, however.

5.4  Extensions using supplementary study populations#

Print this section

Impacts after five to six years for early enrollers

We use a subset of the main study population to estimate impacts over a longer follow-up period. The early enrollers study population comprises students who enrolled and then completed or withdrew from tertiary courses within four calendar years after their enrolment in year 11 in secondary school. This is two years less than the time we allow for students to start and finish their studying in the main analysis. The early enrollers study population includes 68% of the students in the main study population.

Only 44% of the students in the early enrollers study population had completed a qualification by the end of this shorter period (compared with the 51% completion rate of the main study population after six years). Their lower completion rate is probably not due to the fact that students take a long time to complete their studies, because most level 1-3 certificate courses can be completed in a year or less and most students were enrolled for a year or less. It is more likely that there were some systematic differences between the students who enrolled in tertiary education sooner and those who enrolled a few years later, such as age and maturity. For example, students who stayed at school for longer and/or enrolled in tertiary programmes when they were slightly older may have had a greater level of motivation to complete their programme.

The characteristics of students in the early enrollers study population and the impacts of their tertiary study over the first three years are similar to those already reported for the main study population. This is illustrated in Figure 7, which compares the three-year impact estimates for the three study populations using a series of bar graphs. The levels of the bars for the main study population and early enrollers study population are close to each other.

Figure 7: Impacts of tertiary study after three years for each study population

Notes: Figures have been derived from the Integrated Data Infrastructure (IDI). Error bars show the 95% confidence interval. If the error bar crosses the vertical axis at zero, the impact estimate is not significantly different from zero.

In Table 7 and Figure 8, we show what happened to the early enrollers' employment rates, benefit rates and earnings when employed over a longer follow-up period. For level 1-2 certificates, the employment rate impacts are larger after five or six years than after three or four years. For level 3 and 4 certificates, there are signs of small reductions in impact sizes at four, five or six years after the study period compared with two or three years out. For example, the pattern of employment impacts for students who completed a level 3 qualification is 11.9 percentage points after three years, 10.8 percentage points after four years, 8.4 percentage points after five years and 5.5 percentage points after six years. The pattern of benefit rate impacts for this group is -8.0 percentage points after three years, ‑6.1 percentage points after four years, -5.3 percentage points after five years and -4.8 percentage points after six years. The main impact estimates for the level 4 certificate group also show a pattern of slight decline.

Despite experiencing significant growth in their monthly earnings over time, the average earnings of the students who were in employment generally did not surpass those of the matched youth who did not enrol in any tertiary education (and were also in employment at the same time in the follow-up period).

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Earnings were converted to December 2016 values. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

Figure 8: Six-year outcomes of the early enrollers study population

Impacts of tertiary education for more recent birth cohorts

To assess cohort effects, we compare the three-year impact estimates for youth in the early enrollers study population (who were born in 1990-92) with three-year impact estimates for a comparable population of young people who were born in 1992-94. We use exactly the same study design for both samples, with the exception of birth year. In both cases, the study population is restricted to youth who had finished their tertiary studies within four calendar years after their enrolment in year 11 at secondary school. Within both study populations, completers are those who gained a qualification within this window of time.

The impact estimates for the youth who were born in 1992-94 are given in Table 8 and illustrated in Figure 7. These results show slightly smaller benefits were gained by youth who were born in 1992-94 compared with those born in 1990-92, although these differences are not statistically significant. For example, the employment impact for those gaining a level 1-2 certificate is 4.4 percentage points rather than 5.7 percentage points. The employment impact for those gaining a level 3 certificate is 9.1 percentage points rather than 11.9 percentage points. The benefit rate impacts are also somewhat smaller. While the reasons for the smaller effects are unclear, we can conclude that any changes in educational policy that may have affected the more recent birth cohorts did not result in them gaining larger labour market benefits from tertiary education.

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Earnings were converted to December 2016 values. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

5.5  Factors influencing qualification completion rates#

Print this section

We have found persuasive evidence that the completion of a tertiary qualification is associated with positive employment outcomes for our population of poorly qualified school leavers. In this section, we briefly discuss the available evidence on factors that influence the post-school completion rates of low-achieving youth.

Simple completion rate statistics for the main study population were given in Table 2. We noted that students who had completed NCEA level 1 before leaving school were significantly more likely to complete their tertiary study than students without any school qualifications (58% compared with 48%). Females were more likely to complete than males, and Pacific people and NZ Europeans were somewhat more likely to complete than Māori students. Completion rates were higher than average in some broad fields of study (such as food, hospitality and personal services) and lower than average in other fields (such as information technology or mixed-field programmes).

The international literature on low-qualified school leavers indicates that poor learning skills, special learning needs such as dyslexia, drug and alcohol dependency, mental health problems, financial difficulties and housing difficulties can be important barriers to learning for disadvantaged youth (Chowdry et al, 2009; Crawford et al, 2011; National Institute of Adult Continuing Education, 2013). These barriers to learning often persist beyond school and make it more difficult for teenagers who have not succeeded in school to succeed in post-school programmes.

In the Wellington-based Competent Learners study, a longitudinal study of a representative sample of youth, a high proportion of those who left school without formal qualifications started a tertiary course before the age of 20 but left without completing it (Wylie and Hodgen, 2011). The reasons given for not finishing a course included finding the course too difficult and not doing well in it, losing interest, not finding the content or teaching enjoyable and personal reasons.

The literature suggests that youth who have achieved poorly in school often do not have the ability to sustain and complete post-school study programmes without extra support.

 

6  Sub-population impacts#

Print this section

The impacts of completing a tertiary qualification for different types of student and different types of qualification are briefly discussed in this section, using the data for the main study population and estimating impacts at 36 months after the tertiary education ended.

We focus on employment rate and benefit rate impacts and do not report sub-population results for earnings, due to our finding that the beneficial effects of tertiary education for low-qualified youth come mainly through improvements in the likelihood of getting a job. We also focus on the 51% of students who completed a qualification, given the evidence that non-completers do not improve their outcomes on average.

Appendix Tables 8 to 13 give our estimates of the impacts of completing a qualification on the employment rates and benefit receipt rates of various subgroups within the low-qualified school leaver population. Appendix Tables 8 and 9 give results for level 1-2 certificates. Appendix Tables 10 and 11 give results for level 3 certificates. Appendix Tables 12 and 13 give results for level 4 certificates. All tables give results for (a) the main study population and (b) the more recent birth cohorts (1992-94). We focus on statistically significant impact estimates in our summary.

6.1  Employment impacts#

Print this section

Gender

At level 1-2, the employment impacts were slightly larger for male students than females. In contrast, they were larger for females than males at level 3 and level 4. These differences in impact size could be due to differences in subject choices, to differences in unmeasured student characteristics or to other factors.

Ethnic group

At level 1-2, the employment impacts were similar in size across ethnic groups. At levels 3 and 4, the impact sizes for NZ European students and particularly NZ European females were larger than those for Māori and Pacific peoples. These differences in impact size could be due to differences in subject choices, to differences in student characteristics or to other factors.

Highest school qualification

Gaining a level 1-2 tertiary qualification raised the employment rates of students who left school with no qualifications but did not have a significant employment rate impact for those who had gained a level 1 qualification at school. At level 3 and level 4, the employment impacts associated with gaining a tertiary qualification were also larger for the students with no school qualifications than for those with NCEA level 1, although both groups gained significant benefits.

Type of institution

Most level 1-2 certificates were obtained at ITPs. Very few level 1-2 certificates were completed at PTEs, reflecting that PTEs were not able to offer level 1-2 qualifications until quite late in the study period, and the estimated employment impacts associated with these are imprecisely estimated, indicating that no conclusions can be drawn.

When we compare the employment impacts for students who gained level 3 or level 4 certificates at the two main types of institution, we find that those who studied at a PTE experienced slightly larger employment rate increases on average than those who studied at an ITP. These differences in impact size could be due to differences in subject choices, to differences in student characteristics or to other factors.

Field of qualification

At level 1-2, the results for the main study population show that certificates in information technology; engineering and related technologies; health; and food, hospitality and personal services had the largest positive employment impacts. However, inconsistencies in estimated impact sizes across the two study populations and/or a lack of sufficient data for the second study population suggest we should not place too much weight on these findings.

At level 3, the results for the main study population show that certificates in management and commerce; engineering and related technologies; creative arts; food, hospitality and personal services; architecture and building; and mixed-field programmes had the largest positive employment impacts. The impacts for other fields were not statistically significant. Again, the estimates of impact sizes by field of study are not very consistent between the two study populations.

At level 4, the largest positive employment impacts were associated with certificates in food, hospitality and personal services; health; engineering and related technologies; and architecture and building. The results for the later birth cohorts also showed sizeable impacts for students in some of these fields, although the impact sizes are smaller.

6.2  Benefit impacts#

Print this section

The results for level 1-2 certificates show few significant impacts on benefit receipt rates, with the exception of students in a few subject fields. Therefore, we focus on the results for level 3 and level 4 certificates.

Gender

At both level 3 and level 4, the reductions in benefit receipt rates three years after the completion of tertiary study were larger for female students than males.

Ethnic group

At both levels 3 and 4, the reductions in benefit receipt rates were larger for NZ European students and particularly NZ European females. Estimates for Māori and Pacific people were generally not statistically significant.

Highest school qualification

At both level 3 and level 4, the impact of gaining a tertiary qualification on benefit receipt three years later was larger for the students who had no school qualifications than those with NCEA level 1.

Type of institution

Reduced benefit take-up rates were estimated for students at both ITPs and PTEs. There is no clear evidence that one type of institution was associated with larger benefit rate reductions.

Field of qualification

Students who gained level 1-2 certificates in a few fields, such as information technology; engineering and related technologies; health; and food, hospitality and personal services, had lower benefit take up rates three years after completion than their matched comparisons.

At level 3, significant and sizeable benefit rate reductions were estimated for graduates in engineering and related technologies; architecture and building; management and commerce; society and culture; creative arts; and food, hospitality and personal services. The estimates are larger and more consistent across the two study populations for engineering and related technologies; architecture and building; management and commerce; and food, hospitality and personal services.

At level 4, significant benefit rate reductions were estimated for graduates in about two-thirds of the subject fields considered. Larger and more consistent results were estimated for architecture and building; health studies; society and culture; and food, hospitality and personal services.

7  Conclusion#

Print this section

7.1  Summary of findings#

Print this section

This paper has estimated the labour market benefits gained by low-qualified school leavers who attempt to improve their qualifications fairly soon after leaving school by enrolling in post-school certificate programmes. It differs from the previous study (Tumen et al, 2015) in allowing students a longer period of time after secondary school to enrol in and complete a tertiary qualification and then tracking them for a longer follow-up period after they finished their tertiary study (a minimum of three years rather than a minimum of two years). We also use a more extensive set of variables to select the matched comparison groups, which provide the counterfactual against which the impacts of tertiary study are assessed. Therefore, the estimates should be more accurate and are our preferred estimates.

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Earnings were converted to December 2013 values for the 2015 study and December 2016 values in the current study. Figures have been derived from the Integrated Data Infrastructure (IDI).

As a result of the change in study design, the average completion rate is higher - 51% of the youth who enrolled had completed a tertiary qualification after six years, rather than 46% after four years - yet our revised employment and benefit impact estimates are quite similar to the main estimates from the previous study, as shown in Table 9.

The estimated employment impact for all students who enrolled is an increase of 5.5 percentage points at 36 months (similar to the estimate of 5.3 percentage points at 24 months in the previous study). The average impact for all students who completed a qualification is 9.1 percentage points, similar to the previous estimate of 9.2 percentage points. For the combined group of students who completed level 1-3 certificates, the employment impact estimate is 8.6 percentage points, similar to the previous estimate of 8.5 percentage points. For students with level 4 certificates, the employment impact estimate is 10.1 percentage points, little different from the previous estimate of 10.8 percentage points.

Our revised estimates of the impact of tertiary study on benefit take-up rates are mostly smaller than those estimated previously, indicating slightly smaller benefit rate reductions. The most positive estimates are for students who completed a level 3 certificate, whose benefit receipt rate is estimated to be 6.1 percentage points lower three years after tertiary study, and students who completed a level 4 certificate, whose benefit receipt rate is estimated to be 8.3 percentage points lower three years after tertiary study.

Both papers provide evidence that young people who leave school without completing an NCEA level 2 certificate can improve their employment prospects by enrolling in a tertiary programme - but only if they complete a qualification.

Students who had not achieved any qualifications before leaving school were somewhat less likely to complete a tertiary qualification than those who had completed NCEA level 1, and if they did complete, they were substantially less likely to be employed three years after the end of their tertiary study spell. Despite these continuing disadvantages, the employment impacts associated with completing a tertiary qualification were larger for these students than for the NCEA level 1 achievers, suggesting that tertiary education may be more beneficial for the lower-achieving group of school leavers, provided they manage to complete a qualification.[13]

There were substantial variations in the size of the employment impacts for different demographic groups and different fields of study. We did not attempt to establish the reasons for these differences in impacts. There may have been interactions between the effects of personal characteristics and the effects of course characteristics. On the one hand, a higher rate of enrolment in food, hospitality and personal service programmes (for example), which had larger employment impacts on average than programmes in other fields, may have helped boost the post-study employment rates of females relative to males. On the other hand, it is also possible that the gender and ethnic mix of the students in different fields of study influenced the variations in employment impacts across these fields.

The evidence on longer-term impacts suggests that the employment effects of tertiary qualifications at levels 1-4 have generally reached their maximum extent by about three years after the completion of study (although there is some evidence of further increases after three years for the group with level 1-2 certificates).

Even after six years, there is no evidence that having a higher qualification led to higher levels of earnings for those with jobs. It is somewhat surprising that completing a level 4 qualification did not raise the wages and/or hours of this group of graduates. It is possible that any earnings benefits of tertiary education at this level take much longer to achieve.

7.2  Limitations of the study#

Print this section

Uncontrolled selection effects may be affecting our estimates and causing them to be biased. The students in this study were matched to non-participants using an extensive set of variables, but we cannot rule out the possibility that some part of the employment benefit we estimate was due to positive selection on unobserved characteristics such as learning skills, motivation and self-confidence rather than the effects of the education undertaken. In other words, the students who successfully completed a tertiary qualification may have had better employment outcomes than non-participants even if they had not studied. However, it is also possible that students who completed a tertiary qualification may have had worse outcomes than non-participants had they not studied. This would be the case, for example, if school leavers who were more likely to secure employment after leaving school did so and decided not to study, while others who chose to study did so because they were unable to obtain employment. We consider that our estimates are based on the best data currently available and standard methods.

Another limitation of the study is the fact that we cannot estimate the impacts of tertiary education on wage rates or hours of work, due to the lack of suitable measures in the IDI.

7.3  Discussion of implications#

Print this section

The average impact of the tertiary education that is undertaken by low-qualified school leavers represents the net effect of the positive impacts gained by completers combined with the lack of benefits experienced by non-completers. If only half of those who enrol complete a qualification, this significantly reduces the average benefit per student.

On average, the students in our study population experienced a post-study increase in their total monthly earnings of $159(measured in December 2016 dollar values), or 9.5%, as a result of their higher employment rate. The improvement in employment rates that followed completion of the tertiary education was increasing during the first two to three years but fairly constant or even declining after that rather than continuing to increase through time.

From a public policy viewpoint, it is unclear whether the economic benefits flowing from the education undertaken by the low-qualified school leavers in our study population would have exceeded the costs of their education. In principle, the public benefits of higher education include the (net present) value of the individual's increased earnings, higher taxes paid and reduced need for income support over their lifetime. The fact that the increases in total earnings that were apparent during the first two years after course completion were mostly sustained over a longer period (up to six years) is encouraging, suggesting some portion of the impact on the likelihood of employment could be sustained over the students' lifetimes. In addition to any economic benefits, there are likely to be other benefits from undertaking tertiary education that would need to be considered in a cost-benefit analysis.

Policies that either encourage more realistic enrolment decisions for this group of youth or raise their course completion rates (perhaps by providing more in-programme support) have the potential to improve the average benefits of the tertiary education that is undertaken. However, care is needed in assessing the likely improvement in benefits that will be gained. If unobserved differences in ability, persistence or motivation are playing some role in generating the pattern of impacts reported here, students who are provided with additional support to help them complete qualifications will not necessarily achieve outcomes as good as those currently achieved by more able students. Policies that raise achievement in schools and lower the proportion of students who become disengaged from learning and leave school without NCEA level 2 could provide a more effective alternative method of improving the labour market outcomes of this group of youth.

References#

Print this section

Chowdry, Haroon, Claire Crawford and Alissa Goodman (2009) “Drivers and barriers to educational success.” London, Department for Children, School and Families Research Report DCSF-RR102.

Crawford, Claire, Kathryn Duckworth, Anna Vignoles and Gill Wyness (2011) “Young people's education and labour market choices aged 16/17 to 18/19.” London, Department for Education Research Report, DFERR182.

Crichton, Sarah and Sylvia Dixon (2010) “Labour market returns to further education for working-aged adults.” Wellington, Ministry of Business, Innovation and Employment. http://www.dol.govt.nz/publication-view.asp?ID=380 [Treasury adjusted URL at 02 Jul 2024: https://thehub.swa.govt.nz/resources/labour-market-returns-to-further-education-for-working-adults/]

Crichton, Sarah (2013) “The impact of further education of the employment outcomes of beneficiaries.” Wellington, Ministry of Business, Innovation and Employment. http://www.dol.govt.nz/publications/research/publication-view.asp?ID=462

Earle, David (2010) “Benefits of tertiary certificates and diplomas: Exploring social and economic outcomes.” Wellington, Ministry of Education. http://www.educationcounts.govt.nz/publications/80898/benefits-of-tertiary-certificates-and-diplomas-exploring-economic-and-social-outcomes [Treasury adjusted URL at 02 Jul 2024: https://www.educationcounts.govt.nz/publications/80898/98274]

McIntosh, Steven (2004) “The impact of vocational qualifications on the labour market outcomes of low-achieving school leavers.” London, London School of Economics, Centre for Economic Performance Discussion Papers, No. 0621.

National Institute of Adult Continuing Education (2013) “Motivations and barriers to learning for young people not in employment, education or training.” London, Department for Business, Innovation and Skills Research Paper No. 87.

Rosenbaum, Paul and Donald Rubin (1983) “The central role of the propensity score in observational studies for causal effects.” Biometrika 70(1): 41-55.

Stromback, Thorsten (2010) “Earnings, schooling and vocational education and training.” Australian Journal of Labour Economics 13(3): 241-263.

Tumen, Sarah, Sarah Crichton and Sylvia Dixon (2015) “The impact of tertiary study on the labour market outcomes of low-qualified school leavers.” Treasury Working Paper 2015/07. http://www.treasury.govt.nz/publications/research-policy/wp/2015/15-07/

Wylie, Cathy and Edith Hodgen (2011) Forming adulthood: Past present and future in the experiences and views of the competent learners @ 20. Wellington, National Council for Educational Research and Ministry of Education.

Appendix#

Print this section

Notes: The numbers of students have been randomly rounded to base 3. Earnings were converted to December 2016 values. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: The numbers of students have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: The numbers of students have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: The numbers of students have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: The numbers of students have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: The numbers of students have been randomly rounded to base 3. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).

Notes: * Indicates that the impact estimate is statistically significant at the 95% confidence level. The numbers of students have been randomly rounded to base 3. Proportions and averages were calculated excluding those who were overseas three years after the end of the study spell. Figures have been derived from the Integrated Data Infrastructure (IDI).