Appendix A - Measurement issues

Measurement issues, as we discuss them here, refer to difficulties in constructing a measure that precisely corresponds to the phenomenon that you wish to represent. They are not necessarily problems, but they do need to be taken in to account when developing policy based on the indicator. They show that the indicator is not always a good representation of New Zealand's private tertiary returns.

A.1  Comparability of qualifications

A.1.1  ISCED and NCEA classifications

New Zealand has a large proportion of tertiary degrees that are classified outside of the standardised Bologna structure of higher degrees (48% compared to 27% OECD average). This difference partly stems from differences between the NCEA and other international structures of pathways that lead to Type B tertiary degrees (ISCED 5B). The result of some vast international differences is that not all structures including the NCEA can be mapped across to tertiary degrees in a completely standardised fashion across the OECD. In particular, pathways through NCEA (3B and 3C to 4B or 5B) may explain a broader range of degrees with shorter overall durations classified as tertiary Type B in New Zealand, compared to the OECD average.

A.1.2  Lower proportion of post-graduate degrees

A second issue is that Type A (ISCED 5A and 6) degrees in the OECD tertiary qualification specifications are aggregated across Bachelor and higher degrees. However, New Zealand has one of the lowest shares of post-graduate degrees among the OECD (ranked 22^nd with 6% of tertiary degrees at the Post-graduate and higher research qualifications, compared to the 18% OECD average). The significantly lower proportion of post-graduate degrees compared to the OECD is a factor that may contribute to New Zealand's overall tertiary returns gap.

A.2  Earnings comparisons

The OECD uses current earnings by age group to estimate the gross earnings benefit of tertiary education, relative to upper-secondary education. That introduces two important difficulties.

A.2.1  Are wages for tertiary graduates low or are wages high for non-graduates

The gross earnings benefit in the indicator is a measure of the premium that tertiary graduates earn over upper-secondary graduates. Conceptually, a small premium could be due to either low tertiary earnings or high upper-secondary earnings. Distinguishing between the two is important, since high upper-secondary earnings could indicate that the value New Zealand's secondary education is high, rather than the alternative conclusion that our tertiary earnings are low.

Indeed, for upper-secondary qualifications, New Zealand is ranked eighth for gross earnings benefit, relative to lower-secondary qualifications, in the OECD, slightly higher than the average and well above the median. Again, that is a gross earnings premium relative to people without upper-secondary education so it cannot be taken as definitive evidence of relatively high earnings for those with upper-secondary qualifications; however, it suggests that the small gap between upper-secondary and tertiary gross earnings may not be entirely due to low tertiary earnings.

In addition, since age-earnings profile analyses are based on a snapshot (cross-section of degree holders by age at a given time), historical or cohort effects may influence these results. For example, since participation in tertiary education in New Zealand increased significantly in the early 1990s,[64] there is potentially a larger cohort of high ability/academic achievers among the mature and older New Zealanders with upper-secondary as their highest degree. This cohort was also more likely to enter professional jobs directly after the completion of high school before the 1990s, resulting in higher earnings returns for upper-secondary degree holders due to the cohort's effect. This effect is not expected to repeat over time.

A.2.1  The unaccounted effect of partial completion

Low or partial completion rates can result in unmeasured positive returns to tertiary education. Domestic research shows that a person who passes all of their tertiary courses, but does not complete the qualification, can earn 10-25% more than someone with only upper-secondary qualifications can.[65] Including those people's earnings in the average earnings for someone with upper-secondary qualifications is likely to reduce the measured gross earnings benefits for tertiary qualifications, relative to the true value.

Furthermore, New Zealand has an extremely low rate of qualification completion relative to the OECD average with 58% of students completing Type-A qualifications, compared to an OECD average of 69% (Figure 17). Thus, returns to tertiary education in New Zealand may well be much higher than qualification-centric data suggests.

Figure 17 - Tertiary completion rates

Note: Countries where data is unavailable are not reported.

Source:  OECD

However, completion statistics should not be taken at face value. Scott explains that:

• New Zealand has a very high proportion of part-time students, who are less likely to complete. Indeed, it appears that many mature students never intend to complete a qualification at all but are only interested in the individual course, yet they will count as non-completers.
• Many New Zealanders change qualifications during the course of study, which results in the first qualification being counted as incomplete. This is also part of the reason for New Zealand's high proportion of Type-B degrees.

A.2.1  Life cycle and demographic effects

To what extent are returns to tertiary degrees in New Zealand affected by the relative progression of earnings for mature workers? Cohort effects, in particular, could affect returns to education through this effect. An example of cohort effects is when mature workers hold a disproportionately higher share of older and shorter degrees (OECD note older nursing or technical degrees), contributing to lower returns to education due to lower earnings for mature workers.

Tertiary education is generally associated with greater earnings growth over the life cycle compared to upper secondary. One of the main factors that contribute to this greater growth is that work experience is combined with greater new skill formation and further on-the-job learning. Progression of earnings into mature working years is mechanically important in the NPV and IRR calculations due to the greater discounting of more distant future earnings gains. Both cohort effects and actual earnings progression due to learning with experience and job progress can affect earnings progression of the mature age groups.

We calculate earnings progression from the age bracket 25-34 to 55-64, by degree type. We find that the earnings progression for Type A tertiary degrees is very similar for New Zealand and the OECD average (a 30% growth in average earnings for the 55-64, compared to the 25-34 age group, or a 1.3 ratio in both cases).

The earnings progression for Type B tertiary degrees, however, is very flat, showing a much narrower growth of earnings over time: 6% growth in average earnings over a twenty-year-period, or a 1.06 ratio for men and 1.0 for women (and a lower ratio of 0.95 combined across gender compared to the OECD average of 1.04).

Lack of earnings progression and generally lower earnings for Type B degrees relative to upper secondary are important in reducing New Zealand's returns when combined across all tertiary degrees. This type of observed earnings effect usually reflects a combination of lower opportunities for continued on-the-job learning that leads to higher earnings, and/or cohort effects reflecting older or shorter degrees.