The Treasury

Global Navigation

Personal tools

Treasury
Publication

Adult literacy and economic growth - WP 04/24

2.5  International comparisons

Countries differ markedly in the prevalence of Level 1 literacy skills, as Figure 6, Figure 7 and Figure 8 show.[15] The proportion of New Zealanders at Level 1 was more than double that of the best performing countries but less than half that of the poorest performing countries. Given the high correlation between scores on each of the three scales it is not surprising that the relative positions of countries change little between the prose, document and literacy scales. The best performing countries, consistently, were the four Nordic countries – Sweden, Denmark, Norway and Finland – together with the Netherlands, Germany and the Czech Republic. The poorest performing countries were Hungary, Poland, Italy and Portugal. New Zealand was part of a group of countries, mainly English-speaking, in the middle of the distribution. The proportion of the population with Level 1 literacy skills differed little between these countries.

Figure 6 – Prose literacy in IALS: percent of population aged 16-65 at Level 1, by country
Prose literacy in IALS: percent of population aged 16-65 at Level 1, by country.
^ = not significantly different from New Zealand
Source: OECD and Statistics Canada (2000).
Figure 7 – Document literacy in IALS: percent of population aged 16-65 at Level 1, by country
Document literacy in IALS: percent of population aged 16-65 at Level 1, by country.
^ = not significantly different from New Zealand
Source: OECD and Statistics Canada (2000).
Figure 8 – Quantitative literacy in IALS: percent of population aged 16-65 at Level 1, by country
Quantitative literacy in IALS: percent of population aged 16-65 at Level 1, by country.
^ = not significantly different from New Zealand
Source: OECD and Statistics Canada (2000).

These inter-country comparisons show that there is potential for many countries, including New Zealand, to perform better in terms of the proportion of their populations with poor literacy skills. New Zealand does not stand out, however, as having a long tail of achievement compared to other OECD countries, and in particular compared to other English-speaking countries.

Why some countries do better than others, and therefore what aspects of the best-performing countries might be worthy of imitating, is unclear. Differences in educational attainment and age structure between countries appear to explain relatively little of the differences in literacy skills.[16] What little is published on international differences in IALS tends to express doubt about the plausibility of the cross-country comparisons (Carey 2000; Blum et al 2001).[17] This is for a number of reasons including changes in the difficulty of items once they have been translated; differences in the motivation of respondents to undertake a survey of this type and length; sampling differences; and the method of processing missing answers. Blum et al (2001) conclude their analysis by saying:

The IALS survey, as it stands, should be treated with caution at national level and more so at an international level… On the basis of our analyses, it is not possible to assume that IALS measures only literacy. It seems to measure a combination of different factors: motivation (reflected in the different ways of filling in the questionnaire), understandings of what items mean, and differences in test taking behaviour more generally (p.244).

Putting these criticisms to one side for a moment, older New Zealanders seem to perform relatively well, compared to their counterparts in other countries, while younger New Zealanders do relatively poorly (Figure 9). On this basis, it might be thought that New Zealand school leavers are slipping behind most other OECD countries in terms of basic literacy skills.

Figure 9 – Prose literacy in IALS: percent of population aged 16-24 in Level 1, by country
Prose literacy in IALS: percent of population aged 16-24 in Level 1, by country.
Source: Online IALS search tool.

This theory, however, contradicts what is known from another international study of literacy skills. In 2000, New Zealand took part in the OECD’s first Programme for International Student Assessment (PISA) study. PISA measured the performance of 15-year old students in three areas – reading literacy, mathematics literacy and science literacy.[18] International comparisons of achievement are reported in (OECD 2001). In contrast to IALS, young New Zealanders performed particularly well in PISA compared to young people from other OECD countries. Most interestingly, a subset of the IALS prose literacy questions were also included in PISA, and Kirsch, de Jong, LaFontaine, McQueen, Mendelovits and Monseur (2002) are therefore able to express the performance of the 15-year-olds in PISA on the IALS prose literacy scale. Figure 10 shows the proportion of 15-year-olds in PISA who were estimated to be in Level 1 of the prose literacy scale. According to this analysis, New Zealand is now one of the leading countries in the OECD, along with the other English-speaking countries. Germany, Norway, Denmark and the Czech Republic, having done extremely well in IALS, are now amongst the poorer-performing countries.

Figure 10 – IALS prose literacy as measured in PISA: percent of 15-year-olds in Level 1, by country
Source: Kirsch et al (2002).

Why do the literacy skills of young New Zealanders seem relatively poor in IALS but good in PISA? One explanation might be the different time periods over which respondents were at school.[19] It is not clear, though, why there would have been a real improvement in the performance of secondary school students over the intervening period.[20] It is likely that the rankings reported in IALS and PISA are quite sensitive to the details of the survey methodology in the different assessments. In any event, the marked differences between Figure 9 and Figure 10, together with the more general criticisms of the comparative results from IALS, should urge caution in making judgements about inter-country differences in literacy skills.

2.6  Conclusion

According to the IALS survey, around 1 in 5 working-age New Zealanders (currently about 530,000 people) has Level 1 literacy skills. People with Level 1 skills will differ considerably in their ability to perform particular tasks. Most, however, are able to read, and to locate and use information from a straightforward text, but cannot consistently perform more difficult or sophisticated tasks especially those involving unfamiliar types of texts. Many can perform tasks which IALS rated at Levels 2, 3, 4 or even 5, but cannot do these consistently.

Most New Zealanders with Level 1 literacy skills have completed the bare minimum of schooling. This relationship is likely to be complex, though, with poor literacy both the cause and the result of low achievement at school, and with schooling and literacy both influenced by common factors such as innate ability. Some of the reading problems experienced by people in Level 1 are almost certainly related to language barriers, rather than to literacy issues: 19% of the people in Level 1 did not use English as their main language at home.

In 1996, when IALS was conducted, 45% of working-age New Zealanders with Level 1 literacy skills were employed either full-time or part-time. Most were employed as either agricultural and fishery workers, plant and machine operators, or in elementary occupations. Looking only at simple correlations, people with Level 1 skills were much less likely to be employed than people with higher skills and people who were employed tended to have lower earnings.

The introduction to the official IALS report states that Level 3 literacy skills are “considered a suitable minimum for coping with the demands of everyday life and work in a complex, advanced society”. A host of commentators use this comment, unsubstantiated anywhere in the body of the IALS report, to identify everyone with Level 1 and 2 skills as being unable to function in a knowledge economy and therefore, by extension, in need of literacy training. In New Zealand, about 45% of the working-age population are in either Level 1 or 2; in France this figure was as high as 75%. The suggestion that this whole portion of the population is not coping in society is extraordinary, not the least because, in all countries, most people at Levels 1 and 2 considered that their literacy skills were good or excellent, in the context of both their jobs and their daily lives. In any event, the question which the current paper addresses is not whether people are coping with the demands of everyday life and work but whether their participation and productivity would improve with increased literacy skills, and this is covered in the following Chapters.

The international comparisons in IALS should be viewed with a degree of scepticism. At face value, however, New Zealand does not stand out amongst OECD countries as having particularly poor literacy skills. The proportion of people in New Zealand with Level 1 skills is similar to the proportion in other English-speaking countries such as Australia and the United Kingdom, and lies in the middle of the range of OECD countries. This does not mean, though, that there is no capacity for significant improvements in adult literacy in New Zealand.

Notes

  • [15]Results from 19 countries are reported here. Chile and Slovenia, which took part in IALS, are excluded here as they are not in the OECD. Total country results, rather than results for different language or geographical groups, are reported for Canada, Switzerland and the United Kingdom.
  • [16]If New Zealand had the same pattern of educational attainment as Sweden, for example, the proportion with Level 1 prose literacy skills would still be twice as large. New Zealand also has a comparatively youthful age structure compared most European countries. The issue of age and educational attainment is also addressed in Chapter 11 of Carey (2000), where a similar conclusion is reached.
  • [17]Carey (2000) was a report commissioned by the European Union and motivated by large differences in performance between European countries. In particular, 75% of people in France were found to be at either Level 1 or 2 in IALS: a finding that led the French to withdraw from the reporting phase and to be highly critical of the survey.
  • [18]A total of 3,667 New Zealand students from 153 high schools took part. As in IALS, the PISA assessments focused on real-life tasks rather than their mastery of the school curriculum. Students were asked both multi-choice and open-ended written questions after reading magazine articles, graphs, tables, and other pieces of written or visual information.
  • [19]The 16-25 year-olds who took part in IALS were 15 years old at some stage between 1986 and 1995, while those who took part in PISA were 15 years old in 2000.
  • [20]The current emphasis on improved teaching of literacy and numeracy in schools began too recently to have had an effect on students in 2000. Nor has there been evidence of improved performance as measured by other tests of achievement. The proportion of students leaving school with no qualifications (that is, with no School Certificate passes), for example, has been relatively stable since 1989, at between 16% and 19% (Ministry of Education 2004).
Page top