Appendix A

Survey methodology

When SoFIE commenced in 2002 a total of 15,000 households were approached, of whom around 11,500 (77%) agreed to participate. In the initial interview, data was collected from around 22,000 individuals aged 15 and over. All respondents in the original sample (original sample members) are followed over time, even if their household or family circumstances change, forming a longitudinal sample. In later waves new cohabitants of the sample members are interviewed but asked only a reduced set of questions. These additional sample members are not followed if in future waves they no longer live with the original sample member. For these reasons only original sample members are included in this analysis. All SoFIE interviews are carried out face-to-face using computer assisted interviewing.[58][59]

Statistics New Zealand provides a longitudinal weight which accounts for non-response and aligns the composition of the sample with that of the New Zealand population in October 2002. Interviews of SoFIE were conducted throughout the year with the sample spread evenly over the 12-month wave period. Each respondent is asked about the previous 12 months (their annual reference period). As a result of this continuous interviewing, there are 12 reference periods in each wave. Some variables collected in each wave of SoFIE, such as age, can be measured at the household interview date or at a point in the reference period. Figure B1, Appendix B shows the relationship between these dates for a hypothetical SoFIE respondent.

At the end of the SoFIE health module respondents were asked to give permission for their data to be linked to information on hospitalisations and cancer registrations held by the New Zealand Health Information Service (NZHIS) back to 1990. Of the 17,615 adults who responded to the survey in all three waves, 13,980 agreed for their information to be linked, which is a consent rate of 79.4%.[60] Statistics New Zealand provides the name and address information of consenters, along with some basic demographic information, such as date of birth, to MoH who uses this to match it to a National Health Information (NHI) number. This was done automatically were possible, with some manual matches made in more difficult cases. The NHI is a unique identifier that is assigned to every person who uses health and disability support services in New Zealand. It is recorded alongside any hospital visits and cancer registrations. Therefore if respondents are matched to a NHI number their hospital and cancer information can then be obtained. Those respondents who have had no contact with the health sector in New Zealand will have no NHI number and therefore will not be matched successfully. MoH estimates that the current coverage of NHI numbers is over 99% of the resident New Zealand population. As a result, this type of non-match would only be expected to be small. MoH believes that the majority of non-matches are likely to be owing to the respondents' details from SoFIE being significantly different from those held with the NHI, therefore preventing successful matching. Of those respondents who gave consent for their data to be matched, around 96% (13,475 respondents) were successfully matched resulting in a matched, consent rate of 76.5%.

While Statistics New Zealand provides weights to account for non-response and to benchmark results to census population estimates these weights do not adjust for non-consent and non-matching. If the weights provided by Statistics New Zealand were used to make estimates of counts from the matched consenters then these would be underestimates. Further other estimates such as means may be biased if the matched consenters differ from the non-matched or non-consenters in important ways. Analysis of the characteristics of consenters indicated consent rates differed significantly by various characteristics. Of these characteristics age, ethnicity and presence of chronic diseases were of particular importance for this work. The standard longitudinal weights were therefore adjusted to account for non-consent and non-matching using the weighting classes of age, ethnicity and the presence of chronic diseases.[61][62] All analysis in this paper is based on the linked sample with the adjusted longitudinal weights used to realign the sample with the population (adjusted longitudinal weight) as opposed to the weights provided by Statistics New Zealand (standard longitudinal weights). Where possible the same analysis was carried out on the full sample using the standard longitudinal weights to determine the possible impact of using the restricted sample and there was little difference in the results.

Population and sample of interest

The questionnaire is only asked to those aged 15 and over. To ensure there is full information on respondents in all waves, the analysis is focused on those aged 15 and over at the end of the reference period in Wave 1 who remain eligible and respond in all three waves of the survey (adult longitudinal respondents). This is the balanced panel made up of 17,615 respondents in Waves 1-3; an unadjusted attrition rate of 20.5%. Once this is adjusted, to remove those people who move out of the scope of the survey or die, then the adjusted attrition rate is 17.2%. Those over working age or who are full-time students in each wave are excluded from the analysis. The results are therefore representative of the usual adult resident population of New Zealand who lived in private dwellings on the main islands of New Zealand in 2002/03 who are working age non-students. Around three-quarters of the 17,615 adult longitudinal respondents are working age non-students in Wave 1, 2 or 3.[63][64] Just over 76% of respondents (13,475) consent for their responses to be linked to NZHIS information and were subsequently matched.[65] Statistics New Zealand provided the name and address information of consenters, along with some basic demographic information, such as date of birth, to MoH who used this to match it to a National Health Information (NHI) number. This was done automatically were possible (for 80% of cases), with some manual matches made in more difficult cases. The NHI is a unique identifier that is assigned to every person who uses health and disability support services in New Zealand. It is recorded alongside any hospital visits and cancer registrations.

Limitations and strengths of SoFIE

The SoFIE data has a few limitations. As with all surveys, there is potential for non-response error - that is, errors because not all potential respondents take part in the survey. Unlike in cross-sectional surveys, non-response in longitudinal surveys has a second element as respondents can also choose whether to respond in each wave. If this non-response (known as attrition) is non-random (that is, the characteristics of those who do respond are systematically different from those who do not) then any inferences based on analyses of the data may be biased. In addition, where longitudinal data is linked to other sources, information is only observed for part of the sample (those who agree to the linkage) and these differences could also be non-random and potentially bias results. While there are differences in the response, consent and matching rates in SoFIE there are no groups of interest that do not contain any respondents. The weights (both the standard weights provided by Statistics New Zealand and adjusted longitudinal weights to take account of non-consenters) go some way to restore the distribution of respondents over the variables of interest and any bias as a result of this should be small when making inferences about the population as a whole.[66] However, it should be remembered that as a longitudinal survey those who are most unhealthy will die or move into institutions where they may not be able to be traced, meaning that the SoFIE population is likely to be healthier than the wider New Zealand population it represents.

A further limitation is that not all variables are available in all waves. An indicator for psychiatric conditions is only available in Wave 3 and an indicator for cancer is only available for the subset of respondents who agreed for their data to be matched to the Cancer Registrations database and were successfully linked. This potentially reduces the sample size considerably if only Wave 3 matched consenters are considered. Making an assumption about the presence of psychiatric conditions for Waves 1 and 2 and coding the non-consenters' cancer status as “unknown” rather than missing goes someway to countering this problem, allowing analysis to be undertaken on all three waves rather than the restricted sample.

While SoFIE is a longitudinal survey there are only currently three waves of information. While this provides a wealth of information for variables that do not change very frequently, such as diagnosis of new diseases, modelling the impact of these variables with such a short span of data is difficult.

Lastly, if dependants of respondents have ill health or chronic diseases this may also affect the respondent's labour market participation. The SoFIE questionnaire does not allow “carers” to be identified except when the ill health of a family member is given as a reason for inactivity. In addition, when people do report the ill health of a family member as a reason for inactivity the cause of ill health cannot be identified or attributed to a specific chronic disease or illness. The effect of this on labour market participation is therefore not explored in this analysis.

Despite its limitations, SoFIE collects a wealth of information on respondents over time. This allows a range of labour market transitions, durations and repeat occurrences of respondents to be analysed. It allows comparison of labour market activity and disease presence at more than one point in time. Further, attempts to account for the presence of unobserved variables can be made given that the same respondent is being monitored over time. The linking of SoFIE data to cancer and hospitalisation information adds further depth to the SoFIE data and this additional information is subject to less reporting error than additional questioning of respondents.

While there are differences in response and consent rates by respondent characteristics, for a longitudinal survey of this kind the response and consent rates are high by international standards.