KiwiSaver and the Accumulation of Net Wealth (WP 14/22)

Abstract#

The objective of this paper is to analyse the extent to which membership of KiwiSaver has been associated with greater accumulations of net wealth. The paper utilises two linked sources of data which cover the period 2002 to 2010: Statistics New Zealand's Survey of Family, Income and Employment and Inland Revenue Department administrative data on KiwiSaver membership. Two approaches are employed: difference-in-differences (where the outcomes of interest are changes in net wealth) and various panel regression techniques. Results appear consistent with earlier evaluations of KiwiSaver. Neither approach suggests KiwiSaver membership has been associated with any positive effect on net wealth accumulation.

Acknowledgements#

The authors acknowledge the contribution of the Inland Revenue Department and Statistics New Zealand in generating and making available a confidentialised linked data set on which the analyses in this paper is based. We also wish to thank our reviewers, Murat Genc, John Gibson, Mark Vink and Kristie Carter for their helpful comments and suggestions, as well as our editor, John Creedy.

Disclaimer#

The views, opinions, findings, and conclusions or recommendations expressed in this Working Paper are strictly those of the author(s). They do not necessarily reflect the views of the New Zealand Treasury or the New Zealand Government. The New Zealand Treasury and the New Zealand Government take no responsibility for any errors or omissions in, or for the correctness of, the information contained in these working papers. The paper is presented not as policy, but with a view to inform and stimulate wider debate.

Executive Summary#

KiwiSaver is a voluntary savings scheme aimed at increasing the retirement wealth of a target population. Its introduction in 2007 was prompted by a view that household saving in general appeared to be low and declining, and that there may be some who would reach retirement with an accumulation insufficient to allow them to sustain their pre-retirement standard of living.

The objective of this paper is to analyse the extent to which the KiwiSaver scheme has resulted in greater accumulations of net wealth amongst its members, relative to that which might have been expected in the absence of the scheme.

The paper utilises two linked sources of data, the Survey of Family, Income and Employment (SoFIE) and administrative data from the Inland Revenue Department (IRD). SoFIE is a longitudinal data set which includes, as well as a wide range of socio-economic variables, details of individual assets and liabilities. Administrative data from IRD provides individual KiwiSaver membership information. The resulting linked data set covers the eight-year period to 2010. Asset and liability data were measured four times during this period and form the basis for analysing changes in net wealth.

The analysis is based on two approaches. The first uses a difference in difference (DiD) technique, the second uses different panel regression techniques. The DiD technique compares outcomes (in this case changes in net wealth) before and after the introduction of a programme such as KiwiSaver, across two groups (those in the programme and those not). In this way, those who are not members of the scheme form a control group.

Results of the DiD analysis suggest the accumulation of net wealth by members of KiwiSaver was some $16,000 less than the comparable accumulation of non members. Further, in an attempt to hold some of the other factors likely to affect net wealth accumulation constant, the DiD analysis was repeated by age, gender, education, income, wealth, partner and home ownership status. There was a positive effect in only five of the 28 cases examined. In three of these cases the estimated effect was small. In one case, however, the estimated effect was relatively large, at $20,000 in favour of KiwiSaver members. All other cases indicated KiwiSaver members' accumulated less than non-members.

The DiD analysis only holds one factor constant at a time however. To address limitations of the DiD technique various fixed and random effect panel regression models are estimated in which changes in net wealth are related to many factors simultaneously. These include: KiwiSaver membership; income; net wealth; age; gender; partnership status; home and investment property ownership; ethnicity; if the respondent was born in New Zealand; education; labour force and health status. With four observations over time on assets and liabilities in SoFIE it is possible to measure three changes in net wealth for each of approximately 10,000 individuals. This provides nearly 30,000 observations for inclusion in each regression.

The effect of KiwiSaver on net wealth accumulation is estimated to be negative in all model specifications examined, although the coefficient estimates are typically not statistically significant at conventional levels. While the findings of this study appear clear, caution is still warranted in their interpretation. Firstly there is evidence of significant measurement error in key variables. In particular, the distributions of changes in net wealth are very wide and there is little correlation in these changes over time for individuals. Attempts to control for this measurement error in regressions were made; however, these may have only been partially successful. Second, the data are only available up until late 2010, meaning some KiwiSaver members would only have been enrolled for a relatively short time. However, regression estimates suggest that tenure in KiwiSaver has little effect on net wealth accumulation.

The results of this study are consistent with the findings of the previous evaluation of KiwiSaver. For example, Law et al. (2011) found no association between KiwiSaver membership and expected retirement income outcomes (an important element of which must be net wealth at retirement). The current study, which uses completely different techniques and data to that initial evaluation, provides a second piece of evidence which suggests that KiwiSaver has not been associated with greater accumulation of net wealth by its members and hence improved retirement income outcomes.

1 Introduction#

KiwiSaver[1] is a voluntary savings scheme aimed at increasing the retirement wealth of a target population. Its introduction in 2007 was prompted by a view that household saving in general appeared to be low and declining, and in particular, there may be some who would reach retirement with an accumulation insufficient to allow them to sustain their pre-retirement standard of living (see for example, Treasury, 2007). The available micro-evidence relating to individual and household saving, however, does not necessarily support this view (see for example, Scobie, Gibson and Le, 2005, Le, Scobie and Gibson, 2007, Henderson and Scobie, 2009, and Le, Gibson and Stillman, 2010 and 2012).

Law, Meehan and Scobie (2011) evaluate the performance of KiwiSaver in terms of four key dimensions. The authors utilised a comprehensive survey conducted by Colmar Brunton in 2010 on 825 people that included both members and non-members of KiwiSaver. The key findings of the study were that:

1. approximately 1/3 of contributions to KiwiSaver represented new savings, while 2/3 would have resulted anyway in the absence of the scheme;
2. no association was found between KiwiSaver membership and expected retirement income outcomes;
3. in terms of standard measures of programme efficacy (target effectiveness and leakage) KiwiSaver was found to have performed poorly, with leakage to the non-target group estimated to be 93%; and
4. after accounting for other factors, including the schemes impact on public saving, KiwiSavers effect on national saving was found to be negligible.

The current paper is a continuation of the previous work. The objective is to analyse the extent to which the KiwiSaver scheme has resulted in greater accumulations of net wealth amongst its members, relative to that which might have been expected in the absence of the scheme. Though this objective clearly relates to all of the key findings of the earlier evaluation (and particularly the first and second), the approach used and the data employed in the current analysis are very different.

The paper utilises two linked sources of data, the Survey of Family, Income and Employment (SoFIE) and administrative data from the Inland Revenue Department (IRD). SoFIE is a longitudinal data set which includes, as well as a wide range of socio-economic variables, details of individual assets and liabilities. Administrative data from IRD provides individual KiwiSaver membership information. The resulting linked data set covers the eight-year period to 2010. Asset and liability data were measured four times during this period and form the basis for analysing changes in net wealth.

The analysis is based on two approaches. The first uses a difference-in-differences (DiD) technique. This technique compares outcomes (in this case changes in net wealth) before and after the introduction of a programme such as KiwiSaver, across two groups (those in the programme and those not). In this way those who are not members of the scheme form a control group.

The DiD analysis only holds one factor constant at a time however. To address this limitation various fixed and random effect panel regression models are estimated in which changes in net wealth are related to many factors simultaneously. These include: KiwiSaver membership; income; net wealth; age; gender; partnership status; home and investment property ownership; ethnicity; if the respondent was born in New Zealand; education; labour force and health status. With four observations over time on assets and liabilities in SoFIE it is possible to measure three changes in net wealth for each of approximately 10,000 individuals. This provides nearly 30,000 observations for inclusion in each regression.

Results appear consistent with those of the earlier evaluation of KiwiSaver undertaken by Law et al. (2011), particularly with respect to the second of their findings described above. That is, neither the DiD or regression analyses undertaken in the current study suggest that KiwiSaver membership has had any positive effect on net wealth accumulation.

The remainder of this paper proceeds as follows. Section 2 describes the data. In order to inform the choice of an appropriate outcome measure, section 3 presents information on both the distributions and movement within those distributions, of net wealth, changes in net wealth and savings rates respectively. Sections 4 describes the DiD method and presents results. Panel regression techniques are outlined and results discussed in section 5. Conclusions are drawn together in the final section.

2 Data#

This paper uses individual unit record data from two sources. The first is the longitudinal Survey of Family, Income and Employment (SoFIE). The preparation of SoFIE for analysis is similar to that of Law and Meehan (2013), except that the timing of the current analysis allows the incorporation of the full eight waves of the survey. The second data source is administrative. Provided by the Inland Revenue Department (IRD), this data gives information relating to KiwiSaver membership.

SoFIE, the primary data source for this study, is a longitudinal survey conducted by Statistics New Zealand where the original sample members are tracked and surveyed each year. It began in October 2002 with an original sample size of about 11,500 households, amounting to over 22,000 individuals 15 years of age and over. It concluded in September 2010 after running annually for a total of eight years (waves). The core survey collects annual information on individual and family characteristics, as well as labour market and income spells. In alternate years health, and assets and liabilities modules are included respectively.

The assets and liabilities module was included in SoFIE waves 2, 4, 6 and 8 and forms the basis for assessing the effects of KiwiSaver membership on net wealth accumulation. This module allows for the measurement of net wealth at the individual rather than family level, which is relatively rare internationally.

Interviews for each wave were evenly spread over a 12 month period so that some households were interviewed in October and others the following September. However, we index all asset values to the mid-point of the relevant wave. Asset values for wave 2 are therefore indexed to approximately 31 March 2004, wave 4 asset values to 31 March 2006, wave 6 asset values to 31 March 2008 and wave 8 asset values to 31 March 2010.

Indexation was important during this period, with strong house price growth at times potentially leading to non-trivial increases in individuals' net wealth even within the interview period of a particular wave. Fortunately respondents in SoFIE were asked not only for the value of any residential property they owned but also to provide a valuation date. We used this date, together with detailed regional house price indices from Quotable Value (QV) (aggregated to the six major SoFIE regions) to index housing assets as described in the previous paragraph.[2] For all other assets the Consumer Price Index (CPI was used).[3]

SoFIE required a great deal of careful cleaning in order to minimise loss of observations due to question non-response or apparent errors in recording of individual information.[4] Wherever possible we made use of the longitudinal nature of the data to attempt to correct for this. For example, if we observed an individual owning a house worth just $1 in wave four we would examine their housing assets in other waves. If it turned out that that same person in wave two owned a house worth say $900,000 and in wave 6 worth $1,100,000 we changed the value recorded in wave four to $1,000,000. Similar anomalies or non-response were observed across most of the variables used in this analysis and so are too numerous to mention here. For more information about SoFIE and some of the problems researchers can expect to encounter, see for example Scobie and Henderson (2009) or Carter et al. (2010).

Although the final wave of SoFIE included a module on KiwiSaver, for the current analysis information on KiwiSaver membership is instead sourced from administrative data from IRD. The primary reason for this is that the administrative data provides a more complete picture of KiwiSaver membership over time. In addition, it appears that KiwiSaver membership in SoFIE may have been underreported, likely due to the way respondents were routed to the KiwiSaver module.[5] A detailed comparison of the KiwiSaver information contained within SoFIE and provided by IRD is available in Samoilenko and Law (2014). While the process of preparing a combined dataset for analysis was completed by the authors of the current study, Statistics New Zealand (SNZ) independently ‘matched’ individuals’ present in both datasets. In particular, assigning a common individual identifier, or person specific identifier, to individuals present in each data set thereby allowing them to be linked. Details of this process are provided by Gray (2012).

A number of restrictions to our sample are required for analysis. First, only those eligible to join and enjoy the full benefits of KiwiSaver based on their age at wave 1 are included (ie, those aged 15 to 60). Second, Statistics New Zealand only provides longitudinal survey weights for those respondents who were original in scope sample members of SoFIE. As these weights are required for much of our analysis, a further restriction to the sample is necessary. A further requirement, imposed by the difference in difference analysis set out in Section 4, is that individuals have a complete longitudinal history having responded to all eight waves of SoFIE. Finally, to allow for the calculation of savings rates, only those individuals with positive incomes are included.[6] Even with these restrictions a large sample remains, covering approximately 10,000 individuals in each year between 2002 and 2010. 39% of these individuals were KiwiSaver members in 2010 which is remarkably similar to the membership levels reported by Law et al. (2011).

3 Outcome Measures#

Given the data available to us, there exist a range of possible outcome measures that could be used to examine the performance of KiwiSaver in terms of facilitating improved retirement income outcomes for its members. These include net wealth, changes in net wealth and savings rates, with the latter being the most analytically appealing. However, it is well understood that survey data on assets, liabilities and personal income can be prone to measurement error.[7] As each potential outcome measure may be more or less prone to such measurement error, the choice of outcome measure in this case cannot be made solely based on its analytical appeal. Therefore, in order to inform the choice of an appropriate outcome measure, this section presents information on both the distributions and movement within those distributions, of net wealth, changes in net wealth and savings rates respectively.

3.1  Distributions#

The distribution of the nominal unweighted estimates of net wealth (derived as the sum of an individual's assets less the sum of their liabilities) for each of waves 2, 4, 6 and 8 are summarised in Table 1. Both the mean and median levels of net wealth increase substantially over time.[8] However, as the sample members age with successive waves, part of this observed increase is simply due to an ageing effect as typically individuals accumulate wealth over their working lives. In three of the four waves over five percent of observations on net wealth are negative. The spread between the 1^st and 99^th percentiles also increases over time ranging from approximately $1 million in wave 2 to $1.6m in wave 8. Similarly, the inter-quartile range increases from $180,000 in wave 2 to $290,000 in wave 8.

Table 2 presents distributions for the changes in individual net wealth that occur between waves 2 to 4, 4 to 6, and 6 to 8 respectively. The smaller average increase in net wealth between wave 6 and 8 relative to increases in net wealth between earlier waves is likely a reflection of the impact of the global financial crisis which occurred during these years. Compared to net wealth, the distributions of changes in net wealth appear more concentrated, particularly around the middle. In particular, the inter-quartile range remains relatively constant over the period, at around $80,000.

The distributions of individual savings rates between waves 2 to 4, 4 to 6 and 6 to 8 are provided in Table 3.[9] These are calculated by dividing the change in net wealth for an individual between any two consecutive waves (that include asset and liability information) by their gross income over the period. For example, the saving rate for an individual between wave 2 and 4 is simply net wealth in wave 4 less net wealth in wave 2 divided by the sum of income earned in waves 3 and 4. Formally and expressed as a percentage:

Given the wide range in both levels and changes in net wealth already discussed, it is not surprising that when adding income into the equation in order to calculate savings rates; those savings rates have a very wide dispersion. In particular the spread between the 1^st and 99^th percentiles of savings rates is close to or above 3,000% for most of the period. That is thirty multiples of income. The inter-quartile range for savings rates is also substantial, at well over 100% for most of the period.

3.2 Transitions#

As well as consideration of the distributions of our potential outcome measures it is helpful to consider the extent that individuals move within these distributions over time. In particular, lack of persistence over time may be indicative of measurement error. This subsection therefore presents transition probability matrices for net wealth, changes in net wealth and savings rates respectively.

In the present case these indicate the conditional probability that an individual beginning a period in a particular quintile of a distribution, will be in that quintile (or any other) at the end of that period. Such transition probabilities can be calculated over any time period. Given data available to us 2, 4 and 6 year transition probabilities are calculated for net wealth and presented in Table 4.

These results give a picture of the extent of mobility within the distribution of net wealth. In the absence of any mobility across quintiles of the distribution the diagonal elements of each matrix (bold italics) would be 1.00 and all off-diagonal elements would be zero. The greater the off-diagonal elements the greater the degree of mobility. For example, consider the top left hand corner element in panel A of 0.74. This indicates that on average a high proportion of individuals (74%) who were in the first quintile of the net wealth distribution in Waves 2, 4 or 6, were also located in the first quintile of the distribution two years later in Waves 4, 6 or 8 respectively. Reading across that first row of panel A, on average 18% of those in the first net wealth quintile had moved up to the second quintile of the net wealth distribution by the next wave containing asset and liability information.[10]

While the level of persistence in net wealth diminishes somewhat with the length of transition there are some clear patterns common to all transition lengths in Table 4. First, those individuals who begin the period in either the top or bottom quintile of the net wealth distribution are likely to remain in those same quintiles at the end of the period. In general there is also a degree of persistence in the middle of the net wealth distribution with over 50% of individuals starting in quintiles 2, 3, or 4, remaining in those same quintiles 2 years later for example. Further, in most cases where individuals do transition to a different net wealth quintile over time, it is most often one immediately adjacent to that where they started. In other words, someone beginning the period in say quintile 3 is far more likely to end the period in either of quintiles 2 and 4 than quintiles 1 or 5.

Transition probabilities for our second potential output measure, changes in net wealth, are provided in Table 5. As these results are in effect analysing changes in the changes of net wealth, there is now one less observation available in each of panels A and B, and panel C no longer applies.

Compared to levels of net wealth it is clear that there is more volatility over time in changes in net wealth. Typically there is only a 20% to 30% chance that a change in an individual’s net wealth (at the beginning of the period) say between waves 2 and 4 and 4 and 6, would be in the same quintile of a distribution of changes in net wealth between waves 4 and 6 and 6 and 8 (at the end of the period). Further, the probability that an individual could be at any point in the distribution at the end of the period, regardless of their starting position (rather than just moving to an adjacent quintile) is non-trivial. In fact, those starting the period in either quintiles 1 or 5 (of the distribution of changes in net wealth) have more chance of moving to the opposite end of the distribution at the end of say 2 years than anywhere else within it.

Finally, Table 6 presents transition probabilities for our last remaining potential output measure, savings rates. The patterns of transition probabilities in this case are very similar to those for changes in net wealth. There is possibly somewhat more volatility, particularly in the case of four year transitions. This is not surprising however as savings rates are changes in net wealth divided by gross income, introducing additional variation through incomes.[11]

A simple numerical example illustrates how such volatility in savings rates is possible and might, for example, come about through measurement errors in net wealth. Suppose an individual reported total assets of say $320,000 in Wave 4 and when surveyed in Wave 6 failed to recall an item of $50,000, when listing all his or her assets. Even if all other assets and liabilities were to remain unchanged, then the estimate of the saving rate between waves 4 and 6 (for an income of $35,000 over the period) would be -143% ((-50,000/35,000)*100). Now, further suppose that when surveyed again in wave 8 this individual recalled and correctly reported this asset. All else equal, the saving rate for this individual between waves 6 and 8 would increase to 143% (i.e. a change in savings rates between the two periods of 286%). Compound this with say a fall in income, for whatever reason, to $5,000. Then the savings rate between wave 6 and 8 becomes 1,000% (a change in savings rates between the two periods of 1,143%. Referring to the distributions of savings rates provided in the previous subsection and in particular in Table 3, even the first of these examples is more than sufficient to move this individual from the bottom quintile of the savings distribution (for savings between waves 4 and 6) to the top quintile in the following period.

3.3 Conclusion on the appropriate outcome measure#

For the remainder of the analysis in this paper a single outcome measure will be used in order to assess the impact of KiwiSaver. That measure will be changes in net wealth (as opposed to levels of net wealth). As indicated, this choice is informed by both the analytical virtues of each potential outcome measure and their potential to be affected by measurement error.

While observations on the level of net wealth display the most persistence over time the distributions of net wealth in each wave of SoFIE containing assets and liabilities are relatively dispersed. Further, of the three outcome measures it may be relatively more prone to endogeneity (is it KiwiSaver membership that causes higher net wealth or higher net wealth that causes KiwiSaver membership).

Savings rates are perhaps the most intuitively appealing outcome measure, with contributions to KiwiSaver typically set as a percentage of an individual's gross income. However, the distributions of savings rates are highly dispersed with a range of more than 30 times income typical. Further, individual savings rates are not stable over time (in fact they are negatively autocorrelated). To put this in context, a KiwiSaver member may typically have total contributions in any given year of say 6% of gross income. It seems highly unlikely that even an extremely well specified regression model would have much success in being able to isolate the effects of KiwiSaver membership on individual savings rates.[12]

Changes in net wealth are also somewhat volatile over time but have narrower distributions and may be less likely to suffer from endogeniety problems than the level of net wealth. When using changes in net wealth as our dependant variable in the regression analysis undertaken in Section 5 we are also able to address (at least partially) measurement error by including the initial level of net wealth as an explanatory variable. Further, we also include income (the missing component from the saving rate) as an explanatory variable in regressions. This was also the approach taken by (Le et al. 2010) when using SoFIE while attempting to understand the factors associated with net wealth accumulation.

4 Difference-in-Differences#

This section reports the results of an analysis based on the method of difference-in-differences (DiD). This technique has been used extensively internationally (see for example Card and Krueger, 1994) and in New Zealand (see for example MED 2011) for policy and programme evaluation. Section 4.1 describes the methodology and the results are summarised in Section 4.2.

4.1 The difference-in-differences method: brief outline[13]#

Given the choice of outcome measure explained in the previous section our objective is to estimate the direct impact of KiwiSaver membership on changes in net wealth. The intuition for using difference in differences in this context and its application is straightforward.

The need for estimation (using DiD or any other method) arises as it is not possible to directly measure the impact of KiwiSaver on an individual's pattern of net wealth accumulation. This is because while the accumulation of net wealth is observable for members of KiwiSaver after they have joined, the counterfactual is not. That is, the pattern of net wealth accumulation for those same individuals over the same time period that would have resulted had they not joined KiwiSaver is not observable.

To estimate KiwiSavers' impact, the change in net wealth that occurs before and after the introduction of KiwiSaver (in this case between waves 2 and 4, and 6 and 8 respectively) for those individuals who eventually become KiwiSaver members is first compared. Any difference cannot be solely ascribed to the effects of KiwiSaver membership, however, as other factors may also influence net wealth accumulation. The change in net wealth occurring before and after the introduction of KiwiSaver must also be compared for a control group (in this case those that do not join KiiwiSaver). By taking the difference in differences between these two groups an estimate of the effect of KiwiSaver on net wealth accumulation is provided which attempts to hold constant the effects of other influences on net wealth accumulation. Figure 1 illustrates with an example.

Figure 1 - Estimating the impact of KiwiSaver on net wealth accumulation by DiD

 

The vertical red line marks the introduction of KiwiSaver in July 2007 (falling in the fifth wave of SoFIE). The net wealth of two groups is considered both before and after this point; those individuals that will eventually join KiwiSaver (blue) and those that will not (green). Assignment to these groups is based on an individual's KiwiSaver membership status in the final wave of SoFIE.[14]

Before the introduction of KiwiSaver, Figure 1 shows both of these groups accumulating net wealth (between waves 2 and 4) at approximately the same rate. The vertical distance A is the change in net wealth for eventual KiwiSaver members and B is the change in net wealth for non-KiwiSaver members. In the case illustrated A = B for expositional convenience; however, in reality this may not be the case.

After the introduction of KiwiSaver, both KiwiSaver members and non-members experienced an increase in their respective rates of accumulation of net wealth. In other words, C > A and D > B. However, KiwiSaver members experienced a greater change in the speed of net wealth accumulation, i.e. (C - A) > (D - B).

This is all the information that is required for the DiD estimator of the effect of KiwiSaver on the accumulation of net wealth of those belonging to the scheme. The estimate is the difference of the differences, (C - A) - (D - B), and is positive in this example. Crucially, if in order to ascertain the effect of KiwiSaver on net wealth accumulation one simply compared say changes in net wealth accumulation of KiwiSaver members before and after the introduction of the scheme the effect of KiwiSaver would have been overestimated; that is (C - A) > (C - A) - (D - B). Similarly, though not in the case of our illustrative example as we set A = B for convenience, if one simply compared the net wealth accumulation of KiwiSaver members with non-members between waves 6 and 8, the effect of KiwiSaver on net wealth accumulation would likely be incorrect. That is (C - D) ≠ (C - A) - (D - B).

4.2 Results: difference-in-differences analysis#

Results of the DiD analysis described above are now presented. These are weighted using the longitudinal sample weights available in SoFIE at wave 8. Reported changes in net wealth between survey waves are the averages of the changes in net wealth of those individuals belonging to the group of interest.[15]

Table 7 shows that on average both KiwiSaver members and non-members experienced positive changes in net wealth (i.e. they accumulated net wealth) between consecutive asset and liability waves of SoFIE in all cases. However, the rate of growth in net wealth for both groups was much lower later in the period (between wave 6 and 8) than it was earlier in the period. This likely reflects, at least partially, the impact of the Global Financial Crisis (GFC).

What is particularly interesting in the current context, however, is that the reduction in the rate of asset accumulation was more pronounced amongst KiwiSaver members than non-members. In particular, the rate of asset accumulation slowed by approximately $32,500 for KiwiSaver members compared to approximately $16,900 for non-members. In other words KiwiSaver members suffered a greater decline in net wealth relative to non-members. Hence, the difference in differences estimator of the effect of KiwiSaver on the net wealth accumulation of its members is negative and considerable at approximately -$15,700.[16]

As membership of KiwiSaver was not randomly assigned, as individuals self select into the scheme, there is likely to be bias in this estimate, although apriori, the direction of any such bias is not clear. It is possible that differences in characteristics between KiwiSaver members and non-members may be important for net wealth accumulation. To the extent that these are not fully accounted for, they may be confounding results. As a first step toward correcting for the influence of other factors, a series of DiD estimates are made, each conditioning on one selected characteristic at a time. These included: age; gender; partnership status; home ownership status; highest qualification; income; and net wealth. Table 8 summarises these DiD estimates.

The right most column of Table 8 provides the DiD estimator for the effect of KiwiSaver membership on the accumulation of net wealth for each case. The two columns immediately preceding this show the differences between changes in net wealth that occurred between waves 2 and 4, and waves 6 and 8 for KiwiSaver members and non-members respectively.

The results are broadly consistent with the overall DiD estimate in Table 7. There is evidence that membership of the KiwiSaver scheme is associated with greater net wealth accumulation relative to those not in the scheme in only five of the 28 cases examined (in bold). The DiD estimator is positive although small for those individuals belonging to the second and fourth quintiles of the net wealth distribution or who have been partnered for only part of the analysis period. The effect of KiwiSaver membership on net wealth accumulation may be positive and larger for those individuals aged between 25 and 34 or who have a bachelors degree.

There is no evidence of a positive effect on net wealth accumulation from KiwiSaver membership for any subgroup when classified by gender, home ownership or income. The groups who may be most adversely affected by KiwiSaver membership include those individuals with a higher degree or who belong to the top quintile of the net wealth distribution.

To summarise, the results of the analysis in this section suggest KiwiSaver membership is associated with reduced net wealth accumulation. However, there are two clear limitations to the DiD analysis presented.

First, the classification of continuous variables such as income, age or net wealth, is restricted to discrete categories, with a consequent loss of information. Second, and more critical, is the fact that only one conditioning factor at a time is considered, when clearly the impact of KiwiSaver membership is likely conditioned by a whole series of factors acting together. To address these issues, the next section reports the results of a series of multivariate panel regressions.

5 Regression Analyses#

The underlying rationale for the panel regression analysis presented in this section is that changes in net wealth are likely to be influenced by a wide range of variables in addition to KiwiSaver membership. Some of these variables are observed and measured in SoFIE while others are not. It is only by properly correcting for the influence of these variables that we can expect to isolate the true effect of KiwiSaver membership on changes in net wealth.

However, as membership in KiwiSaver was not randomly assigned,[17] it is unlikely that the DiD analysis of the previous section would have properly accounted for the effects of other variables on changes in net wealth. Panel regression techniques allow us to control for both observed and unobserved heterogeneity across our sample. Section 5.1 describes the panel regression methods employed. The results are summarised in Section 5.2.

5.1  Methodology#

Given that changes in net wealth have been chosen as the most appropriate outcome measure by which to assess KiwiSavers performance, a panel of data with the dimensions i = 9,930 and t = 3 is available for regression analysis.[18] In other words, for each of 9,930 survey respondents there are three repeated observations on changes in net wealth that occur between waves 2 and 4, 4 and 6, and 6 and 8 respectively.

The variables available to us cannot possibly capture all influences on net wealth accumulation. In other words, there is likely to be unobserved heterogeneity across our sample. In the current context this could be, for example, due to different preferences for risk (perhaps related to health or longevity) or an individual’s underlying propensity to save. In both cases, these differences could be systematically related to an individual’s decision to join KiwiSaver.

Ordinary cross-sectional techniques cannot deal with this unobserved heterogeneity in the same way that techniques which use panel data can. In fact application of ordinary cross sectional techniques to panel data in the presence of unobserved heterogeneity can lead to incorrect standard errors and biased coefficient estimates.

Use of panel data permits models of the form

where the subscript refers to individual i as before, t refers to the time period,[19] α_i is an unobserved individual-specific effect that represents the permanent cross-individual heterogeneity, γ_t captures time-specific effects, and u_it is a time-varying idiosyncratic error.

The dependant variable

is the change in net wealth that occurs between consecutive waves of SoFIE that contain asset and liability information (for example, between waves 2 and 4) for each individual i. X_it is a vector containing a large set of explanatory variables and β is a vector of the corresponding parameter values. The primary variable of interest in this study is KS_it which is a categorical variable equal to one if an individual i is a member of KiwiSaver at time t and zero otherwise. The estimate of β_KS and its statistical significance therefore will provide an indication as to whether or not KiwiSaver has been associated with greater asset accumulation amongst its members.

Most explanatory variables are defined at the start of the period for which the change in net wealth is being estimated, i.e. at wave 6 when we are estimating the change in net wealth that occurred between waves 6 and 8. One exception is income, which is the sum of the income an individual earns over the period of asset accumulation that is included in regressions (in the current example, the sum of income earned in waves 7 and 8).

The other important exception is KiwiSaver membership. In some regressions we classify membership based on whether or not the respondent was a member of KiwiSaver at the end of the period over which changes in net wealth are measured. In others this classification is based on membership of KiwiSaver at the start of the period. These two approaches were adopted due to rapid growth in KiwiSaver membership over the period.

The first approach has the advantage of classifying many more respondents as KiwiSaver members (over twice as many). However, it means that individuals classified as KiwiSaver members may not actually have been members over the entire period for which changes in net wealth are measured. For example, when estimating changes in net wealth between wave 6 and 8, an individual classified as a KiwSaver member might have only joined the scheme in wave 7. If the true effect of KiwiSaver on members net wealth accumulation is positive this could result in the effect of KiwiSaver being underestimated. It is to allow for this possibility that we also adopt the alternative approach. Descriptions of all of the variables used in regressions are provided in Table 9.

There are two common panel techniques that will be used to estimate our model. The first is Random Effects (RE) and the second is Fixed Effects (FE). Each of these approaches has advantages and disadvantages in the current context.

Under the right conditions the RE approach is the most efficient, providing the greatest power to identify any effect from KiwiSaver membership for example. The RE approach also allows coefficient estimates to be obtained for variables that are constant over time, such as ethnicity. However, the approach requires that the unobserved individual-specific effects (α_i) are uncorrelated with the explanatory variables. This assumption is always difficult to sustain, including in the present case. For example, it might be that an individual’s inherent preference for saving or risk (unobservable traits) might be correlated with their health, income or level of education.

The FE approach, on the other hand, allows for arbitrary correlation between the (α_i) and explanatory variables. The FE estimator is less efficient than the RE estimator however, and it is not possible to obtain coefficients for variables that are constant over time, such as gender.[20]

5.2 Results#

Results of both RE and FE (unweighted and weighted) regressions are summarised in Table 10. In this set of regressions, KiwiSaver membership is defined at the end of each respective period over which changes in net wealth are examined. As previously discussed, this approach has the advantage of classifying the greatest number of respondents as KiwiSaver members.

The left hand column lists the explanatory variables. As is made clear in Table 9, a significant number of these are categorical as distinct from continuous variables. In most cases the categorical variables are not grouped and so are straight forward to interpret. For example, the coefficient estimate on the variable Owns Home provides an estimate of the average effect on net wealth accumulation of owning the home in which the respondent lives, compared to others who do not own their home.

There are four groups of categorical variables, however. These are: ethnicity; employment status; the time period; and health status. The coefficients reported for each category of a group are the differences between the particular category and an omitted category. The respective omitted categories for each group are: European; full-time employed; the first time period over which net wealth changes are measured; and excellent health. In the first column of results, the coefficient estimate on Good Health, for example, should be interpreted in the following way. On average those individuals reporting a good health status had on average a change in net wealth of $31,939 less than those individuals reporting excellent health (the omitted category).

When our model is estimated using RE the effect of KiwiSaver membership on changes in net wealth is estimated to be -$10,024 and is statistically significant. In other words, KiwiSaver membership is associated with a smaller accumulation of net wealth compared to those not in KiwiSaver after allowing for the effects of a large range of other conditioning variables.

Four variables are positively associated with changes in net wealth and are statistically significant. These are income, property ownership (both own home and investment property) and years of schooling. For example a $1 Increase in income is associated with 27 cents greater change in net wealth. The overall effect of age (considering both variables Age and Age Squared) is also positive. Conversely, those with a mortgage had a significantly smaller change in net wealth than those not holding a mortgage. All ethnic groups had significantly lower increases in net wealth than Europeans (the omitted category), and all those reporting any health status less than excellent, likewise had a lower increase in net wealth (relative to excellent health).

Table 10 also presents the results of our model when estimated using FE. As explained in the previous subsection, those variables whose values are not observed to vary over time (specifically gender, ethnicity and whether born in New Zealand) are eliminated by the estimation procedure. Results for two versions of FE estimation are presented. The first uses unweighted data, as does the RE estimation procedure. The second provides FE estimates that are weighted using the longitudinal sample weights available in SoFIE at wave 8.

Notes: Standard errors are given in parentheses. Significance levels are denoted either ** (1% level) or * (5% level).

The two versions are provided because the unweighted estimates give the most appropriate comparison with RE estimation as RE does not allow for the use of weights.[21] FE estimation does, however, provide for the use of weights so the second version allows us to consider whether omitting sample weights is likely to affect our conclusions about the effects of KiwiSaver membership in particular on net wealth accumulation.

The key finding is that there is no evidence of a significant effect on the change in net wealth from KiwiSaver membership. This applies in the case of both the weighted and unweighted FE estimates. As in the case of the RE model, the estimated effect remains negative, implying KiwiSaver membership is associated with a smaller increase in net wealth than for non-members. Not surprisingly, given the discussion of differences between FE and RE estimation in the previous subsection, the effect of KiwiSaver membership on net wealth accumulation is now no longer statistically significantly different from zero. A similar loss of statistical significance can be observed across a number of other variables in the model and in some cases coefficient estimates have changed signs. With such differences in parameter estimates between the RE and FE models FE estimates are likely to be the most robust.

One variable which is statistically significant across all three model specifications in Table 10 (and indeed in Table 11) is Net Wealth. Recall that this is the level of an individual's net wealth at the start of each period over which changes in net wealth (our dependant variable) are observed. The coefficient estimate on this variable in each case is negative which is exactly what one would predict if the type of classical measurement errors in net wealth postulated in section 3 is present. Inclusion of net wealth as an explanatory variable will have, in part at least, mitigated the potential effect of any measurement error on our results.[22]

Finally, Table 11 reports a similar set of results when our alternative KiwiSaver membership definition is adopted. In this set of regressions KiwiSaver membership is defined at the start of each respective period over which changes in net wealth are examined. As previously discussed, this approach has the advantage of classifying only those individuals who have been enrolled in KiwiSaver for the entire period over which changes in net wealth are examined. This minimises the chance that the true effect of KiwiSaver membership on net wealth accumulation is diluted or obscured through inclusion of individuals with very short membership tenure.

Results are very close to those reported in Table 10, indicating that different definitions of KiwiSaver membership do not change our key finding.[23] In particular, the length of membership in KiwiSaver does not appear to have any material impact on net wealth accumulation. Both the RE and FE (weighted and unweighted) estimates of our model yield negative coefficients on the KiwiSaver membership variable. These are marginally significant when RE is applied, but not significantly different from zero when either FE specification is applied.

Notes: Standard errors are given in parentheses. Significance levels are denoted either ** (1% level) or * (5% level).

To summarise, the results of the regression analysis in this section do not support the hypothesis that membership of the KiwiSaver scheme has been associated with greater net wealth accumulation amongst its members. That is, most regressions specifications yield coefficient estimates on KiwiSaver membership that are not statistically different from zero.

On the face of it this may not appear consistent with the first key finding of Law et al. (2011) that around one third of KiwiSaver contributions represented new saving. However, that finding was based on a self reported flow measure of saving whereas the current analysis relies on a stock measure of saving. Difference between the two results can, at least in part, be explained by returns on assets or inflated self reporting of additional saving flows. With respect to the second key finding of Law et al (2011), consistency with results of the current analysis is more obvious. That is, the authors found no association between KiwiSaver membership and expected retirement income outcomes (an important element of which must be net wealth at retirement).

6 Conclusions#

KiwiSaver was introduced in 2007, prompted by a view that household saving in general appeared to be low and declining. Further, that there may be some who would reach retirement with an accumulation of wealth insufficient to allow them to sustain their pre-retirement standard of living. The objective of this paper has been to analyse the extent to which membership of the KiwiSaver scheme has been associated with greater accumulations of net wealth amongst its members.

The paper utilised two linked sources of data, the Survey of Family, Income and Employment (SoFIE) and administrative data from the Inland Revenue Department (IRD). SoFIE was a longitudinal survey which included, as well as a wide range of socio-economic variables, details of individual assets and liabilities. Administrative data from IRD provided individual KiwiSaver membership information. The resulting linked data set covered the eight-year period to 2010. Asset and liability data were measured four times during this period and formed the basis for analysing changes in net wealth.

Careful consideration was given to the choice of an outcome measure upon which to base our assessment of KiwiSavers performance. Three options were considered: net wealth; changes in net wealth and savings rates. Changes in net wealth were chosen due to their analytical appeal and superior distributional properties (particularly in relation to savings rates).

Analysis was based on two approaches. The first used a difference in difference (DiD) technique. This technique compares outcomes (in this case changes in net wealth) before and after the introduction of a programme such as KiwiSaver, across two groups (those in the programme and those not). In this way those who are not members of the scheme formed a control group.

Results of this approach suggested the accumulation of net wealth by members of KiwiSaver was some $16,000 less than the comparable accumulation of non members. Further, in an attempt to hold some of the other factors likely to affect net wealth accumulation constant, the DiD analysis was repeated by age, gender, education, income, wealth, partner and home ownership status. There was a positive effect in only five of the 28 cases examined. In three of these cases the estimated effect was small. In one case, however, the estimated effect was relatively large, at $20,000 in favour of KiwiSaver members. All other cases indicated KiwiSaver members' accumulated less than non-members.

The DiD analysis only held one factor constant at a time however. To address this limitation, various fixed and random effect panel regression models were estimated in which changes in net wealth were related to many factors simultaneously. These included: KiwiSaver membership; income; net wealth; age; gender; partnership status; home and investment property ownership; ethnicity; if the respondent was born in New Zealand; education; labour force and health status. With four observations over time on assets and liabilities in SoFIE it was possible to measure three changes in net wealth for each of approximately 10,000 individuals. This provided nearly 30,000 observations for inclusion in each regression.

The effect of KiwiSaver membership on net wealth accumulation was estimated to be negative in all model specifications examined, although coefficient estimates were typically not statistically significant at conventional levels. While the findings of this study appear clear, caution is still warranted in their interpretation.

First there is evidence of significant measurement error in key variables. In particular, the distributions of changes in net wealth are wide, and there is little correlation in these changes over time for individuals. Attempts to control for this measurement error in regressions were made; however these may have only been partially successful. Second, the data is only available up until late 2010, meaning some KiwiSaver members would only have been enrolled for a relatively short time. However, regression estimates suggest that tenure in KiwiSaver has little effect on net wealth accumulation. Finally, the period over which we analyse changes in net wealth (2002 to 2010) is relatively short, potentially making it difficult to control for the effects of cyclical factors.

One should remember, however, that the results of this study are surprisingly consistent with the findings of the previous evaluation of KiwiSaver. In particular, Law et al. (2011) found no association between KiwiSaver membership and expected retirement income outcomes (an important element of which must be net wealth at retirement). This study, which used completely different techniques and data from the first evaluation, provides a second piece of evidence which suggests that KiwiSaver membership, at least until 2010, had not been associated with greater accumulation of net wealth, and hence improved retirement income outcomes.

References#

Card, David, Krueger, Alan B. (1994). "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania". American Economic Review 84(4):772-793

Carter, Kristie N., Matt Cronin, Tony Blakely, Michael Hayward and Ken Richardson (2010) “Cohort Profile: Survey of Families, Income and Employment (SoFIE) and Health Extension (SoFIE-health).” International Journal of Epidemiology 39:653-659

Gray, Alistair (2012) “Linking IRD KiwiSaver members to SNZ SoFIE respondents.” A report prepared for Statistics New Zealand

Law, David, Lisa Meehan and Grant Scobie (2011) “KiwiSaver: An Initial Evaluation of the Impact on Retirement Saving.” Wellington, The Treasury, Treasury Working Paper 11/04, http://www.treasury.govt.nz/publications/research-policy/wp/2011/11-04/

Law, David and Lisa Meehan (2013) “Housing Affordability in New Zealand: Evidence from Household Surveys.” Wellington, The Treasury, Treasury Working Paper 13/14, http://www.treasury.govt.nz/publications/research-policy/wp/2013/13-14/

Le, Trinh, Grant M. Scobie and John Gibson (2007) “Are kiwis saving enough for retirement? Preliminary evidence from SOFIE.” New Zealand Treasury, Paper prepared for the 8^thAnnual Super-Funds Summit, Wellington, 15-16 March 2007. http://www.treasury.govt.nz/publications/informationreleases/saving/sofie/retirementsofie.pdf

Le, Trinh, John Gibson and Steven Stillman (2010) “Household Wealth and Saving in New Zealand: Evidence from the Longitudinal Survey of Family, Income and Employment.” Working Paper 10-06, Motu Economic and Public Policy Research, Wellington http://www.motu.org.nz/publications/detail/wealth_and_saving

Le, Trinh, John Gibson and Steven Stillman (2012) “Wealth and Saving in New Zealand: Evidence from the Longitudinal Survey of Family, Income and Employment.” New Zealand Economic Papers 46(2):93-118

MED (2011) “Evaluation of the Impacts of Cross-Vote Government Assistance on Firm Performance: Impacts of Direct Financial Support for R&D.” Evaluation Report, Ministry of Economic Development, Wellington. http://www.med.govt.nz/about-us/publications/publications-by-topic/evaluation-of-government-programmes/evaluation-of-cross-vote-government-assistance-on-firm-performance-april-2011.pdf/at_download/file

Samoilenko, Anton and David Law (2014) “KiwiSaver: Comparing Survey and Administrative Data.” Working Paper 14/06, The Treasury, Wellington http://www.treasury.govt.nz/publications/research-policy/wp/2014/14-06/

Scobie, Grant M., John K. Gibson and Trinh V.T. Le (2005) Household Wealth in New Zealand. (Wellington: Institute of Policy Studies)

Scobie, Grant M. and Katherine Henderson (2009) “Saving Rates of New Zealanders: A Net Wealth Approach.” Wellington, The Treasury, Treasury Working Paper 09/04, http://www.treasury.govt.nz/publications/research-policy/wp/2009/09-04/twp09-04.pdf

Treasury (2007) “A synopsis of theory, evidence and recent Treasury analysis on saving', Treasury Report T2007/654, Wellington. http://www.treasury.govt.nz/publications/informationreleases/saving/synopsis/t2007-654v2.pdf

Wooldridge, Jeffrey M. (2006) Introductory Econometrics: A Modern Approach (Mason, OH: Thomson South-Western)