Policy perspectives paper

International Comparative Surveys of Regulatory Impact (PP 06/05)

Formats and related files


This paper was improved thanks to information and comments received from several people in Treasury and in the Ministry of Economic Development.  Particular thanks are due to those who reviewed an earlier draft of this paper, and to participants at a Treasury seminar.


The views, opinions, findings, and conclusions or recommendations expressed in this Policy Perspectives Paper are strictly those of the author(s). They do not necessarily reflect the views of the New Zealand Treasury.  The Treasury takes no responsibility for any errors or omissions in, or for the correctness of, the information contained in these Policy Perspectives Papers. The paper is presented not as policy, but to inform and stimulate wider debate.


Comparing regulations across countries is difficult because information about individual regulatory provisions is scarce and often qualitative in nature. In addition, much of the impact of regulation is not due to the actual law itself, but rather how it is enforced and implemented by the government.

However, quantitative comparisons across countries are valuable for several reasons. While theory can tell us much about the relationship between regulation and key macroeconomic variables, to avoid ambiguity we need to be able to estimate these relationships empirically. Also, cross-country comparisons enable policy makers to situate their country across the range of possible regulatory regimes and infer the economic consequences of different regulatory choices. In addition, the ability to benchmark current regulation and alternative policy options against the regulatory approaches of other countries has proven useful in encouraging countries to implement structural reforms that enhance economic performance.

If we want to use these surveys in debates, it is important that we understand them, and realise their limitations. This paper attempts to shed some light on certain key surveys by discussing their results and methods.

Methods drive results; the results can depend on where data come from and how they are built into the scoring and ranking. There are two types of surveys—“subjective” surveys based on the opinions of business executives, and “objective” surveys based on collecting information from various sources, often published ones. The surveys employ different weighting schemes to form overall indices of regulation from the underlying data. But there is usually a rough agreement between different survey results: countries occupy roughly the same position in different rankings when looking at particular impacts.

The surveys typically only measure the negative impacts of regulations and do not include any benefits associated with the regulatory impacts, nor how effective the regulations are at achieving their stated policy objectives. The main focus is on measuring the ease of doing business and whether regulation is hindering the competitiveness of the national economy.

New Zealand is, generally speaking, rated highly in these surveys. That is, businesses, and the wider economy, bears a low regulatory cost relative to other countries. This is especially so in areas like domestic markets, which are perceived to be contestable to foreign competitors and in which regulations have relatively little effect on prices. On the other hand, New Zealand is ranked lower with regard to foreign ownership restrictions and environmental regulations. In some cases—for example, the labour market—there appears to be some contradiction between what the surveys are telling us.

1.  Introduction#

Various organisations survey the impact of regulation in different countries and publish their results. These surveys are undertaken for different reasons. For instance, the Organisation for Economic Co-operation and Development’s (OECD’s) survey is undertaken on behalf of its member states to enable them to make better policy decisions. On the other hand, the World Competitiveness Yearbook is a product of the Institute for Management Development (IMD) supplied in response to perceived demand from business, academics and policy-makers. The mission of the “think-tank”, the Heritage Foundation, is to “formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defence”.[1] Their index can be seen as an instrument used in pursuit of their goals.[2]

These various indices have been used in the empirical literature to study the relationship between regulation and a range of key macroeconomic variables, such as productivity growth, innovation and entry and exit of firms. The findings of these studies generally confirm that regulatory settings do impact on economic growth.

Frequent references are made in political and business circles, and then picked up by the media, to New Zealand’s standing in international comparisons of regulatory cost and quality. These surveys are then used to justify or discredit calls for regulatory change. Responding to these references in a consistent manner with regard to policy advice is difficult due to the variations in the nature of the questions posed by such surveys, the quality of the data used, and differences in underlying regimes and other country-specific factors not captured by the surveys. Knowing the methods used and how the results were produced will help us to interpret their findings and inform policy decisions.

The aim of this paper is to use these international regulatory surveys to inform meaningful comparisons of the impacts of regulation in New Zealand and elsewhere.

Section 2 gives a basic inventory of the international surveys and a discussion of the results from each one, with emphasis on New Zealand’s standing relative to other countries. Section 3 details the methods used in the surveys, discussing data collection methods and data integrity, and any analytical methods used in the construction of the indices. Section 4 provides an overview of how these surveys have been used in the literature to investigate empirical relationships. Finally, section 5 contains some concluding remarks.


  • [1]Heritage Foundation n.d., About the Heritage Foundation, http://www.heritage.org/about/ (Accessed 10 April 2006).
  • [2]“We believe that ideas have consequences, but that those ideas must be promoted aggressively. So, we constantly try innovative ways to market our ideas” (ibid.)

2.  An Overview of the International Surveys#

The international surveys discussed in this paper are:

1. The Index of Economic Freedom by the Heritage Foundation and the Wall Street Journal.

2. The Doing Business Survey by the World Bank.

3. Economic Freedom of the World by the Fraser Institute.

4. The Global Competitiveness Report by the World Economic Forum (WEF).

5. The World Competitiveness Yearbook by the IMD.

6. The OECD Regulatory Indicators.

WEF’s Global Competitiveness Report#

The Global Competitiveness Report produced by the World Economic Forum (henceforth referred to as the WEF survey) samples 117 countries every year from 1979 until 2004. While the overall indicators presented in this survey are aimed at measuring the broader concept of competitiveness[3] rather than regulation specifically, various sub-indices are relevant to the impact of regulation on competition.

The survey has three overall indicators: the Growth Competitiveness Index (GCI), the Business Competitiveness Index (BCI), and the Global Competitiveness Index. The GCI is of no relevance to this discussion as it contains no information on regulatory policies. The BCI focuses on the underlying microeconomic factors which determine economies’ current sustainable levels of productivity and competitiveness. It specifically measures two areas that are critical to the microeconomic business environment: the sophistication of company operations and strategy, and the quality of the overarching national business environment in which they operate. It is this second aspect of the BCI which is of relevance to the discussion here, as it includes various aspects of the regulatory stance within an economy. For example, under the ‘Administrative Infrastructure’ sub-index (see Figure 1 below), the ‘extent of bureaucratic red tape’ is included as one component of this measure (New Zealand ranks 26th on this component).[4] Another example is the ‘Competition’ sub-index, which includes the component ‘prevalence of trade barriers’ on which New Zealand ranked2nd (that is, it had the second lowest trade barriers) in 2004. New Zealand ranked 16th for the overall ‘Quality of the National Business Environment’ in 2004.

The Global Competitiveness Index takes a more macroeconomic approach and is built around nine pillars, each of which is critical, according to the authors, to driving productivity and competitiveness in national economies. Two of these nine pillars contain sub-categories relating to regulation. The ‘Institutions’ pillar contains a sub-index labelled ‘The Burden of Government Regulation’, which is obtained through responses to the statement, “Complying with administrative requirements (permits, regulations, reporting) issued by the government in your country is 1=burdensome, 7=not burdensome”. New Zealand was ranked 39th on this question.

The other pillar which is important when comparing regulatory stances across countries is the ‘Market Efficiency’ pillar. According to the authors, the efficiency with which the various factor markets in an economy function is critical for its underlying productivity and competitiveness, as it ensures the proper allocation of economic factors to their best use. Three types of market efficiency are measured in this pillar: goods markets, labour markets, and financial markets.

It is important to remember that these “pillars” and what are included in them, have been chosen by the authors. To understand particular aspects of regulatory performance for policy purposes, it is necessary to look at the disaggregate results. For example, some components which are included in the ‘Market Efficiency’ pillar are more relevant than others when assessing regulation and its associated impact on competition. Therefore while New Zealand ranks 10th for Market Efficiency, overall, it may also be useful to know that New Zealand ranks 54th in terms of hiring and firing practices.[5] It might also say something about the impact of regulating hiring and firing: despite its impact on some business decisions, what impact do these regulations have on overall market efficiency?

Also, the WEF asks respondents questions on the environment, but only one question on this is included in the BCI and none are included in the Global Competitiveness Index. New Zealand ranked the 10th most stringent on environmental regulations, and 15th for the clarity and stability of environmental regulations. When asked if complying with environmental standards reduced competitiveness, New Zealand ranked 25th.

Respondents were also asked to select the five (out of a possible 14) most problematic factors for doing business in their country. In New Zealand, approximately 18 per cent of respondents listed the ”inadequate supply of infrastructure” as the most problematic factor, 17 per cent listed ”restrictive labour regulations” as the most problematic factor, and 10 per cent listed tax regulations.

A key message to be taken from this discussion is that it is less informative to study the overall or even the sub-indices of competition published by the WEF—and more informative to drill down to the various components that are likely to be more specific and therefore illuminating. Data at this level are easily accessible for most of the surveys considered here.[6] The advantages and disadvantages of how they have obtained these various components are discussed in the next section.

Figure 1 - Breakdown of the WEF's Business Competitiveness Index.
Figure 1 - Breakdown of the WEF's Business Competitiveness Index.


  • [3]Competitiveness is defined by the authors as “that collection of factors, policies and institutions which determine the level of productivity of a country”.
  • [4]The question asked of respondents for this component was “How much time does your firm’s senior management spend dealing/negotiating with government officials (as a percentage of work time)? Score=1 if 0%, 2 if 1-10%, 3 if 11-20%, 8 if 81-100%.
  • [5]The question asked for this component was “Hiring and firing of workers is 1=impeded by regulations, 7=flexibly determined by employers”.
  • [6]One exception is the Heritage Foundation survey, which does not provide ranks on the components which make up its various sub-factors.

IMD’s World Competitiveness Yearbook#

The World Competitiveness Yearbook, published yearly by the Institute for Management Development (IMD) since 1989, henceforth referred to as the IMD survey, also focuses on the economic competitiveness of nations but, similar to the WEF survey, it also includes many questions dealing with regulation.

The IMD survey samples 61 economies of which 8 are regions and the rest are countries.[7] The overall indicator is made up of 312 criteria that are grouped into 4 competitiveness factors, which are then averaged to obtain the overall World Competitiveness Scoreboard. The four factors are: ‘Economic Performance’ (which includes 77 criteria), ‘Government Efficiency’ (73 criteria), ‘Business Efficiency’ (69 criteria), and ‘Infrastructure’ (95 criteria). New Zealand fell from an overall position of 16th in 2005 to 22nd in 2006. Most of this fall is explained by weaker economic performance.

The ‘Government Efficiency’ factor, which is the factor of interest for this paper as it contains questions relating to regulation, is then split into 5 sub-factors, two of which contain criteria relating to regulation. Within the ‘Institutional Framework’ sub-factor, the survey question “The legal and regulatory framework in your economy 1=restricts the competitiveness of enterprises, 6=encourages the competitiveness of enterprises” is included as one criteria, on which New Zealand ranks 25th out of 61.

The other sub-factor which is of interest is labelled ‘Business legislation’, which is further decomposed into ‘Openness’, ‘Competition and Regulations’, and ‘Labour Regulations’. The various components of the Business Legislation sub-factor are set out in Figure 2 below, along with New Zealand’s rank for each criterion, while Table 1 sets out the best and worst ranked criteria for New Zealand, along with the survey questions for these criteria, in the ‘Openness’ and ‘Competition and Regulation’ components.

Table 1: Worst and Best Ranks for New Zealand in two IMD Components
  Criteria included in the ‘Openness’ component Criteria included in the ‘Competition and Regulation’ component
Worst Rank


“Investment incentives are attractive to foreign investors”


“Regulation intensity does not restrain the ability of companies to compete”

Best Rank


“Customs authorities facilitate the efficient transit of goods”


“Subsidies do not distort fair competition and economic development”

Figure 2 – Decomposition of the IMD’s Business Legislation sub-factor
Figure 2 – Decomposition of the IMD’s Business Legislation sub-factor.

Banking regulation is also included in the IMD survey under the ‘Business Efficiency’ factor. The question asked respondents to rank banking regulation from 1 to 6 depending on whether they felt banking regulation hinders competitiveness or does not hinder competitiveness in their economy. New Zealand ranked 13th on this criterion.

One question on Environmental regulations was also included under the ‘Infrastructure’ factor. The question asked respondents whether environmental laws and compliance hindered the competitiveness of businesses. New Zealand ranked last out of the 60 countries in the survey on this question.

This again highlights the need to consider the underlying components rather than the overall indicators. Since the components which make up each sub-factor are based on the subjective judgment of the authors, banking regulation in this survey has been included in another area entirely to the rest of the regulation criteria, as has the question on environmental regulation.


  • [7]The regions used in the sample are: Bavaria, Catalonia, Ile-de-France, Lombardy, Maharashtra, Scotland, the State of Sao Paulo, and Zhejiang.

Fraser Institute’s Economic Freedom of the World#

The Economic Freedom of the World survey by Gwartney and Lawson focuses on fewer topics (38 components in total), measuring the degree to which the policies and institutions of countries are supportive of economic freedom. This survey has the longest history of those reviewed in this study, starting in 1970. Reports were produced every five years until 2000, and then yearly. After 1997 components were added with the inclusion of survey data from the ”International Country Risk Guide” and the WEF survey, in order to incorporate legal structure and regulatory elements more fully. The overall indicator in 2003 (the latest year available) was decomposed into 5 areas (New Zealand’s rank is in brackets beside each area):

1. The size of government: expenditures, taxes, and enterprises (33rd).

2. Legal structure and protection of property rights (8th).

3. Access to sound money (28th).

4. Freedom to trade internationally (20th).

5. Regulation of Credit, Labour and Business (4th).

The components of areas 4 and 5 (those relevant for the comparison of regulatory practices) are set out in Figure 3 below. For the sub-factors in area 5, New Zealand ranked 2nd for credit market regulations, 38th for labour market regulations, and 4th for business regulations. For the ‘Regulatory Trade Barriers’ sub-factor in area 4, New Zealand ranked 5th in 2003. Both the regulatory components of area 4 and most of the data which makes up the sub-factors in area 5 are obtained from the WEF and IMD surveys.

Figure 3 - Decomposition of the Fraser Institute's Area 4 and Area 5
Figure 3 - Decomposition of the Fraser Institute's Area 4 and Area 5.
Figure 3 - Decomposition of the Fraser Institute's Area 4 and Area 5.

Heritage Foundation, Index of Economic Freedom#

The ‘Index of Economic Freedom’ published by The Heritage Foundation and the Wall Street Journal, defines economic freedom as “the absence of government coercion or constraint on the production, distribution or consumption of goods and services beyond the extent necessary for citizens to protect and maintain liberty”. The Heritage Foundation aims to construct a systematic, empirical measure of economic freedom according to this definition. To this end they have grouped 50 variables into the 10 categories listed below. The Index covers 161 countries and has been produced annually since 1995. In 2004, New Zealand ranked 9th equal with Australia on the overall Index of Economic Freedom, down from 5th in 2003. The 10 categories, along with New Zealand’s rank in 2004 in brackets beside each factor, are:

1. Trade policy (39th equal)

2. Fiscal burden of government (101st equal)

3. Government intervention in the economy (11th equal)

4. Monetary policy (1st equal)

5. Capital flows and foreign investment (13th equal)

6. Banking and finance (1st equal)

7. Wages and prices (2nd equal)

8. Property rights (1st equal)

9. Regulation (4th equal)

10. Informal market activity (1st equal)

Even though all of the 50 variables were studied, not all of them were given an individual score or specific mention in the text. Therefore while we can compare each of the 10 factors across countries and study what the factors comprise, we cannot compare the individual components of each factor across countries. For example, the first factor labelled ‘Trade policy’ comprises three variables: ‘Weighted average tariff rate’, ‘Non-tariff barriers’, and ‘Corruption in the customs service’. No data or scores are provided on these three variables. The factor is given a score of 1 if the weighted average tariff rate is less than or equal to 4%, while a score of 5 is given if the weighted average tariff rate is greater than 19%. If non-tariff barriers exist “in sufficient quantity”, or if there is ample evidence of corruption, a country’s score on this factor based solely on the weighted average tariff rate receives an additional point on the scale.

For the ‘Government intervention’ factor, the total score is derived as the average of the scores assigned to the level of government consumption (as a percentage of GDP), and to the level of the share of government revenues from state-owned enterprises and property.

For factor 5 (‘Capital flows and foreign investment’), countries were assigned a score of 1 if they met the following criteria:

  • open and impartial treatment of foreign investment;
  • accessible foreign investment code;
  • almost no restrictions on foreign investments except for fields related to national security; and
  • no restrictions on capital transactions.

A score of 5 means that the government actively seeks to prevent foreign investment and to prohibit all capital flows, and there is widespread corruption. New Zealand’s score on this factor is 1 point worse than the previous year, dropping from a score of 1 to 2.

Factor 6 (‘Banking and Finance’) assesses the openness of a country’s banking and financial system. The authors score this factor by determining specifically whether foreign banks and financial services firms are able to operate freely, how difficult it is to open domestic banks and other financial services firms, how heavily regulated the financial system is, how great the presence of state-owned banks is, whether the government influences the allocation of credit, and whether banks are free to provide customers with insurance and invest in securities (and vice versa). New Zealand ranked 1st equal on this factor in 2004.

The ‘Wages and prices’ factor is scored by the extent to which a government allows the market to set wages and prices. Specifically, this factor looks at which products have prices set by the government and whether the government has a minimum wage policy or otherwise influences wages.

For the ‘Regulation’ factor, the authors take into consideration the following variables:

  • Licensing requirements to operate a business
  • Ease of obtaining a business license
  • Corruption within the bureaucracy[8]
  • Labour regulations, such as established workweeks, paid vacations, and parental leave
  • Environmental, consumer safety, and worker health regulations
  • Regulations that impose a burden on business

A score of 1 is assigned to a country if:

  • all existing regulations are straightforward and applied uniformly to all businesses;
  • regulations are not much of a burden for business; and
  • corruption is nearly nonexistent.

To obtain a score of 5 on this factor, a country must: have a government which impedes the creation of new businesses; have corruption which is widespread; and have regulations applied randomly.

Thus one difference between this Index and those previously discussed is that the authors explicitly also look at how regulations are imposed. That is, are they imposed uniformly or haphazardly? The argument here is that even if two countries notionally have the same set of regulations, they may still impose different regulatory burdens in practice. If one of them applies its regulations evenly and transparently, it lowers the regulatory burden as businesses can more easily make long-term plans. If the other country applies the same regulations inconsistently, it raises the regulatory burden on businesses by creating an unpredictable business environment.

In the discussion of New Zealand’s score on the ‘Regulation’ factor (New Zealand had a score of 2.0 in 2004 to give it a rank of 4th equal), the authors suggest that the three-layered regulatory system involving national, regional, and local authorities created by the Resource Management Act, has resulted in an inconsistent system in which each of the country’s 83 different local authorities interprets the law in its own way and accusations of environmental violations can be filed on a broad-ranging basis. The authors go on to say that “efforts to fine-tune this act to make it less burdensome have met with only marginal results”.

Finally, the ‘Informal market’ factor takes into account the fact that informal markets are the direct result of some kind of government intervention in the market place. An informal market activity is one that the government has taxed heavily, regulated in a burdensome manner, or simply outlawed in the past. This factor captures effects of government interventions that are not fully measured elsewhere. New Zealand was ranked 1st equal on this factor in 2004.

The “cash economy” and the “black market” are responses to regulation that highlight the ambiguity of the idea of the impact of regulation. On the one hand, informal markets are welfare-enhancing for those directly involved: voluntary participants gain from trade. On the other hand, informal markets might not always be welfare-enhancing from society’s point of view. The existence of these markets in defiance of regulation generates enforcement costs, risk, externalities and whatever ills the regulations are designed to avert.


  • [8]The existence of excessive regulation can support corruption as confused and harassed business owners attempt to navigate the red tape.

World Bank Doing Business Database#

The World Bank “Doing Business” database provides “objective” measures of business regulations and their enforcement, aimed at quantifying the regulatory costs of businesses in 155 countries. That is, the questions asked of respondents relate to observable phenomenon, rather than asking for the opinions of respondents (as in the IMD and WEF Executive Opinion Surveys).[9] New Zealand was ranked 1st on their overall index in 2005. The summary index is the simple average of the following 10 sub-indexes (New Zealand’s ranking is given in brackets beside each category):

1. Starting a business (4th)

2. Dealing with licenses (2nd)

3. Hiring and firing (4th)

4. Registering property (1st)

5. Getting credit (7th)

6. Protecting investors (1st)

7. Paying taxes (16th)

8. Trading across borders (15th)

9. Enforcing contracts (4th)

10. Closing a business (21st)

The components of these sub-indices are set out in Figure 4 below.

Figure 4: Breakdown of the World Bank's "Doing Business" Factors
Figure 4: Breakdown of the World Bank's "Doing Business" Factors.

OECD cross-country product market regulation surveys#

The OECD conducted a survey of several types of regulation in OECD countries in 1998 (Nicoletti et al 2000), and updated it in 2003 (Conway et al 2005). The 1998 survey included 21 OECD countries while the updated 2003 report included all 30 OECD countries.

The OECD also produces “objective” measures of regulation for OECD countries. That is, the indicators are not based on opinion surveys and do not incorporate information about market outcomes. Rather, their survey focuses on assembling the details of governmental regulation, i.e. the questions ask if regulations exist in the country as opposed to getting respondents to rank regulations according to how much they affect the economy. This survey data makes up 90% of the information summarised by the indicators.[10]

Their summary indicators aim to measure the degree to which policies promote or inhibit competition, in areas of the product market where technological and market conditions make competition viable.[11] Their overall indicator is made up of 156 economy-wide (regulations that affect all or most sectors of the economy equally) or industry-specific regulatory provisions, which are averaged using weights determined by principal component analysis into the sub-factors and factors shown in Figure 5 below. The industry-specific information was gathered with particular emphasis on the analysis of service activities, as they have traditionally been highly regulated.

Figure 5 – Breakdown of the OECD Product Market Indicator
Organization Chart.

As well as looking at economic regulations, the OECD also included administrative regulations. Administrative burden is defined as the costs involved in obtaining, reading and understanding procedures and regulations, developing compliance strategies and meeting mandated reporting requirements, including data collection, processing and storage. New Zealand ranked 10th on the administrative regulation index in 2003 (down from 7th in 1998).

New Zealand ranked 7th out of the OECD countries for the overall product market regulation indicator (down from 5th in 1998), 6th for the ‘State control’ factor, 8th for the ‘Barriers to entrepreneurship’ factor, and 16th for the ‘Barriers to trade and investment’ factor (up from 19th equal in 1998). From 1998 to 2003, New Zealand experienced an increase in the scope of the public enterprise sector (dropping from a rank of 2nd with a score of 1.5, to 6th equal with a score of 2.3), and also an increase from a score of 2.0 to 2.6 in the ‘Direct control over business’ factor, dropping from 13th to 20th equal.

The dispersion of regulatory practice—that is, the degree to which regulations within a country vary in aims with respect to promoting competition—has declined between 1998 and 2003 for most OECD countries (measured by the variance of the 16 low-level indicators within a country) (Conway et al. 2005). A declining variance means that regulations within a country are becoming either more liberal, or more restrictive. So, regulatory practice is more consistently focused towards either restriction or liberalisation. A higher variance indicates inconsistency. So, a country with regulations that lowered barriers to entry across all industries would have a low variance, as would a country that raised barriers to entry across all industries. On the other hand, a country that lowered barriers here, but raised them there, would have a high variance.

According to the OECD survey, New Zealand has adopted more varying regulations between the survey periods, as have Turkey, Mexico, Poland, and Spain. For these countries, to the extent that complementarities exist between policy areas, there is a danger that the potential benefits of recent product market reforms may be reduced given ongoing restrictions in other areas (Conway et al 2005).

However, New Zealand has remained (since 1998) in the group of countries labelled “relatively liberal” by the OECD, which also includes the United Kingdom, Australia, the United States, Canada, Iceland, and Denmark. This grouping is based on confidence intervals constructed around the overall product market indicator, which is discussed in the next section.

The range of values taken by the indicators across countries is narrower than the initial 0 to 6 scale on which the individual regulatory provisions were ranked. Although this is partly due to the aggregation effects, the important policy implication is that relative to a worst case scenario, the subset of OECD countries considered (the 21 OECD countries included in the 1998 report) appears to be comfortably placed (according to Nicoletti et al 2000).

The 1998 report (the first survey conducted by the OECD) also included employment protection legislation, but the 2003 report does not include these. The OECD indicators also do not include data on the regulation of professional services, antitrust policy, and financial market regulation. They also ignore other important regulatory areas such as environmental and health and safety regulations.


  • [9]We realise that in collecting the “objective” measures some subjective judgement may be used. However, for the purpose of distinguishing between the different methods of collecting data on regulations between the various surveys we use the label “objective” where opinions of the respondents are not relied on. Where we refer to “hard” data below, we mean “objective” data obtained independently.
  • [10]The remaining 10% of information is based on hard data, such as data on tariff and non-tariff barriers to trade, compiled by the OECD.
  • [11]Restrictions to competition are defined either as barriers to access in markets that are inherently competitive or as government interferences with market mechanisms.


According to Schiantarelli (2005) the main stylised facts about product market regulation that can be identified by the data obtained from these surveys are:

  • Regulatory burdens vary widely across the world. In particular, regulation tends to be more intrusive in poor countries compared to the richer countries;
  • The dispersion of regulatory regimes is greater in developing countries compared to in developed countries;
  • There has been a general trend towards the relaxation of regulation concerning entry, accompanied by a decrease over time in tariff and non-tariff barriers to trade in manufacturing; and
  • There has been substantial deregulation of the service sectors (such as telecommunications, utilities, and transport) in many OECD countries.

Table 3 summarises some of the results found for New Zealand across the different surveys. More specifically, it shows the areas where the surveys appear to reach a consensus and areas where there are inconsistencies between the surveys.

Table 2: Summary of Results for New Zealand
Area Ranks Survey
  1998 Latest Available  
Price Controls


2nd equal

5th equal


1st equal

2nd equal



Heritage Foundation

Hiring and Firing Regulations






World Bank

Trade Barriers


2nd equal

22nd equal


3rd equal

39th equal



Heritage Foundation

Foreign Ownership Barriers

21st equal


7th equal

21st equal


13th equal



Heritage Foundation

Administrative Regulations







Banking and Finance Regulations

1st equal




1st equal




Heritage Foundation


Fraser Institute


Size of Government


56th equal


101st equal

Fraser Institute

Heritage Foundation

Environmental Regulations




10th (most stringent)



a: This ranking is based on data for 1997.

The different surveys tell us consistently that New Zealand has a low level of price controls. The OECD rank New Zealand 1st on their price controls index, while the IMD ranks New Zealand 5th.[12] The Heritage Foundation ranks New Zealand 2nd equal on its Wages and Prices indicator.

For hiring and firing regulations, however, there seems to be some disagreement across the surveys as to how New Zealand compares with other countries. The WEF ranks New Zealand 54th on their indicator of hiring and firing, while the World Bank ranks New Zealand 4th on this score.[13]

On outward-orientated policies, New Zealand does well compared with other countries on trade barriers. The WEF ranks New Zealand 2nd on its ‘prevalence of trade barriers’ index, while the OECD ranks New Zealand 3rd equal on its tariff score. In contrast, the World Bank has New Zealand at a slightly less flattering 15th on ‘trading across borders’ while the Heritage Foundation ranks New Zealand 39th on its ‘trade policy’ indicator.

Note that the World Bank’s ‘trading across borders’ indicator is quite a different measure to the others. It is a measure of all the documents and signatures required to export and import goods, as well as how long it takes to export and import goods. It is unclear why the Heritage Foundation measure should rank New Zealand so much lower than the OECD ranks New Zealand – the Heritage Foundation uses World Bank data on weighted average tariff rates (weighted by imports) while the OECD uses the simple average of most-favoured-nation tariffs.[14]

However, New Zealand does not do so well on ‘Foreign Ownership barriers’: the OECD ranks New Zealand 21st equal on this component.[15] The WEF ranks New Zealand slightly better at 11th place on ‘Foreign Ownership restrictions’. The Heritage Foundation ranks New Zealand 13th equal on their ‘Capital Flows and Foreign Investment’ factor, while the IMD ranks New Zealand 54th on the attractiveness of investment incentives to foreign investors. Therefore, there seems to be a broad consensus among these surveys that New Zealand’s outward-orientated regulation policies (with the exclusion of tariffs) are one area where New Zealand might improve relative to other countries. Of course, certain other countries (European countries, in particular) have cross-border arrangements with each other that lower the cost of investment. Given New Zealand is more geographically isolated than European Union countries, and more politically and economically independent, it would be difficult to imitate their investment rules. Also, preserving local ownership of certain land and resources is a recognised cultural objective. It might also be noted that these surveys precede the passage of the Overseas Investment Act 2005, and the establishment of the Overseas Investment Office.

Another area where inconsistencies arise is administrative regulations. In the WEF survey, New Zealand ranked 39th on their question related to administrative requirements. However, in the OECD survey, New Zealand ranked 10th on their index of administrative regulations.[16]

New Zealand scores well in the Heritage Foundation survey’s ‘Banking and Finance’ component, ranking 1st equal. This includes government ownership of financial institutions, restrictions on the ability of foreign banks to open branches, government influence over the allocation of credit, government regulations, and freedom to offer all types of financial services, securities, and insurance policies. The Fraser Institute also ranks New Zealand 2nd on its indicator of credit market regulations. New Zealand ranks 10th on the IMD’s index of banking regulation, where respondents were asked whether it hinders or does not hinder competitiveness.

Contradictory results, or idiosyncratic ones, may warrant further study. If New Zealand is the 39th worst at anything, the 54th best, it may be a matter of concern. Such results invite further research, perhaps to confirm or refute them, but certainly to understand them.

Another area in which the surveys suggest New Zealand may need to take a closer look at is the size of government. Gwartney and Lawson ranked New Zealand 33rd on this factor, while the Heritage Foundation ranked New Zealand very low (101st) for the fiscal burden of government (which is a measure of the tax burden and government spending). When respondents in the IMD survey were asked whether personal taxes discouraged people from working or seeking advancement, New Zealand ranked 40th, while a similar question on corporate taxes (hampering business activity) saw New Zealand ranked 43rd.

Another finding worthy of note comes from the IMD survey, which ranked New Zealand 61st out of 61 countries on whether Environmental Laws and compliance was seen to hinder the competitiveness of businesses. The World Economic Forum survey also suggests that New Zealand has the 10th most stringent environmental regulations out of 117 countries. (The Heritage Foundation bundles environmental, consumer safety and worker health regulations without reporting separate scores in each category.)

One thing to note about all of these surveys is that they generally do not try and measure the positive impacts of regulation, or how effective they are at achieving their stated policy goals. The main focus is on the ease of doing business, as it affects starting a business, business operations or the ease of trade. There are no assessments, for example, on how consumer welfare is affected by consumer protection regulations, or on how distributional outcomes are being met through service obligations imposed on monopoly industries. One reason for this omission is that those who do the surveys intend to treat the political and social goals of regulation as a given, and ask instead “How much does it cost for a country to pursue its goals?” But this makes comparing countries difficult: a “poor” score on some index might beg the question of whether or not the intervention is achieving some greater good. On the other hand, where policy goals and independent conditions are similar, ranks and scores provide information about how well the regulation is imposed.

An unequivocal response to most of these indices is impossible, because they tend not to provide information on the positive impact of regulation. A “low” rank is not necessarily bad, and a high rank is not necessarily good. It all depends on a country’s policy goals and how effective its regulations are at meeting those goals. Results between countries and over time can be meaningful information, but only when placed in a policy context.


  • [12]New Zealand slipped from 1st in 2005 to 5th in 2006 in the IMD survey.
  • [13]This difference is not due to any difference in the samples of the two surveys – only 5 countries ranked above New Zealand in the WEF survey are not included in the World Bank survey. One difference is that the World Bank measure also includes the costs of hiring and firing, as well as rigidities in hours worked. Thus these extra components, as well as the differences in how the data are collected (subjective judgement of respondents versus “objective” data – discussed in the next section), may account for the differences in rankings.
  • [14]The World Bank also produces a Trade Restrictiveness Index (TRI) in their annual Global Monitoring Report, which includes both tariff and non-tariff barriers. The TRI focuses on the extent to which trade policies at home affect domestic welfare. They aggregate over tariff and non-tariff barriers by using the Ad-Valorem Equivalent approach to convert non-tariff barriers into a monetary value. This involves first estimating the quantity impact of non-tariff barriers on imports using Leamer's (1990) comparative advantage approach - which involves predicting what imports would have been in the absence of trade barriers given the country's factor endowments (measured by agricultural land over GDP, capital over GDP, and labour over GDP, as well as GDP to capture economic size). They then convert this quantity impact into a price equivalent by moving along the import demand curve using estimated import demand elasticities. To solve the problem of aggregating across goods with very different economic importance, they define the TRI as the weighted sum of squared protection levels, where the weights are given by the slope of import demand functions. New Zealand ranked 52nd out of 91 countries in 2004 on this indicator. See Kee, Nicita and Olarreaga (2006) for the methodology of this index.
  • [15]Both the telecommunications industry and the airline industry are each given a weight of ¼ in the construction of this component in the OECD survey, while another 1/3 is made up of restrictions on foreign investors in publicly-controlled firms.
  • [16]This difference seems to be almost entirely due to the different survey methods employed. The WEF asks respondents their opinion of the burden of complying with administrative requirements, whereas the OECD collects information from government officials on the licenses and permits system, the communication and simplification of rules and procedures, administrative burdens for corporations and sole proprietor firms, and sector-specific administrative burdens. This does not appear to be a pattern with New Zealand however. For example, the OECD (who use “objective” data) rank New Zealand worse than the WEF on Foreign Ownership Barriers.

3.  Survey Methods#

Table 2 summarises the survey methods used for the six surveys discussed in this paper.

Table 3: Survey Methods
Survey Data Type Sample Size
(number of
respondents in New Zealand)
Who was sampled
WEF “Subjective” (or “reported”) survey data and hard data 47 Business Executives
IMD “Subjective” survey data and hard data Approximately 67 Business Executives
Gwartney and Lawson Hard data and WEF and IMD survey data    
Heritage Foundation Authors’ rankings based on published statistics Six authors listed Index authors
World Bank “Objective” survey data Unclear Government officials, lawyers, business consultants.
OECD “Objective” survey data and hard data Unclear Civil servants in national administrations


The WEF uses a combination of hard data, and responses to their Executive Opinion Survey. Three components of the ‘Market Efficiency’ pillar in the ‘Global Competitiveness Index’ are examples of the hard data used: the number of procedures and time required to start a business are both drawn from the World Bank database discussed below, and imports (as a percentage of GDP) are also used under the ‘Competition’ sub-index. The responses to the Executive Opinion Survey range from 1 to 7, while each hard data variable was converted to the 1-7 scale using the following formula:

This implicitly assumes that the highest and lowest sample values are the maximum and minimum values which this variable can take in the population. That is, they assumed that the country sample represented the entire population of reference.

The Executive Opinion Survey was designed to capture the expert opinions of business leaders on the most important issues affecting their working environment. The business executives were asked to respond in regard to their own business experience, and to rank their perception on a scale of 1 to 7, with 1 representing the worst-case scenario.

The WEF’s partner institutes—typically consisting of leading research or academic institutes, business organisations, national competitiveness councils, or any other recognised professional entity committed to improving the competitiveness condition of their economy—lead the survey at the national level in accordance with the guidelines set by the WEF. The partner institutes were asked to select a sample of business executives to take part. Specifically, all respondents had to be a CEO or equivalent, unless the business had greater than 500 employees, in which case the participant could hold any of the company’s top five management positions. The companies sampled had to represent the main sectors of the economy, in proportion to its percentage share of total GDP. A master list of companies (compiled from a list of companies registered for a telephone line or companies registered with the national statistical office or tax authorities) was grouped by economic sector, geographical region and size, before sampling from each subgroup. However, the selection of firms to be interviewed within each group could not be completely random, in part because there was a preference for interviewing business executives who have an international perspective, which often meant selecting large companies.

The New Zealand sample consisted of 47 participants. Table 1 below, Panel A sets out the distribution of respondents by firm size, while Panel B provides a breakdown of the distribution of all firms in New Zealand by size (number of employees). It is obvious from a comparison of Panel A with Panel B that the sample drawn for the Executive Opinion Survey is not representative of the actual population of firms in New Zealand. This may be a direct consequence of the preference stated above for the sample to consist of business executives who have an international perspective. So while there may be advantages to surveying business executives with an international perspective (discussed below), there are also disadvantages, as the resulting sample may not be representative of the population of interest.

Table 4: Comparing the distribution of firms sampled in the WEF survey to the population of firms in New Zealand

Panel A: Distribution of respondents to the WEF Executive Opinion Survey by firm size (number of employees)
Size: Less than 101 101-500 501-5,000 5,001-20,000 Greater than 20,000
% of respondents: 11% 45% 36% 6% 2%
Panel B: Distribution of firms in New Zealand by firm size (number of employees) – average 1997-2000, from Mills and Timmins (2004), Table 6.
Size: 0 1-4 5-9 10-19 20+
% of firms: 56.3% 29.9% 5.5% 4.4% 3.9%


Like the WEF survey, the IMD data are made up of hard data, taken from international and regional organisations and private industries, and survey data drawn from their Executive Opinion Survey. The survey of 113 questions was sent to top and middle management in each country or region who explicitly deal with international business situations. The survey questions were designed to measure competitiveness as it is perceived by the business executives. Four thousand executives responded in 2005 for an average of approximately 67 responses per economy.[17]

The survey was sent to executives who represent a cross-section of the business community in each country or region. The distribution reflects a breakdown of industry by sectors: primary, manufacturing, and services, and they state that, in order to be statistically representative, they select a sample size proportional to the GDP of each economy. This could mean that they end up with a very small sample for New Zealand compared to other countries (they do not report sample sizes for each country). If they do want to be statistically representative, the proportion of business executives (out of the entire population of business executives in the country) actually sampled should be higher the smaller the population of business executives, to maintain the same level of confidence in the results. The respondents were asked to evaluate the present and expected competitiveness conditions of the economy in which they work and have resided during the past year, drawing from the wealth of their international experience, and “thereby ensuring that the evaluations portray an in-depth knowledge of their particular environment”.

For each survey question, the IMD calculates the average value for each economy, then the data are converted from a 1-6 scale to a 0-10 scale. Finally, the survey responses are transformed into their standardised values (STD values), from which the rankings are calculated. The STD values are calculated using the following formula:

These STD values enable criteria which were originally scaled differently to be used in the computation of the overall, factor, and sub-factor indicators. The sub-factor rankings are determined by calculating the weighted average of the criteria STD values that make up the sub-factor. All of the hard data have a weight of 1, whereas the survey data are weighted so that the survey accounts for one-third in the determination of the overall ranking (reflecting the two-thirds to one-third split of hard data to survey data in the criteria). For 2005, each survey criterion has a weight of 0.5. When data are unavailable for particular economies, the missing values are replaced by a STD value equal to zero.

The partner institutes of the IMD performed a similar task to those of the WEF, supplying data from national (or regional) sources and helping distribute the survey questionnaire. The IMD states that “A long-established collaboration with Partner Institutes also helps ensure that the data is reliable, accurate, and as up-to-date as possible”. The partner institute in New Zealand is the New Zealand Institute of Management Inc., Wellington.

Gwartney and Lawson#

As mentioned in the previous section, Gwartney and Lawson use both IMD and WEF survey data as components in the areas of interest for this paper (area 4: ‘Freedom to trade internationally’, and area 5: ‘Regulation of Credit, Labour, and Business’). They also use hard data from various sources including the IMF, the World Bank, and the OECD.[18] Each component (both from survey or hard data) was placed on a scale from 0 to 10 that reflected the distribution of the underlying data. This was achieved using the following formula for components comprised of hard data:

where Vi represents a variable of interest for country i – for example the mean tariff rate for country i. This is similar to the WEF computations in that they are assuming that the country sample represents the entire population of interest.

However, for four components included in area 5 which were constructed using hard data, alternative scaling methods were used. For example, for the component ‘Ownership of banks – percentage of deposits held in privately owned banks’, a country was given a score of 10 if privately held deposits totalled between 95% and 100%. When private deposits constituted between 75% and 95% of the total, a rating of 8 was assigned, and so on. Thus, for some of the components, the authors have chosen the categories using subjective opinion, and assigned scores to these categories. For the survey data, the ratings given by the IMD and WEF were rescaled to range from 0 to 10. The component ratings within each area were then averaged (using equal weights) to derive ratings for each of the 5 areas.

Heritage Foundation, Index of economic freedom#

The Heritage Foundation and Wall Street Journal also assign categories to score each factor. That is, some subjective assessment was used. For example, if a country’s banking system received a score of 3, this means that its banking and financial system displayed most of the characteristics for level 3, according to the author’s assessment of the data. All of the subjective assessments, however, are justified by reference to hard data. The main sources of data include the Economist Intelligence Unit Country Commerce, Country Profile and Country Report, official government publications of each country, the US Department of Commerce Country Commercial Guide, the OECD, the IMF, and the World Bank.


While using business evaluations of the regulatory system has the advantage of being able to capture enforcement issues, there are some obvious drawbacks to doing so. For example, since the respondents were asked to rank only their own country in the WEF and IMD surveys, the scale used by business people in particular countries might be more sensitive to government intervention. As a result, they may rate their nation more severely than others. Unfortunately, there is no way to test this. The authors of the WEF survey state that they have made great efforts to reduce this “perception bias” by phrasing the survey questions in such a way that asks respondents to compare their own environment to a world standard, rather than thinking in national terms. Also, Pryor (2002) states that sampling business people with international experience may mitigate this problem somewhat, as they might have some implicit basis of comparison.

The WEF also tested whether the judgements of respondents were affected by a country’s general economic climate, the argument being that in times of recession, business executives will tend to be more pessimistic and may exaggerate the extent of regulations. However, the WEF found no significant relation between either a country’s real growth rate, or the change in the growth rate, to changes in the average level of responses across the survey questions. Thus a country’s general economic climate did not appear to affect the responses.

For each survey question, the WEF also compared the standard deviation of answers within a country to the standard deviation of answers across all countries. In those countries with high within-country variance of responses on many questions it is hard to interpret the country averages, independently of the possible reasons for the variances. As expected, the within-country consensus was higher for cross-cutting business environment indicators and lower for measures where there would be variation within the country across companies and clusters. Thus the WEF concluded that “the country averages, then, capture meaningful differences across countries in competitive circumstances, while limiting idiosyncratic biases that would result if there were only a handful of responses per country”. However, studying the country averages may also lose some valuable information. For example, if respondents from one sector are consistently rating their country worse on some score than respondents in another sector, we might find this information useful from a policy perspective. All of the surveys considered here only report the country averages. They also carried out a “data consistency test”. This test consisted of determining, for each country, whether at least 40 questions out of 57 had lower within-country variance than the cross-country variance. One hundred and ten countries passed this test.

Also, for both the WEF and IMD surveys the samples are small and their degree of randomness can be questioned (Pryor 2002). Plus, the respondents were limited to high level business executives, who presumably have experience in dealing with governmental regulations. Given their attitudes toward governmental regulations, however, they may potentially exaggerate or downplay the extent of these regulations.

Some indication of the quality of the data can be gained by comparing answers to similar questions asked by both the WEF and the IMD. In almost all cases the correlation coefficients were high and significant. Even when the wording of the question differed there was sufficient agreement between the two sources for Pryor (2002) to conclude that “I felt the data reflected the same underlying reality”.


  • [17]They do not specify how many responses they obtained from each country.
  • [18]Areas 1 and 3 (The size of government: expenditures, taxes, and enterprises, and Access to sound money) are made up entirely of hard data from various sources, while area 2 (Legal structure and protection of property rights) uses data from the WEF survey and the International Country Risk Guide.

World Bank#

The World Bank data are based entirely on published information, informed by research of laws and regulations, with input and verification from more than 3,000 local government officials, lawyers, business consultants, and other professionals who routinely administer or advise on legal and regulatory requirements. The data are standardised (to enable comparisons across countries) by asking the respondents to reply with regard to very specific case studies. However, this could be at the expense of losing information. For example, when asked about the procedures, time and costs involved in starting a business, respondents were asked to consider the situation for a business that:

  • is 100% domestically owned;
  • operates in the country’s most populous city;
  • has 5 owners;
  • has start-up capital of 10 times income per capita;
  • performs general industrial or commercial activities;
  • does not perform foreign trade activities;
  • does not qualify for investment incentives;
  • has up to 50 employees one month after the commencement of operations (all of them nationals);
  • has a turnover at least 100 times income per capita; and
  • has a company deed 10 pages long.

The logic behind these criteria is simple: the World Bank wants to compare like with like. However, nations might have “typical” businesses that differ in many ways from the World Bank’s hypothetical business, and the regulatory structure in those nations could be designed with regard to typical native business models. By being so specific about the hypothetical business, the World Bank is improving the integrity of its index from one perspective, but loses information when it fails to account for the diversity of business models within and between nations (and, therefore, the experience of starting typical businesses).


The OECD also use objective data based on a survey of civil servants in national administrations that have knowledge and responsibilities related to the relevant policy areas.

Their overall indicator is aimed at measuring the degree to which policies promote or inhibit competition, in areas of the product market where technological and market conditions make competition viable. Thus, the OECD is making 2 assumptions when constructing their index:

1. that they can “measure” or “know” which policies promote or inhibit competition, and

2. that they know which areas of the market can be competitive.

The OECD indicators only account for formal government regulation (as do the World Bank indicators). This makes them comparable across countries but also implies some limitations. ‘Informal’ regulatory practices, such as administrative guidance or self-disciplinary measures of professional associations, are captured to a very limited extent, and the way in which regulations are applied by enforcement authorities, which the OECD admit can have a considerable impact on competition, is also only reflected in a relatively minor way in the indicators.

As with the Heritage Foundation and Gwartney and Lawson indices, the OECD convert cardinal statistics and qualitative information into discrete, ordinal categories. The often qualitative nature of the data and the need to aggregate different regulatory provisions involved a certain amount of arbitrary judgement in the construction of the detailed indicators. That is, the scoring procedure often involved some subjective judgement. While any mistaken judgements may have had an influence on the ranking of countries in the individual regulatory provisions, the OECD argues that as long as they are not systematic it is unlikely that they can affect significantly the values of the summary and overall indicators of regulation, due to the large number of provisions (156 of them) included in the analysis.

Also, as with the WEF and Gwartney and Lawson calculations, the OECD assumed that the country sample represented the entire population of reference: ie, the least restrictive and the most restrictive provision in the country sample were assigned the values 0 and 6 respectively.

The OECD indicators, while providing detail at the industry level, may not fully cover the range of regulations within a country as they only cover regulations in some industries for some of the indicators. For example, the question in which respondents were asked about restrictions on businesses to enter markets was only asked in relation to the network sectors. The OECD state that the existing indicators could be expanded to incorporate a range of additional economic information. For example, the sectoral coverage of the indicators could be increased by expanding the number of sectors over which indicators such as ‘the scope of public enterprises’, ‘legal barriers to entry’, and ‘barriers to foreign ownership’ are calculated. Currently, only the retail distribution, transportation, and telecommunications sectors are included.

Also, the OECD discuss how new data could be used to refine some of the low-level indicators. For example, data on the number of hours that retail outlets are typically able to trade could be useful in determining the extent to which retail trade is regulated. Another example involves incorporating data on producer support for agriculture into the indicator of barriers to trade.

Nicoletti and Pryor (2005) show evidence that the OECD economy-wide indicators and the Fraser Institute indicators are highly correlated, despite the different methods used in their construction.

Aggregation Methods#

In constructing indicators of regulation, the aggregation procedure must involve weighting the individual components or areas of regulation to form the overall indicator. In principle, the weights should reflect the relative impact of the different policies on the policy objective in question.

The OECD have tried various weighting schemes in the construction of their summary indicators. They concluded that there appears to be only some differences in the ranking of countries (according to the stringency of employment protection legislation) as one moves from a uniform weighting scheme, to subjective weighting and, finally, to a statistically-defined weighting scheme. However, these small differences may have some impact in analytical studies of the impact of employment protection legislation on economic performance, to the extent that the rank position (as opposed to the actual summary values) of countries are used.

The OECD also used a random weighting technique to construct confidence intervals around their indices based on principal components analysis. That is, they used 10,000 sets of randomly generated weights to construct 10,000 overall indicators for each country.

The principal components technique used by the OECD reveals, within each regulatory domain, families of detailed indicators which are most associated with different underlying (unobserved) factors. That is, the underlying detailed indicators are grouped into factors based on the data, rather than relying on judgment as to what components are included in each factor. Principal component analysis is appealing, therefore, since the aggregation of the detailed indicators is data-based and ensures that the resulting summary indicators account for a large part of the cross-country variance of the detailed indicators. It also means that the aggregation is independent of prior views on the relative importance of each regulatory provision.

However, principal component analysis is sensitive to any modifications in the basic data (ie, the weights may change with any change in the data). The results are also likely to be sensitive to the presence of outliers, which may introduce a spurious variability in the data. Data limitations may also imply difficulties in the statistical identification and the economic interpretation of the unobserved factors. The OECD checked the robustness of the results by excluding a few outlier countries.

The principal component analysis of the four detailed indicators describing outward policies ran into some problems. Due to the limited country coverage of some of the basic data, the focus had to be restricted on a few dimensions of outward-oriented regulations, not necessarily fully representative of the countries trade and investment policies. In addition, the cross-country variance of the detailed indicators was much smaller than in the other domain of regulation. As a result, little correlation was found among the indicators.

The same set of weights which were found using principal component analysis in the OECD 1998 report were also used for their 2003 indictors. According to the OECD, maintaining consistent weights in the different estimation periods is an important pre-requisite for making meaningful comparisons of indicator values in different years. The use of the same weights implies that any changes in the indicators over time are entirely due to changes in regulations and not in the aggregation process.

Gwartney and Lawson have also experimented with several weighting methods ranging from the subjective views of “experts” to principal component analysis. They found that in most cases, the choice of weighting method exerts little impact on the rating and ranking of countries. As a result, they decided that it is best to keep the procedure simple and transparent by use of the simple average procedure. By doing this, they do not mean to imply that all components and areas of economic freedom are equally important.

The Heritage Foundation also weights all ten factors equally in the construction of its overall Economic Freedom Index, as they argue, in contrast to Gwartney and Lawson, that all of the factors are of equal importance. They state that this is the common-sense approach, which is consistent with the purpose of the Index: to reflect the economic environment in every country. The index is not designed to measure how much each factor adds to economic growth. In the 2004 Index, Richard Roll concluded that equally weighting the factors reveals as true a picture of economic freedom as the best weighting scheme that statistics can devise (weights based on principal components analysis).

However, while the correlation between the 2 indices (one based on equal weighting and the other based on principal components weighting) was very high (R2 = 0.98—a good argument, according to Roll, in favour of preserving equal weights as they have the advantage of simplicity), the 2 indices did generate somewhat different country rankings. For example, Guatemala moved down 32 positions when using the principal components method rather than equal weights, and quite a few countries changed in rank by at least 10 positions.

As one of the main recommendations of this paper is to look at individual components rather than the overall indicators, the weighting method used does not become an issue (except to the extent that people ignore this recommendation and focus on the consolidated index). Another recommendation may be to compare groups of countries rather than focusing on individual rankings. The OECD identified two broad country groupings (using their constructed confidence intervals) with clearly distinct regulatory regimes—a “relatively liberal” group (which includes New Zealand) and a “relatively restrictive” group. The rest of the OECD countries (”middle-of-the-road” countries) were not statistically different from these two groups.

Kaufmann et al (1999) also warn against concentrating solely on individual rankings compared to other countries. The rather large size of the confidence intervals they constructed makes it clear that small differences in point estimates of governance across countries are not likely to be statistically significant.[19] As a result, they suggest that users of this data should focus on the range of possible governance for each country as summarised in the confidence intervals. For two countries at opposite ends of the scale of governance, whose confidence intervals do not overlap, it is reasonable to conclude that there are in fact significant differences in governance between these two countries. For pairs of countries that are closer together and whose confidence intervals overlap, “one should be much more circumspect about the significance of estimated differences in governance between two such countries”.


  • [19]Their governance indicator is made up of 6 factors: Voice and Accountability, Political Instability and Violence, Government Effectiveness, Regulatory Burden, Rule of law, and Graft. It is constructed using data from a variety of sources including the WEF, IMD and Heritage Foundation surveys.

4.Empirical Relationships#

The indices constructed by the surveys discussed above have been used in the empirical literature to study the relationship between regulation and a range of outcome variables. Appendix Table 1 provides a summary of these studies.

Schianterelli (2005) has reviewed the literature devoted to cross-country evidence on product market regulation and economic performance. He states that, in many cases, researchers have entered measures of regulation directly as an explanatory variable in equations of factor demand, productivity, or innovation. In other cases, the effect of regulation is mediated through its effect on an intermediate variable, such as the mark-up or firms’ entry, exit, and turnover rates. This approach implicitly assumes that regulation only affects the end variable (productivity etc) through its impact on these intermediate variables.

Considering all of the empirical contributions on the relationship between regulation and firm dynamics, Schianterelli concludes that regulatory barriers in the product market have a negative effect on firms’ entry or turnover and are likely to slow the process of resource reallocation.

Those studies which take the more direct estimation approach include Alesina et al (2005). They use time-varying sector-country specific indices of regulation compiled by the OECD (Nicoletti et al 2000) to assess the effect of regulation on capital accumulation, by introducing the regulation indicators directly into an investment equation. Their overall results suggest that a reduction in regulation, particularly if it affects barriers to entry, has a significant and sizable positive effect on investment. They also find evidence that the marginal effect of deregulation on investment is greater when the policy reform is large and when changes occur starting from already low levels of regulation.

This direct evidence, and the indirect evidence provided by Griffith and Harrison (2004) suggests that deregulation has a positive effect on investment.

The cross-country studies provide conflicting evidence on the effect of lower regulation on direct input measures of innovative activity. Both Griffith and Harrison (2004) and Cincera and Galgau (2005) find that deregulation has a negative effect on R&D intensity, while Bassanini and Ernst (2002) and the OECD (2003) find evidence that deregulation has a positive effect on innovation activity. Also, Nicoletti et al (2001) find that strict employment protection legislation (EPL) has a negative effect on innovation activity but the OECD (2003) find no evidence to support this.

There are several papers which address the relationship between product market regulation and productivity or output growth. Most of the studies that include measures of regulation directly in the regression tend to find a negative effect of tighter regulation on total factor productivity or per capita output growth. However, studies that use the mark-up as the channel of transmission find that decreases in the mark-up associated with deregulation are associated with lower productivity growth (or level). On the other hand, relying on turnover or entry as the variable through which the effect of deregulation is transmitted suggests a positive effect of lowering regulatory burdens on overall productivity growth.

Gust and Marquez (2002) find that burdensome regulatory environments and in particular regulations affecting labour market practices have impeded the adoption of information technologies and slowed productivity growth. Their two-step approach uses the EPL index constructed by the OECD to measure labour market regulations, and data on regulatory burdens constructed from the World Competitiveness Report 1993 (by the IMD and WEF) and Global Competitiveness Report (published by the WEF) surveys.

Nicoletti and Scarpetta (2003) use four sets of regulatory indicators to estimate the relationship between regulatory reform and multifactor productivity growth. Their four sets of regulatory indicators are: economy-wide regulation (from the OECD survey 1998), industry-level regulation, regulatory reform (i.e. time-varying indicators of regulation), and privatisation. They find that economy-wide product market regulations that curb competition and private governance have a negative effect on productivity, mainly by slowing down the technological catch-up process.[20]

The OECD Employment Outlook (1999b) finds that EPL may have a positive effect on the employment rate for prime-age men, but provide only weak evidence for a negative effect on other groups. They do not find any relationship between EPL strictness and overall unemployment, although there is some evidence for a reduction in unemployment of prime-age men with an increase in EPL strictness. Stricter EPL was found to be strongly associated with higher rates of self-employment, a result also found by Grubb and Wells (1993), and with lower turnover in the labour market, with both jobs and unemployment spells tending to last longer.

Bertola and Rogerson (1997) find that the degree of flexibility in wage-setting appears to affect the strength of the link between EPL and employment, with rigid wage setting in the presence of strict EPL being a potentially unfortunate mix.

As well as estimating the effect of labour market regulations on employment, Nicoletti and Scarpetta (2004) also assess the effect of product market reforms on employment. They use the time-varying indicator of product market regulation used in Alesina et al (2003) and Nicoletti and Scarpetta (2003), and find that reforms in both labour and product markets are needed to raise significantly long-run employment rates. They also find evidence of significant interactions between regulations in product and labour markets (allowing for the coefficient of the product market regulation indicator to vary according to the stringency of labour market policies). The estimated negative effects of strict product market regulations on employment are stronger when labour institutions – by strengthening workers bargaining power or reservation wage – push workers to seek a higher share of product market rents.


  • [20]The ability of countries to catch up with the more technologically advanced countries (as measured by total factor productivity) is measured by including an interaction term in the regression equation between the gap in total factor productivity (with the frontier country) and the regulation variable (following Griffith et al (2000)).

5.  Concluding remarks#

Indices of regulatory impact will enter policy discussions as they provide convenient, if equivocal, evidence. As noted earlier, the surveys discussed in this paper generally show New Zealand in a flattering light according to the aggregate indices. What weight should they be given, though, when determining reform agenda? And what do they not say about the impact of regulation?

Assessment is comparison. To say that something is performing well or poorly implies a comparison. When we say that New Zealand is performing well or poorly, what are we comparing performance to? Comparison needs an alternative scenario, a counterfactual. There are at least three possible choices of counterfactual. We can assess current performance in relation to:

  • an ideal outcome;
  • past performance; or
  • the performance of others.

These alternatives have advantages and disadvantages. A comparison with an ideal outcome presents two problems. First, how do we define the ideal? And second, if the ideal is unachievable in practice, how useful is a comparison with it? Maybe the answer is to compare performance not with a theoretical ideal, but with a practical optimum—the best potential outcome given limits to our knowledge and resources. But the problems of definition and measurement remain.

Comparisons with past performance seem natural enough, but are they sufficient? Given experience and uncontrollable events, we ought to expect changes in performance. Concentrating on comparisons with the past tells us where we have gone, but begs the question of where we ought to be. We are led back to comparisons with an elusive ideal or potential.

A third way, the pragmatic thinking underlying various kinds of assessment from school examinations to international indices, is to compare performance among cohorts. How has an individual performed compared with others? If we assume that members of a cohort face similar problems, resources and constraints, then differences in performance tell you how well or poorly you did in relation to others in similar circumstances.

This kind of comparison also seems natural to us. People esteem themselves based on differential characteristics within cohorts. And, if any number of sporting contests can show, people like their group to outdo others. But as natural as it might seem, using this approach to assess nations’ regulatory performance is problematic. When it comes to something like regulation, can there ever be (to continue the sporting analogy) a “level playing field”?

New Zealand, and other countries, face particular constraints and possess particular resources. Where these are historical, or natural, or otherwise outside the current control of nations, what are comparative statistics measuring? Are they saying more about how well countries have done, or are they simply measuring the impact of uncontrollable circumstances? What, for example, does New Zealand’s score in an international index of the competitiveness product markets say about the wisdom of our regulations rather than about our remoteness or the size of our population? What resources should New Zealand spend trying to improve its rank on this index rather than trying to achieve other goals?

Notwithstanding these questions, we argue that these indices, these surveys, provide useful information. They can supplement historical comparisons: they offer information about possibilities today that single-country historical studies do not. Other countries are different from New Zealand, but not so different that we can completely disregard carefully drawn comparisons. These surveys must say something about the performance of our regulation. The question is figuring out what that something is.

Surveys such as these ought to be part of debates on regulation in New Zealand. A number of points must be kept in mind, however:

  • Surveys must be credible. This paper has discussed surveys by reputable organisations using published methods. Surveys that do not disclose methods deserve no regard.
  • Surveys must be as informative as possible. Surveys that provide more detailed results are likely to be more useful than those that provide only aggregate results.
  • Even credible surveys have limitations. These limitations will, since the surveys are conducted by reputable organisations, be stated alongside their methods and results, but an awareness of limitations must be carried into any policy discussions that use the survey results. All results are conditional on these limitations.
  • Greater weight should be placed on results that are independently confirmed by different surveys. On the other hand, idiosyncratic and contradictory results warrant closer study.
  • As mentioned above, the aggregate indices do not tell the whole story. Often the most valuable information (and the greatest flaws) of a survey will lie in the details.
  • All surveys are subjective. When composing an index, or selecting what to measure and how, or framing survey questions, authors of surveys must make subjective judgements. It follows that the results of any survey are largely a function of the subjective judgement of those undertaking it. Numbers can give a false impression of objectivity. But in economic matters, quantitative analysis is often simply subjective, qualitative analysis expressed in numbers.
  • A corollary to this last point is that surveys assume some ideal, even implicitly. In the case of the Heritage Foundation, the ideal (economic freedom) is spelled out, but all of the surveys considered in this paper are based on an idea of what regulatory systems should look like.
  • As mentioned, the surveys usually overlook the benefits of regulation. And these benefits depend on the policy goals of the regulation in question. Unless we know what goals our and other countries’ regulations are designed for, and how well they are meeting these goals, we cannot know whether any particular rank is a good or a bad thing.
  • Country-specific constraints should be taken into account when interpreting surveys. For example, in a small economy, industries may be naturally more concentrated than the same industries in larger economies. Location, distance, population density, and local culture may also materially affect scores and ranks.
  • Finally, for better or worse, we ought not to get too concerned with small changes in the results of these surveys. A good score is no excuse for complacency, and a bad score is no excuse for despair. Good regulatory practice is about aiming to achieve a potential, however elusive, and it is not about climbing a league table. However, consistent trends may matter, large changes in rankings may matter, and the failure to improve poor rankings over time may matter. These things may matter, not because getting better ranks is an end in itself, but because they could indicate problems that are adversely and unnecessarily affecting the well-being of New Zealanders.

Appendix One: Summary of empirical literature#

Authors Indicators of Regulation Used Findings
Griffith and Harrison (2004) Fraser Institute
  • Regulation has a significant[21] positive effect on mark-ups. 
  • Mark-ups are negatively and significantly related to employment and investment.
  • Mark-ups have a positive effect on R&D.
  • Positive and significant effect of mark-ups on both the level and growth rate of productivity.
Cincera and Galgau (2005) Fraser Institute
  • Deregulation tends to be significantly associated with more entry and exit. 
  • Entry is not a significant determinant of the growth in investment, but exit is associated with a significant decrease in the pace of capital accumulation.
  • Negative effect of entry on R&D intensity.
  • Entry and exit both have a positive effect on labour productivity growth.
Scarpetta, Hemmings, Tressel and Woo (2002) OECD (time invariant) For firms that employ between 20 and 99 workers product market regulation has a negative and significant effect on entry, but for the 100 to 499 class the effect is positive and significant.
Brandt (2004) OECD (time invariant) Barriers to entry are not significant, but some evidence that its subcomponent representing regulatory and administrative opacity has some explanatory power for entry rates.
Loayza, Oviedo, and Serven (2005) Construct a summary measure of regulation based on information from the World Bank, the Heritage Foundation, and the Fraser Institute.
  • Product market regulation slows down the reallocation of resources following a shock.
  • The portion of turnover explained by business regulation flexibility has a positive and significant effect on overall labour productivity growth.
Alesina, Ardagna, Nicoletti, and Schiantarelli (2005) OECD (time varying) A reduction in regulation, particularly if it affects barriers to entry, has a significant and sizeable positive effect on investment.
Bassanini and Ernst (2002) OECD (time invariant)
  • Non-tariff barriers have a negative effect on R&D intensity. 
  • No effect of tariff barriers on R&D is detected.
  • No evidence of a differential effect of domestic or administrative barriers comparing low tech to high tech firms
  • Evidence of a positive differential effect for EPL in high tech industries relative to low tech in centralised systems of industrial relations.
Koediik and Kremers (1996) ?? Negative relationship between per capita GDP growth or TFP growth and product market regulation.
Card and Freeman (2004) Fraser Institute Fail to find a significant effect of regulation on the level of output per capita (or per worker) or on its growth rate.
Nicoletti and Scarpetta (2003) OECD (time invariant and time varying)
  • Positive effect of privatisation and entry liberalisation on TFP growth. 
  • Entry barriers in manufacturing may affect the pace of technology absorption.
IMF study (2004) OECD (time varying) Both product market reform and trade reform have a positive and significant effect on growth, although it may take time for the full effects to be realised.
Loayza et al (2004) Time invariant indices constructed from the World Bank, the Heritage Foundation, and the Fraser Institute.
  • Negative and significant direct effect of product and labour market regulation on growth. 
  • Interaction between product market regulation and quality of governance positive and significant, suggesting that better governance reduces the negative effect of regulation.
Nicoletti et al (2001) OECD (time invariant and time varying)
  • Anti-competitive product market regulations have a significant negative effect on non-agriculture employment rates
  • The effects of EPL on innovation activity are negative and significant.
OECD (2003) OECD (time invariant and time varying)
  • Negative direct effect of product market regulations on productivity, and this effect is larger the further a country is from the technological frontier, as strict product market regulation hinders the adoption of existing technologies. 
  • Also an indirect negative effect of strict product market regulations on productivity via their impact on innovation activity.
  • Strict EPL tends to hinder productivity, unless these higher firing costs are offset by lower wages and/or more internal training. 
  • No evidence that strict EPL affects innovation activity.
Gust and Marquez (2002) OECD (EPL) and WEF Burdensome regulatory environments and in particular regulations affecting labour market practices have impeded the adoption of information technologies and slowed productivity growth.
OECD Employment Outlook (1999) OECD (EPL) Stricter EPL strongly associated with higher rates of self-employment, and with lower turnover in the labour market, with both jobs and unemployment spells lasting longer.
  OECD (time varying)
  • Reforms in both labour and product markets are needed to raise significantly long-run employment rates. 
  • Significant interactions between regulations in the product and labour markets: the estimated negative effects of strict product market regulations on employment are stronger when labour institutions push workers to seek a higher share of product market rents.


  • [21]In this summary, “significant” means “statistical significance”: that is, the results are robust according to certain statistical tests. It does not necessarily mean that the effect is big.


Analysis, Working Party No. 1 on Macroeconomic and Structural Policy (2003) "The OECD regulatory indicators questionnaire: Regulatory structures and policies in OECD countries." OECD Economics Department.

Conway, Paul, Veronique Janod, and Giuseppe Nicoletti (2005) "Product market regulation in OECD countries: 1998 to 2003." OECD Economics Department Working Papers No. 419.

Griffith, R, S Redding and John Van Reenen (2000) "Mapping the two faces of R&D: Productivity growth in a panel of OECD industries." London, The Institute for Fiscal Studies, IFS Working Papers, W00/02. http://www.ifs.org.uk/publications.php?publication_id=2051>

Grubb, David and William Wells (1993) "Employment regulation and patterns of work in EC countries." OECD Economic Studies No. 21 (Winter): 7-58.

Gust, Christopher, and Jaime Marquez (2002) "International comparisons of productivity growth: The role of information technology and regulatory practices." International Finance Discussion Papers Number 727.

Gwartney, James, Robert Lawson, and Erik Gartzke (2005) "Economic freedom of the world: 2005 annual report." Fraser Institute.

Gwartney, Jim, and Robert Lawson (1998) Economic freedom of the world: 1998/1999 interim report. (Vancouver: The Fraser Institute).

Institute for Management Development (2005) "The world competitiveness yearbook 2005."

Kaufmann, Daniel, Aart Kraay, and Pablo Zoido-Lobaton (1999) "Governance matters." The World Bank Policy Research Working Paper 2196.

Kee, Hiau Looi, Alessandro Nicita, and Marcelo Olarreaga (2006) "Estimating trade restrictiveness indices." World Bank Policy Research Working Paper 3840.

Koedijk, Kees and Jeroen Kremers (1996) "Market opening, regulation and growth in Europe." Economic Policy 23(April): 443-468.

Miles, Marc A., Edwin J. Feulner and Mary Anastasia O'Grady (2005) "2005 index of economic freedom." The Heritage Foundation and The Wall Street Journal.

Nicoletti, Giuseppe, and Frederic L. Pryor (2005) "Subjective and objective measures of the extent of governmental regulations." Journal of Economic Behavior and Organization.

Nicoletti, Giuseppe and Stefano Scarpetta (2004) "Do regulatory reforms in product and labour markets promote employment? Evidence from OECD countries." mimeo.

Nicoletti, Giuseppe, Stefano Scarpetta, and Olivier Boylaud (2000) "Summary indicators of product market regulation with an extension to employment protection legislation." OECD Economics Department Working Papers No. 226.

OECD (1999a) "Cross-country patterns of product market regulation." OECD Economic Outlook 66(Chapter VII).

OECD (1999b) "Employment protection and labour market performance." OECD employment outlook.

OECD (2003) "What drives productivity growth at the industry level?" in OECD ed The source of economic growth in OECD countries (Paris: chp 3.

Pryor, Frederic L. (2002) "Quantitative notes on the extent of governmental regulations in various OECD nations." International Journal of Industrial Organization 20.

Schiantarelli, Fabio (2005) "Product market regulation and macroeconomic performance: A review of cross-country evidence." World Bank Policy Research Working Paper 3770.

The World Economic Forum (2005) "The global competitiveness report 2005."

World Bank (2005) "Doing business."