3.2  Uncertainty and policy making

Policy makers are inevitably faced with making decisions under considerable uncertainty. Gorringe (1991) considered that uncertainty about how the world really works is great because:

• economies are large and complex systems;
• the structure of relationships within economies has limited stability through time;
• often unpredictable human action is involved;
• the use of controlled experiments is generally not possible;
• the quantity and quality of the available data is limited; and
• the tools of economic theory and econometrics are limited.

These uncertainties are especially relevant to areas of theoretical policy that have limited history of application. In contrast to monetary and fiscal policy, where many aspects have evolved into reasonably stable operating principles and practices, Crown financial policy is largely untried in practice.[18] Consequently, policy makers face considerable uncertainty about how alternative Crown financial policies would impact on economic welfare. These uncertainties are likely to remain large for the foreseeable future irrespective of how much empirical research is untaken prior to initial implementation.

Limitations of standard statistical inference

Most published research adheres to a standard statistical methodology for dealing with uncertainty that involves a ‘null’ hypothesis and an ‘alternative’ hypothesis. The null hypothesis is that the variable of interest has no impact. Typically, the alternative hypothesis is rejected in favour of the null hypothesis unless the relevant parameters are different from zero at the 5% level of significance. It has been recognised since at least the 1960s that this method, known as the Neyman-Pearson theory of statistical inference, is inappropriate for policy decisions (Blaug 1992 pp. 21-3).

The key problem with the Neyman-Pearson method is that it assumes the cost of falsely rejecting the status quo (the null hypothesis) is an order of magnitude larger than the cost of falsely rejecting the competing proposal (the alternative hypothesis).[19] While this assumption may be appropriate for guiding the progress of science, for policy (and other decisions) the relative magnitude of costs often are more balanced. To provide a sound basis for policy to maximise economic welfare, it is necessary to estimate of the costs and benefits arising under various scenarios and the probabilities of those scenarios, occurring. Blaug (1992 pp.21-3) and Gorringe (1991, 1998) discuss these issues in more detail.

An example where the inappropriateness of statistical inference for policy making has been taken seriously is the literature on monetary policy under model uncertainty. The monetary policy literature assesses the performance of alternative policy reaction functions in circumstances where the “true” model of the economy is different from that specified in the reaction function.[20]

Bayesian decision theory

In addition to comparative institutional analysis, we adopt a version of Bayesian decision theory. In essence, the Bayesian approach is an application of standard micro-theoretic decision making under uncertainty where the decision maker’s objective is to maximise expected utility.[21] A key assumption in the Bayesian approach is that the decision maker can assign a subjective probability to every potential outcome.[22]

To develop notation for later use, consider a policy option (PO) that is an alternative to the status quo (or another policy option). The policy maker must decide whether to implement the policy option or retain the status quo in the face of uncertainty about the true nature of the economy. A decision to implement is indicated by placing a high weight (denoted “H”) on the policy option, whereas a decision to retain the status quo is indicated by placing a low weight (“L”) on the policy option.[23]

Suppose that the “correct” decision depends on whether the true impact of the policy action exceeds a threshold value. If the impact exceeds the threshold it is called economically significant (“s”) and the policy option should be implemented. If the impact is less than the threshold it is called economically insignificant (“i”) and the status quo should be retained. The policy maker does not know whether the impact would be significant or insignificant but has formed a probability distribution over the true states, s and i.

There are four possible outcomes:

• True Positive (TP) The impact is significant (s) and the policy maker correctly assigns high weight (H); or
• False Negative (FN) The impact is significant (s) but the policy maker incorrectly assigns low weight (L); or
• True Negative (TN) The impact is insignificant (i) and the policy maker correctly assigns low weight (L); or
• False Positive (FP) The impact is insignificant (i) but the policy maker incorrectly assigns high weight (H).[24]

Let the payoffs for the scenarios be random variables Ṽ_TP, Ṽ_FN, Ṽ_TN and Ṽ_FP.

The structure of the decision and possible outcomes are illustrated in Figure 2 below, where the dashed oblong indicates the policy maker (P) does not know whether the true state is ‘s’ or ‘i’. The true state is thought of as being determined by nature (N).

A property of a decision structure of this nature is that attention can be restricted to the false positive and false negative decision errors. This is achieved by defining the loss from a decision error as the value forgone relative to the correct decision.[25]

Bayesian decision theory assumes there exists a utility function that represents the decision makers’ risk preferences over losses incurred under false positive and false negative errors. Consistent with standard micro-economic theory, the optimal decision is to implement the policy option if and only if the expected utility of losses under implementation is less than the expected utility of losses under the status quo.

Figure 2: Decision structure facing policy maker