The Treasury

Global Navigation

Personal tools

Treasury
Publication

Guide to Social Cost Benefit Analysis

Part 2: Issues in Cost Benefit Analysis

CBA's limitations and common criticisms

178.CBA is a partial-equilibrium method. It typically measures direct or ‘first-round’ effects. It also typically assumes that all down-stream markets are perfectly competitive (although some CBAs, for example those of some transport projects, are starting to make an allowance for imperfect competition by estimating the increase in competition that is likely to result from the project) and that all prices equal marginal cost.

179.In many situations it will be quite reasonable to make these simplifying assumptions. However, the analyst must always exercise judgement as to whether there are good grounds for departing from these assumptions and make different assumptions.

180.Other limitations include the fact that all estimates of costs and benefits are based on uncertain forecasts, and some costs and benefits may be very difficult to estimate, or may not even be known. And as discussed above, CBA also typically measures willingness to pay rather than the impact on welfare.

181.These limitations can in some cases be quite severe, but they are not a reason not to use CBA. Other tools, such as multi-criteria analysis (MCA) are likely to be even worse (see discussion on page 52 and following), as is sole reliance on the decision-maker’s intuition. If a decision-maker’s intuition suggests a different estimate of costs or benefits, then rather than disregard the results of the CBA, the better course of action is to re-examine the analysis, test the assumptions, and if appropriate, seek an independent peer review.

Criticisms

182.This section sets out some common criticisms of CBA and explains why they are not generally a good reason not to do a CBA:

  • CBAs produce false accuracy”: It is not unusual to see CBAs that state that the benefit cost ratio is “1.17”. This is most likely to be spurious accuracy. The problem of false accuracy is overcome with the use of ranges.
  • “CBAs can't measure everything”: We acknowledge that there are some intangible benefits that the analyst won't be aware of or that are too hard to measure. As this guide explains, there are more benefits that can be measured than people think. As for those that can genuinely not be measured, this guide recommends that they should be drawn to decision-makers' attention along-side the results of the CBA of those benefits and costs that can be measured.
  • “CBA can be misused to produce self-serving analysis”: CBA is a tool, and like all tools it can be misused. This is not a reason to dismiss CBA in favour of some other tool.
  • “CBA is too complex”: This guide does not recommend that inexperienced policy analysts should carry out CBAs of complex or large projects themselves. Either a ‘rough' CBA can be carried out, or the job should be contracted out to specialists. However, it is important for those carrying out CBAs to produce an accessible report that heeds the recommendations of this guide.
  • “Information requirements are often too onerous”: CBAs can be carried out with whatever information is available. If the information is poor, then the confidence intervals will be larger. There are no other project evaluation methodologies that can produce better results from the same information base.
  • “CBAs overlook equity considerations”: Equity is discussed on page 47, which recommends that equity implications of a project be discussed and drawn to decision-makers’ attention along-side the results of the CBA.
  • “The CBA is not likely to support our Minister's objectives”: This comment ignores the fact that public servants have two distinct roles. The first is to give ministers free and frank advice on what the likely consequences of their decisions are. A CBA is necessary for this role. The second is to implement the Minister's decisions, whether or not those decisions are consistent with the advice given.
Page top