4.4   Scenario Analysis

The simplest form of sensitivity analysis is scenario analysis. Scenarios can be chosen to draw attention to the major uncertainties upon which the success of a proposal depends. Are there any variables (such as exchange rates, salary costs, demand drivers, timing or assumptions) that materially influence the net benefits? These key variables should be identified using the risk assessment process outlined at the start of the chapter. The scenario analysis should then focus on asking “what if” questions and recalculating the expected NPV for several scenarios. For example, what if one or more sensitive/key variables were changed by ±10% or ±50% or whatever is a realistic and possible variation. What if related Government policy altered or critical legislation is not passed? If these events occur, should the proposal proceed? Under what circumstances does the preferred option change? A common approach is to test three combinations of key variables:

• pessimistic or conservative scenario
• most probable or base scenario
• optimistic scenario.

For example, consider the cost of purchasing a large mainframe computer:

If the NPVs do not alter significantly enough to affect the final decision then it is possible to feel more comfortable about the robustness of the analysis and the net benefits of the proposal.

This approach is usually appropriate for smaller proposals or where the outcome of the cost benefit analysis is more certain. For larger proposals with greater uncertainties, the number of variables can be increased and sensitivity analysis can also be used within each scenario. Or a more complex approach such as Monte Carlo analysis can be used (see Annex 1 for more details).

4.5   Intangibles

Intangible costs and benefits are not easily quantifiable in monetary terms. Intangibles may be either internal or external costs or benefits. As discussed in chapter 2, the recommended approach is to capture all identifiable costs and benefits if there is a reasonable basis to include them and include the resultant resource-flows explicitly in the cost benefit analysis. Some intangibles can be quantified in non-monetary terms (for example, one of the benefits from an e-Government initiative might be “time saved” by paying court fines online).

Intangibles that cannot reasonably be quantified in monetary terms should be excluded from the quantitative analysis detailed in chapter 3. However, these intangible benefits and costs can be significant in relation to the quantitative impacts, and significantly influence the final decision. If they are significant they should be explicitly highlighted and explained in the analysis so that decision-makers are aware of the value judgements they are making in pursuing a particular option. This explanation can be quantitative, qualitative, descriptive, or a combination of these.

4.6   Multi-Criteria Analysis

The most common form of qualitative analysis for comparing unvalued costs and benefits is multi-criteria analysis.[55] Multi-criteria analysis is a tool for appraising and ranking alternative policy options against a given set of objectives and criteria. It is less rigorous than cost benefit analysis or cost effectiveness analysis, but is more flexible since it is relatively easy to implement and can be used to assess and compare options that involve both monetary and non-monetary impacts. It can aid decision-making by complementing the quantitative cost benefit analysis.

Multi-criteria analysis usually involves setting a list of success criteria (possibly reflecting public policy goals) and assigning weights to each criterion. The alternative options can be assessed and scored (typically by a representative panel of stakeholders) against the criteria, with the assigned scores multiplied by the weightings, yielding a ranking of alternative options.

It can be used without explicit weighting of the criteria. However this reduces the transparency and validity of the ranking process. Even where the weightings of the criteria are explicit, neither the criteria nor the weights are based on any underlying analysis, and can be easily altered. This can be overcome by consulting experts and stakeholders when determining the criteria, weightings and ranking.

An example of multi-criteria analysis for a major information technology project where scores are out of a maximum of 100 might look something like the following:

In this case, supplier 1 is likely to be the preferred option.

Importantly multi-criteria analysis provides decision-makers with an audit trail as the results are transparent, explicit and documented.

4.7   Concluding Comments

Decision-makers need a consistent basis for assessing competing proposals and to be fully informed about the implications of using economic resources. By quantifying all significant costs and benefits in monetary terms it is possible to determine the net benefits or costs of a given proposal. However some significant benefits or costs may be subject to uncertainty and/or may not be able to be quantified in monetary terms. This chapter has outlined a number of analytical tools to unable analysts to assess the effects of risk and uncertainty, and to allow decision-makers to make informed trade-offs between the quantitative and qualitative factors that would influence the choice of preferred option.

If more detailed guidance is required please contact your Treasury Vote team.