Annex 1: Monte Carlo Analysis
Monte Carlo analysis is a risk modelling technique that uses statistical sampling and probability distributions to simulate the effects of uncertain variables on model outcomes. It can be used to model the effects of key variables such as exchange risk, staff turnover and demand for services on the NPV of a given proposal. The approach provides a systematic assessment of the combined effects of multiple sources of risk in key variables and can also allow for known correlations between these variables.
Monte Carlo provides decision-makers with a visual representation of the:
- expected value and range of variability due to risk and uncertainty on each of the variables modelled
- relationships between these variables and estimated possible outcomes, and
- expected value and range of the possible outcomes, representing the combined effect of the multiple sources of uncertainty.
The process firstly involves the identification and assessment of the key variables. For each variable, a suitable probability distribution (or multivariate distribution) is assigned that best describes the range of uncertainty around the expected value. For each variable being modelled, input values are generated randomly, sampled from the underlying probability distribution function. The computer model combines these inputs to generate an estimated outcome value (for example an NPV). The process is repeated (thousands of times) to generate a probability distribution of possible outcomes. This distribution can be analysed to provide an indication of the variability or robustness of the NPV analysis. For example “there is an 86% chance that the NPV exceeds 0%”.
For example, consider a major IT project where there is significant uncertainty around the net project cost. One output of Monte Carlo analysis is the following diagram:
The diagram shows the probability of delivering the project at the specified cost, given the risk scenarios that have been considered during the analysis. It also gives a visual indication of the range of likely costs associated with the project or module within the project, and is therefore useful in presentation of the proposal to Ministers. The probability curve can also be used to set the thresholds for appropriations and cash-draw down limits for the project/module, based on the likely final costs.
However the technique is complex and requires expert advice to develop the model and interpret the results. The approach should only be used when there are several key variables with significant and/or correlated uncertainties, and when simpler sensitivity analysis approaches are unable to adequately describe the resulting variation in net benefits. For more detailed information see Vose, D. (1996). Quantitative risk analysis : a guide to Monte Carlo simulation modelling, Chichester, U.K.: John Wiley & Sons Ltd.
