- This line was added.
- This line was removed.
- Formatting was changed.
Monte Carlo simulation and Social Cost Benefit Analysis
Monte Carlo simulation is a method based on repeated random sampling of inputs to a deterministic model or calculation procedure. With Monte Carlo simulation, cumulative uncertainties of nature values can be integrated in a socio-economic cost benefit analysis. In the Netherlands, an uncertainty analysis is mandatory in every socio-economic cost-benefit analysis (SCBA), according to the . This also applies to other countries that have SCBA guidelines. The purpose of such an analysis is to determine the influence of uncertain assumptions on the balance (net present value) and ranking of the alternatives. In many SCBA's, the uncertainty analysis is executed in a rather informal way. With the tool presented here, a more formal probabilistic sensitivity analysis can be carried out, based on the Monte Carlo simulation. The advantage of this method is that it provides insight in the cumulative effect of multiple uncertainties, including possible interactions between them. The cumulative effect is especially important for valuation of nature, because these values tend to have rather large uncertainty margins in the balance sheet. The formal sensitivity analysis yields information on which effects contribute most to total uncertainty. This insight can help decision makers in focusing efforts on issues producing the highest uncertainty.
Uncertainties in socio-economic cost-benefit analysis
A socio-economic cost-benefit analysis (SCBA) is used to assess the welfare impacts of large infrastructural projects. The welfare effect of a project is usually expressed in terms of net present value (in euro’s, dollars, or other currency). In practice, an SCBA consists of the following eight steps:
Here, we focus on step 7: sensitivity analysis. Information about the other steps can be found in text books on cost-benefit analysis (see the references of this tool description). Usually, the estimation of uncertainties of the effects is done in a rather simple way by changing the basic assumptions of one effect at a time. The drawback of this so-called sensitivity analysis is that uncertainties are only investigated in the vicinity of the ‘pivot’ point in parameter space (i.e. the standard setting around which each input is varied) and interactions and cumulative effects of uncertainties are poorly understood. The Monte Carlo simulation provides a method to explore the entire parameter space and to include interactions and cumulative effects of uncertainties in a more formal way.
Monte Carlo simulation
Monte Carlo simulation is a method based on repeated random sampling of inputs to a deterministic model or calculation procedure. For each simulation all input variables are randomly drawn from predefined probability density functions. For each set of inputs, a deterministic model run or calculation is made. The outputs of a large number of such simulations are analysed statistically, to yield probability density functions of the output variables.
Relevance for Building with Nature
The Building with Nature strategy aims at including nature value in project design and using natural processes to human benefit. However, it is a strategy with more uncertainties than a traditional design, partly because the natural processes are sometimes poorly understood, partly because they include inherent uncertainties. Building with Nature would therefore benefit from a tool that estimates cumulative uncertainties in a socio-economic cost-benefit analysis, thus enabling to formally incorporate uncertainties in important investment decisions. This tool can be used in any investment project where an SCBA is made and where uncertainties play an important role. The SCBA is usually applied in an early stage (e.g. a feasibility study) of infrastructure developments requiring major investments.
How to Use
SCBA’s are to be made by trained specialists, as they are susceptible to mistakes. Although most people understand the concept of costs and benefits, carrying out a thorough SCBA for a large project is not a simple task. SCBA in combination with Monte Carlo simulation makes the analysis even more complex, because it also requires statistical expertise. This tool may be used by SCBA-specialists with statistical skills or an SCBA-specialist and a statistician working together.
Phased plan process
Performing a Monte Carlo simulation on an existing model or calculation procedure requires the following steps:
The reliability of the results improves with the number of simulations performed. In the example presented in the practical application 10.000 simulations were done. Monte Carlo simulations can be performed with a number of mathematical software packages. Here MatLab was used in combination with an Excel spreadsheet for the SCBA-computations. Other software packages have not been tested when developing this tool description.
In this section we present a case-study in which we have applied a Monte Carlo simulation in combination with a socio-economic cost-benefit analysis to new developments around Lake Grevelingen in the Netherlands (Witteveen+Bos 2011).
Case: socio-economic cost-benefit analysis Lake Grevelingen
Lake Grevelingen is a former estuary, part of the Rhine/Meuse/Scheldt complex in the southwest of the Netherlands. Since the closure of the estuary by the Grevelingendam (1965) at the landward side and the Brouwersdam (1971) at the seaward side, and the construction of a saltwater inlet sluice (1978) ,it is the largest saltwater lake in Europe. In the meantime, Lake Grevelingen has developed into a valuable natural and recreational area of national and international significance. The lake is used for swimming, windsurfing, canoeing, diving, boating and fishing. Commercial fishing of lobster, eel and oyster is also an important function. However, the Brouwersdam has several side-effects, such as the ecological degradation of the lake. In order to counter these side-effects, the national government has started a feasibility study with the following objectives:
In the feasibility study 5 alternatives to reach these objectives were investigated. Here, we use the first alternative to show the applicability of the Monte Carlo Simulation. This alternative consists of the following measures (see figure):
Results of the socio-economic cost-benefit analysis
The results of the socio-economic cost-benefit analysis are presented in the table below, a study completed in 2011 (Witteveen+Bos, 2011). Column 1 presents the results of the original alternative described above. The results show that alternative 1 has a negative net present value. In the sensitivity analysis it was investigated whether it is possible to reduce the costs and to augment the benefits so as to obtain a positive balance. This was done via trial and error by changing the basic assumptions of the most important entries in the balance sheet:
Cost reduction was obtained by changing the design of the turbines, the shipping lock in the Brouwersdam and the design of the opening in the Grevelingendam. It became clear that basic design assumptions strongly affect the investment costs.
Energy prices fluctuate considerably over the years. An important question was how much energy prices should go up so as to obtain a positive balance (break-even point). From this computation it became clear that energy prices should double from 0.06 euro/kWh to 0.12 euro/kWh in order to give alternative 1 a positive balance. Column 2 presents the results under the assumption that all design optimisations are effectuated, without increasing energy prices. The investments are much lower in this alternative because the original alternative was optimised and some other choices were made (Witteveen+Bos, 2011). Column 2 shows that in this case alternative 1 can get a positive balance. However, this is only the case if all design optimisations will be incorporated. It is not clear how realistic this assumption is. Therefore a Monte Carlo simulation is carried out to quantify the probability of obtaining a positive balance.
Table 1: Results of the SCBA in million Euros:
Monte Carlo simulation
For a Monte Carlo simulation it is necessary to have the probability distributions of the values used in the calculations for the SCBA. To limit the research effort we keep the smaller benefits fixed. Moreover, we assume that all probability distributions are normal. We only collect the parameters of the normal probability distributions for the costs and large benefits:
Investment and maintenance costs
In general the height of the project costs depends on many factors. For the case-study of Lake Grevelingen the total costs of the measures vary considerably between 702 and 353 million euro's, depending mainly on the design of the structures. The maintenance costs are calculated as a fixed percentage of the realisation costs.
Flood protection benefits
The benefit of water storage on Lake Grevelingen is that investments for increasing dike height along the lower reaches of the debouching rivers can be cancelled or postponed. Here two types of uncertainty are considered:
Uncertainty about the moment when flood protection measures are necessary depends mainly on uncertainty in expected future water level statistics in the rivers. The meteorological scenario applied shows an uncertainty in the water level forecasts of approximately 0.2 meters (DHV/HKV 2010). On average it takes approximately 30 years for a rise in water level of 0.2 meters. This is the combined effect of sea level rise and increase of river floods. This means that the moment at which the investment has to be made has an uncertainty range of 30 years (see table below).
The cost of increasing the height of dikes varies between EUR 5 million and EUR 10 million per km. In the Monte-Carlo simulation EUR 5 million and EUR 10 million are used as the 5% and 95% probability of exceedance, with a mean value of EUR 7,5 million (see table below).
Energy yield of the tidal power station
Here we consider two types of uncertainty:
A hydraulic model was used to determine the energy production of the tidal power plant (Royal Haskoning 2010). This study gives some information about the bandwidth in energy production (see table below).
Profit of shell-fish (mussels) fisheries
Bringing back the tide in Lake Grevelingen creates opportunities for mussel farming. Here three types of uncertainty related to mussels are considered:
There is a wide variation in the price of mussels. After analysis of various sources, in the MC simulation a 5% probability of exceedance level of EUR 0.25 per kilogram was assumed and a 95% probability of exceedance level of EUR 1.90 per kilogram, with an average of EUR 1.10 per kilogram (see table below).The type of probability distribution can differ for each uncertainty, but in this case no reliable information was available to select distributions other than the normal distribution.
Table 2: Valuation of uncertainties with 5% and 95% thresholds, and their mean values:
Results of the MC simulation
For alternative 1 an MC simulation consisting of 10.000 runs was performed. For each run, input variables were randomly drawn from the probability distributions of table 2. The figure on the right shows the MC simulation results for the total costs, the total benefits and the balance. The expected net present value is EUR -248 million and the standard deviation EUR 337 million. The probability of a positive balance is still 23%.
As can be seen in the figure, the benefits vary more than the costs. The factors contributing to the benefits are added and therefore the uncertainties of these contributions are directly translated into the balance (the same holds true for the costs). To reduce the uncertainties, it is in this case advisable to do more research on the benefits. If you want to know which factor contributes most to the total uncertainty, an extra MC can be performed. This extra analysis has not been done in this practical application.
Effort and time: First conclusion is that performing a cost-benefit analysis with an MC simulation is not too complicated, but it is time consuming. The main difficulty is finding useful information for establishing the probability distributions. For different types of benefits, one can rely on available databases which include standard values for calculating the physical effect and the impacts on welfare (including prices). In most cases they include the mean values only. Data providing information on type and parameters of the most suitable probability distribution are often lacking.
Reducing uncertainties: The MC simulation yields useful information on which items in the balance sheet contribute most to the total uncertainty in the net present value. This information can be used to guide further research so as to reduce uncertainty.
Decision-making: An MC simulation should result in more comprehensive information about uncertainties and thus add to the decision-making process. It is true that an MC simulation leads to more information (see figure above): the graphs show additional information about the bandwidth of costs, benefits and the balance between them. Based on the MC-simulation it is also possible to give a prediction of the probability of a positive balance between benefits and costs. The question is whether this ‘new’ information is useful and workable to the decision maker. Reporting that there is a 23% chance of a positive balance can be very confusing to decision makers, especially if he thinks this is a completely random process.
Suggestions for further reading
Related Building solutions