Mapping uncertainty from multi-criteria analysis of land development suitability, the case of Howth, Dublin

Abstract

This paper presents a method for determining and mapping suitable locations for development using Multi Criteria Analysis and the Analytical Hierarchy Process and considering uncertainties in the process. The method is applied to the case study of Howth (Dublin), where development suitability is assessed against specific protection and conservation areas as well as ground water vulnerability. Uncertainty is incorporated using a Monte Carlo simulation into the Analytical Hierarchy Process calculations to determine criteria weightings. A map is derived, which includes, for all locations, both site suitability for development and the level of uncertainty attached to this suitability. The map combines a double categorization of suitability and uncertainty. The method allows for increased transparency in decision making regarding site suitability for development, as well as increased confidence in decision making to allow for reduced risk in terms of the potential impact of development.

Keywords:

• analytical hierarchy process
• GIS
• land use planning
• multi criteria analysis
• uncertainty
• Monte Carlo analysis

1. Introduction

This paper outlines an approach to determining and mapping uncertainty in land suitability analysis. An application is conducted in Howth (Dublin), where development suitability is assessed against specific protection and conservation areas as well as ground water vulnerability. The popular multi-criteria analysis (MCA) method is used as a tool to find suitable sites for development. A standard approach for determining land suitability is used which involves determining weightings for the MCA using the Analytic Hierarchy Process (AHP) method and mapping site suitability using geographical information systems (GIS). The purpose of this research is to advance the MCA process further by mapping the uncertainty surrounding determinations of site suitability. This approach is likely to prove useful in considering the risks surrounding the selection of suitable sites for development and allows for environmental risks to be mapped. This can allow for greater confidence in decision making, reduced risk in site suitability decisions, and increased transparency in relation to decisions surrounding site selection for development.

A review of recent literature shows the use of GIS incorporating MCA techniques for improving decision-making in relation to land use suitability is well documented (Benke & Pelizaro, 2010; Grandmont, Cardille, Fortier, & Gibéryen, 2012; Kara & Doratli, 2012; Soltani, Mahiny, Monavari, & Alesheikh, 2013; Sposito, Benke, Pelizaro, & Wyatt, 2009) due to the ability to combine geographical data stakeholders preferences in relation to the importance of environmental factors. Whilst there are clear and multiple references to methods for determining land suitability in the literature, much less attention is given to the consideration of uncertainty within this process.

A number of recent papers attempt to measure uncertainty in MCA decision making. Various sources of errors and uncertainty are discussed in the literature. Those related to the judgement of the relative importance of factors are identified as significant (Benke & Pelizaro, 2010; Chen, Yu, & Khan, 2010; Grandmont et al., 2012; Soltani et al., 2013). Chen et al. (2010) and Benke and Pelizaro (2010) make an attempt to address what is described as a lack of attention to evaluating the results of MCA decision-making processes. They note that very little research has been reported in uncertainty analysis and its visualization in model predictions of land suitability, with analysis of MCA often cited without any indication of error or confidence in the results. Hengl (2003) outlines how visualization of uncertainty and its influence on decision-making has not been of much interest in the past giving the false impression that maps are 100% correct in a study area. Foody (2003) notes that end-users such as policy makers and decision-makers often fail to appreciate uncertainty fully and underestimate its relevance. Foody goes on to note that this failure to recognize uncertainty, whatever its source, may lead to erroneous and misleading interpretations of results. Hengl (2003) defines uncertainty as unpredictability or indefiniteness of the prediction models that can, in many cases, be estimated from statistical models. He notes that with emerging quantitative GIS tools, uncertainty is becoming an important aspect of the mapping process with its visualization allowing users to investigate the effects of different decisions, both to visualize the results of the spatial prediction and impact of the propagated uncertainty on decision-making.

The AHP process often relies on expert judgement of the perceived relative importance of environmental factors on a pairwise basis resulting in a set of weights to be used to produce a site suitability map. This judgement of the relative importance of factors can result in uncertainties and errors in the MCA process. This is noted by Grandmont et al. (2012), who continue by stating that this can introduce substantial uncertainty that can translate into weak or imprecise results in site suitability analysis. Benke and Pelizaro (2010) note that a desirable objective of MCA is to gain a sense of the error or uncertainty in the predictions given the uncertainty in the criteria weights. It is therefore useful to quantify and map the potential effects of inaccurate judgement of the relative importance of factors in the MCA AHP process.

Case studies which attempt to quantify and visualize uncertainty include methods for editing the AHP model itself to introduce uncertainty by measuring the confidence value surrounding judgements and incorporating this confidence into the AHP calculation and mapping the results and their confidence values (Grandmont et al., 2012). Benke and Pelizaro (2010) note that land suitability analysis involves the challenge of incorporating some measure of uncertainty in the weight assignments produced by the AHP model. They note that modifying the AHP model itself to incorporate this uncertainty is arguable on theoretical grounds and therefore attempt a posterior sampling strategy on the feasible weights to produce a quantitative and visual representation of possible uncertainty in model predictions. Grandmont et al. (2012), on the other hand, outline an approach which incorporates uncertainty into the AHP calculations themselves. The literature notes that sensitivity analysis should permit weight sensitivity to be visualized geographically and to facilitate the spatial analysis of uncertainty (Hengl, 2003).

Methods for studying uncertainty include the popular Monte Carlo simulation. This method can be used to provide statistical representations of errors as a result of expert opinion and rankings, by plotting the variability in model outputs. Grandmont et al. (2012), Sposito et al. (2009), Benke and Pelizaro (2010) and Chen, Yu, Shahbaz, and Xevi (2009) carry out a Monte Carlo simulation to consider how uncertainty surrounding the AHP rankings can influence site suitability using MCA. Sposito et al. (2009) use Monte Carlo simulation to determine standard deviation to provide a visualization of uncertainty whilst Grandmont et al. (2012) follow a similar approach using the coefficient of variation to determine uncertainty following a Monte Carlo simulation. Grandmont et al. (2012) use this approach to prepare a single map combining site suitability and uncertainty, allowing the end user to easily identify suitable sites for development whilst taking into account the certainty of suitability. Benke and Pelizaro (2010) discuss the usefulness of using a centre-weighted probability distribution where the determined weight is the mode value and using this distribution to randomly sample for each iteration of the model in a Monte Carlo simulation to produce a final output probability distribution. From this distribution, uncertainty metrics can be derived, such as standard deviation or confidence interval. This result provides the scatter or statistical variation needed for uncertainty assessment of the model output. Hengl (2003) outlines an approach for representing uncertainty using continuous variables, represented with the standard deviation of the prediction error mapped through-out the study area.

This paper focuses on uncertainty as a result of judgement-based errors in the weightings used for carrying out the MCA AHP. The paper proposes a method similar to that proposed by Grandmont et al. (2012) but adapting it to consider uncertainty in site suitability for development regarding ecological factors and ground water vulnerability. The method attempts to provide an easy to understand overview of the uncertainty associated with the MCA results by mapping the uncertainty in a fashion inspired from Hengl (2003). Associating both an uncertainty assessment method and effective mapping is argued to help the decision-making process regarding suitable sites for development.

2. Data

The study area is located in Howth, a suburban area of Dublin, Ireland within the administrative boundary of Fingal County Council. This area was chosen as the study area due to the availability of data in relation to the factors chosen for analysis which relate to areas containing protected habitats and species, and ground water vulnerability data. A different set of factors could equally be used to carry out the study. The area was also chosen as it represents a realistic location for urban development to occur. However, it should be noted that the study is purely a methodological approach to consider the potential for considering uncertainty in practice in land use planning. If this approach is to be used in practice then a wider set of criteria would need to be included to determine site suitability. For this reason, the results of this study should not be interpreted as indicating site suitability and certainty for development in this study area.

The area is characterized by an environmentally sensitive coastal area but is ideally located for future development due to its proximity to Dublin city, the existence of a high-quality rail connection serving the area, a high-quality living environment, and an ageing population with capacity in existing infrastructure and services such as childcare, education and transport infrastructure.

However, the environment is particularly sensitive to potential development in this area and a set of environmental factors must be considered in light of certain EU Directives pertaining to the protection of the environment, especially the Habitats Directive regarding the protection of species and their habitats and the Water Framework Directive regarding water quality. The study area includes Special Areas of Conservation (SAC) containing protected habitats and Special Protection Areas (SPA) containing protected bird species, both protected under the EU Habitats Directive. The area is also characterized by extreme and high groundwater vulnerability (GWV) which indicates the likelihood of contamination of groundwater as a result of human activities. GWV is measured across the area to facilitate its protection in line with the EU Water Framework Directive. In order to facilitate future development, yet ensure the continued protection of the environment, there is a need for increased confidence in the decision-making process regarding land use change. Caution must be exercised in making recommendations for the location of future development with consideration given to these environmental sensitivities.

Available data were gathered in the form of GIS vector files. These included the location of SAC's, the location of SPA's and the classification of ground water vulnerability (GWV). Using these data, thematic maps were created for each of the environmental factors after reclassification (see below and Figures 1 –3 in the Main Map). Maps were converted from vector to raster to carry out the MCA process to determine site suitability on a cell-by-cell basis. A cell size of 10 m by 10 m was used, as this is the recommended cell size for planning at the local area level (González Del Campo, 2009).

Cells within SPA, SAC and GWV classification ‘water’ were removed, as these are considered completely unsuitable for development. Jenks classification method was used for all maps where classification was necessary in order to obtain and visualize internally consistent categories and thus development zones.

3. Methods

The research herein uses MCA to find suitable sites for future development and combines SAC's, SPA's and GWV's information. Uncertainty associated with the expert judgement involved in determining the relative weighting of the three criteria is then computed and mapped. The standard MCA method with criteria being weighted after an AHP process is further explained in section (3.1). Uncertainty is then added in the AHP process as in Grandmont et al. (2012) using Monte Carlo simulation.

3.1. Carry out MCA using AHP

MCA is used to create a base map of suitable sites for development. The MCA involves reclassifying the existing values for each of the three environmental factors being considered. Each raster layer was reclassified so that all data have a common measurement scale. This is a necessary procedure for carrying out MCA to allow for factors being considered to be comparable to one another on a numerical scale. This is necessary to transform the factors to interval variables that can be considered on a scale of 0 to 9, with 0 being the most sensitive areas and 9 the least sensitive areas. Table 1 below shows the values for each environmental factor. The reclassified values were determined based on the characteristics of the environmental factor being considered. SAC and SPA were reclassified based on Euclidean distance from the SAC and SPA boundary, with locations inside the SAC and SPA having a value of ‘0’ (completely unsuitable for development) and the reclassification value increasing with increased distance from the SAC and SPA boundaries (i.e. locations become more suitable with increased distance from ecological sites). Regarding GWV, locations with lower vulnerability to groundwater pollution were classified with high values, with higher values reflecting increased suitability for development.

Table 1. Input data reclassification – Showing original classification of each factor and reclassified value.

Environmental Factor	Reclassified Suitability Values
	0	1	2	3	4	5	6	7	8	9
SAC (distance in metres)	In SAC	100	200	300	450	550	700	800	950	1100
SPA (distance in metres)	In SPA	100	200	300	450	600	900	1200	1500	1800
GWV (vulnerability classification)	Water		Exposed Rock	Extreme	High			High-low	Medium	Low

Using the reclassified values, thematic maps were created for each environmental factor to show suitability of land for development, with blue areas unsuitable and purple areas suitable for development (Figures 1 ndash;3 in the Main Map).

The Analytic Hierarchy Process (AHP) was then used to rank and weight the three environmental factors and carry out the MCA. AHP is a method proposed by Thomas Satyr in 1977 and still used extensively today in complex decision-making processes in relation to environmental planning as demonstrated by its wide use in the literature in this regard. Examples include Grandmont et al. (2012) who use AHP to determine weights among factors that can influence permafrost; Chen et al. (2010) and Benke and Pelizaro (2010) use AHP to determine land suitability for agriculture; Soltani et al. (2013) use AHP to determine suitable locations for urban development. In multi-criteria decision-making, AHP determines applicable weights through a pairwise comparison process whereby users prioritize between two elements using a given scale. For example a decision maker determines the priority of SAC over SPA on a scale of 1 to 9 with 1 being of equal importance and 9 indicating extremely high prevalence of one factor over the other. This is repeated through all pairs of criteria (SAC over SPA, SAC over GWV, and SPA over GWV), thereby deriving priorities among all criteria.

The AHP values in the present example were based on the authors experience working in land use planning and taking into account the SAC and SPA environmental protection designations and the relevant EU Directives (Habitats Directive and Water Framework Directive). After these ‘best guess’ ratios, the AHP returns a set of percentage weights for each factor (Table 2). These weights are used in a weighted spatial overlay analysis in order to create a suitability map (Figure 4 in the Main Map). Using the three thematic maps shown in Figures 1 –3 in the Main Map which show the environmental factors reclassified to a common measurement scale of 0 to 9, each factor is then weighted according to a percent weight which reflects its importance. The weight is a relative percentage calculated in this case using the AHP, and the sum of the percent influence weights must equal 100. Each cell value on Figures 1 –3 in the Main Map is then multiplied by its percentage influence shown in Table 2. The three thematic maps are summed with respect to their relative weights from the AHP calculations thereby providing a map indicating the suitability of each cell in the map for development as shown in Figure 4 in the Main Map.

Table 2. AHP matrix, confidence values and MCA weights.

	SPA		GWV		MCA Weight
	Expert Guess	Confidence Value	Expert Guess	Confidence Value	From Expert Guess (%)
SAC	3	1.25	7	3	65
SPA	–		5	0.5	28
GWV	–		–		7

This is normally the output map of the MCA procedure with no further consideration given to the level of uncertainty surrounding cell suitability. This map shows green areas as most suitable for development (least likely to cause pollution to SAC, SPA and GWV if developed) and red areas as most unsuitable for development (most likely to cause pollution to SAC, SPA and GWV if developed).

3.2. Incorporate uncertainty into AHP

The normal AHP process involves choosing a ratio of relative importance between pairs of factors on a scale of 1 (equal importance) to 9 (extremely high prevalence). Whilst expert judgement is accepted as an appropriate method for such evaluation and should not be replaced, caution must be taken due to the potential for inaccuracy. Such inaccuracy can have the effect that the resulting percentages from the AHP process could vary, resulting in a change to the MCA weightings and so a change in suitability. In the case outlined in this report, the best guess AHP values have resulted in the weight values 65%, 28% and 7% for the three factors (Table 2). Following Grandmont et al. (2012) method, a confidence value has been included in relation to each of the expert's AHP values to allow for variation in potential weights to be determined. The confidence values used are shown in Table 2. For example, the AHP pairwise value 5 attributed to SPA: GWV has been given a confidence value of 0.5. This low value indicates that the expert has a high confidence in this value. The AHP value for SAC:GWV is 7 with a confidence value of 3, thus indicating a higher level of uncertainty surrounding this pairwise comparison. The method outlined herein uses expert opinion to determine the importance of factors and the associated uncertainty. By allowing each expert to include a measure of uncertainty the process can accommodate variation in the weights due to disagreement between experts. Another method would derive uncertainties from the dispersion of weights across experts.

From expert best-guess weights and declared uncertainties (Table 2), a Monte Carlo simulation is carried out to compute a potential error for each location so that the suitability map is eventually accompanied with a confidence map.

The Monte Carlo simulation substitutes the deterministic (best-guess) weights from multiple random samples of the MCA weights. About 300 random MCA weights were generated from sampling the AHP pairs. In our case 300 was a reasonable choice after checking that several iterations of these 300 weights set led to less than 1% change of the mean and standard deviation of the AHP weights. In other case studies, one might need to increase the Monte Carlo sample as the number of criteria and dispersion values increase, but still within GIS computational restraints.

The analysis assumes that the AHP values generated are normally distributed around the AHP best-guess, used as the mean of the distribution, and have a standard deviation that equals the declared confidence. For example, SAC has a value of 3 relative to SPA and the confidence surrounding this judgement is determined as 1.25. 300 random numbers were generated for this pairwise comparison with a mean value of 3 and a standard deviation of 1.25. This process was similarly carried out for SAC with respect to GWV. The resulting random values were fed into the AHP process and used to generate 300 series of MCA weights, each series summing up to 100%, e.g SAC: 41%, SPA 52% and GWV 7% or SAC: 70%, SPA: 23% and GW 7%, and so on, overall reflecting both the experts' best-guess and experts uncertainty. Table 3 reports summary statistics over the 300 weights' series. Conversely to the best-guess weights from Table 2, the mean weights include the effect of the declared confidence values. The interplay of values and uncertainties decrease the average weight of SAC (compare 0.60 and 0.65 best-guess) and increases its variability with respect to the other two criteria. The sampled mean of weights would approximate best-guess weights if uncertainties were equal across the three criteria.

Table 3. Descriptive statistics of Monte Carlo simulated MCA weights.

	SAC (%)	SPA (%)	GWV (%)
Mean	60	31	9
Standard Deviation	11	8	5

After the Monte Carlo process, the spatial overlay is then repeated 300 times, resulting in 300 suitability maps.

4. Map design for suitability and uncertainty

Once the 300 weighted overlay maps were carried out using the Monte Carlo simulation, the mean, standard deviation and coefficient of variation were computed for each cell. Following Grandmont et al. (2012), the coefficient of variation (the standard deviation divided by the mean) was used to compare the relative spread of uncertainty in site suitability (see Figure 5 in the Main Map). Sites with a high coefficient of variation (darker) are the most uncertain whereas those with a low coefficient of variation (lighter) have the lowest uncertainty.

The last stage involves integrating Figures 4 and 5 in the Main Map, i.e. the suitability and the uncertainty maps. Using the 4 classes previously used along the two dimensions, the results are organized in tiers to obtain a new matrix. Table 4 outlines the results of this combination by reporting the number of cells occurring within each suitability class – uncertainty pair. Given that a highly discrete reclassification of the inputs was used (Table 1) some pairs have no occurrences, which is a desirable outcome for facilitating suitably sized areas for development.

Table 4. Occurrences of combined suitability and uncertainty classes (Classification method: Jenks).

Suitability Uncertainty	High Suitability	Medium High Suitability	Medium Low Suitability	Low Suitability
High Uncertainty	0	26	24,002	38,410
Medium High Uncertainty	3	7999	36,258	44,771
Medium Low Uncertainty	1978	44,601	34,725	14,369
Low Uncertainty	69,505	54,969	38,186	19,565

In order to obtain a visual representation of both suitability and uncertainty, the best guess map (Figure 4 in the Main Map) and the coefficient of variation map (Figure 5 in the Main Map) were combined to create the Main Map. Conversely to Grandmont et al. (2012) no value reclassification was used, but rather a combination of colour scheme and transparency. Grandmont proposed a framework where uncertainty and suitability are summed, leading to different suitability-uncertainty pairs resulting in the same visual outcome. Preference was given here to cartographic semiology to visualize the two dimensions over the summing procedure.

Hence, the methods consist of representing the land suitability map with varying degrees of transparency based on uncertainty, which is in line with the whitening effect proposed by Hengl (2003). The uncertainty map is reverted: darker values on Figure 5 in the Main Map are made more transparent on the Main Map, so that high uncertainty areas are made less visible and low uncertainty areas are not transparent (brighter) and thus visually emphasized.

Based on the three environmental criteria considered, it is found that the brightest green areas (town centre and far northwest of the map), which are the most suitable with the least uncertainty, should receive particular attention from the experts. Most areas in medium low suitability (orange coloured areas) fade out because of high uncertainty. Those parts of the study area displayed in red, draw attention to the high potentially negative impacts on the environment of developing in these areas.

5. Conclusion

The approach outlined herein follows on from Grandmont et al.’ (2012) approach by providing a method for evaluating uncertainty and displaying the results and allowing for variation in uncertainty between MCA factors. The research (following Grandmont's method), illustrates how confidence values can be used to allow analysts to generate weights for overlaying and to quantify the uncertainty in those weights. This allows for an assessment of the uncertainty that can exist when using the AHP process. The approach deals with uncertainty in the site suitability maps and in the factor weights that represent the best estimate of site suitability. The method presented allows for uncertainty in decisions surrounding site suitability to be measured and visualized on a map. The results from this study show that in practice it is possible to find the most suitable site for development with the least uncertainty. The research highlights the use of GIS techniques combined with MCA and AHP as a valuable tool for risk assessment in land-use planning.

This research measured robustness against the capacity of experts to weight different criteria with more or less confidence. There are other sources of uncertainty and errors in GIS suitability modelling, for which the robustness can be assessed and mapped in a similar way. The objective of this study was to present a method to consider uncertainty surrounding the judgement of the relative importance of factors being considered and provide for the visualization of that uncertainty. We have reduced the analyst's error by using a continuous classification of inputs, but, of course, this is still subject to other scaling possibilities. Future research will be dedicated especially to analysing and mapping the robustness of the methods against thematic aggregation and GIS reclassification procedures.

Software and data source

The following data sets were used in this study: (1) Ground Water Vulnerability (http://www.dcenr.gov.ie/Spatial+Data/Geological+Survey+of+Ireland/GSI+Spatial+Data+Downloads.htm); (2) Special Areas of Conservation and Special Protection Areas (www.dublinked.ie) and (3) OpenStreetmap background mapping. Esri ArgGIS v. 10.2 software was used to carry out the analysis and mapping. Microsoft Excel was used to carry out the AHP process and the Monte Carlo sampling.