Extending the use of scenario planning and MCDA for the evaluation of strategic options

Abstract

Multi-criteria decision analysis (MCDA) is well equipped to deal with conflicting, qualitative objectives when evaluating strategic options. Scenario planning provides a framework for confronting uncertainty, which MCDA lacks. Integration of these methods offers various advantages, yet its effective application in evaluating strategic options would benefit from scenarios that reflect a larger number of wide-ranging scenarios developed in a time-efficient manner, as well as incorporation of MCDA measures that inform within and across scenario comparison of options. The main contribution of this paper is to illustrate how a more diverse set of scenarios could be developed quickly, and to investigate how regret could be used to facilitate comparison of options. First, the reasons for these two areas of development are elaborated with respect to existing techniques. The impacts of applying the proposed method in practice are then assessed through a case study involving food security in Trinidad and Tobago. The paper concludes with a discussion of findings and areas for further research.

Keywords:

• decision analysis
• strategic planning
• scenario

after three revisions

The online version of this article is available Open Access

Introduction

Scenario planning (SP) is an extensively employed method to support strategic decision making through the development of a set of narratives called scenarios. Scenarios are challenging descriptions of futures that are relevant to a strategic decision and representative of plausible developments in the external world (De Geus, 1988; Van der Heijden, 1996). They are an invaluable tool for managers or strategists who want to think through the future dimension of decisions and actions. When combined with option planning (where all options are put forward on a neutral mode) and a clear, structured view of what is desirable, scenarios provide a coherent framework for evaluating strategic options (Wack, 1985a, 1985b). They may also emphasise the importance of developing strategic options so that the final choice is robust (ie capable of responding to a variety of changes in uncontrollable factors) (Roy, 1998, 2010; Pomerol, 2001).

The literature has proposed several ways of integrating scenarios with a Multi-Criteria Decision Analysis (MCDA) framework (Phillips, 1986; Stewart, 1997, 2005; Wright and Goodwin, 1999; Goodwin and Wright, 2001; Belton and Stewart, 2002; Montibeller et al, 2006). The integrated methodology provides a range of contexts within which to systematically consider the implications of trade-offs among multiple objectives. However, there is scope for further development of the methodology with respect to developing scenarios that reflect a larger number of wide-ranging scenarios in a time-efficient manner (Klayman and Schoemaker, 1993; Godet and Roubelat, 1996), and MCDA measures that inform within and across scenario comparison of options (Durbach and Stewart, 2003). This paper explores the former by suggesting a method for developing scenarios using a combinatorial set of key uncertainties, each of which may take a small number of different levels. It addresses the latter by proposing the use of cost-equivalent regret (Keeney, 1992; Lempert et al, 2006). The benefits and drawbacks of applying these in practice will be investigated through a case study involving food security in Trinidad and Tobago.

The paper is organised into three parts. First, the case for the proposed method in light of existing techniques is elaborated. This is followed by a practical illustration of the method using food security in Trinidad and Tobago. The paper then concludes with a discussion of findings and scope for further development of the method.

Applications of SP to the evaluation of strategic options

Scenarios can be used to help the decision maker develop a better understanding of the complex relationships among uncertainties, objectives and strategic options, which are core components in the evaluation of strategic options (Wright and Goodwin, 1999; Goodwin and Wright, 2001; Stewart, 2005). They can direct attention to critical issues and uncertainties, and help define strategic priorities when multiple objectives exist. Scenarios also provide a platform for creating, testing and refining strategic options. To this end, they may highlight potential strengths and weaknesses of options, or provide insights on how to increase the robustness of options.

The multiple uses of scenarios imply that evaluation techniques that seek to integrate them should be capable of:

• Incorporating subjective judgments and dealing with multiple, conflicting objectives—Scenarios are a combination of analysis and judgment about future possibilities (Schoemaker, 1991). An evaluation technique should therefore reflect this, while taking into account the existence of multiple preferences. The literature supports the use of quantitative analysis of how scenarios perform under a set of pre-defined strategic options (Leemhuis, 1985; Huss and Honton, 1987; Godet and Roubelat, 1996; Morgan et al, 1999; Wollenberg et al, 2000; Chermack, 2004; Stewart, 2005) over flexible qualitative descriptions (eg Likert scales).

• Achieving a diverse set of scenario themes quickly—Robustness implies that it is meaningful to consider scenarios that cover a diversity of possibilities. The scenario narrative is typically developed by developing storylines of how the future might unfold from the present to four end states defined by the upper and lower bounds of two key uncertainties (Schwartz and Ogilvy, 1998). Development of the narrative is itself time-consuming, which is a disadvantage when this is only one constituent of the evaluation process.

• Within and across scenario comparison—In order to provide insights for the development and selection of a robust option, a scale that allows comparison must be developed, accompanied by informative visual displays.

MCDA and SP for the evaluation of strategic options

The combined use of MCDA and SP provides a range of advantages with respect to the above implications. Firstly, MCDA is well-equipped to deal with objectives that are difficult to quantify, conflicting and hard to compare. Scenarios provide a framework for confronting uncertainty, which MCDA lacks. Secondly, it may provide a good balance between the analytic and intuitive components of decision making, as well as between the roles of analyst and manager (Schoemaker, 1991). Finally, an integration of the methods that allows for within and across case comparison can provide a documented rationale for a particular choice, or a shortlist of options supported by an elaboration on the conditions in which they perform best (Roy, 1998; Pomerol, 2001).

Table 1 summarises the literature that involves integration of SP and MCDA to date. It assumes a set of scenarios Y={y₁, …, y_s}; a set of strategic options A={a₁, …, a_m} and a set of criteria (measures by which the achievement of a particular objective is gauged) C={c₁, …, c_n}. An MCDA analysis that incorporates scenarios involves elicitation of (a) v_kir—how an option a_k will be perceived to perform in a given scenario y_r with respect to a criterion c_i (value) and (b) w_ir—how important a certain criterion c_i is relative to another criterion c_j in scenario y_r (weight).

Table 1 Summary of MCDA and scenario planning interventions to date

Reference	Number of scenarios	Approach to scenario construction	Method of elicitation		Recommendations for applying MCDA measure ie Σj(vkir·wir)
			Weights (wir)	Values (vkir)
Goodwin and Wright (2001)	2 ‘extreme world’ scenarios; no probabilities attached	Extreme world narratives created by putting all negatively and positively resolved uncertainties in separate scenarios and then checking for internal consistency	• Compare 0–100 for each criterion ci	• Allocate a score of 100 to the best strategy–scenario combination and 0 to the worst in terms of its performance under each criterion	• Use scenario matrix to establish dominance of strategies across scenarios
			• Assign 100 to the most important criterion and compare it with the relative importance of other swings on a 0–100 scale	• Allocate scores between 0 and 100 to represent the improvement in criterion ci that the application of strategy ak may be perceived to bring in scenario yr for all yrɛY	• Perform sensitivity analysis
			• Normalise scores obtained		• Evaluate risk by including a risk criterion in the MCDA model
Belton and Stewart (2002)	2 scenarios	Scenario narratives are external to the organisation (ie, arise from events outside the organisation's control) and are value-free	Different weights assigned for each scenario using approach as in Goodwin and Wright (2001)	Best and worst performances against each criterion across all scenarios define upper and lower limits of the value scale	• Select option that is robust against uncertainties (a direct holistic judgment)
				Allocate scores between 0 and 100 to represent the improvement in criterion ci that the application of strategy ak may be perceived to bring in scenario yr	• Assign relative weights to scenarios and calculate a one-dimensional performance value for each alternative by multiplying the weight of each scenario by MCDA measure
					• Select option that maximises the worst aggregate performance given by the sum of all MCDA measures over all scenarios. This however ignores trade-offs between performances
Montibeller et al (2006)	2–3 scenarios	Narratives based on best; worst and extrapolation of present trends/themed according to one key uncertainty; no probabilities attached	As in Goodwin and Wright (2001) but may assign a different set of weights for each scenario	Focus on a single scenario when expressing preferences as in Belton and Stewart (2002)	• Identify options that are nearer to the ideal performance (100) and that have stable performances close to the ideal
					• Evaluate risk by calculating inter-scenario risk, defined by assessing spread of performances for each option in each scenario
					• Remove dominated strategies if uncertainty is low; otherwise perform sensitivity analysis
Stewart (2005)	Suggests 3–5 scenarios may be sufficient	Unspecified	Treat all criteria–scenario combinations as ‘meta-criteria’	Assumes intensity of preference for different increments in performance on any one criterion may differ from scenario to scenario. Guidance not explicitly provided on elicitation	• Outranking to generate classification into preferences classes
					• Goal programming to measure achievement in terms of distance from a goal or reference level
Phillips (1986)	3 scenarios	High, medium and low based on demand uncertainty, which corresponds to optimistic and pessimistic considerations; probabilities attached	Relative weights elicited which reflect the importance of moving from 0 to 100 on each criterion scale. Weights can also represent the probability of each scenario	As in Belton and Stewart (2002)	• Find differences among best strategies in each scenario and find ways of managing them through further analysis or information

Examination of Table 1 highlights two areas for further development of the combined methodology. Firstly, SP and MCDA interventions have typically involved the use of optimistic, pessimistic and most-likely scenarios. Developing such scenarios goes against the generally accepted view in the scenario literature (Wack, 1985a, 1985b; Schoemaker, 1991; Ringland 1998; Schwartz and Ogilvy, 1998; De Geus, 1999). Several other relevant possibilities are undermined due to a dominance of value-laden notions or assumptions of likelihood, both of which defeat the underlying philosophy of scenarios. Even so, one main disadvantage of SP is the length of time taken to develop scenarios (Mietzner and Reger, 2005). One way to address the time-consuming nature of developing the narrative and difficulty in selecting two uncertainties is to apply a Morphological Analysis (MA) approach. This is based on a combinatorial set of uncertainties, each of which may take a small number (2–5) of different levels (Eden and Ackermann, 1998; Ritchey, 2006).

Secondly, MCDA measures resulting from each option–scenario combination should be compared within and across scenarios (Durbach and Stewart, 2003). Yet, Table 1 shows that some SP and MCDA interventions have employed the use of weights or probabilities to aggregate MCDA measures over all scenarios. This fails to achieve proper integration between the methods for two reasons. Firstly, it violates a key assumption of SP that every future is equally likely (Rosenhead et al, 1972). Secondly, aggregating MCDA measures runs the risk of diluting the rich information derived from the process. These include details on the differences between scenarios that favour one option over another, or scenarios which particular options may perform poorly (Dias, 2006). Selecting those options with stable performances close to the ideal, or assessing the spread of performances for each option in each scenario (Montibeller et al, 2006) provide better comparison. These can be contrasted with the concept of regret, which compares the performance of an option with the maximum achievable performance across all strategic options in that scenario (Lempert et al, 2006). Regret therefore makes explicit use of the information provided by the decision maker, rather than the possible illusion of an ideal world that may never be achievable. However, the concept of regret has not been applied to SP and MCDA interventions.

Method and illustration

This section proposes a MA approach for creating a more diverse cohort of scenarios for evaluating strategic options under the MCDA framework (see Figure 1). It also proposes the use of cost equivalent regret to facilitate within and across scenario comparisons. The method will then be applied practically to identify benefits and challenges of the proposed method. For the practical application, a case study will be used. The case study has been selected as an appropriate research strategy because many uncontrollable variables are involved, and the aim of the research is to explore how the method behaves in a practical setting (Yin, 2008). The case study will be based on the issue of the future of food security in Trinidad and Tobago. A policy context has been chosen because it represents a unique but equally critical and relevant application of the SP and MCDA method compared to traditional business applications. This issue also reflects characteristics of a problem to which the proposed method would be suited, namely:

• The issue implies the existence of long-term consequences that are not known deterministically, but for which provisions must be made in the present to achieve core objectives or mitigate adverse effects.

• The cost criterion is an important consideration in the decision-making process.

• Factors affecting the decision are difficult to quantify, and involve conflicting objectives.

Figure 1 The six steps in the proposed method.

In what follows, the theoretical description of each step is presented together with a rationale for it. This is followed by the corresponding practical implementation of each step.

Step One—Define the strategic question of interest

An appreciation of the context helps define the issue and the time frame within which it is to be considered. In 2002, in pursuance of the Vision 2020 development goals, the government of Trinidad and Tobago outlined the promotion and enhancement of agriculture as a pillar of the national development and diversification of an economy traditionally based on oil and gas (Vision 2020 Operational Plan 2007–2010).

The majority of the country's agricultural resources have traditionally focused on producing export commodities such as sugar, cocoa, coffee and citrus. Under favourable marketing arrangements that assured a ready market and relatively stable prices, export agriculture was profitable (Sector Policy for Food Production and Marine Resources, 2001). However, as international trade regulations (eg food safety standards) became increasingly unfavourable, the contribution of agriculture to national GDP showed a declining trend over time. Local production of staple food items (eg wheat, corn) became increasingly uncompetitive. This meant that average incomes in the agricultural sector were the lowest in the country, and the share of the labour force in agriculture, particularly among younger age groups, was on the decline. However, the motivation to pursue agricultural initiatives weakened as steadily increasing oil revenues, post-2002, were used to fuel a high level of food imports. Thus, growth in the oil and gas sectors resulted in the majority of arable land being traded off for infrastructure development and manufacturing industries. However, the unforeseen fall of oil prices in late 2008 and the steady rise of food prices emphasised the significance of developing an agricultural sector that could consistently provide nutritionally adequate food to its citizens on a sustainable basis. These circumstances provoked the question of which investments were likely to be the most favourable for the country in terms of food security given changes in the regulatory, economic, technological and social environment.

A time frame of 8 years was chosen. This coincided with the election due to take place in 2017. Although the goal of food security does not have to be met by 2020, significant progress towards the goal must be shown by the election year. The current context also stresses the political and economic imperative to make provisions in the present to mitigate further adverse effects. An expert in the field of agriculture was deemed a suitable interviewee.

Step Two—Identify Key Uncertainties and Trends

Key uncertainties are events whose outcomes are uncertain but will significantly affect the issue of concern (Schoemaker, 1995). Trends that can plausibly affect the issue under consideration in constructing scenarios were also included since this is consistent with standard SP formats (Schoemaker, 1991; Van der Heijden, 1996; De Geus, 1999).

In the case study, the traditional approach of brainstorming and then plotting uncertainties on a two-dimensional grid to highlight the most uncertain and most critical uncertainties was used. The most critical uncertainties selected for the development of scenarios were:

• Severity of natural disasters (flood, earthquake, hurricane, drought)

• Regulation in supplier countries

• State of global economy

• Consumer demand for safe foods

• Cost of farming inputs (eg fertilizer, pesticide, land)

Trends expected to continue were:

• High imports from other countries in the Caribbean region of produce such as bananas and ground provisions.

• Competition from other sectors (eg manufacturing and tourism).

• Traditional small farming as a means of livelihood in rural areas.

• Population will increase to 1.5 million by 2017.

Step Three—Identify Criteria and Strategic Options

In keeping with the philosophy of MCDA, the criteria pertinent to end objectives were considered. To derive this, several iterations of the question ‘Why is this measure of success important?’ were made. The criteria corresponding to the objective deemed important in absolute terms (ie its achievement did not imply/aid achievement of some other objective) were listed. This mode of questioning embodied the spirit of the strategic options development and analysis methodology (Eden and Ackermann, 1998).

For the case study, the expert defined food security as the ability of the country to consistently provide nutritionally adequate food to its citizens on a sustainable basis. This implied the following key criteria:

• Quantity of food available—A good proxy for this is the balance of payments account for food (indicates whether net exports exceed net imports). This is influenced by per capita income, cost of food, consumption of homegrown foods and the protection of the agro ecosystem for future generations.

• Quality of food—This refers to the extent to which food available contains basic nutritional value.

• Cost of implementing strategic option—The importance of this criterion is highlighted in the case where two options yield the same quantity and quality of food.

A set of independent options was obtained by considering the option currently being used as well as the main ones under consideration for the future. The discussion about options with the interviewee yielded the following set of strategic options:

• (A) Reduce cost of farming to subsistence farmers (greater access to loans, subsidies and modern technology).

• (B) Provide basic infrastructure for farming (eg land tenure, road access, water access).

• (C) Exclude valuable agricultural land from areas identified as development areas, and exclude areas of high biodiversity from being used for agricultural purposes.

• (D) Promote a positive profile of the agricultural sector, especially towards youths.

• (E) Mega-farm production of higher value local commodities. This approach involves using natural means to grow multiple crops on a large scale. It aims to exploit niche export markets and increase the competitiveness of local alternatives to imported items. This is the option currently being pursued most vigorously by the government.

The ‘Do-Nothing’ option was not considered because it was felt that it was unrealistic and might destroy key implications of the scenarios for strategy. Owing to budget constraints, only one option could be implemented.

Step Four—Develop Scenarios

The development of scenarios in this paper is supported by Morphological Analysis (MA). MA is a method for structuring and analysing multi-dimensional technical, social and political problem complexes where quantification is difficult (Ritchey, 2006). It is based around representations of the objects of interest through sets of variables each of which can take a range of possible states, conventionally represented as a table. As a structuring tool, a key element of MA is checking the consistency (assessing relationships between variables; and that trends are compatible within the time frame (Van der Heijden, 1996)) of these various states in order to avoid a combinatorial explosion in the number of possible configurations. Similar ideas have been invented independently multiple times in the OR literature—for example in strategic choice approach (Friend, 1989) and in the strategy table of Howard (Howard, 1988).

MA is well-suited to scenario development for formal evaluation for three reasons. Firstly, MA can deal with a larger number of uncertainties that are qualitatively and quantitatively defined. Scenarios are best suited for highly complex, uncertain situations where many factors are unquantifiable and virtually every factor is variable, and so there are a large number of variables (Coyle, 2007; Millett, 1988). Secondly, MA encourages the investigation of multiple combinations of extreme boundary values in an efficient manner (Ritchey, 2006). This is very much within the philosophy of SP to provide challenging views of the future. Thirdly, MA can facilitate the description of scenarios to the level of detail that provides the decision maker with enough information for preferences to be elicited (Schoemaker, 1991; Stewart and Scott, 1995; Eden and Ackerman, 1998). This is achieved through the coherent construction of parameter spaces linked by way of logical relationships (Ritchey, 2006).

The steps involved in this stage are:

• Define the limits of each uncertainty and then establish 2–3 intermediate conditions. In determining the limits, the following steps are useful:

  • Consider the best and worst possible achievement levels for the criteria that can be perceived.

  • Extend these further but keeping within plausible levels, and try to envisage what uncertainty levels might lead to that situation. These levels define the limits.

  • Combinations of conditions (one condition from each uncertainty) represent different scenarios.

• Test for consistency—This mode of scenario construction assumes that scenarios are merely a snapshot in time. Consequently, testing for consistency was necessary. The authors support the view that only those relationships which the decision maker is certain about should be included. Therefore, in testing for consistency, any well-established relationships among the set of uncertainties were noted.

In the context of the case study, a spectrum of discrete values or conditions which the uncertainty can take is shown in Table 2. The best case scenario is denoted by BBBBB (ie combination of best/most preferable level of each uncertainty) and represents low severity of natural disasters; supportive regulation in supplier countries; positive growth in the global economy; high consumer demand for safe foods and low cost of farming inputs. Similarly, the scenario denoted by WBBBB holds all conditions as in BBBBB except for the severity of natural disasters, which is high.

Table 2 Key uncertainties and the spectrum of possible discrete values

	Severity of natural disasters	Regulation in supplier countries	State of global economy	Consumer demand for safe foods	Cost of farming inputs
Most preferable (best)	Low	Supportive	Positive growth	High	Low
Least preferable (worst)	High	Neutral	Stagnation	Low	Moderate
		Restrictive	Negative growth		High

There are theoretically 108 possible scenario combinations (2 × 3 × 3 × 2 × 3=108). However, Table 3 only uses 12 of these combinations. Both sets of swings in uncertainties were considered (ie assume all uncertainties at their best (worst) level and observe a swing of each uncertainty in turn to its worst (best) level). Consideration of both swings helped achieve some balance to the extent that if a picture dominated by favourable states was considered, then the trade-offs to be made in bad times would be overlooked. Similarly, sole focus on unfavourable scenarios would not offer a good reference for seizing new opportunities.

Table 3 Elicited value for how strategies are perceived to perform with respect to each criterion in various scenarios

vkir	Scenario
	BBBBB	WBBBB	BWBBB	BBWBB	BBBWB	BBBBW	WWWWW	BWWWW	WBWWW	WWBWW	WWWBW	WWWWB
Quantity (C1)
A	0	100	70	25	100	100	40	100	90	0	100	0
B	100	75	100	100	75	90	60	90	80	50	75	80
C	60	50	60	40	0	50	80	80	0	70	50	60
D	80	0	70	50	85	60	100	40	75	100	0	70
E	95	70	0	0	80	0	0	0	100	80	90	100
Quality (C2)
A	0	100	100	60	100	100	80	90	100	75	80	0
B	85	95	70	100	80	90	60	75	60	70	100	70
C	100	90	60	60	70	70	100	100	25	75	0	100
D	60	0	75	70	80	0	95	0	30	0	75	65
E	90	80	0	0	0	80	0	60	90	100	65	75
Cost (C3)
A	75	100	60	30	100	25	15	0	25	100	10	25
B	90	60	0	100	0	0	30	65	40	65	30	40
C	100	10	100	70	95	100	90	90	85	90	95	85
D	95	50	80	75	90	70	100	100	100	85	100	100
E	0	0	50	0	60	40	0	40	0	0	0	0

This approach to reducing the number of scenarios has two justifications. Firstly, it does not violate the philosophy of SP. Scenarios should be relevant to the concerns of the decision maker; describe generically different futures; and represent states in which the system might exist for some length of time (Schoemaker, 1995). Secondly, the idea of using swings in uncertainties is similar to the use of swing weights in the MCDA framework.

Although the scenarios in the proposed method make use of best and worst-case notions, they were merely used to broaden the interviewee's thinking on a range of plausible uncertainty levels. The best and worst labels have been used in the paper in an illustrative manner to highlight the pattern in the combinations chosen. The interviewee was shown Table 2, but the scenario was outlined to him in a brief narrative format that included trends (see Figure 2) to give a more comprehensive picture.

Figure 2 Sample scenario narrative for BBBBB.

Step Five—Apply the MCDA framework to each Scenario

This step aims to measure how each strategic option performs under a given scenario. The performance of option k under scenario r [denoted Performance (a_k, y_r)] using the MCDA framework is given aswhere v_kir is the value of option k in terms of helping to achieve a desired level of criterion i in scenario r and w_ir is the weight assigned to criterion i in scenario r.

In order to calculate v_kir two questions were posed to the decision maker. Firstly, he was asked ‘Given scenario BBBBB, which strategic option do you think will perform best relative to the other options in terms of the extent to which it will help achieve a desirable level on C1 (quantity)?’ Options were then ranked, with a value of 100 being assigned to the option ranked best and 0 to the option ranked worst. Values for other options on each criterion were assigned relative to this question of value. This prompted the second question—‘How do you think option A will perform in scenario BBBBB relative to the other options in terms of the extent to which it will help achieve a desirable level on C1?’ For example, in scenario BBBBB, rank 1 went to option B and rank 5 went to strategy A. B got a score of 100 and A, a score of 0. A value of 60 assigned to option C meant that the improvement in quantity from using option C over option A was roughly 60% as attractive as the improvement in quantity from using option B over A.

Responses to these questions are shown in Table 3, with Table 4 providing an added illustration of what the values translate to on the scale for each criterion.

Table 4 Estimation of what elicited values translate to on respective criteria scales

	BBBBB			WWWWW
	Quantity* (in US$)	Quality	Cost (in US$)	Quantity (in US$)	Quality	Cost (in US$)
A	−550 million	45% of RDI	330 million	−400 million	35% of RDI	250 million
B	200 million	65% of RDI^†	25 million	−480 million	27% of RDI	20 million
C	−300 million	85% of RDI	1 million	−550 million	50% of RDI	1.5 million
D	−200 million	57% of RDI	3 million	−300 million	40% of RDI	2 million
E	150 million	75% of RDI	1 billion	−700 million	25% of RDI	950 million

*Balance of payments for food=total imports−total exports. The negative (positive) value represents the amount of reduction (increase) in the balance of payments from its current value.

^†RDI—recommended daily intake of basic nutrients per capita.

Eliciting w_ik involved the use of swing weighting. Swing weighting explicitly requires the decision maker to consider the relative value between the most and least preferred levels of two criteria (Goodwin and Wright, 2001). The question asked to elicit weights was—‘If you were in scenario A, and one criterion could be moved to its best level, which would you choose?’ The criterion ranked first received a score of 100, and the other criteria were given a weight relative to this score. Weights were then normalised (Table 5).

Table 5 Normalised criterion weights for scenarios

Wik	BBBBB	WBBBB	BWBBB	BBWBB	BBBWB	BBBBW	WWWWW	BWWWW	WBWWW	WWBWW	WWWBW	WWWWB
Quality	0.255	0.097	0.385	0.156	0.217	0.269	0.204	0.296	0.264	0.143	0.275	0.235
Quantity	0.426	0.645	0.288	0.4	0.435	0.346	0.388	0.334	0.358	0.476	0.333	0.373
Cost	0.319	0.258	0.327	0.444	0.348	0.385	0.408	0.370	0.377	0.381	0.392	0.392

The use of a standard set of weights across scenarios (Goodwin and Wright, 2001) was not adopted because it was felt that eliciting swing weights, given a specific scenario, was more compatible with examining implications of a scenario for strategy (Parnell et al, 1999; Belton and Stewart, 2002; Durbach and Stewart, 2003; Montibeller et al, 2006). The performance of each strategic option under each scenario is shown in Table 6, with the best performance for each scenario highlighted in bold.

Table 6 Overall performance of how strategies perform under various scenarios

Performance Σj(vkir·wir)	BBBBB	WBBBB	BWBBB	BBWBB	BBBWB	BBBBW	WWWWW	BWWWW	WBWWW	WWBWW	WWWBW	WWWWB
A	23.9	100	78.3	32.7	100	71.1	38	60	68	48.8	59.2	9.8
B	93	73.1	55.8	100	50	55.4	47.8	76.3	59.6	58.6	64.2	62
C	83	43.6	73.1	56.4	48.3	74.6	88.2	89.6	38.7	78.3	53.9	79.2
D	79.7	12.9	75.2	64.2	85.7	47.7	99	50.4	72.5	80	59.8	80.6
E	63.4	52.9	16.4	0	55.7	36.9	0	32.6	59.6	52.4	47.9	54.9

The direct elicitation approach in this intervention offers two main advantages when compared with indirect assessment techniques. Firstly, it is consistent with value elicitation in other SP and MCDA interventions to date (see Table 1). Secondly, it facilitates greater integration between the methods since it reinforces the concept of separate evaluations under each scenario while encouraging discussion about the impact of future events on choices in a useful and engaging manner (Belton and Stewart, 2002).

Step Six—Calculate Regret as a measure of Robustness

The regret of a strategic option is defined as the difference between the performance of an option in some future state of the world, given some performance function, and that of what would be the best-performing option in that same future state (Lempert et al, 2006). In other words, if A is the set of options and Y is the set of scenarios, the regret of option a_k, a_kɛA, in scenario y_rɛY, using value v is given aswhere strategic option a′ indexes through all options to determine the one with optimal performance in scenario y_r.

A robust option can be defined as one with relatively small regret compared to the alternatives across a wide range of plausible futures considered (Lempert et al, 2006). A regret-based definition of robustness is used for three main reasons:

• Regret focuses attention on those states of the world in which alternative options have significantly different outcomes (Lempert et al, 2006). The architecture of the set of scenarios, Y, targets a range of these significantly different states.

• The measure explicitly anticipates the emotion of regret when evaluating different options in an effort to make the consequences of choice more salient. This can serve to induce greater deliberation among choices (Zeelenberg, 1999).

• It complements the philosophy of the proposed method since it does not employ the use of probabilities, nor does it recommend the elimination of strategic options through dominance.

Regret thus represents the loss in value relative to the best option, measured on a scenario specific scale, defined by the joint lower and joint upper levels of performance of the options under that scenario.

In order to properly gauge the robustness of an option, the regret values had to be converted to a comparable scale. To achieve this, a cost-equivalent model was proposed (Keeney, 1992). In the case study, scenario BBBBB for example had a cost range of US$949 million (US$950 million–US$1 million), and the range of evaluation units was (100 × 0.319=31.9), each evaluation unit was equivalent to US$29.75 million. In other words, the marginal monetary worth for the overall (i.e. US$949m/31.9) scale was US$29.75 million. To achieve a worthiness equivalent value for option B under this scenario, the overall performance/evaluation figure was multiplied by the marginal monetary worth coefficient before applying the regret calculation. This procedure was repeated for each option–scenario combination, and the results shown in Table 7.

Table 7 Worth-equivalent regret values for each strategy–scenario combination

Cost equivalent regret (US$million)	BBBBB	WBBBB	BWBBB	BBWBB	BBBWB	BBBBW	WWWWW	BWWWW	WBWWW	WWBWW	WWWBW	WWWWB
A	2054.5	0	0	1438.9	0	86.4	1419.3	758.7	111.4	776.1	121.4	1713.6
B	0	990.6	653.9	0	1363.9	475.2	1191.4	341.4	325.0	533.3	0	450.7
C	298.2	2076	150.9	931	1411.2	0	251.7	0	851.5	41.1	250.4	33.5
D	395.7	3203.8	89.5	764.8	391.2	663.6	0	1007	0	0	106.8	0.0
E	879.5	1732.1	1797.3	2137.4	1208.6	929.5	2302.3	1463.5	325.0	687.6	396.8	621.2

The worth equivalent regret values for each strategic option across each scenario are plotted in Figure 3. The most robust option would ideally have a high frequency of low or zero regrets, and have a low spread of values relative to this point. At a first glance, option E always incurs some regret, and has the highest spread of regret values across scenarios. It can be concluded therefore that this option is not robust. Further examination of regret values under this option highlights that this could be due to the heavy dependence of its success on supportive regulation in supplier countries. Options B (provide basic infrastructure for farming) and D (promote a positive profile of the agricultural sector, especially towards youths) appear more worthy candidates. Option B has the lowest overall spread of regret values. Option D would perform better if mechanisms could be included that minimise its highest regret, which occurs in scenario WBBBB, when the severity of natural disaster is high. Such a mechanism might include the development of a comprehensive disaster preparedness plan. This is particularly interesting as it highlights the importance of building capabilities for food security should a sudden disruption in food supply occur, which is precisely what has not been pursued in the status quo. One similarity between options B and D that makes them more successful than other options is their orientation towards empowerment and self-sufficiency. These findings can provide a basis for further discussion of options that are compatible with this theme. Regret values may then be recalculated, but it must be borne in mind that regret depends on a given set of strategic options, and so the regret value may change as new options are added or existing ones deleted (French, 1986).

Figure 3 Plot of cost equivalent regret values.

A number of assumptions underpin the approach outlined here:

1. There are generally preferred directions of movement for criterion that hold no matter what other criteria values are (eg higher quality of food) (Keeney and Raiffa, 1993).

2. The set of criteria are preferentially independent (Keeney, 1992).

3. The decision maker is able to provide the judgments required by the method.

4. The regret measure is valid as a means of assessing robustness.

5. Examining best–worst (worst–best) swings in uncertainties help provide meaningful information on how changes in the environment affect preferences.

6. Attribute functions are linear and cost is a significant attribute in the value model (Keeney and von Winterfeldt, 2007).

Discussion

The proposed method explores ways to achieve a diverse set of scenario themes quickly and facilitate within and across scenario comparison of options, while being able to handle subjective judgments for multiple objectives under uncertainty. The implementation of the proposed method also highlights scope for improvement in these areas.

The proposed method developed scenarios based on swings in extreme possibilities of a set of five uncertainties. The interviewee felt that this approach tried to capture many factors that should be considered in making a decision and helped him to focus on prioritising items to achieve ends objectives with limited financial resources. The proposed method of achieving a diverse set of scenarios to form a basis for measuring robustness can be contrasted with scenario narratives constructed around the impacts of decisions (Schoemaker, 1991; Stewart and Scott, 1995); variations of parameters of a system model, which are mathematically defined (Tietje, 2005); varying perspectives of a desirable future (Gordon, 2008); or minor variations to one or more emergent conditions such that evaluation in accordance with stakeholder concerns is permitted (Karvetski et al, 2010).

The interviewee found that the level of detail was sufficient for eliciting the required answers, even though the scenario presented was an outline of a future point in time (Schoemaker, 1991), and not a storyline of how the future might unfold from the present to an end state (Van der Heijden, 1996; De Geus, 1999). In making value and weight judgments, he was prompted to consider how choosing an option now might plausibly behave in a particular scenario with respect to each criterion. He therefore acknowledged that in-depth knowledge of both the technical aspects of the problem and the decision-making instrument were required in providing judgments.

He felt that applying the method in a group decision-making process would have been more useful, as it would have provided a basis for debate and validation of opinions with respect to criteria. However, the repetition of weight and value elicitation questions was perceived by the interviewee as time-consuming and inconvenient, especially after about the seventh scenario. One way of addressing this issue is based on adapting the framework of the swing weighting method for recalibration of a baseline value function, following incremental adjustment of the baseline (Karvetski et al, 2010). While this may reduce the time and effort needed for elicitation, an anchor and adjust strategy may fail to encourage the decision maker to explore generically different futures that challenge the status quo, which lies at the heart of SP philosophy. In addition, this method would not be applicable in cases like the one presented here, where scenarios alter not only how the decision maker forms his/her preferences across criteria, but also how he/she perceives each option will perform.

With respect to within and across scenario comparison of options, the interviewee felt that visualisation of regret measures helped to crystallise the purpose of the exercise. He thought that cost-equivalent measures were also useful given a circumstance of financial constraints. The main advantage of the illustrative display used in this paper is that it allows the decision maker to see how much better/worse an option performs compared to another. Since a single MCDA model is created for each scenario (Montibeller et al, 2006), within-scenario value functions are not commensurable. Comparison of performance across scenarios is therefore facilitated only through the use of cost-equivalent figures. This can be contrasted with ranking (Karvetski et al, 2010), which does not provide such visualisation; and box plot displays (Lempert et al, 2006) with cumulative frequency percentage charts (Bertsch, 2008), which provide information concerning how often each performance measure occurs. Stacked bar charts, cobweb diagrams (Karvetski et al, 2010) and value paths (Schilling et al, 1983) would also provide practical displays to identify the most important and sensitive criteria across scenarios; as well as those criteria with the largest potential for relative improvement.

Application of the proposed method highlights three main challenges. The first is reducing the demand on decision makers for elicitation of weights and values. The second relates to extending the method to formally incorporate group decision making, which would imply consideration of a range of values and weights that reflect multiple perspectives. The development of a common model may also be possible, assuming communication among stakeholders is desirable (Belton and Stewart, 2002). Applying the method in a group situation would inevitably be very time-consuming and likely to require software support (Wollenberg et al, 2000). The third challenge relates to the incorporation of new options that may develop as a result of the evaluation process. Given that direct elicitation was used, including any new options to test whether they do improve robustness may necessitate the redefinition of scales (Belton and Stewart, 2002).

Conclusions and directions for further research

The main contribution of this paper was to illustrate how a more diverse set of scenarios that copes well with qualitative and quantitatively defined variables could be developed quickly; and to investigate how regret could be applied to MCDA measures facilitate within and across scenario comparisons.

The structure of the method was influenced by three main assumptions adopted by the authors, but arising from examination of the relevant literature. The first assumption is that the combined use of SP and MCDA is beneficial when considering the evaluation of strategic options. The second is that scenarios are intended to be challenging descriptions of futures that are relevant to a strategic decision and representative of plausible developments in the external world (Van der Heijden, 1996). However, the use of scenarios for the evaluation of options has often involved the consideration of optimistic, pessimistic and most-likely scenarios. These are limited in their capacity to provide a representative range of variation that could occur and also goes against the generally accepted view in the SP literature. The third is that MCDA measures should not be aggregated over scenarios through the use of weights or probabilities since it contravenes the philosophies of both methods.

Practical benefits from applying the method included a greater awareness by the interviewee of interactions among key components of a strategic decision; a purposeful display of measures to facilitate comparison of options anchored in cost considerations; and an interest in applying the method to a group decision-making process. The findings in this paper are nonetheless tentative. They have only been based on a single case, and more will be needed to confirm them. From a theoretical perspective, the method sought to stimulate investigation of how many scenarios are sufficient for use in evaluating options, and on the level of detail appropriate for using scenarios to evaluate strategic options, about which there remains a lack of literature and evidence from practice.

The paper suggests various directions for further research. Firstly, there is scope for investigating whether a more diverse set of scenarios with its increased elicitation burden justifies the loss of detail in the traditional narrative format in terms of quality and time taken. This prompts questions on how the demand on decision makers for elicitation of weights and values can be reduced, and on how effective best–worst swings are in scenario selection. It may be possible to achieve this by using incomplete information about preferences with software support. Secondly, how this method may be adapted to accommodate multiple perspectives in an interactive group decision-making process remains an open question. Thirdly, the use of regret as a meaningful measure of robustness in such interventions also deserves further exploration. Finally, this paper has focused on the selection of robust options. However, assessments on whether options are flexible (ie option can easily transform to accommodate new conditions) (Mingers and Rosenhead, 2001) and diversified (ie facilitates investment in a range of different areas relative to the organisation's current major offering(s)) (Wright and Goodwin, 2009) may improve the quality of options entering the evaluation process.

Acknowledgements

The authors wish to thank the agricultural expert for his willingness to provide information for the case study and for his valuable comments on the method applied.