Playing the new devil’s advocate role in facilitated modelling processes to address group homogeneity

Figures & data

Table 1. Comparison between Vennix (1996) adapted and the New Devil’s Advocate.

	Vennix’ Devil’s Advocate	New Devil’s Advocate
Type	Script	Role
Performers	Subgroup of participants	Subgroup of facilitators
Task	Subgroup challenges the assumption and ideas adopted so far	The NDA impersonates the missing stakeholders
Preparation	None	Analysis of the missing stakeholders and information collection
Facilitation requirements	At least one facilitator	At least two facilitators (one acting as NDA)

Figure 1. Project timeline in which the case study was conducted (adapted from Cappuccio et al., 2017).

Table 2. Sample of the role-plot table with the information provided to the facilitators playing the New Devil’s Advocate.

Issue to raise	Stakeholder bearer of that issue	How to frame it	Possible variables related to	Source
Existing dwellings are likely to represent 70–80% of the 2050 stock. 30% are going to be new houses. In 2016, 140,660 houses were completed. Legislation “Building Regulations and associated technical guidance”, regulation called “Zero carbon emissions” that have been stopped	Constructors / other policymakers	“At the moment, new houses are not relevant in the model. It is a stock outside the core structure. But I think that if we adopt a long-term view, they can be important and therefore valuable to be taken into account.” Are there any legislative requirements for energy efficiency and GHG emissions from new buildings?	“New building rate”, “demolition rate”	(Shrubsole et al., 2014), (Government of United Kingdom, 2017)
Top renovation payments methods: 1) Savings/personal finances 85% 2) Cash from home mortgage refinance 14% 3) Credit card—to be paid off over time 14% 4) Gift/inheritance 11% 5) Personal loan from friends/family 4%	Renovation expert	From what I know, only 14% of the people that are renovating choose to pay with money from home mortgage refinance. Is this data realistic according to you? Do you agree that knowing the magnitude of people paying in this way is important?	“Percentage of people using mortgage refinance for paying for renovations”	(Houzz & Home, 2016)
People in the UK are reluctant to take loans and mortgages	Industry stakeholders/ Households	Households in the UK are not very willing to take a loan or refinance their mortgage to pay for their renovations. This can have an impact on the policy.	“Reluctance towards mortgages and loans”→ “Percentage of people using mortgage refinance to pay for renovations”	Interview with a lecturer from the UCL Institute of Environmental Design and Engineering with multi-year work experience in the construction sector

Figure 2. Evaluation framework adopted.

Table 3. New Devil’s advocate coding scheme.

Pattern	Code	Description
Production	Proposing	Turns that introduce, suggest and/or propose new concepts/ideas/items
	Building	Turns that engage with, elaborate, develop and provide information about concepts under discussion
	Clarifying	Turns that demand and/or open up a reflection on clarification of concepts and content
Procedural	Modelling	Turns that direct group discussion explicitly toward the model structure and construction
	Facilitating	Turns that aim to facilitate group discussion and work, like prioritising, prompting brainstorming, etc.
	Inviting	Turns that encourage participation by other group members
	Reiterating	Turns that repeat and summarise previous content
Decisional	Affirming	Turns that agree with the content proposed, under discussion, and/or to be included in the model
	Dismissing	Turns that serve to rebuff or ignore content proposed under discussion and/or to be included in the model
	Challenging	Turns that reject or critique alternative content proposed by other members
	Undermining	Turns that criticize other members for discrediting their proposed content
	Deploying Authority	Turns that contain directives that eliminate alternative content proposed by other participants referring to superior knowledge or expertise to justify the legitimacy of a proposed concept
Other	Interruption	Code to account for interruptions to avoid double counting the same communicative behaviour

Figure 3. New Devil’s Advocates’ impact on the system map (Stock and Flow Diagram). The initial model’s structure at the beginning of the workshop is depicted in black, additions made during the session are presented in red, and the structure directly suggested by the New Devil’s Advocate performers is shown in blue.

Table 4. Second workshop New Devil’s Advocates’ and participants’ cumulative interventions, variables, and connections.

Items accountable to	Interventions	Variables	Connections
NDAs	112 (20%)	15 (28%)	27 (36%)
Other participants	452 (80%)	39 (72%)	48 (64%)
TOTAL	564	54	75

Table 5. Coding results for the New Devil’s Advocates’ interventions.

Pattern	Code	Total NDAs interventions
Production	Proposing	18 (16%)	94 (84%)
	Building	41 (36%)
	Clarifying	35 (31%)
Procedural	Modelling	7 (6%)	13 (12%)
	Facilitating	5 (4%)
	Inviting	0
	Reiterating	1 (1%)
Decisional	Affirming	2 (2%)	2 (2%)
	Dismissing	0
	Challenging	0
	Undermining	0
	Deploying Authority	0
Other	Interruption	3 (3%)	3 (3%)

Figure 4. New Devil’s Advocate SWOT and mitigations analysis.

Table

ISSUE TO RAISE	STAKEHOLDER BEARER OF THAT ISSUE	HOW TO FRAME IT	POSSIBLE VARIABLES RELATED TO	SOURCE

Table

Structural input in the role-plot^a	Corresponding structure in the map^a
“Rebound effect”	“Rebound effect”
“Probability of unintended consequences”	“Unintended consequences”
“Probability of unintended consequences”→ “Negative word of mouth”; “Speed of negative word of mouth”	“Unintended consequences” → “Measure performance” → “Word of mouth effect”
“Quality of retrofit”→ “Probability of unintended consequences”	“Quality” → “Measure performance”
“Energy use” depends on the “EE level of the house” and on the “Compliance of householders behaviour with the new retrofit measure”	“Behavioural compliance” → “Measure performance”
“Fragmentation→ “Quality of retrofit”; “Skills of installer”→ “Quality of retrofit”	“Market fragmentation” → “Standards” → “Quality”; “SC skills” → “Quality”
“Quality of the retrofit”→ “Performance of the retrofit measure”; “Maintenance capacity”→ “Performance of the retrofit measure”; “Customer behaviour”→ ”Performance of the retrofit measure”	“Quality” → “Measure performance”; “Maintenance frequency” → “Measure performance”; “Behavioural compliance” → “Measure performance”;
“Industrial trust for making investments”	“Industry trust”
“Industry capacity”; “time to adjust capacity” (estimated in 1-2 years)	“Capability of response to demand”
“Installer skills”, “Time to gain skills, Skilled workers”	“SC skills”
“Unskilled workers” → “Skilled workers gap” → “Installers skills” → “Quality of retrofit”	“SC skills” → “Quality” → “Measure Performance”
“Quality”→”Credibility of the industry”→ “Demand”	“Quality” → “Measure performance” → “Uncertainty Costs” → “Perceived costs”
“Performance of the measures” → “Actual savings”	“Measure performance” → “Perceived monthly savings”
“Economic cycles”→ “Industry staff dismissed” → “Skills”	“Business Cycles”
“Uncertainty costs”	“Effectiveness uncertainty”
“Monitoring” → “Quality”	“Monitoring” → “Quality”
“Maintenance costs” → “Actual savings”	“Maintenance costs” → “Financial attractiveness”
“Disposable income” → “Priority given to retrofit”	“Households preference for energy efficiency”
“Marginal improvements” → “Technological level” → “Performance of the measure”	“Technological development of deep retrofit” → “Measure performance”
“Trusted information (credibility)”	“Tailored feedback to consumer on impact of their work”

^a The symbol ‘→’ stands for a causal connection.

Vennix, J. A. M. (1996). Group model building. Wiley.

 Google Scholar

Cappuccio, V., Cunico, G., & Zimmermann, N. (2017). Shaping policy development in a UK Government department. HopefullNESS the 13th Nordic Environmental Social Science Conference.

 Google Scholar

Shrubsole, C., Macmillan, a., Davies, M., & May, N. (2014). 100 Unintended consequences of policies to improve the energy efficiency of the UK housing stock. Indoor and Built Environment, 23(3), 340–352. https://doi.org/10.1177/1420326X14524586

 Web of Science ®Google Scholar

Government of United Kingdom (2017). House building; new build dwellings.   December Quarter 2016.

 Google Scholar

Houzz & Home (2016). HOUZZ & HOME – UK August 2016 (Issue August). http://st.hzcdn.com/static/econ/en-GB/U.KHouzzHome2016.pdf

 Google Scholar