How do we make sense of significance? Indications and reflections on an experiment

Abstract

Determination of significance is widely recognised as an important step in environmental assessment (EA) processes. The prescriptive literature and guidance on significance determination is comprehensive within the field of EA, whereas descriptive and explorative studies of how we go about making sense, or construct meaning, of actions to determine significance are few. This article makes use of sense-making theory to explore how sense-making among EA researchers and practitioners influences significance determination. Focus is on the situation when persons have their first look at information about a strategic choice and as part of this make their initial judgement of significance. An experiment is designed and conducted to investigate how persons make sense of a specific case and determine significance in a screening and scoping context. The experiment indicates patterns in the test persons' sense-making, including important differences in the way individuals screen and scope. These patterns concern what we notice, how fast we frame the choice, and when we are critical about the provided information. The indications provide a basis for reflections on practice and on how to organise EA processes.

Keywords:

• Sense-making
• significance
• strategic environmental assessment
• screening
• scoping

Introduction

Significance is a central concept in environmental assessments, since significance formally is the threshold that prompts assessment processes in the screening stage and the threshold for including impacts and alternatives in the scoping stage. Informally, however, assessment of significance occurs throughout the environmental assessment (EA) process and the following implementation, when decisions are made on what to include and investigate, how and at what level of detail, and finally, if and how results of decisions (e.g. mitigation measures for significant impacts) are implemented in practice. Significance also plays an important role in regulations on EA, for example, in the scope of the EU directive on strategic environmental assessment (SEA) (article 1 of the EU Directive 2001/42/EF) and in the Directive's instructions on public involvement, the content of the environmental report and monitoring. This article focuses on how people make sense and determine significance in the screening and scoping stages of SEA.

To guide the significance determination, the EU Directive includes significance criteria that concern the characteristics of the effects, the area to be affected as well as the plans and programmes in question. Significance is, however, not further defined in the Directive and the study concerning the report on the application and effectiveness of the SEA Directive found that ‘neither the Directive itself nor the SEA Guidance provides clear and unambiguous criteria for how to interpret the qualification when deciding to apply the SEA requirement’ (COWI 2009, p. 50). Significance is argued to be one of the elements in the Directive, which ‘many lawyers and environmental assessment practitioners will be employed for many years in sorting out’ (Thérivel 2004, p. 33).

Research has documented problems and challenges in the practice of significance determination. As an example, a study examined the results of discretion involved in screening of climate change plans, and found non-compliance with SEA legislation with lack of screening and following environmental assessment – owing to the subjective judgements of practitioners (Kørnøv & Wejs 2013).

Despite the importance of significance in EA procedures, the concept is rarely explicitly defined in the literature (Weston 2000, p. 193). Significance has been described as dynamic, contextual, political and uncertain (Wood et al. 2004), since increased knowledge among involved actors, change of actors, development in actors' preferences and values, and societal developments may all influence perceptions and conceptions of significance in a given context. The contextual character of significance is emphasised by Lawrence (2007b, p. 778), who points to the fact that ‘perceptions vary among populations and sectors of society regarding which impacts are positive and negative, and to what degree’. Significance determination is therefore widely influenced by discourses and practices constituting ‘dynamic “relational complexes” involving people, things and their many properties, competences and accomplishments’ (Healy 2005, p. 239).

The process of determining significance in EA

The EA literature provides many checklists, criteria, procedures and thresholds to guide significance determination (e.g. Thompson 1990; Thérivel 2004; Lawrence 2007b; Wood 2008). The EU guidance is another example of an attempt to limit discretion while determining significance in screening and scoping (EU 2001, 2003). The literature also includes a suggestion for inserting more ‘common sense’ in the assessment of significance (Ross et al. 2006) – however, without clarifying and reflecting upon differences in sense-making among persons and contexts. Despite the manifold of thresholds and criteria, determination of significance is argued to involve ‘an element of judgement’ (Thérivel 2004, p. 134), ‘subjective decisions’ (Wood et al. 2007, p. 810), personal viewpoints (Weston 2006), value-dependency (Lawrence 2007a, p. 759) as well as intuition (Canter & Canty 1993, p. 291). The process of determining significance has therefore been described as manipulatable (Wood et al. 2007) and imprecise, context-dependent, political and complex (Lawrence 2007a). The range of adjectives seems to be an indicator for how difficult significance determination is to grasp – and the inevitability of discretionary judgement.

The clash between the importance of significance and the complexity of significance determination has given rise to critical questioning of the concept (e.g. Lawrence 2007b), the team determining the significance (e.g. DEAT 2002; Peterson 2010), the process of determining significance (e.g. Wood et al. 2004), the lack of focus (Ross et al. 2006) and the timing and role of significance determination in practice (e.g. Nielsen et al. 2005; Christensen & Kørnøv 2011). Few studies have dealt with how people in practice identify significance and very few – if any – have investigated what happens when SEA practitioners in their first encounter with a case try to make sense of information in order to determine significance in the early phases of screening and scoping. In an environmental impact assessment (EIA) context, Weston (2000) argues that ‘[m]ost research in EIA decision making has focused on the project authorization process and not the crucial decisions made at the earlier stages of screening and scoping’ (p. 185) and Wood (2008, p. 23) points at a ‘paucity of research that critically examines and reflects upon the way in which significance is evaluated and communicated’.

The few studies of significance determination practice reveal elements of how we determine significance. By studying British local authorities, Wood et al. (2004) divide respondents into two profiles: People either demonstrated ‘a smooth, gradual and incremental appraisal of significance’ or demonstrated a step change response ‘punctuated by sharp changes in relation to the size/scale of the proposal’ (pp. 1 and 13). Wood et al. furthermore show that significance determination practice had no direct relationship to government guidance thresholds. The minor importance of official thresholds and checklists is also supported by the finding that only 2% of the local authority practitioners regarded checklists as the single most effective approach in screening practice (Wood & Becker 2005, p. 358). In a study of practitioners' balancing of precaution and efficiency in EIA scoping in the UK, Snell and Cowell find a tendency towards scoping issues in rather than excluding them owing to the concern of legal challenges, thereby enlarging the environmental statements (Snell & Cowell 2006). The results of a quality assessment of Environmental Impact Assessment Statements (EIS), based upon both individual and group assessment, showed important differences with group assessments being more critical than the individual (Peterson 2010). Peterson argues that that the group approach becomes an arena for outbalancing not just expertise but also subjective values and perspectives, and suggests a revision of the current assessment practice.

In addition to the British findings, significance determination processes in an EA context is under-researched (Snell & Cowell 2006). We still do not know the details of what happens when practitioners or researchers are presented with some kind of action and asked to determine whether SEA must be applied and what impacts and alternatives are significant. Insight into similar processes can be found in other fields of study, and the fields of socio-psychology and cognition seem especially relevant for shedding light on the first preliminary significance determination. Within these fields, theories of how we make sense of information has gained increased importance in recent decades with the focus on how people ‘construct what they construct, why, and with what effects’ (Weick 1995, p. 4). In the preliminary significance determination process, we only notice some of the relevant information and we base our judgement on a fraction of noticed information. In this way, significance determination is seen as a process of constructing reality, where sense-making is crucial for the judgement of significance.

Aim and contribution

The article investigates and reflects upon how to improve EA by paying more attention to the sense-making, thus emphasising the social and cognitive elements of assessment – compared with the technical and procedural. The aim of the article is to uncover how we notice and make sense of information in order to determine significance.

In contrast to Wood et al.'s (2004) retrospective investigation of significance determination, the aim is to uncover the process as it unfolds – as a direct observation of how the process evolves without retrospective filtering and reasoning. For this purpose, an experiment is designed to investigate how SEA practitioners and researchers make sense of written information and determine significant impacts and SEA relevance. The experiment is aimed at the very early sense-making, at what happens the first time we see a text. This focus is chosen since research shows that the initial meaning we assign to information and events can be very influential on the process that follows; Gawronski et al. (2010) refer to a large body of research that shows that people's unconscious evaluation of events can be ‘relatively rigid and difficult to change’ (p. 683). In an EA context, this means that our initial sense-making is important for the entire process as it unconsciously may hinder openness towards new information and other actors' opinions.

The research questions that are guiding the article are:

1.	What patterns can be found in the way SEA practitioners make sense of a strategic choice?
2.	How do such patterns influence significance determination?

Since significance determination is a complex process, the investigation will not find universal patterns, but tendencies in a context. The article discusses these tendencies in terms of inspiration for improving practice.

The study is a part of a research project on SEA and strategic choices in the Danish energy sector (see Lyhne 2011), and the experiment is using a hypothetical but realistic case of a strategic choice in the sector.

In the next section, the article unfolds sense-making theory and relates it to EA. We then present the design of the experiments, before setting out the findings of the research. The article concludes with reflections and ideas on how to acknowledge the sense-making taking place at the early stages of SEA.

Insight from literature on sense-making

Karl E. Weick's theory of sense-making describes human sense-making as a social process of continuously enacting events, extracting cues from these events and retrospectively making plausible stories (Weick 1995, p. 18). The sense-making literature is focused on how people make sense of stimuli; people ‘sort through prior cues, label them and connect them, which often result in plausible stories that are good enough to keep going’ (Weick 2001, p. 237). A person's sense-making process is interdependent with her mental framework, which is a set of rules, beliefs and worldviews that govern how the person makes meaning of input. Mental frameworks, identity and articulation are important elements in the process of reducing multiple meanings and generate a locally plausible story (Weick et al. 2005, p. 414), but it is not a clear-cut process. Starbuck and Milliken (1988, p. 49) argue, ‘people have to have numerous sensemaking frameworks that contradict each other. These numerous frameworks create plentiful interpretive opportunities – if an initial framework fails, one can try its equally plausible converse’. Frames serve the function of separating signal from noise and the filtered information, Starbuck and Milliken argue, ‘is less accurate but, if the filtering is effective, more understandable’.

In an SEA context, practitioners apply mental frameworks to organise information and inputs about impacts and alternatives and enact this sense and order back into the society through reports and technical summaries.

Weick describes sense-making as a process initiated when people are experiencing discrepancies and equivocality in their on-going sensing. People first search their frameworks to explain the discrepancies. These frameworks may be ‘Institutional constraints, organizational premises, plans, expectations, acceptable justifications, and traditions inherited from predecessors’ (Weick et al. 2005, p. 409). If no explanation is found, they label and notice cues in order to generate plausible stories. If these stories seem to be adequate, they are retained as guidance for future action and interpretation.

The process of making sense has been studied in socio-psychological research for decades. Starbuck and Milliken (1988) refer to studies that have shown that ‘some stimuli are more available or more likely to attract attention than others’ and ‘the characteristics of perceivers, including their current activities, strongly affect both the availabilities of stimuli and the abilities of stimuli to attract attention’. According to Watzlawick et al. (1974), blind spots are found in all mental frameworks and the blind spots prevent people from solving some problems. Furthermore, Bargh (1982) argues that part of our attention to stimuli is managed by automatic and involuntary processes which ‘can either facilitate or inhibit active attentional processing’ (p. 425).

Learning from sense-making literature, we – as EA practitioners and researchers – need to acknowledge that we cannot fully control what we notice and what we do not notice, the words we use are never accurate, and our initial interpretation may be rigid. Sense-making literature may provide the insight that is needed to better understand and improve how we read signals of importance and frame problems and opportunities (see Woodside 2001). Although the conception and the use of the term ‘significance’ differ between sense-making and SEA literature, significance plays an important role in both fields. It is thus interesting to use sense-making theory to investigate how test persons make sense of significance in an SEA framework and reach a decision upon what aspects are relevant to include in the assessment.

Methodology and set-up of the experiment

The following presentation of the experiment aims at being reproducible, so that others will be able to follow the steps and get comparable results.

To investigate patterns of how people notice information and frame concepts, the experiment is constituted by a case text and a procedure for observing test persons making sense of this text. The test persons are asked to speak out loud and underline words and sentences while reading a text.

The experiment procedure is presented with reference to elements of the sense-making process (Weick 1995) in Table 1. The procedure provides access to the on-going sense-making and judgement of significance as well as occasions for testing a person's reflection on the process (steps 5, 6 and 8).

Table 1 The steps in the experiment process and their relation to sense-making literature.

Step	Task	Elements in the sense-making process
1	A SEA practitioner [A] reads a text and during the reading underlines and comments upon what is especially interesting/useful for understanding (interruptions for clarification if needed)	Noticing and labelling of information in the enactment of the case
2	[A] is asked to explain what she/he noticed (retell the text). ([A] is not informed of the following stages to avoid dominance of interpretation at this stage)	Retrospective account of the noticing of cues, labelling of information and potential beginning of a story of what the case is about
3	[A] is asked to determine possible significant environmental aspects	Creating stories of what is significant
4	[A] is asked how she/he would go on: is EA needed, what analyses, alternatives and measures are especially important?	Creating stories by searching for experience with relevant incidents
5	[A] is asked of her/his idea about why she/he noticed the specific cues and whether the noticing had a personal touch	Retrospective reflection on the noticing process by the test person (steps 1 and 2)
6	[A] is asked of her/his idea about why she/he pointed at the specific significant environmental aspects	Retrospective reflection on the stories created (steps 1 and 4)
7	Before concluding, [A] re-reads text to confirm his/her understanding (with a new pen colour)	A test for a changed perceptual framework owing to the thoughts in steps 4–6 and more detailed knowledge about the experiment
8	[A] is asked about potential changes in understanding caused by the second reading in step 7	Retrospective reflection on potential changes and the reasons for these
9	[A] is confronted with statements uttered during the experiment	Confrontation of statements may give reactions in line with Weick's recipe of ‘How can I know what I think until I hear what I say?’
10	As a recapitulation [A] is asked about reflections on and learning in the experiment	It may give indications of how the test persons think about their sense-making process

Learning from Weick's recipe of ‘How can I know what I think until I hear what I say?’, the test persons are confronted with interesting statements that they have uttered during the experiment. The intention of this confrontation is to make the test person elaborate on interesting elements such as mental frameworks or individual sense-making processes. The number of confrontations per test person is limited to three.

Experiment set-up

The case text has characteristics similar to the actual future strategic energy planning in Denmark, for example, with its point of departure in renewable energy targets and new technologies. The case is formulated so that test persons will most likely recognise elements without being familiar with the situation.

The set-up of the experiment is:

•	A number of EA/SEA researchers and practitioners are test persons (people with ‘variable’ mental frameworks). These are selected to reach a variety in the test persons' backgrounds and occupational positions; see considerations below.
•	Each test person is guided through the experiment in isolation and the interviewers only interact during the test persons' sense-making of the information if clarification is needed.
•	Before the experiment starts, the aim, duration and content of the study are explained to the test persons. They are instructed to continuously speak out loud, underline words in the text that they regard as important for understanding, and explain thoughts and underlining during the reading of the text. To enhance trust and informality, it is emphasised to the test persons that their performance will not be graded or evaluated and that there are no trick questions.
•	The process is audio-recorded, subsequently transcribed and given to the test persons for comment.

Owing to resource limitations, the number of test persons for this study is set to nine. The selection of test persons is intended to include people from a variety of different jobs, expertise in relation to the information/professional field of expertise and educational backgrounds (see Table 2 below).

Table 2 Test persons in the experiment.

	No or little familiarity with SEA	Very familiar with SEA
Very familiar with the energy case	Lotte, Anynomous, Christian	Per, Stine
Little familiar with the energy case	Kristian	Martin, Sanne, Anja

This variety is intended to make differences in mental frameworks more explicit. Furthermore, it is intended to reflect that environment professionals are not the only ones to conduct SEA screening and scoping. In practice, the selection of test persons has resulted in a distribution of four university-based SEA researchers and practitioners, one consultancy-based SEA practitioner, one university-based energy planner, one municipality-based energy planner, one company energy planner and one university-based urban planner.

The case text

The case that the test persons are presented with is shown in Figure 1. The idea behind the text is to present a strategic choice related to a societal need in a way that resembles the sparse information faced by SEA practitioners in the early stage of SEA processes. Information provided at this stage is likely to be uncertain, ambiguous and flawed when it comes to knowledge about the consequences of the strategic choices. Therefore, the aim of the fictional case text is not to be consistent or technically correct, but to be potentially problematic and thought-provoking. For instance, the need for storage is specified as a single, large figure without providing calculations or references. A variety in content is sought so that it involves technical descriptions, a table with numbers and concrete examples of implications.

Figure 1 The case text presented to the test persons.

Research findings

The findings are presented and discussed in the following subsections. Since the number of test persons is limited to nine, the findings are indicative. The most interesting indications for EA practice are:

1.	There are substantial differences in the way test persons notice relevant information and determine significance between their first and second readings of the SEA text.
2.	Personal and professional experience can only partly explain the difference in significance determination.
3.	The way different people conceptually frame the case varies depending on their familiarity with the subject matter and their practical SEA experience: The older and/or more practically experienced persons are, the faster and firmer their framing.

The following sections are structured by the two research questions outlined above.

Patterns in noticing and framing

The experiment provides an empirical demonstration of the variety of how and when test persons notice and frame the case. The most prominent findings are presented below.

Changes in noticing between first and second reading

The experiment shows a tendency for the first reading to be primarily oriented towards the factual details and examples mentioned in the text, whereas the second reading is primarily oriented towards establishing the context and a critical stance towards the text. In the first reading, the underlining thus concerns matters such as the specific technologies presented (e.g. ‘Compressed air energy storage’) and the concrete examples of the implementation (e.g. ‘Closing the Limfjord in one end’). In the second reading, aspects like the strategic context (e.g. ‘store large amounts of energy’) and the strategic alternatives (‘intelligent control of electricity system’) are noticed.

Four of the test persons show awareness of their approach to the case description. In his comments, test person Per states that ‘by the first reading I try to establish the structure and by the second I patch it up, where I have overlooked something or maybe redefine something, because you would see that some other things go on in the text’. Christian explains his way of approaching the text: ‘Then I have some specific elements that I look for … I would not say that I memorise, but I remember the essence – maybe remembering the content more than the meaning of the text. Also because when the text is processed several times, it may be that it is another meaning that you make of the text than the first time you read it through’.

Patterns in the process of making sense of aspects

The experiment shows differences in how the test persons are making sense of aspects of the information. Stine continuously asks questions for a range of elements that she is not familiar with, and points to a range of elements she will have to investigate more in detail. In addition to experiences and knowledge, the experiment also indicates other influential parameters:

•	Talking out loud triggers sense-making. Kristian comments that his own speaking about alternatives and impacts made him notice the descriptions of initiatives and consequences in the text in the second reading. In a similar vein, Anja comments: ‘I am aware of it [the information], but when I have to express it, you also become more attentive to it’.
•	Concrete examples are helpful. Kristian especially notices the concrete examples in the text. On the closing of the Limfjord he comments: ‘It is a concrete proposal for a solution, which actually gives a better picture of what it is all about … If I was to remember something from this case in two weeks, it is probably that’.
•	‘Shocks’ are remembered. The closing of the Limfjord resembles a ‘shock’ to Lotte's mental frameworks: ‘Closing the Fjord! I especially notice that one, because that has indeed an environmental impact … It is absolutely absurd!’
•	Varying accessibility of numbers vs written text to different test persons. The different types of information in the text clearly influence what the test persons notice. The numbers in the table are particularly less accessible to some of the test persons. Anja skips the table and explains: ‘Then there is such a typical engineer table, and then I think, “That is a bit boring and skip it”. … I actually also skipped the table the second time and I did actually not notice that I did so’. The unawareness indicates the importance of the structures Anja imposed on the text in the first reading.
•	Local knowledge plays a role. Identity and local relations seem to play a role in what the persons relate to. Asked about unique aspects in her noticing, Sanne points to her relation to Aalborg, close to the Limfjord: ‘I am, after all, a local. It is not sure that a person from Zealand [other part of Denmark] would think like that’.

Experiences guide critical stance

In the second reading, as opposed to the first reading, the underlining reveals a critical position towards elements such as the strategic choice, the size of the need, government experts and the technologies put forward. As an example, Stine comments: ‘When it is this strategic level, I think it would be relevant to know the premises in terms of the projections and the expectations to the development’. The difference in critical stance between first and second readings is explicitly reflected on by Martin: ‘What I do in the beginning is actually that I accept the premise about the future electricity system, which makes it necessary to store big amounts of energy. … Others may say, “We need a discussion about this, before I go on”’.

The experiment shows a tendency for critical stances to depend on the professional background, so that energy planners are critical towards the correctness of the energy problem and solutions, whereas the environmental managers are critical towards the environmental implications and the need for the energy infrastructure.

Feelings and intuition are influential

Further, the experiment reveals some underlining and significance determination which cannot be rationally explained by the test persons. Instead, the test persons implicitly refer to ‘feelings’ or ‘intuitions’. Martin describes his choice of what is important as a feeling of what is useful; confronted with the meaning of numbers, he argues ‘it is not something that I feel in the moment that I have any use for’. In such cases, noticing thus becomes a guess – a ‘feeling’ – rather than a rational exercise. Lotte does similar non-rational underlining: ‘Now I underline that wind mills must constitute half of it. I do not know why I did it, but I did’.

How we frame the case is related to who we are and what we do

To explain their framing of the case text, Stine and Kristian explicitly refer to their profession; Lotte refers to the projects she is working on at the time of the experiment; Per and Anonymous relate to their experience and professional opinions. Thus, the test persons' familiarity with the energy sector and the familiarity with preparing an assessment seem to be two important dimensions of when and how significance is framed. Based on these two dimensions, Table 3 suggests four personal profiles in which test persons fit based on their approaches to determining significance.

Table 3 Profiles within the dimensions of familiarity with preparing SEA and familiarity with the energy case indicated by the experiment.

	No or low level of familiarity with SEA	High level of familiarity with SEA
High level of familiarity with the energy case	Relating (Lotte, Anonymous, Christian)	Settling (Per, Stine)
Low level of familiarity with the energy case	Seeking (Kristian)	Arranging (Martin, Sanne, Anja)

The ‘relating’ profile found several associations and potentials in the energy case without a certain quick frame on what should be assessed: As an example, Lotte relates cues in the text to a number of experiences she has gained in her profession. The ‘seeking’ profile recognised few elements in the text and did not identify a specific frame for understanding the case: As an example, Kristian explicitly states that he emphasises the implementation examples, because they appear ‘funny’ to him. The test persons familiar with similar cases and with preparing SEA are quick to settle the case in terms of what it is about and how to proceed. These persons are grouped in a ‘settling’ profile. The ‘arranging’ profile found aspects to assess, but did not have the technical insight to develop a specific frame for the energy case.

The framings' influence on significance determination

The experiment shows that the framing of the case is not a straightforward and linear process and the influences vary over time: Noticing ‘storage’, Anja initially suggests that the case is about carbon capture and storage. In line with Starbuck and Milliken's ‘if an initial framework fails, one can try its equally plausible converse’, she quickly realises its incorrectness and instead suggests an energy storage framing of the case.

The influence of the test persons' framing on their significance determination is outlined in Table 4. The findings indicate that test persons' framings of the text are highly influential on their judgement of impacts, alternatives and need for SEA.

Table 4 Test persons' dominant framings of the case and their influence on significance determination.

Person	Framing	Influence on significance determination	Time of expression
Per	Complex systems cause conflicts and are not needed – and nature is not the problem	No doubt about need for application of SEA. Focus on nature and land-use in terms of impacts. Focus on low-tech alternatives	1 min
Stine	How to get a smooth authority approval process	SEA not automatically necessary, but depending on authorities	1 min
Christian	Societal relevance of the technologies	SEA should have been done before delimiting to three technologies	1 min
Anonymous	Inadequate solutions to energy system planning	Critical stance on the choice. Arguing for a range of alternatives	3 min
Sanne	Initiatives are unpopular among locals	Focus on impacts on local citizens	5 min
Lotte	Potential for synergies	Positive potential among significant effects	7 min
Martin	Valid determination of technologies	Initial refraining from suggesting alternatives	14 min
Kristian	A planning task	A spatial focus in terms of significant impacts	14 min
Anja	The big picture	(Not noticeable)	22 min

When we frame the case varies considerably

As seen from Table 4, some test persons develop a specific framing on what the text is about within few minutes, whereas other test persons never seem to create an overall framing. The two test persons with an age over 50 and a professorship were quick (Anonymous and Per, within three minutes) to assign a specific frame to the text. Also the EA practitioners from the consultancy company and the Danish TSO quickly assigned a specific framing to the text. Relevant experience thus seems to lead to quick framings of the text.

Quick framings reduce openness to remaining information

The energy researcher (Anonymous) comments on the text that ‘I immediately see what this is all about. And then you may say that I have been trapped by my first impression’. Anonymous defends his framing: ‘It is obdurate, however, it is reasoned obduracy … There is no reason to use more time on this; it is bad solutions’. Anonymous and Per's quick framings reduce irrelevant stimuli, whereby more attention can be given to the impacts and alternatives that their framings consider as relevant. Automatic and involuntary processes seem to work the other way around for Kristian in noticing certain elements as funny, since they facilitate active attention to these elements.

The experiment findings indicate that a high level of familiarity with the energy case may be both a pitfall and a benefit in terms of significance determination: People who are very familiar with the energy case make a rapid framing that precludes information and at the same time focus their attention on what is (assumed to be) the most important elements. Similarly, a low level of familiarity may mean a more unstructured and slow process, but at the same time a critical stance on the basics of the provided information and openness towards other perspectives on the problem.

Conclusion and perspectives

In this article we have proposed that sense-making is a central activity in significance determination in both screening and scoping stages of SEA. Sense-making theory provides a theoretical and methodological approach to conceptualising and investigating sense-making involved in test persons' determination of significance.

The experimental research has, owing to the small number of test persons, no ambition of making comprehensive and general statements about sense-making in SEA processes. The research is meant as a conceptual and empirical input to the understanding of the social processes that take place initially and continually during the SEA process.

The experiment and findings supplement ideas and concepts within decision-making. Kørnøv and Thissen (2000) disputed the idea that ‘more information leads to better significance determination’ in SEA, and the experiment shows instances where the test person developed a firm frame in the very beginning of the reading of the case, regardless of the remaining information. Simon (1947) proposed the idea of ‘satisficing’ and the experiment shows instances in which test persons are satisficing their need for information in order to get on with the process.

The article furthermore underlines that the individual engaging with the SEA text is not objective and passive, but is a sense-maker. The text is not ‘transmitted’ and received fully by the individual. Instead the test persons construct stories of meaning, which involve ‘negotiations’ between the SEA text and the individual in the reading process and even ‘re-creation’ of elements in the text.

As a consequence of the findings, sense-making should be considered a key element in the process of significance determination. The question is then how we can approach our sense-making in a way that is beneficial for significance determination processes. How can we use this insight to develop a better appreciation of the link between information and significance determination? Three suggestions are provided in the following: Recognition of and reflection upon our own sense-making; frame awareness in team-setting; and reconsideration of guidance and good governance.

Recognition of and reflection on sense-making

As presented, the experiment shows a tendency for test persons to be more critical during the second reading by questioning premises and the intention of the text. Wood and Becker (2005) propose a frame-reflective approach to counteract similar problems: ‘To limit the problems associated with screening errors, further guidance should seek to raise awareness of the existence of frames amongst practitioners and encourage a frame-reflective approach to screening decision making’ (p. 367). They picture ‘frame-reflective practitioners’ who actively question the basis of their assumptions and the subsequent implications, but they do further advise how it can be done in practice.

Insight into how we make sense, like the insight the test persons gained through the experiment, may be a means to be aware of assumptions. Similar to the experiment, an open dialogue with colleagues based on a comparison of what is noticed and what is found significant in a given case may provide a basis for increasing our awareness of our blind spots and rigid framings.

Team-setting for screening and scoping

The findings indicate the importance of setting a team with different profiles and different degrees of familiarity with the case. Furthermore, the findings indicate that differences in background, age and experience are needed if we want a more heterogeneous and holistic perception of the case. In this way, the findings are in line with Peterson's (2010) arguments on group-based significance determination. Awareness of the frames we employ in team-setting may thus make it possible to reduce ‘blind spots’ and enhance a broader perspective on impacts and alternatives. Insight into frames in an organisation may therefore be important knowledge when organising SEA processes and aiming at better quality of the SEA process.

The different levels of sophistication of the framings identified in the experiment call for consideration of familiarity to the case when setting the team. The higher level of sophistication plays an important role in distinguishing between significant and non-significant impacts and alternatives, and sophisticated framings may thus be a necessity to scoping in, rather than excluding, too many impacts and alternatives. At the same time, less familiarity with the case may be needed to question what firmer framings take for granted. The significance determination may thus in practice benefit from openness at different levels of sophistication, so that both basic assumptions and advanced issues are critically questioned.

It may similarly be relevant to consider sense-making processes in the public consultation. DEAT (2002) argues that making the process of significance determination ‘more explicit, open to comment and public input’ would be an improvement of the practice. Public consultation is an opportunity to bring a large number of mental frames into the screening and scoping process, and careful consideration to the sense-making process may provide an opportunity to articulate elements that are not noticed or not labelled.

Guidance and good governance

Guidance on SEA involves a range of checklists on screening and scoping based on targets and thresholds. The limited reference of thresholds in the test persons' sense-making indicates that thresholds do not play an explicit role at this early stage. In line with the study by Wood and Becker (2005), the experiment findings indicate that experience seems to play a far larger role. Thresholds and targets may rather be used as retrospective legitimacy for the choices made during meaning creation.

The experiment also suggests a discussion of good governance. As an example, the IAIA best practice principles state, ‘the [EA] process should result in full consideration of all relevant information on the affected environment, of proposed alternatives and their impacts’ (IAIA 1999). The experiment findings suggest a re-consideration of such formulations, as the meaning of ‘full consideration’, ‘all relevant information’ and ‘affected environment’ differs from person to person and from profile to profile. To acknowledge the constructionism and complexity inherent in sense-making, the best practice principle could instead focus on the openness and communication between actors during the process.

Overall, the perspectives point to a need to notice and recognise significance determination, have conversations in interactions about its nature and role, and make significance determination an object of both social and institutional learning.

The experiment is made on an individual basis, whereas sense-making in practice is taking place in a social interaction between people. The individual basis is chosen to allow for an investigation of the individual's enactment and bracketing of events, which would be difficult to investigate in an experiment with social interaction; if two or more people were brought together, it would be impossible to concurrently access their thoughts as they unfold. An experiment with social interaction is a very relevant extension to the individual experiment and such an extension may reveal how the individuals' enactment and bracketing transform in a social setting. Although the sense-making principles are generic, additional procedural elements would need to be developed in order to go beyond uttered noticing and framing to the individuals' sense-making as it unfolds in the social setting.

Acknowledgements

We acknowledge with thanks the assistance of the test persons. We would also like to thank Pernille Sylvest Andersen for her valuable proofreading of this paper. Finally, we would like to thank Energinet.dk for support of the PhD project within which this experiment was made.