Considering expert takeovers in citizen involvement processes

ABSTRACT

While citizen involvement has become an increasingly accepted and popular procedure in both the theory and practice of responsible research and innovation (RRI), there remains a curious dilemma in that the accomplishment of involvement does not necessarily ensure sought responsibility towards citizens. Instead, experts may easily take over the process of involvement and change the outcomes in quite distinct directions, as is empirically shown in this article, while still claiming to draw legitimacy from citizens. To counteract such unwanted takeovers, citizen contributions should be considered as a point of reference throughout the subsequent activities resulting from the involvement. Alternatively, the citizen contributions should simply be used as they were articulated without any significant expert translation. The article draws on empirical insights from an extensive case study in which citizens in 12 European countries articulated visions on sustainable futures, which experts then formulated to priorities for the European Union's Horizon 2020 framework programme for research and innovation.

KEYWORDS:

• Citizen involvement
• expert assessment
• citizen visions
• research priorities
• statistical analysis

Introduction

Citizen involvement is at the core of responsible research and innovation (RRI), as it considers citizens’ societal concerns and expectations in respect to research and innovation activities. While procedures and methodologies for involving citizens are evolving and maturing (see Bechtold, Capari, and Gudowsky 2017; Rip 2016), how to cope with contributions from citizen involvement remains an open question. This is particularly true when outcomes from involvement cannot be applied as such, as is the case, for instance, in popular votes or referendums. Challenges then emerge concerning how, by whom and at what stage the outcomes should be further translated into viable options in research and innovation.

Experts and policy specialists consequently play a key role when translating or at least considering outcomes from citizen involvement and carrying the outcomes through to the succeeding stages of RRI. Based on the inclusive ideals of RRI, it can be argued that even if citizens are objectively incorrect in their assessments or endorsements, they should be considered with appropriate respect when formulating the succeeding activities (see Dahl 1994; Kahane et al. 2013). Therefore, interplay between experts and policy specialists, on the one hand, and citizens, on the other, is of key importance when translating outcomes from citizen involvement into policy.

Additionally, as outcomes from citizen involvement often provide insights which require further translation, it would seem a worthwhile approach to make a clear distinction between contributions from citizens and contributions from others. This might be challenging in real life, as these roles often are overlapping or at least interdependent, and even the assessment of citizen contributions itself may require special expertise. Nevertheless, as this article will argue through the use of a case study, making such a distinction through additional reflection or at least through validation of the way citizens assess expert preferences may provide quite a different and arguably more comprehensive view of citizen contributions. This is of particular interest not only for the identification of citizen perspectives but also for strategically challenging and disrupting established research and innovation procedures (see Christensen 1997; Fligstein 2013).

This article reviews an extensive case of citizen involvement that produced 50 visions of the future for experts to select amongst in order to formulate research priorities. The article examines the challenges of such procedures and highlights the potential and usually unintended effect of expert takeovers. Here, citizen involvement is defined and understood as an activity that introduces complexity and diversity to established policy and priority formulation processes (see Kahane et al. 2013). Expert takeover, on the other hand, relates to a redirection of the outcomes of citizen involvement towards expert prioritisations.

The article proceeds by reviewing how responsible research and innovation approaches citizen involvement and how involvement methodologies have been developed from this perspective. It then examines an empirical case study where citizens in 12 countries were involved in the development of priorities for the European research and innovation agenda, and identifies two key stages in which the integrity of citizen contributions was challenged by experts who acted as gatekeepers and took over the involvement process. The study empirically demonstrates that expert involvement screened out some citizen contributions while also introducing a contrasting set of prioritisations. The concluding section of the article discusses how and to what extent the unwanted effects of such gatekeeping may be overcome.

Interplay between citizens and experts

Responsible research and innovation (RRI) can be traced back to debates on nanotechnology and studies of laypeople's risk perceptions and has received further impetus in the European Commission's Horizon 2020 research and innovation programme (Rip 2014; von Schomberg 2013). The concept now embraces ideals relating to societal challenges, ethical values and inclusiveness (Koops 2015; Owen, Macnaghten, and Stilgoe 2012). The latter emphasis follows the logic of introducing new policy topics, rather than following up on scientific advances (see Rip 2016). Policy interest has contributed to the popularity of the concept while simultaneously expanding its scope.

Nevertheless, citizen involvement itself is seldom sufficient to address citizen concerns regarding research and innovation. Indeed, expert competences are called for, particularly when tackling complex issues. To elaborate on this interplay, the following sections discuss how citizen involvement relates to RRI and reflect on the rationale of accompanying citizen involvement with the competences of experts, even when this might interfere with the aims of that involvement.

Citizen involvement and RRI

Responsible research and innovation (RRI) has for some time embraced involving citizens in research and innovation activities for instance through the formulation of research ideas (Forsberg et al. 2015; Rip 2016). Adhering to the outcomes of citizen involvement in the agenda-setting stages of research and innovation, i.e. ‘upstream’, merits particular attention, as this also may further alleviate public distrust in science and policy (Anttonen et al. 2018; Burgess and Chilvers 2006; Repo et al. 2018; Wilsdon and Willis 2004). RRI itself can be considered to have transformed into a social innovation containing cultural and institutional dimensions as well as an element of social accountability (see Matschoss and Repo 2018; Rip 2014).

Involving citizens in the formulation of research and innovation priorities provides a demand-oriented assessment of future solutions (see Decker et al. 2017) and further encourages rethinking the roles and responsibilities of different actors in innovation systems (Evans and Plows 2007; Forsberg et al. 2015). Indeed, the approach democratises the formulation of research and innovation agendas (i.e. ‘governance of intent’ as Owen, Macnaghten, and Stilgoe 2012, 754 put it). This also represents a shift which is more than rhetorical from informing and explaining to the public towards a dialogue with citizens (Pieczka and Escobar 2013). The setting is similar to that of user innovation theory, which examines how users have solved emerging problems (von Hippel 2005). In other words, users not only identify problems but also articulate solutions, which are only later introduced to the population at large by organisations such as companies. Following this line of thought, citizens should also be allowed to contribute to solutions or at least to influence how solutions are developed. Again, it is of key importance to ensure that user or citizen contributions are carried through to involvement and innovation processes in their genuine forms rather than being taken over by experts.

As a tool for realising RRI, involvement of citizens is often undertaken to identify their perspectives. In effect, such activities both strengthen knowledge flows and increase mutual learning between decision makers and citizens, thereby complementing the perspectives of other stakeholders and vested interests (Renn and Schweizer 2009). Thus involved citizens are considered to be stakeholders in their own right, as they possess societal, lay-person expertise that reflects the public interest (Brown 2006; Kahane et al. 2013) rather than specialist knowledge or vested interests. The ultimate aim of involvement activities of this kind is to increase the transparency and openness of decision-making processes and bolster legitimacy and the confidence of citizens in public policy processes (Stilgoe, Lock, and Wilsdon 2014). Indeed, expert analysis has given way to public deliberation in attempts to emphasise the role of human aspirations and introduce diversity at the expense of technological determinism (Stirling 2008).

Moreover, involvement supports the adoption of active citizenship and empowers people by offering them means to influence political processes in their early agenda-setting stages (Brown 2006; Howlett, Ramesh, and Perl 2009; Jasanoff 2003). In such undertakings, citizens reformulate policy agendas, set new priorities for research and innovation or question the assessment of technologies rather than display a lack of understanding for science (Wynne 2006). The concept of RRI and citizen involvement have both emerged from an ambition to reform research and innovation agendas, and their key benefits relate to improvements in the quality of science (Stilgoe, Lock, and Wilsdon 2014). Indeed, citizen involvement can offer something novel by generating unforeseen outcomes and stimulating creativity and innovation (Owen, Macnaghten, and Stilgoe 2012). Consequently, it can also contribute not only to the creation of consensus but also to the disruption of established practices by bringing forth varying aims for policies, strategies and agendas. Indeed, this has been confirmed in a recent large-scale comparison of agenda setting for EU research and innovation policy, which empirically indicates that citizen induced policy advice differs from that provided by experts (Rosa, Gudowsky, and Warnke 2018).

Nevertheless, adding an element of citizen involvement to the formulation of research and innovation does not necessarily fulfil aspirations relating to increased ‘responsibility’, and may in fact add new challenges. No standard procedures exist for either selecting which citizens to involve or determining the issues for which involvement would be particularly important. Moreover, the benefits identified for citizen involvement, including improved legitimacy and representation of a diversity of interests, must be contrasted with practical challenges, such as the lack of financial resources to realise involvement and the limited capacities of citizens to participate (Kahane et al. 2013). Acknowledging the shortcomings of existing procedures and methodologies could indeed provide new insights into how to pursue responsible innovation (de Hoop, Pols, and Romijn 2016).

Rationales for involving both citizens and experts

The justification for bringing together contributions from citizens and experts is the notion that citizens possess societal insights which require expertise in order for them to be translated into research and innovation priorities (see Horlick-Jones, Rowe, and Walls 2007). Citizens are acknowledged to have expectations towards and concerns about the future and even possess multiple social imaginaries, which enables the development of social practices (Taylor 2002), but their competences are considered insufficient to formulate them into research priorities (Evans and Plows 2007). The task of experts is, thus, to consider citizen perspectives and combine them with their own knowledge in an effort to formulate priorities in a way that respects citizen views (Linnerooth-Bayer et al. 2016).

The case of citizen involvement analysed in this article examined a key tenet of RRI: citizens’ articulation of research ideas and the assessment of formulated research priorities. The main objective was to gain insights into citizens’ preferences by integrating the views of laypeople into the articulation of novel research concerning sustainability.

This approach to citizen and expert involvement follows a methodology, first developed in the European Civisti project (Jacobi et al. 2011; Rask and Damianova 2009), which highlights the interplay of foresight and participatory technology assessment. In this methodology, citizens describe their visions of the future in a target-setting manner, while experts translate these visions into research priorities and policy recommendations to reach that target. This approach aims for desired futures, rather than problematising the adverse effects of technologies and their accompanying risks, which are often considered in Technology Assessment (TA). The approach is thus closer to that of Constructive Technology Assessment in that it addresses the social implications of new and emerging technologies (Kiran, Oudshoorn, and Verbeek 2015; Schot and Rip 1997). Furthermore, it acknowledges that society and technology interact and develop together, thereby accentuating the need to target overarching and desired futures rather than solving the specific problems of today.

While citizens articulate visions and thereby provide demands and needs for the future, it is the task of experts to formulate them into research priorities that can be focused on to reach that future (Gudowsky and Sotoudeh 2017). Decker et al. (2017) followed a similar division of labour when examining imagined technology futures in care-giving. The approach of comparing insights from two distinct groups is also similar to that of a study by de Jong, Kupper, and Broerse (2016), in which the varying imaginaries (see e.g. Taylor 2002) of scientists and security professionals were examined. The difference here is that this article does not attempt to build consensus between the two participant groups, but rather to contrast their preferences.

In Europe, a number of projects have incorporated citizen contributions into the development of research and innovation programmes. Recent citizen involvement approaches have included focus-group interviews concerning waste management (Voices 2015), and citizen consultations on and technology assessments of public health genomics and aging societies (Pacita 2016). Citizen involvement has been thematically open when addressing forward-looking topics, utilising the methodology reviewed in this article, in which citizens create visions that experts formulate into priorities (Civisti 2011; CASI in Bedsted et al. 2016; Cimulact in Jørgensen and Schøning 2016).

While all these projects have their merits in terms of impacts and outcomes, the CASI project provided indicators which allow an analytical comparison between how citizens and experts prioritise contributions originating from citizens. Accordingly, this article empirically reviews the kinds of citizen visions experts chose for priority formulation and the difference in the way citizens and experts ranked the formulated priorities. Such analysis is possible to carry out, because experts had to make selections and rankings due to the design of citizen involvement in the CASI project. In this way, we aim to show that even when citizens and experts work towards the same aim – a more sustainable future – their priorities reflect competing views, expertise and values. Citizen involvement, in this respect, requires consideration of this observation, even when it cannot thoroughly be explained how the differences come about. The data presented in the following section from the CASI case study are suitable for such analysis, as citizens confirmed that their visions were adequately formulated into respective research priorities (Bedsted et al. 2016).

Case study: citizen-expert involvement

This article reviews data collected in the European CASI project, which organised a three-stage process of citizen-expert-contributions for the development of research priorities for the European Commission's Horizon 2020 framework programme for research and innovation in 2015 (Bedsted et al. 2016). The applied approach is based on a process utilised also in other recent projects (see e.g. Repo and Matschoss 2018; cf. Gudowsky and Sotoudeh 2017) where researchers first ask citizens to articulate visions for the future 30–40 years from now in workshops organised in different European countries. Second, a selected group of experts screen the visions in a dedicated workshop. Then they formulate research priorities which are based on the visions and are to be utilised in the European Union's framework programmes for research and innovation. Finally, the research priorities are introduced in national workshops where citizens assess the connections between the priorities and the visions.

The overall idea is to gain novel insights and unexpected views from citizens about the situation beyond the present day and to build public acceptance and legitimacy for future research and innovation activities while supporting sustainability. The process brings together knowledge formed by experts’ sustained engagement with issues related to sustainability with citizens’ lack of specialist knowledge (in the sense of Evans and Plows 2007) to complement each other in the different stages of the involvement. In general, the ultimate aim of such involvement is to break established power arrangements and introduce new agendas for research and innovation through the combination of these systematic differences in perspectives (see Stirling 2008) in a process where neither is supposed to dominate but to support the other.

Compared to other similar instances of citizen involvement, the CASI exercise was particular because it documented the varying viewpoints of citizens and experts, thereby offering an excellent opportunity to examine expert takeover tendencies in citizen involvement. In the first involvement stage, a total of 230 citizens representing diverse interests and backgrounds were guided through a uniform vision-building process in 12 country panels (Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Finland, Germany, Italy, Poland, Portugal, Slovenia and the United Kingdom) (Kaarakainen et al. 2015). 50 visions articulated in group work by the citizens reflected their different perspectives on sustainable futures and on the kinds of elements that these futures would encompass. Organising the citizen workshops during a short timeframe in the 12 countries ensured the possibility of comparing visions between countries, as no external, transnational or global event or development radically influenced the citizens’ perspectives on sustainable futures. Prior to the workshops, the participating citizens in each country received an inspiration magazine translated into their national language. The magazine consisted of neutrally written articles on sustainability issues, which formed the common citizen knowledge base in the involvement process.

A selection of these visions for a sustainable future were developed into 27 research priorities in the second stage by 22 European experts in innovations, sustainability and public engagement in a two-day workshop in Copenhagen, Denmark (Repo, Kaarakainen, and Matschoss 2015). At the end of the workshop, the experts ranked the research priorities according to their importance. In the third stage, 184 of mostly the same citizens were in national workshops introduced to the research priorities formulated by the experts and requested to assess how well these related to their respective visions as well as ranked them based on the importance (Matschoss et al. 2015). The process and outcomes are presented in Figure 1.

Figure 1. The citizen-expert involvement process and its outcomes in the examined case.

This article's research data are thus formed by (1) citizen visions selected by experts to be further developed into research and innovation priorities, and (2) the varying citizen and expert assessments of those priorities in each of the 12 countries and by the experts. This comparative setting allows an analysis of the contributions of citizens and experts and their evaluations of the outcomes of the involvement process.

Example 1 presents an excerpt from one vision (‘New sustainable energy economy’) and the research priority that was formulated on the basis of it.

Example 1.

Vision and succeeding priority (Sources: Kaarakainen et al. 2015; Repo, Kaarakainen, and Matschoss 2015)

Vision: New sustainable energy economy.

Aim of this vision is to develop visions and ideas for the necessary rapid implementation of the energy transition.

The existing fossil energy sources should be consequently reduced. The use of nuclear energy should be completely renounced on an international level. Instead, renewable energy sources will be supported by politics and their expansion and technological development should be accelerated. If the vision is implemented, only renewable energy will still be used.

As a part of the restructuring process national energy production and supply is decentralized and new decentralized storage technologies are developed. Renewable energies are connected with each other so that they build together with new storage technologies the ‘virtual power plant’. Through this technology the renewable energies from sun, wind and water are stable in a network and are able to secure the base load. The conventional energy sources, which were essential for energy supply until now, are no longer required. Big, monopolistic energy suppliers are no longer necessary and disappear from the market. Accompanying to this energy transition, comprehensive procedures for risk analysis as a part of development for the use of all energy sources are to be implemented by law.

Research priority: Research on business models and changing institutions related to sustainable energy economy.

Research priority is to study the change in the roles of market actors and institutions especially in order to connect small scale energy producers. Research topics include the development of a stable energy market system, risk management, security of the grid, energy storage, prosumerism, energy democracy, and data privacy concerns. Similarly, the transition from a centralized into a decentralized market structure merits research.

From a total of 50 citizen visions, ‘New sustainable energy economy’ was one of the 27 visions selected by the experts for formulation into a research priority. This selection constituted the first major topic-related step in which the experts guided the citizen involvement process in another direction than that envisioned by the citizens. ‘New sustainable energy economy’ was translated into the research priority ‘Research on business models and changing institutions related to sustainable energy economy’, which the experts ranked second in terms of importance amongst the 27 priorities formulated by the experts. The citizens then confirmed that the priority conformed well to its originating vision; however, they assigned it a low citizen rank of 18. This process thus contributes two key sets of data which are used to examine the rival preferences of citizens and experts: (1) vision selection for development into research priorities, and (2) expert and citizen rankings of research priorities. Citizen validations show that the formulation of the research priorities was quite faithful to their originating visions (with an overall score of 3.53, with 3 representing a moderate degree and 4 a large degree of faithfulness), which provides a good basis for comparison.

Expert takeover and its implications for citizen involvement

In the following section, empirical results from the involvement of citizens and experts in the formulation of European research priorities as conducted in the CASI project are used to observe how experts challenge contributions from citizen involvement. First, topic modelling is used to demonstrate how experts screen out citizen contributions by preferring technical and systemic topics over social or personal ones. Second, differences in citizen and expert assessments of research priorities are analysed statistically.

Screening of topics by experts

In the involvement process, experts selected approximately half the citizen visions (27 of 50) for formulation into research priorities. In order to reveal if the experts’ selection was topically balanced or whether they emphasised some topics over others, a review of this selection is required. Nevertheless, conducting such a review is far from simple, as citizen visions are of complex character. The titles of the visions do not always correspond well to their contents, and several visions address the future from many vantage points and at various abstraction levels. Furthermore, categorisation of the visions is challenging, as each vision may address more than one topic. To overcome these challenges, the study applied the methodology of topic modelling to create a uniform base for further comparative analysis. Moreover, topic modelling provided an opportunity to connect each vision to the observed topics.

Topic modelling is a suitable methodology for analysing a large number of comparable texts, as it does not require predefined concepts or categorisations. Instead, topic modelling is used to identify reoccurring patterns in a joint corpus of texts. In the methodology, probabilistically identified word clusters form topics. The present study employed Latent Dirichlet Allocation (LDA) to conduct the modelling, using the MALLET package for statistical processing of natural languages (Blei, Ng, and Jordan 2003; McCallum 2002). The modelled number of topics reflects the granularity of analysis, and seven topics were selected to be modelled because this produced results that were not overly abstract or too specific to analyse. Trial modelling and consultation of earlier research (Repo and Matschoss 2018; Repo, Matschoss, and Timonen 2017) show that a smaller number of topics provides a more abstract level of analysis as it merges identifiable topics. A larger number of modelled topics, on the other hand, provides results which can be hard to tell apart. Seven topics provided a good balance between the two ends, and trialling is a common procedure to establish the sought granularity for analysis. Hyperparameter optimisation every 10 iterations was applied to identify relative weights between the topics. Table 1 depicts the seven topics, their relative weight (Dirichlet parameter) in the corpus of visions, and the key words for each topic.

Table 1. Seven modelled topics in citizen visions.

Name of topic	Dirichlet parameter	Key words
Society and change in human life	0.560	people, social, resources, society, vision, development, life, change, education, sustainable, economic, human, local, environmental, economy, environment, future, values, activities, system
Quality of food products	0.156	quality, products, food, production, high, materials, insects, consumers, waste, cannabis, reducing, market, healthy, development, systems, knowledge, resources, encourage, buy, benefits
Energy production and renewables	0.117	energy, production, system, small, support, renewable, sources, scale, public, water, solutions, infrastructure, individual, technologies, part, buildings, supply, state, due, energies
Personalised working time	0.116	individual, work, time, personal, labour, system, health, working, physical, exercise, beauty, distributive, sport, group, case, diseases, promotion, virtual, activities, trainer
Public transport network in a green city	0.108	citizens, city, green, transport, network, public, services, spaces, functions, areas, important, number, provide, empathy, element, open, community, corridors, surrounding, wellbeing
Education for sustainability	0.104	education, knowledge, world, learning, sustainability, problems, educational, vision, immigrants, exchange, planet, diversity, political, virtual, children, communication, policy, objectives, classroom, relation
Urban farming	0.073	urban, farming, local, city, related, land, production, people, farm, farms, members, garden, gardens, cities, common, roofs, member, food, plants, vegetables

The largest topic identified in the vision corpus was assigned the title ‘Society and change in human life’. The topic concerns people and societal developments. It relates to sustainability issues in terms of resources, education, environment and economy, and presents a value-based view of the future. This was the key topic in 18 citizen visions and a secondary topic in seven (cut-off at .300, i.e. 30%). A more detailed presentation of the extent to which each vision relates to the identified topics is presented in Appendix 1.

The second largest topic was labelled ‘Quality of food products’. It concerns novel sources for food production, the reduction of food waste and knowledge on healthy food. The topic captures visions related to developments in the consumer market for novel kinds of food, such as insects. ‘Quality of food products’ was the key topic in a total of nine visions and a secondary topic in two.

The third topic was termed ‘Energy production and renewables’. The topic is based on views of a future energy system that supports small-scale energy production with renewable energy sources. It covers a broad range of changes prompted by novel technological solutions in the wider infrastructure, such as buildings and water use. ‘Energy production and renewables’ was the key topic in six citizen visions and a secondary topic in one.

The fourth topic, ‘Personalised working time’, is based on visions calling for individual and personal working times. A system with more flexible working hours would create health benefits and enable better possibilities for physical exercise and sport. The topic includes views on how to divide leisure time between novel kinds of activities, such as engaging a virtual trainer. This was the key topic in four citizen visions and a secondary topic in another four.

The fifth topic was labelled ‘Public transport network in a green city’, and it deals with well-functioning public transport networks that help urban areas develop towards greener cities for citizens. Such green, well-connected cities would enable the development of open communities based on empathy and produce well-being in their surroundings. This was a key topic for five citizen visions and a secondary topic for one.

The sixth topic was titled ‘Education for sustainability’, and it promotes a world based on education, learning and knowledge. In such a future, sustainability problems will have been solved through education that considers diverse aspects of life. The topic relates to a future knowledge society where problems are solved through a holistic view of the planet, creating, for instance, better understanding of immigration issues. ‘Education for sustainability’ was the key topic in six citizen visions and a secondary topic in four.

The seventh topic was labelled ‘Urban farming’, which relates to visions of cities that include spaces for farming. For example, people could have gardens on their roofs and members of housing associations could cultivate plants and vegetables for food in urban areas. Two citizen visions featured ‘Urban farming’ as a key topic.

As topic modelling was conducted so that each vision was connected to its respective topic or topics, this brings forth an opportunity to review how experts preferred the topics prevalent in the citizen visions (see Appendix 1 for each vision and its respective topic). Table 2 presents the topics identified in the citizen visions and reviews the topical share of the visions which the experts formulated into research priorities. The number of visions includes key topics and secondary topics (the latter being counted as .5). Due to the inclusion of secondary topics in the analysis, the number of visions is greater than the nominal number. This procedure, however, provides a more comprehensive view of which topics were selected by the experts for formulation into research priorities.

Table 2. Screening effect by experts: share of visions proceeding to research priorities according to topic.

Name of topic	Number of visions	Number of visions selected as a basis for research priorities	Percentage of visions progressing to research priorities
Energy production and renewables	6.5	5.5	85
Quality of food products	10	7.5	75
Public transport network in a green city	5.5	4	73
Urban farming	2	1	50
Education for sustainability	8	3	38
Society and change in human life	21.5	8	37
Personalised working time	6	2	33
All 7 topics	50	27	54

Table 2 demonstrates that the experts preferred to choose visions with technically oriented systemic topics for formulation into research priorities. These included ‘Energy production and renewables’ (with 85% of visions formulated into priorities, ‘Quality of food products’ (75%) and ‘Public transport network in green city’ (73%). A review of these three topics and their corresponding visions construes these topics as technological and systemic to their character. For instance, the vision on distributed small-scale energy generation takes into account various technical solutions in energy production and the public sector (topic: ‘Energy production and renewables’). Similarly, the citizen vision on sustainable agriculture considers healthy production of food, preservation of the countryside and animal welfare (topic: ‘Quality of food products’). In turn, one of the two visions of ‘Urban farming’ was formulated into a research priority, suggesting that that the experts responded neutrally to this topic (50% of these two visions were selected compared to an average of 54% of all visions).

By contrast, the experts screened out visions of a social or personal character. Accordingly, a disproportionately small number of visions connected with ‘Education for sustainability’, ‘Society and change in human life’ and ‘Personalised working time’ were formulated into research priorities (38%, 37% and 33%, respectively). This screening effect was further accentuated by the fact that the number of connected visions was noticeably higher for these topics than for the others (35.5 vs. 24). A review of respective citizen visions shows that these topics are social and human-centred in character. The citizen vision on sustainable education, accordingly, suggests the establishment of family centres, parent schools and community centres to this aim (topic: ‘Education for sustainability’). Correspondingly, the vision on living in community highlights equality among citizens, respect for each other, promotion of social and economic interaction of everybody, enhancement of personal and social balance, and adoption of a healthy lifestyle (topic: ‘Society and change in human life’).

In conclusion, a review of the selection process reveals that experts preferred some citizen topics over others. This selection appears systematic and logical in the sense that technical topics were preferred over social and personal ones. In this respect, resetting the agenda presents experts with an opportunity to take over the agenda initially created by citizens. The outcomes of this are examined in the next section.

Rival expert and citizen assessments of priorities

The detailed reporting procedures of citizen and expert involvement in the CASI project provide a further opportunity to review how citizens assessed the research priority formulations of experts. This is of particular empirical interest, as similar involvement projects have not provided data to carry out such assessment, and hence this forms the key contribution of this article. Table 3 lists the names of the research priorities formulated by the experts. The experts’ ranking of the priority is presented next to a combined citizen rank from the 12 European countries. The combined citizen rank has been balanced so that each country panel has identical weight. Appendix 2 presents country-specific citizen rankings in addition to the expert rank, the combined citizen rank and the statistical correlations between them.

Table 3. Expert and combined citizen rankings of research priorities (Matschoss et al. 2015; Repo, Kaarakainen, and Matschoss 2015).

Research priority	European expert rank	Citizen rank in 12 countries
Improvement of European electricity transmission to increase renewable energy production	1	12
Research on business models and changing institutions related to sustainable energy economy	2	18
Sustainable living environment	3	14
Holistic education for a sustainable future	4	2
A new European food culture	4	15
Access to natural resources as a human right	6	21
Co-developing green technology	7	24
Sustainable economics	8	16
Unified ecological grading system	8	17
Sustainable transformation of existing traffic infrastructure in cities	10	5
Supporting people to become producers of renewable energy	11	3
Supporting an active civil society for sustainable development	12	19
New working models – new economic models	13	6
Sustainable construction of buildings	14	4
Fair and participatory access to limited resources	14	10
Understanding and implementing sustainable electronics	16	9
Innovating agriculture: the sustainability option	16	7
New spaces for public discourse	16	26
Supporting local/regional agricultural production, distribution and consumption system	19	1
Supporting Eco-preneurship	19	20
Collaboration through shared space	21	23
Impact of virtual communities in behaviour change	22	25
Ensuring inclusive and dynamic city centres	23	13
Enhanced physical activity for better quality of life and energy efficiency	24	11
Exploring the introduction of insect food	25	27
More green in cities	26	8
Research on individual urban farming	27	22

Table 3 demonstrates striking differences between the expert and combined citizen rankings. For instance, only one research priority appears in the five most highly ranked on both lists, with just one more featuring in the most highly ranked 10 (‘Holistic education for a sustainable future’ and ‘Sustainable transformation of existing traffic infrastructure in cities’, respectively). By contrast, many of the priorities which citizens rank among the top 10 are ranked lower by the experts, and vice versa. Nevertheless, similarities between citizens and experts are evident in the lower ranked priorities.

Statistical analysis in the form of Spearman's rank-order correlation coefficient was applied to further examine the differences in expert and citizen ranking of the research priorities. The coefficient is a non-parametric measure and expresses the strength of correlation in the quantitative rankings of two different datasets. It is suitable for the analysis of statistical data which are not normally distributed (Hauke and Kossowski 2011; Hotelling and Pabst 1936). The data must conform to two assumptions: that they are interval, ratio level or ordinal in their level of measurement, and that they are monotonically related. These assumptions are fulfilled for our ranking data on research priorities.

Spearman’s rank-order correlation delivers a measure enabling the assessment of the strength of the correlation between two data sets. It is based on computing the sum of the squared differences of the items in the data sets (Fieller, Hartley, and Pearson 1957). The coefficient ranges between −1 and 1, where −1 indicates a complete negative correlation, 0 indicates no correlation and 1 indicates complete positive correlation. Cohen's standard is used to assess the strength of the correlation. A correlation coefficient below .30 represents a small association, a correlation between .30 and .49 a medium association, and one above .50 a large association (Cohen 1988).

In the present study, Spearman’s rank-order correlation was applied to determine the relationship between the expert rankings, the combined European citizen ranking and the citizen rankings in each country (see Appendix 2). The results of the analysis demonstrate that the expert rankings of research priorities indeed differ from those of the citizens. First, no statistically significant association was observed between the combined European citizen ranking and the expert ranking (r_s = .145, p = .470, 2-tailed). Furthermore, none of the 12 citizen rankings were statistically significantly associated with the expert ranking.

By contrast, most of the citizen rankings were statistically significantly associated to the combined European citizen ranking, indicating an emerging European citizen perspective on research priorities. A strong association was found in six countries (r_s = .531−.760, p < .01, 2-tailed) and a medium association in three countries (r_s = .460−.493, p < .05, 2-tailed). The correlations for the remaining three countries were not statistically significant at the level of .05.

In sum, no statistically significant association between expert rankings and citizen rankings could be observed. On the other hand, an emerging and widely spread citizen perspective could be identified, as citizen rankings in nine of the 12 countries correlated with the combined European ranking. Indeed, the statistical analysis confirms that the experts had quite different preferences from those of the citizens. This further indicates that experts are liable to take over the contributions of citizens even when those contributions have been formulated into actionable priorities.

Implications of differences

The case study emphasises that citizens and experts can indeed approach the future from largely different perspectives. In the study, citizens acknowledged that the priorities formulated by the experts were quite faithful to their originating visions (Bedsted et al. 2016), although more detailed analysis shows screening effects and varying prioritisations. Nevertheless, differences in perspectives should not be seen as an unwanted outcome but rather as the essential motivation for engaging both citizens and experts in the first place. Such differences indicate the value of involving both parties when attempting to broaden the development of priorities for forward-looking exercises, such as research and innovation activities.

By contrast, it is the management of the different perspectives of citizens and experts which merits attention. The challenge is then to account for the particular strengths of both parties while still adhering to the task at hand, which in the case study was citizen involvement rather than expert assessment. When the task is to identify future directions desired by citizens, an emphasis on citizen contributions is of key importance. In contrast, if the task requires familiarity with highly complex settings (i.e. expert knowledge), then it would appear worthwhile to emphasise the contributions of experts.

Even so, in this involvement process (as in many others), the knowledge base is jointly built by citizens and experts, which should be reflected in how that knowledge is used and reported. Accordingly, when focusing on the contribution of one party, it would be valuable to consider briefly the contribution of the other. For instance, when arguing for the top expert priority on electricity transmission and renewable energy sources, it could be disclosed that citizens ranked this only 12th in their prioritisation, and the implications of this difference should also be discussed. Similarly, when arguing for technically oriented or systemic priorities, it should be disclosed that citizens in general prefer futures with social or personal characters. Furthermore, if only the expert contribution is valued, then the citizen contribution should be downplayed instead of being used as an additional key argument or legitimisation for expert preferences. The most critical issue from the perspective of responsible research and innovation (RRI) is to ensure that expert viewpoints do not unjustifiably base their legitimacy on citizen involvement.

The findings further challenge knowledge creation processes that begin with agenda setting by citizens and are followed by formulations by experts, thereby emulating public policy development (see Howlett, Ramesh, and Perl 2009). When applying such processes, methodologies or procedures should be established to support the comparison of the differing viewpoints of citizens and experts. The case study explored in this article demonstrates that establishing such comparative methodologies need not be a laborious task.

Discussion and conclusions

Citizen involvement has become one of the key features of responsible research and innovation (RRI). Consequently, it is central to ensure that any contribution originating from citizens receives the appropriate attention. This is by no means an easy task, as citizen involvement typically requires translation by experts in order for it to be utilised in subsequent activities. However, each such stage of translation provides an opportunity for expert takeover of the citizen involvement contribution. Accordingly, it has been suggested that translation quality should be used as an evaluation criteria for engagement processes (Horlick-Jones, Rowe, and Walls 2007). As previously mentioned, reliance on experts becomes particularly problematic if the legitimacy of their viewpoints is drawn from citizens.

This article has approached the conundrum of accomplishing responsible citizen involvement while having experts to translate its contributions into useful outputs for subsequent use. After all, it is precisely this expertise which is required to develop the contributions arising from citizen involvement. To examine this empirically, the article reviewed a case study in which citizens in 12 European countries articulated visions for the future, which experts formulated into research priorities for the European Union's Horizon 2020 framework programme for research and innovation. The case study demonstrated that even when experts are committed to faithfully following the spirit of citizen contributions, screening effects and varying prioritisations are liable to lead to expert takeover. The involvement design applied in the examined CASI project made the differences between citizens and experts explicit, which is not always the case in citizen involvement. Indeed, differences may remain tacit unless they are actively accounted for. The results showed that experts preferred technically oriented, systemic citizen visions over societal and personal ones and that their assessment of priorities differed from that of citizens. Such prioritisation works against the notion of providing multiple policy options that respect legitimate stakeholder differences in values and world views (Evans and Plows 2007; Linnerooth-Bayer et al. 2016; see also Wynne 2006). The results support the idea that policy advice which considers citizens as legitimate actors indeed differs from that offered by experts (Rosa, Gudowsky, and Warnke 2018).

Accordingly, the article challenges the notion of citizen involvement as simply one processual stage and instead prompts practitioners to utilise citizen contributions fully throughout the involvement and formulation process (see Kiran, Oudshoorn, and Verbeek 2015 on principles rather than checklists). Responsibility in research and innovation should then account for both the process and the outcome (de Saille 2015; Sutcliffe 2011). Using citizen contributions as a point of reference for subsequent RRI activities appears a key integrative way to address the challenge of incorporating citizen contributions into the entire RRI process. In practice, this would require reflection on how expert contributions relate to insights from citizen involvement, the provision of an independent and balanced assessment on which contributions come from citizens and which from experts, or simply an open reporting of the true significance of the contribution of citizens.

Resisting the expert takeover of the citizen involvement process is particularly required if the citizen contribution is of key importance for the legitimacy of the outcomes. Nevertheless, as experts, by definition, rely on the best available knowledge, it is arguably more sensible to make use of their expertise than to attempt to spurn it. Moreover, acknowledging the tension between citizens and experts is likely to lead to the development of processual measures that resist expert takeovers. The empirical case study examined here included such feedback loops between citizens and experts (Bedsted et al. 2016).

The involvement of citizens in the realm of responsible research and innovation creates more benefits than simply enhanced inclusion and accountability. For example, where citizen and expert viewpoints differ, the former could be used to challenge conventional practices which rely on established expertise. This is particularly the case when targeted innovations do not yet seem useful (see Christensen 1997) and when the need arises to construct or transform new strategic action fields (Fligstein 2013). Citizen involvement could, accordingly, be used to disrupt traditional agenda setting and thereby contribute to change and diversity as well as accentuate a shift away from technological emphasis (see Kahane et al. 2013; Stirling 2008). Finally, the identification of issues which are difficult for citizens to accept, however important they may seem to experts, provides recognition of the need to develop alternative solutions to such issues (see de Saille 2015).

Citizen involvement has become an established procedure in responsible research and innovation, and for good reason. It strengthens the connection between citizen concerns and agenda setting in research and innovation, fulfils the goal of societal inclusion in the field, and is apt to increase citizen acceptance for the innovations of the future. Alongside these benefits, it also introduces a disruptive element to the formulation of research priorities and creates a tension between citizens and experts. This tension might be the most exciting contribution of citizen involvement, and its careful management indeed strengthens the role of responsibility in research and innovation.

Data availability statement

The data sets used in this article are available in reports which are listed in the reference list and are available at www.casi2020.eu.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Dr. Petteri Repo works as a Research Director at the Centre for Consumer Society Research at the University of Helsinki. He has lead research groups focusing on living environments and innovations which have addressed topics relating to user-oriented technology assessment and user innovations for a sustainable knowledge society. He holds a PhD (Economics) from the Swedish School of Economics and Business Administration in Finland and is Adjunct Professor (Docent) at the Aalto University School of Business. His current research interests include participatory foresight, user driven innovation, consumer studies, policy analysis, and natural language processing.

Dr. Kaisa Matschoss works as a University Researcher at the Centre for Consumer Society Research at the University of Helsinki. Her research interests lie in energy issues, innovations and public engagement for sustainability as well as sustainable consumption. She holds a PhD (Economics) from the Carl von Ossietzky University Oldenburg in Germany and is Adjunct Professor (Docent) at the University of Eastern Finland. Currently she studies smart energy transition and the role of intermediary organisations in it, energy-related experimentation and engagement of households in changing their practices towards reduced energy consumption.

ORCID

Petteri Repo http://orcid.org/0000-0002-0298-5718

Kaisa Matschoss http://orcid.org/0000-0002-4045-6729

Additional information

Funding

The CASI (Public Participation in Developing a Common Framework for Assessment and Management of Sustainable Innovation) project was funded in the 7th Framework Programme of the European Commission FP7 Science in Society (Grant agreement number: 612113).

Appendices

Appendix 1

Citizen visions selected and not selected for research priority formulation by experts according to their topics. Tables 1 and 2 indicate the topical distribution of each vision (total = 1, i.e. 100%).

Table A1. Citizen visions selected for research priority formulation.

Vision	Society and change in human life	Quality of food products	Energy production and renewables	Personali-sed working time	Public transport network in a green city	Education for sustaina-bility	Urban farming
Active civil society for sustainable development	0.727	0.037	0.001	0.001	0.001	0.234	0.000
Ecopreneurship – Sustainable business for the future	0.698	0.094	0.002	0.002	0.201	0.002	0.001
Recognition, rethinking responsible governance / action	0.636	0.001	0.001	0.001	0.001	0.359	0.001
Education – a path to spiritual and sustainable future	0.558	0.001	0.001	0.001	0.001	0.428	0.011
Sustainable living environment, sustainable values	0.516	0.140	0.176	0.000	0.079	0.000	0.088
Eco credits	0.414	0.304	0.002	0.041	0.238	0.001	0.001
Sustainable agriculture	0.127	0.852	0.001	0.001	0.001	0.019	0.000
EUCRES – EU collaboration for recycle systems	0.084	0.764	0.070	0.001	0.001	0.060	0.020
Insects – the dish of the future	0.269	0.728	0.001	0.001	0.001	0.001	0.000
Food for all	0.209	0.567	0.119	0.001	0.103	0.001	0.001
Sustainable electronics	0.307	0.563	0.001	0.126	0.001	0.001	0.001
Self-supply with healthy food	0.042	0.460	0.338	0.000	0.008	0.000	0.151
Conflict free distributive justice	0.278	0.421	0.001	0.251	0.001	0.047	0.001
Sharengy – sharing renewable energy sources	0.105	0.001	0.892	0.001	0.001	0.001	0.000
Distributed small-scale energy generation in mainstream within 30–40 years	0.109	0.000	0.885	0.002	0.000	0.000	0.002
New sustainable energy economy	0.195	0.071	0.672	0.001	0.061	0.001	0.000
The sustainable construction of buildings	0.167	0.271	0.555	0.002	0.002	0.002	0.001
Development of new technologies and improvements of the existing in harmony with nature and society	0.365	0.110	0.523	0.000	0.000	0.000	0.000
1/2 d labour	0.213	0.006	0.011	0.759	0.011	0.001	0.000
From physical activity to electricity	0.207	0.001	0.256	0.535	0.001	0.000	0.000
The city my home / home in the city	0.093	0.001	0.014	0.001	0.891	0.000	0.000
More green in the city	0.069	0.005	0.001	0.001	0.785	0.000	0.139
Reducing traffic congestion through the creation of green transport corridors and the protection and development of open and recreational spaces	0.005	0.178	0.045	0.010	0.734	0.001	0.027
Network for a world as home	0.342	0.001	0.062	0.069	0.438	0.087	0.000
Think coloured	0.291	0.001	0.001	0.079	0.027	0.600	0.001
Education = aware citizen = aware society = sustainability	0.459	0.019	0.001	0.041	0.000	0.480	0.000
Urban farming	0.030	0.000	0.000	0.000	0.000	0.000	0.969
Number of visions in a topic	6	7	5	2	4	2	1
% of selected visions for research priority formulation	33	78	83	50	80	33	50

Table A2. Citizen visions not selected for research priority formulation.

Vision	Society and change in human life	Quality of food products	Energy production and renewables	Personali-sed working time	Public transport network in a green city	Education for sustaina-bility	Urban farming
Living in community	0.817	0.002	0.001	0.014	0.151	0.001	0.013
Eco2Social Industry in 2050	0.676	0.135	0.127	0.035	0.018	0.009	0.001
Global solidarity based on volunteering technological development and regulated distribution of resources	0.638	0.041	0.000	0.251	0.000	0.069	0.000
The happy life. Healthy and contending life as the driver of a holistically sustainable development	0.638	0.037	0.001	0.323	0.001	0.001	0.000
Societal reset	0.634	0.044	0.001	0.001	0.001	0.301	0.019
Human world	0.628	0.041	0.001	0.011	0.001	0.317	0.001
Healthy living	0.597	0.001	0.001	0.381	0.019	0.001	0.001
Homo faber	0.578	0.001	0.001	0.253	0.000	0.167	0.000
Society of understanding (empathic)	0.553	0.001	0.000	0.000	0.425	0.000	0.020
Distributive justice of essential resources	0.551	0.001	0.001	0.445	0.001	0.001	0.000
Clean nature for a better quality of life	0.524	0.346	0.122	0.001	0.007	0.001	0.000
Beauty will save the world	0.505	0.002	0.001	0.429	0.001	0.001	0.062
Vision of quality	0.029	0.969	0.001	0.001	0.000	0.000	0.000
Cannabis utopia	0.200	0.481	0.159	0.039	0.012	0.001	0.110
Energy for humanity ecosystems preservation	0.415	0.002	0.558	0.001	0.001	0.022	0.001
Supporter of body and mind (IPHA – intelligent personal health adviser)	0.053	0.001	0.183	0.739	0.000	0.000	0.023
Society of potential capacities	0.370	0.001	0.001	0.606	0.020	0.001	0.001
Optimal living together in the city and surrounding areas	0.167	0.029	0.025	0.004	0.556	0.000	0.218
New ways for sustainable education	0.084	0.001	0.001	0.001	0.001	0.908	0.006
Assets of the planet on the school curriculum	0.150	0.002	0.001	0.001	0.034	0.810	0.001
Facing immigration of nations	0.167	0.001	0.086	0.000	0.000	0.745	0.000
Union of the earth – World without the borders	0.489	0.002	0.001	0.001	0.001	0.505	0.001
Urban farm	0.052	0.001	0.016	0.000	0.008	0.000	0.923
Number of non-selected visions in a topic	12	2	1	2	1	4	1
% of not selected visions for research priority formulation	67	22	17	50	20	67	50

Appendix 2

Table A3. Citizen and expert rankings of research priorities.

Research priority	COMBINED CITIZEN RANK	AT	BE	BG	CZ	DK	FI	GE	IT	PL	PR	SI	UK	EXPERT RANK
Supporting local/regional agricultural production, distribution and consumption system	1	8	1	2	4	3	5	6	1	2	6	2	5	19
Holistic education for a sustainable future	2	1	5	1	8	8	15	1	15	8	7	1	2	4
Supporting people to become producers of renewable energy	3	3	10	8	8	13	1	4	5	11	3	11	1	11
Sustainable construction of buildings	4	19	3	3	3	1	2	19	12	1	1	3	14	14
Sustainable transformation of existing traffic infrastructure in cities	5	14	1	6	5	13	7	8	5	11	1	20	14	10
New working models – new economic models	6	8	7	11	14	5	11	23	2	11	7	7	10	13
Innovating agriculture: the sustainability option	7	3	13	4	16	8	7	13	15	18	3	15	8	16
More green in cities	8	19	22	11	1	2	20	15	12	2	7	11	14	26
Understanding and implementing sustainable electronics	9	6	3	19	5	5	2	19	4	17	12	15	25	16
Fair and participatory access to limited resources	10	2	17	21	25	21	7	2	21	11	19	3	5	14
Enhanced physical activity for better quality of life and energy efficiency	11	26	25	9	2	5	11	27	15	18	3	5	10	24
Improvement of European electricity transmission to increase renewable energy production	12	15	5	6	21	3	15	23	24	5	10	9	12	1
Ensuring inclusive and dynamic city centres	13	16	10	11	8	8	11	23	10	16	15	15	12	23
Sustainable living environment	14	8	27	11	13	8	5	8	15	18	11	20	19	3
A new European food culture	15	19	17	19	8	16	15	8	24	5	15	20	2	4
Sustainable economics	16	11	7	21	25	8	2	3	15	24	27	11	14	8
Unified ecological grading system	17	5	17	26	23	21	7	15	5	24	19	10	2	8
Research on business models and changing institutions related to sustainable energy economy	18	19	13	21	21	21	15	4	10	11	19	23	5	2
Supporting an active civil society for sustainable development	19	11	17	5	16	26	20	8	20	2	19	26	14	12
Supporting Eco-preneurship	20	6	22	15	8	13	26	13	9	22	15	15	25	19
Access to natural resources as a human right	21	11	13	9	14	21	20	8	21	26	12	7	24	6
Research on individual urban farming	22	16	13	25	16	21	11	6	21	22	15	5	19	27
Collaboration through shared space	23	26	10	16	16	16	27	19	24	5	12	24	8	21
Co-developing green technology	24	19	22	16	16	16	15	15	12	18	19	11	19	7
Impact of virtual communities in behaviour change	25	19	25	18	25	20	20	23	8	8	19	15	19	22
New spaces for public discourse	26	19	7	26	24	27	20	19	3	8	19	26	25	16
Exploring the introduction of insect food	27	16	17	24	5	16	20	15	24	26	19	24	19	25
Correlation with aggregated citizen rank	1	.460*	.493**	.672**	.492**	.689**	.646**	.197	.318	.367	.760**	.531**	.524**	.0145
Sig. (2-tailed)	-	0.016	0.009	0.000	0.009	0.000	0.000	0.324	0.106	0.059	0.000	0.004	0.005	0.470
Correlation with expert rank	.0145	.263	.152	.154	−.305	−.093	.187	.364	−.093	.023	−.054	.006	.268	1
Sig. (2-tailed)	0.470	0.185	0.449	0.444	0.122	0.643	0.350	0.062	0.643	0.908	0.789	0.976	0.177	-

Note: The combined citizen ranking is calculated from equalised priority scores from each country.

*p < .05, **p < .01