3,361
Views
7
CrossRef citations to date
0
Altmetric
Research articles

IMAGINE RRI. A card-based method for reflecting on responsibility in life science research

ORCID Icon, ORCID Icon & ORCID Icon
Pages 201-224 | Received 07 Jul 2017, Accepted 14 Mar 2018, Published online: 06 Apr 2018

ABSTRACT

Responsible Research and Innovation (RRI) has become a new buzzword in science policy, pointing to a shift in the role of research in contemporary societies. While on a discursive level responsibility is easily welcomed, implementing RRI in research practice appears challenging. RRI as an agent for change must compete with other forces shaping the current research system and its institutions, such as innovation orientation, competition and indicator-driven evaluation cultures.

To address these challenges, we created a new format for engaging life science researchers in reflections on the meaning of responsibility in their own research practices. In this conceptual paper, we present and discuss a card-based method: IMAGINE RRI. The method’s aim is twofold. First, it is meant to empower researchers to appropriate RRI through shared reflection while connecting it to their practices. Second, it aims to enable researchers to reflect on how the institutional context of their work and the embedded values fosters or hinders responsible research practices.

Supplementary material (including card design and discussion map) is available under a creative commons license at the following link: https://phaidra.univie.ac.at/view/o:690945.

Introduction

Responsible Research and Innovation (RRI) has become a buzzword at the core of science policy discourses in Europe and beyond (de Saille Citation2015). Its appearance and its embrace (at least discursively) by policy makers can be read as pointing to a shift in the role of research in contemporary societies: it is expected to become better aligned with or even driven by societal concerns (Bos et al. Citation2014; Guston and Stilgoe Citation2017). RRI did not appear on a blank slate, but had important predecessors including research on the ethical, legal and social implications (ELSI) or aspects (ELSA) of the production of knowledge and innovations (Hilgartner, Prainsack, and Hurlbut Citation2017). However, after more than a decade of ELSA/ELSI in practice, critical voices highlighted that it often led to a division of labour between researchers, creating knowledge and technology innovations on the one hand and, interdisciplinary ELSA/ELSI scholars, taking care of societal aspects on the other hand (Felt et al. Citation2013; Rip Citation2016). Additionally, ELSA/ELSI research was seen as in danger of being hijacked by dominant innovation logics, smoothing out societal concerns rather than raising them (Zwart, Landeweerd, and van Rooij Citation2014).

RRI set out to address some of these problems. Indeed, in RRI, researchers are perceived as the key actors expected to engage with societal values and concerns and devote more attention to the role of these values and concerns in research practices. Researchers are expected to cultivate the capacity to respond – of being ‘response-able’ (Felt Citation2017) – to societal challenges and engage with societal values and expectations throughout the process of creating innovations. The broader aim is to move explicit as well as tacit forms of governance away from ‘reactive forms […] to proactive forms’ of addressing societal concerns (Ribeiro, Smith, and Millar Citation2017). However, we know from previous initiatives aiming to address the social and ethical dimensions of research that these engagements ‘cannot be externally imposed … [but] must be nurtured by scientists … themselves’ (Wilsdon, Wynne, and Stilgoe Citation2005). To be effective on an epistemic level, i.e. in shaping the direction and content of research, reflexivity must thus become an embodied skill. It must be part of the tacit knowledge of designing and carrying out complex and often unpredictable research processes.

While on a discursive level, responsibility is easily welcomed as a value already deeply embedded in the self-understanding of researchers and research institutions, implementing RRI in everyday practice appears more complex and challenging for at least two reasons. First, to work as a boundary object across many different communities and sites, the concept of RRI remains rather vague. This, however, leaves RRI practitioners with the challenge of rendering the notion of responsibility and its multiple meanings sufficiently concrete and developing context-specific devices to make RRI a useful concept in research practice. Second, RRI comes at a time when innovation has been assigned a key role in the development of our societies, while excellence and competition and their accompanying indicator-based logics have simultaneously become driving values in the research system. This means that any successful RRI activity must find a way of making RRI a core element in research practice, despite competing values. If this can be achieved, RRI-related work could potentially play the role of ‘a “moral glue” that holds the often simultaneous yet potentially contradictory promises of economic, societal and scientific benefits together’ (Felt Citation2017). Failing to create a collaborative environment to address responsibility in research and innovation may again lead to an outsourcing of reflection to RRI scholars, a ritualization of specific performances at the beginning or the end of research projects (e.g. outreach events), or the rise of a ‘new bureaucracy of virtue’, i.e. the creation of new forms with RRI boxes to be ticked (Felt et al. Citation2013; Felt Citation2017).

A variety of approaches to integrating social and ethical aspects into research and innovation practices exist in the literature (see Fisher et al. Citation2015 for an overview). However, the majority of these approaches, such as socio-technical integration research (STIR) (Fisher and Schuurbiers Citation2013) or constructive technology assessment (CTA) (Rip and Robinson Citation2013), build on a time-intensive process of engagement with a particular research field or group. Due to their methodological approach, they address specific fields or institutional actors and do not necessarily provide tools that can easily be put to broader use in research contexts (e.g. in PhD and post-doc training or in team building contexts) without substantial external funding for RRI activities.

To address these challenges in our work in an interdisciplinary research platform on ‘Responsible Research and Innovation in Academic Practice’ at the University of Vienna, we created a new, rather open and playful (van der Meij, Broerse, and Kupper Citation2017) format for engaging researchers in basic life science research in reflections regarding the multi-layered meanings of responsibility in their own research and innovation practices. Our aim was to create a method that required relatively little time investment by participating researchers, empowered researchers to appropriate the concept of RRI through shared reflection, and enabled them to connect their practices to wider and sometimes abstract RRI concepts and debates. Our method also aimed to foster reflection on how the institutional context of researchers’ work enables or hinders their engagement with ethical and social aspects of their work. This allows us to perceive the limits of RRI approaches and to think about needed changes in the ‘responsibility conditions’ (Felt Citation2017) in which contemporary research is taking place.

In this conceptual paper, we present and discuss this method, known as ‘IMAGINE-RRI’. We adapted the card-based discussion method ‘IMAGINE’ (Felt et al. Citation2014; Felt, Schumann, and Schwarz-Plaschg Citation2018) originally developed as an engagement method for nanotechnologies that allowed citizens with little to no prior experience to access and participate in debates on complex socio-technological issues. It does so by offering citizens a ‘narrative infrastructure’ (Felt Citation2017) with which to talk about these issues. Since researchers may similarly lack experience in addressing issues of responsibility, the idea was to create a card-based approach that opens a space in which researchers are provided an infrastructure that enables them to navigate the wide and often loosely coupled discursive landscapes around RRI.

As we will describe, our method builds on different sets of cards presenting short texts and visual elements and a choreography of how to engage with the cards. The cards sketch different facets of responsibility in the life sciences from different perspectives (statements, practices, contexts). Through inviting researchers to (1) select cards that speak to them and their experiences, (2) develop their ideas and narratives around them and (3) do so in exchange with each other, we aim to gradually construct a shared space in which the abstract concept of responsibility assumes more concrete meaning.

The aim of this paper is to present our method, situate it in the existing literature on RRI interventions, and make it accessible to a community of practitioners. In what follows, we first discuss the wider methodological debates in which our approach is embedded. Second, we describe the genealogy of the card-based method and the concept of narrative infrastructure it builds on. The core part of this article then sketches the basic structure, aim and timing of the discussion. We describe the different card sets used in the three rounds of debate, commenting on the leading question, the basic structure of the cards and how they were created. Next, we discuss the organization of the discussion, its facilitation, and the ideal number and profile of participants. Finally, we illustrate the discussion dynamics by providing examples of actual exchanges between researchers to show how the method allows researchers to engage and interact with issues of responsibility in life science research. In the spirit of an open method publication, we publish the original card sets, discussion board and development documentation along with this paper as supplementary materials. We encourage and welcome interested RRI researchers and professionals to use and adapt our approach.

From public engagement to RRI: methodological reflections

The method we developed is embedded in a diverse and lively field of efforts aiming to address the complex relations of science and society in contemporary research. Under the umbrella of ELSI/ELSA of research are a range of efforts and formats designed to either engage citizens in reflecting on science and technology and its role in contemporary societies or – although to a lesser extent – encourage researchers and engineers to consider societal aspects in their work more closely (Hilgartner, Prainsack, and Hurlbut Citation2017). These efforts date back to the 1990s and have spread from human genome research to many other domains, such as nanotechnology or synthetic biology. We encounter a flurry of activities engaging citizens and addressing their visions and concerns, for which many innovative methodologies have been developed. In addition to classical debate formats, new, more experimental approaches have been tried out such as scenario-building workshops, card-based discussion groups, or walkshops (Macnaghten and Guivant Citation2011; Felt, Schumann, and Schwarz Citation2015; Wickson, Strand, and Kjolberg Citation2015). Much can be learned from these exercises, such as how to trigger a structured debate while leaving it sufficiently open for flexible and context-specific reflections, or the importance of taking people out of their usual social settings to disrupt established ways of thinking and facilitate a change in perspectives — while being aware that participants generally appropriate these discussion spaces and use them in ways that seem best for them (Felt and Fochler Citation2010).

Although far less effort was made to engage researchers in comparison to citizens in ELSA/ELSI contexts, a range of important approaches exist that aim to foster the integration of social and ethical aspects into research practices, such as ‘midstream modulation’ (Fisher, Mahajan, and Mitcham Citation2006), STIR (Fisher and Schuurbiers Citation2013), CTA (Rip and Robinson Citation2013) or ‘ethics on the laboratory floor’ (van der Burg and Swierstra Citation2013).

Among the approaches mentioned, CTA has the longest tradition (Rip and te Kulve Citation2008; Rip and Robinson Citation2013). CTA engages different actors related to a specific emerging technology in a discourse on potential future scenarios, and thus aims to raise actors’ awareness of the ethical and social dimensions of their practices. Particularly later approaches to CTA emphasise the need to reflect on societal effects of innovations (cf. Schot and Rip Citation1997; Rip and te Kulve Citation2008). However, its focus on future technological development in a specific area may render it difficult to apply in fields of fundamental research in which technological development pathways are too remote, yet too unclear or too diverse. Additionally, the method requires a considerable investment of time and resources. The CTA process needs social science researchers to first carry out detailed preliminary studies of the emerging technological field to be addressed. These studies form the basis of scenario building. These scenarios, in turn, are discussed in workshops with central actors in the innovation process with the aim of fostering reflection and learning. Reflection on the conditions and practices of knowledge production itself often remains rather marginal.

Collaborative integration approaches (Fisher et al. Citation2015) set out to reshape the research process itself by reflecting on researchers’ strategic decisions as they are made, often through embedding social scientists or humanities researchers in the actual research laboratories and processes. Their primary aim is to foster reflection on how techno-scientific and societal developments are intertwined in the processes of creating knowledge and innovations. They can successfully increase the capacity of researchers to integrate social considerations into their daily work (Fisher, Mahajan, and Mitcham Citation2006), cause scientists to learn to look at their research through different lenses (Bensaude Vincent Citation2013) and render ongoing decision-making processes more transparent, thus potentially changing the course of research (Fisher Citation2007). However, these approaches have also proven to be very time-consuming and subject to socially and ethically sensitive negotiation processes. This may limit their impact to a relatively small number of labs. It has also been noted that in such approaches, reflexive work is often tacitly delegated to social scientists or scholars from the humanities (Bensaude Vincent Citation2013; Viseu Citation2015), leading to a division of work (Rip Citation2016) and sometimes to open conflict (Rabinow and Bennett Citation2012).

With the advent of RRI, the question of how to better engage researchers in reflecting on issues of responsibility has again been placed higher on the agenda. We have seen studies that reflect on researchers’ imaginations and conceptualisations of responsibility and societal relevance (e.g. Glerup and Horst Citation2014; Glerup, Davies, and Horst Citation2017; Rosenlund, Notini, and Bravo Citation2017). Others, notably in the European Research Framework projects (e.g. RRI tools, HEIRRI, IRRESISTIBLE), have sought tools and methods that could become embedded in academic routines on a larger scale. The recent FP7 project ‘RRI tools’, for example, provides a collection of critical questions,Footnote1 a self-reflection tool,Footnote2 training materialFootnote3 and a search engine for tools and inspiring practices (Groves Citation2017). While this is a potentially valuable collection of efforts, little is known so far about how these approaches actually engage with researchers’ situated thinking about responsibility and about how context-sensitive they are. We see our card-based discussion approach as an addition to this broad agenda of developing methods that can stimulate researchers’ capacity to reflect on the social and ethical aspects of their work and can be applied and adapted relatively widely with limited effort. It has the potential to be integrated into different learning environments.

IMAGINE RRI: methodological and theoretical development

The card-based discussion method IMAGINE RRI presented in this paper is rooted in a long tradition of using card-supported discussion methods in the social sciences and also more specifically builds on previous experiences in developing and using the card-based engagement method IMAGINE in citizen engagement (Felt et al. Citation2014; Felt, Schumann, and Schwarz-Plaschg Citation2018).

The use of cards has a history in qualitative research, particularly in interviews and focus groups. They have been used to facilitate discussion regarding sensitive issues (e.g. Sutton Citation2011) or as part of ‘focusing’ or ‘ranking exercises’ to analyse people’s ways of ordering and classification (Kitzinger Citation1994; Bloor et al. Citation2001). Card-based discussion formats, such as PlayDecide (e.g. Bandelli Citation2010), were meant to foster debate and decision-making on specific issues of concern following an initial information phase. Related to research and innovation, card-based formats, such as ‘dilemma games’,Footnote4 were developed to trigger reflection on issues of research integrity (see van der Meij, Broerse, and Kupper Citation2017 for an overview of playful methods in RRI). Examples from engineering education show how such card-based approaches are suited to training with regard to decision-making capacity in complex work processes (He and Adar Citation2017). Similarly, design communities use ‘ideation cards’ to stimulate creative thinking and reflection on how to put complex legal requirements (e.g. ‘privacy by design’) into practice (Luger et al. Citation2015). Related to cards, concept boards presenting different visual and textual elements (e.g. media articles, illustrations, interview sequences or policy statements) are also used as a resource in group discussions (e.g. DEEPEN Citation2009).

Reflecting this wider background, we built specifically on previous experiences with the card-based discussion method IMAGINE. IMAGINE was developed to allow citizens to engage with nanotechnology. Developed in a context in which citizens might be expected to struggle to find discursive reference points with which to discuss nanotechnologies, its key idea is to allow them to enter the debate by providing them with different types of cards developed out of specific narrative elements in the wider nanotechnology discourse. Together, these different cards offer a narrative infrastructure participants can relate to in forming their perspectives and opinions. Participants can choose from and relate to the many short narrative inputs presented on the card sets. They can use these narratives to either support their argument or to highlight contrasting opinions. The method thus aims to give participants the space to appropriate narrative resources and work with them.

On a theoretical level, our approach was guided by the concept of narrative infrastructure.Footnote5 This concept draws attention to the fact that actors – be they citizens talking about nanotechnology or researchers talking about responsibility – use prevailing narrative resources to make sense of complex concepts such as responsibility and to relate them to their practices. They relate to elements of a publicly available ‘network of temporally stabilized narratives’, gravitating around the different aspects of responsibility. Using the notion of infrastructure, thus draws our attention to both the stability as well as the materiality (i.e. the different media through which narratives take form, circulate and can stabilize) of the narratives which are used by researchers when trying to make sense of what they do.

Such ‘narratives can take different forms, including assessments, reconfigurations of past developments, future-oriented accounts voicing promises […] but also potential threats, and moral reflections of what is good science and innovation and how a good researcher should be’ (Felt Citation2017). At the same time, they also encode the hopes and expectations of individual and institutional actors and thus become ‘the vehicles whereby [values, hopes and expectations] are transmitted and made emotionally real’ (Larkin Citation2013). Narratives are crucial in constituting a broader sense of direction, value and purpose in academic work as key elements in the making of individual and institutional identities and powerful forces enabling or constraining researchers’ actions (Czarniawska Citation2004; see also van der Burg Citation2016).

However, like citizens unaware of the many different facets of nanotechnology futures narratives, most researchers will also at best be marginally in touch with the existing variety of discourses on responsibility. They may be aware of some topics hotly discussed in their own communities and institutions, but are less acquainted with other dimensions of RRI. Also, some RRI discourses may be too abstract for researchers to relate them to their actual work. Our card-based discussion method aims to provide participating researchers with a broad set of narratives that provide the infrastructure they can navigate and use in their sense-making practices throughout the debates. In the context of IMAGINE RRI, the elements of this narrative infrastructure – the cards – have been developed out of different discursive strands that gravitate around science-society relations and values in research. We will describe this in more detail in the following sections.

The idea of the card-based discussion is to provide researchers with this narrative infrastructure in a way that allows them to approach, interpret and use it creatively to formulate their own positions and opinions. Facilitating a collective discussion, we hope, leads to exchanges between researchers on the meanings of their role at the interface of science and society and on the multiple issues of responsibility in their concrete work practices. Unlike in scenario work, for example, which would invite people to move within or at least begin from a fixed scenario, this approach allows participants to build their positions and imaginations more flexibly. Building the discussion in a form that resembles an open card game allowed participants to relate to their experiences of playing board games. As in board games, our card-based discussion method has rules that can be interpreted and played out in different ways and fellow participants are conceptualized as partners even though they may have different strategies and aims.

The basic idea in developing IMAGINE RRI was to provide an infrastructure that allows participants to address issues for which they may lack the narrative repertoire and also to leave as much space as possible for participants’ own agency in developing their positions. We had three specific aims in doing so: (a) rendering a rarely debated issue accessible to all participants without too much pre-framing; we wanted to break down responsibility, which is often seen as a very large and opaque issue, into the many context-specific micro-moments in which it can come into play in life science research; (b) supporting participatory justice; we wanted to empower people with different experiences in research and in different hierarchical positions to speak up and engage in a collective discussion; many crucial RRI issues are deeply related to people’s everyday work, and an open debate may be regarded as difficult for those in more fragile positions; (c) avoiding the classical lay/expert divides. In particular, we aimed to avoid positioning ourselves—the conducting social scientists—as ‘RRI experts’ tutoring the participants. The method aimed to provide a space in which participants may develop an approach to RRI of and on their own through the exchange.

IMAGINE RRI contributes to these aims by providing participating researchers with a set of ideas on the complex issue of RRI from different perspectives and in several stages of debate. The cards offer different types of input (e.g. short text, quote, figure), providing a repertoire of narrative resources and positions. This gives participants the opportunity to respond to questions in the debate flexibly and allows them to build their own framing of the debate. Additionally, to support equal participation in the debate, the cards give ‘material support for participants who possess less rhetorical resources to formulate their opinions’, thus allowing all participants to have voice and to use and relate to the cards creatively to articulate their own positions (Felt et al. Citation2014). Finally, instead of a framing input by the facilitator at the beginning of the debate, we let the cards represent different visions of RRI. This allows the facilitator of the discussion to attend to the process in which participants develop an approach to RRI, rather than act as an RRI expert. The method thus embraces the performative power of research methods, as the card sets and choreography are designed to be flexible enough for participants to actively shape their own reflection process.

IMAGINE RRI

Objectives and structure of the card-based discussion method

The main objectives of our card-facilitated discussion method are to engage life science researchers in reflections on (1) their own role as researchers in relation to different understandings of their responsibilities to society, (2) particular moments when issues of responsibility emerge in their own research practices, and (3) structural contexts that may support or constrain their capacity to consider questions of responsibility in making research-related decisions. The discussion proceeds in three rounds of debate that correspond to these objectives. Each round is supported by a specific deck of cards: statement cards, practice cards and context cards (). These discussion cards and other supplementary material are available under a creative commons license at the following link: https://phaidra.univie.ac.at/view/o:690945.

Figure 1. A typical deck of cards.

Figure 1. A typical deck of cards.

Each participant is provided with a discussion map that visualizes the way the debate is structured and three decks of cards. In each round, participants are asked to pick cards based on rules specified by the facilitator. The choices in the respective rounds are independent, meaning that choices in later rounds do not need to refer to those in earlier ones. As a general principle, the cards allow participants to choose issues that either strongly resonate with their own opinion and practice or that they disagree with. The cards provide participants with a repertoire of narratives (on statements, practices and contexts) that they can choose from, relate to and use creatively during the course of the debate. The facilitator encourages participants to let themselves be inspired by the cards to develop their own positions and imagination about responsible research practices. In each round, blank cards are also available to participants to allow them to express positions and concerns of their own that are not (adequately) covered by the cards. Offering this repertoire of narratives allows participants with little experience in reflecting on RRI issues to gradually develop a position and enter the discussion more easily. For researchers already experienced in such reflections, the cards offer elements to engage with, reformulate or critique the issues.

Each discussion lasts about three hours. After a round of introductions by participants, the facilitator briefly explains the basic logic and purpose of the card-based discussion method. Approximately 45 min are then devoted to each round. A 15-minute break is recommended after round two.

Round 1: statement cards

In the first round, the facilitator asks participants to read the statement cards Footnote6 and choose the card they most strongly agree with and the card they most strongly disagree with. The facilitator also reminds participants of the possibility to use blank cards for personal statements.

The text on the cards describes positions regarding the responsibilities of researchers in relation to society. To develop these cards, we conducted an analysis of existing writing on responsibility in research or analyses of how responsibility is perceived in research practices (such as by Glerup and Horst (Citation2014)). Thus, the phrasing on the cards was inspired by different sources: Card A in , for example, is inspired by a quote by a popular author (C.P. Snow), while other cards relate to how issues of responsibility are addressed in current policy debates (e.g. in terms of grand challenges or market relevance in the European Commission’s framework programme Horizon 2020), and still others consist of rephrased narratives by life science researchers (derived from previous research projects of the authors).Footnote7 Every card features a headline that helps participants memorize the content of cards and that they can use as shorthand to refer to the card in the discussion. The text begins with a one-sentence statement in bold that describes a normative position about what a responsible researcher is, does or should do. The remaining text elaborates on this initial statement in less than forty words.

Figure 2. Example of a statement card.

Figure 2. Example of a statement card.

The phrasing ‘a responsible researcher is’ at the beginning of each card was chosen to invite immediate personal identification with or distancing from the statement. This avoids overly abstract interpretations and encourages researchers to position themselves in relation to the identities and normative positions stated on the cards. This is further supported by the way in which the subsequent text is written. In statement card A (), for example, the first statement is about anticipation as a dimension of RRI. The text, however, represents only one of many possible interpretations of the general statement in bold, and it does not develop it according to a strictly logical interpretation. This consciously opens up room for participants to critically engage with the text and perhaps agree with parts of the statement while disagreeing with others.

The diversity of cards, along with their openness, allows researchers to develop a range of positions, including — for example, with regard to card A — those that interpret responsibility as limited to inner-scientific values and actors, negate the individual responsibility of scientists towards society, or reject the idea of needing to anticipate the societal applications of research results.

Round 2: practice cards

In round 2, the facilitator asks participants to read the 15 practice cards and pick two cards that depict the most important and relevant moments in which issues of responsibility arise in their actual research practices. Following a brief headline, each card features a cartoon that portrays the respective situation in a humoristic manner. This is followed by one or two sentences describing the situation and one to three questions that challenge participants to reflect on the choices to be made (). For choosing the particular situations on the cards, we built on our analyses of research cultures in the life sciences over the last ten years.Footnote8 The situations depicted on the cards were finally vetted by cooperation partners from the life science community prior to their use to ensure their plausibility (Selin Citation2011).

Figure 3. Example of a practice card. Permission by the copyright holder of the cartoon has been obtained.

Figure 3. Example of a practice card. Permission by the copyright holder of the cartoon has been obtained.

The practice cards serve different functions: the visual element of the cartoons often takes an ironic approach to experiences researchers may have had in their work and induces laughter, sarcastic statements or other spontaneous reactions. Humour manages to create closeness and social bonds, and it is often used to address moral issues too delicate to address directly (Kuipers Citation2008). It allows an affective level of engagement, which is often reached by referring to context-specific tacit knowledge or experiences (Davies et al. Citation2012). While researchers read the cards, they often laugh approvingly, remark ‘that’s just the way it is’, or show the cards to each other and have a short chat about their own experiences. The textual element on the cards opens up the issue beyond the irony of the sketched moment and offers a short narrative on its role within the current research system. Questions round out the short text and challenge researchers to reflect on the normative position represented on the card and what it may mean in the context of their concrete research practices.

The practice cards provide a wide repertoire of potential contexts in which responsibility may come to matter in actual research practices. They include aspects directly related to laboratory practices, such as the handling of data (e.g. in the production of images), ethical issues (e.g. related to model organisms), organizational issues (e.g. the creative use of research funding), and situations that link scientific practice to society more directly, such as science communication or the relationship of research questions to societal challenges.

Round 3: context cards

The third and final round revolves around the conditions under which research is done and how these conditions may support or restrain researchers’ capacity to consider questions of responsibility in making research-related decisions — in short, reflecting on the ‘responsibility conditions’. The participants are again asked to read through and pick two of the 13 context cards (or write a card of their own) that they see as most relevant to discuss.

The context cards consist of a headline that names a specific context, and a text (approx. 50 words) that is formulated as a direct quote from a researcher. The choice of contexts is again informed by previous studies on research cultures by the authors. The quotes are either taken from actual interviews, semi-fictional (re-phrased quotes by life scientists), or fictional (from our observations). The context cards cover issues including the effects of metrics and quantification, career dynamics, institutional contexts and their priorities, academic socialization, and the influence of particular forms of funding ().

Figure 4. Example of a context card.

Figure 4. Example of a context card.

The quotes are written from the perspective of life scientists and express reflections on how a specific context influences researchers’ work and their capacity to act responsibly. The narrative form of the direct quote is chosen deliberately. It lends authenticity to the argument, aiming to allow researchers to relate to the topic and potentially recognize its relevance to their personal situation. The relationship of the contexts on the cards to responsible research practice remains mostly implicit, which requires researchers to build their own argument. Again, as with the statement cards, the sentences on the card may make several related points, so participants may agree with parts of the card and disagree with others.

Role of the facilitator and phases of the debate

Each round begins with the facilitator introducing the specific objective of the discussion phase and participants reading and choosing cards. In our experience, this takes 10 min on average. After all participants have made their choice, the facilitator asks participants to explain their card choices in relation to their own experiences and practices. Every participant has an equal opportunity to briefly define her or his position by describing why they chose the specific cards and then providing a rationale for how it relates to their experiences. In doing so, participants build on the narrative infrastructure provided by the cards and add narrative elements of their own; for example, in explaining why they think the specific context is relevant for issues of responsibility. Participants often share own experiences related to the cards to illustrate their point, particularly in the second and third round.

After this first phase, the facilitator asks participants to comment on the choices the others made, and asks which choices were surprising. This usually triggers different forms of productive exchanges and deliberation on the normative issues at stake. Participants may support others’ choices, for example stressing that they had also considered a specific card but then prioritized another. They may question other participants’ interpretations and explain how they differ from their own initial reading, or they may challenge another participant’s position and the importance of the card he/she has chosen. In our experience, the fact that participants can relate their positions to the narratives on the cards allows for quite critical but still constructive debates. Participants feel that the fact that there is a card that relates to their position lends it a certain baseline legitimacy and thus take criticism less personally than they might in non-card-supported discussions.

During the debate, the facilitator aims to maintain a good balance between upholding rules and timelines and allowing flexibility in the discussion. This enables participants to appropriate the game and handle the cards in creative ways; for example, they repeatedly chose not two cards but several different cards to explain their positions, or used a single card to demonstrate different kinds of positions. Additionally, they referred to cards other than those they had chosen to support their arguments during the course of the debate. Thus, their creative ways of handling the cards support them in voicing a variety of positions – even when they were not explicitly mentioned on the cards.

During these first two phases, the facilitator often does not actively intervene by asking questions, but merely ensures that all participants have a turn to speak. Other than that, the dynamics of the discussion emerge from the participants’ choices and reactions. However, if there is time left after the general debate, the facilitator may ask focused questions, e.g. by inquiring further about implicit tensions in the debate, or by addressing cards that have not been mentioned at all and asking why they were not considered. This often motivates researchers to speak about why certain RRI topics are considered less important in the particular research field, and may even lead them to discuss why certain topics are handled as taboo.

The discussion closes with a brief round of statements on how participants experienced the game.

Number and profile of participants

The ideal number of participants for our card-based discussion method is between five and eight. Less than five may impede the discussion dynamics because fewer cards are selected and hence may serve as points of reference in the discussion. With more than eight participants, there is too little interaction time per participant, and the rounds in which participants present their cards are too long and may seem repetitive.

Ideally, participants should first have had experiences with doing actual independent research: in our experience, early PhD researchers up to senior researchers can best engage with the topics offered, but we have also successfully included advanced master’s students.Footnote9

So far, we have conducted the card game mostly in mixed groups in terms of the levels of experience of the researchers present. Sometimes, participants knew each other well beforehand (because they were members of the same research group), but sometimes, participants met for the first time at the discussion table.

The unavoidable hierarchies in such groups are, in our experience, strongly moderated by the card-based format, because the design assigns equal roles to participants and every position is strengthened by the material dimension of the card. We aimed to create equal opportunities for all opinions within a group to be included in the debate. To facilitate this, the cards also include positions that deflect the main ideas of RRI rather than embrace them, presenting them as positions that may legitimately be taken. In that the game allows researchers to reject or embrace positions along a broad range of normative statements, it is designed to include researchers with different opinions regarding how science-society relationships are – or should be – organized in a responsible way.

Illustrating the discussion dynamics

Reflecting on researchers’ role in how responsibility is practised and governed – the statement cards

In the following sections, we will discuss examples of transcripts from the card-based discussions we have conducted to provide the reader with an impression of the actual dynamics in the game. These examples are meant to support our conceptual argument in this paper, not as a systematic discussion of the effectiveness of the intervention or as an interpretive analysis of topics discussed in the debate.

To provide an impression of the discussion and reflection dynamics triggered by this discussion method, let us focus on how statement card I (see ) was discussed. It is one of the more abstract statement cards because it does not propose any concrete role for researchers in relation to society, but rather reflects on their possibilities of action. Nevertheless, this card was chosen rather frequently (by two participants per group on average). Mostly, participants would disagree with it, as in the following quote:

I disagree with this because, I mean, of course one researcher cannot do much (.) but I think it starts from a single researcher, so if you take care of the rules and if you are really thoughtful and you (.) keep this example to other persons in your lab or institute, you can do a lot, actually. And if everyone followed these lines so (.) that would be great. (PDoc5m, group 3)Footnote10

Figure 5. Statement card I.

Figure 5. Statement card I.

Like many others who chose this card, this postdoc argues against the passive role the statement on the card assigns to individual researchers and suggests that responsibility begins with small, everyday actions and the example they set. However, some – although significantly fewer – researchers agreed with the card. Consider how a postdoc in another discussion group puts it:

Of course, that’s a bit of an overstatement, because still I believe that an individual, a researcher, can change or can contribute to changing the system. But I think there are mechanisms these days installed, with regard to funding, with regard to publications, with regard to the whole mechanism of, let’s say, reputation, with regard to where you publish, how often you publish, in which journal, how good […] this is, how good your data have to look in order to publish it in this or that journal, that I think interfere with the whole idea of diligence that most researchers should have by nature. So, I believe that the whole pressure that is built on the individual with regards to surviving within academia, for example, getting funding and so on […] is not a good state at this point, because everything is basically pointing to a direction of producing clear results, clear data, yes or no, black and white results. And so […] I think the system needs to change in order to bring back diligence or a higher degree of diligence and also the possibility to be more honest […]. (PDoc3m, group 2)

While this researcher also rejects the idea that an individual researcher’s actions do not matter, he goes on to stress that irresponsible behaviour is related to certain conditions in current research organization (e.g. career pressure, funding situation). According to this logic, it is the pressure generated by the system that constrains researchers’ quasi-natural tendency to act responsibly.

Let us make two points related to this quote. First, such differences in how cards were interpreted were often taken up for discussion in the group later. In exploring different potential meanings of the cards, participants also debated the different meanings of responsibility, or, as in this example, of the agency of individual researchers in making responsible decisions. Differences in participants’ interpretations thus became a productive resource for the group. Second, as the latter quote shows, researchers often referred to their own experiences when illustrating their positions. In doing so, they used the provided statements as a narrative infrastructure to reflect on their own practices and thereby managed to relate their own experiences to questions of responsibility.

Learning from each other’s experiences – the practice cards

In their choice and explanation of practice cards, researchers usually refer to examples from their own experiences. This allows them to reflect on their own practice and enables mutual learning in the group. Other participants react to the input, add related experiences or compare it to their own experiences. This usually leads to very rich exchanges on the respective topics. As in the example we discuss below, this learning cuts across the classical hierarchies of life science research groups, meaning that PhD students, postdocs and group leaders comment on issues from their respective perspectives. In this process, the perspectives of senior participants are not necessarily ascribed more authority than those of more junior participants. Consider an exchange that was triggered by several participants’ choice of practice cards C and M, which address the diligent handling of data and images, respectively ().

MS5f:

I’m coming back to supervision and team: so, because, if I make some mistake […] the important [thing] is the reaction, and that I get it back, and since the reactions are not that, you are the worst [people chuckling] person […]. It’s, it’s normal, we all make mistakes and it’s normal, and it’s discussed, okay, let’s see how, what we can do about it, and so on […]

MS3m:

I think this trust thing is also, from a student’s perspective, together with responsibility […] it occurs, not, for instance, starting with the master’s, and certainly much, much more in the PhD, is that, on this side, from the supervisor, there trust is put in the student (.) to (.) that things are done properly, and that things are done in a reflected way […]. So, getting this trust is also very motivating […] because you know that there is so much trust put into it, that it has to be done in a proper way. But you won’t start working sloppy because it’s late in the evening […]. So, I think that getting the trust of the supervisor is a very good way of, or it is for me […] to strengthen my feeling of responsibility, to work properly. That is the way to become a good scientist.

MS4f:

That’s very true.

PI4m:

I think it’s also something, I mean you are all also supervisors, maybe you don’t realize that! But in fact, anything that’s happening in the lab and in daily life, in your daily lab life, is not supervised by me at all. I cannot do that. It’s in fact supervised by the other students that have more experience in this specific way, or post-docs or whatever. In a way that is also casual, so you’re supervised in a team, you all are a team, even if you don’t work on the same issue, I guess. And I think […] if you can create this type of atmosphere, then I think this is like a shared responsibility […] it cannot be my responsibility alone, certainly I have to think about it and create a type of atmosphere, but the rest is, it’s us all, no? We have to work together. (group 5)

Figure 6. Context card I.

Figure 6. Context card I.

Exchanges such as this one allow participants to reflect on their everyday experience through the lens of responsibility and identify conditions that foster or inhibit their ability to act responsibly – in this case, specific supervision relationships. If several members of a discussion group are part of the same research team (as was the case here) this may also create space for talking about and reflecting on group dynamics through the lens of responsibility.

This interaction is also a good example of how topics on different cards are related to each other, for while the debate was triggered by a combination of cards about diligence, the narrative of practice card I (Supervision and Teams) gets drawn into the debate as well. Through the ways in which participants combine different cards and develop shared positions (in this case, that of the lab as the location of collective responsibility somehow lying between the institution and the individual), we can observe how they are empowered to use the elements of the narrative infrastructure we offer and build new and more complex ways of thinking about issues of responsibility.

Reflecting on the conditions enabling or hindering responsibility – the context cards

In the last round, the context cards invite participants to reflect on how specific conditions of the current research system foster or inhibit their ability to act responsibly. Again, the quotes on the cards deliberately invite different readings. This can lead participants to disagree about whether specific context conditions foster or inhibit responsibility, as in the exchange below. This particular exchange was triggered by several participants’ choice of the context card I (Temporary Contracts).

While most participants in this specific debate agreed with the general point made on the card, one group leader strongly disagreed. Later in the debate, the facilitator picked up the issue and addressed the apparent difference in opinion. Consider how the different researchers develop very different narratives to relate temporary contracts to responsibility in the following exchange:

PhD1f:

Well, despite the time pressure and your time-limited contract, you should always have some ethics, meaning that even though you are finished, your four years are finished and you have nothing really clear done, maybe sometimes some people take like an easy road to cheat. And that’s not the way you have to go.

PI1m:

Yeah, this is kind of the typical argument that time pressure creates irresponsible behaviour. But also, the other way around is equally true, that time pressure forces upon you that you’re responsible to do your best now and not whenever it is convenient for you. And especially here in the country, we have a long track record of giving, kind of, permanent jobs very, relatively early, like twenty thirty years ago, relatively early in life […]. People then basically stopped working. […] It seems that we, as human beings, of course, we have, we are mortal, we have only a kind of defined amount of time to do our stuff and we didn’t forget about this and in a way kind of reasonably designed temporary contracts or, kind of processes that remind us that the time is now! And the time is not at some point in the future when we feel like it.

PhD2f:

I think the key word here is reasonably timed and, and also flexible again to a certain degree. Because sometimes things just happen that cannot be planned on a three-year, four-year basis. So, I, I agree with giving people permanent contracts very early […]. But having a very, very strict deadline and realizing early on that this deadline cannot be met to due circumstances that just, just happen is also not healthy. Because you feel like you work against, against an obstacle that you cannot overcome. So, there needs to be some kind of flexibility. (group 1)

In this exchange, participants discuss whether and how temporary contracts support or prevent responsible behaviour. While the PI stresses that temporary contracts nudge people to manage their time and efforts efficiently and responsibly, other participants argue that the challenge of making unpredictable epistemic processes fit within the timelines of fixed-term contracts can also invite irresponsible behaviour. While this tension is not resolved in the discussion, the discussion allows us to see how different narratives are placed in conversation with each other or even blend into one larger story.

This demonstrates nicely the baseline of how the group discussions work: rather than leading to clear and ready-made answers to certain questions, our approach is designed to make different positions visible and help recognize different narrative approaches to making sense of responsibility. This, in turn, can be used as a resource for further reflection by participants and thus contributes to advancing researchers’ capacity to reflect on issues of responsibility in research.

Conclusions

A central challenge of Responsible Research and Innovation is to create conditions and initiatives that allow researchers to actively engage with the idea of responsibility and translate it to their practices. RRI can only become a true part of the culture of contemporary research and innovation if it is actively embraced and made sense of by researchers (Felt Citation2017). To achieve this, measures are needed on different levels ranging from ways of empowering researchers to make sense of the idea of RRI both as groups and individuals to more general changes in research environments to provide more space and time to researchers to reflect on issues of RRI. The method we have presented in this paper addresses both concerns, although to different extents. Primarily, it empowers researchers to reflect on RRI and relate it to their own practices. However, through analysing researchers’ lines of argumentation, it can also be used as a research method to explore the ways in which contemporary research environments foster or structurally hinder researchers from realizing the visions and values of RRI.

It is not necessarily easy for researchers to engage with RRI and make sense of it in their own practices. On one hand, RRI as a shorthand represents a diverse and complex set of ideas, ones that are often cast in a very general language that seems distant from the concerns relevant in the lab or the research office. On the other hand, powerful structural forces in contemporary academia, such as increasing competition or the formalization of many parts of academic work, push against researchers’ capacity to reflect on and to act responsibly in their work. In particular, many of these structural tendencies individualize researchers, weakening the capacity of collectives such as research groups, faculties or universities to act as spaces in which a productive discourse on RRI can take place.

The card-based method we have presented in this paper, IMAGINE RRI, is a humble yet effective intervention into this problematic constellation. It empowers researchers to actively think about RRI and relate it to their own practices. As we have described, it does so by putting the concept of the narrative front and centre. Narratives are crucial because they are ways of making sense of a given phenomenon. Our method helps researchers to construct their own narratives by providing them with a narrative infrastructure with which to discuss RRI – represented by the cards. Researchers can appropriate the elements of this infrastructure and modify, supplement or contest them. Whatever they choose to do, they will be making sense of RRI in relation to their own understanding of the place of research in society and their own role. The use of the narrative infrastructure approach (Felt Citation2017) also allows participants to appreciate the breadth of the discussions behind the notion of responsibility even if they personally choose only one particular argumentative position towards this issue. Our method is thus an addition to the repertoire of interventions utilizing narrative approaches (Felt et al. Citation2014; van der Burg Citation2016).

Our method is context sensitive in the sense that it invites researchers to go beyond the level of talking about the idea of responsibility in general and to relate it to practices that matter in their own work. The version we developed focuses on life science practices. Using the method in other fields will require at least partial adaptation to practices relevant in the respective domain.

We perceive our method as an addition to the portfolio of approaches that aim to foster the integration of social and ethical aspects in research practices. It is not capable of reaching as deep an engagement with a specific topical field as, for example, CTA or STIR (Fisher and Schuurbiers Citation2013; Rip and Robinson Citation2013; Fisher et al. Citation2015). However, due to its high adaptability and its limited demand in terms of resources employed and in terms of the time investment by participating researchers, it lends itself to attempts to foster RRI-related reflexivity in institutions on a broader scale. In designing a game-like setting, we also respond to calls for more ‘playful’ methods of post-ELSI/ELSA interdisciplinary collaborations (Felt et al. Citation2014; Balmer et al. Citation2015; van der Meij, Broerse, and Kupper Citation2017).

How does our method engage with the structural tendencies that inhibit the embracing of RRI in contemporary academia? It does so in two ways. First, through narratives and images, it prompts researchers to reflect on and recognize which structural contexts affect their ability to reflect and act responsibly. As researchers are empowered to address these structural constraints, they may also find ways to manage them better or engage in initiatives to change them. Second, our method establishes a collective space for exchange regarding issues of responsibility, working against the individualizing tendencies of contemporary academia. In some instances, this will be rather temporary, such as when the method is used in teaching. In other cases, for example when participants are members of the same research group, it may establish certain aspects of RRI as topics of continuous conversation. In particular, in doing so, it also empowers more junior members of research groups to appropriate issues related to RRI as legitimate topics of conversation.

Although the primary aim of our method is to empower researchers to engage with RRI, it also has considerable potential as a research method for studying how researchers make sense of the idea of responsibility in relation to their practices and which conditions may prevent them from embracing RRI. In this paper, we could only hint at the questions that can be analysed using transcribed recordings of the discussion. For example, it is possible to analyse how researchers conceptualize and relate the agency and responsibility of individuals, groups, institutions and other actors in research governance. We can also ask how researchers perceive the influence of changing structures of research, such as projectification or the increasing time pressure in scientific work, on themselves and their ability to act responsibly. The results of such research may provide important hints as to how much structural conditions of research need to change for ideas of RRI to become embedded as guiding values in research cultures.

As a final reflection, we must address the limits of our method and of RRI in general. As we have described in this paper, we are convinced that our method can present a productive intervention that can empower researchers to make sense of and hence more actively embrace the idea of responsibility and engage more openly with societal issues. At the same time, we recognize that the actual impact of such initiatives will always be limited and uncertain. Certainly, they will not spur every researcher to reflect on his/her responsibility in research beyond the moment of the debate. Such an expectation would not be realistic. Ezrahi (Citation2012) has argued that democracies do not only need to be continuously imagined and performed through many different practices in order to exist, but also rest on ‘necessary fictions’, such as the informed, free and rational citizen. These necessary fictions may have powerful effects despite their obvious repeated failures and shortcomings. In a similar way, we believe that a research system capable to engage with society needs responsible researchers, ready to engage with societal issues and concerns beyond his/her epistemic interests. In Ezrahi’s sense they are the necessary fiction to uphold imaginaries that legitimate the power and authority of research and innovation in contemporary society. To achieve this, however, work is required to bring RRI-related values and concerns into the cultures of contemporary research. The method presented in this paper is one of many means to this end.

Acknowledgement

We would like to thank our collaboration partners, Andreas Richter and Renée Schroeder, in the research platform ‘Responsible Research and Innovation in Academic Practice’ (http://rri.univie.ac.at) for their support in the preparation of the discussion material. Furthermore, our thanks go to the participants of the discussion groups who gave us their time and shared their ideas with us. Finally, we want to express our appreciation to the reviewers for their comments and guidance to improve the manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors .

Notes on contributors

Ulrike Felt is Professor of Science and Technology Studies at the University of Vienna. Her research focuses on governance, democracy and public participation, on shifting research cultures and on time/future in research and society. She was editor-in-chief of Science, Technology, & Human Values and of the 4th Handbook of Science and Technology Studies. Since 2017 she is president of European Association for the Study of Science and Technology.

Maximilian Fochler is assistant professor at the Department of Science and Technology Studies at the University of Vienna. His main current research interests are forms of knowledge production at the interface of science and other societal domains, as well as the impact of new forms of governing science on processes of valuation and subjectification in academia. He has also published on the relations between technosciences and their publics as well as on publics’ engagement with science.

Lisa Sigl is a postdoctoral researcher at the research platform ‘Responsible Research and Innovation in Academic Practice’ at the University of Vienna. In her dissertation, she explored the conditions for uncertainty experiences and how the project modus changes research cultures in the life sciences. Her current research interests lie in implications of commercialization and commodification of research for cultures of knowledge production and in processes of subjectification in academia.

Notes

5 The notion of narrative infrastructures was first used by Deuten and Rip (Citation2000) to study design processes in an organisation. Felt (Citation2017) expanded on and further developed the concept in the context of RRI in academia.

6 There are twelve statement cards in the current deck. In this round, as in others, the number may vary according to the exact focus of the debate.

7 Please refer to the supplementary material for a precise documentation of sources.

8 Input for these cards was taken from previous research projects on work and life in academia carried out by the authors. These projects date back to as early as 2003 in a continuous research trajectory. Material from the projects comprises more than 90 interviews with life science researchers as well as more than 100 hours of focus group discussions and a number of participant observations.

9 At the date of the submission of this paper, the authors had conducted 11 card games in total, all related to the Platform for Responsible Research and Innovation in Academic Practice (http://rri.univie.ac.at) at the University of Vienna. The groups were conducted at different research institutions in Vienna, partly in the framework of training programmes and partly on invitation by specific research groups. The participating researchers had a variety of backgrounds that spanned the entire breadth of life science research conducted at Viennese research institutions.

10 The information following the quote indicates the academic situation of a participant (PDoc … Post-Doc; PhD … doctoral student; MS … Master student; PI … principle investigator), gender as well as the group discussion from which the quote is drawn.

References

  • Balmer, Andrew S., Jane Calvert, Claire Marris, Susan Molyneux-Hodgson, Emma Frow, Matthew Kearnes, Kate Bulpin, Pablo Schyfter, Adrian Mackenzie, and Paul Martin. 2015. “Taking Roles in Interdisciplinary Collaborations: Reflections on Working in Post-ELSI Spaces in the UK Synthetic Biology Community.” Science & Technology Studies 28 (3): 3–25.
  • Bandelli, Andrea. 2010. “Engagement Tools for Scientific Governance.” Journal of Science Communication 9 (2): 1–4.
  • Bensaude Vincent, Bernadette. 2013. “Which Focus for an Ethics in Nanotechnology Laboratories?” In Ethics on the Laboratory Floor, edited by Simone Van der Burg, and Tsjalling Swierstra, 21–37. New York, Houndmills: Palgrave Macmillan.
  • Bloor, Michael, Jane Frankland, Michelle Thomas, and Kate Robinson. 2001. Focus Groups in Social Research. London: SAGE.
  • Bos, Colette, Bart Walhout, Alexander Peine, and Harro van Lente. 2014. “Steering with Big Words: Articulating Ideographs in Research Programs.” Journal of Responsible Innovation 1 (2): 151–170. doi:10.1080/23299460.2014.922732.
  • van der Burg, Simone. 2016. “A Lay Ethics Quest for Technological Futures: About Tradition, Narrative and Decision-Making.” NanoEthics 10 (3): 233–244. doi:10.1007/s11569-016-0273-2.
  • van der Burg, Simone, and Tsjalling Swierstra. 2013. Ethics on the Laboratory Floor. Basingstoke: Palgrave Macmillan.
  • Czarniawska, Barbara. 2004. Narratives in Social Science Research. London: Sage.
  • Davies, Sarah R., Cynthia Selin, Gretchen Gano, and Ângela Guimarães Pereira. 2012. “Citizen Engagement and Urban Change: Three Case Studies of Material Deliberation.” Cities 29 (6): 351–357. doi:10.1016/j.cities.2011.11.012.
  • de Saille, Stevienna. 2015. “Innovating Innovation Policy: the Emergence of ‘Responsible Research and Innovation’.” Journal of Responsible Innovation 2 (2): 152–168. doi:10.1080/23299460.2015.1045280.
  • DEEPEN. 2009. Reconfiguring Responsibility: Deepening Debate on Nanotechnology (FP6 Project Report). Durham: Durham University. Accessed December 4, 2017. http://dro.dur.ac.uk/6399/1/6399.pdf?DDD14+dgg1mbk.
  • Deuten, J. Jasper, and Arie Rip. 2000. “Narrative Infrastructure in Product Creation Processes.” Organization 7 (1): 69–93. doi:10.1177/135050840071005.
  • Ezrahi, Yaron. 2012. Imagined Democracies: Necessary Political Fictions. Cambridge: Cambridge University Press.
  • Felt, Ulrike. 2017. ““Response-Able Practices” or “New Bureaucracies of Virtue”: The Challenges of Making RRI Work in Academic Environments.” In Responsible Innovation 3: A European Agenda?, edited by Lotte Asveld, Rietje van Dam-Mieras, Tsjalling Swierstra, Saskia Lavrijssen, Kees Linse, and Jeroen van den Hoven, 49–68. Cham: Springer International Publishing.
  • Felt, Ulrike, Daniel Barben, Alan Irwin, Pierre-Benoît Joly, Arie Rip, Andy Stirling, and Tereza Stöckelová. 2013. Science in Society: Caring for our Futures in Turbulent Times, Policy Briefing 50. Strasbourg: ESF.
  • Felt, Ulrike, and Maximilian Fochler. 2010. “Machineries for Making Publics: Inscribing and Describing Publics in Public Engagement.” Minerva 48 (3): 219–238. doi:10.1007/s11024-010-9155-x.
  • Felt, Ulrike, Simone Schumann, and Claudia Schwarz. 2015. “(Re)Assembling Natures, Cultures, and (Nano)Technologies in Public Engagement.” Science as Culture 24 (4): 458–483. doi:10.1080/09505431.2015.1055720.
  • Felt, Ulrike, Simone Schumann, Claudia G. Schwarz, and Michael Strassnig. 2014. “Technology of Imagination: A Card-Based Public Engagement Method for Debating Emerging Technologies.” Qualitative Research 14 (2): 233–251. doi:10.1177/1468794112468468.
  • Felt, Ulrike, Simone Schumann, and Claudia Schwarz-Plaschg. 2018. “IMAGINE – A Card-Based Discussion Method.” In Research Methods in Health Social Sciences, edited by Pranee Liamputtong. Singapore: Springer. doi:10.1007/978-981-10-2779-6_9-1.
  • Fisher, Erik. 2007. “Ethnographic Invention: Probing the Capacity of Laboratory Decisions.” NanoEthics 1 (2): 155–165. doi:10.1007/s11569-007-0016-5.
  • Fisher, Erik, Roop L. Mahajan, and Carl Mitcham. 2006. “Midstream Modulation of Technology: Governance From Within.” Bulletin of Science, Technology & Society 26 (6): 485–496. doi: 10.1177/0270467606295402
  • Fisher, Erik, Michael O’Rourke, Robert Evans, Eric B. Kennedy, Michael E. Gorman, and Thomas P. Seager. 2015. “Mapping the Integrative Field: Taking Stock of Socio-Technical Collaborations.” Journal of Responsible Innovation 2 (1): 39–61. doi:10.1080/23299460.2014.1001671.
  • Fisher, Erik, and Daan Schuurbiers. 2013. “Socio-technical Integration Research: Collaborative Inquiry at the Midstream of Research and Development.” In Early Engagement and new Technologies: Opening up the Laboratory, edited by Neelke Doorn, Daan Schuurbiers, Ibo van de Poel, and Michael E. Gorman, 97–110. Dordrecht: Springer Netherlands.
  • Glerup, Cecilie, Sarah R. Davies, and Maja Horst. 2017. “‘Nothing Really Responsible Goes on Here’: Scientists’ Experience and Practice of Responsibility.” Journal of Responsible Innovation 4 (3): 319–336. doi: 10.1080/23299460.2017.1378462
  • Glerup, Cecilie, and Maja Horst. 2014. “Mapping ‘Social Responsibility’ in Science.” Journal of Responsible Innovation 1 (1): 31–50. doi:0.1080/23299460.2014.882077 doi: 10.1080/23299460.2014.882077
  • Groves, Christopher. 2017. “Review of RRI Tools Project, http://www.rri-tools.eu.” Journal of Responsible Innovation 4 (3): 371–374. doi: 10.1080/23299460.2017.1359482
  • Guston, David H., and Jack Stilgoe. 2017. “Responsible Research and Innovation.” In Handbook of Science and Technology Studies, Fourth Edition, edited by Ulrike Felt, Rayvon Fouché, Clark A. Miller, and Laurel Smith-Doerr, 853–880. Cambridge, MA: MIT Press.
  • He, Shiqing, and Eytan Adar. 2017. “VizItCards: A Card-Based Toolkit for Infovis Design Education.” IEEE Transactions on Visualization and Computer Graphics 23 (1): 561–570. doi:http://doi.ieeecomputersociety.org/10.1109/TVCG.2016.2599338.
  • Hilgartner, Stephen, Barbara Prainsack, and J. Benjamin Hurlbut. 2017. “Ethics as Governance in Genomics and Beyond.” In Handbook of Science and Technology Studies, 4th Edition, edited by Ulrike Felt, Rayvon Fouché, Clark A. Miller, and Laurel Smith-Doerr, 823–852. Cambridge, MA: MIT Press.
  • Kitzinger, Jenny. 1994. “The Methodology of Focus Groups: The Importance of Interaction Between Research Participants.” Sociology of Health and Illness 16 (1): 103–121. doi:10.1111/1467-9566.ep11347023.
  • Kuipers, Giselinde. 2008. “The Sociology of Humor.” In The Primer of Humor Research, edited by Victor Raskin, 365–402. Berlin, New York: Mouton de Gruyter.
  • Larkin, Brian. 2013. “The Politics and Poetics of Infrastructure.” Annual Review of Anthropology 42 (1): 327–343. doi:10.1146/annurev-anthro-092412-155522.
  • Luger, Ewa, Lachlan Urquhart, Tom Rodden, and Michael Golembewski. 2015. “Playing the Legal Card.” Paper presented at the crossing, CHI 2015, Seoul, Republic of Korea.
  • Macnaghten, Phil, and Julia Guivant. 2011. “Converging Citizens? Nanotechnology and the Political Imaginary of Public Engagement in Brazil and the UK.” Public Understanding of Science 20 (2): 207–220. doi:10.1177/0963662510379084.
  • van der Meij, Marjoleine G., Jacqueline E. W. Broerse, and Frank Kupper. 2017. “Conceptualizing Playfulness for Reflection Processes in Responsible Research and Innovation Contexts: a Narrative Literature Review.” Journal of Responsible Innovation 4 (1): 43–63. doi:10.1080/23299460.2017.1326258.
  • Rabinow, Paul, and Gaymon Bennett. 2012. Designing Human Practices. An Experiment in Sythetic Biology. Chicago: University of Chicago Press.
  • Ribeiro, Barbara E., Robert D. Smith, and Kate Millar. 2017. “A Mobilising Concept? Unpacking Academic Representations of Responsible Research and Innovation.” Science and Engineering Ethics 23 (1): 81–103. doi:10.1007/s11948-016-9761-6.
  • Rip, Arie. 2016. “The Clothes of the Emperor. An Essay on RRI in and Around Brussels.” Journal of Responsible Innovation 3 (3): 290–304. doi:10.1080/23299460.2016.1255701.
  • Rip, Arie, and Douglas K. R. Robinson. 2013. “Constructive Technology Assessment and the Methodology of Insertion.” In Early Engagement and new Technologies: Opening up the Laboratory, edited by Neelke Doorn, Daan Schuurbiers, Ibo van de Poel, and Michael E. Gorman, 37–53. Dordrecht: Springer Netherlands.
  • Rip, Arie, and Haico te Kulve. 2008. “Constructive Technology Assessment and Socio-Technical Scenarios.” In Presenting Futures, edited by Erik Fisher, Cynthia Selin, and Jameson M. Wetmore, 49–70. Dordrecht: Springer Netherlands.
  • Rosenlund, Joacim, Peter Notini, and Giangiacomo Bravo. 2017. “Exploring Attitudes to Societal Relevance: the Effects of Reflection on Research Practices among Swedish Environmental Scientists.” Journal of Responsible Innovation 4 (3): 337–353. doi: 10.1080/23299460.2017.1387509
  • Schot, J. W., and A. Rip. 1997. “The Past and the Future of Constructive Technology Assessment.” Technological Forecasting & Social Change 54 (2-3): 251–268. doi: 10.1016/S0040-1625(96)00180-1
  • Selin, Cynthia. 2011. “Negotiating Plausibility: Intervening in the Future of Nanotechnology.” Science and Engineering Ethics 17 (4): 723–737. doi:10.1007/s11948-011-9315-x.
  • Sutton, Barbara. 2011. “Playful Cards, Serious Talk: a Qualitative Research Technique to Elicit Women’s Embodied Experiences.” Qualitative Research 11 (2): 177–196. doi:10.1177/1468794110394070.
  • Viseu, Ana. 2015. “Caring for Nanotechnology? Being an Integrated Social Scientist.” Social Studies of Science 45 (5): 642–664. doi:10.1177/0306312715598666.
  • Wickson, Fern, Roger Strand, and Kamilla Lein Kjolberg. 2015. “The Walkshop Approach to Science and Technology Ethics.” Science and Engineering Ethics 21 (1): 241–264. doi:10.1007/s11948-014-9526-z.
  • Wilsdon, James, Brian Wynne, and Jack Stilgoe. 2005. The Public Value of Science. London: Demos.
  • Zwart, Hub, Laurens Landeweerd, and Arjan van Rooij. 2014. ‘Adapt or Perish? Assessing the Recent Shift in the European Research Funding Arena From ‘ELSA’ to ‘RRI’.” Life Sciences, Society and Policy 10 (1): 674. doi:10.1186/s40504-014-0011-x.