Implementing responsible innovation: the role of the meso-level(s) between project and organisation

ABSTRACT

Much of academic discussion of responsible innovation (RI) has focused on RI integration into research projects. In addition, significant attention has also been paid to RI structures and policies at the research policy and institutional level. This article reports experiences of RI implementation with a focus on the intermediate i.e. meso-level. The research described here included a series of interviews that aimed to clarify researchers' perspectives on RI as well as barriers to and benefits of RI implementation. Two cases of engagement with research projects, with the aim of promoting RI, were undertaken. The analysis of the data demonstrates the crucial contribution that the meso-level of a research programme can make in interpreting, implementing and perpetuating RI across related activities. The article provides strong evidence that the scholarly debate surrounding RI should pay more explicit attention to this meso-level, ultimately strengthening RI theory and practice.

KEYWORDS:

• Responsible innovation
• ICT research
• research programme
• meso-level

Introduction

The academic discussion around (RI) has explored numerous ways of realising and implementing RI on the level of a research project (Hoven, Doorn, and Swierstra 2014). While the project level is by no means the only place where RI can play a role, it is probably the most prominent one that is discussed in the literature. This may be because it is where research and innovation activities are realised and where researchers and innovators make decisions that shape the outcomes of their work in ways that are likely to have broader social and ethical impact. Another reason for the prominence of the project level is that this is where it is relatively easy to implement interventions and investigate their consequences. Projects are places where the outcomes and consequences of such policy can be studied, which provide an additional reason for paying attention to the project level when exploring RI.

In addition to the focus on the project level, there is a long-standing discussion on the role of research and innovation policy in guiding RI (Fitjar, Benneworth, and Asheim 2019; Griessler et al. 2023). Research and innovation policy may be shaped by national or international politics, can be driven by research funding organisations and implemented by research performing organisations, such as universities.

The interrelated questions of where RI is – or should be – located and whose role it is to implement it clearly have a bearing on the social reality of the implementation of RI. Given this context we undertook an empirical study that aimed to answer the following research question: What are the main barrier and enablers of RI and how are these reflected at different organisational levels? In response to this question, we undertook an empirical study that explored how RI can be integrated and assessed in research projects, within their broader organisational context. This article approaches its research question by providing an account of an intervention that aimed to integrate RI into a large programme of research funded as a research hub by the UK’s Engineering and Physical Research Council (EPSRC),¹ which forms part of UK Research and Innovation (UKRI),² the main public research funder in the country. Interviews were carried out with researchers across the hub to understand their position on RI. Within this hub, a project focused on RI was funded with the aim to better understand the RI position of the hub itself and to support researchers in identifying RI opportunities and possible gaps in current practice within the hub and linked research programmes. The project worked with two other projects (one from within the hub, and one from another programme also funded by the EPSRC) to understand how RI was interpreted within those projects, with the ultimate aim to co-create a bespoke responsible implementation plan.

Our findings highlight that there are meso-level organisational structures, whose impact go beyond the micro-level of the individual project and the macro-level of the research performing organisation. These structures, which in universities can take the form of faculties, schools, departments, but also research groups, hubs or research programmes, can have a crucial influence on the interpretation and realisation of RI in applied research contexts. These structures have received limited attention in the literature on RI up-to-date, but our work shows that they are of crucial importance and can have a strong influence on the contribution of the projects, including the translation of higher-level policies into project development. The discussion therefore focuses on the location of RI and which interaction between the project and other levels may ultimately promote RI.

This paper is of interest to researchers and innovators who do not consider themselves experts in RI, as it provides pointers to how they can engage with the specific academic community to get support for the integration of RI into their work. The insights presented here are also important for policymakers and funders who wish to understand how RI structures and incentives can impact practice. Crucially, the findings are relevant to individuals within meso-level research structures, such as departments, hubs or programmes in universities and other types of organisations, whose role in the realisation of RI is frequently ignored.

This paper starts with a brief review of the concept of RI as intended within our study. This is followed by a description of the methodology, which details the setup of the study and the data collection and analysis approaches adopted. The findings section provides an overview of the results and the specific interventions in the two exemplar cases examined in this contribution. These underpin the subsequent discussion that highlights the importance of meso-level structures, such as research programmes, for the implementation of RI. The conclusion spells out the contribution to knowledge, as well as benefits and limitations of our approach.

Challenges of realising responsible innovation

In this paper, we use the term ‘responsible innovation’ (RI) to underline that we follow the conceptualisation proposed by Stilgoe, Owen, and Macnaghten (2013) who see the term comprised of anticipation, reflexivity, inclusion and responsiveness. We interpret this concept as compatible with Schomberg’s definition of responsible research and innovation as ‘a transparent, interactive process by which societal actors and innovators become mutually responsive to each other, with a view to the (ethical) acceptability, sustainability and societal desirability of the innovation process and its marketable products (to allow a proper embedding of scientific and technological advances in our society)’ (Von Schomberg 2013, 63) and the European vision of responsible research and innovation as ‘the ongoing process of aligning research and innovation to the values, needs and expectations of society’ (Rome Declaration 2014). While the conceptual discussion of the term remains important (Owen, von Schomberg, and Macnaghten 2021b), we believe that the overall shape of the concept is sufficiently clear and the more daunting challenges can be found in the implementation of RI.³

The question of how to achieve the aims of RI is not new and has been widely discussed in the literature. There are various suggestions as to how individual components of RI, for example, the aspect of anticipation, can build on existing approaches to future and foresight studies (Sardar 2010). Future and foresight research has been identified as one approach to RI (Grunwald 2014) that can support anticipatory governance (Nordmann 2014). Similarly, the ‘inclusion’ aspect of RI can draw on well-established methods for public engagement (Rowe and Frewer 2005) which have been incorporated, in various ways, in RI activities (Fisher and Rip 2013). Beyond the focus on particular activities that correspond to specific components of RI, the integration of RI into the research and innovation system or the activities undertaken during research and innovation processes is frequently recommended. Suggestions include various types of ex-ante assessments, such as risk assessment (Kastenhofer 2011), data protection impact assessment (Ivanova 2020) or more specific technology-focused assessments, such as algorithmic impact assessment (AI Now Institute 2018). These can be seen as part of the broader research and innovation governance landscape which, to a significant extent, supports RI, even though it does not typically use the term RI nor the usually-employed terminology of the RI discourse. Examples of such RI-related processes include research ethics reviews (DuBois and Antes 2018), research integrity processes (European Science Foundation and ALLEA 2011), health and safety principles (IOSH 2012) but also the integration of different disciplines in research programmes (Felt, Fochler, and Sigl 2018). Proposals for the realisation of RI range from institutional changes, on the research funder level (Owen et al. 2021a) to specific suggestions of the use of research methodologies that are deemed to support RI, such as methods incorporating value-sensitive design (de Reuver et al. 2020; Simon 2017).

This very brief overview demonstrates that there is no perfect or unique way to implement RI, but that there are multiple, potentially overlapping, organisational levels and ways to shape and achieve the aims of RI. In addition, there are changes and uncertainties regarding the political and institutional embedding of RI. Probably, the most notable is the changing emphasis of the European Commission with regard to RI. During the Horizon 2020 research framework programme (2014–2020), RI was seen as a cross-cutting activity that had its funding stream but was also meant to inform projects and activities across disciplines and topics (European Commission 2013). In the successor framework programme, called ‘Horizon Europe’, there is much less emphasis on RI and fewer funding opportunities. In the UK, on the other hand, support by research funders for RI seems to still be strong, even though there appear to be difference between funding councils, despite these having been merged into UK Research and Innovation (UKRI) as an umbrella organisation in 2018. Such considerations are important because policy signals and funding opportunities drive the activities of research performing organisations which, in turn, shape the behaviour of research groups, individual researchers and research projects.

This last section of the brief introduction to RI points in a direction that we will return to in the discussion and that constitutes a key finding which is related to the organisational location of RI. To prepare this discussion, it is worth defining the concept of a ‘meso-level’. The term ‘meso’ comes from the Greek mesos which means ‘middle’. Meso-level analysis is typically described as the level between the micro and the macro. To represent what we have said earlier about the focus of RI research, the following figure shows our views about a meso-level within the research ecosystem which shapes and drives RI.

Figure 1 indicates that much attention in RI research has been paid to the micro-level of the individual researcher and the individual project. Similarly, there has been significant work on the role of macro-level institutions such as universities or the larger policy and funding levels. Our work suggests that there are structures that sit between the micro and the macro that call for more attention. This is what our use of the term ‘meso-level’ refers to van Wijk et al. (2019). We use the term meso to refer to programme-level activities. This is not a clearly defined term but one which we use to refer to activities that are larger in terms of funding volume, membership and temporal duration than research projects. Programmes are often run by members from several organisations. At the same time, these programme-level research activities tend to be smaller than institutions and have a limited temporal duration. A good example for a programme would be the UK programme grants⁴ which have a maximum duration of 6 years, a budget of up to £5 m and are typically run across institution. Other uses of the term ‘programme’ exist and the categorisation of entities as macro, meso or micro may be debated. We believe, however, that our use of the term for the purposes of this paper is sufficiently clear.

Figure 1. Levels of RI research and analysis – noting a positioning on research programmes at the meso-level.

Using this concept of a meso-level, we believe that there is very little attention that has been paid to it in the RI discourse. This is not to suggest that nobody has ever thought about it. There have been numerous projects that explored how RI can be realised and implemented and many of those have looked at organisational structures. This includes projects such as the GRRIP⁵ project that explores RI practices in research performing organisations, the CO-CHANGE⁶ project that supports RI implementation through institutional change among research and innovation actors, the HEIRRI⁷ project that aimed to integrate RI in university careers in science and technology, and many others. We have no doubt that the researchers involved in these projects will have observed the relevance of meso-level structures, we believe that these have yet to be formalised and published. A similar comment can be made about the published RI literature. There are numerous publications that look at the integration of RI into organisational structures, some of it arising from projects like the one cited earlier (Wittrock et al. 2021; Wittrock and Forsberg 2019). Particular attention has been paid to the role of RI in commercial organisations which raises additional issues of translation of RI discourses into business terminology and the link of RI to related topics such as business ethics or corporate social responsibility (Iatridis 2015). Despite this wealth of literature (Gurzawska, Mäkinen, and Brey 2017; PRISMA 2019; van de Poel et al. 2020; Yaghmaei 2018), we believe that there remains a gap in attention to the meso-level. This is, of course, difficult to prove. An indication that our contention that there is a lack of attention to the meso-level is correct may be that a recent review of the first decade of publications of the Journal of Responsible Innovation (Fisher et al. 2024) lists ‘institutions and organizations’ as a recurring topic, but fails to refer to the meso-level activities we cover here.

Methodology

The empirical work that underpins this article was undertaken as part of a project that aimed to integrate RI within a larger research programme and critically reflect on this process (Portillo et al. 2022). The research programme that funded this study is a ‘hub’, i.e. a multi-year series of research activities that started in 2009 that was funded by the UK EPSRC, now part of UKRI. The EPSRC has a pivotal role in RI as one of the first research funders to recognise the value of RI and to adopt it as part of the funder’s organisational strategy. It has also been highly influential in shaping the RI discourse (Owen 2014). It is therefore not surprising that the overall structure of the hub already had a strong component of RI as reflected in its research aims and culture. This hub can count an example of a meso-level structure as we use the term in this article.

The research hub’s approach is to allocate resources to specific targeted projects via a co-creation process that allows proposals to be developed using the input of hub members who are university-based researchers and external stakeholders (industrial partners). The study reported in this paper was the outcome of such a co-creation exercise, as was one of the case study projects described below. The other case study project was funded by a different research programme, also funded by the EPSRC. It is important to know that the internal funding processes of the hub, which also applied to this study, included a mandatory specific section dedicated to RI where researchers are encouraged to describe RI challenges relevant to their project proposal, plans for embedding RI practice, how they would promote reflection on RI and would make use of available RI tools and frameworks. Similar requirements applied to the programme that funded the second case study, which had the result that there was already a strong RI presence in all activities described in this article.

The purpose of our RI project that underpins this article was to observe how RI is currently implemented in research projects and to explore researchers’ views on ways of assessing this implementation. It went beyond this pure knowledge interest, however, in that it also aimed to identify and possibly shape the ‘flavour’ of RI that existed in the hub. Furthermore, by being part of the hub and using research to shape the future of the hub, the researchers became part of the research subject. Our approach therefore has an aspect of action research (Ahari et al. 2012; Avison, Baskerville, and Myers 2001), even though we did not use this terminology and did not follow the methodological niceties of action research (Baskerville and Wood-Harper 1998). The important insight for this methodology section is that we were not detached objective observers, but we were explicitly part of the research environment and had a clear motivation to shape this environment with a view to strengthening RI.

In this context, we pursued several strategies to gain insights, collect data and reflect on RI. We worked with two other funded projects and interacted with their project teams to explore their views on and practice of RI. Both groups volunteered to be part of this work. We referred to these as case studies, and designed and planned the work using a case study protocol (Yin 2003a, 2003b). In light of our active involvement in the cases, it might be appropriate to call them action case studies. The case studies work was run in parallel to a set of interviews we conducted with researchers across the hub and linked research programmes, to have a broader understanding of how RI is articulated and implemented within that academic context. The details of our data collection, analysis and interventions are described below.

It is worth highlighting that this study took place during the COVID-19 pandemic, therefore all the research activities were conducted online (in Microsoft Teams). The study was approved by the relevant Research Ethics Committee in July 2021.

Case studies

We engaged with two research projects as example case studies of RI practice, one from each research programme, as mentioned above. Both were short-term projects (12 months). Both case projects were located in the field of computer science even though their detailed research questions, context of application, and objectives were vastly different. Both projects volunteered to engage with us as case studies. Case 1 focused on what could be described as a methodological innovation with the aim of better capturing consumer preferences. Case 2 investigated an application of AI for the support of online moderation for mental health services.

The aim was to co-create an RI activity plan with each project research team, through a series of activities: (1) research interviews (included in those described in Section 3.2, with four participants from case study 1 and three from case study 2); (2) completion of the ORBIT online self-assessments (de Heaver et al. 2020; Stahl 2017) and (3) a reflective team workshop conducted online using a card-based tool: Moral- IT cards (Urquhart and Craigon 2021), to discuss RI in relation to a technology they were developing in their project. Both the ORBIT self-assessment and the Moral-IT cards were used as tools to highlight questions and encourage reflection and engagement around relevant issues. The ORBIT self-assessment is an online tool that researchers were asked to fill independently. The card-based workshops took place close to the start of one of the case study projects and half-way through the other (see more details in the discussion section). One of the sub-questions of our research was to explore the value of using tools to foster RI. There is a significant number of RI tools (Bernstein et al. 2022) and we chose these two because of our familiarity with them and the fact that they were available to us. Following the two interventions using the tools, the research team undertook an analysis of the findings and outcomes which were summarised using an overview document that highlighted current RI practice and open questions regarding RI. This document was shared with the principal investigators (PIs) of the two case study projects as input for a further co-creation activity of their RI action plan. To facilitate the co-creation activity, the analysis was structured in terms of a set of questions covering the following different aspects of RI: anticipation, engagement, reflection, ethics, open science, science communication, EDI (equality, diversity and inclusion), governance and other. For each of these aspects, we extracted the relevant planned activity from the original RI plan in each case study project’s proposal and formulated a set of questions based on the empirical investigation and analysis of the interventions that could trigger further reflection and development of the RI activities in the case study projects.

As part of the overall project structure and to promote critical reflexivity of our approach, we also established an advisory group with nine experts in the field which met three times at different stages of the project.

We did gain insights into the practice of using tools to support reflective processes. We had originally planned to have two workshops with the case projects, one for the Moral IT cards, one for the ORBIT self-assessment. We reduced this to one workshop, due to time reasons, in which a moderated session using the Moral IT cards was run. The ORBIT self-assessment tool, which is implemented in the form of an online survey, was used during the researchers’ time and we only analysed the responses without further input or insights into the thought processes of the researchers. One clear insight emerging from these activities was that tools can be useful to stimulate discussion and highlight particular topics, but they require guidance and resources. An individual researcher can no doubt benefit from the use of one of the many RI-related tools, but for a research team to reflect on the responsibilities they encounter in their work, tools need to be supported and require resources in terms of human expertise to guide reflexivity. Such expertise may be present within scientific research teams, but where this is not the case, it is unlikely that tools like the ones we used here will strengthen the commitment to or practice of RI.

While we thus believe that we successfully engaged in reflexivity, we did not actively support the case projects in their engagement activities, anticipatory work or other RI practice. As a consequence, the work described here remained predominantly on the theoretical level and had little immediate impact on the research and innovation activities of the case project. This led us to question the location of RI and whether a different framing from the one that informed our project would be helpful, as described in the following section.

Research interviews

We completed 14 one-to-one semi-structured interviews with researchers, academics and support staff with different levels of experience, including: three senior academics (two principal investigators and one co-investigator); eight Research Fellows with different levels of experience; a researcher at an external partner organisation; a final year PhD student; and a senior research facilitator. Participants were recruited from different UK research institutions associated with two linked programmes of research addressing trustworthy autonomous systems and the ethical use of personal data research. Participants’ backgrounds included: computer science, social science, mental health, psychology, engineering, law and linguistics. All but two were working on short-term projects (12 months). During the research interviews, participants were asked about the following topics: their understanding of RI and how they put it into practice; examples of enablers and barriers of its implementation within their projects; their experience/training on RI; whether and how their institution could better aid RI practice. Participants were recruited without regard to their level of knowledge of RI and were asked to respond to the questions according to their thoughts and experiences about conducting research within their project at the time the interview took place. Data collection took place between July and September 2021.

Findings

We first describe the findings from our targeted interventions aimed to support the implementation of RI in the two case study projects within the digital technology sector. Then, we highlight key findings from the research interviews (undertaken in parallel to the case studies study) to demonstrate the resulting understanding of, attitudes to, perceived benefits and barriers to RI implementation by individuals who are part of the hub and linked research programmes.

Case study project 1

The first case study project was located in the area of fast-moving consumer goods (FMCGs) and sought to make better use of data. It was predicated on the insight that individual-level personalisation of FMCGs remains uneconomical. Examples of FMCGs might include packaged foods where data could be used to identify taste preferences or seasonal demands. Another example would be household cleaning supplies where data could be used to adjust promotional strategies. The project proposed that data-driven product design could offer rapid adjustment to consumer trends in areas where consensus can be identified. This project built on interdisciplinary research to establish highly efficient methods for active capture of uncertainty and ‘leeway’ in consumer preferences – to identify areas of hidden consensus and inform decisions about both product design and market segmentation, while giving consumers back choice in the product design process. The scientific focus of the project was on new ways of collecting data (beyond established survey technologies) and testing how such data can be used by decision makers. Disciplines involved in the research included computer science, decision science and business studies. By enabling products tailored to those with compatible (rather than identical) preferences, the project aimed to explore the basis for accessible, scalable and thus commercially viable data-driven FMCG customisation. Engagement with stakeholders was part of the research methodology as well as the project’s approach to RI. Key stakeholders were customers of FMCGs as well as decision makers in the FMCG industry.

The proposal covered RI in detail. As a methodological development, a core component of the project was to evaluate the impact this approach may have on RI. This was expected to be two-faceted. First, there was a focus on how this method might offer ethical improvements in research and innovation, by comparison with existing quantitative and qualitative data collection methods. As a method which functions through active (and inherently consensual) engagement with consumers, this could offer substantive benefits in RI – it might also potentially increase trust in resulting products, by re-enfranchising consumers in the design process. However, negative implications were also considered:

• How this new data type might lead to inequitable benefits across different societal groups.

• How it might be combined with existing passive data collection approaches in the future, exacerbating existing ethical concerns.

We developed a set of questions that we shared with the project team with the aim to co-create a more detailed RI plan for the project (see Appendix II: RI-related questions for Case Project 1 – for more detail). These questions started with topics related to anticipation, notably a more detailed discussion of the practical implications of the adoption of the method developed in the project. Part of this was the question of the level of responsibility that the project team felt they had for the consequences of such an adoption. Here some project members felt institutions and funders’ expectations from researchers’ RI practice were unclear. Team members also felt the anticipatory and reflective exercises they undertook at the start of the project (Moral IT cards and ORBIT self-assessment) to be very time-consuming, particularly being a short-term project. With hindsight, the team found the time allocated to that exercise was beneficial to the project as it provided an in-depth reflective exercise on best ways of achieving the benefits of their outputs whilst minimising potential harms. It also aided the completion and institutional approval of their project research ethics application. Timing issues between the life span of the case study and our project (both run in parallel) made the co-creation of an RI plan to be realised within that case study very challenging. However, the project team was asked to reflect on the degree to which the RI-related work in their project was of potential relevance for a follow-on project. Stakeholder engagement was key to the project team from early stages and throughout their project. Questions of engagement were posed, in particular how users of the system might be affected and involved. Clear communication with consumers and companies and provision of meaningful transparency to consumers were elements highlighted by the project team. Broader engagement with consumers and companies was raised as a possible way of disseminating and making findings more widely accessible. In addition, it was asked how participant samples could be scrutinised to avoid lack of representativeness and check the possibility of discrimination.

Case study project 2

The second case study was funded in the context of a programme looking at trustworthy autonomous systems. It explored the use of Socio-Technical Natural Language Processing (NLP) for classifying behavioural online harms within online forum posts (e.g. bullying; drugs & alcohol abuse; harassment; self-harm), especially for young people. The study was led by university-based researchers but worked closely with a company that offers online counselling and that provided the data set and access to stakeholders for interdisciplinary collaboration for NLP experiments. The project’s socio-technical AI explored graph-based NLP algorithms for behaviour classification, using a cyclic socio-technical methodology where human teams from a mix of disciplines and stakeholders worked with NLP models over a period of time incrementally re-training and re-analysing data in experiment cycles until they reached a satisfactory conclusion. This approach was aimed to facilitate incremental use of human feedback for iterative learning and re-ranking, overcoming the limited training data issue and keeping a ‘human in the loop’. The project followed an inclusive multidisciplinary research approach, integrating stakeholders into the experiments from the start.

To implement RI, the project set up an incremental experimental methodology, as well as a diverse team of researchers drawn from multiple disciplines (Computer Science, Criminology, Policy Research, Psychology) and the company with its expertise in mental health and psychotherapy digital interventions. Half the project time was dedicated to running monthly experiments, which was meant to allow time for reflection on the direction of research travel and opportunities to anticipate potential impact, which could then be used to adjust the research plan to enact change. Stakeholder engagement included the company’s forum moderators, who are frequently exposed to sensitive and complex content and lack tools to help performing moderation, and users via focus groups. It was hoped that this would allow the project to engage in a wider conversation about the research and to steer the work towards an inclusive and positive impact.

A key concern in this project was the protection of the data which was discussed with the company in light of GDPR and the company’s privacy policy. All company datasets were to be pseudonymized and checked by moderators for leaks of identifying information. In addition, all members of the project team voiced the importance of co-creation and stakeholder’s welfare to the project. That was also reflected in the project proposal which included ways to consider questions of equality, diversity and inclusion to ensure relevant voices were heard.

The interviews and two interventions through the Moral IT Cards and the ORBIT assessment tool raised a number of questions that were discussed with the project team (see Appendix III: RI-related questions for Case Project 2). These included the degree to which the RI plan had been implemented, including the occasions on which RI issues had been discussed and issues such as parental consent, algorithmic biases, data equality and explainability were explored. Questions were posed about the process of engagement, including support from the institution and the ethics approval, in particular with regard to anonymous data from under 13-year-olds and related issues with explicit consent online. This led to the project team and stakeholders changing the scope of the data processed to avoid the need completely for under 13-year-old data, removing effectively the RI risks from this part of the project. Another area of interest was that of the open-source sharing of the technology and how RI was reflected in the project governance and the broader policy relevance of the project. A plan was put in place for post-project exploitation discussions with stakeholders, allowing for appropriate ring-fencing of open-source academic software outcomes and closed-source versions of models for stakeholder which has more dual-use type risks to consider.

Summary of findings

The analysis of the case studies showed two main groups of factors that have an impact on RI integration: (1) factors related to project dynamic and (2) organisational factors. In the first group, the one related to project dynamics, a key factor identified was that of time available to undertake RI work. While both cases were committed to RI and had plans for integrating RI, the relative weight of RI in the overall scientific work was not clearly determined. At the same time, many RI activities, such as anticipation and reflection, can potentially be very time-consuming. Another factor related to project dynamics is that of stakeholder engagement, a key component of RI which can greatly benefit research but also requires significant resources. Further factors seen as facilitators of responsible co-creation include the consideration of EDI and clear communication.

In addition to project dynamics, the case study findings highlighted several organisational factors that influenced the implementation of RI. It became clear that support on how to put RI into practice could be helpful, in particular for interdisciplinary and short-term projects, but that support was not present consistently. A further organisational issue was that of clarity of expectations. Both cases had integrated RI based on call requirements but were unclear about what exactly were the expectations of the organisations that employed them and how those related to the funder’s principles.

In addition to the findings from the case studies, we gained further insights from the interview studies. Details of the interviews and how our insights were arrived at can be found in Appendix IV: Interview Study. The interviews aimed to understand how researchers interpreted the concept of RI, what they saw as benefits of RI and where they perceived barriers to RI integration.

The concepts of RI that our respondents held reflect to a significant degree the academic RI discourse. They highlighted concepts such as stakeholder engagement and refection as part of RI. Their responses underlined that RI is deemed to have a positive outcome, strengthens research accountability and prevents harm. Overall, RI is meant to support welfare and prevent harm. It is linked to the purpose of research and strengthen social impact. Our respondents saw a close relationship to similar activities, such as research ethics and integrity.

The interviews showed that respondents saw several benefits. Some of these relate to the quality of the research, for example increased transparency and trust which can lead to reputational benefits. Similarly, RI is seen as part of the research process where it can support co-creation, which can also improve research quality but also prevent harms.

These benefits that can promote RI are counteracted by barriers that can impede its implementation. Prominent barriers include some we have already encountered in the case study findings, such as time constraints and unclear expectations. Collaboration between different types of entities, notably between academic institutions and industry, was shown to be a barrier. Institutional governance structures and problems with the embedding of certain research practices.

The findings of the interviews that uncovered the perceptions and attitudes towards RI by our interviewees have been graphically summarised in Figure 2.

Figure 2. Researchers’ perceptions of RI, its benefits and barriers to implementation.

This brief overview of the findings of our empirical work allows us to classify the enablers of and barriers to RI implementation in a way that prepares the following discussion by aligning them with the levels of micro, meso and macro as shown in Table 1.

Table 1. Summary of benefits and barriers of RI integration at micro-, meso- and/or macro- research ecosystems.

Levels of RI	Enablers	Barriers
Micro (individual researcher or research project)	Perceived benefits of RI, such as prevention of harm, transparency, or improved quality of outputs	Time constraints based on project resources and project duration
	Effective communication & co-creation with stakeholders	Lack of understanding of institutional requirements
	Strengthening of research outputs, rendering them more trustworthy
Meso (programme)	Culture of RI, i.e. shared concept and expectation of RI in the relevant research field	Different timings and expectations (academia vs. industry)
	Implementation support (good practice, existing expertise, experience in using RI tools)
	Guidance on interpretation of macro requirement in local context
Macro (institution, funder, research policy)	Clear institutional policies	Unclear RI institutional governance
	Funder mandates	Unclear expectations from institutions and funders

The content of this table picks up the barriers and enablers identified during the interviews as well as our insights from the case studies and aims to allocate them to the three different levels. This categorisation is based on our interaction with the researchers (several of whom are co-authors of this article) and we believe them to sound and justified. However, we are happy to concede that other categorisations might be possible. The point here is not so much to prove that this is the one and only way of interpreting the phenomena we encountered, but to pave the way for the discussion of the importance of the different levels that follows.

Discussion

This study reveals novel gaps in the RI implementation ecosystem in an interdisciplinary research context and provides a bottom-up approach for influencing change for better RI practice. Findings from the two case studies and the interviews gave rise to several insights. Many RI-related elements identified from the interviews correlate with the findings from the case studies’ interventions. In particular, benefits (e.g. anticipation and risks’ mitigation, better outputs) and barriers (e.g. time constraints, unclear expectations) from applying RI, and the value to research projects from stakeholder engagement (e.g. co-creation). The focus of this article is not on the specific findings of the individual case studies, but on the insights we gained with regard to the framing and location of RI as will be discussed now.

Framing and location of RI

Our project was clearly focused on RI as realised on the level of the research project. The representation of a project in Figure 3 gives an indication of this. It shows the project as the large rectangle within which certain RI activities take place, such as anticipation, reflection and others. The project interacts with external stakeholders such as users, policymakers, industry and civil society who forms part of the project’s RI ecosystem.

Figure 3. Project-centred view of RI.

This view of RI at the project level is not wrong, but it is incomplete in important ways as it fails to include relevant aspects that have significant influence on research and innovation practices as well as RI. To start out with, research projects are not normally stand-alone activities, but they have a history and a future which often influence strongly what happens in the project itself. The social reality of research groups and institutions typically means that they are continuously developing ideas and funding opportunities, so that the content of a present project tends to build on findings of a previous one and that the current project is used to shape ideas and proposals for future ones. This is a social reality that is very clear to the researchers involved, where PIs know they will be judged against current and future success of funding applications and the success of a future proposal will depend on the perception of the success of the current one. Often the same research staff will also be employed on successive research projects, bringing with them their values, skills and experiences in relation to RI. A simple representation of this temporal dimension is provided in Figure 4 which shows that the current project, here called project N has a predecessor in project N–1 and a likely successor in project N + 1.

Figure 4. Temporal logic of research projects.

In practice, project constellations are of course much more complex. An individual project can draw from more than one predecessor and spawn more than one successor. This is represented in Figure 5 where project N draws from the three projects M–1, N–1 and O–1 and provides input in the subsequent projects M + 1, N + 1 and O + 1. Projects not only have a temporal dimension and relationships to other projects, but they can form part of larger research programmes, as was the case for all the projects described in this article. Such programmes can determine aspects of the work undertaken in a project. They also tend to have a funding source that has specific priorities which may require, encourage or discourage certain activities with a bearing on RI. Staff may also be involved in and employed by the programme itself, moving between specific projects over the lifetime of the programme. Projects can thus be interpreted as parts of networks of projects and programmes, as indicated in Figure 5.

Figure 5. Network of projects and programmes.

This network-centred view of research projects offers a different perspective of the location of RI and its various components. Some of the RI activities may be best placed on the project level, but in many cases, it may be more useful to have them on the programme level. There will be cases where the individual project has no influence on specific aspects of RI, e.g. where EDI issues may be dealt with on the institutional level of human resources policies or where a research programme institutes a technology foresight programme that informs various projects that are delivered under its auspices. This growing complexity is represented in Figure 6 that adds aspects of RI to the network of projects and programmes proposed in Figure 5. It has added external stakeholders, represented by two groups of users and two groups of policymakers. In this example, project M directly interacts with one of these groups each and the programme that supports project M engages with the other group. In addition, we have added aspects of RI that could happen on the programme level and in our fictitious example of programme A, there are EDI activities and anticipation that happen on the programme level.

Figure 6. Distribution of RI in networks of projects and programmes.

Real networks are of course even more complex than what we suggest in Figure 6. The world of research not only consists of projects and programmes. These are typically hosted by research preforming organisations such as universities which may host a multitude of projects and programmes across different fields and disciplines. Research funders are not only external stakeholders but are intrinsically involved in the shaping of research landscapes, including incentive structures for RI. Similarly, industry may play a role in the research itself, for example as project partners but also as vehicles to move research insights to market and thus help them achieve broader relevance and impact. All of these actors are made up of individuals who may have different roles over time, for example when they move from a university post to a company research position, but who may also simultaneously be involved in different aspects, for example as a researcher on one project who has a leadership role in a programme and a management role in the hosting university.

The case studies and interviews reflected a commonly felt disconnect from funder’s and research organisation’s policies and values in relation to RI, which researchers experienced as vague and somewhat distant from the day-to-day practicalities of research within a specific funded project. We have seen that research programmes can play an important role in interpreting and resourcing RI across a network of related projects. Participants identified several RI activities that may be particularly amenable to support at this level, which are summarised visually in Figure 7, and following.

Figure 7. Potential meso-level RI support activities.

We feel that this emphasis on the framing and location of RI and our insight into the importance of the meso-level as a locus of RI constitute the key novelty and academic contribution, as we argue in more detail in the next subsection.

The importance of meso-level structures for RI

Key aspects highlighting the importance of the meso-level for the theory and practice of RI include:

1. A meso-level structure such as research programme provides the context in which organisational policies and norms can be interpreted RI for that specific research domain, agenda and team. This can provide more specific and locally meaningful expectations and priorities in relation to RI than is typically possible at organisational level (e.g. university policies). For example, one of the programmes studied here requires that RI (and separately EDI) is addressed in every internal project proposal and emphasises the need to identify specific RI activities within the project.

2. A meso-level structure such as research programme can provide a context for training and resources for RI which is specifically tailored to the research domain, topic and team, such as the Moral IT cards used with the case studies. These cards focused in particular on those issues that are typical or unique for IT research projects, thus affording discipline-specific guidance.

3. A meso-level structure can underwrite and support time for RI activities including anticipation, reflection and involvement with stakeholders before and after the official start and end of a project. For example, both programmes studied have centrally coordinated project proposal development processes which require the involvement of external stakeholders and encourage anticipation and reflection.

4. A meso-level structure can help to establish and manage relationships with external partners over a longer timeframe and with efficiencies of scale compared to each individual project attempting to do so independently. This can apply both to commercial partners and also to non-commercial partners or stakeholders, such as Patient and Public Involvement (PPI) groups. In our case, this could be most prominently observed in the collaboration between academics and industry partners where the ongoing collaboration beyond individual projects helped negotiate and clarify crucial questions such as access to data or IP ownership.

5. A meso-level structure such as a research programme, a research group or similar – ideally supported by the institution as a whole – can help to support professional development and career pathways with a specific focus on RI. In our case, the research programme emphasised the importance of RI and offered training and guidance that were independent of and went beyond the RI support offered by the institution.

What is striking when looking back the recommendations that our respondents made with regard to the promotion of RI is that most of them call for action on the meso-level or may even be best placed there. The meso-level such as the programmes in our cases are where disciplinary expertise is located. This is thus where specific RI expectations can be formulated, and the benefits of RI can be showcased. The limited size of a meso-level unit, for example when compared with a large research-intensive university, furthermore, allows the recognition of RI activities and setting of specific incentives and RI-related roles. It may thus be best placed to be the organisational home of RI. These observations are not to deny the importance for RI of individual researchers, research projects, institutions and the policy level. Instead, the point is that all of these are linked through meso-level units which may be more or less permanent but which in terms of organisational memory and organisational practice play a crucial role in RI practice.

The meso-level thus plays a key role in establishing the overall research and innovation ecosystem in which RI takes place. It helps establish and maintain relationships between agents in different systems, such as universities and companies. Meso-level structures can help overcome the limitations of other aspects of the organisational structures within RI plays out. They can offer a similar level of discipline specificity to individual projects, but they offer a longer time frame than projects and allow for the maintenance of ongoing relationships. On the other hand, they can address some of the limitations of RI on the institutional level, where RI policies tend to be located but where there is typically little subject expertise in the discipline in question. Meso-level structures thus combine longevity and subject expertise in a way that is crucial to the long-term development and embedding of RI. Our experience arising from the projects described in this article but also from other projects suggest that meso-level structures provide the glue between the project and the organisational level that is required to create, support and maintain a sustained culture of RI.

In light of this reasoning and the empirical evidence, we found supporting the importance of the meso-level, one can ask why, considering the significant investments into RI on the European level but also by national funders, there has not been more attention paid to it. It seems plausible to assume that this lack of attention can at least partly be explained by the transitory and ill-defined nature of the meso-level. Recipients of research funding tend to be either individual researchers or research institutions, but rarely meso-level entities. Policies are written for or by institutions whereas most research happens on the project level. The meso-level, despite its importance in retaining knowledge and practices, is simply less visible and often falls outside the formal mechanisms of research and research governance, which may explain the lack of attention to it, both by research funders and institutions, and by the RI discourse.

An example of how RI practice can be supported at the meso-level is the approach taken by Responsible AI UK (RAiUK), UKRI funded programme (2023–2028). They have allocated resources to have a programme lead and a coordinator for RI to drive and manage an embedded RI culture within the programme, and to ensure support for RI practice (e.g. workshops, RI toolkit)⁸ is provided to all RAiUK funded projects. Besides, all projects are requested to name an RI lead to ensure project’ commitment to RI practice throughout their projects, and as a point of contact for RI-related activities within the programme. Part of those initiatives and RI activities have been inspired by the research findings and ‘lessons learnt’ from this study.

Conclusion

This paper contains an account of the implementation of RI in an existing large research programme, drawing on data collected through a set of interviews and two project case studies where in each case two tools or interventions were used to stimulate critical reflexivity and co-create an RI action plan with the project participants. It aimed to identify the main barriers and enablers of RI and how these are reflected at different organisational levels.

The study highlighted a complex mix of barriers and enablers that have an effect on how RI is realised in research practice. The key insights that we highlighted in the discussion section is that an important parameter that strongly influences many of the barriers and enablers is that of the organisational level. More specifically, we argue that success or failure of RI implementation hinges to a significant degree on the meso level, the level that we associate with a research programme that brings together several projects and has a duration that exceeds typical project lifetimes.

We believe that in particular our insights concerning the importance of meso-level structures constitute a novel and unique contribution to the RI discourse which so far has not paid much if any attention to them. This may be because they are so familiar to most researchers that they do not seem to call for explicit attention. However, our empirical observations and their interpretation suggest that successful RI implementation relies heavily on these structures and RI research should therefore pay more attention to them. Further investigation is therefore needed to unpack the distinct role(s) that research groups, departments and programmes play in relation to RI. Such future research should also investigate which lessons meso-level structures can learn on how RI can be implemented. The work presented here suggests that meso-level structures can serve as holders of multidisciplinary expertise, they can support learning and development of individual researchers and incentivise successful RI implementation. The meso-level may furthermore prove to be essential in shaping future RI practice and helping to avoid that it degenerates into a box-ticking exercise. This role of the meso-level may be facilitated by its nature as a carrier of local and specific research culture that goes beyond generalised organisational requirements. However, more detailed research is required to confirm these insights and develop more specific guidance. Such future research could also include questions concerning the nature of the research, e.g. whether applied research of the type described here poses different questions from more basic research.

RI, as our research has underlined, should not be understood as a specific targeted intervention but as an ongoing process that accompanies research and innovation activities from their inception all the way to eventual exploitation and use beyond the research environment. It covers the individual action of the researcher as well as the structure of projects and programmes and reaches to research funding and policy. Adopting this perspective of RI will raise many further questions, notably of where the responsibility lies for orchestrating the different activities across organisations, institutions, project and programmes. An answer to this question will be required to ensure that RI can live up to its expectation of strengthening the link between research and society with a view to ensuring that processes and outcomes of research and innovation are ethically acceptable, societally desirable and sustainable.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Economic and Social Research Council: [Grant Number ES/V011278/1]; Engineering and Physical Sciences Research Council: [Grant Number EP/T022493/1]; Engineering and Physical Sciences Research Council: [Grant Number EP/V00784X/1]; Natural Environment Research Council: [Grant Number NE/S015604/1].

Notes

1 https://www.ukri.org/councils/epsrc/, accessed 05.06.2024

2 https://www.ukri.org/, accessed 05.06.2024

3 For the sake of consistency we will use the term RI in this paper, including in cases where the conversations described may have used the term responsible research or responsible research and innovation. The acronym RI thus has an encompassing meaning similar to that of R(R)I proposed by Shanley (Shanley 2021). The only exception to this are quotes from respondents which may include a different use of terminology.

4 https://www.ukri.org/councils/epsrc/guidance-for-applicants/types-of-funding-we-offer/programme-grants/, accessed 05.06.2024

5 https://grrip.eu/, accessed 07.06.2024

6 https://cochangeproject.eu/, accessed 07.06.2024

7 https://www.ecsite.eu/activities-and-services/projects/heirri, accessed 07.06.2024

8 https://rai.ac.uk/toolkits/rri-toolkit/, accessed 05.07.2024

Appendices

Appendix I. Code book.

Table

Pts	Participants, sometimes also participants (of a study)
RR	responsible research
PPI	Patient and public involvement
RR def	responsible research definition (as given by participants)
follow-up	future or follow-up projects
stakeholder [GROUP]	Discussion in regards to different stakeholder groups [group specified in brackets]
Method	discussion of methods/methodologies used, how they might change etc. (both for RR and research in general)
RRI framework	discussion of one of the RRI frameworks (AREA, EPSRC, …)
Team	research team involved in a project
Training	discussion an form of training on RR(I)
Experience	RR based on experience (no formal training), general experience
hr	human resources (e.g. when not having enough people power for a project)
tick box	potential for RRI to become a mere ‘tick box exercise’
individual RR	personal approaches or own responsibility of an individual researcher
terminology	discussion on different terminologies, or similar but different concepts
Outputs	outcomes of a project, publications etc.
responsible design	designing a study responsibly,
pts career/background/position	information to do with previous/current positions/job of pts, professional background etc.
non-rr	none-responsible research
time frame	discussion about time available

Other notes on codes/changes in coding as part of the TA process:

Table

Codes collapsed, renamed or moved for theming:
Original	Now under
PPI	Stakeholders (PPI)
clear objectives	Objective
team	stakeholders (Team)
rri methods	rri approach
individual RR	RR approach
time investment	time scales
HR	Resources
Finances	Resources
support (university)	Institution (university)
support (colleagues)	stakeholders (team)
stakeholders (advice, students)	Academia & Academics
time management	time scales
Outcomes	outputs
Benefits	rr(I) benefits
Publications	outputs
Consent	stakeholders (participants)
research environment (where it is done)	stakeholders (community)
Ethics	various (include responsible design)
academic culture	Academia & Academics, as well as University (institution) when talking about structure &hierarchy
rr tool	rr(I) methods and approaches
best practice	rr*I) methods and approach
rr facilitation	rr(I) methods and approaches
stakeholders (collaborators)	stakeholders (general)
Abandoned codes (due to lack of frequencies or other codes being more appropriate)
Ethical
stakeholders (professionals) – either participants or team
stakeholders (user researcher)
RR as acceptance – was refined as RR def.
RR action
audience
tick box
convincing others
research integrity
risk assessment
research area
hierarchy
networking
confidentiality
proposals
support (others)
working situation – moved into various fitting categories
Other
• pts career/background/position – not including in theming, but added to transcripts for future comparison of participants’ backgrounds
• ’problematic’ – not a complete theme in itself, but added to high some areas or behaviour that was potentially seen as problematic or not up to best practice by participants
• [ … ] – content not used for theming, but left in coding for general information/commenting

Appendix II. RI-related questions for Case Project 1

Table

RI aspect	Questions to consider for a potential follow-on project
Anticipation	How do you anticipate the method being used in practice and how would this affect its ethical implications, e.g. would some application domains be more acceptable than others – How would you influence its use for good? Is this your responsibility?
Reflection	In the transition to the next project what differs from the issues that were identified with the Moral IT-card workshop (we perceived reflection on issues to be considered on a possible follow-up application)?
Engagement	How can you engage with groups of stakeholders, particularly consumers and companies to understand for example: What are consumers’ views of how the tool works and what data it collects – How do they understand it? How might a company use this technology in practice? What method may it replace or augment and how does this fit into other elements e.g. business requirements, product development? How do you engage with potential users of the system to communicate what it can or can’t do in an appropriate way and promote the benefits whilst minimising the potential harms?
Ethics	Could you define an ‘ethical purpose’ for the project which can act as a reference point for other considerations (e.g. this system is intended to help provide better products for consumers) and allow you to view other considerations with this in mind?
Open science	Have you considered any possible venues for disseminating results within wider audiences (e.g. consumer communication venues)?
Science communication	Could your experience of engagement with consumers/companies act as a model for how the operation of methods can be communicated to the public as a potential example of good practice and lessons learned in science communication?
EDI	Challenge identified: participants sample (a representative sample is not always fits for purpose). This aspect of the project was identified early on when participants ‘ recruitment strategy Could you test how the interval capture method is used by different groups to see if it discriminates unfairly between groups and how you may appropriately mitigate this in your system?

Appendix III. RI-related questions for Case Project 2

Table

RI aspect	Questions to consider
Anticipation	Have there been adjustments to the research plan? Has anything changed since the discussion held during the workshop (we perceived reflection on issues that were not discussed before as a team)?
Reflection	Have you planned specific RRI reflective meetings as part of the regular team meetings or as extra sessions? Have you set aside time to discuss these points: parental consent algorithmic biases data quality explainability
Engagement	Has this/these taken place already? Do you foresee any possible barriers on this? Policy impact & seeking support from TAS hub/university policy teams. Have you decided whether and how you will approach this?
Ethics	Ethics conflict: use of under 13’s data (requires parental consent)-How will this be tackled?
Open science	If the technology gets exploited and is Open Sourced – What issues will arise?
Science communication	Has the project team agreed on how they are doing this?
EDI	Inclusion: to be able to provide choices to platform users (E.g. their posts not to be moderated) – Have the team discussed how they are going to get access to this data? Use of company’s past data. Have they got clients’ consent?
Governance	Where is RRI in the project governance (e.g. structure or deliverables that force RRI-related reflection)? Policy Impact strategy – Have you decided whether they would like to pursue this? Existing structures for Innovation impact (policy & government related: mentioned in Engagement section)
Other	Is this a potential issue to consider?

Appendix IV. Interview study

We explored participants’ beliefs, attitudes, experiences, and practice of RI and identified two major themes: (1) Meaning and value of RI and (2) RI integration- benefits and barriers. This paper highlights the most relevant sub-themes identified within these major themes.

Thematic analysis of the interviews

The interviews were analysed using thematic analysis. To begin, the interviews were transcribed, and the resulting data checked for completeness and anonymisation by a member of the research team [X]. The data was formatted in MS Word and put together an initial presentation of themes, based on their first-hand experience doing both the interviews and preparing and cleaning the data. To gain another, more in-depth understanding of the data, the interviews were handed to another member of the research team [X] for further analysis. It was decided to use thematic analysis to identify the underlying themes found in the interview. This was done using reflective analysis as described by Braun and Clarke (2006).

For coding the interviews, they were formatted into MS Excel files, with interviewer and interviewee being in different columns. Longer parts of the transcript were formatted into several cells, keeping in line with the original transcription format and enabling better legibility. Giving the nature of a reflective thematic analysis the researcher considered their positionality vis-à-vis the subject in a reflective statement that was added to as the analysis went on.

The analysis was started with an initial familiarisation phase, through reading the interviews. The researcher went into this process without having been present at the interviews or being aware of the participants’ identities. There was furthermore an effort made towards using an a priori approach to the analysis. Therefore, any previous (pre-)analyses were not considered when coding and forming themes from the data. Interview passages were summarised as a sub-step prior to coding, giving the researcher a more in-depth familiarity with the data. When coding the data and putting together a code book (see Appendix I: Code Book), several individual codes were merged due to being too similar. Depending on frequencies of individual codes, they were considered foundational to individual themes or alternatively put together with other codes in related areas. These general areas were then reviewed and mapped into appropriate themes and sub-themes. Boundaries and names for individual themes were continually refined as the analysis progressed.

Meaning and value of RI

Participants’ understanding of RI was varied. Participants who had gained experience of conducting RI in their work and/or had some knowledge of RI, mentioned: the purpose of the research, anticipation to possible future applications of the innovation to prevent harms, reflection, stakeholder engagement, and social impact, as key elements of doing research responsibly:

It’s these multiple, these multiple facets of it, right, so there’s whether you should do research and there’s how you should do the research. [P1, 47]

I think for me to do research responsibly is to make sure the research you are doing and the implementation of future application of your research findings are first, accountable and second, beneficial to the society. And third, not creating any harms to any members of the society. [P2, 6]

Some participants also provided examples of how they put those elements into practice and throughout the lifetime of their project:

Well, I think that the way we are implementing RRI with, like, the monthly meetings and reflections … […] … is positive because it kind of allows for more of an incremental thing. Like, it’s less, it’s less of a, like, you know, like a box ticking form filling and then forget about it, and more of a continuous process, a monitoring process. [P5, 67–69]

To some participants, RI was understood as research ethics, integrity, and accountability towards their project’s stakeholders, in particular the end users:

It means to me that we do it in an ethical way first and foremost. But also that the research aims and objectives are clearly defined. That the research questions themselves are going to be helpful. You know, I work in digital health and the research questions need to be in the interests of people who will receive care in the future. [P7, 10]

Responsibility was also interpreted as a duty towards the innovation/research community and funding bodies, in particular if that involved private funds. This view was shared by a minority of participants with scarce knowledge and/or experience on RRI. Interestingly, RI practice was also seen by many participants as a two-way process to deliver trustworthy outcomes:

Research responsibility is not only because of the projects, so you also are responsible for the, the – your colleagues or like the scientific area. So you need to provide reasonable and trustable results, and you are not leading them misunderstanding or make the miss-data for the people who is gonna refer your results as their main contact. [P9, 8]

Participants with some knowledge of RI frameworks, in particular the RRI AREA Plus Framework (AREA 4Ps) (Jirotka et al. 2017), referred to the AREA elements when defining RI, in particular: anticipation, reflection and engagement:

So I think for me, the most important aspects are probably the anticipation and reflection areas, and where it’s possible – and it isn’t always possible – to kind of factor in co-design where you get the stakeholders actually engaged as much as you can. [P12, 7]

All participants believed conducting research responsibly was very important to them. Examples provided were very much intertwined with their work experiences. Many realised the value from putting RI into practice by the quality of their outputs, in particular their research data:

I wouldn’t have been able to get the data that I have now, the richness, the multifacetedness of it, I think, if I hadn’t done it in this way. [P14, 84]

Many participants articulated the value or RI through the lenses of co-design with stakeholders. Some pointed out fostering co-creation from early stages and throughout the span of a project, to be extremely important to their project, and to them. One researcher believed that to be a two-way benefit situation:

[…] design it together with the partners as well so that we end up having something useful for us, but also useful for them, or for me. That’s the final, the ultimate hope. [P4, 44]

Many participants emphasised the welfare of research participants and end users of a research output to be key elements when conducting RI. Weighing the risks of research against the benefits to end users was of particular relevance:

So, one of my assumptions is that because the amount of raw data that a potential algorithm will be training on will be quite limited, […] but I’m not willing to put at risk any of our users from the point of having a better trained algorithm, if that makes sense. [P6, 93]

In addition, some mentioned that conducting research responsibly is also caring for the welfare of the researchers conducting a study, not often considered as stakeholders, and for the societal impact of research outputs/innovation:

I think there’s also one thing to think about which maybe isn’t thought about, is the welfare of the researcher. Now obviously that’s fewer – that’s the, the fewest stakeholders, the researcher is the very – like there’s many fewer participants than there are people who it could impact if it’s going to have a broad impact on society. So there’s the – and people weigh that balance. But the thing is participants seem to be given a very high weight. [P1, 131]

RI integration – benefits and barriers

This theme highlights the benefits and barriers of conducting research responsibly, as experienced by this study’s participants. Most of the factors identified as benefits and barriers of RI integration can be grouped within those sub-themes, as project dynamic factors or organisational factors.

Main benefits

The value of RI was mostly intertwined with elements that emerged from participants’ identified benefits, resulted from integrating RI within their research project(s). Those elements are mainly intrinsic to project dynamic factors and include:

Effective communication and co-creation with stakeholders

Many participants shared the view that RI practice improved communication dynamics within their research team. It facilitated a more ‘holistic’ approach to a research project, providing a space to better reflect as a team about their research questions, and in a more inclusive way:

So maybe that is part of an RRI spin that we try and have a holistic, as holistic as possible view of the, the, the, the kind of – the tool development or the research around it … So, this is like what frames your research question. And then you maybe have things like how do we work as a team, right, do we – from gender balance to, you know, trying to make sure that everyone has a voice in the meetings, … And so there’s all of those things, right, they are kind of like the mode of operation of, of the researchers every day. [P3, 17]

Many participants referred to the benefits of applying RI within their projects from the value participatory research activities and stakeholder engagement had in their project. It allowed better communication with research participants:

You, you, you can't guarantee it, but I think it mitigates against this. So the, so the whole process means that you explain to the participants ‘this is how long you're gonna be doing this for. [P10, 63]

A participant shared the influential role of a Personal and Public Involvement (PPI) group in co-creation as part of their responsive research team:

So – yes. The, the strengths of it. So it’s – if, if we didn’t adopt an RRI approach … , we wouldn’t have had that change in trajectory based on the PPI input. And also if we weren’t reflecting, engaging and acting under things that, you know, that happened during the project, if we weren’t responsive to situations that arose and if we weren’t responsive to what the PPI group were presenting, then I think we, we wouldn’t have – I think a lot of our communications as well would have been less effective. [P7, 81]

Prevention of harms

Some participants reflected on the benefits of anticipating to possible harms as part of their project, particularly when involving research participants in their studies:

I know we can run experiments that will stress individuals and things like that. I've never been a participant of one of those and I do think that, you know, we need to be extremely careful when this kind of, like, experimental or research, actually, you know, has an induced stress response. I consider all the research that we do as a potential induce stressor, if that makes sense, because we're asking for personal life experiences, most of the time. [P6, 12]

Quality of research outputs

Many participants highlighted that one of the main benefits of conducting RI was reflected in their project research outcomes:

Yeah, you just get much, much stronger findings, much stronger implications, much better writing. Yeah, every section of, of anything you write on it is, is much stronger because you've thought about the, the weaknesses in the field. So then every sort of, everything you write is just a lot stronger. [P13, 138]

Transparency and trustworthy outcomes

Responsibility was seen by many participants as a means to provide transparency about the purpose and consequences of a research project, and in particular a mechanism to ensure data transparency. Besides, RI leads to trustworthy outcomes:

I feel like the, like responsibility goes hand-in-hand with, like I said, with being accountable, being transparent, being trustworthy, and also for people to understand what happens with the research and why, why am I doing it. And also in the scope of this, I think this also goes in tandem with people knowing what happens to their data. [P14, 8]

Main barriers

When we asked participants their views about the barriers/weakness of implementing RI within their projects, time constraint was a prominent factor. We found this to be the only element within this sub-theme directly linked to project dynamic factors:

Time constraints

Most participants believed that putting RI into practice is the right thing to do. However, many pointed out that allocating time within their project(s) to – for instance – anticipate and reflect about their research outcomes is time-consuming, but some participants still found RI practice beneficial to their project. Nevertheless, some felt their RI activities were not recognised as a ‘proper’ research activities as other mandatory activities are (e.g. seeking project ethical approval):

So there’s a lot of things that wrese as researchers have to do that maybe don’t – still aren’t credited as being a proper part of the research. […] we have time to decide, it’s acknowledged you have to do – for, for doing the ethics forms, because it has to be done. But maybe there isn't time set aside for this more kind of thinking about outcomes? It’s like if it’s not something that is mandatory then it’s not part of your job in a way. It’s like something you should do, but it’s not acknowledged as being something – because it’s not something you have to do. [P1, 150–152]

Most of the barriers identified in this study that impacted participant’s implementation of RI can be grouped as organisational factors and include:

Different timings and expectations – academia versus industrial partners

Participants with work experience within the industrial sector, mentioned that RI practice within an academic project with external partners, often gets affected by time constraints and potential different expectations, timelines, pace and working conduct existing in the academic vs the private sector:

‘I do believe that, yes, there is a lot of potential influences of time constraints and outputs. And basically, you know, also, I think, academic influence, in the sense that if your product doesn't go well, or the findings are not as clear as you initially expected, it's difficult to go back to the drawing board.’ … ‘ It's really difficult to have a very cohesive collaboration with an industry and an academic project, I think. You know, at times, I feel like the academics are constrained with our requests on certain collaborations, and, at times, we [industrial partner] are constrained about the requests of the academic timelines.’ [P6, 55, 58–59]

Unclear expectations from funding bodies regarding RI

Some participants with experience of preparing RI action plans as part of some of their research proposals (a section 200 words max. as requested by EPSRC UKRI), commented that that exercise could be mainly ‘funding driven’:

It's kind of, I guess, funding driven, ‘I need to consider this’ (laughs). It's kind of, in my mind anyway, informally but, you know, from a grant proposal point of view I want to know, you know, like anything, you want to optimise your proposal and make sure you've considered all the angles in the way that the reviewers and the pa’, the funding panel think of it. [P12, 108]

Unclear RI institutional governance

To find out participants’ thoughts about institutional governance of RI within their organisations, we asked them what role they believed their institution had regarding supporting RI embedded practice. Responses included: people in senior leadership positions and strategic decision- making roles should have an active role on that. Many participants acknowledged the focus on promoting RI practice their organisations had, mainly linked to their research programmes (EPRSC funded) but were not aware of programmes and or strategies designed for RI practice as part of their main institution’s research strategy.

It's, I think, also in terms of funding and where money is coming from and all of these pragmatic, practical constraints for research also relate to how long are people in power that make decisions, how long are the policy periods that are happening, right? [P14, 169]

Evaluation of RI implementation – a barrier of facilitator?

All participants agreed that evaluating the impact of putting RI into practice is challenging and were not sure how to go about it and how systematic/comparable it would be between different methodologies/research and/or innovative design, between and even within similar contexts. Many participants argued what the ultimate purpose of it would be, how sustainable that is (this relates to the future impacts of an innovation/research outputs). Most participants believed that evaluating RI implementation in a way researcher/innovations won’t engage (e.g. as a box-ticking exercise) not only should be avoided but would contradict RI practice itself. One participant claimed that more resources are needed to realise and implement an RI evaluation properly done within an institution.

Yeah, I think if the people on the team actually care about it, it would be rea’, it'd [a RRI steered evaluation at the end of the project] be useful for them. It might sort of make them realise that some of the stuff they did was, was not very good, was not very ethical. But I think there, there'll be some people who don't really, yeah, care about that. And it'd also be useful for funders to know, yeah, which, which teams are doing really ethical research (laughs) and managing these things really well. But I, I don't think, I think it'd be quite hard to monitor, unless you have more resources. [P13, 193]

Other participants appreciated the value of evaluating the success of applying RI within a project as ‘to prompt and support that responsibility layer of the project’ [P6, 138], and to better support a reflective exercise within a team, which could aid to change for improvement of RI practice. The idea of evaluating RI practice within short-term projects was also seen as a mechanism to identify opportunities:

[…] But like really highlighting the opportunities. Like strengths, weaknesses, opportunities and threats, and maybe do a – and you could probably use that process, right, to do an RRI SWOT. And […], maybe there’s low hanging fruit for you here, by looking at, you know, how it affects minorities for example, you know. Hey, there’s a paper in there. And that’s usually how you get academics motivated. [P3, 152]

One participant stressed evaluation of RI practice would help to focus on the impact of RI implementation beyond the span of a project and regarding stakeholder’s needs: ‘[…] how many doors does this research open, how, how inspiring is it, does it reenergise other people.’ [P14, 143]

Intrinsic limitations of RI embedded practice

Interestingly, a participant with experience in the theory and practice of RI, argued that RI practice may limit research and innovation action. However, that was seen as a positive element as it leads to a more consistent approach to RI:

Yeah, I was going to say, of course, some of those constraints will also be barriers or weaknesses of the, of the, of the responsible research. And like I said, there are, there are things that we just can't do, there are things that are unnegotiable. So that, that – I think that could be a barrier. But then of course the counter argument is that also makes – that makes things more standardised. So that’s one of the barriers I can think of. [P2, 62]

Recommendations for RI integration

When we asked participants to identify areas for improvement to help fostering embedded RI practice within their projects, many demanded institutions should:

• Provide researchers/academics clear institutional expectations about RI. Academics and researchers wished their institutions to provide specific details (with examples) of that is expected from them in terms of RI embedded practice:

  I would like them [universities] to give more specifics, and they can be examples. […] So, sometimes universities can have these high-level things that are often about protecting the institution or the brand that somebody has, has come up on a very high level rather than going into some specifics or specific examples. [P10, 145]

• Deliver better resources and information about the purpose/benefits of RI practice:

  It’s a few years ago, like, and thinking about that, I put something in. It wasn’t successful. I put it in and, frankly, I had the wrong idea. I looked at AREA, I read the stuff, and basically all I had was the web resources. I had to guess what it was (laughs). There was no training. [P12, 91]

• Acknowledge and promote allocation of time needed to conduct RI activities, and to create and promote sustainable resources for RI practice within research projects. A participant suggested a way to tackle this by developing RI supporting roles to coordinate and facilitate RI focused activities:

  So again, for example, if – let’s say – well, I, I don’t think it would be economically justifiable, or maybe it would anyway, to have a colleague full-time only doing RRI, but maybe, let’s say twenty-five percent of the time of a colleague is to facilitate those quarterly sessions for example. So, you will have an RRI administrator or RRI – I try to avoid the word administrator, but I can't think of something better – so a role like that on a part-time or full-time basis, if this is on a faculty level for example. Again, I think that would, that would make a difference. [P2, 119]

• Institutions and funding bodies should create opportunities for RI champions, and RI incentives for researchers as part of their career (e.g. prize for innovation, publications):

  I’m, I’m a big believer in the carrots. And I think it’s something that (university) is unbelievably bad at in every – in all aspects of the organisation. So, I mean, this is just an, an ad hoc example. But if you did – if, if the institution or (institute) or whoever it is for example had an RRI prize for innovation in RRI in research or whatever. And then, you know, highlight those examples, you know, maybe have someone that works with the project, whatever, and then maybe writes them up. And then maybe, you know, does a blog entry or does a whatever, you know, gets it out. [P3, 162]

• Be responsible to facilitate RI integration as it is an ‘organisational responsibility’, and promote equal opportunities to RI embedded practice:

  Yeah, you could still talk about, oh, you know, how do you – do you have a diverse research team for example, and do you give everyone the same opportunities? But again, I think that’s really much more an organisational responsibility, rather than in an individual research team. [P3, 78]